Overlapping representations of observed actions and action-related features

The lateral occipitotemporal cortex (LOTC) has been shown to capture the representational structure of a smaller range of actions. In the current study, we carried out an fMRI experiment in which we presented human participants with images depicting 100 different actions and used representational similarity analysis (RSA) to determine which brain regions capture the semantic action space established using judgments of action similarity. Moreover, to determine the contribution of a wide range of action-related features to the neural representation of the semantic action space we constructed an action feature model on the basis of ratings of 44 different features. We found that the semantic action space model and the action feature model are best captured by overlapping activation patterns in bilateral LOTC and ventral occipitotemporal cortex (VOTC). An RSA on eight dimensions resulting from principal component analysis carried out on the action feature model revealed partly overlapping representations within bilateral LOTC, VOTC, and the parietal lobe. Our results suggest spatially overlapping representations of the semantic action space of a wide range of actions and the corresponding action-related features. Together, our results add to our understanding of the kind of representations along the LOTC that support action understanding.

The Representation of Observed Actions at the Subordinate, Basic, and Superordinate Level

Actions can be planned and recognized at different hierarchical levels, ranging from very specific (e.g., to swim backstroke) to very broad (e.g., locomotion). Understanding the corresponding neural representation is an important prerequisite to reveal how our brain flexibly assigns meaning to the world around us. To address this question, we conducted an event-related fMRI study in male and female human participants in which we examined distinct representations of observed actions at the subordinate, basic and superordinate level. Using multiple regression representational similarity analysis (RSA) in predefined regions of interest, we found that the three different taxonomic levels were best captured by patterns of activations in bilateral lateral occipitotemporal cortex (LOTC), showing the highest similarity with the basic level model. A whole-brain multiple regression RSA revealed that information unique to the basic level was captured by patterns of activation in dorsal and ventral portions of the LOTC and in parietal regions. By contrast, the unique information for the subordinate level was limited to bilateral occipitotemporal cortex, while no single cluster was obtained that captured unique information for the superordinate level. The behaviorally established action space was best captured by patterns of activation in the LOTC and superior parietal cortex, and the corresponding neural patterns of activation showed the highest similarity with patterns of activation corresponding to the basic level model. Together, our results suggest that occipitotemporal cortex shows a preference for the basic level model, with flexible access across the subordinate and the basic level.SIGNIFICANCE STATEMENT The human brain captures information at varying levels of abstraction. It is debated which brain regions host representations across different hierarchical levels, with some studies emphasizing parietal and premotor regions, while other studies highlight the role of the lateral occipitotemporal cortex (LOTC). To shed light on this debate, here we examined the representation of observed actions at the three taxonomic levels suggested by Rosch et al. (1976) Our results highlight the role of the LOTC, which hosts a shared representation across the subordinate and the basic level, with the highest similarity with the basic level model. These results shed new light on the hierarchical organization of observed actions and provide insights into the neural basis underlying the basic level advantage.

Concepts require flexible grounding

Research on semantic memory has a problem. On the one hand, a robust body of evidence implicates sensorimotor regions in conceptual processing. On the other hand, a different body of evidence implicates a modality independent semantic system. The standard solution to this tension is to posit a hub-and-spoke system with modality independent hubs and modality specific spokes. In this paper, I argue in support of an alternative view of grounding which remains committed to neural reenactment but emphasizes the multimodal and multilevel nature of the semantic system. This view is built upon the recognition that abstraction is a design feature of concepts. Semantic memory employs hierarchically structured representations to capture different degrees of abstraction. Grounding does not work the way that many embodied approaches have assumed.

The social relevance and the temporal constraints of motor resonance in humans

In humans, motor resonance effects can be tracked by measuring the enhancement of corticospinal excitability by action observation. Uncovering factors driving motor resonance is crucial for optimizing action observation paradigms in experimental and clinical settings. In the present study, we deepen motor resonance properties for grasping movements. Thirty-five healthy subjects underwent an action observation task presenting right-hand grasping movements differing from their action goal. Single-pulse transcranial magnetic stimulation was applied over the left primary motor cortex at 100, 200, or 300 ms from the onset of the visual stimulus depicting the action. Motor-evoked potentials were recorded from four muscles of the right hand and forearm. Results show a muscle-specific motor resonance effect at 200 ms after movement but selectively for observing a socially relevant grasp towards another human being. This effect correlates with observers' emotional empathy scores, and it was followed by inhibition of motor resonance at 300 ms post-stimulus onset. No motor resonance facilitation emerged while observing intransitive hand movement or object grasping. This evidence highlights the social side of motor resonance and its dependency on temporal factors.

Conceptual representations in the default, control and attention networks are task-dependent and cross-modal

Conceptual knowledge is central to human cognition. Neuroimaging studies suggest that conceptual processing involves modality-specific and multimodal brain regions in a task-dependent fashion. However, it remains unclear (1) to what extent conceptual feature representations are also modulated by the task, (2) whether conceptual representations in multimodal regions are indeed cross-modal, and (3) how the conceptual system relates to the large-scale functional brain networks. To address these issues, we conducted multivariate pattern analyses on fMRI data. 40 participants performed three tasks-lexical decision, sound judgment, and action judgment-on written words. We found that (1) conceptual feature representations are strongly modulated by the task, (2) conceptual representations in several multimodal regions are cross-modal, and (3) conceptual feature retrieval involves the default, frontoparietal control, and dorsal attention networks. Conceptual representations in these large-scale networks are task-dependent and cross-modal. Our findings support theories that assume conceptual processing to rely on a flexible, multi-level architecture.

Dynamic spatial coding in parietal cortex mediates tactile-motor transformation

Movements towards touch on the body require integrating tactile location and body posture information. Tactile processing and movement planning both rely on posterior parietal cortex (PPC) but their interplay is not understood. Here, human participants received tactile stimuli on their crossed and uncrossed feet, dissociating stimulus location relative to anatomy versus external space. Participants pointed to the touch or the equivalent location on the other foot, which dissociates sensory and motor locations. Multi-voxel pattern analysis of concurrently recorded fMRI signals revealed that tactile location was coded anatomically in anterior PPC but spatially in posterior PPC during sensory processing. After movement instructions were specified, PPC exclusively represented the movement goal in space, in regions associated with visuo-motor planning and with regional overlap for sensory, rule-related, and movement coding. Thus, PPC flexibly updates its spatial codes to accommodate rule-based transformation of sensory input to generate movement to environment and own body alike.

Representational momentum of biological motion in full-body, point-light and single-dot displays

Observing the actions of others triggers, in our brain, an internal and automatic simulation of its unfolding in time. Here, we investigated whether the instantaneous internal representation of an observed action is modulated by the point of view under which an action is observed and the stimulus type. To this end, we motion captured the elliptical arm movement of a human actor and used these trajectories to animate a photorealistic avatar, a point-light stimulus or a single dot rendered either from an egocentric or an allocentric point of view. Crucially, the underlying physical characteristics of the movement were the same in all conditions. In a representational momentum paradigm, we then asked subjects to report the perceived last position of an observed movement at the moment in which the stimulus was randomly stopped. In all conditions, subjects tended to misremember the last configuration of the observed stimulus as being further forward than the veridical last showed position. This misrepresentation was however significantly smaller for full-body stimuli compared to point-light and single dot displays and it was not modulated by the point of view. It was also smaller when first-person full body stimuli were compared with a stimulus consisting of a solid shape moving with the same physical motion. We interpret these findings as evidence that full-body stimuli elicit a simulation process that is closer to the instantaneous veridical configuration of the observed movements while impoverished displays (both point-light and single-dot) elicit a prediction that is further forward in time. This simulation process seems to be independent from the point of view under which the actions are observed.

A shared neural code for the physics of actions and object events

Observing others' actions recruits frontoparietal and posterior temporal brain regions - also called the action observation network. It is typically assumed that these regions support recognizing actions of animate entities (e.g., person jumping over a box). However, objects can also participate in events with rich meaning and structure (e.g., ball bouncing over a box). So far, it has not been clarified which brain regions encode information specific to goal-directed actions or more general information that also defines object events. Here, we show a shared neural code for visually presented actions and object events throughout the action observation network. We argue that this neural representation captures the structure and physics of events regardless of animacy. We find that lateral occipitotemporal cortex encodes information about events that is also invariant to stimulus modality. Our results shed light onto the representational profiles of posterior temporal and frontoparietal cortices, and their roles in encoding event information.

Functional organization of social perception in the human brain

Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.

Two distinct networks containing position-tolerant representations of actions in the human brain

Humans can recognize others' actions in the social environment. This action recognition ability is rarely hindered by the movement of people in the environment. The neural basis of this position tolerance for observed actions is not fully understood. Here, we aimed to identify brain regions capable of generalizing representations of actions across different positions and investigate the representational content of these regions. In a functional magnetic resonance imaging experiment, participants viewed point-light displays of different human actions. Stimuli were presented in either the upper or the lower visual field. Multivariate pattern analysis and a surface-based searchlight approach were employed to identify brain regions that contain position-tolerant action representation: Classifiers were trained with patterns in response to stimuli presented in one position and were tested with stimuli presented in another position. Results showed above-chance classification in the left and right lateral occipitotemporal cortices, right intraparietal sulcus, and right postcentral gyrus. Further analyses exploring the representational content of these regions showed that responses in the lateral occipitotemporal regions were more related to subjective judgments, while those in the parietal regions were more related to objective measures. These results provide evidence for two networks that contain abstract representations of human actions with distinct representational content.

Disentangling hand and tool processing: Distal effects of neuromodulation

Neural processing within a local brain region that responds to more than one object category (e.g., hands and tools) nonetheless have different functional connectivity patterns with other distal brain areas, which suggests that local processing can affect and/or be affected by processing in distal areas, in a category-specific way. Here we wanted to test whether administering either a hand- or tool-related training task in tandem with transcranial direct current stimulation (tDCS) to a region that responds both to hands and tools (posterior middle temporal gyrus; pMTG), modulated local and distal neural processing more for the trained than the untrained category in a subsequent fMRI task. After each combined tDCS/training session, participants viewed images of tools, hands, and animals, in an fMRI scanner. Using multivoxel pattern analysis, we found that tDCS stimulation to pMTG indeed improved the classification accuracy between tools vs. animals, but only when combined with a tool and not a hand training task. Surprisingly, tDCS stimulation to pMTG also improved classification accuracy between hands vs. animals when combined with a tool but not a hand training task. Our findings suggest that overlapping but functionally-specific networks may be engaged separately by using a category-specific training task together with tDCS - a strategy that can be applied more broadly to other cognitive domains using tDCS. By hypothesis, these effects on local processing are a direct result of within-domain connectivity constraints from domain-specific networks that are at play in the processing and organization of object representations.

Representing linguistic communicative functions in the premotor cortex

Linguistic communication is often regarded as an action that serves a function to convey the speaker's goal to the addressee. Here, with an functional magnetic resonance imaging (fMRI) study and a lesion study, we demonstrated that communicative functions are represented in the human premotor cortex. Participants read scripts involving 2 interlocutors. Each script contained a critical sentence said by the speaker with a communicative function of either making a Promise, a Request, or a Reply to the addressee's query. With various preceding contexts, the critical sentences were supposed to induce neural activities associated with communicative functions rather than specific actions literally described by these sentences. The fMRI results showed that the premotor cortex contained more information, as revealed by multivariate analyses, on communicative functions and relevant interlocutors' attitudes than the perisylvian language regions. The lesion study results showed that, relative to healthy controls, the understanding of communicative functions was impaired in patients with lesions in the premotor cortex, whereas no reliable difference was observed between the healthy controls and patients with lesions in other brain regions. These findings convergently suggest the crucial role of the premotor cortex in representing the functions of linguistic communications, supporting that linguistic communication can be seen as an action.

Different Neural Information Flows Affected by Activity Patterns for Action and Verb Generation

Shared brain regions have been found for processing action and language, including the left inferior frontal gyrus (IFG), the premotor cortex (PMC), and the inferior parietal lobule (IPL). However, in the context of action and language generation that shares the same action semantics, it is unclear whether the activity patterns within the overlapping brain regions would be the same. The changes in effective connectivity affected by these activity patterns are also unclear. In this fMRI study, participants were asked to perform hand action and verb generation tasks toward object pictures. We identified shared and specific brain regions for the two tasks in the left PMC, IFG, and IPL. The mean activation level and multi-voxel pattern analysis revealed that the activity patterns in the shared sub-regions were distinct for the two tasks. The dynamic causal modeling results demonstrated that the information flows for the two tasks were different across the shared sub-regions. These results provided the first neuroimaging evidence that the action and verb generation were task context driven in the shared regions, and the distinct patterns of neural information flow across the PMC-IFG-IPL neural network were affected by the polymodal processing in the shared regions.

Updating the Chieti Affective Action Videos database with older adults

Validation of the Chieti Affective Action Videos (CAAV) database was replicated with a sample of older adults (age range 65-93). When designing experimental studies of emotions, it is crucial to take into consideration the differences in emotional processing between young and older adults. Therefore, the main goal of the present study was to provide an appropriate dataset for the use of CAAV in aging research. For this reason, the CAAV administration and the data collection methodology was faithfully replicated in a sample of 302 older adults. All the 360 standardized stimuli were evaluated on the emotional dimensions of valence and arousal. The CAAV validation in an older adults' population increases the potential use of this innovative tool. The present validation supports the use of the CAAV database in future experimental studies on cognitive functions in healthy and pathological aging.

What modulates the Mirror Neuron System during action observation?: Multiple factors involving the action, the actor, the observer, the relationship between actor and observer, and the context

Seeing an agent perform an action typically triggers a motor simulation of that action in the observer's Mirror Neuron System (MNS). Over the past few years, it has become increasingly clear that during action observation the patterns and strengths of responses in the MNS are modulated by multiple factors. The first aim of this paper is therefore to provide the most comprehensive survey to date of these factors. To that end, 22 distinct factors are described, broken down into the following sets: six involving the action; two involving the actor; nine involving the observer; four involving the relationship between actor and observer; and one involving the context. The second aim is to consider the implications of these findings for four prominent theoretical models of the MNS: the Direct Matching Model; the Predictive Coding Model; the Value-Driven Model; and the Associative Model. These assessments suggest that although each model is supported by a wide range of findings, each one is also challenged by other findings and relatively unaffected by still others. Hence, there is now a pressing need for a richer, more inclusive model that is better able to account for all of the modulatory factors that have been identified so far.

The extrastriate body area is involved in reciprocal imitation of hand gestures, vocalizations, and facial expressions: A univariate and multivariate fMRI study

The extrastriate body area (EBA) in the lateral occipito-temporal cortex has an important role in reciprocal interaction, as it detects congruence between self and other's hand actions. However, it is unclear whether the EBA can detect congruence regardless of the type of action. In the present study, we examined the neural substrate underlying congruence detection of three types of actions: hand gestures, vocalizations, and facial expressions. A univariate analysis revealed a congruency effect, especially for imitating action, for all three types of actions in the EBA. A multi-voxel pattern analysis classifier in the EBA was able to distinguish between initiating interaction from responding to interaction in all experiments. Correspondingly, the congruency effect in the EBA revealed by univariate analysis was stronger for responding to than for initiating interaction. These findings suggest that the EBA might contribute to detect congruence regardless of the body part used (i.e. face or hand) and the type of action (i.e. gestural or vocal). Moreover, initiating and responding to interaction might be processed differently within the EBA. This study highlights the role of the EBA in comparing between self and other's actions beyond hand actions.Running head: Function of EBA in reciprocal imitation.

Scene context shapes category representational geometry during processing of tools

Tools are ubiquitous in human environments and to think about them we use concepts. Increasingly, conceptual representation is thought to be dynamic and sensitive to the goals of the observer. Indeed, observer goals can reshape representational geometry within cortical networks supporting concepts. In the present study, we investigated the novel hypothesis that task-irrelevant scene context may implicitly alter the representational geometry of regions within the tool network. Participants performed conceptual judgments on images of tools embedded in scenes that either suggested their use (i.e., a kitchen timer sitting on a kitchen counter with vegetables in a frying pan) or that they would simply be moved (i.e., a kitchen timer sitting in an open drawer with other miscellaneous kitchen items around). We investigated whether representations in the tool network reflect category, grip, and shape information using a representational similarity analysis (RSA). We show that a) a number of regions of the tool network reflect category information about tools and b) category information predicts patterns in supramarginal gyrus more strongly in use contexts than in move contexts. Together, these results show that information about tool category is distributed across different regions of the tool network and that scene context helps shape the representational geometry of the tool network.

Decoding stimuli (tool-hand) and viewpoint invariant grasp-type information

When we see a manipulable object (henceforth tool) or a hand performing a grasping movement, our brain is automatically tuned to how that tool can be grasped (i.e., its affordance) or what kind of grasp that hand is performing (e.g., a power or precision grasp). However, it remains unclear where visual information related to tools or hands are transformed into abstract grasp representations. We therefore investigated where different levels of abstractness in grasp information are processed: grasp information that is invariant to the kind of stimuli that elicits it (tool-hand invariance); and grasp information that is hand-specific but viewpoint-invariant (viewpoint invariance). We focused on brain areas activated when viewing both tools and hands, i.e., the posterior parietal cortices (PPC), ventral premotor cortices (PMv), and lateral occipitotemporal cortex/posterior middle temporal cortex (LOTC/pMTG). To test for invariant grasp representations, we presented participants with tool images and grasp videos (from first or third person perspective; 1pp or 3pp) inside an MRI scanner, and cross-decoded power versus precision grasps across (i) grasp perspectives (viewpoint invariance), (ii) tool images and grasp 1pp videos (tool-hand 1pp invariance), and (iii) tool images and grasp 3pp videos (tool-hand 3pp invariance). Tool-hand 1pp, but not tool-hand 3pp, invariant grasp information was found in left PPC, whereas viewpoint-invariant information was found bilaterally in PPC, left PMv, and left LOTC/pMTG. These findings suggest different levels of abstractness-where visual information is transformed into stimuli-invariant grasp representations/tool affordances in left PPC, and viewpoint invariant but hand-specific grasp representations in the hand network.

Brain mechanisms of visuospatial perspective-taking in relation to object mental rotation and the theory of mind

Visuospatial perspective-taking (VPT) is a process of imagining what can be seen and how a scene looks from a location and orientation in space that differs from one's own. It comprises two levels that are underpinned by distinct neurocognitive processes. Level-2 VPT is often studied in relation to two other cognitive phenomena, object mental rotation (oMR) and theory of mind (ToM). With the aim to describe the broad picture of neurocognitive processes underlying level-2 VPT, here we give an overview of the recent behavioral and neuroscientific findings of level-2 VPT. We discuss its relation to level-1 VPT, which is also referred to as perspective-tracking, and the neighboring topics, oMR and ToM. Neuroscientific research shows that level-2 VPT is a diverse cognitive process, encompassing functionally distinct neural circuits. It shares brain substrates with oMR, especially those parietal brain areas that are specialized in spatial reasoning. However, compared to oMR, level-2 VPT involves additional activations in brain structures that are typically involved in ToM tasks and deal with self/other distinctions. In addition, level-2 VPT has been suggested to engage brain areas coding for internal representations of the body. Thus, the neurocognitive model underpinning level-2 VPT can be understood as a combination of visuospatial processing with social cognition and body schema representations.

Reconstructing neural representations of tactile space

Psychophysical experiments have demonstrated large and highly systematic perceptual distortions of tactile space. Such a space can be referred to our experience of the spatial organisation of objects, at representational level, through touch, in analogy with the familiar concept of visual space. We investigated the neural basis of tactile space by analysing activity patterns induced by tactile stimulation of nine points on a 3 × 3 square grid on the hand dorsum using functional magnetic resonance imaging. We used a searchlight approach within pre-defined regions of interests to compute the pairwise Euclidean distances between the activity patterns elicited by tactile stimulation. Then, we used multidimensional scaling to reconstruct tactile space at the neural level and compare it with skin space at the perceptual level. Our reconstructions of the shape of skin space in contralateral primary somatosensory and motor cortices reveal that it is distorted in a way that matches the perceptual shape of skin space. This suggests that early sensorimotor areas critically contribute to the distorted internal representation of tactile space on the hand dorsum.

Spontaneously emerging patterns in human visual cortex and their functional connectivity are linked to the patterns evoked by visual stimuli

The function of spontaneous brain activity is an important issue in neuroscience. Here we test the hypothesis that patterns of spontaneous activity code representational patterns evoked by stimuli. We compared in human visual cortex multivertex patterns of spontaneous activity to patterns evoked by ecological visual stimuli (faces, bodies, scenes) and low-level visual features (e.g., phase-scrambled faces). Specifically, we identified regions that preferred particular stimulus categories during localizer scans (e.g., extrastriate body area for bodies), measured multivertex patterns for each category during event-related task scans, and then correlated over vertices these stimulus-evoked patterns to the pattern measured on each frame of resting-state scans. The mean correlation coefficient was essentially zero for all regions/stimulus categories, indicating that resting multivertex patterns were not biased toward particular stimulus-evoked patterns. However, the spread of correlation coefficients between stimulus-evoked and resting patterns, positive and negative, was significantly greater for the preferred stimulus category of an ROI. The relationship between spontaneous and stimulus-evoked multivertex patterns also governed the temporal correlation or functional connectivity of patterns of spontaneous activity between individual regions (pattern-based functional connectivity). Resting multivertex patterns related to an object category fluctuated preferentially between ROIs preferring the same category, and fluctuations of the pattern for a category (e.g., body) within its preferred ROIs were largely uncorrelated with fluctuations of the pattern for a disparate category (e.g., scene) within its preferred ROIs. These results support the proposal that spontaneous multivertex activity patterns are linked to stimulus-evoked patterns, consistent with a representational function for spontaneous activity.NEW & NOTEWORTHY Spontaneous brain activity was once thought to reflect only noise, but evidence of strong spatiotemporal regularities has motivated a search for functional explanations. Here we show that the spatial pattern of spontaneous activity in human high-level and early visual cortex is related to the spatial patterns evoked by stimuli. Moreover, these patterns partly govern spontaneous spatiotemporal interactions between regions, so-called functional connectivity. These results support the hypothesis that spontaneous activity serves a representational function.

A shared neural substrate for action verbs and observed actions in human posterior parietal cortex

High-level sensory and motor cortical areas are activated when processing the meaning of language, but it is unknown whether, and how, words share a neural substrate with corresponding sensorimotor representations. We recorded from single neurons in human posterior parietal cortex (PPC) while participants viewed action verbs and corresponding action videos from multiple views. We find that PPC neurons exhibit a common neural substrate for action verbs and observed actions. Further, videos were encoded with mixtures of invariant and idiosyncratic responses across views. Action verbs elicited selective responses from a fraction of these invariant and idiosyncratic neurons, without preference, thus associating with a statistical sampling of the diverse sensory representations related to the corresponding action concept. Controls indicated that the results are not the product of visual imagery or arbitrary learned associations. Our results suggest that language may activate the consolidated visual experience of the reader.

Motor resonance in monkey parietal and premotor cortex during action observation: Influence of viewing perspective and effector identity

Observing others performing motor acts like grasping has been shown to elicit neural responses in the observer`s parieto-frontal motor network, which typically becomes active when the observer would perform these actions him/herself. While some human studies suggested strongest motor resonance during observation of first person or egocentric perspectives compared to third person or allocentric perspectives, other research either report the opposite or did not find any viewpoint-related preferences in parieto-premotor cortices. Furthermore, it has been suggested that these motor resonance effects are lateralized in the parietal cortex depending on the viewpoint and identity of the observed effector (left vs right hand). Other studies, however, do not find such straightforward hand identity dependent motor resonance effects. In addition to these conflicting findings in human studies, to date, little is known about the modulatory role of viewing perspective and effector identity (left or right hand) on motor resonance effects in monkey parieto-premotor cortices. Here, we investigated the extent to which different viewpoints of observed conspecific hand actions yield motor resonance in rhesus monkeys using fMRI. Observing first person, lateral and third person viewpoints of conspecific hand actions yielded significant activations throughout the so-called action observation network, including STS, parietal and frontal cortices. Although region-of-interest analysis of parietal and premotor motor/mirror neuron regions AIP, PFG and F5, showed robust responses in these regions during action observation in general, a clear preference for egocentric or allocentric perspectives was not evident. Moreover, except for lateralized effects due to visual field biases, motor resonance in the monkey brain during grasping observation did not reflect hand identity dependent coding.

Hyperalignment of motor cortical areas based on motor imagery during action observation

Multivariate Pattern Analysis (MVPA) has grown in importance due to its capacity to use both coarse and fine scale patterns of brain activity. However, a major limitation of multivariate analysis is the difficulty of aligning features across brains, which makes MVPA a subject specific analysis. Recent work by Haxby et al. (2011) introduced a method called Hyperalignment that explored neural activity in ventral temporal cortex during object recognition and demonstrated the ability to align individual patterns of brain activity into a common high dimensional space to facilitate Between Subject Classification (BSC). Here we examined BSC based on Hyperalignment of motor cortex during a task of motor imagery of three natural actions (lift, knock and throw). To achieve this we collected brain activity during the combined tasks of action observation and motor imagery to a parametric action space containing 25 stick-figure blends of the three natural actions. From these responses we derived Hyperalignment transformation parameters that were used to map subjects’ representational spaces of the motor imagery task in the motor cortex into a common model representational space. Results showed that BSC of the neural response patterns based on Hyperalignment exceeded both BSC based on anatomical alignment as well as a standard Within Subject Classification (WSC) approach. We also found that results were sensitive to the order in which participants entered the Hyperalignment algorithm. These results demonstrate the effectiveness of Hyperalignment to align neural responses across subject in motor cortex to enable BSC of motor imagery.

The Chieti Affective Action Videos database, a resource for the study of emotions in psychology

The Chieti Affective Action Videos (CAAV) is a new database designed for the experimental study of emotions in psychology. The main goal of the CAAV is to provide a wide range of standardized stimuli based on two emotional dimensions: valence and arousal. The CAAV is the first database to present emotional stimuli through videos of actions filmed and developed specifically for experimental research. 444 young adults were recruited to evaluate this database, which consisted of a sub-set of 90 actions filmed in four versions, for a total of 360 videos. The four versions differ based on the gender of the main actor (male or female) and in the perspective in which each action was shot (first-person or third-person). CAAV validation procedure highlighted a distribution of different stimuli based on valence and arousal indexes. The material provided by CAAV can be used in future experimental studies investigating the role of emotions, perception, attention, and memory in addition to the study of differences between gender and perspective taking.

Hierarchical Action Encoding Within the Human Brain

Humans are able to interact with objects with extreme flexibility. To achieve this ability, the brain does not only control specific muscular patterns, but it also needs to represent the abstract goal of an action, irrespective of its implementation. It is debated, however, how abstract action goals are implemented in the brain. To address this question, we used multivariate pattern analysis of functional magnetic resonance imaging data. Human participants performed grasping actions (precision grip, whole hand grip) with two different wrist orientations (canonical, rotated), using either the left or right hand. This design permitted to investigate a hierarchical organization consisting of three levels of abstraction: 1) “concrete action” encoding; 2) “effector-dependent goal” encoding (invariant to wrist orientation); and 3) “effector-independent goal” encoding (invariant to effector and wrist orientation). We found that motor cortices hosted joint encoding of concrete actions and of effector-dependent goals, while the parietal lobe housed a convergence of all three representations, comprising action goals within and across effectors. The left lateral occipito-temporal cortex showed effector-independent goal encoding, but no convergence across the three levels of representation. Our results support a hierarchical organization of action encoding, shedding light on the neural substrates supporting the extraordinary flexibility of human hand behavior.

The representational space of observed actions

Categorizing and understanding other people’s actions is a key human capability. Whereas there exists a growing literature regarding the organization of objects, the representational space underlying the organization of observed actions remains largely unexplored. Here we examined the organizing principles of a large set of actions and the corresponding neural representations. Using multiple regression representational similarity analysis of fMRI data, in which we accounted for variability due to major action components (body parts, scenes, movements, objects, sociality, transitivity) and three control models (distance between observer and actor, number of people, HMAX-C1), we found that the semantic dissimilarity structure was best captured by patterns of activation in the lateral occipitotemporal cortex (LOTC). Together, our results demonstrate that the organization of observed actions in the LOTC resembles the organizing principles used by participants to classify actions behaviorally, in line with the view that this region is crucial for accessing the meaning of actions.

The role of putative human anterior intraparietal sulcus area in observed manipulative action discrimination

INTRODUCTION

Although it has become widely accepted that the action observation network (AON) includes three levels (occipito-temporal, parietal and premotor), little is known concerning the specific role of these levels within perceptual tasks probing action observation. Recent single cell studies suggest that the parietal level carries the information required to discriminate between two-alternative observed actions, but do not exclude possible contributions from the other two levels.

METHODS

Two functional magnetic resonance imaging experiments used a task-based attentional modulation paradigm in which subjects viewed videos of an actor performing a manipulative action on a coloured object, and discriminated between either two observed manipulative actions, two actors or two colours.

RESULTS

Both experiments demonstrated that relative to actor and colour discrimination, discrimination between observed manipulative actions involved the putative human anterior intraparietal sulcus (phAIP) area in parietal cortex. In one experiment, where the observed actions also differed with regard to effectors, premotor cortex was also specifically recruited.

CONCLUSIONS

Our results highlight the primary role of parietal cortex in discriminating between two-alternative observed manipulative actions, consistent with the view that this level plays a major role in representing the identity of an observed action.

Large-Scale Organization of the Hand Action Observation Network in Individuals Born Without Hands

The human high-level visual cortex comprises regions specialized for the processing of distinct types of stimuli, such as objects, animals, and human actions. How does this specialization emerge? Here, we investigated the role of effector-specific visuomotor coupling experience in shaping the organization of the action observation network (AON) as a window on this question. Observed body movements are frequently coupled with corresponding motor codes, e.g., during monitoring one’s own movements and imitation, resulting in bidirectionally connected circuits between areas involved in body movements observation (e.g., of the hand) and the motor codes involved in their execution. If the organization of the AON is shaped by this effector-specific visuomotor coupling, then, it should not form for body movements that do not belong to individuals’ motor repertoire. To test this prediction, we used fMRI to investigate the spatial arrangement and functional properties of the hand and foot action observation circuits in individuals born without upper limbs. Multivoxel pattern decoding, pattern similarity, and univariate analyses revealed an intact hand AON in the individuals born without upper limbs. This suggests that the organization of the AON does not require effector-specific visuomotor coupling.

Dance Training Shapes Action Perception and Its Neural Implementation within the Young and Older Adult Brain

How we perceive others in action is shaped by our prior experience. Many factors influence brain responses when observing others in action, including training in a particular physical skill, such as sport or dance, and also general development and aging processes. Here, we investigate how learning a complex motor skill shapes neural and behavioural responses among a dance-naïve sample of 20 young and 19 older adults. Across four days, participants physically rehearsed one set of dance sequences, observed a second set, and a third set remained untrained. Functional MRI was obtained prior to and immediately following training. Participants' behavioural performance on motor and visual tasks improved across the training period, with younger adults showing steeper performance gains than older adults. At the brain level, both age groups demonstrated decreased sensorimotor cortical engagement after physical training, with younger adults showing more pronounced decreases in inferior parietal activity compared to older adults. Neural decoding results demonstrate that among both age groups, visual and motor regions contain experience-specific representations of new motor learning. By combining behavioural measures of performance with univariate and multivariate measures of brain activity, we can start to build a more complete picture of age-related changes in experience-dependent plasticity.

Imagined and Executed Actions in the Human Motor System: Testing Neural Similarity Between Execution and Imagery of Actions with a Multivariate Approach

Simulation theory proposes motor imagery (MI) to be a simulation based on representations also used for motor execution (ME). Nonetheless, it is unclear how far they use the same neural code. We use multivariate pattern analysis (MVPA) and representational similarity analysis (RSA) to describe the neural representations associated with MI and ME within the frontoparietal motor network. During functional magnetic resonance imaging scanning, 20 volunteers imagined or executed 3 different types of right-hand actions. Results of MVPA showed that these actions as well as their modality (MI or ME) could be decoded significantly above chance from the spatial patterns of BOLD signals in premotor and posterior parietal cortices. This was also true for cross-modal decoding. Furthermore, representational dissimilarity matrices of frontal and parietal areas showed that MI and ME representations formed separate clusters, but that the representational organization of action types within these clusters was identical. For most ROIs, this pattern of results best fits with a model that assumes a low-to-moderate degree of similarity between the neural patterns associated with MI and ME. Thus, neural representations of MI and ME are neither the same nor totally distinct but exhibit a similar structural geometry with respect to different types of action.

Mirror and (absence of) counter-mirror responses to action sounds measured with TMS

To what extent is the mirror neuron mechanism malleable to experience? The answer to this question can help characterising its ontogeny and its role in social cognition. Some suggest that it develops through sensorimotor associations congruent with our own actions. Others argue for its extreme volatility that will encode any sensorimotor association in the environment. Here, we added to this debate by exploring the effects of short goal-directed 'mirror' and 'counter-mirror' trainings (a 'mirror' training is defined as the first type of training encountered by the participants) on human auditory mirror motor-evoked potentials (MEPs). We recorded MEPs in response to two tones void of previous motor meaning, before and after mirror and counter-mirror trainings in which participants generated two tones of different pitch by performing free-choice button presses. The results showed that mirror MEPs, once established, were protected against an equivalent counter-mirror experience: they became manifest very rapidly and the same number of training trials that lead to the initial association did not suffice to reverse the MEP pattern. This steadiness of the association argues that, by serving direct-matching purposes, the mirror mechanism is a good solution for social cognition.

Visual and visuomotor processing of hands and tools as a case study of cross talk between the dorsal and ventral streams

A major principle of organization of the visual system is between a dorsal stream that processes visuomotor information and a ventral stream that supports object recognition. Most research has focused on dissociating processing across these two streams. Here we focus on how the two streams interact. We tested neurologically-intact and impaired participants in an object categorization task over two classes of objects that depend on processing within both streams-hands and tools. We measured how unconscious processing of images from one of these categories (e.g., tools) affects the recognition of images from the other category (i.e., hands). Our findings with neurologically-intact participants demonstrated that processing an image of a hand hampers the subsequent processing of an image of a tool, and vice versa. These results were not present in apraxic patients (N = 3). These findings suggest local and global inhibitory processes working in tandem to co-register information across the two streams.

Investigating common coding of observed and executed actions in the monkey brain using cross-modal multi-variate fMRI classification

Mirror neurons are generally described as a neural substrate hosting shared representations of actions, by simulating or 'mirroring' the actions of others onto the observer's own motor system. Since single neuron recordings are rarely feasible in humans, it has been argued that cross-modal multi-variate pattern analysis (MVPA) of non-invasive fMRI data is a suitable technique to investigate common coding of observed and executed actions, allowing researchers to infer the presence of mirror neurons in the human brain. In an effort to close the gap between monkey electrophysiology and human fMRI data with respect to the mirror neuron system, here we tested this proposal for the first time in the monkey. Rhesus monkeys either performed reach-and-grasp or reach-and-touch motor acts with their right hand in the dark or observed videos of human actors performing similar motor acts. Unimodal decoding showed that both executed or observed motor acts could be decoded from numerous brain regions. Specific portions of rostral parietal, premotor and motor cortices, previously shown to house mirror neurons, in addition to somatosensory regions, yielded significant asymmetric action-specific cross-modal decoding. These results validate the use of cross-modal multi-variate fMRI analyses to probe the representations of own and others' actions in the primate brain and support the proposed mapping of others' actions onto the observer's own motor cortices.

Attention Selectively Reshapes the Geometry of Distributed Semantic Representation

Humans prioritize different semantic qualities of a complex stimulus depending on their behavioral goals. These semantic features are encoded in distributed neural populations, yet it is unclear how attention might operate across these distributed representations. To address this, we presented participants with naturalistic video clips of animals behaving in their natural environments while the participants attended to either behavior or taxonomy. We used models of representational geometry to investigate how attentional allocation affects the distributed neural representation of animal behavior and taxonomy. Attending to animal behavior transiently increased the discriminability of distributed population codes for observed actions in anterior intraparietal, pericentral, and ventral temporal cortices. Attending to animal taxonomy while viewing the same stimuli increased the discriminability of distributed animal category representations in ventral temporal cortex. For both tasks, attention selectively enhanced the discriminability of response patterns along behaviorally relevant dimensions. These findings suggest that behavioral goals alter how the brain extracts semantic features from the visual world. Attention effectively disentangles population responses for downstream read-out by sculpting representational geometry in late-stage perceptual areas.

MEG adaptation reveals action representations in posterior occipitotemporal regions

When we observe other people's actions, a number of parietal and precentral regions known to be involved in the planning and execution of actions are recruited for example seen as power decreases in alpha and beta frequencies indicative of increased activation. It has been argued that this recruitment reflects the process of simulating the observed action, thereby providing access to the meaning of the action. Alternatively, it has been suggested that rather than providing access to the meaning of an action, parietal and precentral regions might be recruited as a consequence of action understanding. A way to distinguish between these alternatives is to examine where in the brain and at which time point it is possible to discriminate between different types of actions (e.g., pointing or grasping) irrespective of the way these are performed. To this aim, we presented participants with videos of simple hand actions performed with the left or right hand towards a target on the left or the right side while recording magnetoencephalography (MEG) data. In each trial, participants were presented with two subsequent videos (S1, S2) depicting either the same (repeat trials) or different (non-repeat trials) actions. We predicted that areas that are sensitive to the type of action should show stronger adaptation (i.e., a smaller decrease in alpha and beta power) in repeat in comparison to non-repeat trials. Indeed, we observed less alpha and beta power decreases during the presentation of S2 when the action was repeated compared to when two different actions were presented indicating adaptation of neuronal populations that are selective for the type of action. Sources were obtained exclusively in posterior occipitotemporal regions, supporting the notion that an early differentiation of actions occurs outside the motor system.

Encoding of event roles from visual scenes is rapid, spontaneous, and interacts with higher-level visual processing

A crucial component of event recognition is understanding event roles, i.e. who acted on whom: boy hitting girl is different from girl hitting boy. We often categorize Agents (i.e. the actor) and Patients (i.e. the one acted upon) from visual input, but do we rapidly and spontaneously encode such roles even when our attention is otherwise occupied? In three experiments, participants observed a continuous sequence of two-person scenes and had to search for a target actor in each (the male/female or red/blue-shirted actor) by indicating with a button press whether the target appeared on the left or the right. Critically, although role was orthogonal to gender and shirt color, and was never explicitly mentioned, participants responded more slowly when the target's role switched from trial to trial (e.g., the male went from being the Patient to the Agent). In a final experiment, we demonstrated that this effect cannot be fully explained by differences in posture associated with Agents and Patients. Our results suggest that extraction of event structure from visual scenes is rapid and spontaneous.

Investigating the uncanny valley for prosthetic hands

BACKGROUND

In 1970, Mori hypothesised the existence of an 'uncanny valley', whereby stimuli falling short of being fully human are found to be creepy or eerie.

OBJECTIVES

To investigate how eerie people find different prosthetic hands and whether perceptions of eeriness can be accounted for by categorical ambiguity.

STUDY DESIGN

Students participated in computerised experiments during which photographic images of hands were presented.

METHODS

We compared photographs of prosthetic hands pre-selected as more (H+) or less human-like (H-), as well as mechanical and real hands. Participants rated the hands for eeriness and human-likeness, as well as performing a speeded classification (human/non-human) and location judgment (control) task.

RESULTS

The H- prosthetic hands were rated as more eerie than the H+ prosthetic, mechanical and real hands, and this was unaffected by hand orientation. Participants were significantly slower to categorise the H+ prosthetic hands compared to the H- prosthetic and real hands, which was not due to generally slower responses to the H+ prosthetic hands (control task).

CONCLUSION

People find prosthetic hands to be eerie, most consistently for less human-like prosthetic hands. This effect is not driven by ambiguity about whether to categorise the prosthetic hand as human or artificial. Clinical relevance More obviously artificial, less-realistic, prosthetic hands consistently generate a sense of eeriness, while more realistic prosthetic hands avoid the uncanny valley, at least on initial viewing. Thus, greater realism in prosthetic design may not always incur a cost, although the role of movement and cutaneous input requires further investigation.

A fast, invariant representation for human action in the visual system

Humans can effortlessly recognize others’ actions in the presence of complex transformations, such as changes in viewpoint. Several studies have located the regions in the brain involved in invariant action recognition; however, the underlying neural computations remain poorly understood. We use magnetoencephalography decoding and a data set of well-controlled, naturalistic videos of five actions (run, walk, jump, eat, drink) performed by different actors at different viewpoints to study the computational steps used to recognize actions across complex transformations. In particular, we ask when the brain discriminates between different actions, and when it does so in a manner that is invariant to changes in 3D viewpoint. We measure the latency difference between invariant and noninvariant action decoding when subjects view full videos as well as form-depleted and motion-depleted stimuli. We were unable to detect a difference in decoding latency or temporal profile between invariant and noninvariant action recognition in full videos. However, when either form or motion information is removed from the stimulus set, we observe a decrease and delay in invariant action decoding. Our results suggest that the brain recognizes actions and builds invariance to complex transformations at the same time and that both form and motion information are crucial for fast, invariant action recognition.

NEW & NOTEWORTHY The human brain can quickly recognize actions despite transformations that change their visual appearance. We use neural timing data to uncover the computations underlying this ability. We find that within 200 ms action can be read out of magnetoencephalography data and that this representation is invariant to changes in viewpoint. We find form and motion are needed for this fast action decoding, suggesting that the brain quickly integrates complex spatiotemporal features to form invariant action representations.

Brain-to-brain hyperclassification reveals action-specific motor mapping of observed actions in humans

Seeing an action may activate the corresponding action motor code in the observer. It remains unresolved whether seeing and performing an action activates similar action-specific motor codes in the observer and the actor. We used novel hyperclassification approach to reveal shared brain activation signatures of action execution and observation in interacting human subjects. In the first experiment, two "actors" performed four types of hand actions while their haemodynamic brain activations were measured with 3-T functional magnetic resonance imaging (fMRI). The actions were videotaped and shown to 15 "observers" during a second fMRI experiment. Eleven observers saw the videos of one actor, and the remaining four observers saw the videos of the other actor. In a control fMRI experiment, one of the actors performed actions with closed eyes, and five new observers viewed these actions. Bayesian canonical correlation analysis was applied to functionally realign observers' and actors' fMRI data. Hyperclassification of the seen actions was performed with Bayesian logistic regression trained on actors' data and tested with observers' data. Without the functional realignment, between-subjects accuracy was at chance level. With the realignment, the accuracy increased on average by 15 percentage points, exceeding both the chance level and the accuracy without functional realignment. The highest accuracies were observed in occipital, parietal and premotor cortices. Hyperclassification exceeded chance level also when the actor did not see her own actions. We conclude that the functional brain activation signatures underlying action execution and observation are partly shared, yet these activation signatures may be anatomically misaligned across individuals.

Visuomotor effects of body part movements presented in the first-person perspective on imitative behavior

Imitative stimuli presented from a first-person perspective (FPP) produce stronger visuomotor effects than those presented from a third-person perspective (TPP) due to the relatively greater response of the mirror neuron system (MNS) to FPP stimuli. Some previous studies utilizing TPP stimuli have reported no differences in MNS activity between moving and static bodies' stimuli. However, few studies have compared visuomotor effects of such stimuli when presented in the FPP. To clarify this issue, we measured cortical activation in 17 participants during a functional magnetic resonance imaging (MRI) imitation task involving three conditions: moving (a lifting finger was presented), static (an "X" appeared on a static finger), and control (an "X" appeared on a button). All stimuli were presented from the FPP or TPP. Participants were asked to lift the finger corresponding to the imitative stimulus. In the FPP condition, moving stimuli elicited greater MNS activation than static stimuli. Furthermore, such movement effects were stronger in the MNS and insula (a region associated with body-ownership) for FPP stimuli than for TPP stimuli. Psychophysiological interaction analysis revealed increased connectivity between the MNS and insula for moving stimuli in the FPP condition. These findings suggest that bodily movements presented in the FPP elicit a greater visuomotor response than static body presented in the FPP, and that the visuomotor effects of bodily movements were greater in the FPP condition than in the TPP condition. Our analyses further indicated that such responses are processed via the neural system underlying body-ownership. Hum Brain Mapp 38:6218-6229, 2017. © 2017 Wiley Periodicals, Inc.

Decoding early and late cortical contributions to individuation of attended and unattended objects

To isolate a visual stimulus as a unique object with a specific spatial location and time of occurrence, it is necessary to first register (individuate) the stimulus as a distinct perceptual entity. Recent investigations into the neural substrates of object individuation have suggested it is subserved by a distributed neural network, but previous manipulations of individuation load have introduced extraneous visual confounds, which might have yielded ambiguous findings, particularly in early cortical areas. Furthermore, while it has been assumed that selective attention is required for object individuation, there is no definitive evidence on the brain regions recruited for attended and ignored objects. Here we addressed these issues by combining functional magnetic resonance imaging (fMRI) with a novel object-enumeration paradigm in which to-be-individuated objects were defined by illusory contours, such that the physical elements of the display remained constant across individuation conditions. Multi-voxel pattern analyses revealed that attended objects modulated patterns of activity in early visual cortex, as well as frontal and parietal brain areas, as a function of object-individuation load. These findings suggest that object individuation recruits both early and later cortical areas, consistent with theoretical accounts proposing that this operation acts at the junction of feed-forward and feedback processing stages in visual analysis. We also found dissociations between brain regions involved in individuation of attended and unattended objects, suggesting that voluntary spatial attention influences the brain regions recruited for this process.

The neural basis of temporal individuation and its capacity limits in the human brain

Individuation refers to individuals' use of spatial and temporal properties to register objects as distinct perceptual events relative to other stimuli. Although behavioral studies have examined both spatial and temporal individuation, neuroimaging investigations have been restricted to the spatial domain and at relatively late stages of information processing. Here, we used univariate and multivoxel pattern analyses of functional MRI data to identify brain regions involved in individuating temporally distinct visual items and the neural consequences that arise when this process reaches its capacity limit (repetition blindness, RB). First, we found that regional patterns of blood-oxygen-level-dependent activity across the cortex discriminated between instances where repeated and nonrepeated stimuli were successfully individuated-conditions that placed differential demands on temporal individuation. These results could not be attributed to repetition suppression or other stimulus-related factors, task difficulty, regional activation differences, other capacity-limited processes, or artifacts in the data or analyses. Contrary to current theoretical models, this finding suggests that temporal individuation is supported by a distributed set of brain regions, rather than a single neural correlate. Second, conditions that reflect the capacity limit of individuation-instances of RB-lead to changes in the spatial patterns within this network, as well as amplitude changes in the left hemisphere premotor cortex, superior medial frontal cortex, anterior cingulate cortex, and bilateral parahippocampal place area. These findings could not be attributed to response conflict/ambiguity and likely reflect the core brain regions and mechanisms that underlie the capacity-limited process that gives rise to RB.NEW & NOTEWORTHY We present novel findings into the neural bases of temporal individuation and repetition blindness (RB)-the perceptual deficit that arises when this process reaches its capacity limit. Specifically, we found that temporal individuation is a widely distributed process in the brain and identified a number of candidate brain regions that appear to underpin RB. These findings enhance our understanding of how these fundamental perceptual processes are reflected in the human brain.

Spatial and viewpoint selectivity for others' observed actions in monkey ventral premotor mirror neurons

The spatial location and viewpoint of observed actions are closely linked in natural social settings. For example, actions observed from a subjective viewpoint necessarily occur within the observer’s peripersonal space. Neurophysiological studies have shown that mirror neurons (MNs) of the monkey ventral premotor area F5 can code the spatial location of live observed actions. Furthermore, F5 MN discharge can also be modulated by the viewpoint from which filmed actions are seen. Nonetheless, whether and to what extent MNs can integrate viewpoint and spatial location of live observed actions remains unknown. We addressed this issue by comparing the activity of 148 F5 MNs while macaque monkeys observed an experimenter grasping in three different combinations of viewpoint and spatial location, namely, lateral view in the (1) extrapersonal and (2) peripersonal space and (3) subjective view in the peripersonal space. We found that the majority of MNs were space-selective (60.8%): those selective for the peripersonal space exhibited a preference for the subjective viewpoint both at the single-neuron and population level, whereas space-unselective neurons were view invariant. These findings reveal the existence of a previously neglected link between spatial and viewpoint selectivity in MN activity during live-action observation.

Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity

How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC.

SIGNIFICANCE STATEMENT

The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.

Dynamic neural architecture for social knowledge retrieval

Social behavior is often shaped by the rich storehouse of biographical information that we hold for other people. In our daily life, we rapidly and flexibly retrieve a host of biographical details about individuals in our social network, which often guide our decisions as we navigate complex social interactions. Even abstract traits associated with an individual, such as their political affiliation, can cue a rich cascade of person-specific knowledge. Here, we asked whether the anterior temporal lobe (ATL) serves as a hub for a distributed neural circuit that represents person knowledge. Fifty participants across two studies learned biographical information about fictitious people in a 2-d training paradigm. On day 3, they retrieved this biographical information while undergoing an fMRI scan. A series of multivariate and connectivity analyses suggest that the ATL stores abstract person identity representations. Moreover, this region coordinates interactions with a distributed network to support the flexible retrieval of person attributes. Together, our results suggest that the ATL is a central hub for representing and retrieving person knowledge.