First-person and third-person verbs in visual motion-perception regions

The characterization of actions at the superordinate, basic and subordinate level

Objects can be categorized at different levels of abstraction, ranging from the superordinate (e.g., fruit) and the basic (e.g., apple) to the subordinate level (e.g., golden delicious). The basic level is assumed to play a key role in categorization, e.g., in terms of the number of features used to describe these actions and the speed of processing. To which degree do these principles also apply to the categorization of observed actions? To address this question, we first selected a range of actions at the superordinate (e.g., locomotion), basic (e.g., to swim) and subordinate level (e.g., to swim breaststroke), using verbal material (Experiments 1-3). Experiments 4-6 aimed to determine the characteristics of these actions across the three taxonomic levels. Using a feature listing paradigm (Experiment 4), we determined the number of features that were provided by at least six out of twenty participants (common features), separately for the three different levels. In addition, we examined the number of shared (i.e., provided for more than one category) and distinct (i.e., provided for one category only) features. Participants produced the highest number of common features for actions at the basic level. Actions at the subordinate level shared more features with other actions at the same level than those at the superordinate level. Actions at the superordinate and basic level were described with more distinct features compared to those provided at the subordinate level. Using an auditory priming paradigm (Experiment 5), we observed that participants responded faster to action images preceded by a matching auditory cue corresponding to the basic and subordinate level, but not for superordinate level cues, suggesting that the basic level is the most abstract level at which verbal cues facilitate the processing of an upcoming action. Using a category verification task (Experiment 6), we found that participants were faster and more accurate to verify action categories (depicted as images) at the basic and subordinate level in comparison to the superordinate level. Together, in line with the object categorization literature, our results suggest that information about action categories is maximized at the basic level.

Neurophysiological mechanisms of perspective-taking: An MEG investigation of agency

According to the embodied cognition framework, sensory and motor areas are recruited during language understanding through simulation processes. Behavioral and imaging findings point to a dependence of the latter on perspective-taking (e.g., first person "I" versus third person "s/he"). The current study aims at identifying possible neurophysiological correlates of perspective in a linguistic context. Twenty healthy participants were measured with magnetoencephalography (MEG) while semantically processing visually presented inflected German verbs in the first- and third-person perspective, simple present tense. Results show that the first-person perspective induces stronger beta (15-25 Hz) desynchronization in the right-hemispheric posterior superior temporal sulcus, ventral posterior cingulate gyrus, and V5/MT+ area; no modulation of sensorimotor cortex emerged. Moreover, a stronger event-related field (ERF) was observed for the first-person perspective at about 150 ms after pronoun-verb onset, originating in occipital and moving to central and left temporal cortical sites. No effect of perspective on sensory gating was found when targeting the N1 component related to tones following the linguistic stimuli. Results indicate an effect of linguistic perspective-taking on brain activation patterns. The contribution of the single brain areas and their role in self-other distinction is further discussed.

Moving Toward versus Away from Another: How Body Motion Direction Changes the Representation of Bodies and Actions in the Visual Cortex

Representing multiple agents and their mutual relations is a prerequisite to understand social events such as interactions. Using functional magnetic resonance imaging on human adults, we show that visual areas dedicated to body form and body motion perception contribute to processing social events, by holding the representation of multiple moving bodies and encoding the spatial relations between them. In particular, seeing animations of human bodies facing and moving toward (vs. away from) each other increased neural activity in the body-selective cortex [extrastriate body area (EBA)] and posterior superior temporal sulcus (pSTS) for biological motion perception. In those areas, representation of body postures and movements, as well as of the overall scene, was more accurate for facing body (vs. nonfacing body) stimuli. Effective connectivity analysis with dynamic causal modeling revealed increased coupling between EBA and pSTS during perception of facing body stimuli. The perceptual enhancement of multiple-body scenes featuring cues of interaction (i.e., face-to-face positioning, spatial proximity, and approaching signals) was supported by the participants' better performance in a recognition task with facing body versus nonfacing body stimuli. Thus, visuospatial cues of interaction in multiple-person scenarios affect the perceptual representation of body and body motion and, by promoting functional integration, streamline the process from body perception to action representation.

Neural correlates of object and action naming practice

Word retrieval deficits are a common problem in patients with stroke-induced brain damage. While complete recovery of language in chronic aphasia is rare, patients' naming ability can be significantly improved by speech therapy. A growing number of neuroimaging studies have tried to pinpoint the neural changes associated with successful outcome of naming treatment. However, the mechanisms supporting naming practice in the healthy brain have received little attention. Yet, understanding these mechanisms is crucial for teasing them apart from functional reorganization following brain damage. To address this issue, we trained a group of healthy monolingual Italian speakers on naming pictured objects and actions for ten consecutive days and scanned them before and after training. Although activity during object versus action naming dissociated in several regions (lateral occipitotemporal, parietal and left inferior frontal cortices), training effects for the two word classes were similar and included activation decreases in classical language regions of the left hemisphere (posterior inferior frontal gyrus, anterior insula), potentially due to decreased lexical selection demands. Additionally, MVPA revealed training-related activation changes in the left parietal and temporal cortices associated with the retrieval of knowledge from episodic memory (precuneus, angular gyrus) and facilitated access to phonological word forms (posterior superior temporal sulcus).

A norming study of high-quality video clips of pantomimes, emblems, and meaningless gestures

Recent years have witnessed a growing interest in behavioral and neuroimaging studies on the processing of symbolic communicative gestures, such as pantomimes and emblems, but well-controlled stimuli have been scarce. This study describes a dataset of more than 200 video clips of an actress performing pantomimes (gestures that mimic object-directed/object-use actions; e.g., playing guitar), emblems (conventional gestures; e.g., thumbs up), and meaningless gestures. Gestures were divided into four lists. For each of these four lists, 50 Italian and 50 American raters judged the meaningfulness of the gestures and provided names and descriptions for them. The results of these rating and norming measures are reported separately for the Italian and American raters, offering the first normed set of meaningful and meaningless gestures for experimental studies. The stimuli are available for download via the Figshare database.

The two-process theory of biological motion processing

Perception, identification, and understanding of others' actions from motion information are vital for our survival in the social world. A breakthrough in the understanding of action perception was the discovery that our visual system is sensitive to human action from the sparse motion input of only a dozen point lights, a phenomenon known as biological motion (BM) processing. Previous psychological and computational models cannot fully explain the emerging evidence for the existence of BM processing during early ontogeny. Here, we propose a two-process model of the mechanisms underlying BM processing. We hypothesize that the first system, the 'Step Detector,' rapidly processes the local foot motion and feet-below-the-body information that is specific to vertebrates, is less dependent on postnatal learning, and involves subcortical networks. The second system, the 'Bodily Action Evaluator,' slowly processes the fine global structure-from-motion, is specific to conspecific, and dependent on gradual learning processed in cortical networks. This proposed model provides new insight into research on the development of BM processing.

The neurocognitive basis of knowledge about object identity and events: dissociations reflect opposing effects of semantic coherence and control

Semantic memory encompasses knowledge of specific objects and their diverse associations, but the mechanisms that allow us to retrieve aspects of knowledge required for a given task are poorly understood. The dual hub theory suggests that separate semantic stores represent knowledge of (i) taxonomic categories (in the anterior temporal lobes, ATL) and (ii) thematic associations (in angular gyrus, AG or posterior middle temporal gyrus, pMTG). Alternatively, the controlled semantic cognition (CSC) framework suggests that semantic processing emerges from the flexible interaction of heteromodal semantic representations in ATL with a semantic control network, which includes pMTG as well as prefrontal regions. According to this view, ATL supports patterns of coherent auto-associative retrieval, while semantic control sites respond when ongoing conceptual activation needs to be altered to suit the task or context. These theories make different predictions about the nature of functional dissociations within the semantic network. We review evidence for these claims across multiple methods. First, we show ATL is sensitive to the strength of thematic associations as well as taxonomic relations. Next, we document functional dissociations between AG and pMTG: rather than these regions acting as comparable thematic hubs, AG is allied to the default mode network and supports more 'automatic' retrieval, while pMTG responds when control demands are high. However, the semantic control network, including pMTG, also shows a greater response to events/actions and verbs, supporting the claims of both theories. We propose that tasks tapping event semantics often require greater shaping of conceptual retrieval than comparison tasks, because these elements of our knowledge are inherently flexible, with relevant features depending on the context. In this way, the CSC account might be able to account for findings that suggest both a process and a content distinction within the semantic network. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

The Large-Scale Organization of Gestures and Words in the Middle Temporal Gyrus

The middle temporal gyrus (MTG) has been shown to be recruited during the processing of words, but also during the observation of actions. Here we investigated how information related to words and gestures is organized along the MTG. To this aim, we measured the BOLD response in the MTG to video clips of gestures and spoken words in 17 healthy human adults (male and female). Gestures consisted of videos of an actress performing object-use pantomimes (iconic representations of object-directed actions; e.g., playing guitar), emblems (conventional gestures, e.g., thumb up), and meaningless gestures. Word stimuli (verbs, nouns) consisted of video clips of the same actress pronouncing words. We found a stronger response to meaningful compared with meaningless gestures along the whole left and large portions of the right MTG. Importantly, we observed a gradient, with posterior regions responding more strongly to gestures (pantomimes and emblems) than words and anterior regions showing a stronger response to words than gestures. In an intermediate region in the left hemisphere, the response was significantly higher to words and emblems (i.e., items with a greater arbitrariness of the sign-to-meaning mapping) than to pantomimes. These results show that the large-scale organization of information in the MTG is driven by the input modality and may also reflect the arbitrariness of the relationship between sign and meaning.SIGNIFICANCE STATEMENT Here we investigated the organizing principle of information in the middle temporal gyrus, taking into consideration the input-modality and the arbitrariness of the relationship between a sign and its meaning. We compared the middle temporal gyrus response during the processing of pantomimes, emblems, and spoken words. We found that posterior regions responded more strongly to pantomimes and emblems than to words, whereas anterior regions responded more strongly to words than to pantomimes and emblems. In an intermediate region, only in the left hemisphere, words and emblems evoked a stronger response than pantomimes. Our results identify two organizing principles of neural representation: the modality of communication (gestural or verbal) and the (arbitrariness of the) relationship between sign and meanings.

Neural representation of word categories is distinct in the temporal lobe: An activation likelihood analysis

The distinction between nouns and verbs is a language universal. Yet, functional neuroimaging studies comparing noun and verb processing have yielded inconsistent findings, ranging from a complete frontal(verb)-temporal(noun) dichotomy to a complete overlap in activation patterns. The current study addressed the debate about neural distinctions between nouns and verbs by conducting an activation likelihood estimation (ALE) meta-analysis of probabilistic cytoarchitectonic maps. Two levels of analysis were conducted: simple effects (Verbs vs. Baseline, Nouns vs. Baseline), and direct comparisons (Verbs vs. Nouns, Nouns vs. Verbs). Nouns were uniquely associated with a left medial temporal cluster (BA37). Activation foci for verbs included extensive inferior frontal (BA44-47) and mid-temporal (BA22, 21) regions in the left hemisphere. These findings confirm that the two grammatical classes have distinct neural architecture in supra-modal brain regions. Further, nouns and verbs overlapped in a small left lateral inferior temporal activation cluster (BA37), which is a region for modality-independent, grammatical class-independent lexical representations. These findings are most consistent with the view that as one acquires language, linguistic representations for a lexical category shift from the modality specific cortices which represent prototypical members of that category (e.g., motion for verbs) to abstract amodal representations in close proximity to modality specific cortices.

The neural representation of human versus nonhuman bipeds and quadrupeds

How do humans recognize humans among other creatures? Recent studies suggest that a preference for conspecifics may emerge already in perceptual processing, in regions such as the right posterior superior temporal sulcus (pSTS), implicated in visual perception of biological motion. In the current functional MRI study, participants viewed point-light displays of human and nonhuman creatures moving in their typical bipedal (man and chicken) or quadrupedal mode (crawling-baby and cat). Stronger activity for man and chicken versus baby and cat was found in the right pSTS responsive to biological motion. The novel effect of pedalism suggests that, if right pSTS contributes to recognizing of conspecifics, it does so by detecting perceptual features (e.g. bipedal motion) that reliably correlate with their appearance. A searchlight multivariate pattern analysis could decode humans and nonhumans across pedalism in the left pSTS and bilateral posterior cingulate cortex. This result implies a categorical human-nonhuman distinction, independent from within-category physical/perceptual variation. Thus, recognizing conspecifics involves visual classification based on perceptual features that most frequently co-occur with humans, such as bipedalism, and retrieval of information that determines category membership above and beyond visual appearance. The current findings show that these processes are at work in separate brain networks.

Neural bases of action abstraction

There has been recent debate over whether actions are processed primarily by means of motor simulation or cognitive semantics. The current study investigated how abstract action concepts are processed in the brain, independent of the format in which they are presented. Eighteen healthy adult participants viewed different actions (e.g., diving, boxing) in the form of verbs and schematic action pictograms while functional magnetic resonance imaging (fMRI) was collected. We predicted that sensorimotor and semantic brain regions would show similar patterns of neural activity for different instances of the same action (e.g., diving pictogram and the word 'diving'). A representational similarity analysis revealed posterior temporal and sensorimotor regions where specific action concepts were encoded, independent of the format of presentation. These results reveal the neural instantiations of abstract action concepts, and demonstrate that both sensorimotor and semantic systems are involved in processing actions.

Readers select a comprehension mode independent of pronoun: Evidence from fMRI during narrative comprehension

Perspective is a crucial feature for communicating about events. Yet it is unclear how linguistically encoded perspective relates to cognitive perspective taking. Here, we tested the effect of perspective taking with short literary stories. Participants listened to stories with 1st or 3rd person pronouns referring to the protagonist, while undergoing fMRI. When comparing action events with 1st and 3rd person pronouns, we found no evidence for a neural dissociation depending on the pronoun. A split sample approach based on the self-reported experience of perspective taking revealed 3 comprehension preferences. One group showed a strong 1st person preference, another a strong 3rd person preference, while a third group engaged in 1st and 3rd person perspective taking simultaneously. Comparing brain activations of the groups revealed different neural networks. Our results suggest that comprehension is perspective dependent, but not on the perspective suggested by the text, but on the reader's (situational) preference.

Voice over: Audio-visual congruency and content recall in the gallery setting

Experimental research has shown that pairs of stimuli which are congruent and assumed to ‘go together’ are recalled more effectively than an item presented in isolation. Will this multisensory memory benefit occur when stimuli are richer and longer, in an ecological setting? In the present study, we focused on an everyday situation of audio-visual learning and manipulated the relationship between audio guide tracks and viewed portraits in the galleries of the Tate Britain. By varying the gender and narrative style of the voice-over, we examined how the perceived congruency and assumed unity of the audio guide track with painted portraits affected subsequent recall. We show that tracks perceived as best matching the viewed portraits led to greater recall of both sensory and linguistic content. We provide the first evidence that manipulating crossmodal congruence and unity assumptions can effectively impact memory in a multisensory ecological setting, even in the absence of precise temporal alignment between sensory cues.

The lateral occipitotemporal cortex in action

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.