Action selectivity in parietal and temporal cortex

Computational Language Modeling and the Promise of In Silico Experimentation

Language neuroscience currently relies on two major experimental paradigms: controlled experiments using carefully hand-designed stimuli, and natural stimulus experiments. These approaches have complementary advantages which allow them to address distinct aspects of the neurobiology of language, but each approach also comes with drawbacks. Here we discuss a third paradigm-in silico experimentation using deep learning-based encoding models-that has been enabled by recent advances in cognitive computational neuroscience. This paradigm promises to combine the interpretability of controlled experiments with the generalizability and broad scope of natural stimulus experiments. We show four examples of simulating language neuroscience experiments in silico and then discuss both the advantages and caveats of this approach.

Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus

Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. "an apple," "a wedding") presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts.

Using Generative Artificial Intelligence to Classify Primary Progressive Aphasia from Connected Speech

Neurodegenerative dementia syndromes, such as Primary Progressive Aphasias (PPA), have traditionally been diagnosed based in part on verbal and nonverbal cognitive profiles. Debate continues about whether PPA is best subdivided into three variants and also regarding the most distinctive linguistic features for classifying PPA variants. In this study, we harnessed the capabilities of artificial intelligence (AI) and natural language processing (NLP) to first perform unsupervised classification of concise, connected speech samples from 78 PPA patients. Large Language Models discerned three distinct PPA clusters, with 88.5% agreement with independent clinical diagnoses. Patterns of cortical atrophy of three data-driven clusters corresponded to the localization in the clinical diagnostic criteria. We then used NLP to identify linguistic features that best dissociate the three PPA variants. Seventeen features emerged as most valuable for this purpose, including the observation that separating verbs into high and low-frequency types significantly improves classification accuracy. Using these linguistic features derived from the analysis of brief connected speech samples, we developed a classifier that achieved 97.9% accuracy in predicting PPA subtypes and healthy controls. Our findings provide pivotal insights for refining early-stage dementia diagnosis, deepening our understanding of the characteristics of these neurodegenerative phenotypes and the neurobiology of language processing, and enhancing diagnostic evaluation accuracy.

Representation of motion concepts in occipitotemporal cortex: fMRI activation, decoding and connectivity analyses

Embodied theories of semantic cognition predict that brain regions involved in motion perception are engaged when people comprehend motion concepts expressed in language. Left lateral occipitotemporal cortex (LOTC) is implicated in both motion perception and motion concept processing but prior studies have produced mixed findings on which parts of this region are engaged by motion language. We scanned participants performing semantic judgements about sentences describing motion events and static events. We performed univariate analyses, multivariate pattern analyses (MVPA) and psychophysiological interaction (PPI) analyses to investigate the effect of motion on activity and connectivity in different parts of LOTC. In multivariate analyses that decoded whether a sentence described motion or not, the middle and posterior parts of LOTC showed above-chance level performance, with performance exceeding that of other brain regions. Univariate ROI analyses found the middle part of LOTC was more active for motion events than static ones. Finally, PPI analyses found that when processing motion events, the middle and posterior parts of LOTC (overlapping with motion perception regions), increased their connectivity with cognitive control regions. Taken together, these results indicate that the more posterior parts of LOTC, including motion perception cortex, respond differently to motion vs. static events. These findings are consistent with embodiment accounts of semantic processing, and suggest that understanding verbal descriptions of motion engages areas of the occipitotemporal cortex involved in perceiving motion.

Neural correlates of embodied action language processing: a systematic review and meta-analytic study

The neural correlates of action language processing are still debated within embodied cognition research and little is known about the flexible involvement of modality-specific pre-motor system and multimodal high-level temporo-parietal regions as a function of explicit and implicit tasks. A systematic review and the Activation likelihood estimation (ALE) meta-analyses on functional neuroimaging studies were performed to identify neural correlates of action language processing activated during explicit and implicit tasks. The contrast ALE meta-analysis revealed activation of modality-specific premotor area and inferior frontal areas during explicit action language tasks while a greater activation of posterior temporo-occipital areas emerged for implicit tasks. The conjunction analysis revealed overlap in the temporo-parietal multimodal high-level regions for both types of tasks. Functional specialization of the middle temporal gyrus was found where the more posterior-occipital part resulted activated during implicit action language tasks whereas the antero-lateral part was involved in explicit tasks. Our findings were discussed within a conceptual flexibility perspective about the involvement of both the modality-specific and multimodal brain system during action language processing depending on different types of tasks.

Dissociation between function and manipulation in semantic representations of motor impaired subjects: A new test

A fundamental problem in semantic cognition is the representation of human concepts in the brain. Much of the knowledge acquired in the last decades comes from the study of dissociations found in patients with acquired difficulties in language, perception, and action. In particular, some deficits involve loss of knowledge about tools. The dissociation between two relevant aspects of tools, function and manipulation, has been the focus of several studies. In this paper, a new test designed to study the dissociation between function and manipulation is proposed and normative values for a control population are provided. This novel test was additionally administered to and evaluated in a group of Parkinson's disease patients. The Graded-Controlled Hub-and-Spoke model of Lambon Ralph, Jefferies, Patterson and Rogers was used as a theoretical guide to interpret the results.

Semantic and action tool knowledge in the brain: Identifying common and distinct networks

Most cognitive models of apraxia assume that impaired tool use results from a deficit occurring at the conceptual level, which contains dedicated information about tool use, namely, semantic and action tool knowledge. Semantic tool knowledge contains information about the prototypical use of familiar tools, such as function (e.g., a hammer and a mallet share the same purpose) and associative relations (e.g., a hammer goes with a nail). Action tool knowledge contains information about how to manipulate tools, such as hand posture and kinematics. The present review aimed to better understand the neural correlates of action and semantic tool knowledge, by focusing on activation, stimulation and patients' studies (left brain-damaged patients). We found that action and semantic tool knowledge rely upon a large brain network including temporal and parietal regions. Yet, while action tool knowledge calls into play the intraparietal sulcus, function relations mostly involve the anterior and posterior temporal lobe. Associative relations engaged the angular and the posterior middle temporal gyrus. Moreover, we found that hand posture and kinematics both tapped into the inferior parietal lobe and the lateral occipital temporal cortex, but no region specificity was found for one or the other representation. Our results point out the major role of both posterior middle temporal gyrus and inferior parietal lobe for action and semantic tool knowledge. They highlight the common and distinct brain networks involved in action and semantic tool networks and spur future directions on this topic.

Rapid microstructural plasticity in the cortical semantic network following a short language learning session

Despite the clear importance of language in our life, our vital ability to quickly and effectively learn new words and meanings is neurobiologically poorly understood. Conventional knowledge maintains that language learning—especially in adulthood—is slow and laborious. Furthermore, its structural basis remains unclear. Even though behavioural manifestations of learning are evident near instantly, previous neuroimaging work across a range of semantic categories has largely studied neural changes associated with months or years of practice. Here, we address rapid neuroanatomical plasticity accompanying new lexicon acquisition, specifically focussing on the learning of action-related language, which has been linked to the brain’s motor systems. Our results show that it is possible to measure and to externally modulate (using transcranial magnetic stimulation (TMS) of motor cortex) cortical microanatomic reorganisation after mere minutes of new word learning. Learning-induced microstructural changes, as measured by diffusion kurtosis imaging (DKI) and machine learning-based analysis, were evident in prefrontal, temporal, and parietal neocortical sites, likely reflecting integrative lexico-semantic processing and formation of new memory circuits immediately during the learning tasks. These results suggest a structural basis for the rapid neocortical word encoding mechanism and reveal the causally interactive relationship of modal and associative brain regions in supporting learning and word acquisition.

This combined neuroimaging and brain stimulation study reveals rapid and distributed microstructural plasticity after a single immersive language learning session, demonstrating the causal relevance of the motor cortex in encoding the meaning of novel action words.

Semantic association ability mediates the relationship between brain structure and human creativity

Creativity involves the ability to associate relatively weak or distant semantic components and combine them into novel and useful objects. Few studies have explored the brain mechanisms underlying semantic associative ability and its relationship with creativity based on semantic distance. In this study, the chain free association (CFA) task was performed, and semantic distance was quantified to measure individuals' semantic association ability, while the alternative use test (AUT) and creative activity (CAct) tasks were performed to measure creative ability. The behavioral results revealed a significant positive correlation between semantic distance and creativity. The voxel-based morphometry (VBM) analysis found the neural structural basis of semantic distance. Indeed, semantic distance was positively correlated with the gray matter volume (GMV) of the left posterior inferior temporal gyrus (LpITG), which is associated with visual word learning, semantic knowledge retrieval, and semantic memory, in addition to divergent thinking and creative traits. A mediation analysis showed semantic distance mediate the relationship between the regional GMV of LpITG and human creativity. Effectively, highly creative individuals with high regional GMV in LpITG were observed to have higher capacity of spontaneous association process. These findings shed light on the dedication of the brain areas related to remote semantic connectivity to creative thinking via individuals' spontaneous semantic association ability.

Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals

A class of semantic theories defines concepts in terms of statistical distributions of lexical items, basing meaning on vectors of word co-occurrence frequencies. A different approach emphasizes abstract hierarchical taxonomic relationships among concepts. However, the functional relevance of these different accounts and how they capture information-encoding of lexical meaning in the brain still remains elusive. We investigated to what extent distributional and taxonomic models explained word-elicited neural responses using cross-validated representational similarity analysis (RSA) of functional magnetic resonance imaging (fMRI) and model comparisons. Our findings show that the brain encodes both types of semantic information, but in distinct cortical regions. Posterior middle temporal regions reflected lexical-semantic similarity based on hierarchical taxonomies, in coherence with the action-relatedness of specific semantic word categories. In contrast, distributional semantics best predicted the representational patterns in left inferior frontal gyrus (LIFG, BA 47). Both representations coexisted in the angular gyrus supporting semantic binding and integration. These results reveal that neuronal networks with distinct cortical distributions across higher-order association cortex encode different representational properties of word meanings. Taxonomy may shape long-term lexical-semantic representations in memory consistently with the sensorimotor details of semantic categories, whilst distributional knowledge in the LIFG (BA 47) may enable semantic combinatorics in the context of language use. Our approach helps to elucidate the nature of semantic representations essential for understanding human language.

Distinct neural mechanisms underlying conceptual knowledge of manner and instrument verbs

Studies on the organization of conceptual knowledge have examined categories of concrete nouns extensively. Less is known about the neural basis of verb categories suggested by linguistic theories. We used functional MRI to examine the differences between manner verbs, which encode information about the manner of an action, versus instrument verbs, which encode information about an object as part of their meaning. Using both visual and verbal stimuli and a combination of univariate and multivariate pattern analyses, our results show that accessing conceptual representations of instrument class involves brain regions typically associated with complex action and object perception, including the anterior inferior parietal cortex and occipito-temporal cortex. On the other hand, accessing conceptual representations of the manner class involves regions that are commonly associated with the processing of visual and biological motion, in the posterior superior temporal sulcus. These findings support the idea that the semantics of manner and instrument verbs are supported by distinct neural mechanisms.

Effects of motion speed in action representations

Grounded cognition accounts of semantic representation posit that brain regions traditionally linked to perception and action play a role in grounding the semantic content of words and sentences. Sensory-motor systems are thought to support partially abstract simulations through which conceptual content is grounded. However, which details of sensory-motor experience are included in, or excluded from these simulations, is not well understood. We investigated whether sensory-motor brain regions are differentially involved depending on the speed of actions described in a sentence. We addressed this issue by examining the neural signature of relatively fast (The old lady scurried across the road) and slow (The old lady strolled across the road) action sentences. The results showed that sentences that implied fast motion modulated activity within the right posterior superior temporal sulcus and the angular and middle occipital gyri, areas associated with biological motion and action perception. Sentences that implied slow motion resulted in greater signal within the right primary motor cortex and anterior inferior parietal lobule, areas associated with action execution and planning. These results suggest that the speed of described motion influences representational content and modulates the nature of conceptual grounding. Fast motion events are represented more visually whereas motor regions play a greater role in representing conceptual content associated with slow motion.

Age-Related Differences in Reorganization of Functional Connectivity for a Dual Task with Increasing Postural Destabilization

The aged brain may not make good use of central resources, so dual task performance may be degraded. From the brain connectome perspective, this study investigated dual task deficits of older adults that lead to task failure of a suprapostural motor task with increasing postural destabilization. Twelve younger (mean age: 25.3 years) and 12 older (mean age: 65.8 years) adults executed a designated force-matching task from a level-surface or a stabilometer board. Force-matching error, stance sway, and event-related potential (ERP) in the preparatory period were measured. The force-matching accuracy and the size of postural sway of the older adults tended to be more vulnerable to stance configuration than that of the young adults, although both groups consistently showed greater attentional investment on the postural task as sway regularity increased in the stabilometer condition. In terms of the synchronization likelihood (SL) of the ERP, both younger and older adults had net increases in the strengths of the functional connectivity in the whole brain and in the fronto-sensorimotor network in the stabilometer condition. Also, the SL in the fronto-sensorimotor network of the older adults was greater than that of the young adults for both stance conditions. However, unlike the young adults, the older adults did not exhibit concurrent deactivation of the functional connectivity of the left temporal-parietal-occipital network for postural-suprapostural task with increasing postural load. In addition, the older adults potentiated functional connectivity of the right prefrontal area to cope with concurrent force-matching with increasing postural load. In conclusion, despite a universal negative effect on brain volume conduction, our preliminary results showed that the older adults were still capable of increasing allocation of neural sources, particularly via compensatory recruitment of the right prefrontal loop, for concurrent force-matching under the challenging postural condition. Nevertheless, dual-task performance of the older adults tended to be more vulnerable to postural load than that of the younger adults, in relation to inferior neural economy or a slow adaptation process to stance destabilization for scant dissociation of control hubs in the temporal-parietal-occipital cortex.

A unified model of human semantic knowledge and its disorders

How is knowledge about the meanings of words and objects represented in the human brain? Current theories embrace two radically different proposals: either distinct cortical systems have evolved to represent different kinds of things, or knowledge for all kinds is encoded within a single domain-general network. Neither view explains the full scope of relevant evidence from neuroimaging and neuropsychology. Here we propose that graded category-specificity emerges in some components of the semantic network through joint effects of learning and network connectivity. We test the proposal by measuring connectivity amongst cortical regions implicated in semantic representation, then simulating healthy and disordered semantic processing in a deep neural network whose architecture mirrors this structure. The resulting neuro-computational model explains the full complement of neuroimaging and patient evidence adduced in support of both domain-specific and domain-general approaches, reconciling long-standing disputes about the nature and origins of this uniquely human cognitive faculty.

Representational Similarity Mapping of Distributional Semantics in Left Inferior Frontal, Middle Temporal, and Motor Cortex

Language comprehension engages a distributed network of frontotemporal, parietal, and sensorimotor regions, but it is still unclear how meaning of words and their semantic relationships are represented and processed within these regions and to which degrees lexico-semantic representations differ between regions and semantic types. We used fMRI and representational similarity analysis to relate word-elicited multivoxel patterns to semantic similarity between action and object words. In left inferior frontal (BA 44-45-47), left posterior middle temporal and left precentral cortex, the similarity of brain response patterns reflected semantic similarity among action-related verbs, as well as across lexical classes-between action verbs and tool-related nouns and, to a degree, between action verbs and food nouns, but not between action verbs and animal nouns. Instead, posterior inferior temporal cortex exhibited a reverse response pattern, which reflected the semantic similarity among object-related nouns, but not action-related words. These results show that semantic similarity is encoded by a range of cortical areas, including multimodal association (e.g., anterior inferior frontal, posterior middle temporal) and modality-preferential (premotor) cortex and that the representational geometries in these regions are partly dependent on semantic type, with semantic similarity among action-related words crossing lexical-semantic category boundaries.

Shared and Distinct Neuroanatomic Regions Critical for Tool-related Action Production and Recognition: Evidence from 131 Left-hemisphere Stroke Patients

The inferior frontal gyrus and inferior parietal lobe have been characterized as human homologues of the monkey "mirror neuron" system, critical for both action production (AP) and action recognition (AR). However, data from brain lesion patients with selective impairment on only one of these tasks provide evidence of neural and cognitive dissociations. We sought to clarify the relationship between AP and AR, and their critical neural substrates, by directly comparing performance of 131 chronic left-hemisphere stroke patients on both tasks--to our knowledge, the largest lesion-based experimental investigation of action cognition to date. Using voxel-based lesion-symptom mapping, we found that lesions to primary motor and somatosensory cortices and inferior parietal lobule were associated with disproportionately impaired performance on AP, whereas lesions to lateral temporo-occipital cortex were associated with a relatively rare pattern of disproportionately impaired performance on AR. In contrast, damage to posterior middle temporal gyrus was associated with impairment on both AP and AR. The distinction between lateral temporo-occipital cortex, critical for recognition, and posterior middle temporal gyrus, important for both tasks, suggests a rough gradient from modality-specific to abstract representations in posterior temporal cortex, the first lesion-based evidence for this phenomenon. Overall, the results of this large patient study help to bring closure to a long-standing debate by showing that tool-related AP and AR critically depend on both common and distinct left hemisphere neural substrates, most of which are external to putative human mirror regions.

An object-identity probability cueing paradigm during grasping observation: the facilitating effect is present only when the observed kinematics is suitable for the cued object

Electrophysiological and psychophysical data indicate that grasping observation automatically orients attention toward the incoming interactions between the actor’s hand and the object. The aim of the present study was to clarify if this effect facilitates the detection of a graspable object with the observed action as compared to an ungraspable one. We submitted participants to an object-identity probability cueing experiment in which the two possible targets were of the same dimensions but one of them presented sharp tips at one extreme while the other presented flat faces. At the beginning of each trial the most probable target was briefly shown. After a variable interval, at the same position, the same (75%) or a different target (25%) was presented. Participants had to press a key in response to target appearance. Superimposed to the video showing cue and target, an agent performing the reaching and grasping of the target was presented. The kinematics of the action was or was not suitable for grasping the cued target, according to the absence or presence of the sharp tips. Results showed that response was modulated by the probability of target identity but only when the observed kinematics was suitable to grasp the attended target. A further experiment clarified that response modulation was never present when the superimposed video always showed the agent at a rest position. These findings are discussed at the light of neurophysiological and psychophysical literature, considering the relationship between the motor system and the perception of objects and of others’ actions. We conclude that the prediction of the mechanical events that arise from the interactions between the hand and the attended object is at the basis of the capability to select a graspable object in space.

The linguistic context effects on the processing of body-object interaction words: An ERP study on second language learners

Embodied theories of cognition argue that the processing of both concrete and abstract concepts requires the activation of sensorimotor systems. The present study examined the time course for embedding a sensorimotor context in order to elicit sensitivity to the sensorimotor consequences of understanding body-object interaction (BOI) words. In the study, Event-Related Potentials (ERPs) were recorded while subjects performed a sentence acceptability task. Target BOI words were preceded by rich or poor sensorimotor sentential contexts. The behavioural results replicated previous findings in that high BOI words received a response faster than low BOI words. In addition to this, however, there was a context effect in the sensorimotor region as well as a BOI effect in the parietal region (involved in object representation). The results indicate that the sentential sensorimotor context contributes to the subsequent BOI processing and that action-and perception-related language leads to the activation of the same brain areas, which is consistent with the embodiment theory.

The influence of handedness on hemispheric representation of tools: a survey

An important debate exists in contemporary cognitive neuroscience about the innate or experience-dependent origin of the brain representation of conceptual categories. The 'domains of knowledge' hypothesis maintains that innate factors subsume the categorical organization at the brain level of animals, plant life and artefacts. On the other hand, the 'sensory-motor model of conceptual knowledge' and the embodied cognition theory attribute this categorical organization to experience-dependent factors. I tried to clarify this issue by surveying the influence that handedness could have on the lateralization of tools representation in the inferior fronto-parietal and posterior middle temporal cortices of the left hemisphere. The underlying assumption was that, if this lateralization results from innate mechanisms, then handedness should not influence this hemispheric asymmetry. If, on the other hand, this lateralization is due to the motor and somatosensory experiences made with the right dominant hand during the manipulation of tools and other artefacts, then this asymmetry should be inverted or strongly attenuated in left-handers. Results of the review strongly suggest that manual experience acquired during tool manipulation can influence the hemispheric representation of tools and other artefacts. They also suggest, however, that handedness-related embodiment is not fixed, but influenced by personal motor experiences (such as those made by left-handers who have been forced to use their right hand) and by social visual experiences (such as the fact that, living in a right-handed world, left-handers see more people in their environment who use the right rather than the left hand) during tool manipulation.

Frontoparietal networks involved in categorization and item working memory

Categorization and memory for specific items are fundamental processes that allow us to apply knowledge to novel stimuli. This study directly compares categorization and memory using delay match to category (DMC) and delay match to sample (DMS) tasks. In DMC participants view and categorize a stimulus, maintain the category across a delay, and at the probe phase view another stimulus and indicate whether it is in the same category or not. In DMS, a standard item working memory task, participants encode and maintain a specific individual item, and at probe decide if the stimulus is an exact match or not. Constrained Principal Components Analysis was used to identify and compare activity within neural networks associated with these tasks, and we relate these networks to those that have been identified with resting state-fMRI. We found that two frontoparietal networks of particular interest. The first network included regions associated with the dorsal attention network and frontoparietal salience network; this network showed patterns of activity consistent with a role in rapid orienting to and processing of complex stimuli. The second uniquely involved regions of the frontoparietal central-executive network; this network responded more slowly following each stimulus and showed a pattern of activity consistent with a general role in role in decision-making across tasks. Additional components were identified that were associated with visual, somatomotor and default mode networks.

Occipital cortical thickness in very low birth weight born adolescents predicts altered neural specialization of visual semantic category related neural networks

Very low birth weight (VLBW) premature born infants have a high risk to develop visual perceptual and learning deficits as well as widespread functional and structural brain abnormalities during infancy and childhood. Whether and how prematurity alters neural specialization within visual neural networks is still unknown. We used functional and structural brain imaging to examine the visual semantic system of VLBW born (<1250 g, gestational age 25-32 weeks) adolescents (13-15 years, n = 11, 3 males) and matched term born control participants (13-15 years, n = 11, 3 males). Neurocognitive assessment revealed no group differences except for lower scores on an adaptive visuomotor integration test. All adolescents were scanned while viewing pictures of animals and tools and scrambled versions of these pictures. Both groups demonstrated animal and tool category related neural networks. Term born adolescents showed tool category related neural activity, i.e. tool pictures elicited more activity than animal pictures, in temporal and parietal brain areas. Animal category related activity was found in the occipital, temporal and frontal cortex. VLBW born adolescents showed reduced tool category related activity in the dorsal visual stream compared with controls, specifically the left anterior intraparietal sulcus, and enhanced animal category related activity in the left middle occipital gyrus and right lingual gyrus. Lower birth weight of VLBW adolescents correlated with larger thickness of the pericalcarine gyrus in the occipital cortex and smaller surface area of the superior temporal gyrus in the lateral temporal cortex. Moreover, larger thickness of the pericalcarine gyrus and smaller surface area of the superior temporal gyrus correlated with reduced tool category related activity in the parietal cortex. Together, our data suggest that very low birth weight predicts alterations of higher order visual semantic networks, particularly in the dorsal stream. The differences in neural specialization may be associated with aberrant cortical development of areas in the visual system that develop early in childhood.

Effects of navigated TMS on object and action naming

Transcranial magnetic stimulation (TMS) has been used to induce speech disturbances and to affect speech performance during different naming tasks. Lately, repetitive navigated TMS (nTMS) has been used for non-invasive mapping of cortical speech-related areas. Different naming tasks may give different information that can be useful for presurgical evaluation. We studied the sensitivity of object and action naming tasks to nTMS and compared the distributions of cortical sites where nTMS produced naming errors. Eight healthy subjects named pictures of objects and actions during repetitive nTMS delivered to semi-random left-hemispheric sites. Subject-validated image stacks were obtained in the baseline naming of all pictures before nTMS. Thereafter, nTMS pulse trains were delivered while the subjects were naming the images of objects or actions. The sessions were video-recorded for offline analysis. Naming during nTMS was compared with the baseline performance. The nTMS-induced naming errors were categorized by error type and location. nTMS produced no-response errors, phonological paraphasias, and semantic paraphasias. In seven out of eight subjects, nTMS produced more errors during object than action naming. Both intrasubject and intersubject analysis showed that object naming was significantly more sensitive to nTMS. When the number of errors was compared according to a given area, nTMS to postcentral gyrus induced more errors during object than action naming. Object naming is apparently more easily disrupted by TMS than action naming. Different stimulus types can be useful for locating different aspects of speech functions. This provides new possibilities in both basic and clinical research of cortical speech representations.

Action Sentences Activate Sensory Motor Regions in the Brain Independently of Their Status of Reality

Some studies have reported that understanding concrete action-related words and sentences elicits activations of motor areas in the brain. The present fMRI study goes one step further by testing whether this is also the case for comprehension of nonfactual statements. Three linguistic structures were used (factuals, counterfactuals, and negations), referring either to actions or, as a control condition, to visual events. The results showed that action sentences elicited stronger activations than visual sentences in the SMA, extending to the primary motor area, as well as in regions generally associated with the planning and understanding of actions (left superior temporal gyrus, left and right supramarginal gyri). Also, we found stronger activations for action sentences than for visual sentences in the extrastriate body area, a region involved in the visual processing of human body movements. These action-related effects occurred not only in factuals but also in negations and counterfactuals, suggesting that brain regions involved in action understanding and planning are activated by default even when the actions are described as hypothetical or as not happening. Moreover, some of these regions overlapped with those activated during the observation of action videos, indicating that the act of understanding action language and that of observing real actions share neural networks. These results support the claim that embodied representations of linguistic meaning are important even in abstract linguistic contexts.

Hand specific representations in language comprehension

Theories of embodied cognition argue that language comprehension involves sensory-motor re-enactments of the actions described. However, the degree of specificity of these re-enactments as well as the relationship between action and language remains a matter of debate. Here we investigate these issues by examining how hand-specific information (left or right hand) is recruited in language comprehension and action execution. An fMRI study tested self-reported right-handed participants in two separate tasks that were designed to be as similar as possible to increase sensitivity of the comparison across task: an action execution go/no-go task where participants performed right or left hand actions, and a language task where participants read sentences describing the same left or right handed actions as in the execution task. We found that language-induced activity did not match the hand-specific patterns of activity found for action execution in primary somatosensory and motor cortex, but it overlapped with pre-motor and parietal regions associated with action planning. Within these pre-motor regions, both right hand actions and sentences elicited stronger activity than left hand actions and sentences—a dominant hand effect. Importantly, both dorsal and ventral sections of the left pre-central gyrus were recruited by both tasks, suggesting different action features being recruited. These results suggest that (a) language comprehension elicits motor representations that are hand-specific and akin to multimodal action plans, rather than full action re-enactments; and (b) language comprehension and action execution share schematic hand-specific representations that are richer for the dominant hand, and thus linked to previous motor experience.

The left inferior parietal lobe represents stored hand-postures for object use and action prediction

Action semantics enables us to plan actions with objects and to predict others' object-directed actions as well. Previous studies have suggested that action semantics are represented in a fronto-parietal action network that has also been implicated to play a role in action observation. In the present fMRI study it was investigated how activity within this network changes as a function of the predictability of an action involving multiple objects and requiring the use of action semantics. Participants performed an action prediction task in which they were required to anticipate the use of a centrally presented object that could be moved to an associated target object (e.g., hammer—nail). The availability of actor information (i.e., presenting a hand grasping the central object) and the number of possible target objects (i.e., 0, 1, or 2 target objects) were independently manipulated, resulting in different levels of predictability. It was found that making an action prediction based on actor information resulted in an increased activation in the extrastriate body area (EBA) and the fronto-parietal action observation network (AON). Predicting actions involving a target object resulted in increased activation in the bilateral IPL and frontal motor areas. Within the AON, activity in the left inferior parietal lobe (IPL) and the left premotor cortex (PMC) increased as a function of the level of action predictability. Together these findings suggest that the left IPL represents stored hand-postures that can be used for planning object-directed actions and for predicting other's actions as well.

Predication drives verb cortical signatures

Verbs and nouns are fundamental units of language, but their neural instantiation remains poorly understood. Neuropsychological research has shown that nouns and verbs can be damaged independently of each other, and neuroimaging research has found that several brain regions respond differentially to the two word classes. However, the semantic-lexical properties of verbs and nouns that drive these effects remain unknown. Here we show that the most likely candidate is predication: a core lexical feature involved in binding constituent arguments (boy, candies) into a unified syntactic-semantic structure expressing a proposition (the boy likes the candies). We used functional neuroimaging to test whether the intrinsic "predication-building" function of verbs is what drives the verb-noun distinction in the brain. We first identified verb-preferring regions with a localizer experiment including verbs and nouns. Then, we examined whether these regions are sensitive to transitivity--an index measuring its tendency to select for a direct object. Transitivity is a verb-specific property lying at the core of its predication function. Neural activity in the left posterior middle temporal and inferior frontal gyri correlates with transitivity, indicating sensitivity to predication. This represents the first evidence that grammatical class preference in the brain is driven by a word's function to build predication structures.

A piece of the action: modulation of sensory-motor regions by action idioms and metaphors

The idea that the conceptual system draws on sensory and motor systems has received considerable experimental support in recent years. Whether the tight coupling between sensory-motor and conceptual systems is modulated by factors such as context or task demands is a matter of controversy. Here, we tested the context sensitivity of this coupling by using action verbs in three different types of sentences in an fMRI study: literal action, apt but non-idiomatic action metaphors, and action idioms. Abstract sentences served as a baseline. The result showed involvement of sensory-motor areas for literal and metaphoric action sentences, but not for idiomatic ones. A trend of increasing sensory-motor activation from abstract to idiomatic to metaphoric to literal sentences was seen. These results support a gradual abstraction process whereby the reliance on sensory-motor systems is reduced as the abstractness of meaning as well as conventionalization is increased, highlighting the context sensitive nature of semantic processing.

Parkinson's disease disrupts both automatic and controlled processing of action verbs

The problem of how word meaning is processed in the brain has been a topic of intense investigation in cognitive neuroscience. While considerable correlational evidence exists for the involvement of sensory-motor systems in conceptual processing, it is still unclear whether they play a causal role. We investigated this issue by comparing the performance of patients with Parkinson's disease (PD) with that of age-matched controls when processing action and abstract verbs. To examine the effects of task demands, we used tasks in which semantic demands were either implicit (lexical decision and priming) or explicit (semantic similarity judgment). In both tasks, PD patients' performance was selectively impaired for action verbs (relative to controls), indicating that the motor system plays a more central role in the processing of action verbs than in the processing of abstract verbs. These results argue for a causal role of sensory-motor systems in semantic processing.

Social cognition and the cerebellum: a meta-analysis of over 350 fMRI studies

This meta-analysis explores the role of the cerebellum in social cognition. Recent meta-analyses of neuroimaging studies since 2008 demonstrate that the cerebellum is only marginally involved in social cognition and emotionality, with a few meta-analyses pointing to an involvement of at most 54% of the individual studies. In this study, novel meta-analyses of over 350 fMRI studies, dividing up the domain of social cognition in homogeneous subdomains, confirmed this low involvement of the cerebellum in conditions that trigger the mirror network (e.g., when familiar movements of body parts are observed) and the mentalizing network (when no moving body parts or unfamiliar movements are present). There is, however, one set of mentalizing conditions that strongly involve the cerebellum in 50-100% of the individual studies. In particular, when the level of abstraction is high, such as when behaviors are described in terms of traits or permanent characteristics, in terms of groups rather than individuals, in terms of the past (episodic autobiographic memory) or the future rather than the present, or in terms of hypothetical events that may happen. An activation likelihood estimation (ALE) meta-analysis conducted in this study reveals that the cerebellum is critically implicated in social cognition and that the areas of the cerebellum which are consistently involved in social cognitive processes show extensive overlap with the areas involved in sensorimotor (during mirror and self-judgments tasks) as well as in executive functioning (across all tasks). We discuss the role of the cerebellum in social cognition in general and in higher abstraction mentalizing in particular. We also point out a number of methodological limitations of some available studies on the social brain that hamper the detection of cerebellar activity.

People can understand descriptions of motion without activating visual motion brain regions

What is the relationship between our perceptual and linguistic neural representations of the same event? We approached this question by asking whether visual perception of motion and understanding linguistic depictions of motion rely on the same neural architecture. The same group of participants took part in two language tasks and one visual task. In task 1, participants made semantic similarity judgments with high motion (e.g., “to bounce”) and low motion (e.g., “to look”) words. In task 2, participants made plausibility judgments for passages describing movement (“A centaur hurled a spear … ”) or cognitive events (“A gentleman loved cheese …”). Task 3 was a visual motion localizer in which participants viewed animations of point-light walkers, randomly moving dots, and stationary dots changing in luminance. Based on the visual motion localizer we identified classic visual motion areas of the temporal (MT/MST and STS) and parietal cortex (inferior and superior parietal lobules). We find that these visual cortical areas are largely distinct from neural responses to linguistic depictions of motion. Motion words did not activate any part of the visual motion system. Motion passages produced a small response in the right superior parietal lobule, but none of the temporal motion regions. These results suggest that (1) as compared to words, rich language stimuli such as passages are more likely to evoke mental imagery and more likely to affect perceptual circuits and (2) effects of language on the visual system are more likely in secondary perceptual areas as compared to early sensory areas. We conclude that language and visual perception constitute distinct but interacting systems.

Supramodal and modality-sensitive representations of perceived action categories in the human brain

Seeing Suzie bite an apple or reading the sentence 'Suzie munched the apple' both convey a similar idea. But is there a common neural basis for action comprehension when generated through video or text? The current study used functional magnetic resonance imaging to address this question. Participants observed videos or read sentences that described two categories of actions: eating and cleaning. A conjunction analysis of video and sentence stimuli revealed that cleaning actions (compared to eating actions) showed a greater response in dorsal frontoparietal regions, as well as within the medial fusiform gyrus. These findings reveal supramodal representations of perceived actions in the human brain, which are specific to action categories and independent of input modality (video or written words). In addition, some brain regions associated with cleaning and eating actions showed an interaction with modality, which was manifested as a greater sensitivity for video compared with sentence stimuli. Together, this pattern of results demonstrates both supramodal and modality-sensitive representations of action categories in the human brain, a finding with implications for how we understand other people's actions from video and written sources.

Action Concepts in the Brain: An Activation Likelihood Estimation Meta-analysis

Many recent neuroimaging studies have investigated the representation of semantic memory for actions in the brain. We used activation likelihood estimation (ALE) meta-analyses to answer two outstanding questions about the neural basis of action concepts. First, on an “embodied” view of semantic memory, evidence to date is unclear regarding whether visual motion or motor systems are more consistently engaged by action concepts. Second, few studies have directly investigated the possibility that action concepts accessed verbally or nonverbally recruit different areas of the brain. Because our meta-analyses did not include studies requiring the perception of dynamic depictions of actions or action execution, we were able to determine whether conceptual processing alone recruits visual motion and motor systems. Significant concordance in brain regions within or adjacent to visual motion areas emerged in all meta-analyses. By contrast, we did not observe significant concordance in motor or premotor cortices in any analysis. Neural differences between action images and action verbs followed a gradient of abstraction among representations derived from visual motion information in the left lateral temporal and occipital cortex. The consistent involvement of visual motion but not motor brain regions in representing action concepts may reflect differences in the variability of experience across individuals with perceiving versus performing actions.

Contextual processing of abstract concepts reveals neural representations of nonlinguistic semantic content

Concepts develop for many aspects of experience, including abstract internal states and abstract social activities that do not refer to concrete entities in the world. The current study assessed the hypothesis that, like concrete concepts, distributed neural patterns of relevant nonlinguistic semantic content represent the meanings of abstract concepts. In a novel neuroimaging paradigm, participants processed two abstract concepts (convince, arithmetic) and two concrete concepts (rolling, red) deeply and repeatedly during a concept-scene matching task that grounded each concept in typical contexts. Using a catch trial design, neural activity associated with each concept word was separated from neural activity associated with subsequent visual scenes to assess activations underlying the detailed semantics of each concept. We predicted that brain regions underlying mentalizing and social cognition (e.g., medial prefrontal cortex, superior temporal sulcus) would become active to represent semantic content central to convince, whereas brain regions underlying numerical cognition (e.g., bilateral intraparietal sulcus) would become active to represent semantic content central to arithmetic. The results supported these predictions, suggesting that the meanings of abstract concepts arise from distributed neural systems that represent concept-specific content.

Motion and actions in language: semantic representations in occipito-temporal cortex

Understanding verbs typically activates posterior temporal regions and, in some circumstances, motion perception area V5. However, the nature and role of this activation remains unclear: does language alone indeed activate V5? And are posterior temporal representations modality-specific motion representations, or supra-modal motion-independent event representations? Here, we address these issues by investigating human and object motion sentences compared to corresponding state descriptions. We adopted the blank screen paradigm, which is known to encourage visual imagery, and used a localizer to identify V5 and temporal structures responding to motion. Analyses in each individual brain suggested that language modulated activity in the posterior temporal lobe but not within V5 in most participants. Moreover, posterior temporal structures strongly responded to both motion sentences and human static sentences. These results suggest that descriptive language alone need not recruit V5 and instead engages more schematic event representations in temporal cortex encoding animacy and motion.

Behavioral patterns and lesion sites associated with impaired processing of lexical and conceptual knowledge of actions

To further investigate the neural substrates of lexical and conceptual knowledge of actions, we administered a battery of six tasks to 226 brain-damaged patients with widely distributed lesions in the left and right cerebral hemispheres. The tasks probed lexical and conceptual knowledge of actions in a variety of verbal and non-verbal ways, including naming, word-picture matching, attribute judgments involving both words and pictures, and associative comparisons involving both words and pictures. Of the 226 patients who were studied, 61 failed one or more of the six tasks, with four patients being impaired on the entire battery, and varied numbers of patients being impaired on varied combinations of tasks. Overall, the 61 patients manifested a complex array of associations and dissociations across the six tasks. The lesion sites of 147 of the 226 patients were also investigated, using formal methods for lesion-deficit statistical mapping and power analysis of lesion overlap maps. Significant effects for all six tasks were found in the following left-hemisphere regions: the inferior frontal gyrus; the ventral precentral gyrus, extending superiorly into what are likely to be hand-related primary motor and premotor areas; and the anterior insula. In addition, significant effects for 4-5 tasks were found in not only the regions just mentioned, but also in several other left-hemisphere areas: the ventral postcentral gyrus; the supramarginal gyrus; and the posterior middle temporal gyrus. These results converge with previous research on the neural underpinnings of action words and concepts. However, the current study goes considerably beyond most previous investigations by providing extensive behavioral and lesion data for an unusually large and diverse sample of brain-damaged patients, and by incorporating multiple measures of verb comprehension. Regarding theoretical implications, the study provides new support for the Embodied Cognition Framework, which maintains that conceptual knowledge is grounded in sensorimotor systems.

How number processing survives left occipito-temporal damage

We investigated the neural systems that support number processing in a patient (JL) who had damage to the left ventral occipito-temporal cortex (LvOT). JL had severely impaired written word recognition but he was remarkably accurate in number tasks, albeit slower than normal. This suggests LvOT activation is necessary for efficient but not for accurate number decisions. Here we investigated how JL made accurate number decisions using fMRI; we compared JL's brain activation to that in healthy controls and in two patients with frontal lobe damage who, like JL, made slow but accurate responses in number tasks. For semantic relative to perceptual decisions on numbers, JL did not activate the left occipito-temporal area that was involved in all other subjects. However, JL had significantly increased activation in a left posterior middle temporal region. In addition, during semantic and perceptual decisions on numbers, JL showed increased activation in: (1) the right occipito-temporal cortex, (2) right caudate, and (3) bilateral frontal regions. These effects were unique to JL and cannot be explained in terms of abnormally long response times because they were not observed in the other patients who made slow but accurate number decisions. Together these results show that although the LvOT usually contributes to efficient number processing, activation in this region is not essential for accurate performance because (i) perceptual processing of numbers can be supported by right occipital, right caudate, and bilateral frontal activation and (ii) semantic processing of numbers can be supported by increased left posterior middle temporal activation associated with hand actions.

BOLD response to motion verbs in left posterior middle temporal gyrus during story comprehension

A primary focus within neuroimaging research on language comprehension is on the distribution of semantic knowledge in the brain. Studies have shown that the left posterior middle temporal gyrus (LPMT), a region just anterior to area MT/V5, is important for the processing of complex action knowledge. It has also been found that motion verbs cause activation in LPMT. In this experiment we investigated whether this effect could be replicated in a setting resembling real life language comprehension, i.e. without any overt behavioral task during passive listening to a story. During fMRI participants listened to a recording of the story "The Ugly Duckling". We incorporated a nuisance elimination regression approach for factoring out known nuisance variables both in terms of physiological noise, sound intensity, linguistic variables and emotional content. Compared to the remaining text, clauses containing motion verbs were accompanied by a robust activation of LPMT with no other significant effects, consistent with the hypothesis that this brain region is important for processing motion knowledge, even during naturalistic language comprehension conditions.

The functional neuroanatomy of actions

Our current understanding of the neural basis of semantic memory is informed primarily by studies of concrete objects. However, conceptual knowledge encompasses many other, albeit less concrete, domains. This article reviews evidence from neuroimaging and patient studies that speaks to the neural basis of action concepts and the words that refer to them. These data highlight 2 important principles governing the neural instantiation of semantic knowledge. First, the organization of conceptual representations in the brain parallels perception and action. Action concepts are at least partially represented within modality-specific areas responsible for the perception and execution of dynamic actions. Second, unimodal sensory and motor cortices act as "points of entry" for more abstract action knowledge. Increasingly abstract conceptual knowledge derived from these modalities is represented in brain areas located anterior and centripetal to modality-specific regions. Extending research on the neural basis of semantics to include dynamic and relational aspects of the world gives us a more complete appreciation of the range of cognitive and communication impairments that may be experienced by patients with neurologic disease.

Typical neural representations of action verbs develop without vision

Many empiricist theories hold that concepts are composed of sensory-motor primitives. For example, the meaning of the word "run" is in part a visual image of running. If action concepts are partly visual, then the concepts of congenitally blind individuals should be altered in that they lack these visual features. We compared semantic judgments and neural activity during action verb comprehension in congenitally blind and sighted individuals. Participants made similarity judgments about pairs of nouns and verbs that varied in the visual motion they conveyed. Blind adults showed the same pattern of similarity judgments as sighted adults. We identified the left middle temporal gyrus (lMTG) brain region that putatively stores visual-motion features relevant to action verbs. The functional profile and location of this region was identical in sighted and congenitally blind individuals. Furthermore, the lMTG was more active for all verbs than nouns, irrespective of visual-motion features. We conclude that the lMTG contains abstract representations of verb meanings rather than visual-motion images. Our data suggest that conceptual brain regions are not altered by the sensory modality of learning.