Stimulus properties matter more than perspective: an fMRI study of mental imagery and silent reading of action phrases

Differences in perceptual representations in multilinguals' first, second, and third language

Two experiments were conducted to investigate the differences in perceptual representations among multilingual individuals. In Experiment 1, the immediate sentence-picture verification paradigm was used to investigate perceptual representations in the working memory stage. The results suggest a match effect within the first language (Cantonese), but not within the second language (Mandarin) or the third language (English), showing perceptual representations only in first language comprehension. In Experiment 2, the delayed sentence-picture verification paradigm was used to investigate perceptual representations in long-term memory. Similarly, the results suggest a match effect within the first language (Mandarin), but not within the second language (English). The findings of both experiments suggest that the first language was perceptually represented, regardless of whether it was Cantonese or Mandarin, regardless of the processing in working memory or long-term memory. No evidence was found for perceptual representations in the later-learned languages, regardless of high or low proficiency. Our study has implications for theories of language comprehension and embodied cognition.

Modal and amodal cognition: an overarching principle in various domains of psychology

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Distinct brain dynamics and networks for processing short and long auditory time intervals

Psychophysical studies suggest that time intervals above and below 1.2 s are processed differently in the human brain. However, the neural underpinnings of this dissociation remain unclear. Here, we investigate whether distinct or common brain networks and dynamics support the passive perception of short (below 1.2 s) and long (above 1.2 s) empty time intervals. Twenty participants underwent an EEG recording during an auditory oddball paradigm with .8- and 1.6-s standard time intervals and deviant intervals either shorter (early) or longer (delayed) than the standard interval. We computed the auditory ERPs for each condition at the sensor and source levels. We then performed whole brain cluster-based permutation statistics for the CNV, N1 and P2, components, testing deviants against standards. A CNV was found only for above 1.2 s intervals (delayed deviants), with generators in temporo-parietal, SMA, and motor regions. Deviance detection of above 1.2 s intervals occurred during the N1 period over fronto-central sensors for delayed deviants only, with generators in parietal and motor regions. Deviance detection of below 1.2 s intervals occurred during the P2 period over fronto-central sensors for delayed deviants only, with generators in primary auditory cortex, SMA, IFG, cingulate and parietal cortex. We then identified deviance related changes in directed connectivity using bivariate Granger causality to highlight the networks dynamics associated with interval processing above and below 1.2. These results suggest that distinct brain dynamics and networks support the perception of time intervals above and below 1.2 s.

The interaction between motor simulation and spatial perspective-taking in action language: a cross-cultural study

Growing evidence has revealed the crucial role of motor simulation and spatial perspective-taking in action language. However, there is still a lack of understanding of how motor and spatial processes interact when there are multiple actors involved, and if embodied processes are consistent across different cultures. To address this gap, we examined the interaction between motor simulation and spatial perspective-taking in action-sentences comprehension, along with the consistency of embodied processes across cultures. We collected data from Italian and US English speakers using an online sentence-picture verification task. The participants completed four conditions: two congruent (i.e., the participant is the agent in the sentence and the photo; the agent is someone else interacting with the participant in both the sentence and the picture) and two incongruent (i.e., the agents of the sentence and the picture do not match). The results show that when the perspective of the picture matched that described in the sentence-processing reaction times (RTs) were faster than in the incongruent conditions. In the congruent conditions where the agent is someone else, RTs were slower compared to the condition where the participant is the agent. This has been interpreted as claiming that motor simulation and perspective-taking are independent processes interacting during sentence comprehension (e.g., motor simulation is always run in the role of the agent, but we can adopt multiple perspectives depending on the pronouns and the contextual cues). Furthermore, Bayesian analysis provided evidence that embodied processing of action language entwines a common mechanism, suggesting cross-cultural consistency of embodied processes.

Motor imagery training to improve language processing: What are the arguments?

Studies showed that motor expertise was found to induce improvement in language processing. Grounded and situated approaches attributed this effect to an underlying automatic simulation of the motor experience elicited by action words, similar to motor imagery (MI), and suggest shared representations of action conceptualization. Interestingly, recent results also suggest that the mental simulation of action by MI training induces motor-system modifications and improves motor performance. Consequently, we hypothesize that, since MI training can induce motor-system modifications, it could be used to reinforce the functional connections between motor and language system, and could thus lead to improved language performance. Here, we explore these potential interactions by reviewing recent fundamental and clinical literature in the action-language and MI domains. We suggested that exploiting the link between action language and MI could open new avenues for complementary language improvement programs. We summarize the current literature to evaluate the rationale behind this novel training and to explore the mechanisms underlying MI and its impact on language performance.

Neural correlates of embodied action language processing: a systematic review and meta-analytic study

The neural correlates of action language processing are still debated within embodied cognition research and little is known about the flexible involvement of modality-specific pre-motor system and multimodal high-level temporo-parietal regions as a function of explicit and implicit tasks. A systematic review and the Activation likelihood estimation (ALE) meta-analyses on functional neuroimaging studies were performed to identify neural correlates of action language processing activated during explicit and implicit tasks. The contrast ALE meta-analysis revealed activation of modality-specific premotor area and inferior frontal areas during explicit action language tasks while a greater activation of posterior temporo-occipital areas emerged for implicit tasks. The conjunction analysis revealed overlap in the temporo-parietal multimodal high-level regions for both types of tasks. Functional specialization of the middle temporal gyrus was found where the more posterior-occipital part resulted activated during implicit action language tasks whereas the antero-lateral part was involved in explicit tasks. Our findings were discussed within a conceptual flexibility perspective about the involvement of both the modality-specific and multimodal brain system during action language processing depending on different types of tasks.

Multisensory mental representation in covid-19 patients and the possibility of long-lasting gustatory and olfactory dysfunction in the CNS

Gustatory (GD) and olfactory (OD) dysfunctions are the most frequent neurological manifestations of COVID-19. We used mental imagery as an experimental psychological paradigm to access olfactory and gustatory brain representations in 80 Italian COVID-19 adult patients (68.75% reported both OD and GD). COVID-19 patients with OD + GD have a significantly and selectively decreased vividness of odor and taste imagery, indicating that COVID-19 has an effect on their chemosensory mental representations. OD + GD length and type influenced the status of mental chemosensory representations. OD + GD were become all COVID-19 negative at the time of testing. Data suggest that patients are not explicitly aware of long-term altered chemosensory processing. However, differences emerge when their chemosensory function is implicitly assessed using self-ratings. Among patients developing OD + GD, self-ratings of chemosensory function (taste, flavor) were significantly lower as compared to those who did not. At the level of mental representation, such differences can be further detected, in terms of a reduced ability to mentally activate an odor or taste mental image. Our study shows that COVID-19 infection not only frequently causes hyposmia and dysgeusia, but that may also alter the mental representations responsible for olfactory and gustatory perception.

Multisensory mental imagery of fatigue: Evidence from an fMRI study

Functional imaging experimental designs measuring fatigue, defined as a subjective lack of physical and/or mental energy characterizing a wide range of neurologic conditions, are still under development. Nineteen right‐handed healthy subjects (9 M and 10 F, mean age 43.15 ± 8.34 years) were evaluated by means of functional magnetic resonance imaging (fMRI), asking them to perform explicit, first‐person, mental imagery of fatigue‐related multisensory sensations. Short sentences designed to assess the principal manifestations of fatigue from the Multidimensional Fatigue Symptom Inventory were presented. Participants were asked to imagine the corresponding sensations (Sensory Imagery, SI). As a control, they had to imagine the visual scenes (Visual Imagery, VI) described in short phrases. The SI task (vs. VI task) differentially activated three areas: (i) the precuneus, which is involved in first‐person perspective taking; (ii) the left superior temporal sulcus, which is a multisensory integration area; and (iii) the left inferior frontal gyrus, known to be involved in mental imagery network. The SI fMRI task can be used to measure processing involved in mental imagery of fatigue‐related multisensory sensations.

An action-concept processing advantage in a patient with a double motor cortex

Patients with atrophy in motor brain regions exhibit selective deficits in processing action-related meanings, suggesting a link between movement conceptualization and the amount of regional tissue. Here we examine such a relation in a unique opposite model: a rare patient with a double cortex (due to subcortical band heterotopia) in primary/supplementary motor regions, and no double cortex in multimodal semantic regions. We measured behavioral performance in action- and object-concept processing as well and resting-state functional connectivity. Both dimensions involved comparisons with healthy controls. Results revealed preserved accuracy in action and object categories for the patient. However, unlike controls, the patient exhibited faster performance for action than object concepts, a difference that was uninfluenced by general cognitive abilities. Moreover, this pattern was accompanied by heightened functional connectivity between the bilateral primary motor cortices. This suggests that a functionally active double motor cortex may entail action-processing advantages. Our findings offer new constraints for models of action semantics and motor-region function at large.

Is rushing always faster than strolling? A reaction time study on the processing of sentences containing manner of motion verbs

In the context of the embodied cognition debate, an effect of motion verb associated speed information has previously been detected using eye-tracking, functional magnetic resonance imaging (fMRI), and reaction times (RT). The latter, for instance, was implemented by Wender and Weber (1982), who observed that participants were faster in detecting motion in sentences associated with fast motion compared to sentences associated with slow motion after having formed mental images of the sentences' content. It remains open whether the reported effects of speed are associated with automatic lexical-semantic retrieval processes or whether they reflect higher top-down cognitive processes. To answer this question, the paradigm by Wender and Weber (1982) was adopted and further elaborated in the present study. In Experiment 1 visualization instructions were eliminated. Additionally, the stimulus material was manipulated in regards to the agent of the described movement (human vs. object motion) in order to determine the representation's modality (visual vs. motoric). In Experiment 2, the task to detect motion was replaced by the task to judge sensicality. The results suggest that the prompt to perform mental imagery is not a precondition for the engagement of modal representations in this speed of motion paradigm and that the involved representations' modality is visual rather than motoric. However, the modal representations' involvement is dependent on the task. They thus do not seem to be part of the invariant semantic representation of manner of motion verbs.

Neurophysiological mechanisms of perspective-taking: An MEG investigation of agency

According to the embodied cognition framework, sensory and motor areas are recruited during language understanding through simulation processes. Behavioral and imaging findings point to a dependence of the latter on perspective-taking (e.g., first person "I" versus third person "s/he"). The current study aims at identifying possible neurophysiological correlates of perspective in a linguistic context. Twenty healthy participants were measured with magnetoencephalography (MEG) while semantically processing visually presented inflected German verbs in the first- and third-person perspective, simple present tense. Results show that the first-person perspective induces stronger beta (15-25 Hz) desynchronization in the right-hemispheric posterior superior temporal sulcus, ventral posterior cingulate gyrus, and V5/MT+ area; no modulation of sensorimotor cortex emerged. Moreover, a stronger event-related field (ERF) was observed for the first-person perspective at about 150 ms after pronoun-verb onset, originating in occipital and moving to central and left temporal cortical sites. No effect of perspective on sensory gating was found when targeting the N1 component related to tones following the linguistic stimuli. Results indicate an effect of linguistic perspective-taking on brain activation patterns. The contribution of the single brain areas and their role in self-other distinction is further discussed.

Functional MRI of Native and Non-native Speech Sound Production in Sequential German-English Bilinguals

Bilingualism and multilingualism are highly prevalent. Non-invasive brain imaging has been used to study the neural correlates of native and non-native speech and language production, mainly on the lexical and syntactic level. Here, we acquired continuous fast event-related FMRI during visually cued overt production of exclusively German and English vowels and syllables. We analyzed data from 13 university students, native speakers of German and sequential English bilinguals. The production of non-native English sounds was associated with increased activity of the left primary sensorimotor cortex, bilateral cerebellar hemispheres (lobule VI), left inferior frontal gyrus, and left anterior insula compared to native German sounds. The contrast German > English sounds was not statistically significant. Our results emphasize that the production of non-native speech requires additional neural resources already on a basic phonological level in sequential bilinguals.

Investigation of Embodied Language Processing on Command-Swallow Performance in Healthy Participants

Purpose During videofluoroscopic examination of swallowing, patients commonly are instructed to hold a bolus in their mouth until they hear a verbal instruction to swallow, which usually consists of the word swallow and is commonly referred to as the command swallow condition. The language-induced motor facilitation theory suggests that linguistic processes associated with the verbal command to swallow should facilitate the voluntary component of swallowing. As such, the purpose of the study was to examine the linguistic influences of the verbal command on swallowing. Method Twenty healthy young adult participants held a 5-ml liquid bolus in their mouth and swallowed the bolus after hearing one of five acoustic stimuli presented randomly: congruent action word (swallow), incongruent action word (cough), congruent pseudoword (spallow), incongruent pseudoword (pough), and nonverbal stimulus (1000-Hz pure tone). Suprahyoid muscle activity during swallowing was measured via surface electromyography (sEMG). Results The onset and peak sEMG latencies following the congruent action word swallow were shorter than latencies following the pure tone and pseudowords but were not different from the incongruent action word. The lack of difference between swallow and cough did not negate the positive impact of real words on timing. In contrast to expectations, sEMG activity duration and rise time were longer following the word swallow than the pure tone and pseudowords but were not different from cough. No differences were observed for peak suprahyoid muscle activity amplitude and fall times. Conclusions Language facilitation was observed in swallowing. The clinical utility of the information obtained in the study may depend on the purposes for using the command swallow and the type of patient being assessed. However, linguistic processing under the command swallow condition may alter swallow behaviors and suggests that linguistic inducement could be useful as a compensatory technique for patients with difficulty initiating oropharyngeal swallows.

Hand-use norms for Dutch and English manual action verbs: Implicit measures from a pantomime task

Many studies use manual action verbs to test whether people use neural systems for controlling manual actions to understand language about those actions. Yet, few of these studies empirically establish how people use their hands to perform the actions described by those verbs, relying instead on explicit self-report measures. Here, participants pantomimed the manual actions described by a large set of Dutch (N = 251) and English (N = 250) verbs, allowing us to approximate the extent to which people use each of their hands to perform these actions. After the pantomime task, participants also provided explicit ratings of each of these actions. The results from the pantomime task showed that most manual actions cannot be described accurately as either "unimanual" or "bimanual." With a few exceptions, unimanual action verbs do not describe actions that are performed with only one hand, and bimanual verbs do not describe actions that are performed by using both hands equally. Instead, individual actions vary continuously in the extent to which people use their non-dominant hand to perform them, and in the extent to which people consistently prefer one hand or the other to perform them. Finally, by comparing participants' implicit behavior to their explicit ratings, we found that participants' self-report showed only limited correspondence with their observed motor behavior. We provide all of our measures in both raw and summary format, offering researchers a precision tool for constructing stimulus sets for experiments on embodied cognition.

Neurosurgical lesions to sensorimotor cortex do not impair action verb processing

There is ongoing debate regarding the role that sensorimotor regions play in conceptual processing, with embodied theories supporting their direct involvement in processing verbs describing body part movements. Patient lesion studies examining a causal role for sensorimotor activation in conceptual task performance have suffered the caveat of lesions being largely diffuse and extensive beyond sensorimotor cortices. The current study addresses this limitation in reporting on 20 pre-operative neurosurgical patients with focal lesion to the pre- and post-central area corresponding to somatotopic representations. Patients were presented with a battery of neuropsychological tests and experimental tasks tapping into motor imagery and verbal conceptual verb processing in addition to neurophysiological measures including DTI, fMRI, and MEP being measured. Results indicated that left tumor patients who presented with a lesion at or near somatotopic hand representations performed significantly worse on the mental rotation hand task and that performance correlated with MEP amplitudes in the upper limb motor region. Furthermore, performance on tasks of verbal processing was within the normal range. Taken together, while our results evidence the involvement of the motor system in motor imagery processes, they do not support the embodied view that sensorimotor regions are necessary to tasks of action verb processing.

Too late to be grounded? Motor resonance for action words acquired after middle childhood

Though well established for languages acquired in infancy, the role of embodied mechanisms remains poorly understood for languages learned in middle childhood and adulthood. To bridge this gap, we examined 34 experiments that assessed sensorimotor resonance during processing of action-related words in real and artificial languages acquired since age 7 and into adulthood. Evidence from late bilinguals indicates that foreign-language action words modulate neural activity in motor circuits and predictably facilitate or delay physical movements (even in an effector-specific fashion), with outcomes that prove partly sensitive to language proficiency. Also, data from newly learned vocabularies suggest that embodied effects emerge after brief periods of adult language exposure, remain stable through time, and hinge on the performance of bodily movements (and, seemingly, on action observation, too). In sum, our work shows that infant language exposure is not indispensable for the recruitment of embodied mechanisms during language processing, a finding that carries non-trivial theoretical, pedagogical, and clinical implications for neurolinguistics, in general, and bilingualism research, in particular.

Effortful verb retrieval from semantic memory drives beta suppression in mesial frontal regions involved in action initiation

The contribution of the motor cortex to the semantic retrieval of verbs remains a subject of debate in neuroscience. Here, we examined whether additional engagement of the cortical motor system was required when access to verbs semantics was hindered during a verb generation task. We asked participants to produce verbs related to presented noun cues that were either strongly associated with a single verb to prompt fast and effortless verb retrieval, or were weakly associated with multiple verbs and more difficult to respond to. Using power suppression of magnetoencephalography beta oscillations (15-30 Hz) as an index of cortical activation, we performed a whole-brain analysis in order to identify the cortical regions sensitive to the difficulty of verb semantic retrieval. Highly reliable suppression of beta oscillations occurred 250 ms after the noun cue presentation and was sustained until the onset of verbal response. This was localized to multiple cortical regions, mainly in the temporal and frontal lobes of the left hemisphere. Crucially, the only cortical regions where beta suppression was sensitive to the task difficulty, were the higher order motor areas on the medial and lateral surfaces of the frontal lobe. Stronger activation of the premotor cortex and supplementary motor area accompanied the effortful verb retrieval and preceded the preparation of verbal responses for more than 500 ms, thus, overlapping with the time window of verb retrieval from semantic memory. Our results suggest that reactivation of verb-related motor plans in higher order motor circuitry promotes the semantic retrieval of target verbs.

How meaning unfolds in neural time: Embodied reactivations can precede multimodal semantic effects during language processing

Research on how the brain construes meaning during language use has prompted two conflicting accounts. According to the 'grounded view', word understanding involves quick reactivations of sensorimotor (embodied) experiences evoked by the stimuli, with simultaneous or later engagement of multimodal (conceptual) systems integrating information from various sensory streams. Contrariwise, for the 'symbolic view', this capacity depends crucially on multimodal operations, with embodied systems playing epiphenomenal roles after comprehension. To test these contradictory hypotheses, the present magnetoencephalography study assessed implicit semantic access to grammatically constrained action and non-action verbs (n = 100 per category) while measuring spatiotemporally precise signals from the primary motor cortex (M1, a core region subserving bodily movements) and the anterior temporal lobe (ATL, a putative multimodal semantic hub). Convergent evidence from sensor- and source-level analyses revealed that increased modulations for action verbs occurred earlier in M1 (∼130-190 ms) than in specific ATL hubs (∼250-410 ms). Moreover, machine-learning decoding showed that trial-by-trial classification peaks emerged faster in M1 (∼100-175 ms) than in the ATL (∼345-500 ms), with over 71% accuracy in both cases. Considering their latencies, these results challenge the 'symbolic view' and its implication that sensorimotor mechanisms play only secondary roles in semantic processing. Instead, our findings support the 'grounded view', showing that early semantic effects are critically driven by embodied reactivations and that these cannot be reduced to post-comprehension epiphenomena, even when words are individually classified. Briefly, our study offers non-trivial insights to constrain fine-grained models of language and understand how meaning unfolds in neural time.

The mental simulation of state/psychological verbs in the adolescent brain: An fMRI study

This fMRI study investigated mental simulation of state/psychological and action verbs during adolescence. Sixteen healthy subjects silently read verbs describing a motor scene or not (STIMULUS: motor, state/psychological verbs) and they were explicitly asked to imagine the situation or they performed letter detection preventing them from using simulation (TASK: imagery vs. letter detection). A significant task by stimuli interaction showed that imagery of state/psychological verbs, as compared to action stimuli (controlled by the letter detection) selectively increased activation in the right supramarginal gyrus/rolandic operculum and in the right insula, and decreased activation in the right intraparietal sulcus. We compared these data to those from a group of older participants (Tomasino et al. 2014a). Activation in the left supramarginal gyrus decreased for the latter group (as compared to the present group) for imagery of state/psychological verbs. By contrast, activation in the right superior frontal gyrus decreased for the former group (as compared to the older group) for imagery of state/psychological verbs.

The neural exploitation hypothesis and its implications for an embodied approach to language and cognition: Insights from the study of action verbs processing and motor disorders in Parkinson's disease

As it is widely known, Parkinson's disease is clinically characterized by motor disorders such as the loss of voluntary movement control, including resting tremor, postural instability, and bradykinesia (Bocanegra et al., 2015; Helmich, Hallett, Deuschl, Toni, & Bloem, 2012; Liu et al., 2006; Rosin, Topka, & Dichgans, 1997). In the last years, many empirical studies (e.g., Bocanegra et al., 2015; Spadacenta et al., 2012) have also shown that the processing of action verbs is selectively impaired in patients affected by this neurodegenerative disorder. In the light of these findings, it has been suggested that Parkinson disorder can be interpreted within an embodied cognition framework (e.g., Bocanegra et al., 2015). The central tenet of any embodied approach to language and cognition is that high order cognitive functions are grounded in the sensory-motor system. With regard to this point, Gallese (2008) proposed the neural exploitation hypothesis to account for, at the phylogenetic level, how key aspects of human language are underpinned by brain mechanisms originally evolved for sensory-motor integration. Glenberg and Gallese (2012) also applied the neural exploitation hypothesis to the ontogenetic level. On the basis of these premises, they developed a theory of language acquisition according to which, sensory-motor mechanisms provide a neurofunctional architecture for the acquisition of language, while retaining their original functions as well. The neural exploitation hypothesis is here applied to interpret the profile of patients affected by Parkinson's disease. It is suggested that action semantic impairments directly tap onto motor disorders. Finally, a discussion of what theory of language is needed to account for the interactions between language and movement disorders is presented.

Role of medial premotor areas in action language processing in relation to motor skills

The literature reports that the supplementary motor area (SMA) and pre-supplementary motor area (pre-SMA) are involved in motor planning and execution, and in motor-related cognitive functions such as motor imagery. However, their specific role in action language processing remains unclear. In the present study, we investigated the impact of repetitive transcranial magnetic stimulation (rTMS) over SMA and pre-SMA during an action semantic analogy task (SAT) in relation with fine motor skills (i.e., manual dexterity) and motor imagery abilities in healthy non-expert adults. The impact of rTMS over SMA (but not pre-SMA) on reaction times (RT) during SAT was correlated with manual dexterity. Specifically, results show that rTMS over SMA modulated RT for those with lower dexterity skills. Our results therefore demonstrate a causal involvement of SMA in action language processing, as well as the existence of inter-individual differences in this involvement. We discuss these findings in light of neurolinguistic theories of language processing.

Neural reuse of action perception circuits for language, concepts and communication

Neurocognitive and neurolinguistics theories make explicit statements relating specialized cognitive and linguistic processes to specific brain loci. These linking hypotheses are in need of neurobiological justification and explanation. Recent mathematical models of human language mechanisms constrained by fundamental neuroscience principles and established knowledge about comparative neuroanatomy offer explanations for where, when and how language is processed in the human brain. In these models, network structure and connectivity along with action- and perception-induced correlation of neuronal activity co-determine neurocognitive mechanisms. Language learning leads to the formation of action perception circuits (APCs) with specific distributions across cortical areas. Cognitive and linguistic processes such as speech production, comprehension, verbal working memory and prediction are modelled by activity dynamics in these APCs, and combinatorial and communicative-interactive knowledge is organized in the dynamics within, and connections between APCs. The network models and, in particular, the concept of distributionally-specific circuits, can account for some previously not well understood facts about the cortical 'hubs' for semantic processing and the motor system's role in language understanding and speech sound recognition. A review of experimental data evaluates predictions of the APC model and alternative theories, also providing detailed discussion of some seemingly contradictory findings. Throughout, recent disputes about the role of mirror neurons and grounded cognition in language and communication are assessed critically.

Readers select a comprehension mode independent of pronoun: Evidence from fMRI during narrative comprehension

Perspective is a crucial feature for communicating about events. Yet it is unclear how linguistically encoded perspective relates to cognitive perspective taking. Here, we tested the effect of perspective taking with short literary stories. Participants listened to stories with 1st or 3rd person pronouns referring to the protagonist, while undergoing fMRI. When comparing action events with 1st and 3rd person pronouns, we found no evidence for a neural dissociation depending on the pronoun. A split sample approach based on the self-reported experience of perspective taking revealed 3 comprehension preferences. One group showed a strong 1st person preference, another a strong 3rd person preference, while a third group engaged in 1st and 3rd person perspective taking simultaneously. Comparing brain activations of the groups revealed different neural networks. Our results suggest that comprehension is perspective dependent, but not on the perspective suggested by the text, but on the reader's (situational) preference.

Age-Dependent Changes of Thinking about Verbs

We investigated the knowledge of emotional and motor verbs in children and adolescents from three age ranges (8–11, 12–15, 16–19 years). Participants estimated the verbs familiarity, age of acquisition, valence, arousal, imageability, and motor- and emotion-relatedness. Participants were familiar with the verbs in our dataset. The younger (8–11) attributed an emotional character to the verbs less frequently than the middle (12–15) and the older (16–19) groups. In the 8–11 group males rated the verbs as emotion-related less frequently than females. Results indicate that processing verbal concepts as emotion-related develops gradually, and after 12–15 is rather stable. The age of acquisition (AoA) develops late: the older (16–19) had a higher awareness in reporting that they learnt the verbs earlier as compared to the estimations made by the younger (8–11 and 12–15). AoA positively correlated with attribution of emotion relatedness meaning that emotion-related verbs were learned later. Arousal was comparable across ages. Also it increased when attributing motor relatedness to verbs and decreased when attributing emotion relatedness. Reporting the verbs' affective valence (happy vs. unhappy) changes with age: younger (8–11) judged the verbs generally more “happy” than both the older groups. Instead the middle and the older group did not show differences. Happiness increased when processing the verbs as motor related and decreased when processing the verbs as emotion related. Age affected imageability: the younger (8–11) considered the verbs easier to be imagined than the two older groups, suggesting that at this age vividness estimation is still rough, while after 12–15 is stable as the 12–15 and 15–19 group did not differ. Imageability predicted arousal, AoA, emotion- and motor-relatedness indicating that this index influences the way verbs are processed. Imageability was positively correlated to emotion relatedness, indicating that such verbs were harder to be imagined, and negatively to motor relatedness. Imageablity positively correlated with valence meaning that verbs receiving positive valence were also those that were hard to be imagined, and negatively correlated with arousal, meaning that verbs that were harder to be imagined elicited low physiological activation. Our results give an insight in the development of emotional and motor-related verbs representations.

Grasping the Agent's Perspective: A Kinematics Investigation of Linguistic Perspective in Italian and German

Every day, we primarily experience actions as agents, by having a concrete perspective on our actions, their means and goals. This peculiar perspective is what allows us to successfully plan and execute our actions in a dense social environment. Nevertheless, in this environment actions are also perceived from an observer's perspective. Adopting such a perspective helps us to understand and respond to other's people actions and their outcomes. Importantly, similar experiences of being agent and observer occur also when actions are not physically acted/perceived but are merely linguistically shared. In this paper we present two exploratory studies, one in Italian and one in German, in which we applied a direct comparison of three singular perspectives in combination with different verb categories. First, second and third person pronouns were combined with action and interaction verbs, i.e., verbs implying an interaction with an object - e.g., grasp - or an interaction with an object and another person - e.g., give. By means of kinematics recording, we analyzed participants' reaching-grasping responses to a mouse while they were presented with the different combinations of linguistic stimuli (pronouns and verb type). Results of Experiment 1 on reaching show that, when they are preceded by YOU, interaction verbs reached the velocity peak earlier than action verbs, since a further motor act will follow. Thus pronouns influence perspective taking and while comprehending language we are sensitive to the motor chain organization of verbs. The absence of the same effects in Experiment 2 is likely due to the fact that, being the pronoun in German mandatory, it is perceived as less salient than in Italian. Overall our result supports the idea that language is grounded in the motor system in a flexible way, and highlights the need for cross-linguistic studies in the field of embodied language processing.

The Cognitive Side of M1

The primary motor cortex (M1) is traditionally implicated in voluntary movement control. In order to test the hypothesis that there is a functional topography of M1 activation in studies where it has been implicated in higher cognitive tasks we performed activation-likelihood-estimation (ALE) meta-analyses of functional neuroimaging experiments reporting M1 activation in relation to six cognitive functional categories for which there was a sufficient number of studies to include, namely motor imagery, working memory, mental rotation, social/emotion/empathy, language, and auditory processing. The six categories activated different sub-sectors of M1, either bilaterally or lateralized to one hemisphere. Notably, the activations found in the M1 of the left or right hemisphere detected in our study were unlikely due to button presses. In fact, all contrasts were selected in order to eliminate M1 activation due to activity related to the finger button press. In addition, we identified the M1 sub-region of Area 4a commonly activated by 4/6 categories, namely motor imagery and working memory, emotion/empathy, and language. Overall, our findings lend support to the idea that there is a functional topography of M1 activation in studies where it has been found activated in higher cognitive tasks and that the left Area 4a can be involved in a number of cognitive processes, likely as a product of implicit mental simulation processing.

Premotor Cortex Activation Elicited during Word Comprehension Relies on Access of Specific Action Concepts

The relationship between the lexical-semantic and sensory-motor systems is an important topic in cognitive neuroscience. An important finding indicating that these two systems interact is that reading action verbs activates the motor system of the human brain. Two constraints have been proposed to modulate this activation: the effector information associated with the action concepts and statistical regularities between sublexical features and grammatical classes. Using fMRI, we examined whether these two types of information can activate the motor system in the absence of specific motor-semantic content by manipulating the existence of a sublexical cue, called the hand radical, which strongly indicates the semantic feature "hand-related" and grammatical class "verb." Although hand radical characters referring to specific manual actions evoked stronger activation in the premotor cortex than the control characters, hand radical pseudocharacters did not evoke specific activation within the motor system. These results indicated that activation of the premotor cortex during word reading relies on the access of specific action concepts.

Influences of motor contexts on the semantic processing of action-related language

The contribution of the sensory-motor system to the semantic processing of language stimuli is still controversial. To address the issue, the present article focuses on the impact of motor contexts (i.e., comprehenders' motor behaviors, motor-training experiences, and motor expertise) on the semantic processing of action-related language and reviews the relevant behavioral and neuroimaging findings. The existing evidence shows that although motor contexts can influence the semantic processing of action-related concepts, the mechanism of the contextual influences is still far from clear. Future investigations will be needed to clarify (1) whether motor contexts only modulate activity in motor regions, (2) whether the contextual influences are specific to the semantic features of language stimuli, and (3) what factors can determine the facilitatory or inhibitory contextual influences on the semantic processing of action-related language.

Personal experience with narrated events modulates functional connectivity within visual and motor systems during story comprehension

Past experience of everyday life activities, which forms the basis of our knowledge about the world, greatly affects how we understand stories. Yet, little is known about how this influence is instantiated in the human brain. Here, we used functional magnetic resonance imaging to investigate how past experience facilitates functional connectivity during the comprehension of stories rich in perceptual and motor details. We found that comprehenders' past experience with the scenes and actions described in the narratives selectively modulated functional connectivity between lower- and higher-level areas within the neural systems for visual and motor processing, respectively. These intramodal interactions may play an important role in integrating personal knowledge about a narrated situation with an evolving discourse representation. This study provides empirical evidence consistent with the idea that regions related to visual and motor processing are involved in the reenactment of experience as proposed by theories of embodied cognition.

How do conceptual representations interact with processing demands: An fMRI study on action- and abstract-related words

This fMRI study investigated the functional mechanisms related to mental simulation of abstract- and action-related words and measured the effect of the type of stimulus (Abstract vs. action verbs) and the type of task (imagery vs. control task) to explore how conceptual representations interact with processing demands. A significant task by stimuli interaction showed that action-related words activated the left sensorimotor cortex during explicit imagery (as compared to the control task), whereas abstract-related verbs did not automatically activate this area. Rather, as we made sure that Abstract verbs were not associated with motor states (as tested in a rating study of our stimulus list), imagery elicited by abstract verb processing (as compared to Action verbs, and controlled for letter detection) differentially activated a right hemisphere neural network including the right supramarginal (SMG) gyrus and the precuneus which might be related to mental imagery of emotion-related scenes and not to the semantics of the stimuli per se. Our results confirmed the view that the activation of the sensorimotor cortex during language processing of abstract- and action-related words is strategy-dependent.

Picturing words? Sensorimotor cortex activation for printed words in child and adult readers

Learning to read involves associating abstract visual shapes with familiar meanings. Embodiment theories suggest that word meaning is at least partially represented in distributed sensorimotor networks in the brain (Barsalou, 2008; Pulvermueller, 2013). We explored how reading comprehension develops by tracking when and how printed words start activating these "semantic" sensorimotor representations as children learn to read. Adults and children aged 7-10 years showed clear category-specific cortical specialization for tool versus animal pictures during a one-back categorisation task. Thus, sensorimotor representations for these categories were in place at all ages. However, co-activation of these same brain regions by the visual objects' written names was only present in adults, even though all children could read and comprehend all presented words, showed adult-like task performance, and older children were proficient readers. It thus takes years of training and expert reading skill before spontaneous processing of printed words' sensorimotor meanings develops in childhood.

Differentiating Semantic Categories during the Acquisition of Novel Words: Correspondence Analysis Applied to Event-related Potentials

Growing evidence suggests that semantic knowledge is represented in distributed neural networks that include modality-specific structures. Here, we examined the processes underlying the acquisition of words from different semantic categories to determine whether the emergence of visual- and action-based categories could be tracked back to their acquisition. For this, we applied correspondence analysis (CA) to ERPs recorded at various moments during acquisition. CA is a multivariate statistical technique typically used to reveal distance relationships between words of a corpus. Applied to ERPs, it allows isolating factors that best explain variations in the data across time and electrodes. Participants were asked to learn new action and visual words by associating novel pseudowords with the execution of hand movements or the observation of visual images. Words were probed before and after training on two consecutive days. To capture processes that unfold during lexical access, CA was applied on the 100–400 msec post-word onset interval. CA isolated two factors that organized the data as a function of test sessions and word categories. Conventional ERP analyses further revealed a category-specific increase in the negativity of the ERPs to action and visual words at the frontal and occipital electrodes, respectively. The distinct neural processes underlying action and visual words can thus be tracked back to the acquisition of word-referent relationships and may have its origin in association learning. Given current evidence for the flexibility of language-induced sensory-motor activity, we argue that these associative links may serve functions beyond word understanding, that is, the elaboration of situation models.

A case of bilateral frontal tumors without "frontal syndrome"

We report the longitudinal case study of a right-handed patient harboring two frontal tumors that benefited from bilateral simultaneous surgery. The tumors were WHO Grade II gliomas located in the left inferior frontal area (including the cingulate gyrus) and the right anterior superior frontal gyrus. The double tumor resection was guided by direct electrical stimulation of brain areas while the patient was awake. Neuropsychological assessments were administered before and after the surgery to analyse how the brain functions in the presence of two frontal gliomas that affect both hemispheres and reacts to a bilateral resection, which can brutally compromise the neuronal connectivity, progressively established during the infiltrating process. We showed that both the tumor infiltration and their bilateral resection did not lead to a "frontal syndrome" or a "dysexecutive syndrome" predicted by the localization models. However, a subtle fragility was observed in fine-grain language, memory and emotional skills. This case study reveals the significance of brain plasticity in the reorganization of cognitive networks, even in cases of bilateral tumors. It also confirms the clinical relevance of hodotopical brain models, which considers the brain to be organized in parallel-distributed networks around cortical centers and epicenters.

Passive reading and motor imagery about hand actions and tool-use actions: an fMRI study

Recent studies have shown that motor activations in action verb comprehension can be modulated by task demands (e.g., motor imagery vs. passive reading) and the specificity of action verb meaning. However, how the two factors work together to influence the involvement of the motor system during action verb comprehension is still unclear. To address the issue, the current study investigated the brain activations in motor imagery and passive reading of verbs about hand actions and tool-use actions. Three types of Chinese verbs were used, including hand-action verbs and two types of tool-use verbs emphasizing either the hand or tools information. Results indicated that all three types of verbs elicited common activations in hand motor areas during passive reading and motor imagery. Contrast analyses showed that in the hand verbs and the tool verbs where the hand information was emphasized, motor imagery elicited stronger effects than passive reading in the superior frontal gyrus, supplemental motor area and cingulate cortex that are related to motor control and regulation. For tool-use verbs emphasizing tools information, the motor imagery task elicited stronger activity than passive reading in occipital regions related to visual imagery. These results suggest that motor activations during action verb comprehension can be modulated by task demands and semantic features of action verbs. The sensorimotor simulation during language comprehension is flexible and determined by the interactions between linguistic and extralinguistic contexts.

Putting an “End” to the Motor Cortex Representations of Action Words

Language processing is an example of implicit learning of multiple statistical cues that provide probabilistic information regarding word structure and use. Much of the current debate about language embodiment is devoted to how action words are represented in the brain, with motor cortex activity evoked by these words assumed to selectively reflect conceptual content and/or its simulation. We investigated whether motor cortex activity evoked by manual action words (e.g., caress) might reflect sensitivity to probabilistic orthographic–phonological cues to grammatical category embedded within individual words. We first review neuroimaging data demonstrating that nonwords evoke activity much more reliably than action words along the entire motor strip, encompassing regions proposed to be action category specific. Using fMRI, we found that disyllabic words denoting manual actions evoked increased motor cortex activity compared with non-body-part-related words (e.g., canyon), activity which overlaps that evoked by observing and executing hand movements. This result is typically interpreted in support of language embodiment. Crucially, we also found that disyllabic nonwords containing endings with probabilistic cues predictive of verb status (e.g., -eve) evoked increased activity compared with nonwords with endings predictive of noun status (e.g., -age) in the identical motor area. Thus, motor cortex responses to action words cannot be assumed to selectively reflect conceptual content and/or its simulation. Our results clearly demonstrate motor cortex activity reflects implicit processing of ortho-phonological statistical regularities that help to distinguish a word's grammatical class.

Semantic embodiment, disembodiment or misembodiment? In search of meaning in modules and neuron circuits

"Embodied" proposals claim that the meaning of at least some words, concepts and constructions is grounded in knowledge about actions and objects. An alternative "disembodied" position locates semantics in a symbolic system functionally detached from sensorimotor modules. This latter view is not tenable theoretically and has been empirically falsified by neuroscience research. A minimally-embodied approach now claims that action-perception systems may "color", but not represent, meaning; however, such minimal embodiment (misembodiment?) still fails to explain why action and perception systems exert causal effects on the processing of symbols from specific semantic classes. Action perception theory (APT) offers neurobiological mechanisms for "embodied" referential, affective and action semantics along with "disembodied" mechanisms of semantic abstraction, generalization and symbol combination, which draw upon multimodal brain systems. In this sense, APT suggests integrative-neuromechanistic explanations of why both sensorimotor and multimodal areas of the human brain differentially contribute to specific facets of meaning and concepts.

Perspective taking in language: integrating the spatial and action domains

Language is an inherently social behavior. In this paper, we bring together two research areas that typically occupy distinct sections of the literature: perspective taking in spatial language (whether people represent a scene from their own or a different spatial perspective), and perspective taking in action language (the extent to which they simulate an action as though they were performing that action). First, we note that vocabulary is used inconsistently across the spatial and action domains, and propose a more transparent vocabulary that will allow researchers to integrate action- and spatial-perspective taking. Second, we note that embodied theories of language comprehension often make the narrow assumption that understanding action descriptions involves adopting the perspective of an agent carrying out that action. We argue that comprehenders can adopt embodied action-perspectives other than that of the agent, including those of the patient or an observer. Third, we review evidence showing that perspective taking in spatial language is a flexible process. We argue that the flexibility of spatial-perspective taking provides a means for conversation partners engaged in dialogue to maximize similarity between their situation models. These situation models can then be used as the basis for action language simulations, in which language users adopt a particular action-perspective.

How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics

How brain structures and neuronal circuits mechanistically underpin symbolic meaning has recently been elucidated by neuroimaging, neuropsychological, and neurocomputational research. Modality-specific 'embodied' mechanisms anchored in sensorimotor systems appear to be relevant, as are 'disembodied' mechanisms in multimodal areas. In this paper, four semantic mechanisms are proposed and spelt out at the level of neuronal circuits: referential semantics, which establishes links between symbols and the objects and actions they are used to speak about; combinatorial semantics, which enables the learning of symbolic meaning from context; emotional-affective semantics, which establishes links between signs and internal states of the body; and abstraction mechanisms for generalizing over a range of instances of semantic meaning. Referential, combinatorial, emotional-affective, and abstract semantics are complementary mechanisms, each necessary for processing meaning in mind and brain.

Cortico-spinal embodiment of newly acquired, action-related semantic associations

BACKGROUND

Behavioral and neurophysiological studies indicate that the semantic derivation of the motor skills of a given model (e.g., famous tennis or soccer athlete) modulates the reactivity of arm and leg cortico-spinal representations of an onlooker who performs a categorization task. Information on the possible plastic nature of the sensorimotor mapping of action-related knowledge is still lacking.

OBJECTIVE/HYPOTHESIS

Here we explored the time course of any cortico-spinal excitability modulation induced by the creation of arbitrary associations between a personal name and tennis- or soccer-related motor skills.

METHODS

We recorded the amplitude of Transcranial Magnetic Stimulation (TMS) Motor Evoked Potentials (MEPs) from arm and leg muscles during a categorization task concerning names that were learned in association with either soccer players, tennis players or control, non-motor, identities (actors). We stimulated the cortico-spinal system and recorded the MEPs at three different time points (0-24-72 h) after the association learning.

RESULTS

Coherently with previous literature we found a relative dissociation of leg muscles MEPs during reading of soccer-associated personal names with respect to tennis ones. Importantly this modulation was measured only 72 h after having learned the association. This effect was not found in the arm muscle.

CONCLUSION

The results suggest that for the process of embodying semantic associations in the motor system to take place, the strength of the association itself needs to rise above some level of consolidation.

Seeking a bridge between language and motor cortices: a PPI study

The relation between the sensorimotor cortex and the language network has been widely discussed but still remains controversial. Two independent theories compete to explain how this area is involved during action-related verbs processing. The embodied view assumes that action word representations activate sensorimotor representations which are accessed when an action word is processed or when an action is observed. The abstract hypothesis states that the mental representations of words are abstract and independent of the objects' sensorimotor properties they refer to. We combined neuropsychological and fMRI-PPI connectivity data, to address action-related verbs processing in neurosurgical patients with lesions involving (N = 5) or sparing (N = 5) the primary motor cortex and healthy controls (N = 12). A lack of significant changes in the functional coupling between the left M1 cortex and functional nodes of the linguistic network during the verb generation task was found for all the groups. In addition, we found that the ability to perform an action verb naming task was not related to a damaged M1. These data showed that there was not a task-specific functional interaction active between M1 and the inferior frontal gyrus. We will discuss how these findings indicate that action words do not automatically activate the M1 cortex; we suggest rather that its enrolment could be related to other not strictly linguistic processing.

Effect of levodopa on both verbal and motor representations of action in Parkinson's disease: a fMRI study

Previous studies have demonstrated that non-demented Parkinson's disease (PD) patients have a specific impairment of verb production compared with noun generation. One interpretation of this deficit suggested the influence of striato-frontal dysfunction on action-related verb processing. The aim of our study was to investigate cerebral changes after motor improvement due to dopaminergic medication on the neural circuitry supporting action representation in the brain as mediated by verb generation and motor imagery in PD patients. Functional magnetic resonance imaging on 8 PD patients in "ON" dopaminergic treatment state (DTS) and in "OFF" DTS was used to explore the brain activity during three different tasks: Object Naming (ObjN), Generation of Action Verbs (GenA) in which patients were asked to overtly say an action associated with a picture and mental simulation of action (MSoA) was investigated by asking subjects to mentally simulate an action related to a depicted object. The distribution of brain activities associated with these tasks whatever DTS was very similar to results of previous studies. The results showed that brain activity related to semantics of action is modified by dopaminergic treatment in PD patients. This cerebral reorganisation concerns mainly motor and premotor cortex suggesting an involvement of the putaminal motor loop according to the "motor" theory of verb processing.