Masked priming of conceptual features reveals differential brain activation during unconscious access to conceptual action and sound information

Conceptual representations in the default, control and attention networks are task-dependent and cross-modal

Conceptual knowledge is central to human cognition. Neuroimaging studies suggest that conceptual processing involves modality-specific and multimodal brain regions in a task-dependent fashion. However, it remains unclear (1) to what extent conceptual feature representations are also modulated by the task, (2) whether conceptual representations in multimodal regions are indeed cross-modal, and (3) how the conceptual system relates to the large-scale functional brain networks. To address these issues, we conducted multivariate pattern analyses on fMRI data. 40 participants performed three tasks-lexical decision, sound judgment, and action judgment-on written words. We found that (1) conceptual feature representations are strongly modulated by the task, (2) conceptual representations in several multimodal regions are cross-modal, and (3) conceptual feature retrieval involves the default, frontoparietal control, and dorsal attention networks. Conceptual representations in these large-scale networks are task-dependent and cross-modal. Our findings support theories that assume conceptual processing to rely on a flexible, multi-level architecture.

The cross-linguistic comparison of perceptual strength norms for Korean, English and L2 English

This study aimed to establish perceptual strength norms for 1,000 words in the languages of Korean, English, and L2 English, in order to investigate the similarity and difference across languages as well as the influence of the environment on semantic processing. The perceptual strength norms, which are a collection of word profiles that summarize how a word is experienced through different sensory modalities including the five common senses and interoception, provide a valuable tool for testing embodiment cognition theory. The results of this study demonstrated that language users had parallel sensory experiences with concepts, and that L2 learners were also able to associate their sensory experiences with linguistic concepts. Additionally, the results highlighted the importance of incorporating interoception as a sensory modality in the development of perceptual strength norms, as it had a negative correlation with both vision and concreteness. This study was the first to establish norms for Korean and L2 English and directly compare languages using the identical and translation-equivalent word list.

No fruits without color: Cross-modal priming and EEG reveal different roles for different features across semantic categories

Category-specific impairments witnessed in patients with semantic deficits have broadly dissociated into natural and artificial kinds. However, how the category of food (more specifically, fruits and vegetables) fits into this distinction has been difficult to interpret, given a pattern of deficit that has inconsistently mapped onto either kind, despite its intuitive membership to the natural domain. The present study explores the effects of a manipulation of a visual sensory (i.e., color) or functional (i.e., orientation) feature on the consequential semantic processing of fruits and vegetables (and tools, by comparison), first at the behavioral and then at the neural level. The categorization of natural (i.e., fruits/vegetables) and artificial (i.e., utensils) entities was investigated via cross-modal priming. Reaction time analysis indicated a reduction in priming for color-modified natural entities and orientation-modified artificial entities. Standard event-related potentials (ERP) analysis was performed, in addition to linear classification. For natural entities, a N400 effect at central channel sites was observed for the color-modified condition compared relative to normal and orientation conditions, with this difference confirmed by classification analysis. Conversely, there was no significant difference between conditions for the artificial category in either analysis. These findings provide strong evidence that color is an integral property to the categorization of fruits/vegetables, thus substantiating the claim that feature-based processing guides as a function of semantic category.

Time course of brain activity during the processing of motor- and vision-related abstract concepts: flexibility and task dependency

Grounded cognition theories assume that conceptual processing depends on modality-specific brain systems in a context-dependent fashion. Although the relation of abstract concepts to modality-specific systems is less obvious than for concrete concepts, recent behavioral and neuroimaging studies indicated a foundation of abstract concepts in vision and action. However, due to their poor temporal resolution, neuroimaging studies cannot determine whether sensorimotor activity reflects rapid access to conceptual information or later conceptual processes. The present study therefore assessed the time course of abstract concept processing using event-related potentials (ERPs) and compared ERP responses to abstract concepts with a strong relation to vision or action. We tested whether possible ERP effects to abstract word categories would emerge in early or in later time windows and whether these effects would depend on the depth of the conceptual task. In Experiment 1, a shallow lexical decision task, early feature-specific effects starting at 178 ms were revealed, but later effects beyond 300 ms were also observed. In Experiment 2, a deep conceptual decision task, feature-specific effects with an onset of 22 ms were obtained, but effects again extended beyond 300 ms. In congruency with earlier neuroimaging work, the present feature-specific ERP effects suggest a grounding of abstract concepts in modal brain systems. The presence of early and late feature-specific effects indicates that sensorimotor activity observed in neuroimaging experiments may reflect both rapid conceptual and later post-conceptual processing. Results furthermore suggest that a deep conceptual task accelerates access to conceptual sensorimotor features, thereby demonstrating conceptual flexibility.

Brain Activation During Conceptual Processing of Action and Sound Verbs

Grounded cognition approaches to conceptual representations postulate a close link between conceptual knowledge and the sensorimotor brain systems. The present fMRI study tested, whether a feature-specific representation of concepts, as previously demonstrated for nouns, can also be found for action- and sound-related verbs. Participants were presented with action- and soundrelated verbs along with pseudoverbs while performing a lexical decision task. Sound-related verbs activated auditory areas in the temporal cortex, whereas action-related verbs activated brain regions in the superior frontal gyrus and the cerebellum, albeit only at a more liberal threshold. This differential brain activation during conceptual verb processing partially overlapped with or was adjacent to brain regions activated during the functional localizers probing sound perception or action execution. Activity in brain areas involved in the processing of action information was parametrically modulated by ratings of action relevance. Comparisons of action- and sound-related verbs with pseudoverbs revealed activation for both verb categories in auditory and motor areas. In contrast to proposals of strong grounded cognition approaches, our study did not demonstrate a considerable overlap of activations for action- and sound-related verbs and for the corresponding functional localizer tasks. However, in line with weaker variants of grounded cognition theories, the differential activation pattern for action- and sound-related verbs was near corresponding sensorimotor brain regions depending on conceptual feature relevance. Possibly, action-sound coupling resulted in a mutual activation of the motor and the auditory system for both action- and sound-related verbs, thereby reducing the effect sizes for the differential contrasts.

Task-Dependent Recruitment of Modality-Specific and Multimodal Regions during Conceptual Processing

Conceptual knowledge is central to cognitive abilities such as word comprehension. Previous neuroimaging evidence indicates that concepts are at least partly composed of perceptual and motor features that are represented in the same modality-specific brain regions involved in actual perception and action. However, it is unclear to what extent the retrieval of perceptual-motor features and the resulting engagement of modality-specific regions depend on the concurrent task. To address this issue, we measured brain activity in 40 young and healthy participants using functional magnetic resonance imaging, while they performed three different tasks-lexical decision, sound judgment, and action judgment-on words that independently varied in their association with sounds and actions. We found neural activation for sound and action features of concepts selectively when they were task-relevant in brain regions also activated during auditory and motor tasks, respectively, as well as in higher-level, multimodal regions which were recruited during both sound and action feature retrieval. For the first time, we show that not only modality-specific perceptual-motor areas but also multimodal regions are engaged in conceptual processing in a flexible, task-dependent fashion, responding selectively to task-relevant conceptual features.

Auditory cortex sensitivity to the loudness attribute of verbs

The auditory cortex was shown to be activated during the processing of words describing actions with acoustic features. The present study further examines whether processing visually presented action words characterized by different levels of loudness, i.e. "loud" (to shout) and "quiet" actions (to whisper), differentially engage the auditory cortex. Twenty healthy participants were measured with magnetoencephalography (MEG) while reading inflected verbs followed by a short tone and semantic tasks. Based on the results of a localizer task, loudness sensitive temporal Brodmann areas A22, A41/42, and pSTS were inspected in the word paradigm. "Loud" actions induced significantly stronger beta power suppression compared to "quiet" actions in the left hemisphere. Smaller N100m amplitude related to tones following "loud" compared to "quiet" actions confirmed that auditory cortex sensitivity was modulated by action words. Results point to possible selective auditory simulation mechanisms involved in verb processing and support embodiment theories.

The effect of emotional information from eyes on empathy for pain: A subliminal ERP study

Facial expressions are deeply tied to empathy, which plays an important role during social communication. The eye region is effective at conveying facial expressions, especially fear and sadness emotions. Further, it was proved that subliminal stimuli could impact human behavior. This research aimed to explore the effect of subliminal sad, fearful and neutral emotions conveyed by the eye region on a viewer's empathy for pain using event-related potentials (ERP). The experiment used an emotional priming paradigm of 3 (prime: subliminal neutral, sad, fear eye region information) × 2 (target: painful, nonpainful pictures) within-subject design. Participants were told to judge whether the targets were in pain or not. Results showed that the subliminal sad eye stimulus elicited a larger P2 amplitude than the subliminal fearful eye stimulus when assessing pain. For P3 and late positive component (LPC), the amplitude elicited by the painful pictures was larger than the amplitude elicited by the nonpainful pictures. The behavioral results demonstrated that people reacted to targets depicting pain more slowly after the sad emotion priming. Moreover, the subjective ratings of Personal Distress (PD) (one of the dimensions in Chinese version of Interpersonal Reactivity Index scale) predicted the pain effect in empathic neural responses in the N1 and N2 time window. The current study showed that subliminal eye emotion affected the viewer's empathy for pain. Compared with the subliminal fearful eye stimulus, the subliminal sad eye stimulus had a greater impact on empathy for pain. The perceptual level of pain was deeper in the late controlled processing stage.

Processing of Action and Sound Verbs in Context: An FMRI Study

Recent theories propose a flexible recruitment of sensory and motor brain regions during conceptual processing depending on context and task. The present functional magnetic resonance imaging study investigated the influence of context and task on conceptual processing of action and sound verbs. Participants first performed an explicit semantic context decision task, in which action and sound verbs were presented together with a context noun. The same verbs were repeatedly presented in a subsequent implicit lexical decision task together with new action and sound verbs. Thereafter, motor and acoustic localizer tasks were administered to identify brain regions involved in perception and action. During the explicit task, we found differential activations to action and sound verbs near corresponding sensorimotor brain regions. During the implicit lexical decision task, differences between action and sound verbs were absent. However, feature-specific repetition effects were observed near corresponding sensorimotor brain regions. The present results suggest flexible conceptual representations depending on context and task. Feature-specific effects were observed only near, but not within corresponding sensorimotor brain regions, as defined by the localizer tasks. Our results therefore only provide limited evidence in favor of grounded cognition theories assuming a close link between the conceptual and the sensorimotor systems.

Flexibility in Language Action Interaction: The Influence of Movement Type

Recent neuropsychological studies in neurological patients and healthy subjects suggest a close functional relationship between the brain systems for language and action. Facilitation and inhibition effects of motor system activity on language processing have been demonstrated as well as causal effects in the reverse direction, from language processes on motor excitability or performance. However, as the documented effects between motor and language systems were sometimes facilitatory and sometimes inhibitory, the “sign” of these effects still remains to be explained. In a previous study, we reported a word-category-specific differential impairment of verbal working memory for concordant arm- and leg-related action words brought about by complex sequential movements of the hands and feet. In this article, we seek to determine whether the sign of the functional interaction between language and action systems of the human brain can be changed in a predictable manner by changing movement type. We here report that the sign of the effect of motor movement on action word memory can be reversed from interference to facilitation if, instead of complex movement sequences, simple repetitive movements are performed. Specifically, when engaged in finger tapping, subjects were able to remember relatively more arm-related action words (as compared to control conditions), thus documenting an enhancement of working memory brought about by simple hand movements. In contrast, when performing complex sequences of finger movements, an effector-specific degradation of action word memory was found. By manipulating the sign of the effect in accord with theory-driven predictions, these findings provide support for shared neural bases for motor movement and verbal working memory for action-related words and strengthen the argument that motor systems play a causal and functionally relevant role in language processing semantically related to action.

Feature-Specific Event-Related Potential Effects to Action- and Sound-Related Verbs during Visual Word Recognition

Grounded cognition theories suggest that conceptual representations essentially depend on modality-specific sensory and motor systems. Feature-specific brain activation across different feature types such as action or audition has been intensively investigated in nouns, while feature-specific conceptual category differences in verbs mainly focused on body part specific effects. The present work aimed at assessing whether feature-specific event-related potential (ERP) differences between action and sound concepts, as previously observed in nouns, can also be found within the word class of verbs. In Experiment 1, participants were visually presented with carefully matched sound and action verbs within a lexical decision task, which provides implicit access to word meaning and minimizes strategic access to semantic word features. Experiment 2 tested whether pre-activating the verb concept in a context phase, in which the verb is presented with a related context noun, modulates subsequent feature-specific action vs. sound verb processing within the lexical decision task. In Experiment 1, ERP analyses revealed a differential ERP polarity pattern for action and sound verbs at parietal and central electrodes similar to previous results in nouns. Pre-activation of the meaning of verbs in the preceding context phase in Experiment 2 resulted in a polarity-reversal of feature-specific ERP effects in the lexical decision task compared with Experiment 1. This parallels analogous earlier findings for primed action and sound related nouns. In line with grounded cognitions theories, our ERP study provides evidence for a differential processing of action and sound verbs similar to earlier observation for concrete nouns. Although the localizational value of ERPs must be viewed with caution, our results indicate that the meaning of verbs is linked to different neural circuits depending on conceptual feature relevance.

How Language Is Embodied in Bilinguals and Children with Specific Language Impairment

This manuscript explores the role of embodied views of language comprehension and production in bilingualism and specific language impairment. Reconceptualizing popular models of bilingual language processing, the embodied theory is first extended to this area. Issues such as semantic grounding in a second language and potential differences between early and late acquisition of a second language are discussed. Predictions are made about how this theory informs novel ways of thinking about teaching a second language. Secondly, the comorbidity of speech, language, and motor impairments and how embodiment theory informs the discussion of the etiology of these impairments is examined. A hypothesis is presented suggesting that what is often referred to as specific language impairment may not be so specific due to widespread subclinical motor deficits in this population. Predictions are made about how weaknesses and instabilities in speech motor control, even at a subclinical level, may disrupt the neural network that connects acoustic input, articulatory motor plans, and semantics. Finally, I make predictions about how this information informs clinical practice for professionals such as speech language pathologists and occupational and physical therapists. These new hypotheses are placed within the larger framework of the body of work pertaining to semantic grounding, action-based language acquisition, and action-perception links that underlie language learning and conceptual grounding.

Auditory perception modulated by word reading

Theories of embodied cognition positing that sensorimotor areas are indispensable during language comprehension are supported by neuroimaging and behavioural studies. Among others, the auditory system has been suggested to be important for understanding sound-related words (visually presented) and the motor system for action-related words. In this behavioural study, using a sound detection task embedded in a lexical decision task, we show that in participants with high lexical decision performance sound verbs improve auditory perception. The amount of modulation was correlated with lexical decision performance. Our study provides convergent behavioural evidence of auditory cortex involvement in word processing, supporting the view of embodied language comprehension concerning the auditory domain.

The modality-switch effect: visually and aurally presented prime sentences activate our senses

Verifying different sensory modality properties for concepts results in a processing cost known as the modality-switch effect. It has been argued that this cognitive cost is the result of a perceptual simulation. This paper extends this argument and reports an experiment investigating whether the effect is the result of an activation of sensory information which can also be triggered by perceptual linguistically described stimuli. Participants were first exposed to a prime sentence describing a light or a sound’s perceptual property (e.g., “The light is flickering”, “The sound is echoing”), then required to perform a property-verification task on a target sentence (e.g., “Butter is yellowish”, “Leaves rustle”). The content modalities of the prime and target sentences could be compatible (i.e., in the same modality: e.g., visual–visual) or not (i.e., in different modalities). Crucially, we manipulated the stimuli’s presentation modality such that half of the participants was faced with written sentences while the other half was faced with aurally presented sentences. Results show a cost when two different modalities alternate, compared to when the same modality is repeated with both visual and aural stimuli presentations. This result supports the embodied and grounded cognition view which claims that conceptual knowledge is grounded into the perceptual system. Specifically, this evidence suggests that sensory modalities can be pre-activated through the simulation of either read or listened linguistic stimuli describing visual or acoustic perceptual properties.

Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes

While major advances have been made in uncovering the neural processes underlying perceptual representations, our grasp of how the brain gives rise to conceptual knowledge remains relatively poor. Recent work has provided strong evidence that concepts rely, at least in part, on the same sensory and motor neural systems through which they were acquired, but it is still unclear whether the neural code for concept representation uses information about sensory-motor features to discriminate between concepts. In the present study, we investigate this question by asking whether an encoding model based on five semantic attributes directly related to sensory-motor experience - sound, color, visual motion, shape, and manipulation - can successfully predict patterns of brain activation elicited by individual lexical concepts. We collected ratings on the relevance of these five attributes to the meaning of 820 words, and used these ratings as predictors in a multiple regression model of the fMRI signal associated with the words in a separate group of participants. The five resulting activation maps were then combined by linear summation to predict the distributed activation pattern elicited by a novel set of 80 test words. The encoding model predicted the activation patterns elicited by the test words significantly better than chance. As expected, prediction was successful for concrete but not for abstract concepts. Comparisons between encoding models based on different combinations of attributes indicate that all five attributes contribute to the representation of concrete concepts. Consistent with embodied theories of semantics, these results show, for the first time, that the distributed activation pattern associated with a concept combines information about different sensory-motor attributes according to their respective relevance. Future research should investigate how additional features of phenomenal experience contribute to the neural representation of conceptual knowledge.

Concept Representation Reflects Multimodal Abstraction: A Framework for Embodied Semantics

Recent research indicates that sensory and motor cortical areas play a significant role in the neural representation of concepts. However, little is known about the overall architecture of this representational system, including the role played by higher level areas that integrate different types of sensory and motor information. The present study addressed this issue by investigating the simultaneous contributions of multiple sensory-motor modalities to semantic word processing. With a multivariate fMRI design, we examined activation associated with 5 sensory-motor attributes--color, shape, visual motion, sound, and manipulation--for 900 words. Regions responsive to each attribute were identified using independent ratings of the attributes' relevance to the meaning of each word. The results indicate that these aspects of conceptual knowledge are encoded in multimodal and higher level unimodal areas involved in processing the corresponding types of information during perception and action, in agreement with embodied theories of semantics. They also reveal a hierarchical system of abstracted sensory-motor representations incorporating a major division between object interaction and object perception processes.

Nouns, verbs, objects, actions, and abstractions: local fMRI activity indexes semantics, not lexical categories

Noun/verb dissociations in the literature defy interpretation due to the confound between lexical category and semantic meaning; nouns and verbs typically describe concrete objects and actions. Abstract words, pertaining to neither, are a critical test case: dissociations along lexical-grammatical lines would support models purporting lexical category as the principle governing brain organisation, whilst semantic models predict dissociation between concrete words but not abstract items. During fMRI scanning, participants read orthogonalised word categories of nouns and verbs, with or without concrete, sensorimotor meaning. Analysis of inferior frontal/insula, precentral and central areas revealed an interaction between lexical class and semantic factors with clear category differences between concrete nouns and verbs but not abstract ones. Though the brain stores the combinatorial and lexical-grammatical properties of words, our data show that topographical differences in brain activation, especially in the motor system and inferior frontal cortex, are driven by semantics and not by lexical class.

Robust discrimination between EEG responses to categories of environmental sounds in early coma

Humans can recognize categories of environmental sounds, including vocalizations produced by humans and animals and the sounds of man-made objects. Most neuroimaging investigations of environmental sound discrimination have studied subjects while consciously perceiving and often explicitly recognizing the stimuli. Consequently, it remains unclear to what extent auditory object processing occurs independently of task demands and consciousness. Studies in animal models have shown that environmental sound discrimination at a neural level persists even in anesthetized preparations, whereas data from anesthetized humans has thus far provided null results. Here, we studied comatose patients as a model of environmental sound discrimination capacities during unconsciousness. We included 19 comatose patients treated with therapeutic hypothermia (TH) during the first 2 days of coma, while recording nineteen-channel electroencephalography (EEG). At the level of each individual patient, we applied a decoding algorithm to quantify the differential EEG responses to human vs. animal vocalizations as well as to sounds of living vocalizations vs. man-made objects. Discrimination between vocalization types was accurate in 11 patients and discrimination between sounds from living and man-made sources in 10 patients. At the group level, the results were significant only for the comparison between vocalization types. These results lay the groundwork for disentangling truly preferential activations in response to auditory categories, and the contribution of awareness to auditory category discrimination.

Unconscious automatic brain activation of acoustic and action-related conceptual features during masked repetition priming

Classical theories of semantic memory assume that concepts are represented in a unitary amodal memory system. In challenging this classical view, pure or hybrid modality-specific theories propose that conceptual representations are grounded in the sensory-motor brain areas, which typically process sensory and action-related information. Although neuroimaging studies provided evidence for a functional-anatomical link between conceptual processing of sensory or action-related features and the sensory-motor brain systems, it has been argued that aspects of such sensory-motor activation may not directly reflect conceptual processing but rather strategic imagery or postconceptual elaboration. In the present ERP study, we investigated masked effects of acoustic and action-related conceptual features to probe unconscious automatic conceptual processing in isolation. Subliminal feature-specific ERP effects at frontocentral electrodes were observed, which differed with regard to polarity, topography, and underlying brain electrical sources in congruency with earlier findings under conscious viewing conditions. These findings suggest that conceptual acoustic and action representations can also be unconsciously accessed, thereby excluding any postconceptual strategic processes. This study therefore further substantiates a grounding of conceptual and semantic processing in action and perception.

How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics

How brain structures and neuronal circuits mechanistically underpin symbolic meaning has recently been elucidated by neuroimaging, neuropsychological, and neurocomputational research. Modality-specific 'embodied' mechanisms anchored in sensorimotor systems appear to be relevant, as are 'disembodied' mechanisms in multimodal areas. In this paper, four semantic mechanisms are proposed and spelt out at the level of neuronal circuits: referential semantics, which establishes links between symbols and the objects and actions they are used to speak about; combinatorial semantics, which enables the learning of symbolic meaning from context; emotional-affective semantics, which establishes links between signs and internal states of the body; and abstraction mechanisms for generalizing over a range of instances of semantic meaning. Referential, combinatorial, emotional-affective, and abstract semantics are complementary mechanisms, each necessary for processing meaning in mind and brain.