From demonstration to theory in embodied language comprehension: A review

The Effect of Socioeconomically Disadvantaged Stereotype Threat on Inhibitory Control in Individuals with Different Household Incomes

Previous studies have discussed the impact of the socioeconomically disadvantaged stereotype threat (SDST) on inhibitory control. But the specific influences of the SDST on inhibitory control in different household income groups are not clear. We hypothesized that the SDST had different effects on inhibitory control in individuals with distinct household income, and the attribution of stimuli would influence it as well, especially the currency value of the stimuli. To investigate it, two studies were conducted, which required inhibiting their motor responses. Specifically, Study 1 explored the influence of the SDST on basic inhibitory control. Study 2 analyzed the influence of the SDST on inhibitory control when the input stimuli included currency values and monetary conception. The results revealed that the inhibitory control ability was worse in the lower income group but not during the processing of stimuli with currency value. For the effect of the SDST, it found that there was a negative effect on those with a lower household income and a positive effect on those with a higher household income. Based on the findings, the effect of the SDST on inhibitory control in human beings is not stable; instead, it varies depending on the traits of the stimuli in different tasks and of the individuals themselves.

The spatial grounding of politics

In three studies, we advance the research on the association between abstract concepts and spatial dimensions by examining the spatial anchoring of political categories in three different paradigms (spatial placement, memory, and classification) and using non-linguistic stimuli (i.e., photos of politicians). The general hypothesis that politicians of a conservative or socialist party are grounded spatially was confirmed across the studies. In Study 1, photos of politicians were spontaneously placed to the left or right of an unanchored horizontal line depending on their socialist-conservative party affiliation. In Study 2, the political orientation of members of parliament systematically distorted the recall of the spatial positions in which they were originally presented. Finally, Study 3 revealed that classification was more accurate and faster when the politicians were presented in spatially congruent positions (e.g., socialist politician presented on the left side of the monitor) rather than incongruent ones (e.g., socialist on the right side). Additionally, we examined whether participants' political orientation and awareness moderated these effects and showed that spatial anchoring seems independent of political preference but increases with political awareness.

The embodiment of emotion-label words and emotion-laden words: Evidence from late Chinese-English bilinguals

Although increasing studies have confirmed the distinction between emotion-label words (words directly label emotional states) and emotion-laden words (words evoke emotions through connotations), the existing evidence is inconclusive, and their embodiment is unknown. In the current study, the emotional categorization task was adopted to investigate whether these two types of emotion words are embodied by directly comparing how they are processed in individuals' native language (L1) and the second language (L2) among late Chinese-English bilinguals. The results revealed that apart from L2 negative emotion-laden words, both types of emotion words in L1 and L2 produced significant emotion effects, with faster response times and/or higher accuracy rates. In addition, processing facilitation for emotion-label words over emotion-laden words was observed irrespective of language operation; a significant three-way interaction between the language, valence and emotion word type was noted. Taken together, this study suggested that the embodiment of emotion words is modulated by the emotion word type, and L2 negative emotion-laden words tend to be affectively disembodied. The disassociation between emotion-label and emotion-laden words is confirmed in both L1 and L2 and therefore, future emotion word research should take the emotion word type into account.

Evidence for dynamic attentional bias toward positive emotion-laden words: A behavioral and electrophysiological study

There has been no consensus on the neural dissociation between emotion-label and emotion-laden words, which remains one of the major concerns in affective neurolinguistics. The current study adopted dot-probe tasks to investigate the valence effect on attentional bias toward Chinese emotion-label and emotion-laden words. Behavioral data showed that emotional word type and valence interacted in attentional bias scores with an attentional bias toward positive emotion-laden words rather than positive emotion-label words and that this bias was derived from the disengagement difficulty in positive emotion-laden words. In addition, an attentional bias toward negative emotion-label words relative to positive emotion-label words was observed. The event-related potential (ERP) data demonstrated an interaction between emotional word type, valence, and hemisphere. A significant hemisphere effect was observed during the processing of positive emotion-laden word pairs rather than positive emotion-label, negative emotion-label, and negative emotion-laden word pairs, with positive emotion-laden word pairs eliciting an enhanced P1 in the right hemisphere as compared to the left hemisphere. Our results found a dynamic attentional bias toward positive emotion-laden words; individuals allocated more attention to positive emotion-laden words in the early processing stage and had difficulty disengaging attention from them in the late processing stage.

N400 Indexing the Motion Concept Shared by Music and Words

The two event-related potentials (ERP) studies investigated how verbs and nouns were processed in different music priming conditions in order to reveal whether the motion concept via embodiment can be stimulated and evoked across categories. Study 1 (Tasks 1 and 2) tested the processing of verbs (action verbs vs. state verbs) primed by two music types, with tempo changes (accelerating music vs. decelerating music) and without tempo changes (fast music vs. slow music) while Study 2 (Tasks 3 and 4) tested the processing of nouns (animate nouns vs. inanimate nouns) in the same priming condition as adopted in Study 1. During the experiments, participants were required to hear a piece of music prior to judging whether an ensuing word (verb or noun) is semantically congruent with the motion concept conveyed by the music. The results show that in the priming condition of music with tempo changes, state verbs and inanimate nouns elicited larger N400 amplitudes than action verbs and animate nouns, respectively in the anterior regions and anterior to central regions, whereas in the priming condition of music without tempo changes, action verbs elicited larger N400 amplitudes than state verbs and the two categories of nouns revealed no N400 difference, unexpectedly. The interactions between music and words were significant only in Tasks 1, 2, and 3. Taken together, the results demonstrate that firstly, music with tempo changes and music without tempo prime verbs and nouns in different fashions; secondly, action verbs and animate nouns are easier to process than state verbs and inanimate nouns when primed by music with tempo changes due to the shared motion concept across categories; thirdly, bodily experience differentiates between music and words in coding (encoding and decoding) fashion but the motion concept conveyed by the two categories can be subtly extracted on the metaphorical basis, as indicated in the N400 component. Our studies reveal that music tempos can prime different word classes, favoring the notion that embodied motion concept exists across domains and adding evidence to the hypothesis that music and language share the neural mechanism of meaning processing.

Simulating background settings during spoken and written sentence comprehension

Previous findings from the sentence-picture verification task demonstrated that comprehenders simulate visual information about intrinsic attributes of described objects. Of interest is whether comprehenders may also simulate the setting in which an event takes place, such as, for example, the light information. To address this question, four experiments were conducted in which participants (total N = 412) either listened to (Experiment 1) or read (Experiment 3) sentences like “The sun is shining onto a bench” followed by a picture with the matching object (bench) and either the matching lighting condition of the scene (sunlit bench against the sunlit background) or the mismatching one (moonlit bench against the moonlit background). In both experiments, response times (RTs) were shorter when the lighting condition of the pictured scene matched the one implied in the sentence. However, no difference in RTs was observed when the processing of spoken sentences was interfered with visual noise (Experiment 2). Specifically, the results showed that visual interference disrupted incongruent visual content activated by listening to the sentences, as evidenced by faster responses on mismatching trials. Similarly, no difference in RTs was observed when the lighting condition of the pictured scene matched sentence context, but the target object presented for verification mismatched sentence context (Experiment 4). Thus, the locus of simulation effect is on the lighting representation of the target object rather than the lighting representation of the background. These findings support embodied and situated accounts of cognition, suggesting that comprehenders do not simulate objects independently of background settings.

Is rushing always faster than strolling? A reaction time study on the processing of sentences containing manner of motion verbs

In the context of the embodied cognition debate, an effect of motion verb associated speed information has previously been detected using eye-tracking, functional magnetic resonance imaging (fMRI), and reaction times (RT). The latter, for instance, was implemented by Wender and Weber (1982), who observed that participants were faster in detecting motion in sentences associated with fast motion compared to sentences associated with slow motion after having formed mental images of the sentences' content. It remains open whether the reported effects of speed are associated with automatic lexical-semantic retrieval processes or whether they reflect higher top-down cognitive processes. To answer this question, the paradigm by Wender and Weber (1982) was adopted and further elaborated in the present study. In Experiment 1 visualization instructions were eliminated. Additionally, the stimulus material was manipulated in regards to the agent of the described movement (human vs. object motion) in order to determine the representation's modality (visual vs. motoric). In Experiment 2, the task to detect motion was replaced by the task to judge sensicality. The results suggest that the prompt to perform mental imagery is not a precondition for the engagement of modal representations in this speed of motion paradigm and that the involved representations' modality is visual rather than motoric. However, the modal representations' involvement is dependent on the task. They thus do not seem to be part of the invariant semantic representation of manner of motion verbs.

Explicit (Not Implicit) Attitudes Mediate the Focus of Attention During Sentence Processing

Many studies showed that comprehenders monitor changes in protagonists' emotions and actions. This article reports two experiments that explored how focusing comprehenders' attention on a particular property of the protagonist dimension (e.g., emotional or action state) affects the accessibility of information about target objects mentioned in the sentence. Furthermore, the present research examined whether participants' attitudes toward the issues described in the sentence can modulate comprehension processes. To this end, we asked participants to read sentences about environmental issues that focused comprehenders' attention on different mental and physical attributes of the same entities (protagonists and objects) and then self-report their own thoughts on the topic of environment by responding to the items assessing their environmental awareness. Importantly, we manipulated the task requirements across two experiments by administering a self-report task (Experiment 1), which required the participants to rate the seriousness and the frequency of the problem mentioned in a sentence; and administering a sentence-picture verification paradigm (Experiment 2), which required the participants to merely indicate if the object depicted in the picture (related to a certain environmental problem) was mentioned in the preceding sentence. The results of these experiments suggest that the focus of a sentence on the environmental problem (rather than the protagonist's emotion and action) enhances the accessibility of information about environmental issues (e.g., plastic garbage); that the comprehender's level of environmental awareness influences one's attention during sentence processing; and that comprehender characteristics significantly modulate comprehension processes only when the measures tap into explicit (and not implicit) processes.

The Wealth State Awareness Effect on Attention Allocation in People From Impoverished and Affluent Groups

Previous studies have shown that poverty influences cognitive abilities and that those who have a negative living environment exhibit worse cognitive performance. In addition, eye measures vary following the manipulation of cognitive processing. We examined the distinctive changes in impoverished and affluent persons during tasks that require a high level of concentration using eye-tracking measures. Based on the poverty effect in impoverished people, this study explored how wealth state awareness (WSA) influences them. It was found that the pupillary state indexes of the impoverished participants significantly changed when their WSA was regarding poverty. The results suggest that awareness of poverty may cause impoverished individuals to engage in tasks with more attention allocation and more concentration in the more difficult tasks but that a WSA regarding wealth does not have such effect on them. WSA has no significant effects on their more affluent peers. The findings of this study can contribute to research on WSA effects on impoverished individuals from the perspective of eye measures.

Is Complex Visual Information Implicated During Language Comprehension? The Case of Cast Shadows

Previous research showed that sensorimotor information affects the perception of properties associated with implied perceptual context during language comprehension. Three experiments addressed a novel question of whether perceptual context may contribute to a simulation of information about such out-of-sight objects as cast shadows. In Experiment 1, participants read a sentence that implied a particular shadow cast on a target (blinds vs. an open window) and then verified the picture of the object onto which a shadow was cast. Responses were faster when the shadow of blinds cast on the object matched that implied by the sentence. However, the data did not show the same matching effect for pictures with cast shadows from an open window. In Experiments 2 and 3, we found that verification times for pictures with no cast shadows were faster when preceded by an "open window" sentence, thus suggesting that reading the sentence does not elicit a visual simulation of any specific shadow. Experiment 3 showed that the objects superimposed with a cast shadow of the blinds and blinds themselves were verified faster after reading a "blinds" sentence. However, the results of an order analysis showed the temporal stability of the "blinds shadows" effect, but the disappearance of the "blinds" effect in the second half of the data. We conclude that the results are compatible, to a lesser or greater extent, with multiple accounts, and discuss our findings in the context of a mental imagery view, a mental simulation view, and an amodal representation view.

Perceptual Representations in L1, L2 and L3 Comprehension: Delayed Sentence-Picture Verification

We used delayed sentence-picture verification tasks to investigate multilingual perceptual representations. In experiment 1, participants listened to sentences with implied shapes. After a 10-min interval, they judged whether pictures had been mentioned in the preceding sentences or not. Results in experiment 1 showed significant match effect in L1, but not in high proficient L2 or low proficient L3. In experiment 2, Participants listened to one language block, then immediately judged one picture block, totally three language-picture blocks. Results in experiment 2 were parallel to results in experiment 1. Our study supports the view of distributed conception: L2 and L3 are associated with less perceptual symbols than L1, indicating great impact of acquisition styles on perceptual representations. Our results show little impact of language proficiency levels on perceptual representations in delayed tasks.

The role of motor system in action-related language comprehension in L1 and L2: An fMRI study

The framework of embodied cognition has challenged the modular view of a language-cognition divide by suggesting that meaning-retrieval critically involves the sensory-motor system. Despite extensive research into the neural mechanisms underlying language-motor coupling, it remains unclear how the motor system might be differentially engaged by different levels of linguistic abstraction and language proficiency. To address this issue, we used fMRI to quantify neural activations in brain regions underlying motor and language processing in Chinese-English speakers' processing of literal, metaphorical, and abstract language in their L1 and L2. Results overall revealed a response in motor ROIs gradually attenuating in intensity from literal to abstract via metaphorical language in both L1 and L2. Furthermore, contrast analyses between L1 and L2 showed overall greater activations of motor ROIs in the L2. We conclude that motor involvement in language processing is graded rather than all-or-none and that the motor system has a dual-functional role.

Twelve-Month-Old Infants Respond to Speech About Absent Inaccessible Objects

This article investigated the interplay of 12-month-old infants' perception of affordances for locomotion and their ability to respond to the mention of hidden objects. In Experiment I, a toy was hidden in an ottoman that was placed on a cabinet out of infants' reach. Infants were more likely to look at, point to or approach the ottoman when there were stairs leading to it than when there were none. The stairs did not help infants respond by highlighting the target corner of the room (Experiment II) or by boosting their engagement with the study events (Experiment III). This suggests that infants' perception of the accessibility of the hiding location influences their ability to respond to speech about absent things.

Holding a real object during encoding helps the learning of foreign vocabulary

This study aims at assessing and comparing two different methods for learning new vocabulary words in a foreign language. Learning vocabulary with images as non-verbal aids was compared to learning vocabulary with real objects. The Rwandan children who participated in this study learnt French as a third language. They took part in training sessions to learn different French words either seeing the corresponding image or holding the corresponding object. The training program was implemented in a Rwandan primary school with children of different ages (from five to 10 years old). The results showed that the words associated to objects that were held by the children during learning were better memorized than the words associated with images. The global memory performance was lower for the youngest children; however, learning with objects proved to be superior over learning with images for all ages. Taken together, the findings underscore that learning vocabulary with real objects is particularly efficient and support the idea that the embodied theory of language is a key element to effectively master a foreign language.

The Effect of the Embodied Guidance in the Insight Problem Solving: An Eye Movement Study

Insight is an important cognitive process in creative thinking. The present research applied embodied cognitive perspective to explore the effect of embodied guidance on insight problem solving and its underlying mechanisms by two experiments. Experiment 1 used the matchstick arithmetic problem to explore the role of embodied gestures guidance in problem solving. The results showed that the embodied gestures facilitate the participants’ performance. Experiment 2 investigated how embodied attention guidance affects insight problem solving. The results showed that participants performed better in prototypical guidance condition. Experiment 2a adopted the Duncker’s radiation problem to explore how embodied behavior and prototypical guidance influence problem solving by attention tracing techniques. Experiment 2b aimed to further examine whether implicit attention transfer was the real cause which resulted in participants over-performing in prototypical guidance condition in Experiment 2a. The results demonstrated that overt physical motion was unnecessary for individuals to experience the benefits of embodied guidance in problem solving, which supported the reciprocal relation hypothesis of saccades and attention. In addition, the questionnaire completed after experiments showed that participants did not realize the relation between guidance and insight problem solving. Taken together, the current study provided further evidence for that embodied gesture and embodied attention both facilitated the insight problem solving and the facilitation is implicit.

How do German bilingual schoolchildren process German prepositions? - A study on language-motor interactions

While much support is found for embodied language processing in a first language (L1), evidence for embodiment in second language (L2) processing is rather sparse. In a recent study, we found support for L2 embodiment, but also an influence of L1 on L2 processing in adult learners. In the present study, we compared bilingual schoolchildren who speak German as one of their languages with monolingual German schoolchildren. We presented the German prepositions auf (on), über (above), and unter (under) in a Stroop-like task. Upward or downward responses were made depending on the font colour, resulting in compatible and incompatible trials. We found compatibility effects for all children, but in contrast to the adult sample, there were no processing differences between the children depending on the nature of their other language, suggesting that the processing of German prepositions of bilingual children is embodied in a similar way as in monolingual German children.

Action Priority: Early Neurophysiological Interaction of Conceptual and Motor Representations

Handling our everyday life, we often react manually to verbal requests or instruction, but the functional interrelations of motor control and language are not fully understood yet, especially their neurophysiological basis. Here, we investigated whether specific motor representations for grip types interact neurophysiologically with conceptual information, that is, when reading nouns. Participants performed lexical decisions and, for words, executed a grasp-and-lift task on objects of different sizes involving precision or power grips while the electroencephalogram was recorded. Nouns could denote objects that require either a precision or a power grip and could, thus, be (in)congruent with the performed grasp. In a control block, participants pointed at the objects instead of grasping them. The main result revealed an event-related potential (ERP) interaction of grip type and conceptual information which was not present for pointing. Incongruent compared to congruent conditions elicited an increased positivity (100–200 ms after noun onset). Grip type effects were obtained in response-locked analyses of the grasping ERPs (100–300 ms at left anterior electrodes). These findings attest that grip type and conceptual information are functionally related when planning a grasping action but such an interaction could not be detected for pointing. Generally, the results suggest that control of behaviour can be modulated by task demands; conceptual noun information (i.e., associated action knowledge) may gain processing priority if the task requires a complex motor response.

Action Contribution to Competence Judgments: The Use of the Journey Schema

The current research considered the question of how performing an action, or merely preparing the body for action, can have an impact on social judgments related to person perception. Participants were asked to ascribe competence and warmth characteristics to a target person by reading a metaphoric text while their body was manipulated to be prepared for the processing of action-congruent information. In Experiment 1, participants whose forward body action matched the metaphoric action described in the text ascribed more competence characteristics to a politician than did control participants. In Experiment 2, participants whose body was merely prepared for forward movement also ascribed more competence characteristics to a politician than did control participants. In addition, the data from Experiment 2 ruled out an alternative non-embodied explanation (i.e., that effect is due to basic associative processes) grounded in the existing literatures on attitudes by demonstrating that body manipulation had no effect on competence when a non-metaphoric text was used. Finally, no evidence was found that body manipulation affects warmth judgments. These studies converge in demonstrating that forward body movements enhance the favorability of competence judgments when these match the metaphoric forward movements described by text.

A Two-Stage Combining Classifier Model for the Development of Adaptive Dialog Systems

This paper proposes a statistical framework to develop user-adapted spoken dialog systems. The proposed framework integrates two main models. The first model is used to predict the user's intention during the dialog. The second model uses this prediction and the history of dialog up to the current moment to predict the next system response. This prediction is performed with an ensemble-based classifier trained for each of the tasks considered, so that a better selection of the next system can be attained weighting the outputs of these specialized classifiers. The codification of the information and the definition of data structures to store the data supplied by the user throughout the dialog makes the estimation of the models from the training data and practical domains manageable. We describe our proposal and its application and detailed evaluation in a practical spoken dialog system.

Contributions to a neurophysiology of meaning: the interpretation of written messages could be an automatic stimulus-reaction mechanism before becoming conscious processing of information

Background. Even though the interpretation of natural language messages is generally conceived as the result of a conscious processing of the message content, the influence of unconscious factors is also well known. What is still insufficiently known is the way such factors work. We have tackled interpretation assuming it is a process, whose basic features are the same for the whole humankind, and employing a naturalistic approach (careful observation of phenomena in conditions the closest to "natural" ones, and precise description before and independently of data statistical analysis). Methodology. Our field research involved a random sample of 102 adults. We presented them with a complete real world-like case of written communication using unabridged message texts. We collected data (participants' written reports on their interpretations) in controlled conditions through a specially designed questionnaire (closed and opened answers); then, we treated it through qualitative and quantitative methods. Principal Findings. We gathered some evidence that, in written message interpretation, between reading and the attribution of conscious meaning, an intermediate step could exist (we named it "disassembling") which looks like an automatic reaction to the text words/expressions. Thus, the process of interpretation would be a discontinuous sequence of three steps having different natures: the initial "decoding" step (i.e., reading, which requires technical abilities), disassembling (the automatic reaction, an unconscious passage) and the final conscious attribution of meaning. If this is true, words and expressions would firstly function like physical stimuli, before being taken into account as symbols. Such hypothesis, once confirmed, could help explaining some links between the cultural (human communication) and the biological (stimulus-reaction mechanisms as the basis for meanings) dimension of humankind.

Bringing back the body into the mind: gestures enhance word learning in foreign language

Foreign language education in the twenty-first century still teaches vocabulary mainly through reading and listening activities. This is due to the link between teaching practice and traditional philosophy of language, where language is considered to be an abstract phenomenon of the mind. However, a number of studies have shown that accompanying words or phrases of a foreign language with gestures leads to better memory results. In this paper, I review behavioral research on the positive effects of gestures on memory. Then I move to the factors that have been addressed as contributing to the effect, and I embed the reviewed evidence in the theoretical framework of embodiment. Finally, I argue that gestures accompanying foreign language vocabulary learning create embodied representations of those words. I conclude by advocating the use of gestures in future language education as a learning tool that enhances the mind.