How the context matters. Literal and figurative meaning in the embodied language paradigm

The role of motor inhibition in implicit negation processing: two Go/No-Go behavioral studies

Several studies demonstrated that explicit forms of negation processing (e.g., "I don't know") recruits motor inhibitory mechanisms. However, whether this is also true for implicit negation, in which the negative meaning is implicated but not explicitly lexicalized in the sentence (e.g., "I ignore"), has never been studied before. Two Go/No-Go studies, which differed only for the time-windows to respond to the Go stimulus, were carried out. In each, participants (N = 86 in experiment 1; N = 87 in experiment 2) respond to coloured circle while reading task-irrelevant affirmative, explicit negative and implicit negative sentences. We aimed to investigate whether: (i) the processing of implicit negations recruits inhibitory mechanisms; (ii) these inhibitory resources are differently modulated by implicit and explicit negations. Results show that implicit negative sentences recruit the inhibitory resources more strongly when compared to explicit ones, probably due to their inferential nature, likely requiring deeper processing of the negative meaning. Implicit and inferential meaning (i.e., pragmatic information) are grounded too in the same mechanisms that integrate action with perception. Such findings provide further evidence to the embodied account of language, showing that even abstract aspects, like implicit negation, are grounded in the sensory-motor system, by means of functional link between language and motor activity.

The relationship between metaphor skills and Theory of Mind in middle childhood: Task and developmental effects

Theoretical pragmatics in the post-Gricean tradition argued that metaphor requires understanding of how another person sees the world. Yet, it is unclear what role mindreading plays in developing metaphor skills. Here we examined the relationship between metaphor and Theory of Mind (ToM) in middle childhood by using two different tasks. In addition to the Physical and Mental Metaphors task (PMM), based on the verbal explanation of physical and mental metaphors, we revived the Referential Metaphors task for children (Noveck, Bianco, & Castry, 2001), where metaphorical and literal referents are presented in a narrative context. The sample included 169 8-, 9-, and 10-year-old children, assessed also for ToM (via the Strange Stories) and other linguistic and cognitive skills as control variables. In the PMM, ToM supported the understanding of mental (but not physical) metaphors in 9-year-olds only, whereas in the Referential Metaphors task ToM supported accuracy of understanding metaphors (but not literal items) in younger children as well. At age 10, ToM effects were negligible in both tasks. These findings suggest that ToM has a task-specific role in metaphor, linked to the characteristics of the items in the task at stake, being for instance greater for metaphors with mental (compared to physical) content and for non-literal (compared to literal) referents. The findings also suggest that the relationship between ToM and metaphor skills is developmental sensitive, as children start to capitalize on ToM earlier in development when the metaphor context is richer, and these effects fade with age. Theoretically, these data argue in favor of the relevance-theoretic account of metaphor, spelling out different ways in which ToM might support metaphor resolution across tasks, for instance by providing better access to the psychological lexicon (i.e., terms referring to mental states) and better context processing, serving as a springboard to achieve sophisticated pragmatic skills in middle childhood.

Is motor inhibition involved in the processing of sentential negation? An assessment via the Stop-Signal Task

In the last decades, the embodied approach to cognition and language gained momentum in the scientific debate, leading to evidence in different aspects of language processing. However, while the bodily grounding of concrete concepts seems to be relatively not controversial, abstract aspects, like the negation logical operator, are still today one of the main challenges for this research paradigm. In this framework, the present study has a twofold aim: (1) to assess whether mechanisms for motor inhibition underpin the processing of sentential negation, thus, providing evidence for a bodily grounding of this logic operator, (2) to determine whether the Stop-Signal Task, which has been used to investigate motor inhibition, could represent a good tool to explore this issue. Twenty-three participants were recruited in this experiment. Ten hand-action-related sentences, both in affirmative and negative polarity, were presented on a screen. Participants were instructed to respond as quickly and accurately as possible to the direction of the Go Stimulus (an arrow) and to withhold their response when they heard a sound following the arrow. This paradigm allows estimating the Stop Signal Reaction Time (SSRT), a covert reaction time underlying the inhibitory process. Our results show that the SSRT measured after reading negative sentences are longer than after reading affirmative ones, highlighting the recruitment of inhibitory mechanisms while processing negative sentences. Furthermore, our methodological considerations suggest that the Stop-Signal Task is a good paradigm to assess motor inhibition's role in the processing of sentence negation.

On the Embodiment of Negation in Italian Sign Language: An Approach Based on Multiple Representation Theories

Negation can be considered a shared social action that develops since early infancy with very basic acts of refusals or rejection. Inspired by an approach to the embodiment of concepts known as Multiple Representation Theories (MRT, henceforth), the present paper explores negation as an embodied action that relies on both sensorimotor and linguistic/social information. Despite the different variants, MRT accounts share the basic ideas that both linguistic/social and sensorimotor information concur to the processes of concepts formation and representation and that the balance between these components depends on the kind of concept, the context, or the performed task. In the present research we will apply the MRT framework for exploring negation in Italian sign language (LIS). The nature of negation in LIS has been explored in continuity with the co-speech gesture where negative elements are encoded through differentiated prosodic and gestural strategies across languages. Data have been collected in naturalistic settings that may allow a much wider understanding of negation both in speech and in spoken language with a semi-structured interview. Five LIS participants with age range 30–80 were recruited and interviewed with the aim of understanding the continuity between gesture and sign in negation. Results highlight that negation utterances mirror the functions of rejection, non-existence and denial that have been described in language acquisition both in deaf and hearing children. These different steps of acquisition of negation show a different balance between sensorimotor, linguistic and social information in the construction of negative meaning that the MRT is able to enlighten.

Communicative And Affective Components in Processing Auditory Vitality Forms: An fMRI Study

In previous studies on auditory vitality forms, we found that listening to action verbs pronounced gently or rudely, produced, relative to a neutral robotic voice, activation of the dorso-central insula. One might wonder whether this insular activation depends on the conjunction of action verbs and auditory vitality forms, or whether auditory vitality forms are sufficient per se to activate the insula. To solve this issue, we presented words not related to actions such as concrete nouns (e.g.,“ball”), pronounced gently or rudely. No activation of the dorso-central insula was found. As a further step, we examined whether interjections, i.e., speech stimuli conveying communicative intention (e.g., “hello”), pronounced with different vitality forms, would be able to activate, relative to control, the insula. The results showed that stimuli conveying a communicative intention, pronounced with different auditory vitality forms activate the dorsal-central insula. These data deepen our understanding of the vitality forms processing, showing that insular activation is not specific to action verbs, but can be also activated by speech acts conveying communicative intention such as interjections. These findings also show the intrinsic social nature of vitality forms because activation of the insula was not observed in the absence of a communicative intention.

A transformer-based approach to irony and sarcasm detection

Figurative language (FL) seems ubiquitous in all social media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of natural language processing, mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper, we employ advanced deep learning methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work (Potamias et al., in: International conference on engineering applications of neural networks, Springer, Berlin, pp 164–175, 2019), we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which is further enhanced with the employment and devise of a recurrent convolutional neural network. With this setup, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state-of-the-art methodologies and systems. Results demonstrate that the proposed methodology achieves state-of-the-art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.

Language, Gesture, and Emotional Communication: An Embodied View of Social Interaction

Spoken language is an innate ability of the human being and represents the most widespread mode of social communication. The ability to share concepts, intentions and feelings, and also to respond to what others are feeling/saying is crucial during social interactions. A growing body of evidence suggests that language evolved from manual gestures, gradually incorporating motor acts with vocal elements. In this evolutionary context, the human mirror mechanism (MM) would permit the passage from “doing something” to “communicating it to someone else.” In this perspective, the MM would mediate semantic processes being involved in both the execution and in the understanding of messages expressed by words or gestures. Thus, the recognition of action related words would activate somatosensory regions, reflecting the semantic grounding of these symbols in action information. Here, the role of the sensorimotor cortex and in general of the human MM on both language perception and understanding is addressed, focusing on recent studies on the integration between symbolic gestures and speech. We conclude documenting some evidence about MM in coding also the emotional aspects conveyed by manual, facial and body signals during communication, and how they act in concert with language to modulate other’s message comprehension and behavior, in line with an “embodied” and integrated view of social interaction.

A Peircean account of concepts: grounding abstraction in phylogeny through a comparative neuroscientific perspective

The nature of concepts has always been a hotly debated topic in both philosophy and psychology and, more recently, also in cognitive neuroscience. Different accounts have been proposed of what concepts are. These accounts reflect deeply different conceptions of how the human mind works. In the last decades, two diametrically opposed theories of human cognition have been discussed and empirically investigated: the Computational Theory of Mind, on the one hand (Fodor 1983 The modularity of mind: an essay on faculty psychology; Pylyshyn 1984 Computation and cognition: toward a foundation for cognitive science), and Embodied Cognition (Barsalou 2008 Annu. Rev. Psychol.59, 617-645. (doi:10.1146/annurev.psych.59.103006.093639); Gallese & Lakoff 2005 Cogn. Neuropsychol.22, 455-479 (doi:10.1080/02643290442000310); Shapiro 2011 Embodied cognition), on the other hand. The former proposes that concepts are abstract and amodal symbols in the language of thought, while the latter argues for the embodied nature of concepts that are conceived of as grounded in actions and perception. The embodiment of both concrete and abstract concepts has been challenged by many (e.g. Mahon & Caramazza 2008 J. Physiol.102, 59-70. (doi:10.1016/j.jphysparis.2008.03.004); Caramazza et al 2014 Annu. Rev. Neurosci.37, 1-15. (doi:10.1146/annurev-neuro-071013-013950)). These challenges will be here taken seriously and addressed from a comparative perspective. We will provide a phylogenetic and neurobiologically inspired account of the embodied nature of both abstract and concrete concepts. We will propose that, although differing in certain respect, they both might have a bodily foundation. Commonalities between abstract and concrete concepts will be explained by recurring to the Peircean notions of icon and abductive inference (CP 2.247). According to Peirce, icons are the kind of signs on which abductive inferences rest (Peirce CS 1931 in Collected papers of Charles S. Peirce, Hartshorne C, Weiss P, Burks AW. (eds), 40; Peirce CS 1997 In The 1903 Harvard lectures on pragmatism (ed. A. Turrisi)). It will be claimed that the mechanism of Embodied Simulation (Gallese & Sinigaglia 2011 Trends Cogn. Sci.15, 512-519. (doi:10.1016/j.tics.2011.09.003)) can be described as an icon (Cuccio V & Caruana F. 2015 Il corpo come icona. Abduzione, strumenti ed Embodied Simulation. Versus, n. 119, 93-103), and it will then be suggested that on these, basic natural signs rest, both phylogenetically and ontogenetically, the capacity to conceptualize.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

The neural exploitation hypothesis and its implications for an embodied approach to language and cognition: Insights from the study of action verbs processing and motor disorders in Parkinson's disease

As it is widely known, Parkinson's disease is clinically characterized by motor disorders such as the loss of voluntary movement control, including resting tremor, postural instability, and bradykinesia (Bocanegra et al., 2015; Helmich, Hallett, Deuschl, Toni, & Bloem, 2012; Liu et al., 2006; Rosin, Topka, & Dichgans, 1997). In the last years, many empirical studies (e.g., Bocanegra et al., 2015; Spadacenta et al., 2012) have also shown that the processing of action verbs is selectively impaired in patients affected by this neurodegenerative disorder. In the light of these findings, it has been suggested that Parkinson disorder can be interpreted within an embodied cognition framework (e.g., Bocanegra et al., 2015). The central tenet of any embodied approach to language and cognition is that high order cognitive functions are grounded in the sensory-motor system. With regard to this point, Gallese (2008) proposed the neural exploitation hypothesis to account for, at the phylogenetic level, how key aspects of human language are underpinned by brain mechanisms originally evolved for sensory-motor integration. Glenberg and Gallese (2012) also applied the neural exploitation hypothesis to the ontogenetic level. On the basis of these premises, they developed a theory of language acquisition according to which, sensory-motor mechanisms provide a neurofunctional architecture for the acquisition of language, while retaining their original functions as well. The neural exploitation hypothesis is here applied to interpret the profile of patients affected by Parkinson's disease. It is suggested that action semantic impairments directly tap onto motor disorders. Finally, a discussion of what theory of language is needed to account for the interactions between language and movement disorders is presented.

Language for action: Motor resonance during the processing of human and robotic voices

In this fMRI study we evaluated whether the auditory processing of action verbs pronounced by a human or a robotic voice in the imperative mood differently modulates the activation of the mirror neuron system (MNs). The study produced three results. First, the activation pattern found during listening to action verbs was very similar in both the robot and human conditions. Second, the processing of action verbs compared to abstract verbs determined the activation of the fronto-parietal circuit classically involved during the action goal understanding. Third, and most importantly, listening to action verbs compared to abstract verbs produced activation of the anterior part of the supramarginal gyrus (aSMG) regardless of the condition (human and robot) and in the absence of any object name. The supramarginal gyrus is a region considered to underpin hand-object interaction and associated to the processing of affordances. These results suggest that listening to action verbs may trigger the recruitment of motor representations characterizing affordances and action execution, coherently with the predictive nature of motor simulation that not only allows us to re-enact motor knowledge to understand others' actions but also prepares us for the actions we might need to carry out.

Mental Reactivation and Pleasantness Judgment of Experience Related to Vision, Hearing, Skin Sensations, Taste and Olfaction

Language acquisition is based on our knowledge about the world and forms through multiple sensory-motor interactions with the environment. We link the properties of individual experience formed at different stages of ontogeny with the phased development of sensory modalities and with the acquisition of words describing the appropriate forms of sensitivity. To test whether early-formed experience related to skin sensations, olfaction and taste differs from later-formed experience related to vision and hearing, we asked Russian-speaking participants to categorize or to assess the pleasantness of experience mentally reactivated by sense-related adjectives found in common dictionaries. It was found that categorizing adjectives in relation to vision, hearing and skin sensations took longer than categorizing adjectives in relation to olfaction and taste. In addition, experience described by adjectives predominantly related to vision, hearing and skin sensations took more time for the pleasantness judgment and generated less intense emotions than that described by adjectives predominantly related to olfaction and taste. Interestingly the dynamics of skin resistance corresponded to the intensity and pleasantness of reported emotions. We also found that sense-related experience described by early-acquired adjectives took less time for the pleasantness judgment and generated more intense and more positive emotions than that described by later-acquired adjectives. Correlations were found between the time of the pleasantness judgment of experience, intensity and pleasantness of reported emotions, age of acquisition, frequency, imageability and length of sense-related adjectives. All in all these findings support the hypothesis that early-formed experience is less differentiated than later-formed experience.