The case of CAUSE: neurobiological mechanisms for grounding an abstract concept

The primacy of experience in language processing: Semantic priming is driven primarily by experiential similarity

The organization of semantic memory, including memory for word meanings, has long been a central question in cognitive science. Although there is general agreement that word meaning representations must make contact with sensory-motor and affective experiences in a non-arbitrary fashion, the nature of this relationship remains controversial. One prominent view proposes that word meanings are represented directly in terms of their experiential content (i.e., sensory-motor and affective representations). Opponents of this view argue that the representation of word meanings reflects primarily taxonomic structure, that is, their relationships to natural categories. In addition, the recent success of language models based on word co-occurrence (i.e., distributional) information in emulating human linguistic behavior has led to proposals that this kind of information may play an important role in the representation of lexical concepts. We used a semantic priming paradigm designed for representational similarity analysis (RSA) to quantitatively assess how well each of these theories explains the representational similarity pattern for a large set of words. Crucially, we used partial correlation RSA to account for intercorrelations between model predictions, which allowed us to assess, for the first time, the unique effect of each model. Semantic priming was driven primarily by experiential similarity between prime and target, with no evidence of an independent effect of distributional or taxonomic similarity. Furthermore, only the experiential models accounted for unique variance in priming after partialling out explicit similarity ratings. These results support experiential accounts of semantic representation and indicate that, despite their good performance at some linguistic tasks, the distributional models evaluated here do not encode the same kind of information used by the human semantic system.

The Primacy of Experience in Language Processing: Semantic Priming Is Driven Primarily by Experiential Similarity

The organization of semantic memory, including memory for word meanings, has long been a central question in cognitive science. Although there is general agreement that word meaning representations must make contact with sensory-motor and affective experiences in a non-arbitrary fashion, the nature of this relationship remains controversial. One prominent view proposes that word meanings are represented directly in terms of their experiential content (i.e., sensory-motor and affective representations). Opponents of this view argue that the representation of word meanings reflects primarily taxonomic structure, that is, their relationships to natural categories. In addition, the recent success of language models based on word co-occurrence (i.e., distributional) information in emulating human linguistic behavior has led to proposals that this kind of information may play an important role in the representation of lexical concepts. We used a semantic priming paradigm designed for representational similarity analysis (RSA) to quantitatively assess how well each of these theories explains the representational similarity pattern for a large set of words. Crucially, we used partial correlation RSA to account for intercorrelations between model predictions, which allowed us to assess, for the first time, the unique effect of each model. Semantic priming was driven primarily by experiential similarity between prime and target, with no evidence of an independent effect of distributional or taxonomic similarity. Furthermore, only the experiential models accounted for unique variance in priming after partialling out explicit similarity ratings. These results support experiential accounts of semantic representation and indicate that, despite their good performance at some linguistic tasks, the distributional models evaluated here do not encode the same kind of information used by the human semantic system.

Breakdown of category-specific word representations in a brain-constrained neurocomputational model of semantic dementia

The neurobiological nature of semantic knowledge, i.e., the encoding and storage of conceptual information in the human brain, remains a poorly understood and hotly debated subject. Clinical data on semantic deficits and neuroimaging evidence from healthy individuals have suggested multiple cortical regions to be involved in the processing of meaning. These include semantic hubs (most notably, anterior temporal lobe, ATL) that take part in semantic processing in general as well as sensorimotor areas that process specific aspects/categories according to their modality. Biologically inspired neurocomputational models can help elucidate the exact roles of these regions in the functioning of the semantic system and, importantly, in its breakdown in neurological deficits. We used a neuroanatomically constrained computational model of frontotemporal cortices implicated in word acquisition and processing, and adapted it to simulate and explain the effects of semantic dementia (SD) on word processing abilities. SD is a devastating, yet insufficiently understood progressive neurodegenerative disease, characterised by semantic knowledge deterioration that is hypothesised to be specifically related to neural damage in the ATL. The behaviour of our brain-based model is in full accordance with clinical data-namely, word comprehension performance decreases as SD lesions in ATL progress, whereas word repetition abilities remain less affected. Furthermore, our model makes predictions about lesion- and category-specific effects of SD: our simulation results indicate that word processing should be more impaired for object- than for action-related words, and that degradation of white matter should produce more severe consequences than the same proportion of grey matter decay. In sum, the present results provide a neuromechanistic explanatory account of cortical-level language impairments observed during the onset and progress of semantic dementia.

Dyslexia: Causes and Concomitant Impairments

In recent decades, theories have been presented to explain the nature of dyslexia, but the causes of dyslexia remained unclear. Although the investigation of the causes of dyslexia presupposes a clear understanding of the concept of cause, such an understanding is missing. The present paper proposes the absence of at least one necessary condition or the absence of all sufficient conditions as causes for impaired reading. The causes of impaired reading include: an incorrect fixation location, too short a fixation time, the attempt to recognize too many letters simultaneously, too large saccade amplitudes, and too short verbal reaction times. It is assumed that a longer required fixation time in dyslexic readers results from a functional impairment of areas V1, V2, and V3 that require more time to complete temporal summation. These areas and areas that receive input from them, such as the fusiform gyrus, are assumed to be impaired in their ability to simultaneously process a string of letters. When these impairments are compensated by a new reading strategy, reading ability improves immediately.

Influence of language on perception and concept formation in a brain-constrained deep neural network model

A neurobiologically constrained model of semantic learning in the human brain was used to simulate the acquisition of concrete and abstract concepts, either with or without verbal labels. Concept acquisition and semantic learning were simulated using Hebbian learning mechanisms. We measured the network's category learning performance, defined as the extent to which it successfully (i) grouped partly overlapping perceptual instances into a single (abstract or concrete) conceptual representation, while (ii) still distinguishing representations for distinct concepts. Co-presence of linguistic labels with perceptual instances of a given concept generally improved the network's learning of categories, with a significantly larger beneficial effect for abstract than concrete concepts. These results offer a neurobiological explanation for causal effects of language structure on concept formation and on perceptuo-motor processing of instances of these concepts: supplying a verbal label during concept acquisition improves the cortical mechanisms by which experiences with objects and actions along with the learning of words lead to the formation of neuronal ensembles for specific concepts and meanings. Furthermore, the present results make a novel prediction, namely, that such 'Whorfian' effects should be modulated by the concreteness/abstractness of the semantic categories being acquired, with language labels supporting the learning of abstract concepts more than that of concrete ones. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Modelling concrete and abstract concepts using brain-constrained deep neural networks

A neurobiologically constrained deep neural network mimicking cortical areas relevant for sensorimotor, linguistic and conceptual processing was used to investigate the putative biological mechanisms underlying conceptual category formation and semantic feature extraction. Networks were trained to learn neural patterns representing specific objects and actions relevant to semantically 'ground' concrete and abstract concepts. Grounding sets consisted of three grounding patterns with neurons representing specific perceptual or action-related features; neurons were either unique to one pattern or shared between patterns of the same set. Concrete categories were modelled as pattern triplets overlapping in their 'shared neurons', thus implementing semantic feature sharing of all instances of a category. In contrast, abstract concepts had partially shared feature neurons common to only pairs of category instances, thus, exhibiting family resemblance, but lacking full feature overlap. Stimulation with concrete and abstract conceptual patterns and biologically realistic unsupervised learning caused formation of strongly connected cell assemblies (CAs) specific to individual grounding patterns, whose neurons were spread out across all areas of the deep network. After learning, the shared neurons of the instances of concrete concepts were more prominent in central areas when compared with peripheral sensorimotor ones, whereas for abstract concepts the converse pattern of results was observed, with central areas exhibiting relatively fewer neurons shared between pairs of category members. We interpret these results in light of the current knowledge about the relative difficulty children show when learning abstract words. Implications for future neurocomputational modelling experiments as well as neurobiological theories of semantic representation are discussed.

Sound-Action Symbolism

Recent evidence has shown linkages between actions and segmental elements of speech. For instance, close-front vowels are sound symbolically associated with the precision grip, and front vowels are associated with forward-directed limb movements. The current review article presents a variety of such sound-action effects and proposes that they compose a category of sound symbolism that is based on grounding a conceptual knowledge of a referent in articulatory and manual action representations. In addition, the article proposes that even some widely known sound symbolism phenomena such as the sound-magnitude symbolism can be partially based on similar sensorimotor grounding. It is also discussed that meaning of suprasegmental speech elements in many instances is similarly grounded in body actions. Sound symbolism, prosody, and body gestures might originate from the same embodied mechanisms that enable a vivid and iconic expression of a meaning of a referent to the recipient.

Chinese-English bilinguals show linguistic-perceptual links in the brain associating short spoken phrases with corresponding real-world natural action sounds by semantic category

Higher cognitive functions such as linguistic comprehension must ultimately relate to perceptual systems in the brain, though how and why this forms remains unclear. Different brain networks that mediate perception when hearing real-world natural sounds has recently been proposed to respect a taxonomic model of acoustic-semantic categories. Using functional magnetic resonance imaging (fMRI) with Chinese/English bilingual listeners, the present study explored whether reception of short spoken phrases, in both Chinese (Mandarin) and English, describing corresponding sound-producing events would engage overlapping brain regions at a semantic category level. The results revealed a double-dissociation of cortical regions that were preferential for representing knowledge of human versus environmental action events, whether conveyed through natural sounds or the corresponding spoken phrases depicted by either language. These findings of cortical hubs exhibiting linguistic-perceptual knowledge links at a semantic category level should help to advance neurocomputational models of the neurodevelopment of language systems.

Sensorimotor and interoceptive dimensions in concrete and abstract concepts

Recent theories propose that abstract concepts, compared to concrete ones, might activate to a larger extent interoceptive, social and linguistic experiences. At the same time, recent research has underlined the importance of investigating how different sub-kinds of abstract concepts are represented. We report a pre-registered experiment, preceded by a pilot study, in which we asked participants to evaluate the difficulty of 3 kinds of concrete concepts (natural objects, tools, and food concepts) and abstract concepts (Philosophical and Spiritual concepts, PS, Physical Space Time and Quantity concepts, PSTQ, and Emotional, Mental State and Social concepts, EMSS). While rating the words, participants were assigned to different conditions designed to interfere with conceptual processing: they were required to squeeze a ball (hand motor system activation), to chew gum (mouth motor system activation), to self-estimate their heartbeats (interoception), and to perform a motor articulatory task (inner speech involvement). In a control condition they simply rated the difficulty of words. A possible interference should result in the increase of the difficulty ratings. Bayesian analyses reveal that, compared to concrete ones, abstract concepts are more grounded in interoceptive experience and concrete concepts less in linguistic experience (mouth motor system involvement), and that the experience on which different kinds of abstract and concrete concepts differs widely. For example, within abstract concepts interoception plays a major role for EMSS and PS concepts, while the ball squeezing condition interferes more for PSTQ concepts, confirming that PSTQ are the most concrete among abstract concepts, and tap into sensorimotor manual experience. Implications of the results for current theories of conceptual representation are discussed.

Varieties of abstract concepts: development, use and representation in the brain

The capacity for abstract thought is one of the hallmarks of human cognition. However, the mechanisms underlying the ability to form and use abstract concepts like 'fantasy' and 'grace' have not been elucidated yet. This theme issue brings together developmental, social and cognitive psychologists, linguists, anthropologists, cognitive scientists, neuroscientists, philosophers and computer scientists to present theoretical insights and novel evidence on how abstract concepts are acquired, used and represented in the brain. Many of the contributions conceive concepts as grounded in sensorimotor systems and constrained by bodily mechanisms and structures. The theme issue develops along two main axes, related to the most promising research directions on abstract concepts. The axes focus on (i) the different kinds of abstract concepts (numbers, emotions, evaluative concepts like moral and aesthetic ones, social concepts); (ii) the role played by perception and action, language and sociality, and inner processes (emotions, interoception, metacognition) in grounding abstract concepts. Most papers adopt a cognitive science/neuroscience approach, but the theme issue also includes studies on development, on social cognition, and on how linguistic diversity shapes abstract concepts. Overall, the theme issue provides an integrated theoretical account that highlights the importance of language, sociality and inner processes for abstract concepts, and that offers new methodological tools to investigate them.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.