Modeling the Structure and Dynamics of Semantic Processing

CONcreTEXT norms: Concreteness ratings for Italian and English words in context

Concreteness is a fundamental dimension of word semantic representation that has attracted more and more interest to become one of the most studied variables in the psycholinguistic and cognitive neuroscience literature in the last decade. Concreteness effects have been found at both the brain and the behavioral levels, but they may vary depending on the constraints of the context and task demands. In this study, we collected concreteness norms for English and Italian words presented in different context sentences to allow better control and manipulation of concreteness in future psycholinguistic research. First, we observed high split-half correlations and Cronbach's alpha coefficients, suggesting that our ratings were highly reliable and can be used in Italian- and English-speaking populations. Second, our data indicate that the concreteness ratings are related to the lexical density and accessibility of the sentence in both English and Italian. We also found that the concreteness of words in isolation was highly correlated with that of words in context. Finally, we analyzed differences between nouns and verbs in concreteness ratings without significant effects. Our new concreteness norms of words in context are a valuable source of information for future research in both the English and Italian language. The complete database is available on the Open Science Framework (doi: 10.17605/OSF.IO/U3PC4).

Brain Signatures of Embodied Semantics and Language: A Consensus Paper

According to embodied theories (including embodied, embedded, extended, enacted, situated, and grounded approaches to cognition), language representation is intrinsically linked to our interactions with the world around us, which is reflected in specific brain signatures during language processing and learning. Moving on from the original rivalry of embodied vs. amodal theories, this consensus paper addresses a series of carefully selected questions that aim at determining when and how rather than whether motor and perceptual processes are involved in language processes. We cover a wide range of research areas, from the neurophysiological signatures of embodied semantics, e.g., event-related potentials and fields as well as neural oscillations, to semantic processing and semantic priming effects on concrete and abstract words, to first and second language learning and, finally, the use of virtual reality for examining embodied semantics. Our common aim is to better understand the role of motor and perceptual processes in language representation as indexed by language comprehension and learning. We come to the consensus that, based on seminal research conducted in the field, future directions now call for enhancing the external validity of findings by acknowledging the multimodality, multidimensionality, flexibility and idiosyncrasy of embodied and situated language and semantic processes.

Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.

Decoding semantic representations in mind and brain

A key goal for cognitive neuroscience is to understand the neurocognitive systems that support semantic memory. Recent multivariate analyses of neuroimaging data have contributed greatly to this effort, but the rapid development of these novel approaches has made it difficult to track the diversity of findings and to understand how and why they sometimes lead to contradictory conclusions. We address this challenge by reviewing cognitive theories of semantic representation and their neural instantiation. We then consider contemporary approaches to neural decoding and assess which types of representation each can possibly detect. The analysis suggests why the results are heterogeneous and identifies crucial links between cognitive theory, data collection, and analysis that can help to better connect neuroimaging to mechanistic theories of semantic cognition.

Towards a unifying theory of linguistic meaning

Fundamental tensions exist between formal-logical approaches and cognitive approaches to linguistic meaning. The divergence arises from the fundamental differences in nature and form between formal/mathematical structures of natural language meaning and their cognitive representations. While the former are abstract and logical categories of representations, the latter are ultimately embodied and grounded in sensory-motor systems of the brain. This article aims to motivate a unifying theory/formalism of linguistic meaning from a general biologically integrative perspective in the context of current theorizing in linguistics, neurobiology and cognitive sciences on human language meaning within which two divergent approaches for the mathematical and cognitive aspects of linguistic meaning exist. The tensions can be somewhat neutralized if formal-mathematical structures and cognitive representations of natural language meaning can be shown to have representational duality and unity in brain dynamics. This work shows a broad outline of one, if not the only one, path toward this vision.

Computational Models of Readers' Apperceptive Mass

Recent progress in machine-learning-based distributed semantic models (DSMs) offers new ways to simulate the apperceptive mass (AM; Kintsch, 1980) of reader groups or individual readers and to predict their performance in reading-related tasks. The AM integrates the mental lexicon with world knowledge, as for example, acquired via reading books. Following pioneering work by Denhière and Lemaire (2004), here, we computed DSMs based on a representative corpus of German children and youth literature (Jacobs et al., 2020) as null models of the part of the AM that represents distributional semantic input, for readers of different reading ages (grades 1–2, 3–4, and 5–6). After a series of DSM quality tests, we evaluated the performance of these models quantitatively in various tasks to simulate the different reader groups' hypothetical semantic and syntactic skills. In a final study, we compared the models' performance with that of human adult and children readers in two rating tasks. Overall, the results show that with increasing reading age performance in practically all tasks becomes better. The approach taken in these studies reveals the limits of DSMs for simulating human AM and their potential for applications in scientific studies of literature, research in education, or developmental science.

Semantic memory: A review of methods, models, and current challenges

Adult semantic memory has been traditionally conceptualized as a relatively static memory system that consists of knowledge about the world, concepts, and symbols. Considerable work in the past few decades has challenged this static view of semantic memory, and instead proposed a more fluid and flexible system that is sensitive to context, task demands, and perceptual and sensorimotor information from the environment. This paper (1) reviews traditional and modern computational models of semantic memory, within the umbrella of network (free association-based), feature (property generation norms-based), and distributional semantic (natural language corpora-based) models, (2) discusses the contribution of these models to important debates in the literature regarding knowledge representation (localist vs. distributed representations) and learning (error-free/Hebbian learning vs. error-driven/predictive learning), and (3) evaluates how modern computational models (neural network, retrieval-based, and topic models) are revisiting the traditional "static" conceptualization of semantic memory and tackling important challenges in semantic modeling such as addressing temporal, contextual, and attentional influences, as well as incorporating grounding and compositionality into semantic representations. The review also identifies new challenges regarding the abundance and availability of data, the generalization of semantic models to other languages, and the role of social interaction and collaboration in language learning and development. The concluding section advocates the need for integrating representational accounts of semantic memory with process-based accounts of cognitive behavior, as well as the need for explicit comparisons of computational models to human baselines in semantic tasks to adequately assess their psychological plausibility as models of human semantic memory.

Inferior parietal lobule is sensitive to different semantic similarity relations for concrete and abstract words

Similarity measures, the extent to which two concepts have similar meanings, are the key to understand how concepts are represented, with different theoretical perspectives relying on very different sources of data from which similarity can be calculated. While there is some commonality in similarity measures, the extent of their correlation is limited. Previous studies also suggested that the relative performance of different similarity measures may also vary depending on concept concreteness and that the inferior parietal lobule (IPL) may be involved in the integration of conceptual features in a multimodal system for the semantic categorization. Here, we tested for the first time whether theory-based similarity measures predict the pattern of brain activity in the IPL differently for abstract and concrete concepts. English speakers performed a semantic decision task, while we recorded their brain activity in IPL through fNIRS. Using representational similarity analysis, results indicated that the neural representational similarity in IPL conformed to the lexical co-occurrence among concrete concepts (regardless of the hemisphere) and to the affective similarity among abstract concepts in the left hemisphere only, implying that semantic representations of abstract and concrete concepts are characterized along different organizational principles in the IPL. We observed null results for the decoding accuracy. Our study suggests that the use of the representational similarity analysis as a complementary analysis to the decoding accuracy is a promising tool to reveal similarity patterns between theoretical models and brain activity recorded through fNIRS.

Catching the intangible: a role for emotion?

A crucial aspect of Gilead and colleagues’ ontology is the dichotomy between tangible and intangible representations, but the latter remains rather ill-defined. We propose a fundamental role for interoceptive experience and the statistical distribution of entities in language, especially for intangible representations, that we believe Gilead and colleagues’ ontology needs to incorporate.