A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data

Language proficiency is associated with neural representational dimensionality of semantic concepts

Previous studies suggest that semantic concepts are characterized by high-dimensional neural representations and that language proficiency affects semantic processing. However, it is not clear whether language proficiency modulates the dimensional representations of semantic concepts at the neural level. To address this question, the present study adopted principal component analysis (PCA) and representational similarity analysis (RSA) to examine the differences in representational dimensionalities (RDs) and in semantic representations between words in highly proficient (Chinese) and less proficient (English) language. PCA results revealed that language proficiency increased the dimensions of lexical representations in the left inferior frontal gyrus, temporal pole, inferior temporal gyrus, supramarginal gyrus, angular gyrus, and fusiform gyrus. RSA results further showed that these regions represented semantic information and that higher semantic representations were observed in highly proficient language relative to less proficient language. These results suggest that language proficiency is associated with the neural representational dimensionality of semantic concepts.

Mapping and modeling the semantic space of math concepts

Mathematics is an underexplored domain of human cognition. While many studies have focused on subsets of math concepts such as numbers, fractions, or geometric shapes, few have ventured beyond these elementary domains. Here, we attempted to map out the full space of math concepts and to answer two specific questions: can distributed semantic models, such a GloVe, provide a satisfactory fit to human semantic judgements in mathematics? And how does this fit vary with education? We first analyzed all of the French and English Wikipedia pages with math contents, and used a semi-automatic procedure to extract the 1000 most frequent math terms in both languages. In a second step, we collected extensive behavioral judgements of familiarity and semantic similarity between them. About half of the variance in human similarity judgements was explained by vector embeddings that attempt to capture latent semantic structures based on cooccurence statistics. Participants' self-reported level of education modulated familiarity and similarity, allowing us to create a partial hierarchy among high-level math concepts. Our results converge onto the proposal of a map of math space, organized as a database of math terms with information about their frequency, familiarity, grade of acquisition, and entanglement with other concepts.

What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

Why concepts are (probably) vectors

For decades, cognitive scientists have debated what kind of representation might characterize human concepts. Whatever the format of the representation, it must allow for the computation of varied properties, including similarities, features, categories, definitions, and relations. It must also support the development of theories, ad hoc categories, and knowledge of procedures. Here, we discuss why vector-based representations provide a compelling account that can meet all these needs while being plausibly encoded into neural architectures. This view has become especially promising with recent advances in both large language models and vector symbolic architectures. These innovations show how vectors can handle many properties traditionally thought to be out of reach for neural models, including compositionality, definitions, structures, and symbolic computational processes.

The primacy of experience in language processing: Semantic priming is driven primarily by experiential similarity

The organization of semantic memory, including memory for word meanings, has long been a central question in cognitive science. Although there is general agreement that word meaning representations must make contact with sensory-motor and affective experiences in a non-arbitrary fashion, the nature of this relationship remains controversial. One prominent view proposes that word meanings are represented directly in terms of their experiential content (i.e., sensory-motor and affective representations). Opponents of this view argue that the representation of word meanings reflects primarily taxonomic structure, that is, their relationships to natural categories. In addition, the recent success of language models based on word co-occurrence (i.e., distributional) information in emulating human linguistic behavior has led to proposals that this kind of information may play an important role in the representation of lexical concepts. We used a semantic priming paradigm designed for representational similarity analysis (RSA) to quantitatively assess how well each of these theories explains the representational similarity pattern for a large set of words. Crucially, we used partial correlation RSA to account for intercorrelations between model predictions, which allowed us to assess, for the first time, the unique effect of each model. Semantic priming was driven primarily by experiential similarity between prime and target, with no evidence of an independent effect of distributional or taxonomic similarity. Furthermore, only the experiential models accounted for unique variance in priming after partialling out explicit similarity ratings. These results support experiential accounts of semantic representation and indicate that, despite their good performance at some linguistic tasks, the distributional models evaluated here do not encode the same kind of information used by the human semantic system.

Semantic encoding during language comprehension at single-cell resolution

From sequences of speech sounds1,2 or letters3, humans can extract rich and nuanced meaning through language. This capacity is essential for human communication. Yet, despite a growing understanding of the brain areas that support linguistic and semantic processing4-12, the derivation of linguistic meaning in neural tissue at the cellular level and over the timescale of action potentials remains largely unknown. Here we recorded from single cells in the left language-dominant prefrontal cortex as participants listened to semantically diverse sentences and naturalistic stories. By tracking their activities during natural speech processing, we discover a fine-scale cortical representation of semantic information by individual neurons. These neurons responded selectively to specific word meanings and reliably distinguished words from nonwords. Moreover, rather than responding to the words as fixed memory representations, their activities were highly dynamic, reflecting the words' meanings based on their specific sentence contexts and independent of their phonetic form. Collectively, we show how these cell ensembles accurately predicted the broad semantic categories of the words as they were heard in real time during speech and how they tracked the sentences in which they appeared. We also show how they encoded the hierarchical structure of these meaning representations and how these representations mapped onto the cell population. Together, these findings reveal a finely detailed cortical organization of semantic representations at the neuron scale in humans and begin to illuminate the cellular-level processing of meaning during language comprehension.

The language network as a natural kind within the broader landscape of the human brain

Language behaviour is complex, but neuroscientific evidence disentangles it into distinct components supported by dedicated brain areas or networks. In this Review, we describe the 'core' language network, which includes left-hemisphere frontal and temporal areas, and show that it is strongly interconnected, independent of input and output modalities, causally important for language and language-selective. We discuss evidence that this language network plausibly stores language knowledge and supports core linguistic computations related to accessing words and constructions from memory and combining them to interpret (decode) or generate (encode) linguistic messages. We emphasize that the language network works closely with, but is distinct from, both lower-level - perceptual and motor - mechanisms and higher-level systems of knowledge and reasoning. The perceptual and motor mechanisms process linguistic signals, but, in contrast to the language network, are sensitive only to these signals' surface properties, not their meanings; the systems of knowledge and reasoning (such as the system that supports social reasoning) are sometimes engaged during language use but are not language-selective. This Review lays a foundation both for in-depth investigations of these different components of the language processing pipeline and for probing inter-component interactions.

Representation of event and object concepts in ventral anterior temporal lobe and angular gyrus

Semantic knowledge includes understanding of objects and their features and also understanding of the characteristics of events. The hub-and-spoke theory holds that these conceptual representations rely on multiple information sources that are integrated in a central hub in the ventral anterior temporal lobes. The dual-hub theory expands this framework with the claim that the ventral anterior temporal lobe hub is specialized for object representation, while a second hub in angular gyrus is specialized for event representation. To test these ideas, we used representational similarity analysis, univariate and psychophysiological interaction analyses of fMRI data collected while participants processed object and event concepts (e.g. "an apple," "a wedding") presented as images and written words. Representational similarity analysis showed that angular gyrus encoded event concept similarity more than object similarity, although the left angular gyrus also encoded object similarity. Bilateral ventral anterior temporal lobes encoded both object and event concept structure, and left ventral anterior temporal lobe exhibited stronger coding for events. Psychophysiological interaction analysis revealed greater connectivity between left ventral anterior temporal lobe and right pMTG, and between right angular gyrus and bilateral ITG and middle occipital gyrus, for event concepts compared to object concepts. These findings support the specialization of angular gyrus for event semantics, though with some involvement in object coding, but do not support ventral anterior temporal lobe specialization for object concepts.

Local similarity and global variability characterize the semantic space of human languages

How does meaning vary across the world's languages? Scholars recognize the existence of substantial variability within specific domains, ranging from nature and color to kinship. The emergence of large language models enables a systems-level approach that directly characterizes this variability through comparison of word organization across semantic domains. Here, we show that meanings across languages manifest lower variability within semantic domains and greater variability between them, using models trained on both 1) large corpora of native language text comprising Wikipedia articles in 35 languages and also 2) Test of English as a Foreign Language (TOEFL) essays written by 38,500 speakers from the same native languages, which cluster into semantic domains. Concrete meanings vary less across languages than abstract meanings, but all vary with geographical, environmental, and cultural distance. By simultaneously examining local similarity and global difference, we harmonize these findings and provide a description of general principles that govern variability in semantic space across languages. In this way, the structure of a speaker's semantic space influences the comparisons cognitively salient to them, as shaped by their native language, and suggests that even successful bilingual communicators likely think with "semantic accents" driven by associations from their native language while writing English. These findings have dramatic implications for language education, cross-cultural communication, and literal translations, which are impossible not because the objects of reference are uncertain, but because associations, metaphors, and narratives interlink meanings in different, predictable ways from one language to another.

The Primacy of Experience in Language Processing: Semantic Priming Is Driven Primarily by Experiential Similarity

The organization of semantic memory, including memory for word meanings, has long been a central question in cognitive science. Although there is general agreement that word meaning representations must make contact with sensory-motor and affective experiences in a non-arbitrary fashion, the nature of this relationship remains controversial. One prominent view proposes that word meanings are represented directly in terms of their experiential content (i.e., sensory-motor and affective representations). Opponents of this view argue that the representation of word meanings reflects primarily taxonomic structure, that is, their relationships to natural categories. In addition, the recent success of language models based on word co-occurrence (i.e., distributional) information in emulating human linguistic behavior has led to proposals that this kind of information may play an important role in the representation of lexical concepts. We used a semantic priming paradigm designed for representational similarity analysis (RSA) to quantitatively assess how well each of these theories explains the representational similarity pattern for a large set of words. Crucially, we used partial correlation RSA to account for intercorrelations between model predictions, which allowed us to assess, for the first time, the unique effect of each model. Semantic priming was driven primarily by experiential similarity between prime and target, with no evidence of an independent effect of distributional or taxonomic similarity. Furthermore, only the experiential models accounted for unique variance in priming after partialling out explicit similarity ratings. These results support experiential accounts of semantic representation and indicate that, despite their good performance at some linguistic tasks, the distributional models evaluated here do not encode the same kind of information used by the human semantic system.

Grammatical cues to subjecthood are redundant in a majority of simple clauses across languages

Grammatical cues are sometimes redundant with word meanings in natural language. For instance, English word order rules constrain the word order of a sentence like "The dog chewed the bone" even though the status of "dog" as subject and "bone" as object can be inferred from world knowledge and plausibility. Quantifying how often this redundancy occurs, and how the level of redundancy varies across typologically diverse languages, can shed light on the function and evolution of grammar. To that end, we performed a behavioral experiment in English and Russian and a cross-linguistic computational analysis measuring the redundancy of grammatical cues in transitive clauses extracted from corpus text. English and Russian speakers (n = 484) were presented with subjects, verbs, and objects (in random order and with morphological markings removed) extracted from naturally occurring sentences and were asked to identify which noun is the subject of the action. Accuracy was high in both languages (∼89% in English, ∼87% in Russian). Next, we trained a neural network machine classifier on a similar task: predicting which nominal in a subject-verb-object triad is the subject. Across 30 languages from eight language families, performance was consistently high: a median accuracy of 87%, comparable to the accuracy observed in the human experiments. The conclusion is that grammatical cues such as word order are necessary to convey subjecthood and objecthood in a minority of naturally occurring transitive clauses; nevertheless, they can (a) provide an important source of redundancy and (b) are crucial for conveying intended meaning that cannot be inferred from the words alone, including descriptions of human interactions, where roles are often reversible (e.g., Ray helped Lu/Lu helped Ray), and expressing non-prototypical meanings (e.g., "The bone chewed the dog.").

How does linguistic context influence word learning?

While there are well-known demonstrations that children can use distributional information to acquire multiple components of language, the underpinnings of these achievements are unclear. In the current paper, we investigate the potential pre-requisites for a distributional learning model that can explain how children learn their first words. We review existing literature and then present the results of a series of computational simulations with Vector Space Models, a type of distributional semantic model used in Computational Linguistics, which we evaluate against vocabulary acquisition data from children. We focus on nouns and verbs, and we find that: (i) a model with flexibility to adjust for the frequency of events provides a better fit to the human data, (ii) the influence of context words is very local, especially for nouns, and (iii) words that share more contexts with other words are harder to learn.

Signatures of cross-modal alignment in children's early concepts

Whether supervised or unsupervised, human and machine learning is usually characterized as event-based. However, learning may also proceed by systems alignment in which mappings are inferred between entire systems, such as visual and linguistic systems. Systems alignment is possible because items that share similar visual contexts, such as a car and a truck, will also tend to share similar linguistic contexts. Because of the mirrored similarity relationships across systems, the visual and linguistic systems can be aligned at some later time absent either input. In a series of simulation studies, we considered whether children's early concepts support systems alignment. We found that children's early concepts are close to optimal for inferring novel concepts through systems alignment, enabling agents to correctly infer more than 85% of visual-word mappings absent supervision. One possible explanation for why children's early concepts support systems alignment is that they are distinguished structurally by their dense semantic neighborhoods. Artificial agents using these structural features to select concepts proved highly effective, both in environments mirroring children's conceptual world and those that exclude the concepts that children commonly acquire. For children, systems alignment and event-based learning likely complement one another. Likewise, artificial systems can benefit from incorporating these developmental principles.

Bayesian Surprise Predicts Human Event Segmentation in Story Listening

Event segmentation theory posits that people segment continuous experience into discrete events and that event boundaries occur when there are large transient increases in prediction error. Here, we set out to test this theory in the context of story listening, by using a deep learning language model (GPT-2) to compute the predicted probability distribution of the next word, at each point in the story. For three stories, we used the probability distributions generated by GPT-2 to compute the time series of prediction error. We also asked participants to listen to these stories while marking event boundaries. We used regression models to relate the GPT-2 measures to the human segmentation data. We found that event boundaries are associated with transient increases in Bayesian surprise but not with a simpler measure of prediction error (surprisal) that tracks, for each word in the story, how strongly that word was predicted at the previous time point. These results support the hypothesis that prediction error serves as a control mechanism governing event segmentation and point to important differences between operational definitions of prediction error.

A common neural code for meaning in discourse production and comprehension

How does the brain code the meanings conveyed by language? Neuroimaging studies have investigated this by linking neural activity patterns during discourse comprehension to semantic models of language content. Here, we applied this approach to the production of discourse for the first time. Participants underwent fMRI while producing and listening to discourse on a range of topics. We used a distributional semantic model to quantify the similarity between different speech passages and identified where similarity in neural activity was predicted by semantic similarity. When people produced discourse, speech on similar topics elicited similar activation patterns in a widely distributed and bilateral brain network. This network was overlapping with, but more extensive than, the regions that showed similarity effects during comprehension. Critically, cross-task neural similarities between comprehension and production were also predicted by similarities in semantic content. This result suggests that discourse semantics engages a common neural code that is shared between comprehension and production. Effects of semantic similarity were bilateral in all three RSA analyses, even while univariate activation contrasts in the same data indicated left-lateralised BOLD responses. This indicates that right-hemisphere regions encode semantic properties even when they are not activated above baseline. We suggest that right-hemisphere regions play a supporting role in processing the meaning of discourse during both comprehension and production.

SCOPE: The South Carolina psycholinguistic metabase

The number of databases that provide various measurements of lexical properties for psycholinguistic research has increased rapidly in recent years. The proliferation of lexical variables, and the multitude of associated databases, makes the choice, comparison, and standardization of these variables in psycholinguistic research increasingly difficult. Here, we introduce The South Carolina Psycholinguistic Metabase (SCOPE), which is a metabase (or a meta-database) containing an extensive, curated collection of psycholinguistic variable values from major databases. The metabase currently contains 245 lexical variables, organized into seven major categories: General (e.g., frequency), Orthographic (e.g., bigram frequency), Phonological (e.g., phonological uniqueness point), Orth-Phon (e.g., consistency), Semantic (e.g., concreteness), Morphological (e.g., number of morphemes), and Response variables (e.g., lexical decision latency). We hope that SCOPE will become a valuable resource for researchers in psycholinguistics and affiliated disciplines such as cognitive neuroscience of language, computational linguistics, and communication disorders. The availability and ease of use of the metabase with comprehensive set of variables can facilitate the understanding of the unique contribution of each of the variables to word processing, and that of interactions between variables, as well as new insights and development of improved models and theories of word processing. It can also help standardize practice in psycholinguistics. We demonstrate use of the metabase by measuring relationships between variables in multiple ways and testing their individual contribution towards a number of dependent measures, in the most comprehensive analysis of this kind to date. The metabase is freely available at go.sc.edu/scope.

Items Outperform Adjectives in a Computational Model of Binary Semantic Classification

Semantic memory encompasses one's knowledge about the world. Distributional semantic models, which construct vector spaces with embedded words, are a proposed framework for understanding the representational structure of human semantic knowledge. Unlike some classic semantic models, distributional semantic models lack a mechanism for specifying the properties of concepts, which raises questions regarding their utility for a general theory of semantic knowledge. Here, we develop a computational model of a binary semantic classification task, in which participants judged target words for the referent's size or animacy. We created a family of models, evaluating multiple distributional semantic models, and mechanisms for performing the classification. The most successful model constructed two composite representations for each extreme of the decision axis (e.g., one averaging together representations of characteristically big things and another of characteristically small things). Next, the target item was compared to each composite representation, allowing the model to classify more than 1,500 words with human-range performance and to predict response times. We propose that when making a decision on a binary semantic classification task, humans use task prompts to retrieve instances representative of the extremes on that semantic dimension and compare the probe to those instances. This proposal is consistent with the principles of the instance theory of semantic memory.

How do valence and meaning interact? The contribution of semantic control

The hub-and-spoke model of semantic cognition proposes that conceptual representations in a heteromodal 'hub' interact with and emerge from modality-specific features or 'spokes', including valence (whether a concept is positive or negative), along with visual and auditory features. As a result, valence congruency might facilitate our ability to link words conceptually. Semantic relatedness may similarly affect explicit judgements about valence. Moreover, conflict between meaning and valence may recruit semantic control processes. Here we tested these predictions using two-alternative forced-choice tasks, in which participants matched a probe word to one of two possible target words, based on either global meaning or valence. Experiment 1 examined timed responses in healthy young adults, while Experiment 2 examined decision accuracy in semantic aphasia patients with impaired controlled semantic retrieval following left hemisphere stroke. Across both experiments, semantically related targets facilitated valence matching, while related distractors impaired performance. Valence congruency was also found to facilitate semantic decision-making. People with semantic aphasia showed impaired valence matching and had particular difficulty when semantically related distractors were presented, suggesting that the selective retrieval of valence information relies on semantic control processes. Taken together, the results are consistent with the hypothesis that automatic access to the global meaning of written words affects the processing of valence, and that the valence of words is also retrieved even when this feature is task-irrelevant, affecting the efficiency of global semantic judgements.

Decoding semantic representations in mind and brain

A key goal for cognitive neuroscience is to understand the neurocognitive systems that support semantic memory. Recent multivariate analyses of neuroimaging data have contributed greatly to this effort, but the rapid development of these novel approaches has made it difficult to track the diversity of findings and to understand how and why they sometimes lead to contradictory conclusions. We address this challenge by reviewing cognitive theories of semantic representation and their neural instantiation. We then consider contemporary approaches to neural decoding and assess which types of representation each can possibly detect. The analysis suggests why the results are heterogeneous and identifies crucial links between cognitive theory, data collection, and analysis that can help to better connect neuroimaging to mechanistic theories of semantic cognition.

Different computational relations in language are captured by distinct brain systems

A critical way for humans to acquire information is through language, yet whether and how language experience drives specific neural semantic representations is still poorly understood. We considered statistical properties captured by 3 different computational principles of language (simple co-occurrence, network-(graph)-topological relations, and neural-network-vector-embedding relations) and tested the extent to which they can explain the neural patterns of semantic representations, measured by 2 functional magnetic resonance imaging experiments that shared common semantic processes. Distinct graph-topological word relations, and not simple co-occurrence or neural-network-vector-embedding relations, had unique explanatory power for the neural patterns in the anterior temporal lobe (capturing graph-common-neighbors), inferior frontal gyrus, and posterior middle/inferior temporal gyrus (capturing graph-shortest-path). These results were relatively specific to language: they were not explained by sensory-motor similarities and the same computational relations of visual objects (based on visual image database) showed effects in the visual cortex in the picture naming experiment. That is, different topological properties within language and the same topological computations (common-neighbors) for language and visual inputs are captured by different brain regions. These findings reveal the specific neural semantic representations along graph-topological properties of language, highlighting the information type-specific and statistical property-specific manner of semantic representations in the human brain.

Correspondence between cognitive and neural representations for phonology, orthography, and semantics in supramarginal compared to angular gyrus

The angular and supramarginal gyri (AG and SMG) together constitute the inferior parietal lobule (IPL) and have been associated with cognitive functions that support reading. How those functions are distributed across the AG and SMG is a matter of debate, the resolution of which is hampered by inconsistencies across stereotactic atlases provided by the major brain image analysis software packages. Schematic results from automated meta-analyses suggest primarily semantic (word meaning) processing in the left AG, with more spatial overlap among phonological (auditory word form), orthographic (visual word form), and semantic processing in the left SMG. To systematically test for correspondence between patterns of neural activation and phonological, orthographic, and semantic representations, we re-analyze a functional magnetic resonance imaging data set of participants reading aloud 465 words. Using representational similarity analysis, we test the hypothesis that within cytoarchitecture-defined subregions of the IPL, phonological representations are primarily associated with the SMG, while semantic representations are primarily associated with the AG. To the extent that orthographic representations can be de-correlated from phonological representations, they will be associated with cortex peripheral to the IPL, such as the intraparietal sulcus. Results largely confirmed these hypotheses, with some nuanced exceptions, which we discuss in terms of neurally inspired computational cognitive models of reading that learn mappings among distributed representations for orthography, phonology, and semantics. De-correlating constituent representations making up complex cognitive processes, such as reading, by careful selection of stimuli, representational formats, and analysis techniques, are promising approaches for bringing additional clarity to brain structure-function relationships.

How far is brain-inspired artificial intelligence away from brain?

Fueled by the development of neuroscience and artificial intelligence (AI), recent advances in the brain-inspired AI have manifested a tipping-point in the collaboration of the two fields. AI began with the inspiration of neuroscience, but has evolved to achieve a remarkable performance with little dependence upon neuroscience. However, in a recent collaboration, research into neurobiological explainability of AI models found that these highly accurate models may resemble the neurobiological representation of the same computational processes in the brain, although these models have been developed in the absence of such neuroscientific references. In this perspective, we review the cooperation and separation between neuroscience and AI, and emphasize on the current advance, that is, a new cooperation, the neurobiological explainability of AI. Under the intertwined development of the two fields, we propose a practical framework to evaluate the brain-likeness of AI models, paving the way for their further improvements.

Evaluation of word embedding models to extract and predict surgical data in breast cancer

Background

Decisions in healthcare usually rely on the goodness and completeness of data that could be coupled with heuristics to improve the decision process itself. However, this is often an incomplete process. Structured interviews denominated Delphi surveys investigate experts' opinions and solve by consensus complex matters like those underlying surgical decision-making. Natural Language Processing (NLP) is a field of study that combines computer science, artificial intelligence, and linguistics. NLP can then be used as a valuable help in building a correct context in surgical data, contributing to the amelioration of surgical decision-making.

Results

We applied NLP coupled with machine learning approaches to predict the context (words) owning high accuracy from the words nearest to Delphi surveys, used as input.

Conclusions

The proposed methodology has increased the usefulness of Delphi surveys favoring the extraction of keywords that can represent a specific clinical context. It permits the characterization of the clinical context suggesting words for the evaluation process of the data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-022-05038-6.

Flexing the principal gradient of the cerebral cortex to suit changing semantic task demands

Understanding how thought emerges from the topographical structure of the cerebral cortex is a primary goal of cognitive neuroscience. Recent work has revealed a principal gradient of intrinsic connectivity capturing the separation of sensory-motor cortex from transmodal regions of the default mode network (DMN); this is thought to facilitate memory-guided cognition. However, studies have not explored how this dimension of connectivity changes when conceptual retrieval is controlled to suit the context. We used gradient decomposition of informational connectivity in a semantic association task to establish how the similarity in connectivity across brain regions changes during familiar and more original patterns of retrieval. Multivoxel activation patterns at opposite ends of the principal gradient were more divergent when participants retrieved stronger associations; therefore, when long-term semantic information is sufficient for ongoing cognition, regions supporting heteromodal memory are functionally separated from sensory-motor experience. In contrast, when less related concepts were linked, this dimension of connectivity was reduced in strength as semantic control regions separated from the DMN to generate more flexible and original responses. We also observed fewer dimensions within the neural response towards the apex of the principal gradient when strong associations were retrieved, reflecting less complex or varied neural coding across trials and participants. In this way, the principal gradient explains how semantic cognition is organised in the human cerebral cortex: the separation of DMN from sensory-motor systems is a hallmark of the retrieval of strong conceptual links that are culturally shared.

Semantic diversity is best measured with unscaled vectors: Reply to Cevoli, Watkins and Rastle (2020)

Semantic diversity refers to the degree of semantic variability in the contexts in which a particular word is used. We have previously proposed a method for measuring semantic diversity based on latent semantic analysis (LSA). In a recent paper, Cevoli et al. (2020) attempted to replicate our method and obtained different semantic diversity values. They suggested that this discrepancy occurred because they scaled their LSA vectors by their singular values, while we did not. Using their new results, they argued that semantic diversity is not related to ambiguity in word meaning, as we originally proposed. In this reply, we demonstrate that the use of unscaled vectors provides better fits to human semantic judgements than scaled ones. Thus we argue that our original semantic diversity measure should be preferred over the Cevoli et al. version. We replicate Cevoli et al.'s analysis using the original semantic diversity measure and find (a) our original measure is a better predictor of word recognition latencies than the Cevoli et al. equivalent and (b) that, unlike Cevoli et al.'s measure, our semantic diversity is reliably associated with a measure of polysemy based on dictionary definitions. We conclude that the Hoffman et al. semantic diversity measure is better-suited to capturing the contextual variability among words and that words appearing in a more diverse set of contexts have more variable semantic representations. However, we found that homonyms did not have higher semantic diversity values than non-homonyms, suggesting that the measure does not capture this special case of ambiguity.

Beyond the Benchmarks: Toward Human-Like Lexical Representations

To process language in a way that is compatible with human expectations in a communicative interaction, we need computational representations of lexical properties that form the basis of human knowledge of words. In this article, we concentrate on word-level semantics. We discuss key concepts and issues that underlie the scientific understanding of the human lexicon: its richly structured semantic representations, their ready and continual adaptability, and their grounding in crosslinguistically valid conceptualization. We assess the state of the art in natural language processing (NLP) in achieving these identified properties, and suggest ways in which the language sciences can inspire new approaches to their computational instantiation.

Verb Metaphoric Extension Under Semantic Strain

This paper explores the processes underlying verb metaphoric extension. Work on metaphor processing has largely focused on noun metaphor, despite evidence that verb metaphor is more common. Across three experiments, we collected paraphrases of simple intransitive sentences varying in semantic strain—for example, The motor complained → The engine made strange noises—and assessed the degree of meaning change for the noun and the verb. We developed a novel methodology for this assessment using word2vec. In Experiments 1 and 2, we found that (a) under semantic strain, verb meanings were more likely to be adjusted than noun meanings; (b) the degree of verb meaning adjustment—but not noun meaning adjustment—increased with semantic strain; and (c) verb meaning extension is primarily driven by online adjustment, although sense selection also plays a role. In Experiment 3, we replicated the word2vec results with an assessment using human subjects. The results further showed that nouns and verbs change meaning in qualitatively different ways, with verbs more likely to change meaning metaphorically and nouns more likely to change meaning taxonomically or metonymically. These findings bear on the origin and processing of verb metaphors and provide a link between online sentence processing and diachronic change over language evolution.

Semantic projection recovers rich human knowledge of multiple object features from word embeddings

How is knowledge about word meaning represented in the mental lexicon? Current computational models infer word meanings from lexical co-occurrence patterns. They learn to represent words as vectors in a multidimensional space, wherein words that are used in more similar linguistic contexts-that is, are more semantically related-are located closer together. However, whereas inter-word proximity captures only overall relatedness, human judgements are highly context dependent. For example, dolphins and alligators are similar in size but differ in dangerousness. Here, we use a domain-general method to extract context-dependent relationships from word embeddings: 'semantic projection' of word-vectors onto lines that represent features such as size (the line connecting the words 'small' and 'big') or danger ('safe' to 'dangerous'), analogous to 'mental scales'. This method recovers human judgements across various object categories and properties. Thus, the geometry of word embeddings explicitly represents a wealth of context-dependent world knowledge.

Neural decoding of semantic concepts: A systematic literature review

Objective Semantic concepts are coherent entities within our minds. They underpin our thought processes and are a part of the basis for our understanding of the world. Modern neuroscience research is increasingly exploring how individual semantic concepts are encoded within our brains and a number of studies are beginning to reveal key patterns of neural activity that underpin specific concepts. Building upon this basic understanding of the process of semantic neural encoding, neural engineers are beginning to explore tools and methods for semantic decoding: identifying which semantic concepts an individual is focused on at a given moment in time from recordings of their neural activity. In this paper we review the current literature on semantic neural decoding. Approach We conducted this review according to the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. Specifically, we assess the eligibility of published peer-reviewed reports via a search of PubMed and Google Scholar. We identify a total of 74 studies in which semantic neural decoding is used to attempt to identify individual semantic concepts from neural activity. Results Our review reveals how modern neuroscientific tools have been developed to allow decoding of individual concepts from a range of neuroimaging modalities. We discuss specific neuroimaging methods, experimental designs, and machine learning pipelines that are employed to aid the decoding of semantic concepts. We quantify the efficacy of semantic decoders by measuring information transfer rates. We also discuss current challenges presented by this research area and present some possible solutions. Finally, we discuss some possible emerging and speculative future directions for this research area. Significance Semantic decoding is a rapidly growing area of research. However, despite its increasingly widespread popularity and use in neuroscientific research this is the first literature review focusing on this topic across neuroimaging modalities and with a focus on quantifying the efficacy of semantic decoders.

Patterns in CAOSS: Distributed representations predict variation in relational interpretations for familiar and novel compound words

While distributional semantic models that represent word meanings as high-dimensional vectors induced from large text corpora have been shown to successfully predict human behavior across a wide range of tasks, they have also received criticism from different directions. These include concerns over their interpretability (how can numbers specifying abstract, latent dimensions represent meaning?) and their ability to capture variation in meaning (how can a single vector representation capture multiple different interpretations for the same expression?). Here, we demonstrate that semantic vectors can indeed rise up to these challenges, by training a mapping system (a simple linear regression) that predicts inter-individual variation in relational interpretations for compounds such as wood brush (for example brush FOR wood, or brush MADE OF wood) from (compositional) semantic vectors representing the meanings of these compounds. These predictions consistently beat different random baselines, both for familiar compounds (moon light, Experiment 1) as well as novel compounds (wood brush, Experiment 2), demonstrating that distributional semantic vectors encode variations in qualitative interpretations that can be decoded using techniques as simple as linear regression.

Voluntary control of semantic neural representations by imagery with conflicting visual stimulation

Neural representations of visual perception are affected by mental imagery and attention. Although attention is known to modulate neural representations, it is unknown how imagery changes neural representations when imagined and perceived images semantically conflict. We hypothesized that imagining an image would activate a neural representation during its perception even while watching a conflicting image. To test this hypothesis, we developed a closed-loop system to show images inferred from electrocorticograms using a visual semantic space. The successful control of the feedback images demonstrated that the semantic vector inferred from electrocorticograms became closer to the vector of the imagined category, even while watching images from different categories. Moreover, modulation of the inferred vectors by mental imagery depended asymmetrically on the perceived and imagined categories. Shared neural representation between mental imagery and perception was still activated by the imagery under semantically conflicting perceptions depending on the semantic category.

In this study, intracranial EEG recordings show that neural representations of imagined images can still be present in humans even when they are shown conflicting images.

Computational Methods for Predicting and Understanding Food Judgment

People make subjective judgments about the healthiness of different foods every day, and these judgments in turn influence their food choices and health outcomes. Despite the importance of such judgments, there are few quantitative theories about their psychological underpinnings. This article introduces a novel computational approach that can approximate people's knowledge representations for thousands of common foods. We used these representations to predict how both lay decision-makers (the general population) and experts judge the healthiness of individual foods. We also applied our method to predict the impact of behavioral interventions, such as the provision of front-of-pack nutrient and calorie information. Across multiple studies with data from 846 adults, our models achieved very high accuracy rates (r² = .65-.77) and significantly outperformed competing models based on factual nutritional content. These results illustrate how new computational methods applied to established psychological theory can be used to better predict, understand, and influence health behavior.

Context Matters: Recovering Human Semantic Structure from Machine Learning Analysis of Large-Scale Text Corpora

Applying machine learning algorithms to automatically infer relationships between concepts from large‐scale collections of documents presents a unique opportunity to investigate at scale how human semantic knowledge is organized, how people use it to make fundamental judgments (“How similar are cats and bears?”), and how these judgments depend on the features that describe concepts (e.g., size, furriness). However, efforts to date have exhibited a substantial discrepancy between algorithm predictions and human empirical judgments. Here, we introduce a novel approach to generating embeddings for this purpose motivated by the idea that semantic context plays a critical role in human judgment. We leverage this idea by constraining the topic or domain from which documents used for generating embeddings are drawn (e.g., referring to the natural world vs. transportation apparatus). Specifically, we trained state‐of‐the‐art machine learning algorithms using contextually‐constrained text corpora (domain‐specific subsets of Wikipedia articles, 50+ million words each) and showed that this procedure greatly improved predictions of empirical similarity judgments and feature ratings of contextually relevant concepts. Furthermore, we describe a novel, computationally tractable method for improving predictions of contextually‐unconstrained embedding models based on dimensionality reduction of their internal representation to a small number of contextually relevant semantic features. By improving the correspondence between predictions derived automatically by machine learning methods using vast amounts of data and more limited, but direct empirical measurements of human judgments, our approach may help leverage the availability of online corpora to better understand the structure of human semantic representations and how people make judgments based on those.

Decoding the information structure underlying the neural representation of concepts

The ability to identify individual objects or events as members of a kind (e.g., “knife,” “dog,” or “party”) is a fundamental aspect of human cognition. It allows us to quickly access a wealth of information pertaining to a newly encountered object or event and use it to guide our behavior. How is this information represented in the brain? We used functional MRI to analyze patterns of brain activity corresponding to hundreds of familiar concepts and quantitatively characterized the informational structure of these patterns. Our results indicate that conceptual knowledge is stored as patterns of neural activity that encode sensory-motor and affective information about each concept, contrary to the long-held idea that concept representations are independent of sensory-motor experience.

The nature of the representational code underlying conceptual knowledge remains a major unsolved problem in cognitive neuroscience. We assessed the extent to which different representational systems contribute to the instantiation of lexical concepts in high-level, heteromodal cortical areas previously associated with semantic cognition. We found that lexical semantic information can be reliably decoded from a wide range of heteromodal cortical areas in the frontal, parietal, and temporal cortex. In most of these areas, we found a striking advantage for experience-based representational structures (i.e., encoding information about sensory-motor, affective, and other features of phenomenal experience), with little evidence for independent taxonomic or distributional organization. These results were found independently for object and event concepts. Our findings indicate that concept representations in the heteromodal cortex are based, at least in part, on experiential information. They also reveal that, in most heteromodal areas, event concepts have more heterogeneous representations (i.e., they are more easily decodable) than object concepts and that other areas beyond the traditional “semantic hubs” contribute to semantic cognition, particularly the posterior cingulate gyrus and the precuneus.

A tale of two gradients: differences between the left and right hemispheres predict semantic cognition

Decomposition of whole-brain functional connectivity patterns reveals a principal gradient that captures the separation of sensorimotor cortex from heteromodal regions in the default mode network (DMN). Functional homotopy is strongest in sensorimotor areas, and weakest in heteromodal cortices, suggesting there may be differences between the left and right hemispheres (LH/RH) in the principal gradient, especially towards its apex. This study characterised hemispheric differences in the position of large-scale cortical networks along the principal gradient, and their functional significance. We collected resting-state fMRI and semantic, working memory and non-verbal reasoning performance in 175 + healthy volunteers. We then extracted the principal gradient of connectivity for each participant, tested which networks showed significant hemispheric differences on the gradient, and regressed participants’ behavioural efficiency in tasks outside the scanner against interhemispheric gradient differences for each network. LH showed a higher overall principal gradient value, consistent with its role in heteromodal semantic cognition. One frontotemporal control subnetwork was linked to individual differences in semantic cognition: when it was nearer heteromodal DMN on the principal gradient in LH, participants showed more efficient semantic retrieval—and this network also showed a strong hemispheric difference in response to semantic demands but not working memory load in a separate study. In contrast, when a dorsal attention subnetwork was closer to the heteromodal end of the principal gradient in RH, participants showed better visual reasoning. Lateralization of function may reflect differences in connectivity between control and heteromodal regions in LH, and attention and visual regions in RH.

Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison

Similarity is one of the most important relations humans perceive, arguably subserving category learning and categorization, generalization and discrimination, judgment and decision making, and other cognitive functions. Researchers have proposed a wide range of representations and metrics that could be at play in similarity judgment, yet have not comprehensively compared the power of these representations and metrics for predicting similarity within and across different semantic categories. We performed such a comparison by pairing nine prominent vector semantic representations with seven established similarity metrics that could operate on these representations, as well as supervised methods for dimensional weighting in the similarity function. This approach yields a factorial model structure with 126 distinct representation-metric pairs, which we tested on a novel dataset of similarity judgments between pairs of cohyponymic words in eight categories. We found that cosine similarity and Pearson correlation were the overall best performing unweighted similarity functions, and that word vectors derived from free association norms often outperformed word vectors derived from text (including those specialized for similarity). Importantly, models that used human similarity judgments to learn category-specific weights on dimensions yielded substantially better predictions than all unweighted approaches across all types of similarity functions and representations, although dimension weights did not generalize well across semantic categories, suggesting strong category context effects in similarity judgment. We discuss implications of these results for cognitive modeling and natural language processing, as well as for theories of the representations and metrics involved in similarity.

Distinct and common neural coding of semantic and non-semantic control demands

The flexible retrieval of knowledge is critical in everyday situations involving problem solving, reasoning and social interaction. Current theories emphasise the importance of a left-lateralised semantic control network (SCN) in supporting flexible semantic behaviour, while a bilateral multiple-demand network (MDN) is implicated in executive functions across domains. No study, however, has examined whether semantic and non-semantic demands are reflected in a common neural code within regions specifically implicated in semantic control. Using functional MRI and univariate parametric modulation analysis as well as multivariate pattern analysis, we found that semantic and non-semantic demands gave rise to both similar and distinct neural responses across control-related networks. Though activity patterns in SCN and MDN could decode the difficulty of both semantic and verbal working memory decisions, there was no shared common neural coding of cognitive demands in SCN regions. In contrast, regions in MDN showed common patterns across manipulations of semantic and working memory control demands, with successful cross-classification of difficulty across tasks. Therefore, SCN and MDN can be dissociated according to the information they maintain about cognitive demands.

Behavioral correlates of cortical semantic representations modeled by word vectors

The quantitative modeling of semantic representations in the brain plays a key role in understanding the neural basis of semantic processing. Previous studies have demonstrated that word vectors, which were originally developed for use in the field of natural language processing, provide a powerful tool for such quantitative modeling. However, whether semantic representations in the brain revealed by the word vector-based models actually capture our perception of semantic information remains unclear, as there has been no study explicitly examining the behavioral correlates of the modeled brain semantic representations. To address this issue, we compared the semantic structure of nouns and adjectives in the brain estimated from word vector-based brain models with that evaluated from human behavior. The brain models were constructed using voxelwise modeling to predict the functional magnetic resonance imaging (fMRI) response to natural movies from semantic contents in each movie scene through a word vector space. The semantic dissimilarity of brain word representations was then evaluated using the brain models. Meanwhile, data on human behavior reflecting the perception of semantic dissimilarity between words were collected in psychological experiments. We found a significant correlation between brain model- and behavior-derived semantic dissimilarities of words. This finding suggests that semantic representations in the brain modeled via word vectors appropriately capture our perception of word meanings.

Judgment errors in naturalistic numerical estimation

People estimate numerical quantities (such as the calories of foods) on a day-to-day basis. Although these estimates influence behavior and determine wellbeing, they are prone to two important types of errors. Scaling errors occur when people make mistakes reporting their beliefs about a particular numerical quantity (e.g. by inflating small numbers). Belief errors occur when people make mistakes using their knowledge of the judgment target to form their beliefs about the numerical quantity (e.g. by overweighting certain cues). In this paper, we quantitatively model numerical estimates, and in turn, scaling and belief errors, in everyday judgment tasks. Our approach is unique in using insights from semantic memory research to specify knowledge for naturalistic judgment targets, allowing our models to formally describe nuanced errors in belief not considered in prior research. In Studies 1 and 2, we find that belief error models predict participant estimates and errors with very high out-of-sample accuracy rates, significantly outperforming the predictions of scaling error models. In fact, the best-fitting belief error models can closely mimic the inverse-S shaped patterns captured by scaling error models, suggesting that the types of responses previously attributed to scaling errors can be seen as errors of belief. In Studies 3 to 8, we find that belief error models are also able to predict people's responses in semantic judgment, free association, and verbal protocol tasks related to numerical judgment, and thus provide a good account of the cognitive underpinnings of judgment.

Probing Lexical Ambiguity: Word Vectors Encode Number and Relatedness of Senses

Lexical ambiguity-the phenomenon of a single word having multiple, distinguishable senses-is pervasive in language. Both the degree of ambiguity of a word (roughly, its number of senses) and the relatedness of those senses have been found to have widespread effects on language acquisition and processing. Recently, distributional approaches to semantics, in which a word's meaning is determined by its contexts, have led to successful research quantifying the degree of ambiguity, but these measures have not distinguished between the ambiguity of words with multiple related senses versus multiple unrelated meanings. In this work, we present the first assessment of whether distributional meaning representations can capture the ambiguity structure of a word, including both the number and relatedness of senses. On a very large sample of English words, we find that some, but not all, distributional semantic representations that we test exhibit detectable differences between sets of monosemes (unambiguous words; N = 964), polysemes (with multiple related senses; N = 4,096), and homonyms (with multiple unrelated senses; N = 355). Our findings begin to answer open questions from earlier work regarding whether distributional semantic representations of words, which successfully capture various semantic relationships, also reflect fine-grained aspects of meaning structure that influence human behavior. Our findings emphasize the importance of measuring whether proposed lexical representations capture such distinctions: In addition to standard benchmarks that test the similarity structure of distributional semantic models, we need to also consider whether they have cognitively plausible ambiguity structure.

Knowing what you need to know in advance: The neural processes underpinning flexible semantic retrieval of thematic and taxonomic relations

Semantic retrieval is flexible, allowing us to focus on subsets of features and associations that are relevant to the current task or context: for example, we use taxonomic relations to locate items in the supermarket (carrots are a vegetable), but thematic associations to decide which tools we need when cooking (carrot goes with peeler). We used fMRI to investigate the neural basis of this form of semantic flexibility; in particular, we asked how retrieval unfolds differently when participants have advanced knowledge of the type of link to retrieve between concepts (taxonomic or thematic). Participants performed a semantic relatedness judgement task: on half the trials, they were cued to search for a taxonomic or thematic link, while on the remaining trials, they judged relatedness without knowing which type of semantic relationship would be relevant. Left inferior frontal gyrus showed greater activation when participants knew the trial type in advance. An overlapping region showed a stronger response when the semantic relationship between the items was weaker, suggesting this structure supports both top-down and bottom-up forms of semantic control. Multivariate pattern analysis further revealed that the neural response in left inferior frontal gyrus reflects goal information related to different conceptual relationships. Top-down control specifically modulated the response in visual cortex: when the goal was unknown, there was greater deactivation to the first word, and greater activation to the second word. We conclude that top-down control of semantic retrieval is primarily achieved through the gating of task-relevant 'spoke' regions.

Analyzing Machine-Learned Representations: A Natural Language Case Study

As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances-similar to human zero-shot reasoning. However, we also note some shortcomings in this generalization behavior-similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human-like language understanding.

Images of the unseen: extrapolating visual representations for abstract and concrete words in a data-driven computational model

Theories of grounded cognition assume that conceptual representations are grounded in sensorimotor experience. However, abstract concepts such as jealousy or childhood have no directly associated referents with which such sensorimotor experience can be made; therefore, the grounding of abstract concepts has long been a topic of debate. Here, we propose (a) that systematic relations exist between semantic representations learned from language on the one hand and perceptual experience on the other hand, (b) that these relations can be learned in a bottom-up fashion, and (c) that it is possible to extrapolate from this learning experience to predict expected perceptual representations for words even where direct experience is missing. To test this, we implement a data-driven computational model that is trained to map language-based representations (obtained from text corpora, representing language experience) onto vision-based representations (obtained from an image database, representing perceptual experience), and apply its mapping function onto language-based representations for abstract and concrete words outside the training set. In three experiments, we present participants with these words, accompanied by two images: the image predicted by the model and a random control image. Results show that participants' judgements were in line with model predictions even for the most abstract words. This preference was stronger for more concrete items and decreased for the more abstract ones. Taken together, our findings have substantial implications in support of the grounding of abstract words, suggesting that we can tap into our previous experience to create possible visual representation we don't have.

Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals

A class of semantic theories defines concepts in terms of statistical distributions of lexical items, basing meaning on vectors of word co-occurrence frequencies. A different approach emphasizes abstract hierarchical taxonomic relationships among concepts. However, the functional relevance of these different accounts and how they capture information-encoding of lexical meaning in the brain still remains elusive. We investigated to what extent distributional and taxonomic models explained word-elicited neural responses using cross-validated representational similarity analysis (RSA) of functional magnetic resonance imaging (fMRI) and model comparisons. Our findings show that the brain encodes both types of semantic information, but in distinct cortical regions. Posterior middle temporal regions reflected lexical-semantic similarity based on hierarchical taxonomies, in coherence with the action-relatedness of specific semantic word categories. In contrast, distributional semantics best predicted the representational patterns in left inferior frontal gyrus (LIFG, BA 47). Both representations coexisted in the angular gyrus supporting semantic binding and integration. These results reveal that neuronal networks with distinct cortical distributions across higher-order association cortex encode different representational properties of word meanings. Taxonomy may shape long-term lexical-semantic representations in memory consistently with the sensorimotor details of semantic categories, whilst distributional knowledge in the LIFG (BA 47) may enable semantic combinatorics in the context of language use. Our approach helps to elucidate the nature of semantic representations essential for understanding human language.

Neural Components of Reading Revealed by Distributed and Symbolic Computational Models

Determining how the cognitive components of reading - orthographic, phonological, and semantic representations - are instantiated in the brain has been a longstanding goal of psychology and human cognitive neuroscience. The two most prominent computational models of reading instantiate different cognitive processes, implying different neural processes. Artificial neural network (ANN) models of reading posit non-symbolic, distributed representations. The dual-route cascaded (DRC) model instead suggests two routes of processing, one representing symbolic rules of spelling-sound correspondence, the other representing orthographic and phonological lexicons. These models are not adjudicated by behavioral data and have never before been directly compared in terms of neural plausibility. We used representational similarity analysis to compare the predictions of these models to neural data from participants reading aloud. Both the ANN and DRC model representations corresponded with neural activity. However, ANN model representations correlated to more reading-relevant areas of cortex. When contributions from the DRC model were statistically controlled, partial correlations revealed that the ANN model accounted for significant variance in the neural data. The opposite analysis, examining the variance explained by the DRC model with contributions from the ANN model factored out, revealed no correspondence to neural activity. Our results suggest that ANNs trained using distributed representations provide a better correspondence between cognitive and neural coding. Additionally, this framework provides a principled approach for comparing computational models of cognitive function to gain insight into neural representations.

subs2vec: Word embeddings from subtitles in 55 languages

This paper introduces a novel collection of word embeddings, numerical representations of lexical semantics, in 55 languages, trained on a large corpus of pseudo-conversational speech transcriptions from television shows and movies. The embeddings were trained on the OpenSubtitles corpus using the fastText implementation of the skipgram algorithm. Performance comparable with (and in some cases exceeding) embeddings trained on non-conversational (Wikipedia) text is reported on standard benchmark evaluation datasets. A novel evaluation method of particular relevance to psycholinguists is also introduced: prediction of experimental lexical norms in multiple languages. The models, as well as code for reproducing the models and all analyses reported in this paper (implemented as a user-friendly Python package), are freely available at: https://github.com/jvparidon/subs2vec.

Conceptual relations predict colexification across languages

In natural language, multiple meanings often share a single word form, a phenomenon known as colexification. Some sets of meanings are more frequently colexified across languages than others, but the source of this variation is not well understood. We propose that cross-linguistic variation in colexification frequency is non-arbitrary and reflects a general principle of cognitive economy: More commonly colexified meanings across languages are those that require less cognitive effort to relate. To evaluate our proposal, we examine patterns of colexification of varying frequency from about 250 languages. We predict these colexification data based on independent measures of conceptual relatedness drawn from large-scale psychological and linguistic resources. Our results show that meanings that are more frequently colexified across these languages tend to be more strongly associated by speakers of English, suggesting that conceptual associativity provides an important constraint on the development of the lexicon. Our work extends research on polysemy and the evolution of word meanings by grounding cross-linguistic regularities in colexification in basic principles of human cognition.

Connecting concepts in the brain by mapping cortical representations of semantic relations

In the brain, the semantic system is thought to store concepts. However, little is known about how it connects different concepts and infers semantic relations. To address this question, we collected hours of functional magnetic resonance imaging data from human subjects listening to natural stories. We developed a predictive model of the voxel-wise response and further applied it to thousands of new words. Our results suggest that both semantic categories and relations are represented by spatially overlapping cortical patterns, instead of anatomically segregated regions. Semantic relations that reflect conceptual progression from concreteness to abstractness are represented by cortical patterns of activation in the default mode network and deactivation in the frontoparietal attention network. We conclude that the human brain uses distributed networks to encode not only concepts but also relationships between concepts. In particular, the default mode network plays a central role in semantic processing for abstraction of concepts.