A Complex Network Approach to Distributional Semantic Models

Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.

Feature-rich multiplex lexical networks reveal mental strategies of early language learning

Knowledge in the human mind exhibits a dualistic vector/network nature. Modelling words as vectors is key to natural language processing, whereas networks of word associations can map the nature of semantic memory. We reconcile these paradigms-fragmented across linguistics, psychology and computer science-by introducing FEature-Rich MUltiplex LEXical (FERMULEX) networks. This novel framework merges structural similarities in networks and vector features of words, which can be combined or explored independently. Similarities model heterogenous word associations across semantic/syntactic/phonological aspects of knowledge. Words are enriched with multi-dimensional feature embeddings including frequency, age of acquisition, length and polysemy. These aspects enable unprecedented explorations of cognitive knowledge. Through CHILDES data, we use FERMULEX networks to model normative language acquisition by 1000 toddlers between 18 and 30 months. Similarities and embeddings capture word homophily via conformity, which measures assortative mixing via distance and features. Conformity unearths a language kernel of frequent/polysemous/short nouns and verbs key for basic sentence production, supporting recent evidence of children's syntactic constructs emerging at 30 months. This kernel is invisible to network core-detection and feature-only clustering: It emerges from the dual vector/network nature of words. Our quantitative analysis reveals two key strategies in early word learning. Modelling word acquisition as random walks on FERMULEX topology, we highlight non-uniform filling of communicative developmental inventories (CDIs). Biased random walkers lead to accurate (75%), precise (55%) and partially well-recalled (34%) predictions of early word learning in CDIs, providing quantitative support to previous empirical findings and developmental theories.

Brands, networks, communities: How brand names are wired in the mind

Brands can be defined as psychological constructs residing in our minds. By analyzing brand associations, we can study the mental constructs around them. In this paper, we study brands as parts of an associative network based on a word association database. We explore the communities–closely-knit groups in the mind–around brand names in this structure using two community detection algorithms in the Hungarian word association database ConnectYourMind. We identify brand names inside the communities of a word association network and explain why these brand names are part of the community. Several detected communities contain brand names from the same product category, and the words in these categories were connected either to brands in the category or to words describing the product category. Based on our findings, we describe the mental position of brand names. We show that brand knowledge, product knowledge and real word knowledge interact with each other. We also show how the meaning of a product category arises and how this meaning is related to brand meaning. Our results suggest that words sharing the same community with brand names can be used in brand communication and brand positioning.

Exploring What Is Encoded in Distributional Word Vectors: A Neurobiologically Motivated Analysis

The pervasive use of distributional semantic models or word embeddings for both cognitive modeling and practical application is because of their remarkable ability to represent the meanings of words. However, relatively little effort has been made to explore what types of information are encoded in distributional word vectors. Knowing the internal knowledge embedded in word vectors is important for cognitive modeling using distributional semantic models. Therefore, in this paper, we attempt to identify the knowledge encoded in word vectors by conducting a computational experiment using Binder et al.'s (2016) featural conceptual representations based on neurobiologically motivated attributes. In an experiment, these conceptual vectors are predicted from text-based word vectors using a neural network and linear transformation, and prediction performance is compared among various types of information. The analysis demonstrates that abstract information is generally predicted more accurately by word vectors than perceptual and spatiotemporal information, and specifically, the prediction accuracy of cognitive and social information is higher. Emotional information is also found to be successfully predicted for abstract words. These results indicate that language can be a major source of knowledge about abstract attributes, and they support the recent view that emphasizes the importance of language for abstract concepts. Furthermore, we show that word vectors can capture some types of perceptual and spatiotemporal information about concrete concepts and some relevant word categories. This suggests that language statistics can encode more perceptual knowledge than often expected.

Structure of communities in semantic networks of biomedical research on disparities in health and sexism

Introducción.

Como una iniciativa para mejorar la calidad de la atención sanitaria, en la investigación biomédica se ha incrementado la tendencia centrada en el estudio de las disparidades en salud y sexismo.

Objetivo.

Caracterizar la evidencia científica sobre la disparidad en salud definida como la brecha existente entre la distribución de la salud y el posible sesgo por sexo en el acceso a los servicios médicos.

Materiales y métodos.

Se hizo una búsqueda simultánea de la literatura científica en la base de datos Medline PubMed de dos descriptores fundamentales: Healthcare disparities y Sexism. Posteriormente, se construyó una red semántica principal y se determinaron algunas subunidades estructurales (comunidades) para el análisis de los patrones de organización de la información. Se utilizó el programa de código abierto Cytoscape para el analisis y la visualización de las redes y el MapEquation, para la detección de comunidades. Asimismo, se desarrolló código ex profeso disponible en un repositorio de acceso público.

Resultados.

El corpus de la red principal mostró que los términos sobre las enfermedades del corazón fueron los descriptores de condiciones médicas más concurrentes. A partir de las subunidades estructurales, se determinaron los patrones de información relacionada con las políticas públicas, los servicios de salud, los factores sociales determinantes y los factores de riesgo, pero con cierta tendencia a mantenerse indirectamente conectados con los nodos relacionados con condiciones médicas.

Conclusiones.

La evidencia científica indica que la disparidad por sexo sí importa para la calidad de la atención de muchas enfermedades, especialmente aquellas relacionadas con el sistema circulatorio. Sin embargo, aún se percibe un distanciamiento entre los factores médicos y los sociales que dan lugar a las posibles disparidades por sexo.

Architecture and evolution of semantic networks in mathematics texts

Knowledge is a network of interconnected concepts. Yet, precisely how the topological structure of knowledge constrains its acquisition remains unknown, hampering the development of learning enhancement strategies. Here, we study the topological structure of semantic networks reflecting mathematical concepts and their relations in college-level linear algebra texts. We hypothesize that these networks will exhibit structural order, reflecting the logical sequence of topics that ensures accessibility. We find that the networks exhibit strong core–periphery architecture, where a dense core of concepts presented early is complemented with a sparse periphery presented evenly throughout the exposition; the latter is composed of many small modules each reflecting more narrow domains. Using tools from applied topology, we find that the expositional evolution of the semantic networks produces and subsequently fills knowledge gaps, and that the density of these gaps tracks negatively with community ratings of each textbook. Broadly, our study lays the groundwork for future efforts developing optimal design principles for textbook exposition and teaching in a classroom setting.

Modeling the Structure and Dynamics of Semantic Processing

The contents and structure of semantic memory have been the focus of much recent research, with major advances in the development of distributional models, which use word co-occurrence information as a window into the semantics of language. In parallel, connectionist modeling has extended our knowledge of the processes engaged in semantic activation. However, these two lines of investigation have rarely been brought together. Here, we describe a processing model based on distributional semantics in which activation spreads throughout a semantic network, as dictated by the patterns of semantic similarity between words. We show that the activation profile of the network, measured at various time points, can successfully account for response times in lexical and semantic decision tasks, as well as for subjective concreteness and imageability ratings. We also show that the dynamics of the network is predictive of performance in relational semantic tasks, such as similarity/relatedness rating. Our results indicate that bringing together distributional semantic networks and spreading of activation provides a good fit to both automatic lexical processing (as indexed by lexical and semantic decisions) as well as more deliberate processing (as indexed by ratings), above and beyond what has been reported for previous models that take into account only similarity resulting from network structure.

Local Patterns to Global Architectures: Influences of Network Topology on Human Learning

A core question in cognitive science concerns how humans acquire and represent knowledge about their environments. To this end, quantitative theories of learning processes have been formalized in an attempt to explain and predict changes in brain and behavior. We connect here statistical learning approaches in cognitive science, which are rooted in the sensitivity of learners to local distributional regularities, and network science approaches to characterizing global patterns and their emergent properties. We focus on innovative work that describes how learning is influenced by the topological properties underlying sensory input. The confluence of these theoretical approaches and this recent empirical evidence motivate the importance of scaling-up quantitative approaches to learning at both the behavioral and neural levels.