Distributional social semantics: Inferring word meanings from communication patterns

Recruitment of magnitude representations to understand graded words

Language understanding and mathematics understanding are two fundamental forms of human thinking. Prior research has largely focused on the question of how language shapes mathematical thinking. The current study considers the converse question. Specifically, it investigates whether the magnitude representations that are thought to anchor understanding of number are also recruited to understand the meanings of graded words. These are words that come in scales (e.g., Anger) whose members can be ordered by the degree to which they possess the defining property (e.g., calm, annoyed, angry, furious). Experiment 1 uses the comparison paradigm to find evidence that the distance, ratio, and boundary effects that are taken as evidence of the recruitment of magnitude representations extend from numbers to words. Experiment 2 uses a similarity rating paradigm and multi-dimensional scaling to find converging evidence for these effects in graded word understanding. Experiment 3 evaluates an alternative hypothesis - that these effects for graded words simply reflect the statistical structure of the linguistic environment - by using machine learning models of distributional word semantics: LSA, word2vec, GloVe, counterfitted word vectors, BERT, RoBERTa, and GPT-2. These models fail to show the full pattern of effects observed of humans in Experiment 2, suggesting that more is needed than mere statistics. This research paves the way for further investigations of the role of magnitude representations in sentence and text comprehension, and of the question of whether language understanding and number understanding draw on shared or independent magnitude representations. It also informs the role of machine learning models in cognitive psychology research.

Mapping semantic space: Exploring the higher-order structure of word meaning

Multiple representation theories posit that concepts are represented via a combination of properties derived from sensorimotor, affective, and linguistic experiences. Recently, it has been proposed that information derived from social experience, or socialness, represents another key aspect of conceptual representation. How these various dimensions interact to form a coherent conceptual space has yet to be fully explored. To address this, we capitalized on openly available word property norms for 6339 words and conducted a large-scale investigation into the relationships between 18 dimensions. An exploratory factor analysis reduced the dimensions to six higher-order factors: sub-lexical, distributional, visuotactile, body action, affective and social interaction. All these factors explained unique variance in performance on lexical and semantic tasks, demonstrating that they make important contributions to the representation of word meaning. An important and novel finding was that the socialness dimension clustered with the auditory modality and with mouth and head actions. We suggest this reflects experiential learning from verbal interpersonal interactions. Moreover, formally modelling the network structure of semantic space revealed pairwise partial correlations between most dimensions and highlighted the centrality of the interoception dimension. Altogether, these findings provide new insights into the architecture of conceptual space, including the importance of inner and social experience, and highlight promising avenues for future research.

BERTs of a feather: Studying inter- and intra-group communication via information theory and language models

When communicating, individuals alter their language to fulfill a myriad of social functions. In particular, linguistic convergence and divergence are fundamental in establishing and maintaining group identity. Quantitatively characterizing linguistic convergence is important when testing hypotheses surrounding language, including interpersonal and group communication. We provide a quantitative interpretation of linguistic convergence grounded in information theory. We then construct a computational model, built on top of a neural network model of language, that can be deployed to measure and test hypotheses about linguistic convergence in "big data." We demonstrate the utility of our convergence measurement in two case studies: (1) showing that our measurement is indeed sensitive to linguistic convergence across turns in dyadic conversation, and (2) showing that our convergence measurement is sensitive to social factors that mediate convergence in Internet-based communities (specifically, r/MensRights and r/MensLib). Our measurement also captures differences in which social factors influence web-based communities. We conclude by discussing methodological and theoretical implications of this semantic convergence analysis.

Determining the Relativity of Word Meanings Through the Construction of Individualized Models of Semantic Memory

Distributional models of lexical semantics are capable of acquiring sophisticated representations of word meanings. The main theoretical insight provided by these models is that they demonstrate the systematic connection between the knowledge that people acquire and the experience that they have with the natural language environment. However, linguistic experience is inherently variable and differs radically across people due to demographic and cultural variables. Recently, distributional models have been used to examine how word meanings vary across languages and it was found that there is considerable variability in the meanings of words across languages for most semantic categories. The goal of this article is to examine how variable word meanings are across individual language users within a single language. This was accomplished by assembling 500 individual user corpora attained from the online forum Reddit. Each user corpus ranged between 3.8 and 32.3 million words each, and a count-based distributional framework was used to extract word meanings for each user. These representations were then used to estimate the semantic alignment of word meanings across individual language users. It was found that there are significant levels of relativity in word meanings across individuals, and these differences are partially explained by other psycholinguistic factors, such as concreteness, semantic diversity, and social aspects of language usage. These results point to word meanings being fundamentally relative and contextually fluid, with this relativeness being related to the individualized nature of linguistic experience.

Contextual dynamics in lexical encoding across the ageing spectrum: A simulation study

The field of psycholinguistics has recently questioned the primacy of word frequency (WF) in influencing word recognition and production, instead focusing on the importance of a word's contextual diversity (CD). WF is operationalised by counting the number of occurrences of a word in a corpus, while a word's CD is a count of the number of contexts that a word occurs in, with repetitions within a context being ignored. Numerous studies have converged on the conclusion that CD is a better predictor of word recognition latency and accuracy than frequency. These findings support a cognitive mechanism based on the principle of likely need over the principle of repetition in lexical organisation. In the current study, we trained the semantic distinctiveness model on communication patterns in social media platforms consisting of over 55-billion-word tokens and examined the ability of theoretically distinct models to explain word recognition latency and accuracy data from over 1 million participants from the Mandera et al. English Crowdsourding Project norms, consisting of approximately 59,000 words across six age bands ranging from ages 10 to 60 years. There was a clear quantitative trend across the age bands, where there is a shift from a social environment-based attention mechanism in the "younger" models, to a clear dominance for a discourse-based attention mechanism as models "aged." This pattern suggests that there is a dynamical interaction between the cognitive mechanisms of lexical organisation and environmental information that emerges across ageing.

Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations

Distributional semantic models (DSMs) are a primary method for distilling semantic information from corpora. However, a key question remains: What types of semantic relations among words do DSMs detect? Prior work typically has addressed this question using limited human data that are restricted to semantic similarity and/or general semantic relatedness. We tested eight DSMs that are popular in current cognitive and psycholinguistic research (positive pointwise mutual information; global vectors; and three variations each of Skip-gram and continuous bag of words (CBOW) using word, context, and mean embeddings) on a theoretically motivated, rich set of semantic relations involving words from multiple syntactic classes and spanning the abstract-concrete continuum (19 sets of ratings). We found that, overall, the DSMs are best at capturing overall semantic similarity and also can capture verb-noun thematic role relations and noun-noun event-based relations that play important roles in sentence comprehension. Interestingly, Skip-gram and CBOW performed the best in terms of capturing similarity, whereas GloVe dominated the thematic role and event-based relations. We discuss the theoretical and practical implications of our results, make recommendations for users of these models, and demonstrate significant differences in model performance on event-based relations.

Environment Context Variability and Incidental Word Learning: A Virtual Reality Study

Previous research has shown that changes in the scenarios in which something is learned and recalled, respectively, may result in a subpar performance in memory recollection. The current study aimed to evaluate how changes in the visuo-perceptual environmental learning context impact incidental vocabulary learning. To this end, a highly immersive virtual reality setting was created, and participants were required to read eight distinct stories visually presented to them. A novel word was delivered twice in every paragraph and embedded in each story. Stories could be displayed either in a high variability condition, where each paragraph was shown in a new environment context (four different classrooms) or in a low variability condition, where each paragraph was shown in the same context. The findings obtained across four assessment tasks (free recall, recognition, picture matching, and sentence completion) demonstrated that significant visuo-perceptual variability did not bring about any disadvantages in word learning. Thus, perceptual information from a physically diverse environment could provide a variety of instructional and educational beneficial possibilities in the absence of a learning disadvantage.