Structured Semantic Knowledge Can Emerge Automatically from Predicting Word Sequences in Child-Directed Speech

Wordform variability in infants' language environment and its effects on early word learning

Most research regarding early word learning in English tends to make the simplifying assumption that there exists a one-to-one mapping between concrete objects and their labels. In the current work, we provide evidence that runs counter to this assumption, aligning English with more morphologically-rich languages. We suggest that even in a morphologically-poor language like English, real world language input to infants does not provide tidy 1-to-1 mappings. Instead, infants encounter many variant wordforms for familiar nouns (e.g. dog∼doggy∼dogs). We explore this wordform variability in 44 English-learning infants' naturalistic environments using a longitudinal corpus of infant-available speech. We look at both the frequency and composition of wordform variability. We find two broad categories of variability: referent-changing alterations, where words were pluralized or compounded (e.g. coat∼raincoats); and wordplay, where words changed form without a notable change in referent (e.g. bird∼birdie). We further find that wordplay occurs with a limited number of lemmas that are usually early-learned, high-frequency, and shorter. When looking at all wordform variability, we find that individual words with higher levels of wordform variability are learned earlier than words with fewer wordforms, over and above the effect of frequency.

Large Language Models: A Historical and Sociocultural Perspective

This letter explores the intricate historical and contemporary links between large language models (LLMs) and cognitive science through the lens of information theory, statistical language models, and socioanthropological linguistic theories. The emergence of LLMs highlights the enduring significance of information-based and statistical learning theories in understanding human communication. These theories, initially proposed in the mid-20th century, offered a visionary framework for integrating computational science, social sciences, and humanities, which nonetheless was not fully fulfilled at that time. The subsequent development of sociolinguistics and linguistic anthropology, especially since the 1970s, provided critical perspectives and empirical methods that both challenged and enriched this framework. This letter proposes that two pivotal concepts derived from this development, metapragmatic function and indexicality, offer a fruitful theoretical perspective for integrating the semantic, textual, and pragmatic, contextual dimensions of communication, an amalgamation that contemporary LLMs have yet to fully achieve. The author believes that contemporary cognitive science is at a crucial crossroads, where fostering interdisciplinary dialogues among computational linguistics, social linguistics and linguistic anthropology, and cognitive and social psychology is in particular imperative. Such collaboration is vital to bridge the computational, cognitive, and sociocultural aspects of human communication and human-AI interaction, especially in the era of large language and multimodal models and human-centric Artificial Intelligence (AI).

Sampling statistics are like story creation: a network analysis of parent-toddler exploratory play

Actions in the world elicit data for learning and do so in a stream of interconnected events. Here, we provide evidence on how toddlers with their parent sample information by acting on toys during exploratory play. We observed 10 min of free-flowing and unconstrained object exploration of by toddlers (mean age 21 months) and parents in a room with many available objects (n = 32). Borrowing concepts and measures from the study of narratives, we found that the toy selections are not a string of unrelated events but exhibit a suite of what we call coherence statistics: Zipfian distributions, burstiness and a network structure. We discuss the transient memory processes that underlie the moment-to-moment toy selections that create this coherence and the role of these statistics in the development of abstract and generalizable systems of knowledge. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Experience and maturation: The contribution of co-occurrence regularities in language to the development of semantic organization

With development knowledge becomes organized according to semantic links, including early-developing associative (e.g., juicy-apple) and gradually developing taxonomic links (e.g., apple-pear). Word co-occurrence regularities may foster these links: Associative links may form from direct co-occurrence (e.g., juicy-apple), and taxonomic links from shared co-occurrence (e.g., apple and pear co-occur with juicy). Four experiments (2017-2020) investigated this possibility with 4- to 8-year-olds (N = 148, 82 female) and adults (N = 116, 35 female) in a U.S. city with 58.6% White; 29.0% Black, and 5.8% Asian demographics. Results revealed earlier development of the abilities to form direct (ds > 0.536) than the abilities to form shared co-occurrence-based links (ds > 1.291). We argue that the asynchronous development of abilities to form co-occurrence-based links may explain developmental changes in semantic organization.

Child-oriented word associations improve models of early word learning

How words are associated within the linguistic environment conveys semantic content; however, different contexts induce different linguistic patterns. For instance, it is well known that adults speak differently to children than to other adults. We present results from a new word association study in which adult participants were instructed to produce either unconstrained or child-oriented responses to each cue, where cues included 672 nouns, verbs, adjectives, and other word forms from the McArthur-Bates Communicative Development Inventory (CDI; Fenson et al., 2006). Child-oriented responses consisted of higher frequency words with fewer letters, earlier ages of acquisition, and higher contextual diversity. Furthermore, the correlations among the responses generated for each pair of cues differed between unconstrained (adult-oriented) and child-oriented responses, suggesting that child-oriented associations imply different semantic structure. A comparison of growth models guided by a semantic network structure revealed that child-oriented associations are more predictive of early lexical growth. Additionally, relative to a growth model based on a corpus of naturalistic child-directed speech, the child-oriented associations explain added unique variance to lexical growth. Thus, these new child-oriented word association norms provide novel insight into the semantic context of young children and early lexical development.

Exploring Patterns of Stability and Change in Caregivers' Word Usage Across Early Childhood

The linguistic input children receive across early childhood plays a crucial role in shaping their knowledge about the world. To study this input, researchers have begun applying distributional semantic models to large corpora of child-directed speech, extracting various patterns of word use/co-occurrence. Previous work using these models has not measured how these patterns may change throughout development, however. In this work, we leverage natural language processing methods-originally developed to study historical language change-to compare caregivers' use of words when talking to younger versus older children. Some words' usage changed more than others; this variability could be predicted based on the word's properties at both the individual and category levels. These findings suggest that caregivers' changing patterns of word use may play a role in scaffolding children's acquisition of conceptual structure in early development.

The map trap: Why and how word learning research should move beyond mapping

A pervasive goal in the study of how children learn word meanings is to explain how young children solve the mapping problem. The mapping problem asks how language learners connect a label to its referent. Mapping is one part of word learning, however, it does not reflect other critical components of word meaning construction, such as the encoding of lexico-semantic relations and socio-pragmatic context. In this paper, we argue that word learning researchers' overemphasis of mapping has constrained our experimental paradigms and hypotheses, leading to misconceived theories and policy interventions. We first explain how the mapping focus limits our ability to study the richness and complexity of what infants and children learn about, and do with, word meanings. Then, we describe how our focus on mapping has constrained theory development. Specifically, we show how it has led to (a) the misguided emphasis on referent selection and ostensive labeling, and (b) the undervaluing of diverse pathways to word knowledge, both within and across cultures. We also review the consequences of the mapping focus outside of the lab, including myopic language learning interventions. Last, we outline an alternative, more inclusive approach to experimental study and theory construction in word learning research. This article is categorized under: Psychology > Language Psychology > Theory and Methods Psychology > Learning.

Research on Emotion Analysis and Psychoanalysis Application With Convolutional Neural Network and Bidirectional Long Short-Term Memory

This study mainly focuses on the emotion analysis method in the application of psychoanalysis based on sentiment recognition. The method is applied to the sentiment recognition module in the server, and the sentiment recognition function is effectively realized through the improved convolutional neural network and bidirectional long short-term memory (C-BiL) model. First, the implementation difficulties of the C-BiL model and specific sentiment classification design are described. Then, the specific design process of the C-BiL model is introduced, and the innovation of the C-BiL model is indicated. Finally, the experimental results of the models are compared and analyzed. Among the deep learning models, the accuracy of the C-BiL model designed in this study is relatively high irrespective of the binary classification, the three classification, or the five classification, with an average improvement of 2.47% in Diary data set, 2.16% in Weibo data set, and 2.08% in Fudan data set. Therefore, the C-BiL model designed in this study can not only successfully classify texts but also effectively improve the accuracy of text sentiment recognition.

Semantic projection recovers rich human knowledge of multiple object features from word embeddings

How is knowledge about word meaning represented in the mental lexicon? Current computational models infer word meanings from lexical co-occurrence patterns. They learn to represent words as vectors in a multidimensional space, wherein words that are used in more similar linguistic contexts-that is, are more semantically related-are located closer together. However, whereas inter-word proximity captures only overall relatedness, human judgements are highly context dependent. For example, dolphins and alligators are similar in size but differ in dangerousness. Here, we use a domain-general method to extract context-dependent relationships from word embeddings: 'semantic projection' of word-vectors onto lines that represent features such as size (the line connecting the words 'small' and 'big') or danger ('safe' to 'dangerous'), analogous to 'mental scales'. This method recovers human judgements across various object categories and properties. Thus, the geometry of word embeddings explicitly represents a wealth of context-dependent world knowledge.

Computational Models of Readers' Apperceptive Mass

Recent progress in machine-learning-based distributed semantic models (DSMs) offers new ways to simulate the apperceptive mass (AM; Kintsch, 1980) of reader groups or individual readers and to predict their performance in reading-related tasks. The AM integrates the mental lexicon with world knowledge, as for example, acquired via reading books. Following pioneering work by Denhière and Lemaire (2004), here, we computed DSMs based on a representative corpus of German children and youth literature (Jacobs et al., 2020) as null models of the part of the AM that represents distributional semantic input, for readers of different reading ages (grades 1–2, 3–4, and 5–6). After a series of DSM quality tests, we evaluated the performance of these models quantitatively in various tasks to simulate the different reader groups' hypothetical semantic and syntactic skills. In a final study, we compared the models' performance with that of human adult and children readers in two rating tasks. Overall, the results show that with increasing reading age performance in practically all tasks becomes better. The approach taken in these studies reveals the limits of DSMs for simulating human AM and their potential for applications in scientific studies of literature, research in education, or developmental science.

Modelling brain representations of abstract concepts

Abstract conceptual representations are critical for human cognition. Despite their importance, key properties of these representations remain poorly understood. Here, we used computational models of distributional semantics to predict multivariate fMRI activity patterns during the activation and contextualization of abstract concepts. We devised a task in which participants had to embed abstract nouns into a story that they developed around a given background context. We found that representations in inferior parietal cortex were predicted by concept similarities emerging in models of distributional semantics. By constructing different model families, we reveal the models’ learning trajectories and delineate how abstract and concrete training materials contribute to the formation of brain-like representations. These results inform theories about the format and emergence of abstract conceptual representations in the human brain.

How do we conceive abstract concepts, like love, peace, or truth? In this study, we investigate how our brains support the activation and contextualization of such abstract concepts. We asked participants to embed abstract nouns into a coherent story while we recorded functional MRI. Using multivariate analysis techniques, we computed how similar different abstract concepts were represented during this task. We then modelled these neural similarities among concepts with computational models of distributional semantics which capture the words’ co-occurance statistics in large natural language corpora. Our results reveal a correspondence between the computational models and brain representations in the inferior parietal cortex. This correspondence held when the computational models were only trained on subsets of the corpora that contained as few as 100,000 sentences and only abstract or concrete words. Our findings establish a neural correlate of abstract concept representation in the inferior parietal cortex, and they provide a first characterization of the format of these representations.

We Know More Than We Ever Learned: Processes Involved in the Accumulation of World Knowledge

Accumulating information and knowledge is a major task of development. A common assumption is that we build our storehouse of world knowledge, our semantic memory, through direct experience. Although direct experience is involved, to explain fully how we know all that we know, we also must consider processes that allow for integration of information learned in separate yet related episodes of direct learning, as well as inferential processes that operate over integrated representations and permit productive extension of knowledge. In this article, I describe the self-derivation through integration paradigm my colleagues and I developed to model these processes. Using this paradigm, we charted individual and developmental variability throughout childhood and in young adults. Several findings support the contention that the self-derivation through integration paradigm provides a valid model for how we build semantic knowledge, including the observations that performance on the task correlates with and predicts individuals' world knowledge and academic success.

Exploring What Is Encoded in Distributional Word Vectors: A Neurobiologically Motivated Analysis

The pervasive use of distributional semantic models or word embeddings for both cognitive modeling and practical application is because of their remarkable ability to represent the meanings of words. However, relatively little effort has been made to explore what types of information are encoded in distributional word vectors. Knowing the internal knowledge embedded in word vectors is important for cognitive modeling using distributional semantic models. Therefore, in this paper, we attempt to identify the knowledge encoded in word vectors by conducting a computational experiment using Binder et al.'s (2016) featural conceptual representations based on neurobiologically motivated attributes. In an experiment, these conceptual vectors are predicted from text-based word vectors using a neural network and linear transformation, and prediction performance is compared among various types of information. The analysis demonstrates that abstract information is generally predicted more accurately by word vectors than perceptual and spatiotemporal information, and specifically, the prediction accuracy of cognitive and social information is higher. Emotional information is also found to be successfully predicted for abstract words. These results indicate that language can be a major source of knowledge about abstract attributes, and they support the recent view that emphasizes the importance of language for abstract concepts. Furthermore, we show that word vectors can capture some types of perceptual and spatiotemporal information about concrete concepts and some relevant word categories. This suggests that language statistics can encode more perceptual knowledge than often expected.

The Emergence of Richly Organized Semantic Knowledge from Simple Statistics: A Synthetic Review

As adults, we draw upon our ample knowledge about the world to support such vital cognitive feats as using language, reasoning, retrieving knowledge relevant to our current goals, planning for the future, adapting to unexpected events, and navigating through the environment. Our knowledge readily supports these feats because it is not merely a collection of stored facts, but rather functions as an organized, semantic network of concepts connected by meaningful relations. How do the relations that fundamentally organize semantic concepts emerge with development? Here, we cast a spotlight on a potentially powerful but often overlooked driver of semantic organization: Rich statistical regularities that are ubiquitous in both language and visual input. In this synthetic review, we show that a driving role for statistical regularities is convergently supported by evidence from diverse fields, including computational modeling, statistical learning, and semantic development. Finally, we identify a number of key avenues of future research into how statistical regularities may drive the development of semantic organization.

Avoiding Conflict: When Speaker Coordination Does Not Require Conceptual Agreement

In this paper we discuss the socialization hypothesis-the idea that speakers of the same (linguistic) community should share similar concepts given that they are exposed to similar environments and operate in highly-coordinated social contexts-and challenge the fact that it is assumed to constitute a prerequisite to successful communication. We do so using distributional semantic models of meaning (DSMs) which create lexical representations via latent aggregation of co-occurrence information between words and contexts. We argue that DSMs constitute particularly adequate tools for exploring the socialization hypothesis given that 1) they provide full control over the notion of background environment, formally characterized as the training corpus from which distributional information is aggregated; and 2) their geometric structure allows for exploiting alignment-based similarity metrics to measure inter-subject alignment over an entire semantic space, rather than a set of limited entries. We propose to model coordination between two different DSMs trained on two distinct corpora as dimensionality selection over a dense matrix obtained via Singular Value Decomposition This approximates an ad-hoc coordination scenario between two speakers as the attempt to align their similarity ratings on a set of word pairs. Our results underline the specific way in which linguistic information is spread across singular vectors, and highlight the need to distinguish agreement from mere compatibility in alignment-based notions of conceptual similarity. Indeed, we show that compatibility emerges from idiosyncrasy so that the unique and distinctive aspects of speakers' background experiences can actually facilitate-rather than impede-coordination and communication between them. We conclude that the socialization hypothesis may constitute an unnecessary prerequisite to successful communication and that, all things considered, communication is probably best formalized as the cooperative act of avoiding conflict, rather than maximizing agreement.

Adjacent and Non-Adjacent Word Contexts Both Predict Age of Acquisition of English Words: A Distributional Corpus Analysis of Child-Directed Speech

Children show a remarkable degree of consistency in learning some words earlier than others. What patterns of word usage predict variations among words in age of acquisition? We use distributional analysis of a naturalistic corpus of child-directed speech to create quantitative features representing natural variability in word contexts. We evaluate two sets of features: One set is generated from the distribution of words into frames defined by the two adjacent words. These features primarily encode syntactic aspects of word usage. The other set is generated from non-adjacent co-occurrences between words. These features encode complementary thematic aspects of word usage. Regression models using these distributional features to predict age of acquisition of 656 early-acquired English words indicate that both types of features improve predictions over simpler models based on frequency and appearance in salient or simple utterance contexts. Syntactic features were stronger predictors of children's production than comprehension, whereas thematic features were stronger predictors of comprehension. Overall, earlier acquisition was predicted by features representing frames that select for nouns and verbs, and by thematic content related to food and face-to-face play topics; later acquisition was predicted by features representing frames that select for pronouns and question words, and by content related to narratives and object play.

The Role of Co-Occurrence Statistics in Developing Semantic Knowledge

The organization of our knowledge about the world into an interconnected network of concepts linked by relations profoundly impacts many facets of cognition, including attention, memory retrieval, reasoning, and learning. It is therefore crucial to understand how organized semantic representations are acquired. The present experiment investigated the contributions of readily observable environmental statistical regularities to semantic organization in childhood. Specifically, we investigated whether co-occurrence regularities with which entities or their labels more reliably occur together than with others (a) contribute to relations between concepts independently and (b) contribute to relations between concepts belonging to the same taxonomic category. Using child-directed speech corpora to estimate reliable co-occurrences between labels for familiar items, we constructed triads consisting of a target, a related distractor, and an unrelated distractor in which targets and related distractors consistently co-occurred (e.g., sock-foot), belonged to the same taxonomic category (e.g., sock-coat), or both (e.g., sock-shoe). We used an implicit, eye-gaze measure of relations between concepts based on the degree to which children (N = 72, age 4-7 years) looked at related versus unrelated distractors when asked to look for a target. The results indicated that co-occurrence both independently contributes to relations between concepts and contributes to relations between concepts belonging to the same taxonomic category. These findings suggest that sensitivity to the regularity with which different entities co-occur in children's environments shapes the organization of semantic knowledge during development. Implications for theoretical accounts and empirical investigations of semantic organization are discussed.

Cultural influences on word meanings revealed through large-scale semantic alignment

If the structure of language vocabularies mirrors the structure of natural divisions that are universally perceived, then the meanings of words in different languages should closely align. By contrast, if shared word meanings are a product of shared culture, history and geography, they may differ between languages in substantial but predictable ways. Here, we analysed the semantic neighbourhoods of 1,010 meanings in 41 languages. The most-aligned words were from semantic domains with high internal structure (number, quantity and kinship). Words denoting natural kinds, common actions and artefacts aligned much less well. Languages that are more geographically proximate, more historically related and/or spoken by more-similar cultures had more aligned word meanings. These results provide evidence that the meanings of common words vary in ways that reflect the culture, history and geography of their users.

Experience-Driven Semantic Differentiation: Effects of a Naturalistic Experience on Within- and Across-Domain Differentiation in Children

Organized semantic networks reflecting distinctions within and across domains of knowledge are critical for higher-level cognition. Thus, understanding how semantic structure changes with experience is a fundamental question in developmental science. This study probed changes in semantic structure in 4-6 year-old children (N = 29) as a result of participating in an enrichment program at a local botanical garden. This study presents the first direct evidence that (a) the accumulation of experience with items in a domain promoted increases in both within- and across-domain semantic differentiation, and that (b) this experience-driven semantic differentiation generalized to nonexperienced items. These findings have implications for understanding the role of experience in building semantic networks, and for conceptualizing the contribution of enrichment experiences to academic success.

Statistical regularities shape semantic organization throughout development

Our knowledge about the world is represented not merely as a collection of concepts, but as an organized lexico-semantic network in which concepts can be linked by relations, such as "taxonomic" relations between members of the same stable category (e.g., cat and sheep), or association between entities that occur together or in the same context (e.g., sock and foot). To date, accounts of the origins of semantic organization have largely overlooked how sensitivity to statistical regularities ubiquitous in the environment may play a powerful role in shaping semantic development. The goal of the present research was to investigate how associations in the form of statistical regularities with which labels for concepts co-occur in language (e.g., sock and foot) and taxonomic relatedness (e.g., sock and pajamas) shape semantic organization of 4-5-year-olds and adults. To examine these aspects of semantic organization across development, we conducted three experiments examining effects of co-occurrence and taxonomic relatedness on cued recall (Experiment 1), word-picture matching (Experiment 2), and looking dynamics in a Visual World paradigm (Experiment 3). Taken together, the results of the three experiments provide evidence that co-occurrence-based links between concepts manifest in semantic organization from early childhood onward, and are increasingly supplemented by taxonomic links. We discuss these findings in relation to theories of semantic development.

Rapid, experience-related changes in the organization of children's semantic knowledge

The organization of knowledge according to relations between concepts is crucially important for many cognitive processes, and its emergence during childhood is a key focus of cognitive development research. Prior evidence about the role of learning and experience in the development of knowledge organization primarily comes from studies investigating differences between preexisting, naturally occurring groups (e.g., children from rural vs. urban settings, children who own a pet vs. children who do not) and a handful of studies on the effects of researcher-developed educational interventions. However, we know little about whether knowledge organization can be relatively rapidly molded by shorter-term real-world learning experiences (e.g., on a timescale of days vs. years or months). The current study investigated whether naturalistic learning experiences can drive rapid measurable changes in knowledge organization in children by investigating the effects of a week-long zoo summer camp (compared with a control school-based camp) on the degree to which 4- to 9-year-old children's knowledge about animals was organized according to taxonomic relations. Although there were no differences in taxonomic organization between the zoo camp and the school-based camp at pretest, only children who participated in the zoo camp showed increases in taxonomic organization at posttest. Moreover, analyses of changes in taxonomic organization in zoo camp children suggested that these changes were primarily driven by improvements in the degree to which children differentiated between taxonomic categories. These findings provide novel evidence that naturalistic experiences can drive rapid changes in knowledge organization.

Interaction in Spoken Word Recognition Models: Feedback Helps

Human perception, cognition, and action requires fast integration of bottom-up signals with top-down knowledge and context. A key theoretical perspective in cognitive science is the interactive activation hypothesis: forward and backward flow in bidirectionally connected neural networks allows humans and other biological systems to approximate optimal integration of bottom-up and top-down information under real-world constraints. An alternative view is that online feedback is neither necessary nor helpful; purely feed forward alternatives can be constructed for any feedback system, and online feedback could not improve processing and would preclude veridical perception. In the domain of spoken word recognition, the latter view was apparently supported by simulations using the interactive activation model, TRACE, with and without feedback: as many words were recognized more quickly without feedback as were recognized faster with feedback, However, these simulations used only a small set of words and did not address a primary motivation for interaction: making a model robust in noise. We conducted simulations using hundreds of words, and found that the majority were recognized more quickly with feedback than without. More importantly, as we added noise to inputs, accuracy and recognition times were better with feedback than without. We follow these simulations with a critical review of recent arguments that online feedback in interactive activation models like TRACE is distinct from other potentially helpful forms of feedback. We conclude that in addition to providing the benefits demonstrated in our simulations, online feedback provides a plausible means of implementing putatively distinct forms of feedback, supporting the interactive activation hypothesis.

Quantity and Diversity: Simulating Early Word Learning Environments

The words in children's language learning environments are strongly predictive of cognitive development and school achievement. But how do we measure language environments and do so at the scale of the many words that children hear day in, day out? The quantity and quality of words in a child's input are typically measured in terms of total amount of talk and the lexical diversity in that talk. There are disagreements in the literature whether amount or diversity is the more critical measure of the input. Here we analyze the properties of a large corpus (6.5 million words) of speech to children and simulate learning environments that differ in amount of talk per unit time, lexical diversity, and the contexts of talk. The central conclusion is that what researchers need to theoretically understand, measure, and change is not the total amount of words, or the diversity of words, but the function that relates total words to the diversity of words, and how that function changes across different contexts of talk.