iMinerva: a mathematical model of distributional statistical learning

Varying Syntax to Enhance Verb-Focused Intervention for 30-Month-Olds With Language Delay: A Concurrent Multiple Baseline Design

PURPOSE

The purpose of this study is to determine if 2.5-year-olds with language delay would learn verbs (spill) when presented with varying syntactic structure ("The woman is spilling the milk"/"The milk is spilling"; milk = patient or theme) in a therapeutic context. Children with language delay have proportionally small inventories of verbs, which limits expressive language development. Children who have typical language development learn verbs more robustly when presented with alternating arguments than with a single argument structure.

METHOD

Three toddlers with expressive language delay (29-30 months of age) participated in a verb-focused treatment study using a concurrent multiple baseline design. Participants were shown action videos accompanied by sentences with varied argument structure for each target verb. To assess learning pre- and posttreatment, participants were asked to demonstrate actions corresponding to each verb.

RESULTS

Visual inspection and tau analyses reveal significant posttreatment gains of target verbs taught with varying argument structures.

CONCLUSIONS

Our results indicate that learning verbs with high variability of argument roles may facilitate a strong link between lexical representations of verbs and their syntactic structures. Using argument structure variability to teach verbs as an intervention strategy has great potential and should be tested further in larger group studies.

Non-adjacent dependency learning from variable input: investigating the effects of bilingualism, phonological memory, and cognitive control

Introduction

One proposed advantage of bilingualism concerns the ability to extract regularities based on frequency information (statistical learning). Specifically, it has been proposed that bilinguals have an advantage in statistical learning that particularly holds in situations of variable input. Empirical evidence on this matter is scarce. An additional question is whether a potential bilingual advantage in statistical learning can be attributed to enhancements in phonological memory and cognitive control. Previous findings on effects of bilingualism on phonological memory and cognitive control are not consistent.

Method

In the present study, we compared statistical learning from consistent and variable input in monolingual and bilingual children (Study 1) and adults (Study 2). We also explored whether phonological memory and cognitive control might account for any potential group differences found.

Results

The findings suggest that there might be some advantage of bilinguals in statistical learning, but that this advantage is not robust: It largely surfaced only in t-tests against chance for the groups separately, did not surface in the same way for children and adults, and was modulated by experiment order. Furthermore, our results provide no evidence that any enhancement in bilinguals' statistical learning was related to improved phonological memory and cognitive control: bilinguals did not outperform monolinguals on these cognitive measures and performance on these measures did not consistently relate to statistical learning outcomes.

Discussion

Taken together, these findings suggest that any potential effects of bilingualism on statistical learning probably do not involve enhanced cognitive abilities associated with bilingualism.

Everyday Parameters for Episode-to-Episode Dynamics in the Daily Music of Infancy

Experience-dependent change pervades early human development. Though trajectories of developmental change have been well charted in many domains, the episode-to-episode schedules of experiences on which they are hypothesized to depend have not. Here, we took up this issue in a domain known to be governed in part by early experiences: music. Using a corpus of longform audio recordings, we parameterized the daily schedules of music encountered by 35 infants ages 6-12 months. We discovered that everyday music episodes, as well as the interstices between episodes, typically persisted less than a minute, with most daily schedules also including some very extended episodes and interstices. We also discovered that infants encountered music episodes in a bursty rhythm, rather than a periodic or random rhythm, over the day. These findings join a suite of recent discoveries from everyday vision, motor, and language that expand our imaginations beyond artificial learning schedules and enable theorists to model the history-dependence of developmental process in ways that respect everyday sensory histories. Future theories about how infants build knowledge across multiple episodes can now be parameterized using these insights from infants' everyday lives.

A Computational Model of Context-Dependent Encodings During Category Learning

Although current exemplar models of category learning are flexible and can capture how different features are emphasized for different categories, they still lack the flexibility to adapt to local changes in category learning, such as the effect of different sequences of study. In this paper, we introduce a new model of category learning, the Sequential Attention Theory Model (SAT-M), in which the encoding of each presented item is influenced not only by its category assignment (global context) as in other exemplar models, but also by how its properties relate to the properties of temporally neighboring items (local context). By fitting SAT-M to data from experiments comparing category learning with different sequences of trials (interleaved vs. blocked), we demonstrate that SAT-M captures the effect of local context and predicts when interleaved or blocked training will result in better testing performance across three different studies. Comparatively, ALCOVE, SUSTAIN, and a version of SAT-M without locally adaptive encoding provided poor fits to the results. Moreover, we evaluated the direct prediction of the model that different sequences of training change what learners encode and determined that the best-fit encoding parameter values match learners' looking times during training.

Statistically Induced Chunking Recall: A Memory-Based Approach to Statistical Learning

The computations involved in statistical learning have long been debated. Here, we build on work suggesting that a basic memory process, chunking, may account for the processing of statistical regularities into larger units. Drawing on methods from the memory literature, we developed a novel paradigm to test statistical learning by leveraging a robust phenomenon observed in serial recall tasks: that short-term memory is fundamentally shaped by long-term distributional learning. In the statistically induced chunking recall (SICR) task, participants are exposed to an artificial language, using a standard statistical learning exposure phase. Afterward, they recall strings of syllables that either follow the statistics of the artificial language or comprise the same syllables presented in a random order. We hypothesized that if individuals had chunked the artificial language into word-like units, then the statistically structured items would be more accurately recalled relative to the random controls. Our results demonstrate that SICR effectively captures learning in both the auditory and visual modalities, with participants displaying significantly improved recall of the statistically structured items, and even recall specific trigram chunks from the input. SICR also exhibits greater test-retest reliability in the auditory modality and sensitivity to individual differences in both modalities than the standard two-alternative forced-choice task. These results thereby provide key empirical support to the chunking account of statistical learning and contribute a valuable new tool to the literature.

The Emergence of Richly Organized Semantic Knowledge from Simple Statistics: A Synthetic Review

As adults, we draw upon our ample knowledge about the world to support such vital cognitive feats as using language, reasoning, retrieving knowledge relevant to our current goals, planning for the future, adapting to unexpected events, and navigating through the environment. Our knowledge readily supports these feats because it is not merely a collection of stored facts, but rather functions as an organized, semantic network of concepts connected by meaningful relations. How do the relations that fundamentally organize semantic concepts emerge with development? Here, we cast a spotlight on a potentially powerful but often overlooked driver of semantic organization: Rich statistical regularities that are ubiquitous in both language and visual input. In this synthetic review, we show that a driving role for statistical regularities is convergently supported by evidence from diverse fields, including computational modeling, statistical learning, and semantic development. Finally, we identify a number of key avenues of future research into how statistical regularities may drive the development of semantic organization.

PPM-Decay: A computational model of auditory prediction with memory decay

Statistical learning and probabilistic prediction are fundamental processes in auditory cognition. A prominent computational model of these processes is Prediction by Partial Matching (PPM), a variable-order Markov model that learns by internalizing n-grams from training sequences. However, PPM has limitations as a cognitive model: in particular, it has a perfect memory that weights all historic observations equally, which is inconsistent with memory capacity constraints and recency effects observed in human cognition. We address these limitations with PPM-Decay, a new variant of PPM that introduces a customizable memory decay kernel. In three studies-one with artificially generated sequences, one with chord sequences from Western music, and one with new behavioral data from an auditory pattern detection experiment-we show how this decay kernel improves the model's predictive performance for sequences whose underlying statistics change over time, and enables the model to capture effects of memory constraints on auditory pattern detection. The resulting model is available in our new open-source R package, ppm (https://github.com/pmcharrison/ppm).

Individual and Developmental Differences in Distributional Learning

Purpose

This study examined whether children and adults with developmental language disorder (DLD) could use distributional information in an artificial language to learn about grammatical category membership similarly to their typically developing (TD) peers and whether developmental differences existed within and between DLD and TD groups.

Method

Sixteen children ages 7-9 with DLD, 26 age-matched TD children, 17 college students with DLD, and 17 TD college students participated in this task. We used an artificial grammar learning paradigm in which participants had to use knowledge of category membership to determine the acceptability of test items that they had not heard during a training phase.

Results

Individuals with DLD performed similarly to TD peers in distinguishing grammatical from ungrammatical combinations, with no differences between age groups. The order in which items were heard at test differentially affected child versus adult participants and showed a relation with attention and phonological working memory as well.

Conclusion

Differences in ratings between grammatical and ungrammatical items in this task suggest that individuals with DLD can form grammatical categories from novel input and more broadly use distributional information. Differences in order effects suggest a developmental timeline for sensitivity to updating distributional information.

Language development and disorders: Possible genes and environment interactions

Language development requires both basic cognitive mechanisms for learning language and a rich social context from which learning takes off. Disruptions in learning mechanisms, processing abilities, and/or social interactions increase the risks associated with social exclusion or developmental delays. Given the complexity of language processes, a multilevel approach is proposed where both cognitive mechanisms, genetic and environmental factors need to be probed together with their possible interactions. Here we review and discuss such interplay between environment and genetic predispositions in understanding language disorders, with a particular focus on a possible endophenotype, the ability for statistical sequential learning.

Evaluative Conditioning as Memory-Based Judgment

The article proposes a view of evaluative conditioning (EC) as resulting from judgments based on learning instances stored in memory. It is based on the formal episodic memory model MINERVA 2. Additional assumptions specify how the information retrieved from memory is used to inform specific evaluative dependent measures. The present approach goes beyond previous accounts in that it uses a well-specified formal model of episodic memory; it is however more limited in scope as it aims to explain EC phenomena that do not involve reasoning processes. The article illustrates how the memory-based-judgment view accounts for several empirical findings in the EC literature that are often discussed as evidence for dual-process models of attitude learning. It sketches novel predictions, discusses limitations of the present approach, and identifies challenges and opportunities for its future development.

Infant Statistical Learning

Perception involves making sense of a dynamic, multimodal environment. In the absence of mechanisms capable of exploiting the statistical patterns in the natural world, infants would face an insurmountable computational problem. Infant statistical learning mechanisms facilitate the detection of structure. These abilities allow the infant to compute across elements in their environmental input, extracting patterns for further processing and subsequent learning. In this selective review, we summarize findings that show that statistical learning is both a broad and flexible mechanism (supporting learning from different modalities across many different content areas) and input specific (shifting computations depending on the type of input and goal of learning). We suggest that statistical learning not only provides a framework for studying language development and object knowledge in constrained laboratory settings, but also allows researchers to tackle real-world problems, such as multilingualism, the role of ever-changing learning environments, and differential developmental trajectories.

Rapid Statistical Learning Supporting Word Extraction From Continuous Speech

The identification of words in continuous speech, known as speech segmentation, is a critical early step in language acquisition. This process is partially supported by statistical learning, the ability to extract patterns from the environment. Given that speech segmentation represents a potential bottleneck for language acquisition, patterns in speech may be extracted very rapidly, without extensive exposure. This hypothesis was examined by exposing participants to continuous speech streams composed of novel repeating nonsense words. Learning was measured on-line using a reaction time task. After merely one exposure to an embedded novel word, learners demonstrated significant learning effects, as revealed by faster responses to predictable than to unpredictable syllables. These results demonstrate that learners gained sensitivity to the statistical structure of unfamiliar speech on a very rapid timescale. This ability may play an essential role in early stages of language acquisition, allowing learners to rapidly identify word candidates and "break in" to an unfamiliar language.

Infant learning is influenced by local spurious generalizations

In previous work, 11-month-old infants were able to learn rules about the relation of the consonants in CVCV words from just four examples. The rules involved phonetic feature relations (same voicing or same place of articulation), and infants' learning was impeded when pairs of words allowed alternative possible generalizations (e.g. two words both contained the specific consonants p and t). Experiment 1 asked whether a small number of such spurious generalizations found in a randomly ordered list of 24 different words would also impede learning. It did - infants showed no sign of learning the rule. To ask whether it was the overall set of words or their order that prevented learning, Experiment 2 reordered the words to avoid local spurious generalizations. Infants showed robust learning. Infants thus appear to entertain spurious generalizations based on small, local subsets of stimuli. The results support a characterization of infants as incremental rather than batch learners.

TRACX2: a connectionist autoencoder using graded chunks to model infant visual statistical learning

Even newborn infants are able to extract structure from a stream of sensory inputs; yet how this is achieved remains largely a mystery. We present a connectionist autoencoder model, TRACX2, that learns to extract sequence structure by gradually constructing chunks, storing these chunks in a distributed manner across its synaptic weights and recognizing these chunks when they re-occur in the input stream. Chunks are graded rather than all-or-nothing in nature. As chunks are learnt their component parts become more and more tightly bound together. TRACX2 successfully models the data from five experiments from the infant visual statistical learning literature, including tasks involving forward and backward transitional probabilities, low-salience embedded chunk items, part-sequences and illusory items. The model also captures performance differences across ages through the tuning of a single-learning rate parameter. These results suggest that infant statistical learning is underpinned by the same domain-general learning mechanism that operates in auditory statistical learning and, potentially, in adult artificial grammar learning.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

What's statistical about learning? Insights from modelling statistical learning as a set of memory processes

Statistical learning has been studied in a variety of different tasks, including word segmentation, object identification, category learning, artificial grammar learning and serial reaction time tasks (e.g. Saffran et al. 1996 Science 274: , 1926-1928; Orban et al. 2008 Proceedings of the National Academy of Sciences 105: , 2745-2750; Thiessen & Yee 2010 Child Development 81: , 1287-1303; Saffran 2002 Journal of Memory and Language 47: , 172-196; Misyak & Christiansen 2012 Language Learning 62: , 302-331). The difference among these tasks raises questions about whether they all depend on the same kinds of underlying processes and computations, or whether they are tapping into different underlying mechanisms. Prior theoretical approaches to statistical learning have often tried to explain or model learning in a single task. However, in many cases these approaches appear inadequate to explain performance in multiple tasks. For example, explaining word segmentation via the computation of sequential statistics (such as transitional probability) provides little insight into the nature of sensitivity to regularities among simultaneously presented features. In this article, we will present a formal computational approach that we believe is a good candidate to provide a unifying framework to explore and explain learning in a wide variety of statistical learning tasks. This framework suggests that statistical learning arises from a set of processes that are inherent in memory systems, including activation, interference, integration of information and forgetting (e.g. Perruchet & Vinter 1998 Journal of Memory and Language 39: , 246-263; Thiessen et al. 2013 Psychological Bulletin 139: , 792-814). From this perspective, statistical learning does not involve explicit computation of statistics, but rather the extraction of elements of the input into memory traces, and subsequent integration across those memory traces that emphasize consistent information (Thiessen and Pavlik 2013 Cognitive Science 37: , 310-343).This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Listening through voices: Infant statistical word segmentation across multiple speakers

To learn from their environments, infants must detect structure behind pervasive variation. This presents substantial and largely untested learning challenges in early language acquisition. The current experiments address whether infants can use statistical learning mechanisms to segment words when the speech signal contains acoustic variation produced by changes in speakers' voices. In Experiment 1, 8- and 10-month-old infants listened to a continuous stream of novel words produced by 8 different female voices. The voices alternated frequently, potentially interrupting infants' detection of transitional probability patterns that mark word boundaries. Infants at both ages successfully segmented words in the speech stream. In Experiment 2, 8-month-olds demonstrated the ability to generalize their learning about the speech stream when presented with a new, acoustically distinct voice during testing. However, in Experiments 3 and 4, when the same speech stream was produced by only 2 female voices, infants failed to segment the words. The results of these experiments indicate that low acoustic variation may interfere with infants' efficiency in segmenting words from continuous speech, but that infants successfully use statistical cues to segment words in conditions of high acoustic variation. These findings contribute to our understanding of whether statistical learning mechanisms can scale up to meet the demands of natural learning environments.

Modelling the Noise-Robustness of Infants' Word Representations: The Impact of Previous Experience

During language acquisition, infants frequently encounter ambient noise. We present a computational model to address whether specific acoustic processing abilities are necessary to detect known words in moderate noise--an ability attested experimentally in infants. The model implements a general purpose speech encoding and word detection procedure. Importantly, the model contains no dedicated processes for removing or cancelling out ambient noise, and it can replicate the patterns of results obtained in several infant experiments. In addition to noise, we also addressed the role of previous experience with particular target words: does the frequency of a word matter, and does it play a role whether that word has been spoken by one or multiple speakers? The simulation results show that both factors affect noise robustness. We also investigated how robust word detection is to changes in speaker identity by comparing words spoken by known versus unknown speakers during the simulated test. This factor interacted with both noise level and past experience, showing that an increase in exposure is only helpful when a familiar speaker provides the test material. Added variability proved helpful only when encountering an unknown speaker. Finally, we addressed whether infants need to recognise specific words, or whether a more parsimonious explanation of infant behaviour, which we refer to as matching, is sufficient. Recognition involves a focus of attention on a specific target word, while matching only requires finding the best correspondence of acoustic input to a known pattern in the memory. Attending to a specific target word proves to be more noise robust, but a general word matching procedure can be sufficient to simulate experimental data stemming from young infants. A change from acoustic matching to targeted recognition provides an explanation of the improvements observed in infants around their first birthday. In summary, we present a computational model incorporating only the processes infants might employ when hearing words in noise. Our findings show that a parsimonious interpretation of behaviour is sufficient and we offer a formal account of emerging abilities.

Infants generalize from just (the right) four words

Infants in the lab can generalize from 2min of language-like input. Given that infants might fail to fully encode so much input, how many examples do they actually need? And if infants only encode a subset of their input at one time, does generalization change when that subset supports multiple generalizations? Exp. 1 asked whether 11-month-olds generalize the relation between two consonants in a word when just four input words provided non-conflicting vs. partially conflicting support for a phonological feature-based generalization. Infants learned under both conditions, although the latter appears to be more difficult. Exp. 2 asked whether infants' robust learning reflects a bias toward feature-based generalizations. Infants failed to generalize when input provided completely conflicting support for two generalizations. Together, the data suggest that infants are able to generalize from much less input than previously observed, but generalization depends on the specific subset of the input they encounter.

Domain generality versus modality specificity: the paradox of statistical learning

Statistical learning (SL) is typically considered to be a domain-general mechanism by which cognitive systems discover the underlying distributional properties of the input. However, recent studies examining whether there are commonalities in the learning of distributional information across different domains or modalities consistently reveal modality and stimulus specificity. Therefore, important questions are how and why a hypothesized domain-general learning mechanism systematically produces such effects. Here, we offer a theoretical framework according to which SL is not a unitary mechanism, but a set of domain-general computational principles that operate in different modalities and, therefore, are subject to the specific constraints characteristic of their respective brain regions. This framework offers testable predictions and we discuss its computational and neurobiological plausibility.

Neurocognitive mechanisms of statistical-sequential learning: what do event-related potentials tell us?

Statistical-sequential learning (SL) is the ability to process patterns of environmental stimuli, such as spoken language, music, or one's motor actions, that unfold in time. The underlying neurocognitive mechanisms of SL and the associated cognitive representations are still not well understood as reflected by the heterogeneity of the reviewed cognitive models. The purpose of this review is: (1) to provide a general overview of the primary models and theories of SL, (2) to describe the empirical research - with a focus on the event-related potential (ERP) literature - in support of these models while also highlighting the current limitations of this research, and (3) to present a set of new lines of ERP research to overcome these limitations. The review is articulated around three descriptive dimensions in relation to SL: the level of abstractness of the representations learned through SL, the effect of the level of attention and consciousness on SL, and the developmental trajectory of SL across the life-span. We conclude with a new tentative model that takes into account these three dimensions and also point to several promising new lines of SL research.

A role for the developing lexicon in phonetic category acquisition

Infants segment words from fluent speech during the same period when they are learning phonetic categories, yet accounts of phonetic category acquisition typically ignore information about the words in which sounds appear. We use a Bayesian model to illustrate how feedback from segmented words might constrain phonetic category learning by providing information about which sounds occur together in words. Simulations demonstrate that word-level information can successfully disambiguate overlapping English vowel categories. Learning patterns in the model are shown to parallel human behavior from artificial language learning tasks. These findings point to a central role for the developing lexicon in phonetic category acquisition and provide a framework for incorporating top-down constraints into models of category learning.

Discovering words in fluent speech: the contribution of two kinds of statistical information

To efficiently segment fluent speech, infants must discover the predominant phonological form of words in the native language. In English, for example, content words typically begin with a stressed syllable. To discover this regularity, infants need to identify a set of words. We propose that statistical learning plays two roles in this process. First, it provides a cue that allows infants to segment words from fluent speech, even without language-specific phonological knowledge. Second, once infants have identified a set of lexical forms, they can learn from the distribution of acoustic features across those word forms. The current experiments demonstrate both processes are available to 5-month-old infants. This demonstration of sensitivity to statistical structure in speech, weighted more heavily than phonological cues to segmentation at an early age, is consistent with theoretical accounts that claim statistical learning plays a role in helping infants to adapt to the structure of their native language from very early in life.