What Mechanisms Underlie Implicit Statistical Learning? Transitional Probabilities Versus Chunks in Language Learning

Hebbian learning can explain rhythmic neural entrainment to statistical regularities

In many domains, learners extract recurring units from continuous sequences. For example, in unknown languages, fluent speech is perceived as a continuous signal. Learners need to extract the underlying words from this continuous signal and then memorize them. One prominent candidate mechanism is statistical learning, whereby learners track how predictive syllables (or other items) are of one another. Syllables within the same word predict each other better than syllables straddling word boundaries. But does statistical learning lead to memories of the underlying words-or just to pairwise associations among syllables? Electrophysiological results provide the strongest evidence for the memory view. Electrophysiological responses can be time-locked to statistical word boundaries (e.g., N400s) and show rhythmic activity with a periodicity of word durations. Here, I reproduce such results with a simple Hebbian network. When exposed to statistically structured syllable sequences (and when the underlying words are not excessively long), the network activation is rhythmic with the periodicity of a word duration and activation maxima on word-final syllables. This is because word-final syllables receive more excitation from earlier syllables with which they are associated than less predictable syllables that occur earlier in words. The network is also sensitive to information whose electrophysiological correlates were used to support the encoding of ordinal positions within words. Hebbian learning can thus explain rhythmic neural activity in statistical learning tasks without any memory representations of words. Learners might thus need to rely on cues beyond statistical associations to learn the words of their native language. RESEARCH HIGHLIGHTS: Statistical learning may be utilized to identify recurring units in continuous sequences (e.g., words in fluent speech) but may not generate explicit memory for words. Exposure to statistically structured sequences leads to rhythmic activity with a period of the duration of the underlying units (e.g., words). I show that a memory-less Hebbian network model can reproduce this rhythmic neural activity as well as putative encodings of ordinal positions observed in earlier research. Direct tests are needed to establish whether statistical learning leads to declarative memories for words.

Chunking Versus Transitional Probabilities: Differentiating Between Theories of Statistical Learning

There are two main approaches to how statistical patterns are extracted from sequences: The transitional probability approach proposes that statistical learning occurs through the computation of probabilities between items in a sequence. The chunking approach, including models such as PARSER and TRACX, proposes that units are extracted as chunks. Importantly, the chunking approach suggests that the extraction of full units weakens the processing of subunits while the transitional probability approach suggests that both units and subunits should strengthen. Previous findings using sequentially organized, auditory stimuli or spatially organized, visual stimuli support the chunking approach. However, one limitation of prior studies is that most assessed learning with the two-alternative forced-choice task. In contrast, this pre-registered experiment examined the two theoretical approaches in sequentially organized, visual stimuli using an online self-paced task-arguably providing a more sensitive index of learning as it occurs-and a secondary offline familiarity judgment task. During the self-paced task, abstract shapes were covertly organized into eight triplets (ABC) where one in every eight was altered (BCA) from the canonical structure in a way that disrupted the full unit while preserving a subunit (BC). Results from the offline familiarity judgment task revealed that the altered triplets were perceived as highly familiar, suggesting the learned representations were relatively flexible. More importantly, results from the online self-paced task demonstrated that processing for subunits, but not unit-initial stimuli, was impeded in the altered triplet. The pattern of results is in line with the chunking approach to statistical learning and, more specifically, the TRACX model.

Statistical Learning in Vision

Vision and learning have long been considered to be two areas of research linked only distantly. However, recent developments in vision research have changed the conceptual definition of vision from a signal-evaluating process to a goal-oriented interpreting process, and this shift binds learning, together with the resulting internal representations, intimately to vision. In this review, we consider various types of learning (perceptual, statistical, and rule/abstract) associated with vision in the past decades and argue that they represent differently specialized versions of the fundamental learning process, which must be captured in its entirety when applied to complex visual processes. We show why the generalized version of statistical learning can provide the appropriate setup for such a unified treatment of learning in vision, what computational framework best accommodates this kind of statistical learning, and what plausible neural scheme could feasibly implement this framework. Finally, we list the challenges that the field of statistical learning faces in fulfilling the promise of being the right vehicle for advancing our understanding of vision in its entirety. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Knowledge of Statistics or Statistical Learning? Readers Prioritize the Statistics of their Native Language Over the Learning of Local Regularities

A large body of evidence suggests that people spontaneously and implicitly learn about regularities present in the visual input. Although theorized as critical for reading, this ability has been demonstrated mostly with pseudo-fonts or highly atypical artificial words. We tested whether local statistical regularities are extracted from materials that more closely resemble one’s native language. In two experiments, Italian speakers saw a set of letter strings modelled on the Italian lexicon and guessed which of these strings were words in a fictitious language and which were foils. Unknown to participants, words could be distinguished from foils based on their average bigram frequency. Surprisingly, in both experiments, we found no evidence that participants relied on this regularity. Instead, lexical decisions were guided by minimal bigram frequency, a cue rooted in participants’ native language. We discuss the implications of these findings for accounts of statistical learning and visual word processing.

Cognitive mechanisms of statistical learning and segmentation of continuous sensory input

Two classes of cognitive mechanisms have been proposed to explain segmentation of continuous sensory input into discrete recurrent constituents: clustering and boundary-finding mechanisms. Clustering mechanisms are based on identifying frequently co-occurring elements and merging them together as parts that form a single constituent. Bracketing (or boundary-finding) mechanisms work by identifying rarely co-occurring elements that correspond to the boundaries between discrete constituents. In a series of behavioral experiments, I tested which mechanisms are at play in the visual modality both during segmentation of a continuous syllabic sequence into discrete word-like constituents and during recognition of segmented constituents. Additionally, I explored conscious awareness of the products of statistical learning—whole constituents versus merged clusters of smaller subunits. My results suggest that both online segmentation and offline recognition of extracted constituents rely on detecting frequently co-occurring elements, a process likely based on associative memory. However, people are more aware of having learnt whole tokens than of recurrent composite clusters.

Statistically Induced Chunking Recall: A Memory-Based Approach to Statistical Learning

The computations involved in statistical learning have long been debated. Here, we build on work suggesting that a basic memory process, chunking, may account for the processing of statistical regularities into larger units. Drawing on methods from the memory literature, we developed a novel paradigm to test statistical learning by leveraging a robust phenomenon observed in serial recall tasks: that short-term memory is fundamentally shaped by long-term distributional learning. In the statistically induced chunking recall (SICR) task, participants are exposed to an artificial language, using a standard statistical learning exposure phase. Afterward, they recall strings of syllables that either follow the statistics of the artificial language or comprise the same syllables presented in a random order. We hypothesized that if individuals had chunked the artificial language into word-like units, then the statistically structured items would be more accurately recalled relative to the random controls. Our results demonstrate that SICR effectively captures learning in both the auditory and visual modalities, with participants displaying significantly improved recall of the statistically structured items, and even recall specific trigram chunks from the input. SICR also exhibits greater test-retest reliability in the auditory modality and sensitivity to individual differences in both modalities than the standard two-alternative forced-choice task. These results thereby provide key empirical support to the chunking account of statistical learning and contribute a valuable new tool to the literature.

Short-Term Memory for Serial Order Moderates Aspects of Language Acquisition in Children With Developmental Language Disorder: Findings From the HelSLI Study

Previous studies of verbal short-term memory (STM) indicate that STM for serial order may be linked to language development and developmental language disorder (DLD). To clarify whether a domain-general mechanism is impaired in DLD, we studied the relations between age, non-verbal serial STM, and language competence (expressive language, receptive language, and language reasoning). We hypothesized that non-verbal serial STM differences between groups of children with DLD and typically developing (TD) children are linked to their language acquisition differences. Fifty-one children with DLD and sixty-six TD children participated as part of the HelSLI project in this cross-sectional study. The children were 4-6-year-old monolingual native Finnish speakers. They completed several tests of language and cognitive functioning, as well as new game-like tests of visual and auditory non-verbal serial STM. We used regression analyses to examine how serial STM moderates the effect of age on language. A non-verbal composite measure of serial visual and auditory STM moderated cross-sectional development of receptive language in the children with DLD. This moderation was not observed in the TD children. However, we found more rapid cross-sectional development of non-verbal serial STM in the TD children than in the children with DLD. The results suggest that children with DLD may be more likely to have compromised general serial STM processing and that superior non-verbal serial STM may be associated with better language acquisition in children with DLD.

Statistically defined visual chunks engage object-based attention

Although objects are the fundamental units of our representation interpreting the environment around us, it is still not clear how we handle and organize the incoming sensory information to form object representations. By utilizing previously well-documented advantages of within-object over across-object information processing, here we test whether learning involuntarily consistent visual statistical properties of stimuli that are free of any traditional segmentation cues might be sufficient to create object-like behavioral effects. Using a visual statistical learning paradigm and measuring efficiency of 3-AFC search and object-based attention, we find that statistically defined and implicitly learned visual chunks bias observers' behavior in subsequent search tasks the same way as objects defined by visual boundaries do. These results suggest that learning consistent statistical contingencies based on the sensory input contributes to the emergence of object representations.

Statistical Learning and Language Impairments: Toward More Precise Theoretical Accounts

Statistical-learning (SL) theory offers an experience-based account of typical and atypical spoken and written language acquisition. Recent work has provided initial support for this view, tying individual differences in SL abilities to linguistic skills, including language impairments. In the current article, we provide a critical review of studies testing SL abilities in participants with and without developmental dyslexia and specific language impairment and discuss the directions that this field of research has taken so far. We identify substantial vagueness in the demarcation lines between different theoretical constructs (e.g., “statistical learning,” “implicit learning,” and “procedural learning”) as well as in the mappings between experimental tasks and these theoretical constructs. Moreover, we argue that current studies are not designed to contrast different theoretical approaches but rather test singular confirmatory predictions without including control tasks showing normal performance. We end by providing concrete suggestions for how to advance research on SL deficits in language impairments.

Beta-Band Activity Is a Signature of Statistical Learning

Through statistical learning (SL), cognitive systems may discover the underlying regularities in the environment. Testing human adults (n = 35, 21 females), we document, in the context of a classical visual SL task, divergent rhythmic EEG activity in the interstimulus delay periods within patterns versus between patterns (i.e., pattern transitions). Our findings reveal increased oscillatory activity in the beta band (∼20 Hz) at triplet transitions that indexes learning: it emerges with increased pattern repetitions; and importantly, it is highly correlated with behavioral learning outcomes. These findings hold the promise of converging on an online measure of learning regularities and provide important theoretical insights regarding the mechanisms of SL and prediction.SIGNIFICANCE STATEMENT Statistical learning has become a major theoretical construct in cognitive science, providing the primary means by which organisms learn about regularities in the environment. As such, it is a critical building block for basic and higher-order cognitive functions. Here we identify, for the first time, a spectral neural index in the time window before stimulus presentation, which evolves with increased pattern exposure, and is predictive of learning performance. The manifestation of learning that is revealed, not in stimulus processing but in the blank interval between stimuli, makes a direct link between the fields of statistical learning on the one hand and either prediction or consolidation on the other hand, suggesting a possible mechanistic account of visual statistical learning.

Exploring and Exploiting Uncertainty: Statistical Learning Ability Affects How We Learn to Process Language Along Multiple Dimensions of Experience

While the effects of pattern learning on language processing are well known, the way in which pattern learning shapes exploratory behavior has long gone unnoticed. We report on the way in which individual differences in statistical pattern learning affect performance in the domain of language along multiple dimensions. Analyzing data from healthy monolingual adults' performance on a serial reaction time task and a self-paced reading task, we show how individual differences in statistical pattern learning are reflected in readers' knowledge of linguistic co-occurrence patterns and in their exploration and exploitation of content-specific and task-general information. First, we investigated the extent to which an individual's pattern learning correlates with his or her sensitivity to systematic morphological and syntactic co-occurrences, as evidenced while reading authentic sentences. We found that the stream of morphological and syntactic information has a more pronounced effect on the reading speed of, as we will label them, content-sensitive learners in that the more probable the co-occurrence pattern, the faster their reading of that pattern will be. Next, we investigated how differences in pattern learning are reflected in the ways in which individuals approach the reading task itself and adapt to it. Casting this relation in terms of exploration/exploitation strategies, known from Reinforcement Learning, we conclude that content-sensitive learners are also more likely to initially probe (explore) a wider range of directly relevant patterns, which they can later use (exploit) to optimize their reading performance further. By affecting exploratory behavior, pattern learning influences the information that is gathered and becomes available for exploitation, thereby increasing the effect pattern learning has on language cognition.

Editors' Introduction: Aligning Implicit Learning and Statistical Learning: Two Approaches, One Phenomenon

This editors' introduction provides the background to the special issue. We first outline the rationale for bringing together, in a single volume, leading researchers from two distinct, yet related research strands, implicit learning and statistical learning. The aim of the special issue is to facilitate the development of a shared understanding of research questions and methodologies, to provide a platform for discussing similarities and differences between the two strands, and to encourage the formulation of joint research agendas. We then introduce the new contributions solicited for this special issue and provide our perspective on the agenda setting that results from combining these two approaches.

A common probabilistic framework for perceptual and statistical learning

System-level learning of sensory information is traditionally divided into two domains: perceptual learning that focuses on acquiring knowledge suitable for fine discrimination between similar sensory inputs, and statistical learning that explores the mechanisms that develop complex representations of unfamiliar sensory experiences. The two domains have been typically treated in complete separation both in terms of the underlying computational mechanisms and the brain areas and processes implementing those computations. However, a number of recent findings in both domains call in question this strict separation. We interpret classical and more recent results in the general framework of probabilistic computation, provide a unifying view of how various aspects of the two domains are interlinked, and suggest how the probabilistic approach can also alleviate the problem of dealing with widely different types of neural correlates of learning. Finally, we outline several directions along which our proposed approach fosters new types of experiments that can promote investigations of natural learning in humans and other species.