When forgetting fosters learning: A neural network model for statistical learning

Associative Learning of an Unnormalized Successor Representation

The successor representation is known to relate to temporal associations learned in the temporal context model (Gershman et al., 2012), and subsequent work suggests a wide relevance of the successor representation across spatial, visual, and abstract relational tasks. I demonstrate that the successor representation and purely associative learning have an even deeper relationship than initially indicated: Hebbian temporal associations are an unnormalized form of the successor representation, such that the two converge on an identical representation whenever all states are equally frequent and can correlate highly in practice even when the state distribution is nonuniform.

Long-Horizon Associative Learning Explains Human Sensitivity to Statistical and Network Structures in Auditory Sequences

Networks are a useful mathematical tool for capturing the complexity of the world. In a previous behavioral study, we showed that human adults were sensitive to the high-level network structure underlying auditory sequences, even when presented with incomplete information. Their performance was best explained by a mathematical model compatible with associative learning principles, based on the integration of the transition probabilities between adjacent and nonadjacent elements with a memory decay. In the present study, we explored the neural correlates of this hypothesis via magnetoencephalography (MEG). Participants (N = 23, 16 females) passively listened to sequences of tones organized in a sparse community network structure comprising two communities. An early difference (∼150 ms) was observed in the brain responses to tone transitions with similar transition probability but occurring either within or between communities. This result implies a rapid and automatic encoding of the sequence structure. Using time-resolved decoding, we estimated the duration and overlap of the representation of each tone. The decoding performance exhibited exponential decay, resulting in a significant overlap between the representations of successive tones. Based on this extended decay profile, we estimated a long-horizon associative learning novelty index for each transition and found a correlation of this measure with the MEG signal. Overall, our study sheds light on the neural mechanisms underlying human sensitivity to network structures and highlights the potential role of Hebbian-like mechanisms in supporting learning at various temporal scales.

Chunking as a function of sequence length

Chunking mechanisms are central to several cognitive processes. During the acquisition of visuo-motor sequences, it is commonly reported that these sequences are segmented into chunks leading to more fluid, rapid, and accurate performances. The question of a chunk's storage capacity has been often investigated but little is known about the dynamics of chunk size evolution relative to sequence length. In two experiments, we studied the dynamics and the evolution of a sequence's chunking pattern as a function of sequence length in a non-human primate species (Guinea baboons, Papio papio). Using an operant conditioning device, baboons had to point on a touch screen to a moving target. In Experiment 1, they had to produce repeatedly the same sequence of 4 movements during 2000 trials. In Experiment 2, the sequence was composed of 5 movements and was repeated 4000 times. For both lengths, baboons initially produced small chunks that became fewer and longer with practice. Moreover, the dynamics and the evolution of the chunking pattern varied as a function of sequence length. Finally, with extended practice (i.e., more than 2000 trials), we observed that the mean chunk size reached a plateau indicating that there are fundamental limits to chunking processes that also depend on sequence length. These data therefore provide new empirical evidence for understanding the general properties of chunking mechanisms in sequence learning.

Hebbian learning can explain rhythmic neural entrainment to statistical regularities

In many domains, learners extract recurring units from continuous sequences. For example, in unknown languages, fluent speech is perceived as a continuous signal. Learners need to extract the underlying words from this continuous signal and then memorize them. One prominent candidate mechanism is statistical learning, whereby learners track how predictive syllables (or other items) are of one another. Syllables within the same word predict each other better than syllables straddling word boundaries. But does statistical learning lead to memories of the underlying words-or just to pairwise associations among syllables? Electrophysiological results provide the strongest evidence for the memory view. Electrophysiological responses can be time-locked to statistical word boundaries (e.g., N400s) and show rhythmic activity with a periodicity of word durations. Here, I reproduce such results with a simple Hebbian network. When exposed to statistically structured syllable sequences (and when the underlying words are not excessively long), the network activation is rhythmic with the periodicity of a word duration and activation maxima on word-final syllables. This is because word-final syllables receive more excitation from earlier syllables with which they are associated than less predictable syllables that occur earlier in words. The network is also sensitive to information whose electrophysiological correlates were used to support the encoding of ordinal positions within words. Hebbian learning can thus explain rhythmic neural activity in statistical learning tasks without any memory representations of words. Learners might thus need to rely on cues beyond statistical associations to learn the words of their native language. RESEARCH HIGHLIGHTS: Statistical learning may be utilized to identify recurring units in continuous sequences (e.g., words in fluent speech) but may not generate explicit memory for words. Exposure to statistically structured sequences leads to rhythmic activity with a period of the duration of the underlying units (e.g., words). I show that a memory-less Hebbian network model can reproduce this rhythmic neural activity as well as putative encodings of ordinal positions observed in earlier research. Direct tests are needed to establish whether statistical learning leads to declarative memories for words.

The dynamics of multiword sequence extraction

Being able to process multiword sequences is central for both language comprehension and production. Numerous studies support this claim, but less is known about the way multiword sequences are acquired, and more specifically how associations between their constituents are established over time. Here we adapted the Hebb naming task into a Hebb lexical decision task to study the dynamics of multiword sequence extraction. Participants had to read letter strings presented on a computer screen and were required to classify them as words or pseudowords. Unknown to the participants, a triplet of words or pseudowords systematically appeared in the same order and random words or pseudowords were inserted between two repetitions of the triplet. We found that response times (RTs) for the unpredictable first position in the triplet decreased over repetitions (i.e., indicating the presence of a repetition effect) but more slowly and with a different dynamic compared with items appearing at the predictable second and third positions in the repeated triplet (i.e., showing a slightly different predictability effect). Implicit and explicit learning also varied as a function of the nature of the triplet (i.e., unrelated words, pseudowords, semantically related words, or idioms). Overall, these results provide new empirical evidence about the dynamics of multiword sequence extraction, and more generally about the role of statistical learning in language acquisition.

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

Simple questions on simple associations: regularity extraction in non-human primates

When human and non-human animals learn sequences, they manage to implicitly extract statistical regularities through associative learning mechanisms. In two experiments conducted with a non-human primate species (Guinea baboons, Papio papio), we addressed simple questions on the learning of simple AB associations appearing in longer noisy sequences. Using a serial reaction time task, we manipulated the position of AB within the sequence, such that it could be either fixed (by appearing always at the beginning, middle, or end of a four-element sequence; Experiment 1) or variable (Experiment 2). We also tested the effect of sequence length in Experiment 2 by comparing the performance on AB when it was presented at a variable position within a sequence of four or five elements. The slope of RTs from A to B was taken for each condition as a measurement of learning rate. While all conditions differed significantly from a no-regularity baseline, we found strong evidence that the learning rate did not differ between the conditions. These results indicate that regularity extraction is not impacted by the position of the regularity within a sequence and by the length of the sequence. These data provide novel general empirical constraints for modeling associative mechanisms in sequence learning.

Hebbian, correlational learning provides a memory-less mechanism for Statistical Learning irrespective of implementational choices: Reply to Tovar and Westermann (2022)

Statistical learning relies on detecting the frequency of co-occurrences of items and has been proposed to be crucial for a variety of learning problems, notably to learn and memorize words from fluent speech. Endress and Johnson (2021) (hereafter EJ) recently showed that such results can be explained based on simple memory-less correlational learning mechanisms such as Hebbian Learning. Tovar and Westermann (2022) (hereafter TW) reproduced these results with a different Hebbian model. We show that the main differences between the models are whether temporal decay acts on both the connection weights and the activations (in TW) or only on the activations (in EJ), and whether interference affects weights (in TW) or activations (in EJ). Given that weights and activations are linked through the Hebbian learning rule, the networks behave similarly. However, in contrast to TW, we do not believe that neurophysiological data are relevant to adjudicate between abstract psychological models with little biological detail. Taken together, both models show that different memory-less correlational learning mechanisms provide a parsimonious account of Statistical Learning results. They are consistent with evidence that Statistical Learning might not allow learners to learn and retain words, and Statistical Learning might support predictive processing instead.

Temporal Dynamics of Competition between Statistical Learning and Episodic Memory in Intracranial Recordings of Human Visual Cortex

The function of long-term memory is not just to reminisce about the past, but also to make predictions that help us behave appropriately and efficiently in the future. This predictive function of memory provides a new perspective on the classic question from memory research of why we remember some things but not others. If prediction is a key outcome of memory, then the extent to which an item generates a prediction signifies that this information already exists in memory and need not be encoded. We tested this principle using human intracranial EEG as a time-resolved method to quantify prediction in visual cortex during a statistical learning task and link the strength of these predictions to subsequent episodic memory behavior. Epilepsy patients of both sexes viewed rapid streams of scenes, some of which contained regularities that allowed the category of the next scene to be predicted. We verified that statistical learning occurred using neural frequency tagging and measured category prediction with multivariate pattern analysis. Although neural prediction was robust overall, this was driven entirely by predictive items that were subsequently forgotten. Such interference provides a mechanism by which prediction can regulate memory formation to prioritize encoding of information that could help learn new predictive relationships.SIGNIFICANCE STATEMENT When faced with a new experience, we are rarely at a loss for what to do. Rather, because many aspects of the world are stable over time, we rely on past experiences to generate expectations that guide behavior. Here we show that these expectations during a new experience come at the expense of memory for that experience. From intracranial recordings of visual cortex, we decoded what humans expected to see next in a series of photographs based on patterns of neural activity. Photographs that generated strong neural expectations were more likely to be forgotten in a later behavioral memory test. Prioritizing the storage of experiences that currently lead to weak expectations could help improve these expectations in future encounters.

Detecting non-adjacent dependencies is the exception rather than the rule

Statistical learning refers to our sensitivity to the distributional properties of our environment. Humans have been shown to readily detect the dependency relationship of events that occur adjacently in a stream of stimuli but processing non-adjacent dependencies (NADs) appears more challenging. In the present study, we tested the ability of human participants to detect NADs in a new Hebb-naming task that has been proposed recently to study regularity detection in a noisy environment. In three experiments, we found that most participants did not manage to extract NADs. These results suggest that the ability to learn NADs in noise is the exception rather than the rule. They provide new information about the limits of statistical learning mechanisms.

Statistical Learning in Vision

Vision and learning have long been considered to be two areas of research linked only distantly. However, recent developments in vision research have changed the conceptual definition of vision from a signal-evaluating process to a goal-oriented interpreting process, and this shift binds learning, together with the resulting internal representations, intimately to vision. In this review, we consider various types of learning (perceptual, statistical, and rule/abstract) associated with vision in the past decades and argue that they represent differently specialized versions of the fundamental learning process, which must be captured in its entirety when applied to complex visual processes. We show why the generalized version of statistical learning can provide the appropriate setup for such a unified treatment of learning in vision, what computational framework best accommodates this kind of statistical learning, and what plausible neural scheme could feasibly implement this framework. Finally, we list the challenges that the field of statistical learning faces in fulfilling the promise of being the right vehicle for advancing our understanding of vision in its entirety. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Learning Higher-Order Transitional Probabilities in Nonhuman Primates

The extraction of cooccurrences between two events, A and B, is a central learning mechanism shared by all species capable of associative learning. Formally, the cooccurrence of events A and B appearing in a sequence is measured by the transitional probability (TP) between these events, and it corresponds to the probability of the second stimulus given the first (i.e., p(B|A)). In the present study, nonhuman primates (Guinea baboons, Papio papio) were exposed to a serial version of the XOR (i.e., exclusive-OR), in which they had to process sequences of three stimuli: A, B, and C. In this manipulation, first-order TPs (i.e., AB and BC) were uninformative due to their transitional probabilities being equal to .5 (i.e., p(B|A) = p(C|B) = .5), while second-order TPs were fully predictive of the upcoming stimulus (i.e., p(C|AB) = 1). In Experiment 1, we found that baboons were able to learn second-order TPs, while no learning occurred on first-order TPs. In Experiment 2, this pattern of results was replicated, and a final test ruled out an alternative interpretation in terms of proximity to the reward. These results indicate that a nonhuman primate species can learn a nonlinearly separable problem such as the XOR. They also provide fine-grained empirical data to test models of statistical learning on the interaction between the learning of different orders of TPs. Recent bioinspired models of associative learning are also introduced as promising alternatives to the modeling of statistical learning mechanisms.

Sleeping neonates track transitional probabilities in speech but only retain the first syllable of words

Extracting statistical regularities from the environment is a primary learning mechanism that might support language acquisition. While it has been shown that infants are sensitive to transition probabilities between syllables in speech, it is still not known what information they encode. Here we used electrophysiology to study how full-term neonates process an artificial language constructed by randomly concatenating four pseudo-words and what information they retain after a few minutes of exposure. Neural entrainment served as a marker of the regularities the brain was tracking during learning. Then in a post-learning phase, evoked-related potentials (ERP) to different triplets explored which information was retained. After two minutes of familiarization with the artificial language, neural entrainment at the word rate emerged, demonstrating rapid learning of the regularities. ERPs in the test phase significantly differed between triplets starting or not with the correct first syllables, but no difference was associated with subsequent violations in transition probabilities. Thus, our results revealed a two-step learning process: neonates segmented the stream based on its statistical regularities, but memory encoding targeted during the word recognition phase entangled the ordinal position of the syllables but was still incomplete at that age.

Pitches that Wire Together Fire Together: Scale Degree Associations Across Time Predict Melodic Expectations

The ongoing generation of expectations is fundamental to listeners' experience of music, but research into types of statistical information that listeners extract from musical melodies has tended to emphasize transition probabilities and n-grams, with limited consideration given to other types of statistical learning that may be relevant. Temporal associations between scale degrees represent a different type of information present in musical melodies that can be learned from musical corpora using expectation networks, a computationally simple method based on activation and decay. Expectation networks infer the expectation of encountering one scale degree followed in the near (but not necessarily immediate) future by another given scale degree, with previous work suggesting that scale degree associations learned by expectation networks better predict listener ratings of pitch similarity than transition probabilities. The current work outlines how these learned scale degree associations can be combined to predict melodic continuations and tests the resulting predictions on a dataset of listener responses to a musical cloze task previously used to compare two other models of melodic expectation, a variable-order Markov model (IDyOM) and Temperley's music-theoretically motivated model. Under multinomial logistic regression, all three models explain significant unique variance in human melodic expectations, with coefficient estimates highest for expectation networks. These results suggest that generalized scale degree associations informed by both adjacent and nonadjacent relationships between melodic notes influence listeners' melodic predictions above and beyond n-gram context, highlighting the need to consider a broader range of statistical learning processes that may underlie listeners' expectations for upcoming musical events.