Learning Predictive Statistics: Strategies and Brain Mechanisms

Contributions of Basal Ganglia Circuits to Perception, Attention, and Consciousness

Research into ascending sensory pathways and cortical networks has generated detailed models of perception. These same cortical regions are strongly connected to subcortical structures, such as the basal ganglia (BG), which have been conceptualized as playing key roles in reinforcement learning and action selection. However, because the BG amasses experiential evidence from higher and lower levels of cortical hierarchies, as well as higher-order thalamus, it is well positioned to dynamically influence perception. Here, we review anatomical, functional, and clinical evidence to demonstrate how the BG can influence perceptual processing and conscious states. This depends on the integrative relationship between cortex, BG, and thalamus, which allows contributions to sensory gating, predictive processing, selective attention, and representation of the temporal structure of events.

Response times are affected by mispredictions in a stochastic game

Acting as a goalkeeper in a video-game, a participant is asked to predict the successive choices of the penalty taker. The sequence of choices of the penalty taker is generated by a stochastic chain with memory of variable length. It has been conjectured that the probability distribution of the response times is a function of the specific sequence of past choices governing the algorithm used by the penalty taker to make his choice at each step. We found empirical evidence that besides this dependence, the distribution of the response times depends also on the success or failure of the previous prediction made by the participant. Moreover, we found statistical evidence that this dependence propagates up to two steps forward after the prediction failure.

Potential cognitive and neural benefits of a computerised cognitive training programme based on Structure Learning in healthy adults: study protocol for a randomised controlled trial

BACKGROUND

Cognitive flexibility refers to the capacity to shift between conceptual representations particularly in response to changes in instruction and feedback. It enables individuals to swiftly adapt to changes in their environment and has significant implications for learning. The present study focuses on investigating changes in cognitive flexibility following an intervention programme-Structure Learning training.

METHODS

Participants are pseudo-randomised to either the Training or Control group, while matched on age, sex, intelligence and cognitive flexibility performance. In the Training group, participants undergo around 2 weeks of training (at least 13 sessions) on Structure Learning. In the Control group, participants do not have to undergo any training and are never exposed to the Structure Learning task. The effects of Structure Learning training are investigated at both the behavioural and neural level. We measured covariates that can influence an individual's training performance before the training phase and outcome measures that can potentially show training benefits after the training phase. At the behavioural level, we investigated outcomes in both cognitive and social aspects with a primary focus on executive functions. At the neural level, we employed a multimodality approach and investigated potential changes to functional connectivity patterns, neurometabolite concentration in the frontal brain regions, and brain microstructure and myelination.

DISCUSSION

We reported the development of a novel training programme based on Structure Learning that aims to hone a general learning ability to potentially achieve extensive transfer benefits across various cognitive constructs. Potential transfer benefits can be exhibited through better performance in outcome measures between Training and Control participants, and positive associations between training performance and outcomes after the training in Training participants. Moreover, we attempt to substantiate behavioural findings with evidence of neural changes across different imaging modalities by the Structure Learning training.

TRIAL REGISTRATION

National Institutes of Health U.S. National Library of Medicine ClinicalTrials.gov NCT05611788. Registered on 7 November 2022.

PROTOCOL VERSION

11 May 2023.

Flexible structure learning under uncertainty

Experience is known to facilitate our ability to interpret sequences of events and make predictions about the future by extracting temporal regularities in our environments. Here, we ask whether uncertainty in dynamic environments affects our ability to learn predictive structures. We exposed participants to sequences of symbols determined by first-order Markov models and asked them to indicate which symbol they expected to follow each sequence. We introduced uncertainty in this prediction task by manipulating the: (a) probability of symbol co-occurrence, (b) stimulus presentation rate. Further, we manipulated feedback, as it is known to play a key role in resolving uncertainty. Our results demonstrate that increasing the similarity in the probabilities of symbol co-occurrence impaired performance on the prediction task. In contrast, increasing uncertainty in stimulus presentation rate by introducing temporal jitter resulted in participants adopting a strategy closer to probability maximization than matching and improving in the prediction tasks. Next, we show that feedback plays a key role in learning predictive statistics. Trial-by-trial feedback yielded stronger improvement than block feedback or no feedback; that is, participants adopted a strategy closer to probability maximization and showed stronger improvement when trained with trial-by-trial feedback. Further, correlating individual strategy with learning performance showed better performance in structure learning for observers who adopted a strategy closer to maximization. Our results indicate that executive cognitive functions (i.e., selective attention) may account for this individual variability in strategy and structure learning ability. Taken together, our results provide evidence for flexible structure learning; individuals adapt their decision strategy closer to probability maximization, reducing uncertainty in temporal sequences and improving their ability to learn predictive statistics in variable environments.

Study protocol: How does cognitive flexibility relate to other executive functions and learning in healthy young adults?

BACKGROUND

Cognitive flexibility (CF) enables individuals to readily shift from one concept or mode of practice/thoughts to another in response to changes in the environment and feedback, making CF vital to optimise success in obtaining goals. However, how CF relates to other executive functions (e.g., working memory, response inhibition), mental abilities (e.g., creativity, literacy, numeracy, intelligence, structure learning), and social factors (e.g., multilingualism, tolerance of uncertainty, perceived social support, social decision-making) is less well understood. The current study aims to (1) establish the construct validity of CF in relation to other executive function skills and intelligence, and (2) elucidate specific relationships between CF, structure learning, creativity, career decision making and planning, and other life skills.

METHODS

This study will recruit up to 400 healthy Singaporean young adults (age 18-30) to complete a wide range of cognitive tasks and social questionnaires/tasks. The richness of the task/questionnaire battery and within-participant administration enables us to use computational modelling and structural equation modelling to examine connections between the latent constructs of interest.

SIGNIFICANCE AND IMPACT

The current study is the first systematic investigation into the construct validity of CF and its interrelationship with other important cognitive skills such as learning and creativity, within an Asian context. The study will further explore the concept of CF as a non-unitary construct, a novel theoretical proposition in the field. The inclusion of a structure learning paradigm is intended to inform future development of a novel intervention paradigm to enhance CF. Finally, the results of the study will be useful for informing classroom pedagogy and the design of lifelong learning policies and curricula, as part of the wider remit of the Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition (CLIC).

Computational and neural mechanisms of statistical pain learning

Pain invariably changes over time. These fluctuations contain statistical regularities which, in theory, could be learned by the brain to generate expectations and control responses. We demonstrate that humans learn to extract these regularities and explicitly predict the likelihood of forthcoming pain intensities in a manner consistent with optimal Bayesian inference with dynamic update of beliefs. Healthy participants received probabilistic, volatile sequences of low and high-intensity electrical stimuli to the hand during brain fMRI. The inferred frequency of pain correlated with activity in sensorimotor cortical regions and dorsal striatum, whereas the uncertainty of these inferences was encoded in the right superior parietal cortex. Unexpected changes in stimulus frequencies drove the update of internal models by engaging premotor, prefrontal and posterior parietal regions. This study extends our understanding of sensory processing of pain to include the generation of Bayesian internal models of the temporal statistics of pain.

Bottom-up and top-down modulation of route selection in imitation

The cognitive system selects the most appropriate action imitative process: a semantic process - relying on long-term memory representations for known actions, and low-level visuomotor transformations for unknown actions. These two processes work in parallel; however, how context regularities and cognitive control modulate them is unclear. In this study, process selection was triggered contextually by presenting mixed known and new actions in predictable or unpredictable lists, while a cue on the forthcoming action triggered top-down control. Known were imitated faster than the new actions in the predictable lists only. Accuracy was higher and reaction times faster in the uncued conditions, and the predictable faster than the unpredictable list in the uncued condition only. In the latter condition, contextual factors modulate process selection, as participants use statistical regularities to perform the task at best. With the cue, the cognitive system tries to control response selection, resulting in more errors and longer reaction times.

A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High-Quality Data: Development and Usability Study

Background

The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the forefront as a promising solution for the rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. This study examines the opportunities and challenges afforded by the societal shift toward web-based testing and highlights an urgent need to establish a standard data quality assurance framework for online studies.

Objective

This study aims to develop and validate a new supervised online testing methodology, remote guided testing (RGT).

Methods

A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory, and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (n=41) or online using remote guided testing (n=44) and delivered using identical web-based platforms (Cambridge Neuropsychological Test Automated Battery, Inquisit, and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, and response times) and overall task performance measures.

Results

The results indicated that, across all data quality and performance measures, RGT data was statistically-equivalent to in-person data collected in the lab (P>.40 for all comparisons). Moreover, RGT participants out-performed the lab group on measured verbal intelligence (P<.001), which could reflect test environment differences, including possible effects of mask-wearing on communication.

Conclusions

These data suggest that the RGT methodology could help ameliorate concerns regarding online data quality—particularly for studies involving high-risk or rare cohorts—and offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

Retrieving the structure of probabilistic sequences of auditory stimuli from EEG data

Using a new probabilistic approach we model the relationship between sequences of auditory stimuli generated by stochastic chains and the electroencephalographic (EEG) data acquired while 19 participants were exposed to those stimuli. The structure of the chains generating the stimuli are characterized by rooted and labeled trees whose leaves, henceforth called contexts, represent the sequences of past stimuli governing the choice of the next stimulus. A classical conjecture claims that the brain assigns probabilistic models to samples of stimuli. If this is true, then the context tree generating the sequence of stimuli should be encoded in the brain activity. Using an innovative statistical procedure we show that this context tree can effectively be extracted from the EEG data, thus giving support to the classical conjecture.

fMRI reveals language-specific predictive coding during naturalistic sentence comprehension

Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.

Content-based Dissociation of Hippocampal Involvement in Prediction

Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top-down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli.

A common probabilistic framework for perceptual and statistical learning

System-level learning of sensory information is traditionally divided into two domains: perceptual learning that focuses on acquiring knowledge suitable for fine discrimination between similar sensory inputs, and statistical learning that explores the mechanisms that develop complex representations of unfamiliar sensory experiences. The two domains have been typically treated in complete separation both in terms of the underlying computational mechanisms and the brain areas and processes implementing those computations. However, a number of recent findings in both domains call in question this strict separation. We interpret classical and more recent results in the general framework of probabilistic computation, provide a unifying view of how various aspects of the two domains are interlinked, and suggest how the probabilistic approach can also alleviate the problem of dealing with widely different types of neural correlates of learning. Finally, we outline several directions along which our proposed approach fosters new types of experiments that can promote investigations of natural learning in humans and other species.

Specific patterns of brain alterations underlie distinct clinical profiles in Huntington's disease

Huntington's disease (HD) is a genetic neurodegenerative disease which involves a triad of motor, cognitive and psychiatric disturbances. However, there is great variability in the prominence of each type of symptom across individuals. The neurobiological basis of such variability remains poorly understood but would be crucial for better tailored treatments. Multivariate multimodal neuroimaging approaches have been successful in disentangling these profiles in other disorders. Thus we applied for the first time such approach to HD. We studied the relationship between HD symptom domains and multimodal measures sensitive to grey and white matter structural alterations. Forty-three HD gene carriers (23 manifest and 20 premanifest individuals) were scanned and underwent behavioural assessments evaluating motor, cognitive and psychiatric domains. We conducted a multimodal analysis integrating different structural neuroimaging modalities measuring grey matter volume, cortical thickness and white matter diffusion indices - fractional anisotropy and radial diffusivity. All neuroimaging measures were entered into a linked independent component analysis in order to obtain multimodal components reflecting common inter-subject variation across imaging modalities. The relationship between multimodal neuroimaging independent components and behavioural measures was analysed using multiple linear regression. We found that cognitive and motor symptoms shared a common neurobiological basis, whereas the psychiatric domain presented a differentiated neural signature. Behavioural measures of different symptom domains correlated with different neuroimaging components, both the brain regions involved and the neuroimaging modalities most prominently associated with each type of symptom showing differences. More severe cognitive and motor signs together were associated with a multimodal component consisting in a pattern of reduced grey matter, cortical thickness and white matter integrity in cognitive and motor related networks. In contrast, depressive symptoms were associated with a component mainly characterised by reduced cortical thickness pattern in limbic and paralimbic regions. In conclusion, using a multivariate multimodal approach we were able to disentangle the neurobiological substrates of two distinct symptom profiles in HD: one characterised by cognitive and motor features dissociated from a psychiatric profile. These results open a new view on a disease classically considered as a uniform entity and initiates a new avenue for further research considering these qualitative individual differences.

Multimodal imaging of brain connectivity reveals predictors of individual decision strategy in statistical learning

Successful human behaviour depends on the brain's ability to extract meaningful structure from information streams and make predictions about future events. Individuals can differ markedly in the decision strategies they use to learn the environment's statistics, yet we have little idea why. Here, we investigate whether the brain networks involved in learning temporal sequences without explicit reward differ depending on the decision strategy that individuals adopt. We demonstrate that individuals alter their decision strategy in response to changes in temporal statistics and engage dissociable circuits: extracting the exact sequence statistics relates to plasticity in motor corticostriatal circuits, while selecting the most probable outcomes relates to plasticity in visual, motivational and executive corticostriatal circuits. Combining graph metrics of functional and structural connectivity, we provide evidence that learning-dependent changes in these circuits predict individual decision strategy. Our findings propose brain plasticity mechanisms that mediate individual ability for interpreting the structure of variable environments.

Brain signatures of a multiscale process of sequence learning in humans

Extracting the temporal structure of sequences of events is crucial for perception, decision-making, and language processing. Here, we investigate the mechanisms by which the brain acquires knowledge of sequences and the possibility that successive brain responses reflect the progressive extraction of sequence statistics at different timescales. We measured brain activity using magnetoencephalography in humans exposed to auditory sequences with various statistical regularities, and we modeled this activity as theoretical surprise levels using several learning models. Successive brain waves related to different types of statistical inferences. Early post-stimulus brain waves denoted a sensitivity to a simple statistic, the frequency of items estimated over a long timescale (habituation). Mid-latency and late brain waves conformed qualitatively and quantitatively to the computational properties of a more complex inference: the learning of recent transition probabilities. Our findings thus support the existence of multiple computational systems for sequence processing involving statistical inferences at multiple scales.

Neural substrates of internally-based and externally-cued timing: An activation likelihood estimation (ALE) meta-analysis of fMRI studies

A dynamic interplay exists between Internally-Based (IBT) and Externally-Cued (ECT) time processing. While IBT processes support the self-generation of context-independent temporal representations, ECT mechanisms allow constructing temporal representations primarily derived from the structure of the sensory environment. We performed an activation likelihood estimation (ALE) meta-analysis on 177 fMRI experiments, from 79 articles, to identify brain areas involved in timing; two individual ALEs tested the hypothesis of a neural segregation between IBT and ECT. The general ALE highlighted a network involving supplementary motor area (SMA), intraparietal sulcus, inferior frontal gyrus (IFG), insula (INS) and basal ganglia. We found evidence of a partial dissociation between IBT and ECT. IBT relies on a subset of areas also involved in ECT, however ECT tasks activate SMA, right IFG, left precentral gyrus and INS in a significantly stronger way. Present results suggest that ECT involves the detection of environmental temporal regularities and their integration with the output of the IBT processing, to generate a representation of time which reflects the temporal metric of the environment.

White-Matter Pathways for Statistical Learning of Temporal Structures

Extracting the statistics of event streams in natural environments is critical for interpreting current events and predicting future ones. The brain is known to rapidly find structure and meaning in unfamiliar streams of sensory experience, often by mere exposure to the environment (i.e., without explicit feedback). Yet, we know little about the brain pathways that support this type of statistical learning. Here, we test whether changes in white-matter (WM) connectivity due to training relate to our ability to extract temporal regularities. By combining behavioral training and diffusion tensor imaging (DTI), we demonstrate that humans adapt to the environment's statistics as they change over time from simple repetition to probabilistic combinations. In particular, we show that learning relates to the decision strategy that individuals adopt when extracting temporal statistics. We next test for learning-dependent changes in WM connectivity and ask whether they relate to individual variability in decision strategy. Our DTI results provide evidence for dissociable WM pathways that relate to individual strategy: extracting the exact sequence statistics (i.e., matching) relates to connectivity changes between caudate and hippocampus, while selecting the most probable outcomes in a given context (i.e., maximizing) relates to connectivity changes between prefrontal, cingulate and basal ganglia (caudate, putamen) regions. Thus, our findings provide evidence for distinct cortico-striatal circuits that show learning-dependent changes of WM connectivity and support individual ability to learn behaviorally-relevant statistics.