Dissociating hippocampal and striatal contributions to sequential prediction learning

A dopaminergic basis of behavioral control

Both goal-directed and automatic processes shape human behavior, but these processes often conflict. Behavioral control is the decision about which process guides behavior. Despite the importance of behavioral control for adaptive decision-making, its neural mechanisms remain unclear. Critically, it is unknown if there are mechanisms for behavioral control that are distinct from those supporting the formation of goal-relevant knowledge. We performed deep phenotyping of individual dopamine system function by combining multiple PET scans, fMRI, and dopaminergic drug administration in a within-subject, double-blind, placebo-controlled design. Subjects performed a rule-based response time task, with goal-directed and automatic decision-making operationalized as model-based and model-free influences on behavior. We found a double dissociation between two aspects of ventral striatal dopamine physiology: D2/3 receptor availability and dopamine synthesis capacity. Convergent and causal evidence indicated that D2/3 receptors regulate behavioral control by enhancing model-based and blunting model-free influences on behavior but do not affect model-based knowledge formation. In contrast, dopamine synthesis capacity was linked to the formation of model-based knowledge but not behavioral control. D2/3 receptors also modulated frontostriatal functional connectivity, suggesting they regulate behavioral control by gating prefrontal inputs to the striatum. These results identify central mechanisms underlying individual and state differences in behavioral control and point to striatal D2/3 receptors as targets for interventions for improving goal-directed behavior.

Intentional learning establishes multiple attentional sets that simultaneously guide attention

One of the key human cognitive capabilities is to extract regularities from the environment to guide behavior. An attentional set for a target feature can be established through statistical learning of probabilistic target associations; however, whether an array of attentional sets of predictive target features can be established during intentional learning, and how they might guide attention, is not known yet. To address these questions, we had human observers perform a visual search task where we instructed them to try to use color to find their target shape. We structured the task with a fine-grained statistical regularity such that the target shapes appeared in different colors with five unique probabilities (i.e., 33%, 26%, 19%, 12%, and 5%) while we recorded their electroencephalogram. Observers rapidly learned these regularities, evidenced by being faster to report targets that appeared in higher probability colors. These effects were not due to unequal sample sizes or simple feature priming. More importantly, equivalent speeding across a set of high-probability colors suggests that the brain was driving attention to multiple targets simultaneously. Our electrophysiological results showed larger amplitude N2 posterior contralateral component, indexing perceptual attention, and late positive complex (LPC) component, indexing postperceptual processes, for targets paired with high-probability colors. These electrophysiological data suggest that the learned attentional sets change both perceptual selection and how postperceptual decisions are made. In sum, we show that multiple attentional sets can be established during intentional learning that accompanies general task acquisition and that these attentional sets can simultaneously guide attention by enhancing both perceptual attention and postperceptual processes. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Associative memory retrieval modulates upcoming perceptual decisions

Expectations can inform fast, accurate decisions. But what informs expectations? Here we test the hypothesis that expectations are set by dynamic inference from memory. Participants performed a cue-guided perceptual decision task with independently-varying memory and sensory evidence. Cues established expectations by reminding participants of past stimulus-stimulus pairings, which predicted the likely target in a subsequent noisy image stream. Participant's responses used both memory and sensory information, in accordance to their relative reliability. Formal model comparison showed that the sensory inference was best explained when its parameters were set dynamically at each trial by evidence sampled from memory. Supporting this model, neural pattern analysis revealed that responses to the probe were modulated by the specific content and fidelity of memory reinstatement that occurred before the probe appeared. Together, these results suggest that perceptual decisions arise from the continuous sampling of memory and sensory evidence.

Structured memory representations develop at multiple time scales in hippocampal-cortical networks

Influential views of systems memory consolidation posit that the hippocampus rapidly forms representations of specific events, while neocortical networks extract regularities across events, forming the basis of schemas and semantic knowledge. Neocortical extraction of schematic memory representations is thought to occur on a protracted timescale of months, especially for information that is unrelated to prior knowledge. However, this theorized evolution of memory representations across extended timescales, and differences in the temporal dynamics of consolidation across brain regions, lack reliable empirical support. To examine the temporal dynamics of memory representations, we repeatedly exposed human participants to structured information via sequences of fractals, while undergoing longitudinal fMRI for three months. Sequence-specific activation patterns emerged in the hippocampus during the first 1-2 weeks of learning, followed one week later by high-level visual cortex, and subsequently the medial prefrontal and parietal cortices. Schematic, sequence-general representations emerged in the prefrontal cortex after 3 weeks of learning, followed by the medial temporal lobe and anterior temporal cortex. Moreover, hippocampal and most neocortical representations showed sustained rather than time-limited dynamics, suggesting that representations tend to persist across learning. These results show that specific hippocampal representations emerge early, followed by both specific and schematic representations at a gradient of timescales across hippocampal-cortical networks as learning unfolds. Thus, memory representations do not exist only in specific brain regions at a given point in time, but are simultaneously present at multiple levels of abstraction across hippocampal-cortical networks.

Predictive motor activation: Modulated by expectancy or predictability?

Predicting actions is a fundamental ability that helps us to comprehend what is happening in our environment and to interact with others. The motor system was previously identified as source of action predictions. Yet, which aspect of the statistical likelihood of upcoming actions the motor system is sensitive to remains an open question. This EEG study investigated how regularities in observed actions are reflected in the motor system and utilized to predict upcoming actions. Prior to measuring EEG, participants watched videos of action sequences with different transitional probabilities. After training, participants' brain activity over motor areas was measured using EEG while watching videos of action sequences with the same statistical structure. Focusing on the mu and beta frequency bands we tested whether activity of the motor system reflects the statistical likelihood of upcoming actions. We also explored two distinct aspects of the statistical structure that capture different prediction processes, expectancy and predictability. Expectancy describes participants' expectation of the most likely action, whereas predictability represents all possible actions and their relative probabilities. Results revealed that mu and beta oscillations play different roles during action prediction. While the mu rhythm reflected anticipatory activity without any link to the statistical structure, the beta rhythm was related to the expectancy of an action. Our findings support theories proposing that the motor system underlies action prediction, and they extend such theories by showing that multiple forms of statistical information are extracted when observing action sequences. This information is integrated in the prediction generated by the neural motor system of which action is going to happen next.

Glutamatergic dysfunction leads to a hyper-dopaminergic phenotype through deficits in short-term habituation: a mechanism for aberrant salience

Psychosis in disorders like schizophrenia is commonly associated with aberrant salience and elevated striatal dopamine. However, the underlying cause(s) of this hyper-dopaminergic state remain elusive. Various lines of evidence point to glutamatergic dysfunction and impairments in synaptic plasticity in the etiology of schizophrenia, including deficits associated with the GluA1 AMPAR subunit. GluA1 knockout (Gria1^-/-) mice provide a model of impaired synaptic plasticity in schizophrenia and exhibit a selective deficit in a form of short-term memory which underlies short-term habituation. As such, these mice are unable to reduce attention to recently presented stimuli. In this study we used fast-scan cyclic voltammetry to measure phasic dopamine responses in the nucleus accumbens of Gria1^-/- mice to determine whether this behavioral phenotype might be a key driver of a hyper-dopaminergic state. There was no effect of GluA1 deletion on electrically-evoked dopamine responses in anaesthetized mice, demonstrating normal endogenous release properties of dopamine neurons in Gria1^-/- mice. Furthermore, dopamine signals were initially similar in Gria1^-/- mice compared to controls in response to both sucrose rewards and neutral light stimuli. They were also equally sensitive to changes in the magnitude of delivered rewards. In contrast, however, these stimulus-evoked dopamine signals failed to habituate with repeated presentations in Gria1^-/- mice, resulting in a task-relevant, hyper-dopaminergic phenotype. Thus, here we show that GluA1 dysfunction, resulting in impaired short-term habituation, is a key driver of enhanced striatal dopamine responses, which may be an important contributor to aberrant salience and psychosis in psychiatric disorders like schizophrenia.

The brain correlates of hostile attribution bias and their relation to the displaced aggression

BACKGROUND

Hostile attribution bias (HAB) has been considered as a risk factor of various types of psychosocial adjustment problem, and contributes to displaced aggression (DA). The neural basis of HAB and the underlying mechanisms of how HAB predicts DA remain unclear.

METHODS

The current study used degree centrality (DC) and resting-sate functional connectivity (RSFC) to investigate the functional connection pattern related to HAB in 503 undergraduate students. Furthermore, the "Decoding" was used to investigate which psychological components the maps of the RSFC-behavior may be related to. Finally, to investigate whether and how the RSFC pattern, HAB predicts DA, we performed mediation analyses.

RESULTS

We found that HAB was negatively associated with DC in bilateral temporal poles (TP) and positively correlated with DC in the putamen and thalamus; Moreover, HAB was negatively associated with the strength of functional connectivity between TP and brain regions in the theory of mind network (ToM), and positively related to the strength of functional connectivity between the thalamus and regions in the ToM network. The "Decoding" showed the maps of the RSFC-behavior may involve the theory mind, autobiographic, language, comprehension and working memory. Mediation analysis further showed that HAB mediated the relationship between some neural correlates of the HAB and DA.

LIMITATIONS

The current results need to be further tested by experimental methods or longitudinal design in further studies.

CONCLUSIONS

These findings shed light on the neural underpinnings of HAB and provide a possible mediation model regarding the relationships among RSFC pattern, HAB, and displaced aggression.

Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

Mixing memory and desire: How memory reactivation supports deliberative decision-making

Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.

Neural Substrates of Incidental Associations and Mediated Learning: The Role of Cannabinoid Receptors

The ability to form associations between different stimuli in the environment to guide adaptive behavior is a central element of learning processes, from perceptual learning in humans to Pavlovian conditioning in animals. Like so, classical conditioning paradigms that test direct associations between low salience sensory stimuli and high salience motivational reinforcers are extremely informative. However, a large part of everyday learning cannot be solely explained by direct conditioning mechanisms - this includes to a great extent associations between individual sensory stimuli, carrying low or null immediate motivational value. This type of associative learning is often described as incidental learning and can be captured in animal models through sensory preconditioning procedures. Here we summarize the evolution of research on incidental and mediated learning, overview the brain systems involved and describe evidence for the role of cannabinoid receptors in such higher-order learning tasks. This evidence favors a number of contemporary hypotheses concerning the participation of the endocannabinoid system in psychosis and psychotic experiences and provides a conceptual framework for understanding how the use of cannabinoid drugs can lead to altered perceptive states.

Visual Predictions Operate on Different Timescales

Humans live in a volatile environment, subject to changes occurring at different timescales. The ability to adjust internal predictions accordingly is critical for perception and action. We studied this ability with two EEG experiments in which participants were presented with sequences of four Gabor patches, simulating a rotation, and instructed to respond to the last stimulus (target) to indicate whether or not it continued the direction of the first three stimuli. Each experiment included a short-term learning phase in which the probabilities of these two options were very different (p = .2 vs. p = .8, Rules A and B, respectively), followed by a neutral test phase in which both probabilities were equal. In addition, in one of the experiments, prior to the short-term phase, participants performed a much longer long-term learning phase where the relative probabilities of the rules predicting targets were opposite to those of the short-term phase. Analyses of the RTs and P3 amplitudes showed that, in the neutral test phase, participants initially predicted targets according to the probabilities learned in the short-term phase. However, whereas participants not pre-exposed to the long-term learning phase gradually adjusted their predictions to the neutral probabilities, for those who performed the long-term phase, the short-term associations were spontaneously replaced by those learned in that phase. This indicates that the long-term associations remained intact whereas the short-term associations were learned, transiently used, and abandoned when the context changed. The spontaneous recovery suggests independent storage and control of long-term and short-term associations.

Reward and fictive prediction error signals in ventral striatum: asymmetry between factual and counterfactual processing

Reward prediction error, the difference between the expected and obtained reward, is known to act as a reinforcement learning neural signal. In the current study, we propose a model fitting approach that combines behavioral and neural data to fit computational models of reinforcement learning. Briefly, we penalized subject-specific fitted parameters that moved away too far from the group median, except when that deviation led to an improvement in the model's fit to neural responses. By means of a probabilistic monetary learning task and fMRI, we compared our approach with standard model fitting methods. Q-learning outperformed actor-critic at both behavioral and neural level, although the inclusion of neuroimaging data into model fitting improved the fit of actor-critic models. We observed both action-value and state-value prediction error signals in the striatum, while standard model fitting approaches failed to capture state-value signals. Finally, left ventral striatum correlated with reward prediction error while right ventral striatum with fictive prediction error, suggesting a functional hemispheric asymmetry regarding prediction-error driven learning.

Sequence Structure Has a Differential Effect on Underlying Motor Learning Processes

Current methods to understand implicit motor sequence learning inadequately assess motor skill acquisition in daily life. Using fixed sequences in the serial reaction time task is not ideal as participants may become aware of the sequence, thereby changing the learning from implicit to explicit. Probabilistic sequences, in which stimuli are linked by statistical, rather than deterministic, associations can ensure that learning remains implicit. Additionally, the processes underlying the learning of motor sequences may differ based on sequence structure. Here, the authors compared the learning of fixed and probabilistic sequences to randomly ordered stimuli using a modified serial reaction time task. Both the fixed and probabilistic sequence groups exhibited learning as indicated by decreased response time and variability. In the initial stage of learning, fixed sequences exhibited both online and offline gains in response time; however, only the offline gain was observed during the learning of probabilistic sequences. These results indicated that probabilistic structures may be learned differently from fixed structures and have important implications for our current understanding of motor learning. Probabilistic sequences more accurately reflect motor skill acquisition in daily life, offer ecological validity to the serial reaction time framework, and advance our understanding of motor learning.

Retrieving the structure of probabilistic sequences of auditory stimuli from EEG data

Using a new probabilistic approach we model the relationship between sequences of auditory stimuli generated by stochastic chains and the electroencephalographic (EEG) data acquired while 19 participants were exposed to those stimuli. The structure of the chains generating the stimuli are characterized by rooted and labeled trees whose leaves, henceforth called contexts, represent the sequences of past stimuli governing the choice of the next stimulus. A classical conjecture claims that the brain assigns probabilistic models to samples of stimuli. If this is true, then the context tree generating the sequence of stimuli should be encoded in the brain activity. Using an innovative statistical procedure we show that this context tree can effectively be extracted from the EEG data, thus giving support to the classical conjecture.

Convergent and Distinct Effects of Multisensory Combination on Statistical Learning Using a Computer Glove

Learning to play a musical instrument involves mapping visual + auditory cues to motor movements and anticipating transitions. Inspired by the serial reaction time task and artificial grammar learning, we investigated explicit and implicit knowledge of statistical learning in a sensorimotor task. Using a between-subjects design with four groups, one group of participants were provided with visual cues and followed along by tapping the corresponding fingertip to their thumb, while using a computer glove. Another group additionally received accompanying auditory tones; the final two groups received sensory (visual or visual + auditory) cues but did not provide a motor response—all together following a 2 × 2 design. Implicit knowledge was measured by response time, whereas explicit knowledge was assessed using probe tests. Findings indicate that explicit knowledge was best with only the single modality, but implicit knowledge was best when all three modalities were involved.

Distinct Neural Circuits Underlie Prospective and Concurrent Memory-Guided Behavior

The past is the best predictor of the future. This simple postulate belies the complex neurobiological mechanisms that facilitate an individual's use of memory to guide decisions. Previous research has shown integration of memories bias decision-making. Alternatively, memories can prospectively guide our choices. Here, we elucidate the mechanisms and timing of hippocampal (HPC), medial prefrontal cortex (mPFC), and striatal contributions during prospective memory-guided decision-making. We develop an associative learning task in which the correct choice is conditional on the preceding stimulus. Two distinct networks emerge: (1) a prospective circuit consisting of the HPC, putamen, mPFC, and other cortical regions, which exhibit increased activation preceding successful conditional decisions and (2) a concurrent circuit comprising the caudate, dorsolateral prefrontal cortex (dlPFC), and additional cortical structures that engage during the execution of correct conditional choices. Our findings demonstrate distinct neurobiological circuits through which memory prospectively biases decisions and influences choice execution.

Beyond the mean reaction time: Trial-by-trial reaction time reveals the distraction effect on perceptual-motor sequence learning

Perceptual-motor sequences can be learned quickly under distraction, often demonstrated by the mean reaction time (RT) change in a serial reaction time (SRT) task. However, any arbitrary mean RT can arise from one of many distinct trial-by-trial RT patterns. It is surprising that previous sequence learning studies have hinged only on the mean RT metrics while little is known about the distraction effect on its trial-by-trial processes. In an SRT task with or without distraction, we found that initially learning a fixed repeating sequence without distraction was expressed by a micro-online learning process where reaction time (RT) progressively improved within learning blocks as adults continuously performed the SRT task. Such online RT improvements, however, vanished when the SRT task was performed under distraction. Despite the detrimental effect of distraction on micro-online RT improvements, we observed offline enhancements in RT following rest intervals of 3 min that emerged to secure sequence learning under distraction. We reasoned that distraction may exert influence on the micro-online and offline learning by mediating the engagement of explicit and implicit memory. Given the offline RT change under distraction, a short rest between learning blocks may be a key player in early perceptual-motor sequence learning under distraction. We thus suggest that future studies investigating the distraction effect on sequence learning need to control the length of rest between learning blocks, while previous research with equivocal interpretations of the distraction effect failed to do so.

Goal-oriented and habitual decisions: Neural signatures of model-based and model-free learning

Human decision-making is mainly driven by two fundamental learning processes: a slow, deliberative, goal-directed model-based process that maps out the potential outcomes of all options and a rapid habitual model-free process that enables reflexive repetition of previously successful choices. Although many model-informed neuroimaging studies have examined the neural correlates of model-based and model-free learning, the concordant activity among these two processes remains unclear. We used quantitative meta-analyses of functional magnetic resonance imaging experiments to identify the concordant activity pertaining to model-based and model-free learning over a range of reward-related paradigms. We found that: 1) both processes yielded concordant ventral striatum activity, 2) model-based learning activated the medial prefrontal cortex and orbital frontal cortex, and 3) model-free learning specifically activated the left globus pallidus and right caudate head. Our findings suggest that model-free and model-based decision making engage overlapping yet distinct neural regions. These stereotaxic maps improve our understanding of how deliberative goal-directed and reflexive habitual learning are implemented in the brain.

Distentangling the systems contributing to changes in learning during adolescence

Multiple neurocognitive systems contribute simultaneously to learning. For example, dopamine and basal ganglia (BG) systems are thought to support reinforcement learning (RL) by incrementally updating the value of choices, while the prefrontal cortex (PFC) contributes different computations, such as actively maintaining precise information in working memory (WM). It is commonly thought that WM and PFC show more protracted development than RL and BG systems, yet their contributions are rarely assessed in tandem. Here, we used a simple learning task to test how RL and WM contribute to changes in learning across adolescence. We tested 187 subjects ages 8 to 17 and 53 adults (25-30). Participants learned stimulus-action associations from feedback; the learning load was varied to be within or exceed WM capacity. Participants age 8-12 learned slower than participants age 13-17, and were more sensitive to load. We used computational modeling to estimate subjects' use of WM and RL processes. Surprisingly, we found more protracted changes in RL than WM during development. RL learning rate increased with age until age 18 and WM parameters showed more subtle, gender- and puberty-dependent changes early in adolescence. These results can inform education and intervention strategies based on the developmental science of learning.

Zen and the Art of Making a Bayesian Espresso

In this issue of Neuron, Konovalov and Krajbich (2018) argue that a Bayesian inference is employed when learning new sequences and identify distinct brain networks that track the uncertainty of both the current state and the underlying pattern structure.

A Roadmap for Understanding Memory: Decomposing Cognitive Processes into Operations and Representations

Thanks to patients Phineas Gage and Henry Molaison, we have long known that behavioral control depends on the frontal lobes, whereas declarative memory depends on the medial temporal lobes (MTL). For decades, cognitive functions-behavioral control, declarative memory-have served as labels for characterizing the division of labor in cortex. This approach has made enormous contributions to understanding how the brain enables the mind, providing a systems-level explanation of brain function that constrains lower-level investigations of neural mechanism. Today, the approach has evolved such that functional labels are often applied to brain networks rather than focal brain regions. Furthermore, the labels have diversified to include both broadly-defined cognitive functions (declarative memory, visual perception) and more circumscribed mental processes (recollection, familiarity, priming). We ask whether a process-a high-level mental phenomenon corresponding to an introspectively-identifiable cognitive event-is the most productive label for dissecting memory. For example, recollection conflates a neurocomputational operation (pattern completion-based retrieval) with a class of representational content (associative, high-dimensional memories). Because a full theory of memory must identify operations and representations separately, and specify how they interact, we argue that processes like recollection constitute inadequate labels for characterizing neural mechanisms. Instead, we advocate considering the component operations and representations of processes like recollection in isolation. For the organization of memory, the evidence suggests that pattern completion is recapitulated widely across the ventral visual stream and MTL, but the division of labor between sites within this pathway can be explained by representational content.

Predictions as a window into learning: Anticipatory fixation offsets carry more information about environmental statistics than reactive stimulus-responses

A core question underlying neurobiological and computational models of behavior is how individuals learn environmental statistics and use them to make predictions. Most investigations of this issue have relied on reactive paradigms, in which inferences about predictive processes are derived by modeling responses to stimuli that vary in likelihood. Here we deployed a novel anticipatory oculomotor metric to determine how input statistics impact anticipatory behavior that is decoupled from target-driven-response. We implemented transition constraints between target locations, so that the probability of a target being presented on the same side as the previous trial was 70% in one condition (pret70) and 30% in the other (pret30). Rather than focus on responses to targets, we studied subtle endogenous anticipatory fixation offsets (AFOs) measured while participants fixated the screen center, awaiting a target. These AFOs were small (<0.4° from center on average), but strongly tracked global-level statistics. Speaking to learning dynamics, trial-by-trial fluctuations in AFO were well-described by a learning model, which identified a lower learning rate in pret70 than pret30, corroborating prior suggestions that pret70 is subjectively treated as more regular. Most importantly, direct comparisons with saccade latencies revealed that AFOs: (a) reflected similar temporal integration windows, (b) carried more information about the statistical context than did saccade latencies, and (c) accounted for most of the information that saccade latencies also contained about inputs statistics. Our work demonstrates how strictly predictive processes reflect learning dynamics, and presents a new direction for studying learning and prediction.

Computing Social Value Conversion in the Human Brain

Social signals play powerful roles in shaping self-oriented reward valuation and decision making. These signals activate social and valuation/decision areas, but the core computation for their integration into the self-oriented decision machinery remains unclear. Here, we study how a fundamental social signal, social value (others' reward value), is converted into self-oriented decision making in the human brain. Using behavioral analysis, modeling, and neuroimaging, we show three-stage processing of social value conversion from the offer to the effective value and then to the final decision value. First, a value of others' bonus on offer, called offered value, was encoded uniquely in the right temporoparietal junction (rTPJ) and also in the left dorsolateral prefrontal cortex (ldlPFC), which is commonly activated by offered self-bonus value. The effective value, an intermediate value representing the effective influence of the offer on the decision, was represented in the right anterior insula (rAI), and the final decision value was encoded in the medial prefrontal cortex (mPFC). Second, using psychophysiological interaction and dynamic causal modeling analyses, we demonstrated three-stage feedforward processing from the rTPJ and ldPFC to the rAI and then from rAI to the mPFC. Further, we showed that these characteristics of social conversion underlie distinct sociobehavioral phenotypes. We demonstrate that the variability in the conversion underlies the difference between prosocial and selfish subjects, as seen from the differential strength of the rAI and ldlPFC coupling to the mPFC responses, respectively. Together, these findings identified fundamental neural computation processes for social value conversion underlying complex social decision making behaviors.SIGNIFICANCE STATEMENT In daily life, we make decisions based on self-interest, but also in consideration for others' status. These social influences modulate valuation and decision signals in the brain, suggesting a fundamental process called value conversion that translates social information into self-referenced decisions. However, little is known about the conversion process and its underlying brain mechanisms. We investigated value conversion using human fMRI with computational modeling and found three essential stages in a progressive brain circuit from social to empathic and decision areas. Interestingly, the brain mechanism of conversion differed between prosocial and individualistic subjects. These findings reveal how the brain processes and merges social information into the elemental flow of self-interested decision making.

Understanding the Neural Bases of Implicit and Statistical Learning

Both implicit learning and statistical learning focus on the ability of learners to pick up on patterns in the environment. It has been suggested that these two lines of research may be combined into a single construct of "implicit statistical learning." However, by comparing the neural processes that give rise to implicit versus statistical learning, we may determine the extent to which these two learning paradigms do indeed describe the same core mechanisms. In this review, we describe current knowledge about neural mechanisms underlying both implicit learning and statistical learning, highlighting converging findings between these two literatures. A common thread across all paradigms is that learning is supported by interactions between the declarative and nondeclarative memory systems of the brain. We conclude by discussing several outstanding research questions and future directions for each of these two research fields. Moving forward, we suggest that the two literatures may interface by defining learning according to experimental paradigm, with "implicit learning" reserved as a specific term to denote learning without awareness, which may potentially occur across all paradigms. By continuing to align these two strands of research, we will be in a better position to characterize the neural bases of both implicit and statistical learning, ultimately improving our understanding of core mechanisms that underlie a wide variety of human cognitive abilities.

Hippocampal pattern separation supports reinforcement learning

Animals rely on learned associations to make decisions. Associations can be based on relationships between object features (e.g., the three leaflets of poison ivy leaves) and outcomes (e.g., rash). More often, outcomes are linked to multidimensional states (e.g., poison ivy is green in summer but red in spring). Feature-based reinforcement learning fails when the values of individual features depend on the other features present. One solution is to assign value to multi-featural conjunctive representations. Here, we test if the hippocampus forms separable conjunctive representations that enables the learning of response contingencies for stimuli of the form: AB+, B-, AC-, C+. Pattern analyses on functional MRI data show the hippocampus forms conjunctive representations that are dissociable from feature components and that these representations, along with those of cortex, influence striatal prediction errors. Our results establish a novel role for hippocampal pattern separation and conjunctive representation in reinforcement learning.

Joint modeling of reaction times and choice improves parameter identifiability in reinforcement learning models

BACKGROUND

Reinforcement learning models provide excellent descriptions of learning in multiple species across a variety of tasks. Many researchers are interested in relating parameters of reinforcement learning models to neural measures, psychological variables or experimental manipulations. We demonstrate that parameter identification is difficult because a range of parameter values provide approximately equal quality fits to data. This identification problem has a large impact on power: we show that a researcher who wants to detect a medium sized correlation (r = .3) with 80% power between a variable and learning rate must collect 60% more subjects than specified by a typical power analysis in order to account for the noise introduced by model fitting.

NEW METHOD

We derive a Bayesian optimal model fitting technique that takes advantage of information contained in choices and reaction times to constrain parameter estimates.

RESULTS

We show using simulation and empirical data that this method substantially improves the ability to recover learning rates.

COMPARISON WITH EXISTING METHODS

We compare this method against the use of Bayesian priors. We show in simulations that the combined use of Bayesian priors and reaction times confers the highest parameter identifiability. However, in real data where the priors may have been misspecified, the use of Bayesian priors interferes with the ability of reaction time data to improve parameter identifiability.

CONCLUSIONS

We present a simple technique that takes advantage of readily available data to substantially improve the quality of inferences that can be drawn from parameters of reinforcement learning models.

Distinct brain areas process novel and repeating tone sequences

The auditory dorsal stream has been implicated in sensorimotor integration and concatenation of sequential sound events, both being important for processing of speech and music. The auditory ventral stream, by contrast, is characterized as subserving sound identification and recognition. We studied the respective roles of the dorsal and ventral streams, including recruitment of basal ganglia and medial temporal lobe structures, in the processing of tone sequence elements. A sequence was presented incrementally across several runs during functional magnetic resonance imaging in humans, and we compared activation by sequence elements when heard for the first time ("novel") versus when the elements were repeating ("familiar"). Our results show a shift in tone-sequence-dependent activation from posterior-dorsal cortical areas and the basal ganglia during the processing of less familiar sequence elements towards anterior and ventral cortical areas and the medial temporal lobe after the encoding of highly familiar sequence elements into identifiable auditory objects.

Modulation of associative learning in the hippocampal-striatal circuit based on item-set similarity

Mounting evidence suggests that the medial temporal lobe (MTL) and striatal learning systems support different forms of learning, which can be competitive or cooperative depending on task demands. We have previously shown how activity in these regions can be modulated in a conditional visuomotor associative learning task based on the consistency of response mappings or reward feedback (Mattfeld & Stark, 2015). Here, we examined the shift in learning towards the MTL and away from the striatum by placing strong demands on pattern separation, a process of orthogonalizing similar inputs into distinct representations. Mnemonically, pattern separation processes have been shown to rely heavily on processing in the hippocampus. Therefore, we predicted modulation of hippocampal activity by pattern separation demands, but no such modulation of striatal activity. Using a variant of the conditional visuomotor associative learning task that we have used previously, we presented participants with two blocked conditions: items with high and low perceptual overlap during functional magnetic resonance imaging (fMRI). As predicted, we observed learning-related activity in the hippocampus, which was greater in the high than the low overlap condition, particularly in the dentate gyrus. In contrast, the associative striatum also showed learning related activity, but it was not modulated by overlap condition. Using functional connectivity analyses, we showed that the correlation between the hippocampus and dentate gyrus with the associative striatum was differentially modulated by high vs. low overlap, suggesting that the coordination between these regions was affected when pattern separation demands were high. These findings contribute to a growing literature that suggests that the hippocampus and striatal network both contribute to the learning of arbitrary associations that are computationally distinct and can be altered by task demands.

Forget-me-some: General versus special purpose models in a hierarchical probabilistic task

Humans build models of their environments and act according to what they have learnt. In simple experimental environments, such model-based behaviour is often well accounted for as if subjects are ideal Bayesian observers. However, more complex probabilistic tasks require more sophisticated forms of inference that are sufficiently computationally and statistically taxing as to demand approximation. Here, we study properties of two approximation schemes in the context of a serial reaction time task in which stimuli were generated from a hierarchical Markov chain. One, pre-existing, scheme was a generically powerful variational method for hierarchical inference which has recently become popular as an account of psychological and neural data across a wide swathe of probabilistic tasks. A second, novel, scheme was more specifically tailored to the task at hand. We show that the latter model fit significantly better than the former. This suggests that our subjects were sensitive to many of the particular constraints of a complex behavioural task. Further, the tailored model provided a different perspective on the effects of cholinergic manipulations in the task. Neither model fit the behaviour on more complex contingencies that competently. These results illustrate the benefits and challenges that come with the general and special purpose modelling approaches and raise important questions of how they can advance our current understanding of learning mechanisms in the brain.

New insights into statistical learning and chunk learning in implicit sequence acquisition

Implicit sequence learning is ubiquitous in our daily life. However, it is unclear whether the initial acquisition of sequences results from learning to chunk items (i.e., chunk learning) or learning the underlying statistical regularities (i.e., statistical learning). By grouping responses with or without a distinct chunk or statistical structure into segments and comparing these responses, previous studies have demonstrated both chunk and statistical learning. However, few studies have considered the response sequence as a whole and examined the temporal dependency of the entire sequence, where the temporal dependencies could disclose the internal representations of chunk and statistical learning. Participants performed a serial reaction time (SRT) task under different stimulus interval conditions. We found that sequence learning reflected by reaction time (RT) rather than motor improvements represented by movement time (MT). The temporal dependency of RT and MT revealed that both RT and MT displayed recursive patterns caused by biomechanical effects of response locations and foot transitions. Chunking was noticeable only in the presence of the recurring RT or MT but vanished after the recursive component was removed, implying that chunk formation may result from biomechanical constraints rather than learning itself. In addition, we observed notable first-order autocorrelations in RT. This trial-to-trial association enhanced as learning progressed regardless of stimulus intervals, reflecting the internal cognitive representation of the first-order stimulus contingencies. Our results suggest that initial acquisition of implicit sequences may arise from first-order statistical learning rather than chunk learning.

Manipulating memory efficacy affects the behavioral and neural profiles of deterministic learning and decision-making

When making a decision, we have to identify, collect, and evaluate relevant bits of information to ensure an optimal outcome. How we approach a given choice can be influenced by prior experience. Contextual factors and structural elements of these past decisions can cause a shift in how information is encoded and can in turn influence later decision-making. In this two-experiment study, we sought to manipulate declarative memory efficacy and decision-making in a concurrent discrimination learning task by altering the amount of information to be learned. Subjects learned correct responses to pairs of items across several repetitions of a 50- or 100-pair set and were tested for memory retention. In one experiment, this memory test interrupted learning after an initial encoding experience in order to test for early encoding differences and associate those differences with changes in decision-making. In a second experiment, we used fMRI to probe neural differences between the two list-length groups related to decision-making across learning and assessed subsequent memory retention. We found that a striatum-based system was associated with decision-making patterns when learning a longer list of items, while a medial cortical network was associated with patterns when learning a shorter list. Additionally, the hippocampus was exclusively active for the shorter list group. Altogether, these behavioral, computational, and imaging results provide evidence that multiple types of mnemonic representations contribute to experienced-based decision-making. Moreover, contextual and structural factors of the task and of prior decisions can influence what types of evidence are drawn upon during decision-making.

Predictive Movements and Human Reinforcement Learning of Sequential Action

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse movement trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model.

The Role of Working Memory in the Probabilistic Inference of Future Sensory Events

The ability to represent the emerging regularity of sensory information from the external environment has been thought to allow one to probabilistically infer future sensory occurrences and thus optimize behavior. However, the underlying neural implementation of this process is still not comprehensively understood. Through a convergence of behavioral and neurophysiological evidence, we establish that the probabilistic inference of future events is critically linked to people's ability to maintain the recent past in working memory. Magnetoencephalography recordings demonstrated that when visual stimuli occurring over an extended time series had a greater statistical regularity, individuals with higher working-memory capacity (WMC) displayed enhanced slow-wave neural oscillations in the θ frequency band (4-8 Hz.) prior to, but not during stimulus appearance. This prestimulus neural activity was specifically linked to contexts where information could be anticipated and influenced the preferential sensory processing for this visual information after its appearance. A separate behavioral study demonstrated that this process intrinsically emerges during continuous perception and underpins a realistic advantage for efficient behavioral responses. In this way, WMC optimizes the anticipation of higher level semantic concepts expected to occur in the near future.

Cross-modal and non-monotonic representations of statistical regularity are encoded in local neural response patterns

Current neurobiological models assign a central role to predictive processes calibrated to environmental statistics. Neuroimaging studies examining the encoding of stimulus uncertainty have relied almost exclusively on manipulations in which stimuli were presented in a single sensory modality, and further assumed that neural responses vary monotonically with uncertainty. This has left a gap in theoretical development with respect to two core issues: (i) are there cross-modal brain systems that encode input uncertainty in way that generalizes across sensory modalities, and (ii) are there brain systems that track input uncertainty in a non-monotonic fashion? We used multivariate pattern analysis to address these two issues using auditory, visual and audiovisual inputs. We found signatures of cross-modal encoding in frontoparietal, orbitofrontal, and association cortices using a searchlight cross-classification analysis where classifiers trained to discriminate levels of uncertainty in one modality were tested in another modality. Additionally, we found widespread systems encoding uncertainty non-monotonically using classifiers trained to discriminate intermediate levels of uncertainty from both the highest and lowest uncertainty levels. These findings comprise the first comprehensive report of cross-modal and non-monotonic neural sensitivity to statistical regularities in the environment, and suggest that conventional paradigms testing for monotonic responses to uncertainty in a single sensory modality may have limited generalizability.

Parsing the Role of the Hippocampus in Approach-Avoidance Conflict

The hippocampus plays a central role in the approach-avoidance conflict that is central to the genesis of anxiety. However, its exact functional contribution has yet to be identified. We designed a novel gambling task that generated approach-avoidance conflict while controlling for spatial processing. We fit subjects' behavior using a model that quantified the subjective values of choice options, and recorded neural signals using functional magnetic resonance imaging (fMRI). Distinct functional signals were observed in anterior hippocampus, with inferior hippocampus selectively recruited when subjects rejected a gamble, to a degree that covaried with individual differences in anxiety. The superior anterior hippocampus, in contrast, uniquely demonstrated value signals that were potentiated in the context of approach-avoidance conflict. These results implicate the anterior hippocampus in behavioral avoidance and choice monitoring, in a manner relevant to understanding its role in anxiety. Our findings highlight interactions between subregions of the hippocampus as an important focus for future study.

Lost in Time and Space: States of High Arousal Disrupt Implicit Acquisition of Spatial and Sequential Context Information

Biased cognition during high arousal states is a relevant phenomenon in a variety of topics: from the development of post-traumatic stress disorders or stress-triggered addictive behaviors to forensic considerations regarding crimes of passion. Recent evidence indicates that arousal modulates the engagement of a hippocampus-based "cognitive" system in favor of a striatum-based "habit" system in learning and memory, promoting a switch from flexible, contextualized to more rigid, reflexive responses. Existing findings appear inconsistent, therefore it is unclear whether and which type of context processing is disrupted by enhanced arousal. In this behavioral study, we investigated such arousal-triggered cognitive-state shifts in human subjects. We validated an arousal induction procedure (three experimental conditions: violent scene, erotic scene, neutral control scene) using pupillometry (Preliminary Experiment, n = 13) and randomly administered this method to healthy young adults to examine whether high arousal states affect performance in two core domains of contextual processing, the acquisition of spatial (spatial discrimination paradigm; Experiment 1, n = 66) and sequence information (learned irrelevance paradigm; Experiment 2, n = 84). In both paradigms, spatial location and sequences were encoded incidentally and both displacements when retrieving spatial position as well as the predictability of the target by a cue in sequence learning changed stepwise. Results showed that both implicit spatial and sequence learning were disrupted during high arousal states, regardless of valence. Compared to the control group, participants in the arousal conditions showed impaired discrimination of spatial positions and abolished learning of associative sequences. Furthermore, Bayesian analyses revealed evidence against the null models. In line with recent models of stress effects on cognition, both experiments provide evidence for decreased engagement of flexible, cognitive systems supporting encoding of context information in active cognition during acute arousal, promoting reduced sensitivity for contextual details. We argue that arousal fosters cognitive adaptation towards less demanding, more present-oriented information processing, which prioritizes a current behavioral response set at the cost of contextual cues. This transient state of behavioral perseverance might reduce reliance on context information in unpredictable environments and thus represent an adaptive response in certain situations.

Predicting the Past, Remembering the Future

Rational analyses of memory suggest that retrievability of past experience depends on its usefulness for predicting the future: memory is adapted to the temporal structure of the environment. Recent research has enriched this view by applying it to semantic memory and reinforcement learning. This paper describes how multiple forms of memory can be linked via common predictive principles, possibly subserved by a shared neural substrate in the hippocampus. Predictive principles offer an explanation for a wide range of behavioral and neural phenomena, including semantic fluency, temporal contiguity effects in episodic memory, and the topological properties of hippocampal place cells.

Learning Predictive Statistics: Strategies and Brain Mechanisms

When immersed in a new environment, we are challenged to decipher initially incomprehensible streams of sensory information. However, quite rapidly, the brain finds structure and meaning in these incoming signals, helping us to predict and prepare ourselves for future actions. This skill relies on extracting the statistics of event streams in the environment that contain regularities of variable complexity from simple repetitive patterns to complex probabilistic combinations. Here, we test the brain mechanisms that mediate our ability to adapt to the environment's statistics and predict upcoming events. By combining behavioral training and multisession fMRI in human participants (male and female), we track the corticostriatal mechanisms that mediate learning of temporal sequences as they change in structure complexity. We show that learning of predictive structures relates to individual decision strategy; that is, selecting the most probable outcome in a given context (maximizing) versus matching the exact sequence statistics. These strategies engage distinct human brain regions: maximizing engages dorsolateral prefrontal, cingulate, sensory-motor regions, and basal ganglia (dorsal caudate, putamen), whereas matching engages occipitotemporal regions (including the hippocampus) and basal ganglia (ventral caudate). Our findings provide evidence for distinct corticostriatal mechanisms that facilitate our ability to extract behaviorally relevant statistics to make predictions.SIGNIFICANCE STATEMENT Making predictions about future events relies on interpreting streams of information that may initially appear incomprehensible. Past work has studied how humans identify repetitive patterns and associative pairings. However, the natural environment contains regularities that vary in complexity from simple repetition to complex probabilistic combinations. Here, we combine behavior and multisession fMRI to track the brain mechanisms that mediate our ability to adapt to changes in the environment's statistics. We provide evidence for an alternate route for learning complex temporal statistics: extracting the most probable outcome in a given context is implemented by interactions between executive and motor corticostriatal mechanisms compared with visual corticostriatal circuits (including hippocampal cortex) that support learning of the exact temporal statistics.

Reminders of past choices bias decisions for reward in humans

We provide evidence that decisions are made by consulting memories for individual past experiences, and that this process can be biased in favour of past choices using incidental reminders. First, in a standard rewarded choice task, we show that a model that estimates value at decision-time using individual samples of past outcomes fits choices and decision-related neural activity better than a canonical incremental learning model. In a second experiment, we bias this sampling process by incidentally reminding participants of individual past decisions. The next decision after a reminder shows a strong influence of the action taken and value received on the reminded trial. These results provide new empirical support for a decision architecture that relies on samples of individual past choice episodes rather than incrementally averaged rewards in evaluating options and has suggestive implications for the underlying cognitive and neural mechanisms.

Brain networks for confidence weighting and hierarchical inference during probabilistic learning

Learning is difficult when the world fluctuates randomly and ceaselessly. Classical learning algorithms, such as the delta rule with constant learning rate, are not optimal. Mathematically, the optimal learning rule requires weighting prior knowledge and incoming evidence according to their respective reliabilities. This "confidence weighting" implies the maintenance of an accurate estimate of the reliability of what has been learned. Here, using fMRI and an ideal-observer analysis, we demonstrate that the brain's learning algorithm relies on confidence weighting. While in the fMRI scanner, human adults attempted to learn the transition probabilities underlying an auditory or visual sequence, and reported their confidence in those estimates. They knew that these transition probabilities could change simultaneously at unpredicted moments, and therefore that the learning problem was inherently hierarchical. Subjective confidence reports tightly followed the predictions derived from the ideal observer. In particular, subjects managed to attach distinct levels of confidence to each learned transition probability, as required by Bayes-optimal inference. Distinct brain areas tracked the likelihood of new observations given current predictions, and the confidence in those predictions. Both signals were combined in the right inferior frontal gyrus, where they operated in agreement with the confidence-weighting model. This brain region also presented signatures of a hierarchical process that disentangles distinct sources of uncertainty. Together, our results provide evidence that the sense of confidence is an essential ingredient of probabilistic learning in the human brain, and that the right inferior frontal gyrus hosts a confidence-based statistical learning algorithm for auditory and visual sequences.

Simulating future value in intertemporal choice

The laboratory study of how humans and other animals trade-off value and time has a long and storied history, and is the subject of a vast literature. However, despite a long history of study, there is no agreed upon mechanistic explanation of how intertemporal choice preferences arise. Several theorists have recently proposed model-based reinforcement learning as a candidate framework. This framework describes a suite of algorithms by which a model of the environment, in the form of a state transition function and reward function, can be converted on-line into a decision. The state transition function allows the model-based system to make decisions based on projected future states, while the reward function assigns value to each state, together capturing the necessary components for successful intertemporal choice. Empirical work has also pointed to a possible relationship between increased prospection and reduced discounting. In the current paper, we look for direct evidence of a relationship between temporal discounting and model-based control in a large new data set (n = 168). However, testing the relationship under several different modeling formulations revealed no indication that the two quantities are related.

Children and Adults Both Learn Motor Sequences Quickly, But Do So Differently

Both children and adults can learn motor sequences quickly in one learning session, yet little is known about potential age-related processes that underlie this fast sequence acquisition. Here, we examined the progressive performance changes in a one-session modified serial reaction time task in 6- and 10-year-old children and adults. We found that rapid sequence learning, as reflected by reaction time (RT), was comparable between groups. The learning was expressed through two behavioral processes: online progressive changes in RT while the task was performed in a continuous manner and offline changes in RT that emerged following a short rest. These offline and online RT changes were age-related; learning in 6-year-olds was primarily reflected through the offline process. In contrast, learning in adults was reflected through the online process; and both online and offline processes occurred concurrently in 10-year-olds. Our results suggest that early rapid sequence learning has a developmental profile. Although the unifying mechanism underlying these two age-related processes is unclear, we discuss possible explanations that need to be systematically elucidated in future studies.

The neurobiology of uncertainty: implications for statistical learning

The capacity for assessing the degree of uncertainty in the environment relies on estimating statistics of temporally unfolding inputs. This, in turn, allows calibration of predictive and bottom-up processing, and signalling changes in temporally unfolding environmental features. In the last decade, several studies have examined how the brain codes for and responds to input uncertainty. Initial neurobiological experiments implicated frontoparietal and hippocampal systems, based largely on paradigms that manipulated distributional features of visual stimuli. However, later work in the auditory domain pointed to different systems, whose activation profiles have interesting implications for computational and neurobiological models of statistical learning (SL). This review begins by briefly recapping the historical development of ideas pertaining to the sensitivity to uncertainty in temporally unfolding inputs. It then discusses several issues at the interface of studies of uncertainty and SL. Following, it presents several current treatments of the neurobiology of uncertainty and reviews recent findings that point to principles that serve as important constraints on future neurobiological theories of uncertainty, and relatedly, SL. This review suggests it may be useful to establish closer links between neurobiological research on uncertainty and SL, considering particularly mechanisms sensitive to local and global structure in inputs, the degree of input uncertainty, the complexity of the system generating the input, learning mechanisms that operate on different temporal scales and the use of learnt information for online prediction.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning

A growing literature suggests that the hippocampus is critical for the rapid extraction of regularities from the environment. Although this fits with the known role of the hippocampus in rapid learning, it seems at odds with the idea that the hippocampus specializes in memorizing individual episodes. In particular, the Complementary Learning Systems theory argues that there is a computational trade-off between learning the specifics of individual experiences and regularities that hold across those experiences. We asked whether it is possible for the hippocampus to handle both statistical learning and memorization of individual episodes. We exposed a neural network model that instantiates known properties of hippocampal projections and subfields to sequences of items with temporal regularities. We found that the monosynaptic pathway-the pathway connecting entorhinal cortex directly to region CA1-was able to support statistical learning, while the trisynaptic pathway-connecting entorhinal cortex to CA1 through dentate gyrus and CA3-learned individual episodes, with apparent representations of regularities resulting from associative reactivation through recurrence. Thus, in paradigms involving rapid learning, the computational trade-off between learning episodes and regularities may be handled by separate anatomical pathways within the hippocampus itself.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Prefrontal Cortex Activation Reflects Efficient Exploitation of Higher-order Statistical Structure

Because everyday actions are statistically structured, knowing which action a person has just completed allows predicting the most likely next action step. Taking even more than the preceding action into account improves this predictability but also causes higher processing costs. Using fMRI, we investigated whether observers exploit second-order statistical regularities preferentially if information on possible upcoming actions provided by first-order regularities is insufficient. We hypothesized that anterior pFC balances whether or not second-order information should be exploited. Participants watched videos of actions that were structured by first- and second-order conditional probabilities. Information provided by the first and by the second order was manipulated independently. BOLD activity in the action observation network was more attenuated the more information on upcoming actions was provided by first-order structure, reflecting expectation suppression for more predictable actions. Activation in posterior parietal sites decreased further with second-order information but increased in temporal areas. As expected, second-order information was integrated more when less first-order information was provided, and this interaction was mediated by anterior pFC (BA 10). Observers spontaneously used both the present and the preceding action to predict the upcoming action, and integration of the preceding action was enhanced when the present action was uninformative.