Knowledge acquisition is governed by striatal prediction errors

The neuroscience of active learning and direct instruction

Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.

Impaired flexible reward learning in ADHD patients is associated with blunted reinforcement sensitivity and neural signals in ventral striatum and parietal cortex

Reward-based learning and decision-making are prime candidates to understand symptoms of attention deficit hyperactivity disorder (ADHD). However, only limited evidence is available regarding the neurocomputational underpinnings of the alterations seen in ADHD. This concerns flexible behavioral adaption in dynamically changing environments, which is challenging for individuals with ADHD. One previous study points to elevated choice switching in adolescent ADHD, which was accompanied by disrupted learning signals in medial prefrontal cortex. Here, we investigated young adults with ADHD (n = 17) as compared to age- and sex-matched controls (n = 17) using a probabilistic reversal learning experiment during functional magnetic resonance imaging (fMRI). The task requires continuous learning to guide flexible behavioral adaptation to changing reward contingencies. To disentangle the neurocomputational underpinnings of the behavioral data, we used reinforcement learning (RL) models, which informed the analysis of fMRI data. ADHD patients performed worse than controls particularly in trials before reversals, i.e., when reward contingencies were stable. This pattern resulted from 'noisy' choice switching regardless of previous feedback. RL modelling showed decreased reinforcement sensitivity and enhanced learning rates for negative feedback in ADHD patients. At the neural level, this was reflected in a diminished representation of choice probability in the left posterior parietal cortex in ADHD. Moreover, modelling showed a marginal reduction of learning about the unchosen option, which was paralleled by a marginal reduction in learning signals incorporating the unchosen option in the left ventral striatum. Taken together, we show that impaired flexible behavior in ADHD is due to excessive choice switching ('hyper-flexibility'), which can be detrimental or beneficial depending on the learning environment. Computationally, this resulted from blunted sensitivity to reinforcement of which we detected neural correlates in the attention-control network, specifically in the parietal cortex. These neurocomputational findings remain preliminary due to the relatively small sample size.

Long-term, multi-event surprise correlates with enhanced autobiographical memory

Neurobiological and psychological models of learning emphasize the importance of prediction errors (surprises) for memory formation. This relationship has been shown for individual momentary surprising events; however, it is less clear whether surprise that unfolds across multiple events and timescales is also linked with better memory of those events. We asked basketball fans about their most positive and negative autobiographical memories of individual plays, games and seasons, allowing surprise measurements spanning seconds, hours and months. We used advanced analytics on National Basketball Association play-by-play data and betting odds spanning 17 seasons, more than 22,000 games and more than 5.6 million plays to compute and align the estimated surprise value of each memory. We found that surprising events were associated with better recall of positive memories on the scale of seconds and months and negative memories across all three timescales. Game and season memories could not be explained by surprise at shorter timescales, suggesting that long-term, multi-event surprise correlates with memory. These results expand notions of surprise in models of learning and reinforce its relevance in real-world domains.

Prediction error and memory across the lifespan

The influence of Prediction Errors (PEs) on episodic memory has generated growing empirical and theoretical interest. This review explores how the relationship between PE and memory may evolve throughout lifespan. Drawing upon the predictive processing framework and the Predictive, Interactive Multiple Memory System (PIMMS) model in particular, the paper highlights the hierarchical organization of memory systems and the interaction between top-down predictions and bottom-up sensory input, proposing that PEs promote synaptic change and improve encoding and consolidation processes. We discuss the neuroscientific mechanisms underlying PE-driven memory enhancement, focusing on the involvement of the hippocampus, the entorhinal cortex-hippocampus pathway, and the noradrenergic sympathetic system. Recognizing the divergent trajectories of episodic and semantic memory across the lifespan is crucial when examining the effects of PEs on memory. This review underscores the heterogeneity of memory processes and neurocognitive mechanisms underlying PE-driven memory enhancement across age. Future research is suggested to directly compare neural networks involved in learning from PEs across different age groups and to contribute to a deeper understanding of PE-driven learning across age.

Remembering unexpected beauty: Contributions of the ventral striatum to the processing of reward prediction errors regarding the facial attractiveness in face memory

The COVID-19 pandemic has led people to predict facial attractiveness from partially covered faces. Differences in the predicted and observed facial attractiveness (i.e., masked and unmasked faces, respectively) are defined as reward prediction error (RPE) in a social context. Cognitive neuroscience studies have elucidated the neural mechanisms underlying RPE-induced memory improvements in terms of monetary rewards. However, little is known about the mechanisms underlying RPE-induced memory modulation in terms of social rewards. To elucidate this, the present functional magnetic resonance imaging (fMRI) study investigated activity and functional connectivity during face encoding. In encoding trials, participants rated the predicted attractiveness of faces covered except for around the eyes (prediction phase) and then rated the observed attractiveness of these faces without any cover (outcome phase). The difference in ratings between these phases was defined as RPE in facial attractiveness, and RPE was categorized into positive RPE (increased RPE from the prediction to outcome phases), negative RPE (decreased RPE from the prediction to outcome phases), and non-RPE (no difference in RPE between the prediction and outcome phases). During retrieval, participants were presented with individual faces that had been seen and unseen in the encoding trials, and were required to judge whether or not each face had been seen in the encoding trials. Univariate activity in the ventral striatum (VS) exhibited a linear increase with increased RPE in facial attractiveness. In the multivariate pattern analysis (MVPA), activity patterns in the VS and surrounding areas (extended VS) significantly discriminated between positive/negative RPE and non-RPE. In the functional connectivity analysis, significant functional connectivity between the extended VS and the hippocampus was observed most frequently in positive RPE. Memory improvements by face-based RPE could be involved in functional networks between the extended VS (representing RPE) and the hippocampus, and the interaction could be modulated by RPE values in a social context.

Beta Oscillations in Monkey Striatum Encode Reward Prediction Error Signals

Reward prediction error (RPE) signals are crucial for reinforcement learning and decision-making as they quantify the mismatch between predicted and obtained rewards. RPE signals are encoded in the neural activity of multiple brain areas, such as midbrain dopaminergic neurons, prefrontal cortex, and striatum. However, it remains unclear how these signals are expressed through anatomically and functionally distinct subregions of the striatum. In the current study, we examined to which extent RPE signals are represented across different striatal regions. To do so, we recorded local field potentials (LFPs) in sensorimotor, associative, and limbic striatal territories of two male rhesus monkeys performing a free-choice probabilistic learning task. The trial-by-trial evolution of RPE during task performance was estimated using a reinforcement learning model fitted on monkeys' choice behavior. Overall, we found that changes in beta band oscillations (15-35 Hz), after the outcome of the animal's choice, are consistent with RPE encoding. Moreover, we provide evidence that the signals related to RPE are more strongly represented in the ventral (limbic) than dorsal (sensorimotor and associative) part of the striatum. To conclude, our results suggest a relationship between striatal beta oscillations and the evaluation of outcomes based on RPE signals and highlight a major contribution of the ventral striatum to the updating of learning processes.SIGNIFICANCE STATEMENT Reward prediction error (RPE) signals are crucial for reinforcement learning and decision-making as they quantify the mismatch between predicted and obtained rewards. Current models suggest that RPE signals are encoded in the neural activity of multiple brain areas, including the midbrain dopaminergic neurons, prefrontal cortex and striatum. However, it remains elusive whether RPEs recruit anatomically and functionally distinct subregions of the striatum. Our study provides evidence that RPE-related modulations in local field potential (LFP) power are dominant in the striatum. In particular, they are stronger in the rostro-ventral rather than the caudo-dorsal striatum. Our findings contribute to a better understanding of the role of striatal territories in reward-based learning and may be relevant for neuropsychiatric and neurologic diseases that affect striatal circuits.

Contextual incongruency triggers memory reinstatement and the disruption of neural stability

Schemas, or internal representation models of the environment, are thought to be central in organising our everyday life behaviour by giving stability and predictiveness to the structure of the world. However, when an element from an unfolding event mismatches the schema-derived expectations, the coherent narrative is interrupted and an update to the current event model representation is required. Here, we asked whether the perceived incongruence of an item from an unfolding event and its impact on memory relied on the disruption of neural stability patterns preceded by the neural reactivation of the memory representations of the just-encoded event. Our study includes data from two different experiments whereby human participants (N = 33, 26 females and N = 18, 16 females, respectively) encoded images of objects preceded by trial-unique sequences of events depicting daily routine. We found that neural stability patterns gradually increased throughout the ongoing exposure to a schema-consistent episode, which was corroborated by the re-analysis of data from two other experiments, and that the brain stability pattern was interrupted when the encoding of an object of the event was incongruent with the ongoing schema. We found that the decrease in neural stability for low-congruence items was seen at ∼1000 ms from object encoding onset and that it was preceded by an enhanced N400 ERP and an increased degree of neural reactivation of the just-encoded episode. Current results offer new insights into the neural mechanisms and their temporal orchestration that are engaged during online encoding of schema-consistent episodic narratives and the detection of incongruencies.

Patterns of retrieval-related cortico-striatal connectivity are stable across the adult lifespan

Memory retrieval effects in the striatum are well documented and robust across experimental paradigms. However, the functional significance of these effects, and whether they are moderated by age, remains unclear. We used functional magnetic resonance imaging paired with an associative recognition task to examine retrieval effects in the striatum in a sample of healthy young, middle-aged, and older adults. We identified anatomically segregated patterns of enhanced striatal blood oxygen level-dependent (BOLD) activity during recollection- and familiarity-based memory judgments. Successful recollection was associated with enhanced BOLD activity in bilateral putamen and nucleus accumbens, and neither of these effects were reliably moderated by age. Familiarity effects were evident in the head of the caudate nucleus bilaterally, and these effects were attenuated in middle-aged and older adults. Using psychophysiological interaction analyses, we observed a monitoring-related increase in functional connectivity between the caudate and regions of the frontoparietal control network, and between the putamen and bilateral retrosplenial cortex and intraparietal sulcus. In all instances, monitoring-related increases in cortico-striatal connectivity were unmoderated by age. These results suggest that the striatum, and the caudate in particular, couples with the frontoparietal control network to support top-down retrieval-monitoring operations, and that the strength of these inter-regional interactions is preserved in later life.

Probabilistic Decision-Making in Children With Dyslexia

Background

Neurocognitive mechanisms underlying developmental dyslexia (dD) remain poorly characterized apart from phonological and/or visual processing deficits. Assuming such deficits, the process of learning complex tasks like reading requires the learner to make decisions (i.e., word pronunciation) based on uncertain information (e.g., aberrant phonological percepts)-a cognitive process known as probabilistic decision making, which has been linked to the striatum. We investigate (1) the relationship between dD and probabilistic decision-making and (2) the association between the volume of striatal structures and probabilistic decision-making in dD and typical readers.

Methods

Twenty four children diagnosed with dD underwent a comprehensive evaluation and MRI scanning (3T). Children with dD were compared to age-matched typical readers (n = 11) on a probabilistic, risk/reward fishing task that utilized a Bayesian cognitive model with game parameters of risk propensity (γ+) and behavioral consistency (β), as well as an overall adjusted score (average number of casts, excluding forced-fail trials). Volumes of striatal structures (caudate, putamen, and nucleus accumbens) were analyzed between groups and associated with game parameters.

Results

dD was associated with greater risk propensity and decreased behavioral consistency estimates compared to typical readers. Cognitive model parameters associated with timed pseudoword reading across groups. Risk propensity related to caudate volumes, particularly in the dD group.

Conclusion

Decision-making processes differentiate dD, associate with the caudate, and may impact learning mechanisms. This study suggests the need for further research into domain-general probabilistic decision-making in dD, neurocognitive mechanisms, and targeted interventions in dD.

A neural network model of when to retrieve and encode episodic memories

Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.

Fractal-Based Analysis of fMRI BOLD Signal During Naturalistic Viewing Conditions

Background: Temporal fractals are characterized by prominent scale-invariance and self-similarity across time scales. Monofractal analysis quantifies this scaling behavior in a single parameter, the Hurst exponent (H). Higher H reflects greater correlation in the signal structure, which is taken as being more fractal. Previous fMRI studies have observed lower H during conventional tasks relative to resting state conditions, and shown that H is negatively correlated with task difficulty and novelty. To date, no study has investigated the fractal dynamics of BOLD signal during naturalistic conditions. Methods: We performed fractal analysis on Human Connectome Project 7T fMRI data (n = 72, 41 females, mean age 29.46 ± 3.76 years) to compare H across movie-watching and rest. Results: In contrast to previous work using conventional tasks, we found higher H values for movie relative to rest (mean difference = 0.014; p = 5.279 × 10^-7; 95% CI [0.009, 0.019]). H was significantly higher in movie than rest in the visual, somatomotor and dorsal attention networks, but was significantly lower during movie in the frontoparietal and default networks. We found no cross-condition differences in test-retest reliability of H. Finally, we found that H of movie-derived stimulus properties (e.g., luminance changes) were fractal whereas H of head motion estimates were non-fractal. Conclusions: Overall, our findings suggest that movie-watching induces fractal signal dynamics. In line with recent work characterizing connectivity-based brain state dynamics during movie-watching, we speculate that these fractal dynamics reflect the configuring and reconfiguring of brain states that occurs during naturalistic processing, and are markedly different than dynamics observed during conventional tasks.

Prediction errors disrupt hippocampal representations and update episodic memories

The brain supports adaptive behavior by generating predictions, learning from errors, and updating memories to incorporate new information. Prediction error, or surprise, triggers learning when reality contradicts expectations. Prior studies have shown that the hippocampus signals prediction errors, but the hypothesized link to memory updating has not been demonstrated. In a human functional MRI study, we elicited mnemonic prediction errors by interrupting familiar narrative videos immediately before the expected endings. We found that prediction errors reversed the relationship between univariate hippocampal activation and memory: greater hippocampal activation predicted memory preservation after expected endings, but memory updating after surprising endings. In contrast to previous studies, we show that univariate activation was insufficient for understanding hippocampal prediction error signals. We explain this surprising finding by tracking both the evolution of hippocampal activation patterns and the connectivity between the hippocampus and neuromodulatory regions. We found that hippocampal activation patterns stabilized as each narrative episode unfolded, suggesting sustained episodic representations. Prediction errors disrupted these sustained representations and the degree of disruption predicted memory updating. The relationship between hippocampal activation and subsequent memory depended on concurrent basal forebrain activation, supporting the idea that cholinergic modulation regulates attention and memory. We conclude that prediction errors create conditions that favor memory updating, prompting the hippocampus to abandon ongoing predictions and make memories malleable.

Pairing facts with imagined consequences improves pandemic-related risk perception

The COVID-19 pandemic reached staggering new peaks during a global resurgence more than a year after the crisis began. Although public health guidelines initially helped to slow the spread of disease, widespread pandemic fatigue and prolonged harm to financial stability and mental well-being contributed to this resurgence. In the late stage of the pandemic, it became clear that new interventions were needed to support long-term behavior change. Here, we examined subjective perceived risk about COVID-19 and the relationship between perceived risk and engagement in risky behaviors. In study 1 (n = 303), we found that subjective perceived risk was likely inaccurate but predicted compliance with public health guidelines. In study 2 (n = 735), we developed a multifaceted intervention designed to realign perceived risk with actual risk. Participants completed an episodic simulation task; we expected that imagining a COVID-related scenario would increase the salience of risk information and enhance behavior change. Immediately following the episodic simulation, participants completed a risk estimation task with individualized feedback about local viral prevalence. We found that information prediction error, a measure of surprise, drove beneficial change in perceived risk and willingness to engage in risky activities. Imagining a COVID-related scenario beforehand enhanced the effect of prediction error on learning. Importantly, our intervention produced lasting effects that persisted after a 1- to 3-wk delay. Overall, we describe a fast and feasible online intervention that effectively changed beliefs and intentions about risky behaviors.

Pairing facts with imagined consequences improves pandemic-related risk perception

The COVID-19 pandemic reached staggering new peaks during a global resurgence more than a year after the crisis began. Although public health guidelines initially helped to slow the spread of disease, widespread pandemic fatigue and prolonged harm to financial stability and mental well-being contributed to this resurgence. In the late stage of the pandemic, it became clear that new interventions were needed to support long-term behavior change. Here, we examined subjective perceived risk about COVID-19 and the relationship between perceived risk and engagement in risky behaviors. In study 1 (n = 303), we found that subjective perceived risk was likely inaccurate but predicted compliance with public health guidelines. In study 2 (n = 735), we developed a multifaceted intervention designed to realign perceived risk with actual risk. Participants completed an episodic simulation task; we expected that imagining a COVID-related scenario would increase the salience of risk information and enhance behavior change. Immediately following the episodic simulation, participants completed a risk estimation task with individualized feedback about local viral prevalence. We found that information prediction error, a measure of surprise, drove beneficial change in perceived risk and willingness to engage in risky activities. Imagining a COVID-related scenario beforehand enhanced the effect of prediction error on learning. Importantly, our intervention produced lasting effects that persisted after a 1- to 3-wk delay. Overall, we describe a fast and feasible online intervention that effectively changed beliefs and intentions about risky behaviors.

Pairing facts with imagined consequences improves pandemic-related risk perception

The COVID-19 pandemic reached staggering new peaks during a global resurgence more than a year after the crisis began. Although public health guidelines initially helped to slow the spread of disease, widespread pandemic fatigue and prolonged harm to financial stability and mental well-being contributed to this resurgence. In the late stage of the pandemic, it became clear that new interventions were needed to support long-term behavior change. Here, we examined subjective perceived risk about COVID-19 and the relationship between perceived risk and engagement in risky behaviors. In study 1 (n = 303), we found that subjective perceived risk was likely inaccurate but predicted compliance with public health guidelines. In study 2 (n = 735), we developed a multifaceted intervention designed to realign perceived risk with actual risk. Participants completed an episodic simulation task; we expected that imagining a COVID-related scenario would increase the salience of risk information and enhance behavior change. Immediately following the episodic simulation, participants completed a risk estimation task with individualized feedback about local viral prevalence. We found that information prediction error, a measure of surprise, drove beneficial change in perceived risk and willingness to engage in risky activities. Imagining a COVID-related scenario beforehand enhanced the effect of prediction error on learning. Importantly, our intervention produced lasting effects that persisted after a 1- to 3-wk delay. Overall, we describe a fast and feasible online intervention that effectively changed beliefs and intentions about risky behaviors.

The basal ganglia and the cerebellum in human emotion

The basal ganglia (BG) and the cerebellum historically have been relegated to a functional role in producing or modulating motor output. Recent research, however, has emphasized the importance of these subcortical structures in multiple functional domains, including affective processes such as emotion recognition, subjective feeling elicitation and reward valuation. The pathways through the thalamus that connect the BG and cerebellum directly to each other and with extensive regions of the cortex provide a structural basis for their combined influence on limbic function. By regulating cortical oscillations to guide learning and strengthening rewarded behaviors or thought patterns to achieve a desired goal state, these regions can shape the way an individual processes emotional stimuli. This review will discuss the basic structure and function of the BG and cerebellum and propose an updated view of their functional role in human affective processing.

Timing matters when correcting fake news

Countering misinformation can reduce belief in the moment, but corrective messages quickly fade from memory. We tested whether the longer-term impact of fact-checks depends on when people receive them. In two experiments (total N = 2,683), participants read true and false headlines taken from social media. In the treatment conditions, “true” and “false” tags appeared before, during, or after participants read each headline. Participants in a control condition received no information about veracity. One week later, participants in all conditions rated the same headlines’ accuracy. Providing fact-checks after headlines (debunking) improved subsequent truth discernment more than providing the same information during (labeling) or before (prebunking) exposure. This finding informs the cognitive science of belief revision and has practical implications for social media platform designers.

Contextual prediction errors reorganize naturalistic episodic memories in time

Episodic memories are contextual experiences ordered in time. This is underpinned by associative binding between events within the same contexts. The role of prediction errors in declarative memory is well established but has not been investigated in the time dimension of complex episodic memories. Here we combine these two properties of episodic memory, extend them into the temporal domain and demonstrate that prediction errors in different naturalistic contexts lead to changes in the temporal ordering of event structures in them. The wrongly predicted older sequences were weakened despite their reactivation. Interestingly the newly encoded sequences with prediction errors, seen once, showed accuracy as high as control sequences which were viewed repeatedly without change. Drift-diffusion modelling revealed a lower decision threshold for the newer sequences than older sequences, reflected by their faster recall. Moreover, participants' adjustments to their decision threshold significantly correlated with their relative speed of sequence memory recall. These results suggest a temporally distinct and adaptive role for prediction errors in learning and reorganizing episodic temporal sequences.

Closed-minded cognition: Right-wing authoritarianism is negatively related to belief updating following prediction error

When confronted with information that challenges our beliefs, we must often learn from error in order to successfully navigate the world. Past studies in reinforcement learning and educational psychology have linked prediction error, a measure of surprise, to successful learning from feedback. However, there are substantial individual differences in belief-updating success, and the psychological factors that influence belief updating remain unclear. Here, we identify a novel factor that may predict belief updating: right-wing authoritarianism (RWA), which is characterized by a desire for order, structure, and preservation of social norms. We hypothesized that because people who score high on RWA are motivated to preserve entrenched beliefs, they may often fail to successfully update their beliefs when confronted with new information. Using a novel paradigm, we challenged participants’ false beliefs and misconceptions to elicit prediction error. In two studies, we found consistent evidence that high-RWA individuals were less successful at correcting their false beliefs. Relative to low-RWA individuals, high-RWA individuals were less likely to revise beliefs in response to prediction error. We argue that RWA is associated with a relatively closed-minded cognitive style that negatively influences belief updating.

A predictive account of how novelty influences declarative memory

A rich body of studies in the human and non-human literature has examined the question how novelty influences memory. For a variety of different stimuli, ranging from simple objects and words to vastly complex scenarios, the literature reports that novelty improves memory in some cases, but impairs memory in other cases. In recent attempts to reconcile these conflicting findings, novelty has been divided into different subtypes, such as relative versus absolute novelty, or stimulus versus contextual novelty. Nevertheless, a single overarching theory of novelty and memory has been difficult to attain, probably due to the complexities in the interactions among stimuli, environmental factors (e.g., spatial and temporal context) and level of prior knowledge (but see Duszkiewicz et al., 2019; Kafkas & Montaldi, 2018b; Schomaker & Meeter, 2015). Here we describe how a predictive coding framework might be able to shed new light on different types of novelty and how they affect declarative memory in humans. More precisely, we consider how prior expectations modulate the influence of novelty on encoding episodes into memory, e.g., in terms of surprise, and how novelty/surprise affect memory for surrounding information. By reviewing a range of behavioural findings and their possible underlying neurobiological mechanisms, we highlight where a predictive coding framework succeeds and where it appears to struggle.

The surprising role of the default mode network in naturalistic perception

The default mode network (DMN) is a group of high-order brain regions recently implicated in processing external naturalistic events, yet it remains unclear what cognitive function it serves. Here we identified the cognitive states predictive of DMN fMRI coactivation. Particularly, we developed a state-fluctuation pattern analysis, matching network coactivations across a short movie with retrospective behavioral sampling of movie events. Network coactivation was selectively correlated with the state of surprise across movie events, compared to all other cognitive states (e.g. emotion, vividness). The effect was exhibited in the DMN, but not dorsal attention or visual networks. Furthermore, surprise was found to mediate DMN coactivations with hippocampus and nucleus accumbens. These unexpected findings point to the DMN as a major hub in high-level prediction-error representations.

Brandman et al compared fMRI coactivations and behavioral responses across participants who watched the same short movie. They demonstrated that the default mode network plays a central role in the representation of surprise – a process which is important for updating internal narrative representations

Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.

Testing the reinforcement learning hypothesis of social conformity

Our preferences are influenced by the opinions of others. The past human neuroimaging studies on social conformity have identified a network of brain regions related to social conformity that includes the posterior medial frontal cortex (pMFC), anterior insula, and striatum. Since these brain regions are also known to play important roles in reinforcement learning (i.e., processing prediction error), it was previously hypothesized that social conformity and reinforcement learning have a common neural mechanism. However, although this view is currently widely accepted, these two processes have never been directly compared; therefore, the extent to which they shared a common neural mechanism had remained unclear. This study aimed to formally test the hypothesis. The same group of participants (n = 25) performed social conformity and reinforcement learning tasks inside a functional magnetic resonance imaging (fMRI) scanner. Univariate fMRI data analyses revealed activation overlaps in the pMFC and bilateral insula between social conflict and unsigned prediction error and in the striatum between social conflict and signed prediction error. We further conducted multivoxel pattern analysis (MVPA) for more direct evidence of a shared neural mechanism. MVPA did not reveal any evidence to support the hypothesis in any of these regions but found that activation patterns between social conflict and prediction error in these regions were largely distinct. Taken together, the present study provides no clear evidence of a common neural mechanism between social conformity and reinforcement learning.

Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing

Surprise signals a discrepancy between past and current beliefs. It is theorized to be linked to affective experiences, the creation of particularly resilient memories, and segmentation of the flow of experience into discrete perceived events. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural correlates in human subjects observing basketball games. We found that surprise was associated with segmentation of ongoing experiences, as reflected by subjectively perceived event boundaries and shifts in neocortical patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, activation in subcortical regions associated with dopamine, game enjoyment, and long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.

States of curiosity and interest enhance memory differently in adolescents and in children

Curiosity - broadly defined as the desire to acquire new information - enhances learning and memory in adults. In addition, interest in the information (i.e., when the information is processed) can also facilitate later memory. To date, it is not known how states of pre-information curiosity and post-information interest enhance memory in childhood and adolescence. We used a trivia paradigm in which children and adolescents (N = 60, 10-14 years) encoded trivia questions and answers associated with high or low curiosity. States of high pre-answer curiosity enhanced later memory for trivia answers in both children and adolescents. However, higher positive post-answer interest enhanced memory for trivia answers beyond the effects of curiosity more strongly in adolescents than in children. These results suggest that curiosity and interest have positive effects on learning and memory in childhood and adolescence, but might need to be harnessed in differential ways across child development to optimize learning.

Dynamic Brain Responses Modulated by Precise Timing Prediction in an Opposing Process

The brain function of prediction is fundamental for human beings to shape perceptions efficiently and successively. Through decades of effort, a valuable brain activation map has been obtained for prediction. However, much less is known about how the brain manages the prediction process over time using traditional neuropsychological paradigms. Here, we implemented an innovative paradigm for timing prediction to precisely study the temporal dynamics of neural oscillations. In the experiment recruiting 45 participants, expectation suppression was found for the overall electroencephalographic activity, consistent with previous hemodynamic studies. Notably, we found that N1 was positively associated with predictability while N2 showed a reversed relation to predictability. Furthermore, the matching prediction had a similar profile with no timing prediction, both showing an almost saturated N1 and an absence of N2. The results indicate that the N1 process showed a 'sharpening' effect for predictable inputs, while the N2 process showed a 'dampening' effect. Therefore, these two paradoxical neural effects of prediction, which have provoked wide confusion in accounting for expectation suppression, actually co-exist in the procedure of timing prediction but work in separate time windows. These findings strongly support a recently-proposed opposing process theory.

Novelty Manipulations, Memory Performance, and Predictive Coding: the Role of Unexpectedness

Novelty is central to the study of memory, but the wide range of experimental manipulations aimed to reveal its effects on learning produced inconsistent results. The novelty/encoding hypothesis suggests that novel information undergoes enhanced encoding and thus leads to benefits in memory, especially in recognition performance; however, recent studies cast doubts on this assumption. On the other hand, data from animal studies provided evidence on the robust effects of novelty manipulations on the neurophysiological correlates of memory processes. Conceptualizations and operationalizations of novelty are remarkably variable and were categorized into different subtypes, such as stimulus, context, associative or spatial novelty. Here, we summarize previous findings about the effects of novelty on memory and suggest that predictive coding theories provide a framework that could shed light on the differential influence of novelty manipulations on memory performance. In line with predictive coding theories, we emphasize the role of unexpectedness as a crucial property mediating the behavioral and neural effects of novelty manipulations.

Reward Prediction Error and Declarative Memory

Learning based on reward prediction error (RPE) was originally proposed in the context of nondeclarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. We provide a brief overview of these studies, identify emerging patterns, and discuss open issues such as the role of signed versus unsigned RPEs in declarative learning.

Elucidating the role of the posterior medial frontal cortex in social conflict processing

A fundamental function of the brain is learning via new information. Studies investigating the neural basis of information-based learning processes indicate an important role played by the posterior medial frontal cortex (pMFC) in representing conflict between an individual's expectation and new information. However, specific function of the pMFC in this process remains relatively indistinct. Particularly, it's unclear whether the pMFC plays a role in the detection of conflict of incoming information, or the update of their belief after new information is provided. In an fMRI scanner, twenty-eight Japanese students viewed scenarios depicting various pro-social/anti-social behaviors. Participants rated how likely Japanese and South Korean students would perform each behavior, followed by feedback of the actual likelihood. They were then asked to rerate the scenarios after the fMRI session. Participants updated their second estimates based on feedback, with estimate changes more pronounced for favorable feedback (e.g., higher likelihood of pro-social behavior than expected) despite nationality, indicating participants were willing to view other people favorably. The fMRI results demonstrated activity in a part of the pMFC, the dorsomedial prefrontal cortex (dmPFC), was correlated with social conflict (difference between participant's estimate and actual likelihood), but not the corresponding belief update. Importantly, activity in a different part within the dmPFC was more sensitive to unfavorable trials compared to favorable trials. These results indicate sensitivity in the pMFC (at least within the dmPFC) relates to conflict between desirable outcomes versus reality, as opposed to the associated update of belief.

Flexible updating of dynamic knowledge structures

Schemas are knowledge structures that allow us to make efficient judgments about the world without the cost of memorizing every detail of previous experiences. It has long been known that schemas can enhance long-term memory for related information. The usefulness of schemas, however, critically depends on their adaptability: how flexibly a schema can be updated according to changing environmental conditions. Prior consolidation of a schema supports new learning of schema-consistent information. Yet, the effect of consolidation on inconsistent information, and how schemas may be subsequently updated, are not well understood. It is difficult to track the dynamic updating of knowledge structures with traditional memory measures. Here, using a continuous-report paradigm, we were able to show that schematization increases incrementally with consolidation and that the strength with which schemas are initially established predicts schema-guided responding in a later test. Critically, schema updating in response to inconsistent information was more pronounced in a group which was given time to consolidate compared to a group that was not given time to consolidate. Importantly, the later group reverted back to the no longer relevant schema, indicating that systematic bias towards old information, rather than increased forgetting, underlies reduced memory for schema-inconsistent information.

Previously rewarding environments enhance incidental memory formation

Declarative memory performance is superior for items that were encoded in temporal proximity to reward delivery or expectancy. How reward-predicting contexts affect subsequent declarative memory formation in those contexts are, however, unknown. Using an ecological experimental setup in the form of a naturalistic driving simulator task, we examined the effect that previously rewarded environments may have on incidental memory formation. After driving in two distinct environments, one of which associated with monetary reward, participants drove again in the environments, which were embedded with unique images on billboards. A recognition test 24 h later demonstrated that incidental memory was superior for items presented in the reward-associated environment. These findings suggest that environmental cues imbued with incentive salience promote memory processes even in the absence of reward.

Surprise and destabilize: prediction error influences episodic memory reconsolidation

Through the process of "reconsolidation," reminders can temporarily destabilize memories and render them vulnerable to change. Recent rodent research has proposed that prediction error, or the element of surprise, is a key component of this process; yet, this hypothesis has never before been extended to complex episodic memories in humans. In our novel paradigm, we used naturalistic stimuli to demonstrate that prediction error enables adaptive updating of episodic memories. In Study 1, participants (N = 48) viewed 18 videos, each depicting an action-outcome event. The next day, we reactivated these memories by presenting the videos again. We found that incomplete reminders, which interrupted videos before the outcome, made memories vulnerable to subsequent interference from a new set of videos, producing false memories. In Study 2 (N = 408), an independent sample rated qualities of the stimuli. We found that videos that were more surprising when interrupted produced more false memories. Last, in Study 3 (N = 24), we tested competing predictions of reconsolidation theory and the Temporal Context Model, an alternative account of source confusion. Consistent with the mechanistic time-course of reconsolidation, our effects were crucially time-dependent. Overall, we synthesize prior animal and human research to present compelling evidence that prediction error destabilizes episodic memories and drives dynamic updating in the face of new information.