Striatal prediction errors support dynamic control of declarative memory decisions

Patterns of retrieval-related cortico-striatal connectivity are stable across the adult lifespan

Memory retrieval effects in the striatum are well documented and robust across experimental paradigms. However, the functional significance of these effects, and whether they are moderated by age, remains unclear. We used functional magnetic resonance imaging paired with an associative recognition task to examine retrieval effects in the striatum in a sample of healthy young, middle-aged, and older adults. We identified anatomically segregated patterns of enhanced striatal blood oxygen level-dependent (BOLD) activity during recollection- and familiarity-based memory judgments. Successful recollection was associated with enhanced BOLD activity in bilateral putamen and nucleus accumbens, and neither of these effects were reliably moderated by age. Familiarity effects were evident in the head of the caudate nucleus bilaterally, and these effects were attenuated in middle-aged and older adults. Using psychophysiological interaction analyses, we observed a monitoring-related increase in functional connectivity between the caudate and regions of the frontoparietal control network, and between the putamen and bilateral retrosplenial cortex and intraparietal sulcus. In all instances, monitoring-related increases in cortico-striatal connectivity were unmoderated by age. These results suggest that the striatum, and the caudate in particular, couples with the frontoparietal control network to support top-down retrieval-monitoring operations, and that the strength of these inter-regional interactions is preserved in later life.

Distinct neural mechanisms underlie subjective and objective recollection and guide memory-based decision making

Accurate memories are often associated with vivid experiences of recollection. However, the neural mechanisms underlying subjective recollection and their unique role in decision making beyond accuracy have received limited attention. We dissociated subjective recollection from accuracy during a forced-choice task. Distractors corresponded either to non-studied exemplars of the targets (A-A’ condition) or to non-studied exemplars of different studied items (A-B’ condition). The A-A’ condition resulted in higher accuracy and greater activation in the superior parietal lobe, whereas the A-B’ condition resulted in higher subjective recollection and greater activation in the precuneus and retrosplenial regions, indicating a dissociation between objective and subjective memory. Activation in insular, cingulate, and lateral prefrontal regions was also associated with subjective recollection; however, during a subsequent decision phase, activation in these same regions was greater for discarded than for selected responses in anticipation of a social reward, underscoring their role in evaluating memory evidence flexibly based on current goals.

Advances in the computational understanding of mental illness

Computational psychiatry is a rapidly growing field attempting to translate advances in computational neuroscience and machine learning into improved outcomes for patients suffering from mental illness. It encompasses both data-driven and theory-driven efforts. Here, recent advances in theory-driven work are reviewed. We argue that the brain is a computational organ. As such, an understanding of the illnesses arising from it will require a computational framework. The review divides work up into three theoretical approaches that have deep mathematical connections: dynamical systems, Bayesian inference and reinforcement learning. We discuss both general and specific challenges for the field, and suggest ways forward.

Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory

Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.

Neural Development of Memory and Metamemory in Childhood and Adolescence: Toward an Integrative Model of the Development of Episodic Recollection

Memory and metamemory processes are essential to retrieve detailed memories and appreciate the phenomenological experience of recollection. Developmental cognitive neuroscience has made strides in revealing the neural changes associated with improvements in memory and metamemory during childhood and adolescence. We argue that hippocampal changes, in concert with surrounding cortical regions, support developmental improvements in the precision, complexity, and flexibility of memory representations. In contrast, changes in frontoparietal regions promote efficient encoding and retrieval strategies. A smaller body of literature on the neural substrates of metamemory development suggests that error monitoring processes implemented in the anterior insula and dorsal anterior cingulate cortex trigger, and perhaps support the development of, metacognitive evaluationsin the prefrontal cortex, while developmental changes in the parietal cortex support changes in the phenomenological experience of episodic retrieval. Our conclusions highlight the necessity of integrating these lines of research into a comprehensive model on the neurocognitive development of episodic recollection.

Neural Systems for Memory-based Value Judgment and Decision-making

Real-life choices often require that we draw inferences about the value of options based on structured, schematic knowledge about their utility for our current goals. Other times, value information may be retrieved directly from a specific prior experience with an option. In an fMRI experiment, we investigated the neural systems involved in retrieving and assessing information from different memory sources to support value-based choice. Participants completed a task in which items could be conferred positive or negative value based on schematic associations (i.e., schema value) or learned directly from experience via deterministic feedback (i.e., experienced value). We found that ventromedial pFC (vmPFC) activity correlated with the influence of both experience- and schema-based values on participants' decisions. Connectivity between the vmPFC and middle temporal cortex also tracked the inferred value of items based on schematic associations on the first presentation of ingredients, before any feedback. In contrast, the striatum responded to participants' willingness to bet on ingredients as a function of the unsigned strength of their memory for those options' values. These results argue that the striatum and vmPFC play distinct roles in memory-based value judgment and decision-making. Specifically, the vmPFC assesses the value of options based on information inferred from schematic knowledge and retrieved from prior direct experience, whereas the striatum controls a decision to act on options based on memory strength.

Oscillatory signatures of reward prediction errors in declarative learning

Reward prediction errors (RPEs) are crucial to learning. Whereas these mismatches between reward expectation and reward outcome are known to drive procedural learning, their role in declarative learning remains underexplored. Earlier work from our lab addressed this, and consistently found that signed reward prediction errors (SRPEs; "better-than-expected" signals) boost declarative learning. In the current EEG study, we sought to explore the neural signatures of SRPEs. Participants studied 60 Dutch-Swahili word pairs while RPE magnitudes were parametrically manipulated. Behaviorally, we replicated our previous findings that SRPEs drive declarative learning, with increased recognition for word pairs accompanied by large, positive RPEs. In the EEG data, at the start of reward feedback processing, we found an oscillatory (theta) signature consistent with unsigned reward prediction errors (URPEs; "different-than-expected" signals). Slightly later during reward feedback processing, we observed oscillatory (high-beta and high-alpha) signatures for SRPEs during reward feedback, similar to SRPE signatures during procedural learning. These findings illuminate the time course of neural oscillations in processing reward during declarative learning, providing important constraints for future theoretical work.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Signed reward prediction errors drive declarative learning

Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

Retrieval Demands Adaptively Change Striatal Old/New Signals and Boost Subsequent Long-Term Memory

The striatum is a central part of the dopaminergic mesolimbic system and contributes both to the encoding and retrieval of long-term memories. In this regard, the co-occurrence of striatal novelty and retrieval success effects in independent studies underlines the structure's double duty and suggests dynamic contextual adaptation. To test this hypothesis and further investigate the underlying mechanisms of encoding and retrieval dynamics, human subjects viewed pre-familiarized scene images intermixed with new scenes and classified them as indoor versus outdoor (encoding task) or old versus new (retrieval task), while fMRI and eye tracking data were recorded. Subsequently, subjects performed a final recognition task. As hypothesized, striatal activity and pupil size reflected task-conditional salience of old and new stimuli, but, unexpectedly, this effect was not reflected in the substantia nigra and ventral tegmental area (SN/VTA), medial temporal lobe, or subsequent memory performance. Instead, subsequent memory generally benefitted from retrieval, an effect possibly driven by task difficulty and activity in a network including different parts of the striatum and SN/VTA. Our findings extend memory models of encoding and retrieval dynamics by pinpointing a specific contextual factor that differentially modulates the functional properties of the mesolimbic system.SIGNIFICANCE STATEMENT The mesolimbic system is involved in the encoding and retrieval of information but it is unclear how these two processes are achieved within the same network of brain regions. In particular, memory retrieval and novelty encoding were considered in independent studies, implying that novelty (new > old) and retrieval success (old > new) effects may co-occur in the striatum. Here, we used a common framework implicating the striatum, but not other parts of the mesolimbic system, in tracking context-dependent salience of old and new information. The current study, therefore, paves the way for a more comprehensive understanding of the functional properties of the mesolimbic system during memory encoding and retrieval.

Neural activations associated with feedback and retrieval success

There is substantial behavioral evidence for a phenomenon commonly called "the testing effect", i.e. superior memory performance after repeated testing compared to re-study of to-be-learned materials. However, considerably less is known about the underlying neuro-cognitive processes that are involved in the initial testing phase, and thus underlies the actual testing effect. Here, we investigated functional brain activity related to test-enhanced learning with feedback. Subjects learned foreign vocabulary across three consecutive tests with correct-answer feedback. Functional brain-activity responses were analyzed in relation to retrieval and feedback events, respectively. Results revealed up-regulated activity in fronto-striatal regions during the first successful retrieval, followed by a marked reduction in activity as a function of improved learning. Whereas feedback improved behavioral performance across consecutive tests, feedback had a negligable role after the first successful retrieval for functional brain-activity modulations. It is suggested that the beneficial effects of test-enhanced learning is regulated by feedback-induced updating of memory representations, mediated via the striatum, that might underlie the stabilization of memory commonly seen in behavioral studies of the testing effect.

Independent contributions of fMRI familiarity and novelty effects to recognition memory and their stability across the adult lifespan

The impact of age on the neural correlates of familiarity-driven recognition memory has received relatively little attention. Here, the relationships between age, the neural correlates of familiarity, and memory performance were investigated using an associative recognition test in young, middle-aged and older participants. Test items comprised studied, rearranged (items studied on different trials) and new word pairs. fMRI 'familiarity effects' were operationalized as greater activity for studied test pairs incorrectly identified as 'rearranged' than for correctly rejected new pairs. The reverse contrast was employed to identify 'novelty' effects. Estimates of familiarity strength were slightly but significantly lower for the older relative to the younger group. With the exception of one region in dorsal medial prefrontal cortex, fMRI familiarity effects (which were identified in medial and lateral parietal cortex, dorsal medial and left lateral prefrontal cortex, and bilateral caudate among other regions) did not differ significantly with age. Age-invariant 'novelty effects' were identified in the anterior hippocampus and the perirhinal cortex. When entered into the same regression model, familiarity and novelty effects independently predicted familiarity strength across participants, suggesting that the two classes of memory effect reflect functionally distinct mnemonic processes. It is concluded that the neural correlates of familiarity-based memory judgments, and their relationship with familiarity strength, are largely stable across much of the healthy adult lifespan.