Parallel contributions of distinct human memory systems during probabilistic learning

Prediction-error-dependent processing of immediate and delayed positive feedback

Learning often involves trial-and-error, i.e. repeating behaviours that lead to desired outcomes, and adjusting behaviour when outcomes do not meet our expectations and thus lead to prediction errors (PEs). PEs have been shown to be reflected in the reward positivity (RewP), an event-related potential (ERP) component between 200 and 350 ms after performance feedback which is linked to striatal processing and assessed via electroencephalography (EEG). Here we show that this is also true for delayed feedback processing, for which a critical role of the hippocampus has been suggested. We found a general reduction of the RewP for delayed feedback, but the PE was similarly reflected in the RewP and the later P300 for immediate and delayed positive feedback, while no effect was found for negative feedback. Our results suggest that, despite processing differences between immediate and delayed feedback, positive PEs drive feedback processing and learning irrespective of delay.

The balance of hippocampal and caudate network functional connectivity is associated with episodic memory performance and its decline across adulthood

The hippocampal and caudate networks interact to support episodic memory, but the relationship between hippocampal and caudate connectivity strength and episodic memory is unclear. In general, cognition is optimally supported when connectivity within a functional network dominates connectivity from other networks. For example, episodic memory may be optimally supported when the hippocampal and caudate networks express this pattern of connectivity, consistent with research showing that the two networks are organized competitively. Alternatively, episodic memory may be optimally supported when connectivity in both networks is more balanced, consistent with fMRI reports showing cooperation between networks. Using cross-sectional behavioral and resting state fMRI data from a diverse sample (N = 347; Ages 18-85), I tested the hypothesis that reduced hippocampal and caudate network dominance would be associated with reduced episodic memory across individuals and age. Consistent with this hypothesis, lower caudate network dominance in bilateral thalamic regions was associated with worse episodic memory regardless of age. Age-related differences in caudate network dominance in the pallidum and putamen were also associated with worse episodic memory performance, but through their shared variance with age. I found no evidence that network dominance was related to processing speed or executive function, or that hippocampal network dominance was relate to episodic memory performance. These results show that ongoing biological dynamics between the hippocampal and caudate networks throughout adulthood are related to episodic memory performance and support a growing literature specifying the role of the caudate network in episodic memory.

The anterior medial hippocampus contributes to both recall and familiarity-based memory for scenes

The hippocampus is usually associated with recall memory, whereas its contribution to familiarity-based memory is debated. Growing evidence support the idea that this structure participates to any cognitive process performed on scene representations. In parallel, differences in functional specialisation and cortical connectivity were found across the longitudinal and transverse axes of the hippocampus. Here we reanalysed functional MRI data from 51 participants showing stronger engagement of the hippocampus in recall, familiarity-based recognition and rejection, and visual discrimination, of scenes compared to single objects. A conjunction analysis between these four tasks revealed a set of occipital, medial temporal, posterior cingulate, and parietal regions, matching the scene construction network described in the literature. Crucially, we found that the anterior medial part of the hippocampus was consistently involved in all tasks investigated for scene stimuli. These findings support that the hippocampus can contribute to both recall and familiarity-based memory, depending on stimulus type. More generally, this bolsters the recent proposal that circumscribed regions within the hippocampus may underpin specific cognitive mechanisms.

Learning new words via feedback-Association between feedback-locked ERPs and recall performance-An exploratory study

Feedback learning is thought to involve the dopamine system and its projection sites in the basal ganglia and anterior cingulate cortex (ACC), regions associated with procedural learning. Under certain conditions, such as when feedback is delayed, feedback-locked activation is pronounced in the medial temporal lobe (MTL), which is associated with declarative learning. In event-related potential research, the feedback-related negativity (FRN) has been linked to immediate feedback processing, while the N170, possibly reflecting MTL activity, has been related to delayed feedback processing. In the current study, we performed an exploratory investigation on the relation between N170 and FRN amplitude and memory performance in a test for declarative memory (free recall), also exploring the role of feedback delay. To this end, we adapted a paradigm in which participants learned associations between non-objects and non-words with either immediate or delayed feedback, and added a subsequent free recall test. We indeed found that N170, but not FRN amplitudes, depended on later free recall performance, with smaller amplitudes for later remembered non-words. In an additional analysis with memory performance as dependent variable, the N170, but not the FRN amplitude predicted free recall, modulated by feedback timing and valence. This finding shows that the N170 reflects an important process during feedback processing, possibly related to expectations and their violation, but is distinct from the process reflected by the FRN.

Neural substrates of reward anticipation and outcome in schizophrenia: a meta-analysis of fMRI findings in the monetary incentive delay task

Dysfunction of the mesocorticolimbic dopaminergic reward system is a core feature of schizophrenia (SZ), yet its precise contributions to different stages of reward processing and their relevance to disease symptomology are not fully understood. We performed a coordinate-based meta-analysis, using the monetary incentive delay task, to identify which brain regions are implicated in different reward phases in functional magnetic resonance imaging in SZ. A total of 17 studies (368 SZ and 428 controls) were included in the reward anticipation, and 10 studies (229 SZ and 281 controls) were included in the reward outcome. Our meta-analysis revealed that during anticipation, patients showed hypoactivation in the striatum, anterior cingulate cortex, median cingulate cortex (MCC), amygdala, precentral gyrus, and superior temporal gyrus compared with controls. Striatum hypoactivation was negatively associated with negative symptoms and positively associated with the proportion of second-generation antipsychotic users (percentage of SGA users). During outcome, patients displayed hyperactivation in the striatum, insula, amygdala, hippocampus, parahippocampal gyrus, cerebellum, postcentral gyrus, and MCC, and hypoactivation in the dorsolateral prefrontal cortex (DLPFC) and medial prefrontal cortex (mPFC). Hypoactivity of mPFC during outcome was negatively associated with positive symptoms. Moderator analysis showed that the percentage of SGA users was a significant moderator of the association between symptom severity and brain activity in both the anticipation and outcome stages. Our findings identified the neural substrates for different reward phases in SZ and may help explain the neuropathological mechanisms underlying reward processing deficits in the disorder.

Hippocampal Contribution to Probabilistic Feedback Learning: Modeling Observation- and Reinforcement-based Processes

Simple probabilistic reinforcement learning is recognized as a striatum-based learning system, but in recent years, has also been associated with hippocampal involvement. This study examined whether such involvement may be attributed to observation-based learning (OL) processes, running in parallel to striatum-based reinforcement learning. A computational model of OL, mirroring classic models of reinforcement-based learning (RL), was constructed and applied to the neuroimaging data set of Palombo, Hayes, Reid, and Verfaellie (2019). Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia. Cognitive, Affective & Behavioral Neuroscience, 19(3), 523-536. Results suggested that OL processes may indeed take place concomitantly to reinforcement learning and involve activation of the hippocampus and central orbitofrontal cortex. However, rather than independent mechanisms running in parallel, the brain correlates of the OL and RL prediction errors indicated collaboration between systems, with direct implication of the hippocampus in computations of the discrepancy between the expected and actual reinforcing values of actions. These findings are consistent with previous accounts of a role for the hippocampus in encoding the strength of observed stimulus-outcome associations, with updating of such associations through striatal reinforcement-based computations. In addition, enhanced negative RL prediction error signaling was found in the anterior insula with greater use of OL over RL processes. This result may suggest an additional mode of collaboration between the OL and RL systems, implicating the error monitoring network.

Response Flexibility: The Role of the Lateral Habenula

The ability to make appropriate decisions that result in an optimal outcome is critical for survival. This process involves assessing the environment as well as integrating prior knowledge about the environment with information about one's current internal state. There are many neural structures that play critical roles in mediating these processes, but it is not yet known how such information coalesces to influence behavioral output. The lateral habenula (LHb) has often been cited as a structure critical for adaptive and flexible responding when environmental contexts and internal state changes. A challenge, however, has been understanding how LHb promotes response flexibility. In this review, we hypothesize that the LHb enables flexible responding following the integration of context memory and internal state information by signaling downstream brainstem structures known to drive hippocampal theta. In this way, animals respond more flexibly in a task situation not because the LHb selects a particular action, but rather because LHb enhances a hippocampal neural state that is often associated with greater attention, arousal, and exploration. In freely navigating animals, these are essential conditions that are needed to discover and implement appropriate alternative choices and behaviors. As a corollary to our hypothesis, we describe short- and intermediate-term functions of the LHb. Finally, we discuss the effects on the behavior of LHb dysfunction in short- and intermediate-timescales, and then suggest that new therapies may act on the LHb to alleviate the behavioral impairments following long-term LHb disruption.

A direct test of competitive versus cooperative episodic-procedural network dynamics in human memory

Classical lesion studies led to a consensus that episodic and procedural memory arises from segregated networks identified with the hippocampus and the caudate nucleus, respectively. Neuroimaging studies, however, show that competitive and cooperative interactions occur between networks during memory tasks. Furthermore, causal experiments to manipulate connectivity between these networks have not been performed in humans. Although nodes common to both networks, such as the precuneus and ventrolateral thalamus, may mediate their interaction, there is no experimental evidence for this. We tested how network-targeted noninvasive brain stimulation affects episodic-procedural network interactions and how these network manipulations affect episodic and procedural memory in healthy young adults. Compared to control (vertex) stimulation, hippocampal network-targeted stimulation increased within-network functional connectivity and hippocampal connectivity with the caudate. It also increased episodic, relative to procedural, memory, and this persisted one week later. The differential effect on episodic versus procedural memory was associated with increased functional connectivity between the caudate, precuneus, and ventrolateral thalamus. These findings provide direct evidence of episodic-procedural network competition, mediated by regions common to both networks. Enhanced hippocampal network connectivity may boost episodic, but decrease procedural, memory by co-opting resources shared between networks.

Prediction errors disrupt hippocampal representations and update episodic memories

The brain supports adaptive behavior by generating predictions, learning from errors, and updating memories to incorporate new information. Prediction error, or surprise, triggers learning when reality contradicts expectations. Prior studies have shown that the hippocampus signals prediction errors, but the hypothesized link to memory updating has not been demonstrated. In a human functional MRI study, we elicited mnemonic prediction errors by interrupting familiar narrative videos immediately before the expected endings. We found that prediction errors reversed the relationship between univariate hippocampal activation and memory: greater hippocampal activation predicted memory preservation after expected endings, but memory updating after surprising endings. In contrast to previous studies, we show that univariate activation was insufficient for understanding hippocampal prediction error signals. We explain this surprising finding by tracking both the evolution of hippocampal activation patterns and the connectivity between the hippocampus and neuromodulatory regions. We found that hippocampal activation patterns stabilized as each narrative episode unfolded, suggesting sustained episodic representations. Prediction errors disrupted these sustained representations and the degree of disruption predicted memory updating. The relationship between hippocampal activation and subsequent memory depended on concurrent basal forebrain activation, supporting the idea that cholinergic modulation regulates attention and memory. We conclude that prediction errors create conditions that favor memory updating, prompting the hippocampus to abandon ongoing predictions and make memories malleable.

Value network engagement and effects of memory-related processing during encoding and retrieval of value

Decision makers rely on episodic memory to calculate choice values in everyday life, yet it is unclear how neural mechanisms of valuation differ when value-related information is encoded versus retrieved from episodic memory. The current fMRI study compared neural correlates of value while information was encoded versus retrieved from memory. Scanned tasks were followed by a behavioral episodic memory test for item-attribute associations. Our analyses sought to (i) identify neural correlates of value that were distinct and common across encoding and retrieval, and (ii) determine whether neural mechanisms of valuation and episodic memory interact. The study yielded three primary findings. First, value-related activation in the fronto-striatal reward circuit and posterior parietal cortex was comparable across valuation phases. Second, value-related activation in select fronto-parietal and salience regions was significantly greater at value retrieval than encoding. Third, there was no interaction between neural correlates of valuation and episodic memory. Taken with prior research, the present study indicates that fronto-parietal and salience regions play a key role in retrieval-dependent valuation and context-specific effects likely determine whether neural correlates of value interact with episodic memory.

Probabilistic value learning in medial temporal lobe amnesia

A prevailing view in cognitive neuroscience suggests that different forms of learning are mediated by dissociable memory systems, with a mesolimbic (i.e., midbrain and basal ganglia) system supporting incremental trial-and-error reinforcement learning and a hippocampal-based system supporting episodic memory. Yet, growing evidence suggests that the hippocampus may also be important for trial-and-error learning, particularly value or reward-based learning. In the present report, we use a lesion-based neuropsychological approach to clarify hippocampal contributions to such learning. Six amnesic patients with medial temporal lobe damage and a group of healthy controls were administered a simple value-based learning task involving probabilistic trial-and-error acquisition of stimulus-response-outcome (reward or none) contingencies modeled after Li et al. (Proceedings of the National Academy of Sciences , 2011, 108 (1), 55-60). As predicted, patients were significantly impaired on the task, demonstrating reduced learning of the contingencies. Our results provide further supportive evidence that the hippocampus' role in cognition extends beyond episodic memory tasks and call for further refinement of theoretical models of hippocampal functioning.

Neural correlates of reward processing: Functional dissociation of two components within the ventral striatum

INTRODUCTION

Rewarding and punishing stimuli elicit BOLD responses in the affective division of the striatum. The responses typically traverse from the affective to the associative division of the striatum, suggesting an involvement of associative processes during the modulation of stimuli valance. In this study, we hypothesized that fMRI responses to rewards versus punishments in a guessing card game can be disassociated into two functional component processes that reflect the convergence of limbic and associative functional networks in the ventral striatum.

METHODS

We used fMRI data of 175 (92 female) subjects from the human connectome project´s gambling task, working memory task, and resting-state scans. A reward > punish contrast identified a ventral striatum cluster from which voxelwise GLM parameter estimates were entered into a k-means clustering algorithm. The k-means analysis supported separating the cluster into two spatially distinct components. These components were used as seeds to investigate their functional connectivity profile. GLM parameter estimates were extracted and compared from the task contrasts reward > punish and 2-back > 0-back from two ROIs in the ventral striatum and one ROI in hippocampus.

RESULTS

The analyses converged to show that a superior striatal component, coupled with the ventral attention and frontal control networks, was responsive to both a modulation of cognitive control in working memory and to rewards, whereas the most inferior part of the ventral striatum, coupled with the limbic and default mode networks including the hippocampus, was selectively responsive to rewards.

CONCLUSION

We show that the fMRI response to rewards in the ventral striatum reflects a mixture of component processes of reward. An inferior ventral striatal component and hippocampus are part of an intrinsically coupled network that responds to reward-based processing during gambling. The more superior ventral striatal component is intrinsically coupled to networks involved with executive functioning and responded to both reward and cognitive control demands.

Differential reinforcement encoding along the hippocampal long axis helps resolve the explore-exploit dilemma

When making decisions, should one exploit known good options or explore potentially better alternatives? Exploration of spatially unstructured options depends on the neocortex, striatum, and amygdala. In natural environments, however, better options often cluster together, forming structured value distributions. The hippocampus binds reward information into allocentric cognitive maps to support navigation and foraging in such spaces. Here we report that human posterior hippocampus (PH) invigorates exploration while anterior hippocampus (AH) supports the transition to exploitation on a reinforcement learning task with a spatially structured reward function. These dynamics depend on differential reinforcement representations in the PH and AH. Whereas local reward prediction error signals are early and phasic in the PH tail, global value maximum signals are delayed and sustained in the AH body. AH compresses reinforcement information across episodes, updating the location and prominence of the value maximum and displaying goal cell-like ramping activity when navigating toward it.

Hippocampal contributions to value-based learning: Converging evidence from fMRI and amnesia

Recent evidence suggests that the human hippocampus-known primarily for its involvement in episodic memory-plays a role in a host of motivationally relevant behaviors, including some forms of value-based decision-making. However, less is known about the role of the hippocampus in value-based learning. Such learning is typically associated with a striatal system, yet a small number of studies, both in human and nonhuman species, suggest hippocampal engagement. It is not clear, however, whether this engagement is necessary for such learning. In the present study, we used both functional MRI (fMRI) and lesion-based neuropsychological methods to clarify hippocampal contributions to value-based learning. In Experiment 1, healthy participants were scanned while learning value-based contingencies (whether players in a "game" win money) in the context of a probabilistic learning task. Here, we observed recruitment of the hippocampus, in addition to the expected ventral striatal (nucleus accumbens) activation that typically accompanies such learning. In Experiment 2, we administered this task to amnesic patients with medial temporal lobe damage and to healthy controls. Amnesic patients, including those with damage circumscribed to the hippocampus, failed to acquire value-based contingencies, thus confirming that hippocampal engagement is necessary for task performance. Control experiments established that this impairment was not due to perceptual demands or memory load. Future research is needed to clarify the mechanisms by which the hippocampus contributes to value-based learning, but these findings point to a broader role for the hippocampus in goal-directed behaviors than previously appreciated.

Competitive and cooperative interactions between medial temporal and striatal learning systems

The striatum and medial temporal lobes (MTL) exhibit dissociable roles during learning. Whereas the striatum and its network of thalamic relays and cortical nodes are necessary for nondeclarative learning, the MTL and associated network are required for declarative learning. Several studies have suggested that these networks are functionally competitive during learning. Since these discoveries, however, evidence has accumulated that they can operate in a cooperative fashion. In this review, we discuss evidence for both competition and cooperation between these systems during learning, with the aim of reconciling these seemingly contradictory findings. Examples of cooperation between the striatum and MTL have been provided, especially during consolidation and generalization of knowledge, and do not appear to be precluded by differences in functional specialization. However, whether these systems cooperate or compete does seem to depend on the phase of learning and cognitive or motor aspects of the task. The involvement of other regions, such as midbrain dopaminergic nuclei and the prefrontal cortex, may promote and mediate cooperation between the striatum and the MTL during learning. Building on this body of research, we propose a model for striatum-MTL interactions in learning and memory and attempt to predict, in general terms, when cooperation or competition will occur.

Reward Learning over Weeks Versus Minutes Increases the Neural Representation of Value in the Human Brain

Over the past few decades, neuroscience research has illuminated the neural mechanisms supporting learning from reward feedback. Learning paradigms are increasingly being extended to study mood and psychiatric disorders as well as addiction. However, one potentially critical characteristic that this research ignores is the effect of time on learning: human feedback learning paradigms are usually conducted in a single rapidly paced session, whereas learning experiences in ecologically relevant circumstances and in animal research are almost always separated by longer periods of time. In our experiments, we examined reward learning in short condensed sessions distributed across weeks versus learning completed in a single "massed" session in male and female participants. As expected, we found that after equal amounts of training, accuracy was matched between the spaced and massed conditions. However, in a 3-week follow-up, we found that participants exhibited significantly greater memory for the value of spaced-trained stimuli. Supporting a role for short-term memory in massed learning, we found a significant positive correlation between initial learning and working memory capacity. Neurally, we found that patterns of activity in the medial temporal lobe and prefrontal cortex showed stronger discrimination of spaced- versus massed-trained reward values. Further, patterns in the striatum discriminated between spaced- and massed-trained stimuli overall. Our results indicate that single-session learning tasks engage partially distinct learning mechanisms from distributed training. Our studies begin to address a large gap in our knowledge of human learning from reinforcement, with potential implications for our understanding of mood disorders and addiction.SIGNIFICANCE STATEMENT Humans and animals learn to associate predictive value with stimuli and actions, and these values then guide future behavior. Such reinforcement-based learning often happens over long time periods, in contrast to most studies of reward-based learning in humans. In experiments that tested the effect of spacing on learning, we found that associations learned in a single massed session were correlated with short-term memory and significantly decayed over time, whereas associations learned in short massed sessions over weeks were well maintained. Additionally, patterns of activity in the medial temporal lobe and prefrontal cortex discriminated the values of stimuli learned over weeks but not minutes. These results highlight the importance of studying learning over time, with potential applications to drug addiction and psychiatry.

Adolescent Development of Value-Guided Goal Pursuit

Adolescents are challenged to orchestrate goal-directed actions in increasingly independent and consequential ways. In doing so, it is advantageous to use information about value to select which goals to pursue and how much effort to devote to them. Here, we examine age-related changes in how individuals use value signals to orchestrate goal-directed behavior. Drawing on emerging literature on value-guided cognitive control and reinforcement learning, we demonstrate how value and task difficulty modulate the execution of goal-directed action in complex ways across development from childhood to adulthood. We propose that the scope of value-guided goal pursuit expands with age to include increasingly challenging cognitive demands, and scaffolds on the emergence of functional integration within brain networks supporting valuation, cognition, and action.

The network architecture of value learning

Value guides behavior. With knowledge of stimulus values and action consequences, behaviors that maximize expected reward can be selected. Prior work has identified several brain structures critical for representing both stimuli and their values. Yet, it remains unclear how these structures interact with one another and with other regions of the brain to support the dynamic acquisition of value-related knowledge. Here, we use a network neuroscience approach to examine how BOLD functional networks change as 20 healthy human subjects learn the values of novel visual stimuli over the course of four consecutive days. We show that connections between regions of the visual, frontal, and cingulate cortices become stronger as learning progresses, with some of these changes being specific to the type of feedback received during learning. These results demonstrate that functional networks dynamically track behavioral improvement in value judgments, and that interactions between network communities form predictive biomarkers of learning.

Rational human behavior is the pursuit of actions that maximize expected reward. These rewards can be understood as stimulus-value contingencies, learned by experience throughout our lives. Various structures have been recognized to participate in these learning processes. Yet, an understanding of how these structures interact with one another and with other brain regions remains vastly unexplored. Here, we propose a novel analytical framework utilizing and extending techniques from the dynamic network neuroscience to ask “How do our brains change when we learn values?” We find that interactions between sensory and fronto-cingulate structures grow stronger as learning progresses, bringing together several isolated findings in the cognitive neuroscience of value-based behavior and extending our understanding of human learning in general.

Effects of feedback delay on learning from positive and negative feedback in patients with Parkinson's disease off medication

Phasic dopamine (DA) signals conveyed from the substantia nigra to the striatum and the prefrontal cortex crucially affect learning from feedback, with DA bursts facilitating learning from positive feedback and DA dips facilitating learning from negative feedback. Consequently, diminished nigro-striatal dopamine levels as in unmedicated patients suffering from Parkinson's Disease (PD) have been shown to lead to a negative learning bias. Recent studies suggested a diminished striatal contribution to feedback processing when the outcome of an action is temporally delayed. This study investigated whether the bias towards negative feedback learning induced by a lack of DA in PD patients OFF medication is modulated by feedback delay. To this end, PD patients OFF medication and healthy controls completed a probabilistic selection task, in which feedback was given immediately (after 800 ms) or delayed (after 6800 ms). PD patients were impaired in immediate but not delayed feedback learning. However, differences in the preference for positive/negative learning between patients and controls were seen for both learning from immediate and delayed feedback, with evidence of stronger negative learning in patients than controls. A Bayesian analysis of the data supports the conclusion that feedback timing did not affect the learning bias in the patients. These results hint at reduced, but still relevant nigro-striatal contribution to feedback learning, when feedback is delayed.

Knowledge acquisition is governed by striatal prediction errors

Discrepancies between expectations and outcomes, or prediction errors, are central to trial-and-error learning based on reward and punishment, and their neurobiological basis is well characterized. It is not known, however, whether the same principles apply to declarative memory systems, such as those supporting semantic learning. Here, we demonstrate with fMRI that the brain parametrically encodes the degree to which new factual information violates expectations based on prior knowledge and beliefs-most prominently in the ventral striatum, and cortical regions supporting declarative memory encoding. These semantic prediction errors determine the extent to which information is incorporated into long-term memory, such that learning is superior when incoming information counters strong incorrect recollections, thereby eliciting large prediction errors. Paradoxically, by the same account, strong accurate recollections are more amenable to being supplanted by misinformation, engendering false memories. These findings highlight a commonality in brain mechanisms and computational rules that govern declarative and nondeclarative learning, traditionally deemed dissociable.

More Than the Sum of Its Parts: A Role for the Hippocampus in Configural Reinforcement Learning

People often perceive configurations rather than the elements they comprise, a bias that may emerge because configurations often predict outcomes. But how does the brain learn to associate configurations with outcomes and how does this learning differ from learning about individual elements? We combined behavior, reinforcement learning models, and functional imaging to understand how people learn to associate configurations of cues with outcomes. We found that configural learning depended on the relative predictive strength of elements versus configurations and was related to both the strength of BOLD activity and patterns of BOLD activity in the hippocampus. Configural learning was further related to functional connectivity between the hippocampus and nucleus accumbens. Moreover, configural learning was associated with flexible knowledge about associations and differential eye movements during choice. Together, this suggests that configural learning is associated with a distinct computational, cognitive, and neural profile that is well suited to support flexible and adaptive behavior.

Aging and a genetic KIBRA polymorphism interactively affect feedback- and observation-based probabilistic classification learning

Probabilistic category learning involves complex interactions between the hippocampus and striatum that may depend on whether acquisition occurs via feedback or observation. Little is known about how healthy aging affects these processes. We tested whether age-related behavioral differences in probabilistic category learning from feedback or observation depend on a genetic factor known to influence individual differences in hippocampal function, the KIBRA gene (single nucleotide polymorphism rs17070145). Results showed comparable age-related performance impairments in observational as well as feedback-based learning. Moreover, genetic analyses indicated an age-related interactive effect of KIBRA on learning: among older adults, the beneficial T-allele was positively associated with learning from feedback, but negatively with learning from observation. In younger adults, no effects of KIBRA were found. Our results add behavioral genetic evidence to emerging data showing age-related differences in how neural resources relate to memory functions, namely that hippocampal and striatal contributions to probabilistic category learning may vary with age. Our findings highlight the effects genetic factors can have on differential age-related decline of different memory functions.

Altered Microstructural Caudate Integrity in Posttraumatic Stress Disorder but Not Traumatic Brain Injury

OBJECTIVE

Given the high prevalence and comorbidity of combat-related PTSD and TBI in Veterans, it is often difficult to disentangle the contributions of each disorder. Examining these pathologies separately may help to understand the neurobiological basis of memory impairment in PTSD and TBI independently of each other. Thus, we investigated whether a) PTSD and TBI are characterized by subcortical structural abnormalities by examining diffusion tensor imaging (DTI) metrics and volume and b) if these abnormalities were specific to PTSD versus TBI.

METHOD

We investigated whether individuals with PTSD or TBI display subcortical structural abnormalities in memory regions by examining DTI metrics and volume of the hippocampus and caudate in three groups of Veterans: Veterans with PTSD, Veterans with TBI, and Veterans with neither PTSD nor TBI (Veteran controls).

RESULTS

While our results demonstrated no macrostructural differences among the groups in these regions, there were significant alterations in microstructural DTI indices in the caudate for the PTSD group but not the TBI group compared to Veteran controls.

CONCLUSIONS

The result of increased mean, radial, and axial diffusivity, and decreased fractional anisotropy in the caudate in absence of significant volume atrophy in the PTSD group suggests the presence of subtle abnormalities evident only at a microstructural level. The caudate is thought to play a role in the physiopathology of PTSD, and the habit-like behavioral features of the disorder could be due to striatal-dependent habit learning mechanisms. Thus, DTI appears to be a vital tool to investigate subcortical pathology, greatly enhancing the ability to detect subtle brain changes in complex disorders.

Contributions of the hippocampus to feedback learning

Humans learn about the world in a variety of manners, including by observation, by associating cues in the environment, and via feedback. Across species, two brain structures have been predominantly involved in these learning processes: the hippocampus--supporting learning via observation and paired association--and the striatum--critical for feedback learning. This simple dichotomy, however, has recently been challenged by reports of hippocampal engagement in feedback learning, although the role of the hippocampus is not fully understood. The purpose of this experiment was to characterize the hippocampal response during feedback learning by manipulating varying levels of memory interference. Consistent with prior reports, feedback learning recruited the striatum and midbrain. Notably, feedback learning also engaged the hippocampus. The level of activity in these regions was modulated by the degree of memory interference, such that the greatest activation occurred during the highest level of memory interference. Importantly, the accuracy of information learned via feedback correlated with hippocampal activation and was reduced by the presence of high memory interference. Taken together, these findings provide evidence of hippocampal involvement in feedback learning by demonstrating both its relevance for the accuracy of information learned via feedback and its susceptibility to interference.

A tunable local field potentials computer simulator to assess minimal requirements for phase-amplitude cross-frequency-coupling estimation

The quantitative study of cross-frequency coupling (CFC) is a relevant issue in neuroscience. In local field potentials (LFPs), measured either in the cortex or in the hippocampus, how γ-oscillation amplitude is modulated by changes in θ-rhythms-phase is thought to be important in memory formation. Several methods were proposed to quantify CFC, but reported evidence suggests that experimental parameters affect the results. Therefore, a simulation tool to support the determination of minimal requirements for CFC estimation in order to obtain reliable results is particularly useful. An approach to generate computer-simulated signals having CFC intensity, sweep duration, signal-to-noise ratio (SNR), and multiphasic-coupling tunable by the user has been developed. Its utility has been proved by a study evaluating minimal sweep duration and SNR required for reliable θ-γ CFC estimation from signals simulating LFP measured in the mouse hippocampus. A MATLAB® software was made available to facilitate methodology reproducibility. The analysis of the synthetic LFPs created by the simulator shows how the minimal sweep duration for achieving accurate θ-γ CFC estimates increases as SNR decreases and the number of CFC levels to discriminate increases. In particular, a sufficient reliability in discriminating five different predetermined CFC levels is reached with 35-s sweep with SNR = 20, while SNR = 5 requires at least 140-s sweep.

Paired-Associate and Feedback-Based Weather Prediction Tasks Support Multiple Category Learning Systems

It remains unclear whether probabilistic category learning in the feedback-based weather prediction task (FB-WPT) can be mediated by a non-declarative or procedural learning system. To address this issue, we compared the effects of training time and verbal working memory, which influence the declarative learning system but not the non-declarative learning system, in the FB and paired-associate (PA) WPTs, as the PA task recruits a declarative learning system. The results of Experiment 1 showed that the optimal accuracy in the PA condition was significantly decreased when the training time was reduced from 7 to 3 s, but this did not occur in the FB condition, although shortened training time impaired the acquisition of explicit knowledge in both conditions. The results of Experiment 2 showed that the concurrent working memory task impaired the optimal accuracy and the acquisition of explicit knowledge in the PA condition but did not influence the optimal accuracy or the acquisition of self-insight knowledge in the FB condition. The apparent dissociation results between the FB and PA conditions suggested that a non-declarative or procedural learning system is involved in the FB-WPT and provided new evidence for the multiple-systems theory of human category learning.

Using fMRI to study reward processing in humans: past, present, and future

Functional magnetic resonance imaging (fMRI) is a noninvasive tool used to probe cognitive and affective processes. Although fMRI provides indirect measures of neural activity, the advent of fMRI has allowed for1) the corroboration of significant animal findings in the human brain, and2) the expansion of models to include more common human attributes that inform behavior. In this review, we briefly consider the neural basis of the blood oxygenation level dependent signal to set up a discussion of how fMRI studies have applied it in examining cognitive models in humans and the promise of using fMRI to advance such models. Specifically, we illustrate the contribution that fMRI has made to the study of reward processing, focusing on the role of the striatum in encoding reward-related learning signals that drive anticipatory and consummatory behaviors. For instance, we discuss how fMRI can be used to link neural signals (e.g., striatal responses to rewards) to individual differences in behavior and traits. While this functional segregation approach has been constructive to our understanding of reward-related functions, many fMRI studies have also benefitted from a functional integration approach that takes into account how interconnected regions (e.g., corticostriatal circuits) contribute to reward processing. We contend that future work using fMRI will profit from using a multimodal approach, such as combining fMRI with noninvasive brain stimulation tools (e.g., transcranial electrical stimulation), that can identify causal mechanisms underlying reward processing. Consequently, advancements in implementing fMRI will promise new translational opportunities to inform our understanding of psychopathologies.

The Role of Informative and Ambiguous Feedback in Avoidance Behavior: Empirical and Computational Findings

Avoidance behavior is a critical component of many psychiatric disorders, and as such, it is important to understand how avoidance behavior arises, and whether it can be modified. In this study, we used empirical and computational methods to assess the role of informational feedback and ambiguous outcome in avoidance behavior. We adapted a computer-based probabilistic classification learning task, which includes positive, negative and no-feedback outcomes; the latter outcome is ambiguous as it might signal either a successful outcome (missed punishment) or a failure (missed reward). Prior work with this task suggested that most healthy subjects viewed the no-feedback outcome as strongly positive. Interestingly, in a later version of the classification task, when healthy subjects were allowed to opt out of (i.e. avoid) responding, some subjects (“avoiders”) reliably avoided trials where there was a risk of punishment, but other subjects (“non-avoiders”) never made any avoidance responses at all. One possible interpretation is that the “non-avoiders” valued the no-feedback outcome so positively on punishment-based trials that they had little incentive to avoid. Another possible interpretation is that the outcome of an avoided trial is unspecified and that lack of information is aversive, decreasing subjects’ tendency to avoid. To examine these ideas, we here tested healthy young adults on versions of the task where avoidance responses either did or did not generate informational feedback about the optimal response. Results showed that provision of informational feedback decreased avoidance responses and also decreased categorization performance, without significantly affecting the percentage of subjects classified as “avoiders.” To better understand these results, we used a modified Q-learning model to fit individual subject data. Simulation results suggest that subjects in the feedback condition adjusted their behavior faster following better-than-expected outcomes, compared to subjects in the no-feedback condition. Additionally, in both task conditions, “avoiders” adjusted their behavior faster following worse-than-expected outcomes, and treated the ambiguous no-feedback outcome as less rewarding, compared to non-avoiders. Together, results shed light on the important role of ambiguous and informative feedback in avoidance behavior.

Ongoing behavioral state information signaled in the lateral habenula guides choice flexibility in freely moving rats

The lateral habenula (LHb) plays a role in a wide variety of behaviors ranging from maternal care, to sleep, to various forms of cognition. One prominent theory with ample supporting evidence is that the LHb serves to relay basal ganglia and limbic signals about negative outcomes to midbrain monoaminergic systems. This makes it likely that the LHb is critically involved in behavioral flexibility as all of these systems have been shown to contribute when flexible behavior is required. Behavioral flexibility is commonly examined across species and is impaired in various neuropsychiatric conditions including autism, depression, addiction, and schizophrenia; conditions in which the LHb is thought to play a role. Therefore, a thorough examination of the role of the LHb in behavioral flexibility serves multiple functions including understanding possible connections with neuropsychiatric illnesses and additional insight into its role in cognition in general. Here, we assess the LHb’s role in behavioral flexibility through comparisons of the roles its afferent and efferent pathways are known to play. Additionally, we provide new evidence supporting the LHb contributions to behavioral flexibility through organization of specific goal directed actions under cognitively demanding conditions. Specifically, in the first experiment, a majority of neurons recorded from the LHb were found to correlate with velocity on a spatial navigation task and did not change significantly when reward outcomes were manipulated. Additionally, measurements of local field potential (LFP) in the theta band revealed significant changes in power relative to velocity and reward location. In a second set of experiments, inactivation of the LHb with the gamma-aminobutyric acid (GABA) agonists baclofen and muscimol led to an impairment in a spatial/response based repeated probabilistic reversal learning task. Control experiments revealed that this impairment was likely due to the demands of repeated switching behaviors as rats were unimpaired on initial discrimination acquisition or retention of probabilistic learning. Taken together, these novel findings compliment other work discussed supporting a role for the LHb in action selection when cognitive or emotional demands are increased. Finally, we discuss future mechanisms by which a superior understanding of the LHb can be obtained through additional examination of behavioral flexibility tasks.

Large-scale intrinsic functional network organization along the long axis of the human medial temporal lobe

The medial temporal lobe (MTL), encompassing the hippocampus and parahippocampal gyrus (PHG), is a heterogeneous structure which plays a critical role in memory and cognition. Here, we investigate functional architecture of the human MTL along the long axis of the hippocampus and PHG. The hippocampus showed stronger connectivity with striatum, ventral tegmental area and amygdala-regions important for integrating reward and affective signals, whereas the PHG showed stronger connectivity with unimodal and polymodal association cortices. In the hippocampus, the anterior node showed stronger connectivity with the anterior medial temporal lobe and the posterior node showed stronger connectivity with widely distributed cortical and subcortical regions including those involved in sensory and reward processing. In the PHG, differences were characterized by a gradient of increasing anterior-to-posterior connectivity with core nodes of the default mode network. Left and right MTL connectivity patterns were remarkably similar, except for stronger left than right MTL connectivity with regions in the left MTL, the ventral striatum and default mode network. Graph theoretical analysis of MTL-based networks revealed higher node centrality of the posterior, compared to anterior and middle hippocampus. The PHG showed prominent gradients in both node degree and centrality along its anterior-to-posterior axis. Our findings highlight several novel aspects of functional heterogeneity in connectivity along the long axis of the human MTL and provide new insights into how its network organization supports integration and segregation of signals from distributed brain areas. The implications of our findings for a principledunderstanding of distributed pathways that support memory and cognition are discussed.

The influence of trial order on learning from reward vs. punishment in a probabilistic categorization task: experimental and computational analyses

Previous research has shown that trial ordering affects cognitive performance, but this has not been tested using category-learning tasks that differentiate learning from reward and punishment. Here, we tested two groups of healthy young adults using a probabilistic category learning task of reward and punishment in which there are two types of trials (reward, punishment) and three possible outcomes: (1) positive feedback for correct responses in reward trials; (2) negative feedback for incorrect responses in punishment trials; and (3) no feedback for incorrect answers in reward trials and correct answers in punishment trials. Hence, trials without feedback are ambiguous, and may represent either successful avoidance of punishment or failure to obtain reward. In Experiment 1, the first group of subjects received an intermixed task in which reward and punishment trials were presented in the same block, as a standard baseline task. In Experiment 2, a second group completed the separated task, in which reward and punishment trials were presented in separate blocks. Additionally, in order to understand the mechanisms underlying performance in the experimental conditions, we fit individual data using a Q-learning model. Results from Experiment 1 show that subjects who completed the intermixed task paradoxically valued the no-feedback outcome as a reinforcer when it occurred on reinforcement-based trials, and as a punisher when it occurred on punishment-based trials. This is supported by patterns of empirical responding, where subjects showed more win-stay behavior following an explicit reward than following an omission of punishment, and more lose-shift behavior following an explicit punisher than following an omission of reward. In Experiment 2, results showed similar performance whether subjects received reward-based or punishment-based trials first. However, when the Q-learning model was applied to these data, there were differences between subjects in the reward-first and punishment-first conditions on the relative weighting of neutral feedback. Specifically, early training on reward-based trials led to omission of reward being treated as similar to punishment, but prior training on punishment-based trials led to omission of reward being treated more neutrally. This suggests that early training on one type of trials, specifically reward-based trials, can create a bias in how neutral feedback is processed, relative to those receiving early punishment-based training or training that mixes positive and negative outcomes.

Integrative hippocampal and decision-making neurocircuitry during goal-relevant predictions and encoding

It has become clear that the hippocampus plays a critical role in the identification of new contexts and for the detection of changes in familiar contexts. The hippocampus accomplishes these goals through a continual process of comparing predicted features of a context or situation to those actually experienced. A mismatch between expected and experienced context expectations is thought to lead to the generation of a context prediction error (Mizumori, 2013) that functionally alerts connected brain areas to alter subsequent decision making and response selection. Little is understood about how hippocampal context analyses impact downstream decision processes. This issue is evaluated here first by comparing the nature of the information represented in hippocampus and decision-related midbrain-striatal structures, while rats perform a hippocampal-dependent spatial memory task in which rewards of different value are found at different locations. In contrast to place-specific and egocentric neural representations, neural representations of goal information are broadly distributed in hippocampal and decision neural circuitry, but they appear in different forms for different brain structures. It is suggested that further researching on how goal information processing occurs in hippocampus and decision neural circuitry may reveal insights into the nature of the interaction between memory and decision systems. The second part of this review describes neural pathways by which hippocampal context information might arrive within the decision circuit. The third section presents a hypothesis that the nature of the interactions between hippocampal and midbrain-striatal circuitry is regulated by the prefrontal cortex.

Increased cortico-striatal connectivity during motor practice contributes to the consolidation of motor memory in writer's cramp patients

Sensorimotor representations of movements are created in the sensorimotor network through repeated practice to support successful and effortless performance. Writer's cramp (WC) is a disorder acquired through extensive practice of finger movements, and it is likely associated with the abnormal acquisition of sensorimotor representations. We investigated (i) the activation and connectivity changes in the brain network supporting the acquisition of sensorimotor representations of finger sequences in patients with WC and (ii) the link between these changes and consolidation of motor performance 24 h after the initial practice. Twenty-two patients with WC and 22 age-matched healthy volunteers practiced a complex sequence with the right (pathological) hand during functional MRI recording. Speed and accuracy were measured immediately before and after practice (day 1) and 24 h after practice (day 2). The two groups reached equivalent motor performance on day 1 and day 2. During motor practice, patients with WC had (i) reduced hippocampal activation and hippocampal-striatal functional connectivity; and (ii) overactivation of premotor-striatal areas, whose connectivity correlated with motor performance after consolidation. These results suggest that patients with WC use alternative networks to reach equiperformance in the acquisition of new motor memories.

Episodic memory encoding interferes with reward learning and decreases striatal prediction errors

Learning is essential for adaptive decision making. The striatum and its dopaminergic inputs are known to support incremental reward-based learning, while the hippocampus is known to support encoding of single events (episodic memory). Although traditionally studied separately, in even simple experiences, these two types of learning are likely to co-occur and may interact. Here we sought to understand the nature of this interaction by examining how incremental reward learning is related to concurrent episodic memory encoding. During the experiment, human participants made choices between two options (colored squares), each associated with a drifting probability of reward, with the goal of earning as much money as possible. Incidental, trial-unique object pictures, unrelated to the choice, were overlaid on each option. The next day, participants were given a surprise memory test for these pictures. We found that better episodic memory was related to a decreased influence of recent reward experience on choice, both within and across participants. fMRI analyses further revealed that during learning the canonical striatal reward prediction error signal was significantly weaker when episodic memory was stronger. This decrease in reward prediction error signals in the striatum was associated with enhanced functional connectivity between the hippocampus and striatum at the time of choice. Our results suggest a mechanism by which memory encoding may compete for striatal processing and provide insight into how interactions between different forms of learning guide reward-based decision making.

Processing of action- but not stimulus-related prediction errors differs between active and observational feedback learning

Learning of stimulus-response-outcome associations is driven by outcome prediction errors (PEs). Previous studies have shown larger PE-dependent activity in the striatum for learning from own as compared to observed actions and the following outcomes despite comparable learning rates. We hypothesised that this finding relates primarily to a stronger integration of action and outcome information in active learners. Using functional magnetic resonance imaging, we investigated brain activations related to action-dependent PEs, reflecting the deviation between action values and obtained outcomes, and action-independent PEs, reflecting the deviation between subjective values of response-preceding cues and obtained outcomes. To this end, 16 active and 15 observational learners engaged in a probabilistic learning card-guessing paradigm. On each trial, active learners saw one out of five cues and pressed either a left or right response button to receive feedback (monetary win or loss). Each observational learner observed exactly those cues, responses and outcomes of one active learner. Learning performance was assessed in active test trials without feedback and did not differ between groups. For both types of PEs, activations were found in the globus pallidus, putamen, cerebellum, and insula in active learners. However, only for action-dependent PEs, activations in these structures and the anterior cingulate were increased in active relative to observational learners. Thus, PE-related activity in the reward system is not generally enhanced in active relative to observational learning but only for action-dependent PEs. For the cerebellum, additional activations were found across groups for cue-related uncertainty, thereby emphasising the cerebellum's role in stimulus-outcome learning.

Mechanisms for widespread hippocampal involvement in cognition

The quintessential memory system in the human brain--the hippocampus and surrounding medial temporal lobe--is often treated as a module for the formation of conscious, or declarative, memories. However, growing evidence suggests that the hippocampus plays a broader role in memory and cognition and that theories organizing memory into strictly dedicated systems may need to be updated. We first consider the historical evidence for the specialized role of the hippocampus in declarative memory. Then, we describe the serendipitous encounter that motivated the special section in this issue, based on parallel research from our labs that suggested a more pervasive contribution of the hippocampus to cognition beyond declarative memory. Finally, we develop a theoretical framework that describes 2 general mechanisms for how the hippocampus interacts with other brain systems and cognitive processes: the memory modulation hypothesis, in which mnemonic representations in the hippocampus modulate the operation of other systems, and the adaptive function hypothesis, in which specialized computations in the hippocampus are recruited as a component of both mnemonic and nonmnemonic functions. This framework is consistent with an emerging view that the most fertile ground for discovery in cognitive psychology and neuroscience lies at the interface between parts of the mind and brain that have traditionally been studied in isolation.

Homeostatic regulation of memory systems and adaptive decisions

While it is clear that many brain areas process mnemonic information, understanding how their interactions result in continuously adaptive behaviors has been a challenge. A homeostatic-regulated prediction model of memory is presented that considers the existence of a single memory system that is based on a multilevel coordinated and integrated network (from cells to neural systems) that determines the extent to which events and outcomes occur as predicted. The “multiple memory systems of the brain” have in common output that signals errors in the prediction of events and/or their outcomes, although these signals differ in terms of what the error signal represents (e.g., hippocampus: context prediction errors vs. midbrain/striatum: reward prediction errors). The prefrontal cortex likely plays a pivotal role in the coordination of prediction analysis within and across prediction brain areas. By virtue of its widespread control and influence, and intrinsic working memory mechanisms. Thus, the prefrontal cortex supports the flexible processing needed to generate adaptive behaviors and predict future outcomes. It is proposed that prefrontal cortex continually and automatically produces adaptive responses according to homeostatic regulatory principles: prefrontal cortex may serve as a controller that is intrinsically driven to maintain in prediction areas an experience-dependent firing rate set point that ensures adaptive temporally and spatially resolved neural responses to future prediction errors. This same drive by prefrontal cortex may also restore set point firing rates after deviations (i.e. prediction errors) are detected. In this way, prefrontal cortex contributes to reducing uncertainty in prediction systems. An emergent outcome of this homeostatic view may be the flexible and adaptive control that prefrontal cortex is known to implement (i.e. working memory) in the most challenging of situations. Compromise to any of the prediction circuits should result in rigid and suboptimal decision making and memory as seen in addiction and neurological disease. © 2013 The Authors. Hippocampus Published by Wiley Periodicals, Inc.

A role for the lateral dorsal tegmentum in memory and decision neural circuitry

A role for the hippocampus in memory is clear, although the mechanism for its contribution remains a matter of debate. Converging evidence suggests that hippocampus evaluates the extent to which context-defining features of events occur as expected. The consequence of mismatches, or prediction error, signals from hippocampus is discussed in terms of its impact on neural circuitry that evaluates the significance of prediction errors: Ventral tegmental area (VTA) dopamine cells burst fire to rewards or cues that predict rewards (Schultz, Dayan, & Montague, 1997). Although the lateral dorsal tegmentum (LDTg) importantly controls dopamine cell burst firing (Lodge & Grace, 2006) the behavioral significance of the LDTg control is not known. Therefore, we evaluated LDTg functional activity as rats performed a spatial memory task that generates task-dependent reward codes in VTA (Jo, Lee, & Mizumori, 2013; Puryear, Kim, & Mizumori, 2010) and another VTA afferent, the pedunculopontine nucleus (PPTg, Norton, Jo, Clark, Taylor, & Mizumori, 2011). Reversible inactivation of the LDTg significantly impaired choice accuracy. LDTg neurons coded primarily egocentric information in the form of movement velocity, turning behaviors, and behaviors leading up to expected reward locations. A subset of the velocity-tuned LDTg cells also showed high frequency bursts shortly before or after reward encounters, after which they showed tonic elevated firing during consumption of small, but not large, rewards. Cells that fired before reward encounters showed stronger correlations with velocity as rats moved toward, rather than away from, rewarded sites. LDTg neural activity was more strongly regulated by egocentric behaviors than that observed for PPTg or VTA cells that were recorded by Puryear et al. and Norton et al. While PPTg activity was uniquely sensitive to ongoing sensory input, all three regions encoded reward magnitude (although in different ways), reward expectation, and reward encounters. Only VTA encoded reward prediction errors. LDTg may inform VTA about learned goal-directed movement that reflects the current motivational state, and this in turn may guide VTA determination of expected subjective goal values. When combined it is clear the LDTg and PPTg provide only a portion of the information that dopamine cells need to assess the value of prediction errors, a process that is essential to future adaptive decisions and switches of cognitive (i.e. memorial) strategies and behavioral responses.

Differential reward responses during competition against in- and out-of-network others

Social interactions occur within a variety of different contexts--cooperative/competitive--and often involve members of our social network. Here, we investigated whether social network modulated the value placed on positive outcomes during a competitive context. Eighteen human participants played a simple card-guessing game with three different competitors: a close friend (in-network), a confederate (out-of-network) and a random number generator (non-social condition) while undergoing functional magnetic resonance imaging. Neuroimaging results at the time of outcome receipt demonstrated a significant main effect of competitor across multiple regions of medial prefrontal cortex, with Blood Oxygen Level Dependent (BOLD) responses strongest when competing against one's friend compared with all other conditions. Striatal BOLD responses demonstrated a more general sensitivity to positive compared with negative monetary outcomes, which an exploratory analysis revealed to be stronger when interacting with social, compared with non-social, competitors. Interestingly, a Granger causality analysis indicated directed influences sent from an medial prefrontal cortex (mPFC) region, which shows social network differentiation of outcomes, and the ventral striatum bilaterally. Our results suggest that when competing against others of varying degrees of social network, mPFC differentially values these outcomes, perhaps treating in-network outcomes as more informative, leaving the striatum to more general value computations.

Changes in GABA and glutamate concentrations during memory tasks in patients with Parkinson's disease undergoing DBS surgery

Until now direct neurochemical measurements during memory tasks have not been accomplished in the human basal ganglia. It has been proposed, based on both functional imaging studies and psychometric testing in normal subjects and in patients with Parkinson’s disease (PD), that the basal ganglia is responsible for the performance of feedback-contingent implicit memory tasks. To measure neurotransmitters, we used in vivo microdialysis during deep brain stimulation (DBS) surgery. We show in the right subthalamic nucleus (STN) of patients with PD a task-dependent change in the concentrations of glutamate and GABA during an implicit memory task relative to baseline, while no difference was found between declarative memory tasks. The five patients studied had a significant decrease in the percent concentration of GABA and glutamate during the performance of the weather prediction task (WPT). We hypothesize, based on current models of basal ganglia function, that this decrease in the concentration is consistent with expected dysfunction in basal ganglia networks in patients with PD.

Context prediction analysis and episodic memory

Events that happen at a particular place and time come to define our episodic memories. Extensive experimental and clinical research illustrate that the hippocampus is central to the processing of episodic memories, and this is in large part due to its analysis of context information according to spatial and temporal references. In this way, hippocampus defines ones expectations for a given context as well as detects errors in predicted contextual features. The detection of context prediction errors is hypothesized to distinguished events into meaningful epochs that come to be recalled as separate episodic memories. The nature of the spatial and temporal context information processed by hippocampus is described, as is a hypothesis that the apparently self-regulatory nature of hippocampal context processing may ultimately be mediated by natural homeostatic operations and plasticity. Context prediction errors by hippocampus are suggested to be valued by the midbrain dopamine system, the output of which is ultimately fed back to hippocampus to update memory-driven context expectations for future events. Thus, multiple network functions (both within and outside hippocampus) combine to result in adaptive episodic memories.

Learning to obtain reward, but not avoid punishment, is affected by presence of PTSD symptoms in male veterans: empirical data and computational model

Post-traumatic stress disorder (PTSD) symptoms include behavioral avoidance which is acquired and tends to increase with time. This avoidance may represent a general learning bias; indeed, individuals with PTSD are often faster than controls on acquiring conditioned responses based on physiologically-aversive feedback. However, it is not clear whether this learning bias extends to cognitive feedback, or to learning from both reward and punishment. Here, male veterans with self-reported current, severe PTSD symptoms (PTSS group) or with few or no PTSD symptoms (control group) completed a probabilistic classification task that included both reward-based and punishment-based trials, where feedback could take the form of reward, punishment, or an ambiguous “no-feedback” outcome that could signal either successful avoidance of punishment or failure to obtain reward. The PTSS group outperformed the control group in total points obtained; the PTSS group specifically performed better than the control group on reward-based trials, with no difference on punishment-based trials. To better understand possible mechanisms underlying observed performance, we used a reinforcement learning model of the task, and applied maximum likelihood estimation techniques to derive estimated parameters describing individual participants’ behavior. Estimations of the reinforcement value of the no-feedback outcome were significantly greater in the control group than the PTSS group, suggesting that the control group was more likely to value this outcome as positively reinforcing (i.e., signaling successful avoidance of punishment). This is consistent with the control group’s generally poorer performance on reward trials, where reward feedback was to be obtained in preference to the no-feedback outcome. Differences in the interpretation of ambiguous feedback may contribute to the facilitated reinforcement learning often observed in PTSD patients, and may in turn provide new insight into how pathological behaviors are acquired and maintained in PTSD.