Deep brain stimulation of the subthalamic nucleus restores spatial reversal learning in patients with Parkinson's disease

Spatial learning and navigation are supported by distinct memory systems in the human brain such as the hippocampus-based navigational system and the striatum-cortex-based system involved in motor sequence, habit and reversal learning. Here, we studied the role of subthalamic circuits in hippocampus-associated spatial memory and striatal-associated spatial reversal learning formation in patients with Parkinson's disease, who underwent a deep brain stimulation of the subthalamic nucleus. Deep brain stimulation patients (Parkinson's disease-subthalamic nucleus: n = 26) and healthy subjects (n = 15) were tested in a novel experimental spatial memory task based on the Morris water maze that assesses both hippocampal place memory as well as spatial reversal learning. All subjects were trained to navigate to a distinct spatial location hidden within the virtual environment during 16 learning trials in a subthalamic nucleus Stim-On condition. Patients were then randomized into two groups with either a deep brain stimulation On or Off condition. Four hours later, subjects were retested in a delayed recall and reversal learning condition. The reversal learning was realized with a new hidden location that should be memorized during six consecutive trials. The performance was measured by means of an index indicating the improvement during the reversal learning. In the delayed recall condition, neither patients, healthy subjects nor the deep brain stimulation On- versus Off groups showed a difference in place memory performance of the former trained location. In the reversal learning condition, healthy subjects (reversal index 2.0) and patients in the deep brain stimulation On condition (reversal index 1.6) showed a significant improvement. However, patients in the deep brain stimulation Off condition (reversal index 1.1) performed significantly worse and did not improve. There were no differences between all groups in a final visual guided navigation task with a visible target. These results suggest that deep brain stimulation of subthalamic nucleus restores spatial reversal learning in a virtual navigation task in patients with Parkinson's disease and gives insight into the neuromodulation effects on cognition of subthalamic circuits in Parkinson's disease.

Declarative Learning Mechanisms Support Declarative but Not Probabilistic Feedback-Based Learning in Children with Developmental Language Disorder (DLD)

Declarative and probabilistic feedback-based learning was evaluated in 8-12-year-old school-age children with developmental language disorder (DLD; n = 14) and age-matched children with typical development (TD; n = 15). Children performed a visual two-choice word-learning task and a visual probabilistic classification task while their electroencephalogram (EEG) was recorded non-invasively from the scalp. Behavioral measures of accuracy and response to feedback, and electrophysiological responses to feedback were collected and compared between the two groups. While behavioral data indicated poorer performance by children with DLD in both learning paradigms, and similar response patterns to positive and negative feedback, electrophysiological data highlighted processing patterns in the DLD group that differed by task. More specifically, in this group, feedback processing in the context of declarative learning, which is known to be dominated by the medial temporal lobe (MTL), was associated with enhanced N170, an event-related brain potential (ERP) associated with MTL activation. The N170 amplitude was found to be correlated with declarative task performance in the DLD group. During probabilistic learning, known to be governed by the striatal-based learning system, the feedback-related negativity (FRN) ERP, which is the product of the cortico-striatal circuit dominated feedback processing. Within the context of probabilistic learning, enhanced N170 was associated with poor learning in the TD group, suggesting that MTL activation during probabilistic learning disrupts learning. These results are interpreted within the context of a proposed feedback parity hypothesis suggesting that in children with DLD, the system that dominates learning (i.e., MTL during declarative learning and the striatum during probabilistic learning) dominates and supports feedback processing.

Instructed motivational states bias reinforcement learning and memory formation

Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use. We manipulated motivational states via cover stories for a reinforcement learning task: The imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded. In an initial sample and a preregistered replication, we demonstrated that imperative motivation increased exploitation during reinforcement learning. Conversely, interrogative motivation increased directed (but not random) exploration, despite the cost to participants' earnings. At test, the interrogative group was more accurate at recognizing paintings and recalling associated values. In the interrogative group, higher value paintings were more likely to be remembered; imperative motivation disrupted this effect of reward modulating memory. Overall, we demonstrate that a prelearning motivational manipulation can bias learning and memory, bearing implications for education, behavior change, clinical interventions, and communication.

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders

BACKGROUND

Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neurodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, including reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dissociate model-based from model-free learning in human learners.

RESULTS

Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised.

CONCLUSIONS

Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-outcome associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks' structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them.

An Implicit-Explicit Framework for Intervention Methods in Developmental Language Disorder

PURPOSE

The growing interest in framing intervention approaches as either implicit or explicit calls for a discussion of what makes intervention approaches engage each of these learning systems, with the goal of achieving a shared framework. This tutorial presents evidence for the interaction between implicit and explicit learning systems, and it highlights the intervention characteristics that promote implicit or explicit learning as well as outcome measures that tap into implicit or explicit knowledge. This framework is then applied to eight common intervention approaches and notable combinations of approaches to unpack their differential engagement of implicit and explicit learning.

CONCLUSIONS

Many intervention characteristics (e.g., instructions, elicitation techniques, feedback) can be manipulated to move an intervention along the implicit-explicit continuum. Given the bias for using explicit learning strategies that develops throughout childhood and into adulthood, clinicians should be aware that most interventions (even those that promote implicit learning) will engage the explicit learning system. However, increased awareness of the implicit and explicit learning systems and their cognitive demands will allow clinicians to choose the most appropriate intervention for the target behavior.

Reward timing matters in motor learning

Reward timing, that is, the delay after which reward is delivered following an action is known to strongly influence reinforcement learning. Here, we asked if reward timing could also modulate how people learn and consolidate new motor skills. In 60 healthy participants, we found that delaying reward delivery by a few seconds influenced motor learning. Indeed, training with a short reward delay (1 s) induced continuous improvements in performance, whereas a long reward delay (6 s) led to initially high learning rates that were followed by an early plateau in the learning curve and a lower performance at the end of training. Participants who learned the skill with a long reward delay also exhibited reduced overnight memory consolidation. Overall, our data show that reward timing affects the dynamics and consolidation of motor learning, a finding that could be exploited in future rehabilitation programs.

Delaying feedback compensates for impaired reinforcement learning in developmental dyslexia

A theoretical framework suggests that developmental dyslexia is characterized by abnormalities in brain structures underlying the procedural learning and memory systems while the declarative learning and memory systems are presumed to remain intact or even enhanced (Procedural Deficit Hypothesis). This notion has been supported by a substantial body of research, which focused on each system independently. However, less attention has been paid to interactions between these memory systems which may provide insights as to learning situations and conditions in which learning in dyslexia can be improved. The current study was undertaken to examine these important but unresolved issues. To this end, probabilistic reinforcement learning and episodic memory tasks were examined in participants with dyslexia and neurotypicals simultaneously within a single task. Feedback timing presentation was manipulated, building on prior research indicating that delaying feedback timing shifts striatal-based probabilistic learning, to become more hippocampal-dependent. It was hypothesized that if the procedural learning and memory systems are impaired in dyslexia, performance will be impaired under conditions that encourage procedural memory engagement (immediate feedback trials) but not under conditions that promote declarative memory processing (long delayed feedback trials). It was also predicted that the ability to incidentally acquire episodic information would be preserved in dyslexia. The results supported these predictions. Participants with dyslexia were impaired in probabilistic learning of cue-outcome associations compared to neurotypicals in an immediate feedback condition, but not when feedback on choices was presented after a long delay. Furthermore, participants with dyslexia demonstrated similar performance to neurotypicals in a task requiring incidental episodic memory formation. These findings attest to a dissociation between procedural-based and declarative-based learning in developmental dyslexia within a single task, a finding that adds discriminative validity to the Procedural Deficit Hypothesis. Just as important, the present findings suggest that training conditions designed to shift the load from midbrain/striatal systems to declarative memory mechanisms have the potential to compensate for impaired learning in developmental dyslexia.

Signed and unsigned reward prediction errors dynamically enhance learning and memory

Memory helps guide behavior, but which experiences from the past are prioritized? Classic models of learning posit that events associated with unpredictable outcomes as well as, paradoxically, predictable outcomes, deploy more attention and learning for those events. Here, we test reinforcement learning and subsequent memory for those events, and treat signed and unsigned reward prediction errors (RPEs), experienced at the reward-predictive cue or reward outcome, as drivers of these two seemingly contradictory signals. By fitting reinforcement learning models to behavior, we find that both RPEs contribute to learning by modulating a dynamically changing learning rate. We further characterize the effects of these RPE signals on memory and show that both signed and unsigned RPEs enhance memory, in line with midbrain dopamine and locus-coeruleus modulation of hippocampal plasticity, thereby reconciling separate findings in the literature.

Dopamine is associated with prioritization of reward-associated memories in Parkinson's disease

Patients with Parkinson's disease have reduced reward sensitivity related to dopaminergic neuron loss, which is associated with impairments in reinforcement learning. Increasingly, however, dopamine-dependent reward signals are recognized to play an important role beyond reinforcement learning. In particular, it has been shown that reward signals mediated by dopamine help guide the prioritization of events for long-term memory consolidation. Meanwhile, studies of memory in patients with Parkinson's disease have focused on overall memory capacity rather than what is versus what isn't remembered, leaving open questions about the effect of dopamine replacement on the prioritization of memories by reward and the time-dependence of this effect. The current study sought to fill this gap by testing the effect of reward and dopamine on memory in patients with Parkinson's disease. We tested the effect of dopamine modulation and reward on two forms of long-term memory: episodic memory for neutral objects and memory for stimulus-value associations. We measured both forms of memory in a single task, adapting a standard task of reinforcement learning with incidental episodic encoding events of trial-unique objects. Objects were presented on each trial at the time of feedback, which was either rewarding or not. Memory for the trial-unique images and for the stimulus-value associations, and the influence of reward on both, was tested immediately after learning and 2 days later. We measured performance in Parkinson's disease patients tested either ON or OFF their dopaminergic medications and in healthy older control subjects. We found that dopamine was associated with a selective enhancement of memory for reward-associated images, but that it did not influence overall memory capacity. Contrary to predictions, this effect did not differ between the immediate and delayed memory tests. We also found that while dopamine had an effect on reward-modulated episodic memory, there was no effect of dopamine on memory for stimulus-value associations. Our results suggest that impaired prioritization of cognitive resource allocation may contribute to the early cognitive deficits of Parkinson's disease.

Does Acute Stress Impact Declarative and Procedural Learning?

It is well established that acute stress can influence memory function, yet its influence may differ across memory systems. Whereas stress sometimes exerts a negative influence on declarative learning, it does not necessarily harm learning in general, as demonstrated in the case of procedural learning. Probabilistic category learning is mediated by the striatum, but delaying feedback by a few seconds shifts learning to become more hippocampal-dependent. Here, we examined the influence of acute stress on this type of learning, under different conditions that favor either procedural-based (immediate feedback) vs. declarative-based (delayed feedback) learning. Sixty-two participants randomly assigned to either stress or non-stress groups, performed a probabilistic category learning task, in which they were instructed to learn associations between cues and outcomes under different feedback conditions (immediate feedback, short-delayed feedback, and long-delayed feedback). Acute stress was induced by the Maastricht Acute Stress Test (MAST), and stress levels were gauged by Galvanic Skin Response (GSR) measures and a self-reported questionnaire. Results showed that although the MAST was effective in inducing stress, this did not harm learning in either of the feedback conditions. These findings suggest that not all hippocampal-based learning types are negatively influenced by stress.

Competitive and cooperative interactions between medial temporal and striatal learning systems

The striatum and medial temporal lobes (MTL) exhibit dissociable roles during learning. Whereas the striatum and its network of thalamic relays and cortical nodes are necessary for nondeclarative learning, the MTL and associated network are required for declarative learning. Several studies have suggested that these networks are functionally competitive during learning. Since these discoveries, however, evidence has accumulated that they can operate in a cooperative fashion. In this review, we discuss evidence for both competition and cooperation between these systems during learning, with the aim of reconciling these seemingly contradictory findings. Examples of cooperation between the striatum and MTL have been provided, especially during consolidation and generalization of knowledge, and do not appear to be precluded by differences in functional specialization. However, whether these systems cooperate or compete does seem to depend on the phase of learning and cognitive or motor aspects of the task. The involvement of other regions, such as midbrain dopaminergic nuclei and the prefrontal cortex, may promote and mediate cooperation between the striatum and the MTL during learning. Building on this body of research, we propose a model for striatum-MTL interactions in learning and memory and attempt to predict, in general terms, when cooperation or competition will occur.

Motor Learning Deficits in Parkinson's Disease (PD) and Their Effect on Training Response in Gait and Balance: A Narrative Review

Parkinson's disease (PD) is a neurological disorder traditionally associated with degeneration of the dopaminergic neurons within the substantia nigra, which results in bradykinesia, rigidity, tremor, and postural instability and gait disability (PIGD). The disorder has also been implicated in degradation of motor learning. While individuals with PD are able to learn, certain aspects of learning, especially automatic responses to feedback, are faulty, resulting in a reliance on feedforward systems of movement learning and control. Because of this, patients with PD may require more training to achieve and retain motor learning and may require additional sensory information or motor guidance in order to facilitate this learning. Furthermore, they may be unable to maintain these gains in environments and situations in which conscious effort is divided (such as dual-tasking). These shortcomings in motor learning could play a large part in degenerative gait and balance symptoms often seen in the disease, as patients are unable to adapt to gradual sensory and motor degradation. Research has shown that physical and exercise therapy can help patients with PD to adapt new feedforward strategies to partially counteract these symptoms. In particular, balance, treadmill, resistance, and repeated perturbation training therapies have been shown to improve motor patterns in PD. However, much research is still needed to determine which of these therapies best alleviates which symptoms of PIGD, the needed dose and intensity of these therapies, and long-term retention effects. The benefits of such technologies as augmented feedback, motorized perturbations, virtual reality, and weight-bearing assistance are also of interest. This narrative review will evaluate the effect of PD on motor learning and the effect of motor learning deficits on response to physical therapy and training programs, focusing specifically on features related to PIGD. Potential methods to strengthen therapeutic effects will be discussed.

Modulation of associative learning in the hippocampal-striatal circuit based on item-set similarity

Mounting evidence suggests that the medial temporal lobe (MTL) and striatal learning systems support different forms of learning, which can be competitive or cooperative depending on task demands. We have previously shown how activity in these regions can be modulated in a conditional visuomotor associative learning task based on the consistency of response mappings or reward feedback (Mattfeld & Stark, 2015). Here, we examined the shift in learning towards the MTL and away from the striatum by placing strong demands on pattern separation, a process of orthogonalizing similar inputs into distinct representations. Mnemonically, pattern separation processes have been shown to rely heavily on processing in the hippocampus. Therefore, we predicted modulation of hippocampal activity by pattern separation demands, but no such modulation of striatal activity. Using a variant of the conditional visuomotor associative learning task that we have used previously, we presented participants with two blocked conditions: items with high and low perceptual overlap during functional magnetic resonance imaging (fMRI). As predicted, we observed learning-related activity in the hippocampus, which was greater in the high than the low overlap condition, particularly in the dentate gyrus. In contrast, the associative striatum also showed learning related activity, but it was not modulated by overlap condition. Using functional connectivity analyses, we showed that the correlation between the hippocampus and dentate gyrus with the associative striatum was differentially modulated by high vs. low overlap, suggesting that the coordination between these regions was affected when pattern separation demands were high. These findings contribute to a growing literature that suggests that the hippocampus and striatal network both contribute to the learning of arbitrary associations that are computationally distinct and can be altered by task demands.

Feedback Timing Modulates Probabilistic Learning in Adults with ADHD

Attention deficit hyperactivity disorder (ADHD) has been associated primarily with executive function deficits. Emerging findings suggest, however, that procedural learning may be compromised as well. To this effect, we recently showed that feedback-based procedural learning is selectively impaired in ADHD, results that coincide with dopaminergic alterations associated with ADHD. Key questions, however, remain unresolved, among which are the learning conditions that may improve procedural learning in ADHD. Here we examined feedback-based probabilistic learning during conditions that engage procedural and declarative learning systems to different degrees, depending on feedback timing. ADHD and control participants carried out a probabilistic learning task in which they were required to learn to associate between cues and outcomes, where outcomes were presented either immediately or with a short/long delays. Whereas performance in probabilistic learning in ADHD participants was comparable to controls in delayed feedback conditions, during both learning and test phases, their performance diminished when feedback was immediate. Furthermore, ADHD symptom severity was negatively correlated with the ability to learn from immediate feedback. These results suggest that feedback-based probabilistic learning can be improved in ADHD, provided appropriate conditions. By shifting the load from midbrain/striatal systems to declarative memory mechanisms, behavioral performance in ADHD populations can be remediated.

Paradoxical Decision-Making: A Framework for Understanding Cognition in Parkinson's Disease

People with Parkinson's disease (PD) show impaired decision-making when sensory and memory information must be combined. This recently identified impairment results from an inability to accumulate the proper amount of information needed to make a decision and appears to be independent of dopamine tone and reinforcement learning mechanisms. Although considerable work focuses on PD and decisions involving risk and reward, in this Opinion article we propose that the emerging findings in perceptual decision-making highlight the multisystem nature of PD, and that unraveling the neuronal circuits underlying perceptual decision-making impairment may help in understanding other cognitive impairments in people with PD. We also discuss how a decision-making framework may be extended to gain insights into mechanisms of motor impairments in PD.

Dissociable effects of surprising rewards on learning and memory

Reward-prediction errors track the extent to which rewards deviate from expectations, and aid in learning. How do such errors in prediction interact with memory for the rewarding episode? Existing findings point to both cooperative and competitive interactions between learning and memory mechanisms. Here, we investigated whether learning about rewards in a high-risk context, with frequent, large prediction errors, would give rise to higher fidelity memory traces for rewarding events than learning in a low-risk context. Experiment 1 showed that recognition was better for items associated with larger absolute prediction errors during reward learning. Larger prediction errors also led to higher rates of learning about rewards. Interestingly we did not find a relationship between learning rate for reward and recognition-memory accuracy for items, suggesting that these two effects of prediction errors were caused by separate underlying mechanisms. In Experiment 2, we replicated these results with a longer task that posed stronger memory demands and allowed for more learning. We also showed improved source and sequence memory for items within the high-risk context. In Experiment 3, we controlled for the difficulty of reward learning in the risk environments, again replicating the previous results. Moreover, this control revealed that the high-risk context enhanced item-recognition memory beyond the effect of prediction errors. In summary, our results show that prediction errors boost both episodic item memory and incremental reward learning, but the two effects are likely mediated by distinct underlying systems. (PsycINFO Database Record

Dynamic Flexibility in Striatal-Cortical Circuits Supports Reinforcement Learning

Complex learned behaviors must involve the integrated action of distributed brain circuits. Although the contributions of individual regions to learning have been extensively investigated, much less is known about how distributed brain networks orchestrate their activity over the course of learning. To address this gap, we used fMRI combined with tools from dynamic network neuroscience to obtain time-resolved descriptions of network coordination during reinforcement learning in humans. We found that learning to associate visual cues with reward involves dynamic changes in network coupling between the striatum and distributed brain regions, including visual, orbitofrontal, and ventromedial prefrontal cortex (n = 22; 13 females). Moreover, we found that this flexibility in striatal network coupling correlates with participants' learning rate and inverse temperature, two parameters derived from reinforcement learning models. Finally, we found that episodic learning, measured separately in the same participants at the same time, was related to dynamic connectivity in distinct brain networks. These results suggest that dynamic changes in striatal-centered networks provide a mechanism for information integration during reinforcement learning.SIGNIFICANCE STATEMENT Learning from the outcomes of actions, referred to as reinforcement learning, is an essential part of life. The roles of individual brain regions in reinforcement learning have been well characterized in terms of updating values for actions or cues. Missing from this account, however, is an understanding of how different brain areas interact during learning to integrate sensory and value information. Here we characterize flexible striatal-cortical network dynamics that relate to reinforcement learning behavior.

Perceptual decisions based on previously learned information are independent of dopaminergic tone

Both cognitive and motor symptoms in people with Parkinson's disease (PD) arise from either too little or too much dopamine (DA). Akinesia stems from DA neuronal cell loss, and dyskinesia often stems from an overdose of DA medication. Cognitive behaviors typically associated with frontal cortical function, such as working memory and task switching, are also affected by too little or too much DA in PD. Whether motor and cognitive circuits overlap in PD is unknown. In this article, we show that whereas motor performance improves in people with PD when on dopaminergic medication compared with off medication, perceptual decision-making based on previously learned information (priors) remains impaired whether on or off medications. To rule out effects of long-term DA treatment and dopaminergic neuronal loss such as occur in PD, we also tested a group of people with dopa-unresponsive focal dystonia, a disease that involves the basal ganglia, like PD, but has motor symptoms that are insensitive to dopamine treatment and is not thought to involve frontal cortical DA circuits, unlike PD. We found that people with focal dystonia showed intact perceptual decision-making performance but impaired use of priors in perceptual decision-making, similar to people with PD. Together, the results show a dissociation between motor and cognitive performance in people with PD and reveal a novel cognitive impairment, independent of sensory and motor impairment, in people with focal dystonia. The combined results from people with PD and people with focal dystonia provide mechanistic insights into the role of basal ganglia non-dopaminergic circuits in perceptual decision-making based on priors.

Local striatal reward signals can be predicted from corticostriatal connectivity

A defining feature of the basal ganglia is their anatomical organization into multiple cortico-striatal loops. A central tenet of this architecture is the idea that local striatal function is determined by its precise connectivity with cortex, creating a functional topography that is mirrored within cortex and striatum. Here we formally test this idea using both human anatomical and functional imaging, specifically asking whether within striatal subregions one can predict between-voxel differences in functional signals based on between-voxel differences in corticostriatal connectivity. We show that corticostriatal connectivity profiles predict local variation in reward signals in bilateral caudate nucleus and putamen, expected value signals in bilateral caudate nucleus, and response effector activity in bilateral putamen. These data reveal that, even within individual striatal regions, local variability in corticostriatal anatomical connectivity predicts functional differentiation.

•

We use diffusion imaging to predict striatal reinforcement learning BOLD responses.

•

Structural cortico-striatal connectivity can explain intra-regional BOLD variability.

•

These results suggest a fine functional parcellation based on afferent connectivity.

Cognitive states influence dopamine-driven aberrant learning in Parkinson's disease

Individual differences in dopaminergic tone underlie tendencies to learn from reward versus punishment. These effects are well documented in Parkinson's patients, who vacillate between low and high tonic dopaminergic states as a function of medication. Yet very few studies have investigated the influence of higher-level cognitive states known to affect downstream dopaminergic learning in Parkinson's patients. A dopamine-dependent cognitive influence over learning would provide a candidate mechanism for declining cognitive integrity and motivation in Parkinson's patients. In this report we tested the influence of two high-level cognitive states (cost of conflict and value of volition) that have recently been shown to cause predictable learning biases in healthy young adults as a function of dopamine receptor subtype and dopaminergic challenge. It was hypothesized that Parkinson's patients OFF medication would have an enhanced cost of conflict and a decreased value of volition, and that these effects would be remediated or reversed ON medication. Participants included N = 28 Parkinson's disease patients who were each tested ON and OFF dopaminergic medication and 28 age- and sex-matched controls. The expected cost of conflict effect was observed in Parkinson's patients OFF versus ON medication, but only in those that were more recently diagnosed (<5 years). We found an unexpected effect in the value of volition task: medication compromised the ability to learn from difficult a-volitional (instructed) choices. This novel finding was also enhanced in recently diagnosed patients. The difference in learning biases ON versus OFF medication between these two tasks was strongly correlated, bolstering the idea that they tapped into a common underlying imbalance in dopaminergic tone that is particularly variable in earlier stage Parkinsonism. The finding that these decision biases are specific to earlier but not later stage disease may offer a chance for future studies to quantify phenotypic expressions of idiosyncratic disease progression.

Contributions of the hippocampus to feedback learning

Humans learn about the world in a variety of manners, including by observation, by associating cues in the environment, and via feedback. Across species, two brain structures have been predominantly involved in these learning processes: the hippocampus--supporting learning via observation and paired association--and the striatum--critical for feedback learning. This simple dichotomy, however, has recently been challenged by reports of hippocampal engagement in feedback learning, although the role of the hippocampus is not fully understood. The purpose of this experiment was to characterize the hippocampal response during feedback learning by manipulating varying levels of memory interference. Consistent with prior reports, feedback learning recruited the striatum and midbrain. Notably, feedback learning also engaged the hippocampus. The level of activity in these regions was modulated by the degree of memory interference, such that the greatest activation occurred during the highest level of memory interference. Importantly, the accuracy of information learned via feedback correlated with hippocampal activation and was reduced by the presence of high memory interference. Taken together, these findings provide evidence of hippocampal involvement in feedback learning by demonstrating both its relevance for the accuracy of information learned via feedback and its susceptibility to interference.

Dopamine Modulation of Intertemporal Decision-making: Evidence from Parkinson Disease

Choosing between smaller prompt rewards and larger later rewards is a common choice problem, and studies widely agree that frontostriatal circuits heavily innervated by dopamine are centrally involved. Understanding how dopamine modulates intertemporal choice has important implications for neurobiological models and for understanding the mechanisms underlying maladaptive decision-making. However, the specific role of dopamine in intertemporal decisions is not well understood. Dopamine may play a role in multiple aspects of intertemporal choices--the valuation of choice outcomes and sensitivity to reward delays. To assess the role of dopamine in intertemporal decisions, we tested Parkinson disease patients who suffer from dopamine depletion in the striatum, in either high (on medication, PDON) or low (off medication, PDOFF) dopaminergic states. Compared with both PDOFF and healthy controls, PDON made more farsighted choices and reduced their valuations less as a function of increasing time to reward. Furthermore, reduced discounting in the high dopaminergic state was robust across multiple measures, providing new evidence for dopamine's role in making decisions about the future.

Dopamine and Consolidation of Episodic Memory: Timing is Everything

Memory consolidation underpins adaptive behavior and dopaminergic networks may be critical for prolonged, selective information storage. To understand the time course of the dopaminergic contribution to memory consolidation in humans, here we investigate the effect of dopaminergic medication on recall and recognition in the short and longer term in Parkinson disease (PD). Fifteen people with PD were each tested on or off dopaminergic medication during learning/early consolidation (Day 1) and/or late consolidation (Day 2). Fifteen age-matched healthy participants were tested only once. On Day 1 participants learned new information, and early episodic memory was tested after 30 min. Then on Day 2, recall and recognition were retested after a 24-hr delay. Participants on medication on Day 1 recalled less information at 30 min and 24 hr. In contrast, patients on medication on Day 2 (8-24 hr after learning) recalled more information at 24 hr than those off medication. Although recognition sensitivity was unaffected by medication, response bias was dependent on dopaminergic state: Medication during learning induced a more liberal bias 24 hr later, whereas patients off medication during learning were more conservative responders 24 hr later. We use computational modeling to propose possible mechanisms for this change in response bias. In summary, dopaminergic medication in PD patients during learning impairs early consolidation of episodic memory and makes delayed responses more liberal, but enhances late memory consolidation presumably through a dopamine-dependent consolidation pathway that may be active during sleep.

Basal Ganglia Activity Mirrors a Benefit of Action and Reward on Long-Lasting Event Memory

The expectation of reward is known to enhance a consolidation of long-term memory for events. We tested whether this effect is driven by positive valence or action requirements tied to expected reward. Using a functional magnetic resonance imaging (fMRI) paradigm in young adults, novel images predicted gain or loss outcomes, which in turn were either obtained or avoided by action or inaction. After 24 h, memory for these images reflected a benefit of action as well as a congruence of action requirements and valence, namely, action for reward and inaction for avoidance. fMRI responses in the hippocampus, a region known to be critical for long-term memory function, reflected the anticipation of inaction. In contrast, activity in the putamen mirrored the congruence of action requirement and valence, whereas other basal ganglia regions mirrored overall action benefits on long-lasting memory. The findings indicate a novel type of functional division between the hippocampus and the basal ganglia in the motivational regulation of long-term memory consolidation, which favors remembering events that are worth acting for.

Decision-making impairments in Parkinson's disease as a by-product of defective cost-benefit analysis and feedback processing

Studies examining decision-making in people with Parkinson's disease (PD) show impaired performance on a variety of tasks. However, there are also demonstrations that patients with PD can make optimal decisions just like healthy age-matched controls. We propose that the reason for these mixed findings is that PD does not produce a generalized impairment of decision-making, but rather affects sub-components of this process. In this review we evaluate this hypothesis by considering the empirical evidence examining decision-making in PD. We suggest that of the various stages of the decision-making process, the most affected in PD are (1) the cost-benefit analysis stage and (2) the outcome evaluation stage. We consider the implications of this proposal for research in this area.

Functional compensation in the ventromedial prefrontal cortex improves memory-dependent decisions in older adults

Everyday consumer choices frequently involve memory, as when we retrieve information about consumer products when making purchasing decisions. In this context, poor memory may affect decision quality, particularly in individuals with memory decline, such as older adults. However, age differences in choice behavior may be reduced if older adults can recruit additional neural resources that support task performance. Although such functional compensation is well documented in other cognitive domains, it is presently unclear whether it can support memory-guided decision making and, if so, which brain regions play a role in compensation. The current study engaged younger and older humans in a memory-dependent choice task in which pairs of consumer products from a popular online-shopping site were evaluated with different delays between the first and second product. Using functional imaging (fMRI), we found that the ventromedial prefrontal cortex (vmPFC) supports compensation as defined by three a priori criteria: (1) increased vmPFC activation was observed in older versus younger adults; (2) age-related increases in vmPFC activity were associated with increased retrieval demands; and (3) increased vmPFC activity was positively associated with performance in older adults-evidence of successful compensation. Extending these results, we observed evidence for compensation in connectivity between vmPFC and the dorsolateral PFC during memory-dependent choice. In contrast, we found no evidence for age differences in value-related processing or age-related compensation for choices without delayed retrieval. Together, these results converge on the conclusion that age-related decline in memory-dependent choice performance can be minimized via functional compensation in vmPFC.

Distress from Motivational Dis-integration: When Fundamental Motives Are Too Weak or Too Strong

Past research has shown that satisfying different kinds of fundamental motives contributes to well-being. More recently, advances in motivational theory have shown that z is also tied to the integration of different motives. In other words, well-being depends not only on maximizing effectiveness in satisfying specific motives, but also on ensuring that motives work together such that no individual motive is too weak or too strong. In this chapter, we review existing research to show that specific forms of psychological distress can be linked to specific types of motivational imbalance or dis-integration. Such disintegration can arise from either excessive weakness of a specific motive or the excessive strength and/or dominance of a specific motive, thereby inhibiting other motives. Possible neural correlates and avenues of intervention are discussed.

Motivational modes and learning in Parkinson's disease

Learning and motivation are intrinsically related, and both have been linked to dopamine. Parkinson's disease results from a progressive loss of dopaminergic inputs to the striatum and leads to impairments in motivation and learning from feedback. However, the link between motivation and learning in Parkinson's disease is not well understood. To address this gap, we leverage a well-established psychological theory of motivation, regulatory mode theory, which distinguishes between two functionally independent motivational concerns in regulating behavior: a concern with having an effect by initiating and maintaining movement (Locomotion) and a concern with establishing what is correct by critically evaluating goal pursuit means and outcomes (Assessment). We examined Locomotion and Assessment in patients with Parkinson's disease and age-matched controls. Parkinson's disease patients demonstrated a selective decrease in Assessment motivation but no change in Locomotion motivation, suggesting that Parkinson's disease leads to a reduced tendency to evaluate and monitor outcomes. Moreover, weaker Assessment motivation was correlated with poorer performance on a feedback-based learning task previously shown to depend on the striatum. Together, these findings link a questionnaire-based personality inventory with performance on a well-characterized experimental task, advancing our understanding of how Parkinson's disease affects motivation with implications for well-being and treatment outcomes.

Functionally dissociable influences on learning rate in a dynamic environment

Maintaining accurate beliefs in a changing environment requires dynamically adapting the rate at which one learns from new experiences. Beliefs should be stable in the face of noisy data but malleable in periods of change or uncertainty. Here we used computational modeling, psychophysics, and fMRI to show that adaptive learning is not a unitary phenomenon in the brain. Rather, it can be decomposed into three computationally and neuroanatomically distinct factors that were evident in human subjects performing a spatial-prediction task: (1) surprise-driven belief updating, related to BOLD activity in visual cortex; (2) uncertainty-driven belief updating, related to anterior prefrontal and parietal activity; and (3) reward-driven belief updating, a context-inappropriate behavioral tendency related to activity in ventral striatum. These distinct factors converged in a core system governing adaptive learning. This system, which included dorsomedial frontal cortex, responded to all three factors and predicted belief updating both across trials and across individuals.

How attentional boost interacts with reward: the effect of dopaminergic medications in Parkinson's disease

There is widespread evidence that dopamine is implicated in the regulation of reward and salience. However, it is less known how these processes interact with attention and recognition memory. To explore this question, we used the attentional boost test in patients with Parkinson's disease (PD) before and after the administration of dopaminergic medications. Participants performed a visual letter detection task (remembering rewarded target letters and ignoring distractor letters) while also viewing a series of photos of natural and urban scenes in the background of the letters. The aim of the game was to retrieve the target letter after each trial and to win as much virtual money as possible. The recognition of background scenes was not rewarded. We enrolled 26 drug-naïve, newly diagnosed patients with PD and 25 healthy controls who were evaluated at baseline and follow-up. Patients with PD received dopamine agonists (pramipexole, ropinirole, rotigotine) during the 12-week follow-up period. At baseline, we found intact attentional boost in patients with PD: they were able to recognize target-associated scenes similarly to controls. At follow-up, patients with PD outperformed controls for both target- and distractor-associated scenes, but not when scenes were presented without letters. The alerting, orienting and executive components of attention were intact in PD. Enhanced attentional boost was replicated in a smaller group of patients with PD (n = 15) receiving l-3,4-dihydroxyphenylalanine (L-DOPA). These results suggest that dopaminergic medications facilitate attentional boost for background information regardless of whether the central task (letter detection) is rewarded or not.

A role for the medial temporal lobe in feedback-driven learning: evidence from amnesia

The ability to learn from feedback is a key component of adaptive behavior. This type of learning is traditionally thought to depend on neural substrates in the striatum and not on the medial temporal lobe (MTL). Here we show that in humans the MTL becomes necessary for feedback-based learning when feedback is delayed. Specifically, amnesic patients with MTL damage were impaired at probabilistic learning of cue-outcome associations when response-contingent feedback was delayed by a few seconds, but not when feedback was immediate. By contrast, patients with striatal dysfunction due to Parkinson's disease demonstrated the opposite pattern: impaired learning when trial-by-trial feedback was immediate but not when feedback was delayed, indicating that the striatum is necessary for learning only when feedback is immediate. Together, these results reveal that multiple complementary learning processes support what appears to be identical behavior in healthy individuals and point to an important role for the MTL in feedback-driven learning.