Preservation of function in Parkinson's disease: what's learning got to do with it?

(A)symmetry during gait initiation in people with Parkinson's disease: A motor and cortical activity exploratory study

Background

Gait asymmetry and deficits in gait initiation (GI) are among the most disabling symptoms in people with Parkinson's disease (PwPD). Understanding if PwPD with reduced asymmetry during GI have higher asymmetry in cortical activity may provide support for an adaptive mechanism to improve GI, particularly in the presence of an obstacle.

Objective

This study quantified the asymmetry of anticipatory postural adjustments (APAs), stepping parameters and cortical activity during GI, and tested if the presence of an obstacle regulates asymmetry in PwPD.

Methods

Sixteen PwPD and 16 control group (CG) performed 20-trials in two conditions: unobstructed and obstructed GI with right and left limbs. We measured, through symmetry index, (i) motor parameters: APAs and stepping, and (ii) cortical activity: the PSD of the frontal, sensorimotor and occipital areas during APA, STEP-I (moment of heel-off of the leading foot in the GI until the heel contact of the same foot); and STEP-II (moment of the heel-off of the trailing foot in the GI until the heel contact of the same foot) phases.

Results

Parkinson's disease showed higher asymmetry in cortical activity during APA, STEP-I and STEP-II phases and step velocity (STEP-II phase) during unobstructed GI than CG. However, unexpectedly, PwPD reduced the level of asymmetry of anterior-posterior displacement (p < 0.01) and medial-lateral velocity (p < 0.05) of the APAs. Also, when an obstacle was in place, PwPD showed higher APAs asymmetry (medial-lateral velocity: p < 0.002), with reduced and increased asymmetry of the cortical activity during APA and STEP-I phases, respectively.

Conclusion

Parkinson's disease were not motor asymmetric during GI, indicating that higher cortical activity asymmetry can be interpreted as an adaptive behavior to reduce motor asymmetry. In addition, the presence of obstacle did not regulate motor asymmetry during GI in PwPD.

Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study

Cognitive flexibility is essential to modify our behavior in a non-stationary environment and is often explored by reversal learning tasks. The basal ganglia (BG) dopaminergic system, under a top-down control of the pre-frontal cortex, is known to be involved in flexible action selection through reinforcement learning. However, how adaptive dopamine changes regulate this process and learning mechanisms for training the striatal synapses remain open questions. The current study uses a neurocomputational model of the BG, based on dopamine-dependent direct (Go) and indirect (NoGo) pathways, to investigate reinforcement learning in a probabilistic environment through a task that associates different stimuli to different actions. Here, we investigated: the efficacy of several versions of the Hebb rule, based on covariance between pre- and post-synaptic neurons, as well as the required control in phasic dopamine changes crucial to achieving a proper reversal learning. Furthermore, an original mechanism for modulating the phasic dopamine changes is proposed, assuming that the expected reward probability is coded by the activity of the winner Go neuron before a reward/punishment takes place. Simulations show that this original formulation for an automatic phasic dopamine control allows the achievement of a good flexible reversal even in difficult conditions. The current outcomes may contribute to understanding the mechanisms for active control of dopamine changes during flexible behavior. In perspective, it may be applied in neuropsychiatric or neurological disorders, such as Parkinson’s or schizophrenia, in which reinforcement learning is impaired.

Acute exercise following skill practice promotes motor memory consolidation in Parkinson's disease

Acute cardiovascular exercise has shown to promote neuroplastic processes supporting the consolidation of newly acquired motor skills in healthy adults. First results suggest that this concept may be transferred to populations with motor and cognitive dysfunctions. In this context, Parkinson's disease (PD) is highly relevant since patients demonstrate deficits in motor learning. Hence, in the present study we sought to explore the effect of a single post-practice exercise bout on motor memory consolidation in PD. For this purpose, 17 patients with PD (Hoehn and Yahr: 1 - 2.5, age: 60.1 ± 7.9 y) practiced a whole-body skill followed by either (i) a moderate-intense bout of cycling, or (ii) seated rest for a total of 30 min. The motor skill required the participants to balance on a tiltable platform (stabilometer) for 30 s. During skill practice, participants performed 15 trials followed by a retention test 1 day and 7 days later. We calculated time in balance (platform within ± 5° from horizontal) for each trial and within- and between-group differences in memory consolidation (i.e. offline learning = skill change from last acquisition block to retention tests) were analyzed. Groups revealed similar improvements during skill practice (F_4,60 = 0.316, p = 0.866), but showed differences in offline learning, which were only evident after 7 days (F_1,14 = 5.602, p = 0.033). Our results suggest that a single post-practice exercise bout is effective in enhancing long-term motor memory consolidation in a population with motor learning impairments. This may point at unique promoting effects of exercise on dopamine neurotransmission involved in memory formation. Future studies should investigate the potential role of exercise-induced effects on the dopaminergic system.

A non-linear deterministic model of action selection in the basal ganglia to simulate motor fluctuations in Parkinson's disease

Motor fluctuations and dyskinesias are severe complications of Parkinson's disease (PD), especially evident at its advanced stage, under long-term levodopa therapy. Despite their strong clinical prevalence, the neural origin of these motor symptoms is still a subject of intense debate. In this work, a non-linear deterministic neurocomputational model of the basal ganglia (BG), inspired by biology, is used to provide more insights into possible neural mechanisms at the basis of motor complications in PD. In particular, the model is used to simulate the finger tapping task. The model describes the main neural pathways involved in the BG to select actions [the direct or Go, the indirect or NoGo, and the hyperdirect pathways via the action of the sub-thalamic nucleus (STN)]. A sensitivity analysis is performed on some crucial model parameters (the dopamine level, the strength of the STN mechanism, and the strength of competition among different actions in the motor cortex) at different levels of synapses, reflecting major or minor motor training. Depending on model parameters, results show that the model can reproduce a variety of clinically relevant motor patterns, including normokinesia, bradykinesia, several attempts before movement, freezing, repetition, and also irregular fluctuations. Motor symptoms are, especially, evident at low or high dopamine levels, with excessive strength of the STN and with weak competition among alternative actions. Moreover, these symptoms worsen if the synapses are subject to insufficient learning. The model may help improve the comprehension of motor complications in PD and, ultimately, may contribute to the treatment design.

Mathematical modeling and parameter estimation of levodopa motor response in patients with parkinson disease

Parkinson disease (PD) is characterized by a clear beneficial motor response to levodopa (LD) treatment. However, with disease progression and longer LD exposure, drug-related motor fluctuations usually occur. Recognition of the individual relationship between LD concentration and its effect may be difficult, due to the complexity and variability of the mechanisms involved. This work proposes an innovative procedure for the automatic estimation of LD pharmacokinetics and pharmacodynamics parameters, by a biologically-inspired mathematical model. An original issue, compared with previous similar studies, is that the model comprises not only a compartmental description of LD pharmacokinetics in plasma and its effect on the striatal neurons, but also a neurocomputational model of basal ganglia action selection. Parameter estimation was achieved on 26 patients (13 with stable and 13 with fluctuating LD response) to mimic plasma LD concentration and alternate finger tapping frequency along four hours after LD administration, automatically minimizing a cost function of the difference between simulated and clinical data points. Results show that individual data can be satisfactorily simulated in all patients and that significant differences exist in the estimated parameters between the two groups. Specifically, the drug removal rate from the effect compartment, and the Hill coefficient of the concentration-effect relationship were significantly higher in the fluctuating than in the stable group. The model, with individualized parameters, may be used to reach a deeper comprehension of the PD mechanisms, mimic the effect of medication, and, based on the predicted neural responses, plan the correct management and design innovative therapeutic procedures.

A Single Bout of Aerobic Exercise Improves Motor Skill Consolidation in Parkinson's Disease

Background: Motor learning is impaired in Parkinson’s disease (PD), with patients demonstrating deficits in skill acquisition (online learning) and consolidation (offline learning) compared to healthy adults of similar age. Recent studies in young adults suggest that single bouts of aerobic exercise (AEX), performed in close temporal proximity to practicing a new motor task, may facilitate motor skill learning. Thus, we aimed at investigating the effects of a single bout of aerobic cycling on online and offline learning in PD patients.

Methods: 17 PD patients (Hoehn and Yahr 1 – 2.5, age: 64.4 ± 6.2) participated in this crossover study. Immediately prior to practicing a novel balance task, patients either performed 30 min of (i) moderate intensity (60–70% VO_2max) aerobic cycling, or (ii) seated rest (order counterbalanced). The task required patients to stabilize a balance platform (stabilometer) in a horizontal position for 30 s. For each experimental condition, patients performed 15 acquisition trials, followed by a retention test 24 h later. We calculated time in balance (platform within ± 5° from horizontal) for each trial, and analyzed within- and between-subjects differences in skill acquisition (online learning) and skill retention (offline learning) using mixed repeated-measures ANOVA.

Results: We found that the exercise bout had no effect on performance level or online gains during acquisition, despite affecting the time course of skill improvements (larger initial and reduced late skill gains). Aerobic cycling significantly improved offline learning, as reflected by larger 24-h skill retention compared to the rest condition.

Conclusion: Our results suggest that a single bout of moderate-intensity AEX is effective in improving motor skill consolidation in PD patients. Thus, acute exercise may represent an effective strategy to enhance motor memory formation in this population. More work is necessary to understand the underlying mechanisms, the optimal scheduling of exercise, and the applicability to other motor tasks. Further, the potential for patients in later disease stages need to be investigated. The study was a priori registered at ClinicalTrials.gov (NCT03245216).

Exercise Ameliorates Motor Deficits and Improves Dopaminergic Functions in the Rat Hemi-Parkinson's Model

To determine the influences of exercise on motor deficits and dopaminergic transmission in a hemiparkinson animal model, we measured the effects of exercise on the ambulatory system by estimating spatio-temporal parameters during walking, striatal dopamine (DA) release and reuptake and synaptic plasticity in the corticostriatal pathway after unilateral 6-OHDA lesions. 6-OHDA lesioned hemiparkinsonian rats were exercised on a fixed speed treadmill for 30 minutes per day. Controls received the same lesion but no exercise. Animals were subsequently analyzed for behavior including gait analysis, rotarod performance and apomorphine induced rotation. Subsequently, in vitro striatal dopamine release was analyzed by using FSCV and activity-dependent plasticity in the corticostriatal pathway was measured in each group. Our data indicated that exercise could improve motor walking speed and increase the apomorphine-induced rotation threshold. Exercise also ameliorated spatiotemporal impairments in gait in PD animals. Exercise increased the parameters of synaptic plasticity formation in the corticostriatal pathway of PD animals as well as the dynamics of dopamine transmission in PD animals. Fixed speed treadmill training 30 minutes per day could ameliorate spatial-temporal gait impairment, improve walking speed, dopamine transmission as well as corticostriatal synaptic plasticity in the unilateral 6-OHDA lesioned rat model.

To Do or Not to Do: Dopamine, Affordability and the Economics of Opportunity

Five years ago, we introduced the thrift hypothesis of dopamine (DA), suggesting that the primary role of DA in adaptive behavior is regulating behavioral energy expenditure to match the prevailing economic conditions of the environment. Here we elaborate that hypothesis with several new ideas. First, we introduce the concept of affordability, suggesting that costs must necessarily be evaluated with respect to the availability of resources to the organism, which computes a value not only for the potential reward opportunity, but also the value of resources expended. Placing both costs and benefits within the context of the larger economy in which the animal is functioning requires consideration of the different timescales against which to compute resource availability, or average reward rate. Appropriate windows of computation for tracking resources requires corresponding neural substrates that operate on these different timescales. In discussing temporal patterns of DA signaling, we focus on a neglected form of DA plasticity and adaptation, changes in the physical substrate of the DA system itself, such as up- and down-regulation of receptors or release probability. We argue that changes in the DA substrate itself fundamentally alter its computational function, which we propose mediates adaptations to longer temporal horizons and economic conditions. In developing our hypothesis, we focus on DA D2 receptors (D2R), arguing that D2R implements a form of “cost control” in response to the environmental economy, serving as the “brain’s comptroller”. We propose that the balance between the direct and indirect pathway, regulated by relative expression of D1 and D2 DA receptors, implements affordability. Finally, as we review data, we discuss limitations in current approaches that impede fully investigating the proposed hypothesis and highlight alternative, more semi-naturalistic strategies more conducive to neuroeconomic investigations on the role of DA in adaptive behavior.

Postural motor learning in Parkinson's disease: The effect of practice on continuous compensatory postural regulation

INTRODUCTION

Although balance training is considered the most effective treatment for balance impairments in Parkinson's disease (PD), few studies have examined if learning for balance control remains intact with PD. This study aimed to determine if learning for automatic postural responses is preserved in people with PD.

METHODS

Eleven participants with moderate PD (68±6.4years; H&Y: 2-3) on their usual medication maintained balance on a platform that oscillated forward and backward with variable amplitude and constant frequency. Participants completed 42 trials during one training session, and retention and transfer tests following a 24-h delay. Performance was measured by comparing spatial and temporal measures of whole-body centre of mass (COM) with platform displacements. Learning was compared between participants with PD and previously reported, age-matched older adults (Van Ooteghem et al., 2010).

RESULTS

Although postural responses in participants with PD were impaired compared to control participants, a majority of PD participants improved their postural responses with practice as revealed by reduced COM displacements and improved phase relationships between COM and platform motion. Rates of improvement were comparable between groups demonstrating preserved adaptive capacity for participants with PD. Similar to control participants, the PD group moved toward anticipatory COM control as a strategy for improving stability, exhibited short-term retention of performance improvements, and demonstrated generalizability of the learned responses. Rate of improvement with practice, but not retention, was related to severity of motor impairments.

CONCLUSIONS

Patients with moderate PD on medication demonstrate retention of improvements in automatic postural responses with practice suggesting that intrinsic postural motor learning is preserved in this group.

Chronic Nicotine Mitigates Aberrant Inhibitory Motor Learning Induced by Motor Experience under Dopamine Deficiency

Although dopamine receptor antagonism has long been associated with impairments in motor performance, more recent studies have shown that dopamine D2 receptor (D2R) antagonism, paired with a motor task, not only impairs motor performance concomitant with the pharmacodynamics of the drug, but also impairs future motor performance once antagonism has been relieved. We have termed this phenomenon "aberrant motor learning" and have suggested that it may contribute to motor symptoms in movement disorders such as Parkinson's disease (PD). Here, we show that chronic nicotine (cNIC), but not acute nicotine, treatment mitigates the acquisition of D2R-antagonist-induced aberrant motor learning in mice. Although cNIC mitigates D2R-mediated aberrant motor learning, cNIC has no effect on D1R-mediated motor learning. β2-containing nicotinic receptors in dopamine neurons likely mediate the protective effect of cNIC against aberrant motor learning, because selective deletion of β2 nicotinic subunits in dopamine neurons reduced D2R-mediated aberrant motor learning. Finally, both cNIC treatment and β2 subunit deletion blunted postsynaptic responses to D2R antagonism. These results suggest that a chronic decrease in function or a downregulation of β2-containing nicotinic receptors protects the striatal network against aberrant plasticity and aberrant motor learning induced by motor experience under dopamine deficiency.

SIGNIFICANCE STATEMENT

Increasingly, aberrant plasticity and aberrant learning are recognized as contributing to the development and progression of movement disorders. Here, we show that chronic nicotine (cNIC) treatment or specific deletion of β2 nicotinic receptor subunits in dopamine neurons mitigates aberrant motor learning induced by dopamine D2 receptor (D2R) blockade in mice. Moreover, both manipulations also reduced striatal dopamine release and blunt postsynaptic responses to D2R antagonists. These results suggest that chronic downregulation of function and/or receptor expression of β2-containing nicotinic receptors alters presynaptic and postsynaptic striatal signaling to protect against aberrant motor learning. Moreover, these results suggest that cNIC treatment may alleviate motor symptoms and/or delay the deterioration of motor function in movement disorders by blocking aberrant motor learning.

Cyclic AMP and afferent activity govern bidirectional synaptic plasticity in striatopallidal neurons

Recent experimental evidence suggests that the low dopamine conditions in Parkinson's disease (PD) cause motor impairment through aberrant motor learning. Those data, along with computational models, suggest that this aberrant learning results from maladaptive corticostriatal plasticity and learned motor inhibition. Dopaminergic modulation of both corticostriatal long-term depression (LTD) and long-term potentiation (LTP) is proposed to be critical for these processes; however, the regulatory mechanisms underlying bidirectional corticostriatal plasticity are not fully understood. Previously, we demonstrated a key role for cAMP signaling in corticostriatal LTD. In this study, mouse brain slices were used to perform a parametric experiment that tested the impact of varying both intracellular cAMP levels and the strength of excitatory inputs on corticostriatal plasticity. Using slice electrophysiology in the dorsolateral striatum, we demonstrate that both LTP and LTD can be sequentially induced in the same D2-expressing neuron and that LTP was strongest with high intracellular cAMP and LFS, whereas LTD required low intracellular cAMP and high-frequency stimulation. Our results provide a molecular and cellular basis for regulating bidirectional corticostriatal synaptic plasticity and may help to identify novel therapeutic targets for blocking or reversing the aberrant synaptic plasticity that likely contributes to motor deficits in PD.

Dissociable effects of dopamine on learning and performance within sensorimotor striatum

Striatal dopamine is an important modulator of current behavior, as seen in the rapid and dramatic effects of dopamine replacement therapy in Parkinson Disease (PD). Yet there is also extensive evidence that dopamine acts as a learning signal, modulating synaptic plasticity within striatum to affect future behavior. Disentangling these "performance" and "learning" functions is important for designing effective, long-term PD treatments. We conducted a series of unilateral drug manipulations and dopamine terminal lesions in the dorsolateral striatum of rats highly-trained to perform brief instructed head/neck movements (two-alternative forced choice task). Reaction times and accuracy were measured longitudinally to determine if task behavior changed immediately, progressed over time, and/or persisted after drug withdrawal. Enhanced dopamine signaling with amphetamine caused an immediate, nonprogressive, and bilateral decrease in reaction times (RT). The altered RT distributions were consistent with reduced distance to threshold in the linear approach to threshold with ergodic rate (LATER) model of decision-making. Conversely, the dopamine antagonist flupenthixol caused experience-dependent, persistent changes in RT and accuracy indicative of a "learning" effect. These RT distributions were consistent with a slowed rate of approach to decision threshold. Our results show that dopaminergic signaling makes dissociable contributions to current and future behavior even within a single striatal subregion, and provide important clues for both models of normal decision-making and the design of novel drug therapies in PD.

Performance of a motor task learned on levodopa deteriorates when subsequently practiced off

Studies in animals and in people with Parkinson's disease (PD) demonstrate complex effects of dopamine on learning motor tasks; its effect on retention of motor learning has received little attention. Recent animal studies demonstrate that practicing a task in the off state, when initially learned in the on state, leads to progressive deterioration in performance. We measured the acquisition and retention of 3 different motor tasks in the presence and absence of levodopa. Twenty individuals with Hoehn and Yahr Stage 1.5 to 3 PD practiced the tasks daily for two 4-day weeks, one half practicing on L-dopa the first week and off the second week. The other half practiced off l-dopa both weeks. The tasks were (1) alternate tapping of 2 keys, (2) moving the body toward 2 targets on a posturography device, and (3) mirror drawing of a star. For the tapping and body movement tests, those who practiced on the first week had a progressive decline in performance with practice during week 2, while subjects off during week 1 maintained or improved. In contrast, for the mirror task, subjects on L-dopa initially had much more difficulty completing the task compared to subjects who practiced off. Both groups improved with practice the first week and had flat performance the second week. These data suggest that performance of speed-accuracy tasks learned in the on state may progressively worsen if subsequently practiced in the off state. In addition, performance, but not learning, of some tasks may be impeded by L-dopa.

Nicotinic receptors regulate the dynamic range of dopamine release in vivo

Nicotinic acetylcholine receptors (nAChRs) are expressed presynaptically on dopamine axon terminals, and their activation by endogenous acetylcholine from striatal cholinergic interneurons enhances dopamine release both independently of and in concert with dopamine neuron activity. Acute nAChR inactivation is believed to enhance the contrast between low- and high-frequency dopamine cell activity. Although these studies reveal a key role for acute activation and inactivation of nAChRs in striatal microcircuitry, it remains unknown if chronic inactivation/desensitization of nAChRs can alter dopamine release dynamics. Using in vivo cyclic voltammetry in anaesthetized mice, we examined whether chronic inactivation of nAChRs modulates dopamine release across a parametric range of stimulation, varying both frequency and pulse number. Deletion of β2*nAChRs and chronic nicotine exposure greatly diminished dopamine release across the entire range of stimulation parameters. In addition, we observed a facilitation of dopamine release at low frequency and pulse number in wild-type mice that is absent in the β2* knockout and chronic nicotine mice. These data suggest that deletion or chronic desensitization of nAChRs reduces the dynamic range of dopamine release in response to dopamine cell activity, decreasing rather than increasing contrast between high and low dopamine activity.

The Enemy within: Propagation of Aberrant Corticostriatal Learning to Cortical Function in Parkinson's Disease

Motor dysfunction in Parkinson’s disease is believed to arise primarily from pathophysiology in the dorsal striatum and its related corticostriatal and thalamostriatal circuits during progressive dopamine denervation. One function of these circuits is to provide a filter that selectively facilitates or inhibits cortical activity to optimize cortical processing, making motor responses rapid and efficient. Corticostriatal synaptic plasticity mediates the learning that underlies this performance-optimizing filter. Under dopamine denervation, corticostriatal plasticity is altered, resulting in aberrant learning that induces inappropriate basal ganglia filtering that impedes rather than optimizes cortical processing. Human imaging suggests that increased cortical activity may compensate for striatal dysfunction in PD patients. In this Perspective article, we consider how aberrant learning at corticostriatal synapses may impair cortical processing and learning and undermine potential cortical compensatory mechanisms. Blocking or remediating aberrant corticostriatal plasticity may protect cortical function and support cortical compensatory mechanisms mitigating the functional decline associated with progressive dopamine denervation.

Changes to saccade behaviors in Parkinson's disease following dancing and observation of dancing

Background: The traditional view of Parkinson’s disease (PD) as a motor disorder only treated by dopaminergic medications is now shifting to include non-pharmacologic interventions. We have noticed that patients with PD obtain an immediate, short-lasting benefit to mobility by the end of a dance class, suggesting some mechanism by which dancing reduces bradykinetic symptoms. We have also found that patients with PD are unimpaired at initiating highly automatic eye movements to visual stimuli (pro-saccades) but are impaired at generating willful eye movements away from visual stimuli (anti-saccades). We hypothesized that the mechanisms by which a dance class improves movement initiation may generalize to the brain networks impacted in PD (frontal lobe and basal ganglia, BG), and thus could be assessed objectively by measuring eye movements, which rely on the same neural circuitry.

Methods: Participants with PD performed pro- and anti-saccades before, and after, a dance class. “Before” and “after” saccade performance measurements were compared. These measurements were then contrasted with a control condition (observing a dance class in a video), and with older and younger adult populations, who rested for an hour between measurements.

Results: We found an improvement in anti-saccade performance following the observation of dance (but not following dancing), but we found a detriment in pro-saccade performance following dancing.

Conclusion: We suggest that observation of dance induced plasticity changes in frontal-BG networks that are important for executive control. Dancing, in contrast, increased voluntary movement signals that benefited mobility, but interfered with the automaticity of efficient pro-saccade execution.

A role for dopamine-mediated learning in the pathophysiology and treatment of Parkinson's disease

Dopamine contributes to corticostriatal plasticity and motor learning. Dopamine denervation profoundly alters motor performance, as in Parkinson's disease (PD); however, the extent to which these symptoms reflect impaired motor learning is unknown. Here, we demonstrate a D2 receptor blockade-induced aberrant learning that impedes future motor performance when dopamine signaling is restored, an effect diminished by coadministration of adenosine antagonists during blockade. We hypothesize that an inappropriate corticostriatal potentiation in striatopallidal cells of the indirect pathway underlies aberrant learning. We demonstrate synaptic potentiation in striatopallidal neurons induced by D2 blockade and diminished by application of an adenosine antagonist, consistent with behavioral observations. A neurocomputational model of the basal ganglia recapitulates the behavioral pattern and further links aberrant learning to plasticity in the indirect pathway. Thus, D2-mediated aberrant learning may contribute to motor deficits in PD, suggesting new avenues for the development of therapeutics.

Thorndike's Law 2.0: Dopamine and the Regulation of Thrift

Dopamine is widely associated with reward, motivation, and reinforcement learning. Research on dopamine has emphasized its contribution to compulsive behaviors, such as addiction and overeating, with less examination of its potential role in behavioral flexibility in normal, non-pathological states. In the study reviewed here, we investigated the effect of increased tonic dopamine in a two-lever homecage operant paradigm where the relative value of the levers was dynamic, requiring the mice to constantly monitor reward outcome and adapt their behavior. The data were fit to a temporal difference learning model that showed that mice with elevated dopamine exhibited less coupling between reward history and behavioral choice. This work suggests a way to integrate motivational and learning theories of dopamine into a single formal model where tonic dopamine regulates the expression of prior reward learning by controlling the degree to which learned reward values bias behavioral choice. Here I place these results in a broader context of dopamine's role in instrumental learning and suggest a novel hypothesis that tonic dopamine regulates thrift, the degree to which an animal needs to exploit its prior reward learning to maximize return on energy expenditure. Our data suggest that increased dopamine decreases thriftiness, facilitating energy expenditure, and permitting greater exploration. Conversely, this implies that decreased dopamine increases thriftiness, favoring the exploitation of prior reward learning, and diminishing exploration. This perspective provides a different window onto the role dopamine may play in behavioral flexibility and its failure, compulsive behavior.

Putting desire on a budget: dopamine and energy expenditure, reconciling reward and resources

Accumulating evidence indicates integration of dopamine function with metabolic signals, highlighting a potential role for dopamine in energy balance, frequently construed as modulating reward in response to homeostatic state. Though its precise role remains controversial, the reward perspective of dopamine has dominated investigation of motivational disorders, including obesity. In the hypothesis outlined here, we suggest instead that the primary role of dopamine in behavior is to modulate activity to adapt behavioral energy expenditure to the prevailing environmental energy conditions, with the role of dopamine in reward and motivated behaviors derived from its primary role in energy balance. Dopamine has long been known to modulate activity, exemplified by psychostimulants that act via dopamine. More recently, there has been nascent investigation into the role of dopamine in modulating voluntary activity, with some investigators suggesting that dopamine may serve as a final common pathway that couples energy sensing to regulated voluntary energy expenditure. We suggest that interposed between input from both the internal and external world, dopamine modulates behavioral energy expenditure along two axes: a conserve-expend axis that regulates generalized activity and an explore-exploit axes that regulates the degree to which reward value biases the distribution of activity. In this view, increased dopamine does not promote consumption of tasty food. Instead increased dopamine promotes energy expenditure and exploration while decreased dopamine favors energy conservation and exploitation. This hypothesis provides a mechanistic interpretation to an apparent paradox: the well-established role of dopamine in food seeking and the findings that low dopaminergic functions are associated with obesity. Our hypothesis provides an alternative perspective on the role of dopamine in obesity and reinterprets the "reward deficiency hypothesis" as a perceived energy deficit. We propose that dopamine, by facilitating energy expenditure, should be protective against obesity. We suggest the apparent failure of this protective mechanism in Western societies with high prevalence of obesity arises as a consequence of sedentary lifestyles that thwart energy expenditure.