Distinct dopaminergic control of the direct and indirect pathways in reward-based and avoidance learning behaviors

Loss of glycine receptors in the nucleus accumbens and ethanol reward in an Alzheimer´s Disease mouse model

Alterations in cognitive and non-cognitive cerebral functions characterize Alzheimer's disease (AD). Cortical and hippocampal impairments related to extracellular accumulation of Aβ in AD animal models have been extensively investigated. However, recent reports have also implicated intracellular Aβ in limbic regions, such as the nucleus accumbens (nAc). Accumbal neurons express high levels of inhibitory glycine receptors (GlyRs) that are allosterically modulated by ethanol and have a role in controlling its intake. In the present study, we investigated how GlyRs in the 2xTg mice (AD model) affect nAc functions and ethanol intake behavior. Using transgenic and control aged-matched litter mates, we found that the GlyRα2 subunit was significantly decreased in AD mice (6-month-old). We also examined intracellular calcium dynamics using the fluorescent calcium protein reporter GCaMP in slice photometry. We also found that the calcium signal mediated by GlyRs, but not GABAAR, was also reduced in AD neurons. Additionally, ethanol potentiation was significantly decreased in accumbal neurons in the AD mice. Finally, we performed drinking in the dark (DID) experiments and found that 2xTg mice consumed less ethanol on the last day of DID, in agreement with a lower blood ethanol concentration. 2xTg mice also showed lower sucrose consumption, indicating that overall food reward was altered. In conclusion, the data support the role of GlyRs in nAc neuron excitability and a decreased glycinergic activity in the 2xTg mice that might lead to impairment in reward processing at an early stage of the disease.

The cortico-striatal circuitry in autism-spectrum disorders: a balancing act

The basal ganglia are major targets of cortical inputs and, in turn, modulate cortical function via their projections to the motor and prefrontal cortices. The role of the basal ganglia in motor control and reward is well documented and there is also extensive evidence that they play a key role in social and repetitive behaviors. The basal ganglia influence the activity of the cerebral cortex via two major projections from the striatum to the output nuclei, the globus pallidus internus and the substantia nigra, pars reticulata. This modulation involves a direct projection known as the direct pathway and an indirect projection via the globus pallidus externus and the subthalamic nucleus, known as the indirect pathway. This review discusses the respective contribution of the direct and indirect pathways to social and repetitive behaviors in neurotypical conditions and in autism spectrum disorders.

α4-Containing GABAA Receptors on DRD2 Neurons of the Nucleus Accumbens Mediate Instrumental Responding for Conditioned Reinforcers and Its Potentiation by Cocaine

Extrasynaptic GABAA receptors (GABAARs) composed of α4, β, and δ subunits mediate GABAergic tonic inhibition and are potential molecular targets in the modulation of behavioral responses to natural and drug rewards. These GABAARs are highly expressed within the nucleus accumbens (NAc), where they influence the excitability of the medium spiny neurons. Here, we explore their role in modulating behavioral responses to food-conditioned cues and the behavior-potentiating effects of cocaine. α4-Subunit constitutive knock-out mice (α4-/-) showed higher rates of instrumental responding for reward-paired stimuli in a test of conditioned reinforcement (CRf). A similar effect was seen following viral knockdown of GABAAR α4 subunits within the NAc. Local infusion of the α4βδ-GABAAR-preferring agonist THIP (4,5,6,7-tetrahydroisoxazolo[5,4-c]pyridin-3-ol; Gaboxadol) into the NAc had no effect on responding when given alone but reduced cocaine potentiation of responding for conditioned reinforcers in wild-type, but not α4-/- mice. Finally, specific deletion of α4-subunits from dopamine D2, but not D1, receptor-expressing neurons (DRD2 and DRD1 neurons), mimicked the phenotype of the constitutive knockout, potentiating CRf responding, and blocking intra-accumbal THIP attenuation of cocaine-potentiated CRf responding. These data demonstrate that α4-GABAAR-mediated inhibition of DRD2 neurons reduces instrumental responding for a conditioned reinforcer and its potentiation by cocaine and emphasize the importance of GABAergic signaling within the NAc in mediating the effects of cocaine.

Overexpression of wild type glycine alpha 1 subunit rescues ethanol sensitivity in accumbal receptors and reduces binge drinking in mice

The nucleus accumbens (nAc) is a critical region in the brain reward system since it integrates abundant synaptic inputs contributing to the control of neuronal excitability in the circuit. The presence of inhibitory α1 glycine receptor (GlyRs) subunits, sensitive to ethanol, has been recently reported in accumbal neurons suggesting that they are protective against excessive binge consumption. In the present study, we used viral vectors (AAV) to overexpress mutant and WT α1 subunits in accumbal neurons in D1 Cre and α1 KI mice. Injection of a Cre-inducible AAV carrying an ethanol insensitive α1 subunit in D1 Cre neurons was unable to affect sensitivity to ethanol in GlyRs or affect ethanol drinking. On the other hand, using an AAV that transduced WT α1 GlyRs in GABAergic neurons in the nAc of high-ethanol consuming mice caused a reduction in ethanol intake as reflected by lowered drinking in the dark and reduced blood ethanol concentration. As expected, the AAV increased the glycine current density by 5-fold without changing the expression of GABAA receptors. Examination of the ethanol sensitivity in isolated accumbal neurons indicated that the GlyRs phenotype changed from an ethanol resistant to an ethanol sensitive type. These results support the conclusion that increased inhibition in the nAc can control excessive ethanol consumption and that selective targeting of GlyRs by pharmacotherapy might provide a mechanistic procedure to reduce ethanol binge.

Neuroproteomic mapping of kinases and their substrates downstream of acetylcholine: finding and implications

INTRODUCTION

Since the emergence of the cholinergic hypothesis of Alzheimer's disease (AD), acetylcholine has been viewed as a mediator of learning and memory. Donepezil improves AD-associated learning deficits and memory loss by recovering brain acetylcholine levels. However, it is associated with side effects due to global activation of acetylcholine receptors. Muscarinic acetylcholine receptor M1 (M1R), a key mediator of learning and memory, has been an alternative target. The importance of targeting a specific pathway downstream of M1R has recently been recognized. Elucidating signaling pathways beyond M1R that lead to learning and memory holds important clues for AD therapeutic strategies.

AREAS COVERED

This review first summarizes the role of acetylcholine in aversive learning, one of the outputs used for preliminary AD drug screening. It then describes the phosphoproteomic approach focused on identifying acetylcholine intracellular signaling pathways leading to aversive learning. Finally, the intracellular mechanism of donepezil and its effect on learning and memory is discussed.

EXPERT OPINION

The elucidation of signaling pathways beyond M1R by phosphoproteomic approach offers a platform for understanding the intracellular mechanism of AD drugs and for developing AD therapeutic strategies. Clarifying the molecular mechanism that links the identified acetylcholine signaling to AD pathophysiology will advance the development of AD therapeutic strategies.

Striatal adenosine A2A receptor involvement in normal and large nest building deer mice: perspectives on compulsivity and anxiety

Obsessive-compulsive disorder (OCD) is characterized by recurring obsessive thoughts and repetitive behaviors that are often associated with anxiety and perturbations in cortico-striatal signaling. Given the suboptimal response of OCD to current serotonergic interventions, there is a need to better understand the psychobiological mechanisms that may underlie the disorder. In this regard, investigations into adenosinergic processes might be fruitful. Indeed, adenosine modulates both anxiety- and motor behavioral output. Thus, we aimed to explore the potential associations between compulsive-like large nest building (LNB) behavior in deer mice, anxiety and adenosinergic processes. From an initial pool of 120 adult deer mice, 34 normal nest building (NNB)- and 32 LNB-expressing mice of both sexes were selected and exposed to either a normal water (wCTRL) or vehicle control (vCTRL), lorazepam (LOR) or istradefylline (ISTRA) for 7- (LOR) or 28 days after which nesting assessment was repeated and animals screened for anxiety-like behavior in an anxiogenic open field. Mice were then euthanized, the striatal tissue removed on ice and the adenosine A_2A receptor expression quantified. Our findings indicate that NNB and LNB behavior are not distinctly associated with measures of generalized anxiety and that ISTRA-induced changes in nesting expression are dissociated from changes in anxiety scores. Further, data from this investigation show that nesting in deer mice is directly related to striatal adenosine signaling, and that LNB is founded upon a lower degree of adenosinergic A_2A stimulation.

Error-related signaling in nucleus accumbens D2 receptor-expressing neurons guides inhibition-based choice behavior in mice

Learned associations between environmental cues and the outcomes they predict (cue-outcome associations) play a major role in behavioral control, guiding not only which responses we should perform, but also which we should inhibit, in order to achieve a specific goal. The encoding of such cue-outcome associations, as well as the performance of cue-guided choice behavior, is thought to involve dopamine D1 and D2 receptor-expressing medium spiny neurons (D1-/D2-MSNs) of the nucleus accumbens (NAc). Here, using a visual discrimination task in male mice, we assessed the role of NAc D1-/D2-MSNs in cue-guided inhibition of inappropriate responding. Cell-type specific neuronal silencing and in-vivo imaging revealed NAc D2-MSNs to contribute to inhibiting behavioral responses, with activation of NAc D2-MSNs following response errors playing an important role in optimizing future choice behavior. Our findings indicate that error-signaling by NAc D2-MSNs contributes to the ability to use environmental cues to inhibit inappropriate behavior.

Dopamine error signal to actively cope with lack of expected reward

To obtain more of a particular uncertain reward, animals must learn to actively overcome the lack of reward and adjust behavior to obtain it again. The neural mechanisms underlying such coping with reward omission remain unclear. Here, we developed a task in rats to monitor active behavioral switch toward the next reward after no reward. We found that some dopamine neurons in the ventral tegmental area exhibited increased responses to unexpected reward omission and decreased responses to unexpected reward, following the opposite responses of the well-known dopamine neurons that signal reward prediction error (RPE). The dopamine increase reflected in the nucleus accumbens correlated with behavioral adjustment to actively overcome unexpected no reward. We propose that these responses signal error to actively cope with lack of expected reward. The dopamine error signal thus cooperates with the RPE signal, enabling adaptive and robust pursuit of uncertain reward to ultimately obtain more reward.

Cellular bases for reward-related dopamine actions

Dopamine neurons exhibit transient increases and decreases in their firing rate upon reward and punishment for learning. This bidirectional modulation of dopamine dynamics occurs on the order of hundreds of milliseconds, and it is sensitively detected to reinforce the preceding sensorimotor events. These observations indicate that the mechanisms of dopamine detection at the projection sites are of remarkable precision, both in time and concentration. A major target of dopamine projection is the striatum, including the ventral region of the nucleus accumbens, which mainly comprises dopamine D1 and D2 receptor (D1R and D2R)-expressing spiny projection neurons. Although the involvement of D1R and D2R in dopamine-dependent learning has been suggested, the exact cellular bases for detecting transient dopamine signaling remain unclear. This review discusses recent cellular studies on the novel synaptic mechanisms for detecting dopamine transient signals associated with learning. Analyses of behavior based on these mechanisms have further revealed new behavioral aspects that are closely associated with these synaptic mechanisms. Thus, it is gradually possible to mechanistically explain behavioral learning via synaptic and cellular bases in rodents.

Aging in nucleus accumbens and its impact on alcohol use disorders

Ethanol is one of the most widely consumed drugs in the world and prolonged excessive ethanol intake might lead to alcohol use disorders (AUDs), which are characterized by neuroadaptations in different brain regions, such as in the reward circuitry. In addition, the global population is aging, and it appears that they are increasing their ethanol consumption. Although research involving the effects of alcohol in aging subjects is limited, differential effects have been described. For example, studies in human subjects show that older adults perform worse in tests assessing working memory, attention, and cognition as compared to younger adults. Interestingly, in the field of the neurobiological basis of ethanol actions, there is a significant dichotomy between what we know about the effects of ethanol on neurochemical targets in young animals and how it might affect them in the aging brain. To be able to understand the distinct effects of ethanol in the aging brain, the following questions need to be answered: (1) How does physiological aging impact the function of an ethanol-relevant region (e.g., the nucleus accumbens)? and (2) How does ethanol affect these neurobiological systems in the aged brain? This review discusses the available data to try to understand how aging affects the nucleus accumbens (nAc) and its neurochemical response to alcohol. The data show that there is little information on the effects of ethanol in aged mice and rats, and that many studies had considered 2-3-month-old mice as adults, which needs to be reconsidered since more recent literature defines 6 months as young adults and >18 months as an older mouse. Considering the actual relevance of an aged worldwide population and that this segment is drinking more frequently, it appears at least reasonable to explore how ethanol affects the brain in adult and aged models.

Importin α3 (KPNA3) Deficiency Augments Effortful Reward-Seeking Behavior in Mice

Importin α3 (Gene: Kpna3, the ortholog of human Importin α4) is a member of the importin α family and participates in nucleocytoplasmic transport by forming trimeric complexes between cargo proteins and importin β1. Evidence from human studies has indicated that single nucleotide polymorphisms (SNP) in the KPNA3 gene are associated with the occurrence of several psychiatric disorders accompanied by abnormal reward-related behavior, including schizophrenia, major depression, and substance addiction. However, the precise roles of importin α3 in controlling reward processing and motivation are still unclear. In this study, we evaluated the behavioral effects of Kpna3 knockout (KO) in mice on performance in touchscreen operant chamber-based tasks evaluating simple (fixed-ratio) and effortful (progressive-ratio) reward-seeking behaviors. While Kpna3 KO mice showed no significant differences in operant reward learning on a fixed-ratio schedule, they demonstrated significantly increased motivation (increased break point) to instrumentally respond for sucrose on a progressive-ratio schedule. We additionally measured the number of c-Fos-positive cells, a marker of neural activity, in 20 regions of the brain and identified a network of brain regions based on their interregional correlation coefficients. Network and graph-theoretic analyses suggested that Kpna3 deficiency enhanced overall interregional functional connectivity. These findings suggest the importance of Kpna3 in motivational control and indicate that Kpna3 KO mice may be an attractive line for modeling motivational abnormalities associated with several psychiatric disorders.

Computational Mechanisms of Osmoregulation: A Reinforcement Learning Model for Sodium Appetite

Homeostatic control with oral nutrient intake is a vital complex system involving the orderly interactions between the external and internal senses, behavioral control, reward learning, and decision-making. Sodium appetite is a representative system and has been intensively investigated in animal models of homeostatic systems and oral nutrient intake. However, the system-level mechanisms for regulating sodium intake behavior and homeostatic control remain unclear. In the current study, we attempted to provide a mechanistic understanding of sodium appetite behavior by using a computational model, the homeostatic reinforcement learning model, in which homeostatic behaviors are interpreted as reinforcement learning processes. Through simulation experiments, we confirmed that our homeostatic reinforcement learning model successfully reproduced homeostatic behaviors by regulating sodium appetite. These behaviors include the approach and avoidance behaviors to sodium according to the internal states of individuals. In addition, based on the assumption that the sense of taste is a predictor of changes in the internal state, the homeostatic reinforcement learning model successfully reproduced the previous paradoxical observations of the intragastric infusion test, which cannot be explained by the classical drive reduction theory. Moreover, we extended the homeostatic reinforcement learning model to multimodal data, and successfully reproduced the behavioral tests in which water and sodium appetite were mediated by each other. Finally, through an experimental simulation of chemical manipulation in a specific neural population in the brain stem, we proposed a testable hypothesis for the function of neural circuits involving sodium appetite behavior. The study results support the idea that osmoregulation via sodium appetitive behavior can be understood as a reinforcement learning process, and provide a mechanistic explanation for the underlying neural mechanisms of decision-making related to sodium appetite and homeostatic behavior.

Acute cannabinoids impair association learning via selectively enhancing synaptic transmission in striatonigral neurons

Background

Cannabinoids and their derivatives attract strong interest due to the tremendous potential of their psychoactive effects for treating psychiatric disorders and symptoms. However, their clinical application is restricted by various side-effects such as impaired coordination, anxiety, and learning and memory disability. Adverse impact on dorsal striatum-dependent learning is an important side-effect of cannabinoids. As one of the most important forms of learning mediated by the dorsal striatum, reinforcement learning is characterized by an initial association learning phase, followed by habit learning. While the effects of cannabinoids on habit learning have been well-studied, little is known about how cannabinoids influence the initial phase of reinforcement learning.

Results

We found that acute activation of cannabinoid receptor type 1 (CB1R) by the synthetic cannabinoid HU210 induced dose-dependent impairment of association learning, which could be alleviated by intra-dorsomedial striatum (DMS) injection of CB1R antagonist. Moreover, acute exposure to HU210 elicited enhanced synaptic transmission in striatonigral “direct” pathway medium spiny neurons (MSNs) but not indirect pathway neurons in DMS. Intriguingly, enhancement of synaptic transmission that is also observed after learning was abolished by HU210, indicating cannabinoid system might disrupt reinforcement learning by confounding synaptic plasticity normally required for learning. Remarkably, the impaired response-reinforcer learning was also induced by selectively enhancing the D1-MSN (MSN that selectively expresses the dopamine receptor type 1) activity by virally expressing excitatory hM3Dq DREADD (designer receptor exclusively activated by a designer drug), which could be rescued by specifically silencing the D1-MSN activity via hM4Di DREADD.

Conclusion

Our findings demonstrate dose-dependent deleterious effects of cannabinoids on association learning by disrupting plasticity change required for learning associated with the striatal direct pathway, which furthers our understanding of the side-effects of cannabinoids and the underlying mechanisms.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12915-022-01307-1.

Protracted abstinence in males with an opioid use disorder: partial recovery of nucleus accumbens function

Opioid use disorder (OUD) affects more than 27 million people globally accounting for more than 300,000 deaths annually. Protracted abstinence among individuals with OUD is rare due to a high relapse rate among those not receiving medications for OUD. Extensive preclinical studies form the basis of the allostasis theory, which proposes long-lasting functional brain abnormalities that persist after opioid withdrawal and contribute to relapse. Few studies have tested the allostasis theory in humans using neuroimaging. Here, we used fMRI and an instrumental learning task to test allostasis theory predictions (ATP) of functional abnormalities in both positive valence (PVS) and negative valence (NVS) accumbens systems in OUD patients with protracted abstinence (n = 15), comparing them with OUD patients receiving methadone treatment (MT) (n = 33), and with healthy controls (n = 23). As hypothesized, protracted abstinence OUD patients showed incomplete recovery of nucleus accumbens function, as evidenced by the blunted response to aversive events (NVS) during negative reinforcement, as observed in MT patients. In contrast, their accumbens response to rewarding events (PVS) during positive reinforcement was similar to that of controls and different from that in MT patients whose response was blunted. Protracted abstinence OUD patients also showed improvements in depression symptoms compared to MT patients. Residual depressive symptoms and pre-MT intravenous drug measures were associated with worse accumbens function in protracted abstinence. These results support the ATP of long-lasting dysfunction of NVS after withdrawal and show preliminary evidence of recovery of PVS function with protracted withdrawal. Therapeutic strategies that target NVS may facilitate recovery.

The Signaling and Pharmacology of the Dopamine D1 Receptor

The dopamine D1 receptor (D1R) is a Gα_s/olf-coupled GPCR that is expressed in the midbrain and forebrain, regulating motor behavior, reward, motivational states, and cognitive processes. Although the D1R was initially identified as a promising drug target almost 40 years ago, the development of clinically useful ligands has until recently been hampered by a lack of suitable candidate molecules. The emergence of new non-catechol D1R agonists, biased agonists, and allosteric modulators has renewed clinical interest in drugs targeting this receptor, specifically for the treatment of motor impairment in Parkinson's Disease, and cognitive impairment in neuropsychiatric disorders. To develop better therapeutics, advances in ligand chemistry must be matched by an expanded understanding of D1R signaling across cell populations in the brain, and in disease states. Depending on the brain region, the D1R couples primarily to either Gα_s or Gα_olf through which it activates a cAMP/PKA-dependent signaling cascade that can regulate neuronal excitability, stimulate gene expression, and facilitate synaptic plasticity. However, like many GPCRs, the D1R can signal through multiple downstream pathways, and specific signaling signatures may differ between cell types or be altered in disease. To guide development of improved D1R ligands, it is important to understand how signaling unfolds in specific target cells, and how this signaling affects circuit function and behavior. In this review, we provide a summary of D1R-directed signaling in various neuronal populations and describe how specific pathways have been linked to physiological and behavioral outcomes. In addition, we address the current state of D1R drug development, including the pharmacology of newly developed non-catecholamine ligands, and discuss the potential utility of D1R-agonists in Parkinson's Disease and cognitive impairment.

Phosphoproteomic of the acetylcholine pathway enables discovery of the PKC-β-PIX-Rac1-PAK cascade as a stimulatory signal for aversive learning

Acetylcholine is a neuromodulator critical for learning and memory. The cholinesterase inhibitor donepezil increases brain acetylcholine levels and improves Alzheimer's disease (AD)-associated learning disabilities. Acetylcholine activates striatal/nucleus accumbens dopamine receptor D2-expressing medium spiny neurons (D2R-MSNs), which regulate aversive learning through muscarinic receptor M1 (M1R). However, how acetylcholine stimulates learning beyond M1Rs remains unresolved. Here, we found that acetylcholine stimulated protein kinase C (PKC) in mouse striatal/nucleus accumbens. Our original kinase-oriented phosphoproteomic analysis revealed 116 PKC substrate candidates, including Rac1 activator β-PIX. Acetylcholine induced β-PIX phosphorylation and activation, thereby stimulating Rac1 effector p21-activated kinase (PAK). Aversive stimulus activated the M1R-PKC-PAK pathway in mouse D2R-MSNs. D2R-MSN-specific expression of PAK mutants by the Cre-Flex system regulated dendritic spine structural plasticity and aversive learning. Donepezil induced PAK activation in both accumbal D2R-MSNs and in the CA1 region of the hippocampus and enhanced D2R-MSN-mediated aversive learning. These findings demonstrate that acetylcholine stimulates M1R-PKC-β-PIX-Rac1-PAK signaling in D2R-MSNs for aversive learning and imply the cascade's therapeutic potential for AD as aversive learning is used to preliminarily screen AD drugs.

Impaired Learning From Negative Feedback in Stimulant Use Disorder: Dopaminergic Modulation

BACKGROUND

Drug-induced alterations to the dopamine system in stimulant use disorder (SUD) are hypothesized to impair reinforcement learning (RL). Computational modeling enables the investigation of the latent processes of RL in SUD patients, which could elucidate the nature of their impairments.

METHODS

We investigated RL in 44 SUD patients and 41 healthy control participants using a probabilistic RL task that assesses learning from reward and punishment separately. In an independent sample, we determined the modulatory role of dopamine in RL following a single dose of the dopamine D2/3 receptor antagonist amisulpride (400 mg) and the agonist pramipexole (0.5 mg) in a randomised, double-blind, placebo-controlled, crossover design. We analyzed task performance using computational modelling and hypothesized that RL impairments in SUD patients would be differentially modulated by a dopamine D2/3 receptor antagonist and agonist.

RESULTS

Computational analyses in both samples revealed significantly reduced learning rates from punishment in SUD patients compared with healthy controls, whilst their reward learning rates were not measurably impaired. In addition, the dopaminergic receptor agents modulated RL parameters differentially in both groups. Both amisulpride and pramipexole impaired RL parameters in healthy participants, but ameliorated learning from punishment in SUD patients.

CONCLUSION

Our findings suggest that RL impairments seen in SUD patients are associated with altered dopamine function.

Involvement of Midbrain Dopamine Neuron Activity in Negative Reinforcement Learning in Mice

The activity of the midbrain dopamine system reflects the valence of environmental events and modulates various brain structures to modify an organism's behavior. A series of recent studies reported that the direct and indirect pathways in the striatum are critical for instrumental learning, but the dynamic changes in dopamine neuron activity that occur during negative reinforcement learning are still largely unclear. In the present study, by using a negative reinforcement learning paradigm employing foot shocks as aversive stimuli, bidirectional changes in substantia nigra pars compacta (SNc) dopamine neuron activity in the learning and habituation phases were observed. The results showed that in the learning phase, before mice had mastered the skill of escaping foot shocks, the presence of foot shocks induced a transient reduction in the activity of SNc dopamine neurons; however, in the habituation phase, in which the learned skill was automated, it induced a transient increase. Microinjection of a dopamine D1 receptor (D1R) or D2 receptor (D2R) antagonist into the dorsomedial striatum (DMS) significantly impaired learning behavior, suggesting that the modulatory effects of dopamine on both the direct and indirect pathways are required. Moreover, during the learning phase, excitatory synaptic transmission to DMS D2R-expressing medium spiny neurons (D2-MSNs) was potentiated. However, upon completion of the learning and habituation phases, the synapses onto D1R-expressing medium spiny neurons (D1-MSNs) were potentiated, and those onto D2-MSNs were restored to normal levels. The bidirectional changes in both SNc dopamine neuron activity and DMS synaptic plasticity might be the critical neural correlates for negative reinforcement learning.

The asymmetric learning rates of murine exploratory behavior in sparse reward environments

Goal-oriented behaviors of animals can be modeled by reinforcement learning algorithms. Such algorithms predict future outcomes of selected actions utilizing action values and updating those values in response to the positive and negative outcomes. In many models of animal behavior, the action values are updated symmetrically based on a common learning rate, that is, in the same way for both positive and negative outcomes. However, animals in environments with scarce rewards may have uneven learning rates. To investigate the asymmetry in learning rates in reward and non-reward, we analyzed the exploration behavior of mice in five-armed bandit tasks using a Q-learning model with differential learning rates for positive and negative outcomes. The positive learning rate was significantly higher in a scarce reward environment than in a rich reward environment, and conversely, the negative learning rate was significantly lower in the scarce environment. The positive to negative learning rate ratio was about 10 in the scarce environment and about 2 in the rich environment. This result suggests that when the reward probability was low, the mice tend to ignore failures and exploit the rare rewards. Computational modeling analysis revealed that the increased learning rates ratio could cause an overestimation of and perseveration on rare-rewarding events, increasing total reward acquisition in the scarce environment but disadvantaging impartial exploration.

The Role of Glia in Addiction: Dopamine as a Modulator of Glial Responses in Addiction

Addiction is a chronic and potentially deadly disease considered a global health problem. Nevertheless, there is still no ideal treatment for its management. The alterations in the reward system are the most known pathophysiological mechanisms. Dopamine is the pivotal neurotransmitter involved in neuronal drug reward mechanisms and its neuronal mechanisms have been intensely investigated in recent years. However, neuroglial interactions and their relation to drug addiction development and maintenance of drug addiction have been understudied. Many reports have found that most neuroglial cells express dopamine receptors and that dopamine activity may induce neuroimmunomodulatory effects. Furthermore, current research has also shown that pro- and anti-inflammatory molecules modulate dopaminergic neuron activity. Thus, studying the immune mechanisms of dopamine associated with drug abuse is vital in researching new pathophysiological mechanisms and new therapeutic targets for addiction management.

Impulse Control Disorders in Parkinson's Disease: From Bench to Bedside

Parkinson’s disease (PD) is a neurodegenerative disorder that is characterized by symptoms that impact both motor and non-motor domains. Outside of motor impairments, PD patients are at risk for impulse control disorders (ICDs), which include excessively disabling impulsive and compulsive behaviors. ICD symptoms in PD (PD + ICD) can be broadly conceptualized as a synergistic interaction between dopamine agonist therapy and the many molecular and circuit-level changes intrinsic to PD. Aside from discontinuing dopamine agonist treatment, there remains a lack of consensus on how to best address ICD symptoms in PD. In this review, we explore recent advances in the molecular and neuroanatomical mechanisms underlying ICD symptoms in PD by summarizing a rapidly accumulating body of clinical and preclinical studies, with a special focus on the utility of rodent models in gaining new insights into the neurochemical basis of PD + ICD. We also discuss the relevance of these findings to the broader problem of impulsive and compulsive behaviors that impact a range of neuropsychiatric syndromes.

Dopamine D2R is Required for Hippocampal-dependent Memory and Plasticity at the CA3-CA1 Synapse

Dopamine receptors play an important role in motivational, emotional, and motor responses. In addition, growing evidence suggests a key role of hippocampal dopamine receptors in learning and memory. It is well known that associative learning and synaptic plasticity of CA3-CA1 requires the dopamine D1 receptor (D1R). However, the specific role of the dopamine D2 receptor (D2R) on memory-related neuroplasticity processes is still undefined. Here, by using two models of D2R loss, D2R knockout mice (Drd2-/-) and mice with intrahippocampal injections of Drd2-small interfering RNA (Drd2-siRNA), we aimed to investigate how D2R is involved in learning and memory as well as in long-term potentiation of the hippocampus. Our studies revealed that the genetic inactivation of D2R impaired the spatial memory, associative learning, and the classical conditioning of eyelid responses. Similarly, deletion of D2R reduced the activity-dependent synaptic plasticity in the hippocampal CA1-CA3 synapse. Our results demonstrate the first direct evidence that D2R is essential in behaving mice for trace eye blink conditioning and associated changes in hippocampal synaptic strength. Taken together, these results indicate a key role of D2R in regulating hippocampal plasticity changes and, in consequence, acquisition and consolidation of spatial and associative forms of memory.

Maternal Morphine Exposure and Post-Weaning Social Isolation Impair Memory and Ventral Striatum Dopamine System in Male Offspring: Is an Enriched Environment Beneficial?

Maternal opioids abuse has some deleterious consequences on next generations. Besides, children's rearing conditions can affect the behavioral states and brain plasticity in their later life. In the present study, we investigated the effects of maternal morphine (MOR) treatment and post-weaning rearing conditions on memory, pain threshold, and the ventral striatum dopaminergic activity in male offspring. Female Wistar rats were treated twice daily either with escalating doses of MOR or with normal saline (NS) one week before mating, during pregnancy and lactation. After weaning, the male pups were assigned to six groups and then raised for an 8-week period under three different conditions: standard (STD), isolated (ISO) or enriched environment (EE). The behavioral tests, including passive avoidance task, novel object recognition, and tail-flick test, were also performed. Moreover, the ventral striatum dopamine's content (DA), mRNA expressions of dopamine receptor 1(D1R) and dopamine receptor 2 (D2R), and dopamine transporter (DAT) were evaluated. The obtained data showed that maternal MOR exposure and post-weaning social isolation could dramatically impair memory in offspring, while EE could reverse these adverse outcomes. Moreover, results of tail flick latency indicated the increased pain threshold in EE animals. At molecular level, maternal MOR injections and social isolation reduced DA levels and altered expressions of D1R, D2R, and DAT within the ventral striatum of these male offspring. However, post-weaning EE partially buffered these changes. Our finding signified the effects of maternal MOR exposure and social isolation on the behaviors and neurochemistry of brain in next generation, and it also provided evidence on reversibility of these alterations following EE.

Accumbal D2R-medium spiny neurons regulate aversive behaviors through PKA-Rap1 pathway

The nucleus accumbens (NAc) plays a crucial role in various mental activities, including positive and negative reinforcement. We previously hypothesized that a balance between dopamine (DA) and adenosine signals regulates the PKA-Rap1 pathway in medium spiny neurons expressing DA D1 receptors (D1R-MSNs) or D2 receptors (D2R-MSNs) and demonstrated that the PKA-Rap1 pathway in D1R-MSNs is responsible for positive reinforcement. Here, we show the role of the PKA-Rap1 pathway in accumbal D2R-MSNs in negative reinforcement. Mice were exposed to electric foot shock as an aversive stimulus. We monitored the phosphorylation level of Rap1gap S563, which leads to the activation of Rap1. Electric foot shocks increased the phosphorylation level of GluN1 S897 and Rap1gap S563 in the NAc. The aversive stimulus-evoked phosphorylation of Rap1gap S563 was detected in accumbal D2R-MSNs and inhibited by pretreatment with adenosine A2a receptor (A2aR) antagonist. A2aR antagonist-treated mice showed impaired aversive memory in passive avoidance tests. AAV-mediated inhibition of PKA, Rap1, or MEK1 in accumbal D2R-MSNs impaired aversive memory in passive avoidance tests, whereas activation of this pathway potentiated aversive memory. Optogenetic inactivation of mesolimbic DAergic neurons induced place aversion in real-time place aversion tests. Aversive response was attenuated by inhibition of PKA-Rap1 signaling in accumbal D2R-MSNs. These results suggested that accumbal D2R-MSNs regulate aversive behaviors through the A2aR-PKA-Rap1-MEK pathway. Our findings provide a novel molecular mechanism for regulating negative reinforcement.

Blunted medial prefrontal cortico-limbic reward-related effective connectivity and depression

Major depressive disorder is a leading cause of disability and significant mortality, yet mechanistic understanding remains limited. Over the past decade evidence has accumulated from case-control studies that depressive illness is associated with blunted reward activation in the basal ganglia and other regions such as the medial prefrontal cortex. However it is unclear whether this finding can be replicated in a large number of subjects. The functional anatomy of the medial prefrontal cortex and basal ganglia has been extensively studied and the former has excitatory glutamatergic projections to the latter. Reduced effect of glutamatergic projections from the prefrontal cortex to the nucleus accumbens has been argued to underlie motivational disorders such as depression, and many prominent theories of major depressive disorder propose a role for abnormal cortico-limbic connectivity. However, it is unclear whether there is abnormal reward-linked effective connectivity between the medial prefrontal cortex and basal ganglia related to depression. While resting state connectivity abnormalities have been frequently reported in depression, it has not been possible to directly link these findings to reward-learning studies. Here, we tested two main hypotheses. First, mood symptoms are associated with blunted striatal reward prediction error signals in a large community-based sample of recovered and currently ill patients, similar to reports from a number of studies. Second, event-related directed medial prefrontal cortex to basal ganglia effective connectivity is abnormally increased or decreased related to the severity of mood symptoms. Using a Research Domain Criteria approach, data were acquired from a large community-based sample of subjects who participated in a probabilistic reward learning task during event-related functional MRI. Computational modelling of behaviour, model-free and model-based functional MRI, and effective connectivity dynamic causal modelling analyses were used to test hypotheses. Increased depressive symptom severity was related to decreased reward signals in areas which included the nucleus accumbens in 475 participants. Decreased reward-related effective connectivity from the medial prefrontal cortex to striatum was associated with increased depressive symptom severity in 165 participants. Decreased striatal activity may have been due to decreased cortical to striatal connectivity consistent with glutamatergic and cortical-limbic related theories of depression and resulted in reduced direct pathway basal ganglia output. Further study of basal ganglia pathophysiology is required to better understand these abnormalities in patients with depressive symptoms and syndromes.

A Neurofunctional Domains Approach to Evaluate D1/D5 Dopamine Receptor Partial Agonism on Cognition and Motivation in Healthy Volunteers With Low Working Memory Capacity

BACKGROUND

Dopamine D1 receptor signaling plays key roles in core domains of neural function, including cognition and reward processing; however, many questions remain about the functions of circuits modulated by dopamine D1 receptor, largely because clinically viable, selective agonists have yet to be tested in humans.

METHODS

Using a novel, exploratory neurofunctional domains study design, we assessed the safety, tolerability, pharmacodynamics, and pharmacokinetics of PF-06412562, a selective D1/D5R partial agonist, in healthy male volunteers who met prespecified criteria for low working memory capacity. Functional magnetic resonance imaging, electrophysiologic endpoints, and behavioral paradigms were used to assess working memory, executive function, and motivation/reward processing following multiple-dose administration of PF-06412562. A total of 77 patients were assigned PF-06412562 (3 mg twice daily and 15 mg twice daily) or placebo administered for 5 to 7 days. Due to the exploratory nature of the study, it was neither powered for any specific treatment effect nor corrected for multiple comparisons.

RESULTS

Nominally significant improvements from baseline in cognitive endpoints were observed in all 3 groups; however, improvements in PF-06412562-treated patients were less than in placebo-treated participants. Motivation/reward processing endpoints were variable. PF-06412562 was safe and well tolerated, with no serious adverse events, severe adverse events, or adverse events leading to dose reduction or temporary discontinuation except for 1 permanent discontinuation due to increased orthostatic heart rate.

CONCLUSIONS

PF-06412562, in the dose range and patient population explored in this study, did not improve cognitive function or motivation/reward processing more than placebo over the 5- to 7-day treatment period.

CLINICALTRIALS.GOV IDENTIFIER

NCT02306876.

Neurotransmission through dopamine D1 receptors is required for aversive memory formation and Arc activation in the cerebral cortex

Dopaminergic neurotransmission is considered to play an important role not only in reward-based learning, but also in aversive learning. Here, we investigated the role of dopaminergic neurotransmission via dopamine D1 receptors (D1Rs) in aversive memory formation in a passive avoidance test using D1R knockdown (KD) mice, in which the expression of D1Rs can conditionally and reversibly be controlled by doxycycline (Dox) treatment. We also performed whole-brain imaging after aversive footshock stimulation in activity-regulated cytoskeleton protein (Arc)-dVenus D1RKD mice, which were crossbred from Arc-dVenus transgenic mice and D1RKD mice, to examine the distribution of Arc-controlled dVenus expression in the hippocampus and cerebral cortex during aversive memory formation. Knockdown of D1R expression following Dox treatment resulted in impaired performance in the passive avoidance test and was associated with a decrease in dVenus expression in the cerebral cortex (visual, somatosensory, and motor cortices), but not the hippocampus, compared with control mice without Dox treatment. These findings indicate that D1R-mediated dopaminergic transmission is critical for aversive memory formation, specifically by influencing Arc expression in the cerebral cortex.

Influence of nonsynaptic α1 glycine receptors on ethanol consumption and place preference

Here, we used knock-in (KI) mice that have ethanol-insensitive alpha 1 glycine receptors (GlyRs) (KK385/386AA) to examine how alpha 1 GlyRs might affect binge drinking and conditioned place preference. Data show that tonic alpha 1 GlyR-mediated currents were exclusively sensitive to ethanol only in wild-type mice. Behavioral studies showed that the KI mice have a higher intake of ethanol upon first exposure to drinking and greater conditioned place preference to ethanol. This study suggests that nonsynaptic alpha 1-containing GlyRs have a role in motivational and early reinforcing effects of ethanol.

Maladaptive striatal plasticity and abnormal reward-learning in cervical dystonia

In monogenetic generalized forms of dystonia, in vitro neurophysiological recordings have demonstrated direct evidence for abnormal plasticity at the level of the cortico-striatal synapse. It is unclear whether similar abnormalities contribute to the pathophysiology of cervical dystonia, the most common type of focal dystonia. We investigated whether abnormal cortico-striatal synaptic plasticity contributes to abnormal reward-learning behavior in patients with focal dystonia. Forty patients and 40 controls performed a reward gain and loss avoidance reversal learning task. Participant's behavior was fitted to a computational model of the basal ganglia incorporating detailed cortico-striatal synaptic learning rules. Model comparisons were performed to assess the ability of four hypothesized receptor specific abnormalities of cortico-striatal long-term potentiation (LTP) and long-term depression (LTD): increased or decreased D1:LTP/LTD and increased or decreased D2: LTP/LTD to explain abnormal behavior in patients. Patients were selectively impaired in the post-reversal phase of the reward task. Individual learning rates in the reward reversal task correlated with the severity of the patient's motor symptoms. A model of the striatum with decreased D2:LTP/ LTD best explained the patient's behavior, suggesting excessive D2 cortico-striatal synaptic depotentiation could underpin biased reward-learning in patients with cervical dystonia. Reversal learning impairment in cervical dystonia may be a behavioral correlate of D2-specific abnormalities in cortico-striatal synaptic plasticity. Reinforcement learning tasks with computational modeling could allow the identification of molecular targets for novel treatments based on their ability to restore normal reward-learning behavior in these patients.

High ethanol sensitive glycine receptors regulate firing in D1 medium spiny neurons in the nucleus accumbens

Inhibitory glycine receptors (GlyRs) are widely expressed in spinal cord and brain stem. They are also expressed in the nucleus Accumbens (nAc) where they have been implicated in the release of dopamine from the ventral tegmental area to the nAc in the presence of ethanol. One of the major types of neurons in the nAc are the Dopamine 1 receptor-expressing (D1+) medium spiny neurons (MSNs) that are activated when addictive drugs, like ethanol, are administrated. Thus, D1(+) MSNs are a relevant target for the study of ethanol effects. Here, using electrophysiological recordings, we report that GlyRs in D1(+) MSNs are highly sensitive to ethanol, with potentiation starting at 5 mM (26 ± 5%). Single channel recordings in D1(+) MSNs showed that 10 mM ethanol increased the open probability of the channel (0.22 ± 0.05 versus 0.66 ± 0.16), but did not affect channel conductance (~40 pS). A glycinergic mediated tonic current in D1(+) MSNs was potentiated by 10 and 50 mM ethanol causing a reduction in the excitability of these cells. A 34 ± 7% reduction in action potential firing was observed in these neurons in the presence of 50 mM ethanol. Interestingly, no effects of ethanol were detected in the presence of strychnine or in D1(-) MSNs in the nAc. These results indicate that GlyRs present in D1(+) MSNs are sensitive to low concentrations of ethanol, and that potentiation of this inhibitory current regulates the activation of nAc, acting as a homeostatic signal that would prevent over-activation of the reward system when drugs like ethanol are consumed.

A comparison of implicit and explicit reward learning in low risk alcohol users versus people who binge drink and people with alcohol dependence

Chronic alcohol use leads to specific neurobiological alterations in the dopaminergic brain reward system, which probably are leading to a reward deficiency syndrome in alcohol dependence. The purpose of our study was to examine the effects of such hypothesized neurobiological alterations on the behavioral level, and more precisely on the implicit and explicit reward learning. Alcohol users were classified as dependent drinkers (using the DSM-IV criteria), binge drinkers (using criteria of the USA National Institute on Alcohol Abuse and Alcoholism) or low-risk drinkers (following recommendations of the Scientific board of trustees of the German Health Ministry). The final sample (n = 94) consisted of 36 low-risk alcohol users, 37 binge drinkers and 21 abstinent alcohol dependent patients. Participants were administered a probabilistic implicit reward learning task and an explicit reward- and punishment-based trial-and-error-learning task. Alcohol dependent patients showed a lower performance in implicit and explicit reward learning than low risk drinkers. Binge drinkers learned less than low-risk drinkers in the implicit learning task. The results support the assumption that binge drinking and alcohol dependence are related to a chronic reward deficit. Binge drinking accompanied by implicit reward learning deficits could increase the risk for the development of an alcohol dependence.

•

Alcohol dependent patients were impaired in implicit and explicit reward learning.

•

Alcohol dependence may lead to implicit and explicit reward learning deficits.

•

Binge drinkers learned less than low-risk drinkers in the implicit learning task.

•

Binge drinking is related to implicit reward learning deficits.

Role of basal ganglia neurocircuitry in the pathology of psychiatric disorders

Over the last few decades, advances in human and animal-based techniques have greatly enhanced our understanding of the neural mechanisms underlying psychiatric disorders. Many of these studies have indicated connectivity between and alterations within basal ganglia structures to be particularly pertinent to the development of symptoms associated with several of these disorders. Here we summarize the connectivity, molecular composition, and function of sites within basal ganglia neurocircuits. Then we review the current literature from both human and animal studies concerning altered basal ganglia function in five common psychiatric disorders: obsessive-compulsive disorder, substance-related and addiction disorders, major depressive disorder, generalized anxiety disorder, and schizophrenia. Finally, we present a model based upon the findings of these studies that highlights the striatum as a particularly attractive target for restoring normal function to basal ganglia neurocircuits altered within psychiatric disorder patients.

Compulsive methamphetamine taking and abstinence in the presence of adverse consequences: Epigenetic and transcriptional consequences in the rat brain

Methamphetamine addiction is characterized by compulsive binges of drug intake despite adverse life consequences. A model of methamphetamine self-administration that includes contingent footshocks to constitute adverse consequences has helped to segregate rats that reduce or stop lever pressing for methamphetamine (sensitive) from those that continue to lever press for the drug (resistant) in the presence of negative outcomes. We have observed differential DNA hydroxymethylation and increased expression of potassium channel mRNAs in the nucleus accumbens of sensitive compared to resistant rats, suggesting a role of these channels in suppressing methamphetamine intake. There were also significant increases in nerve growth factor (NGF) expression and activation of its downstream signaling pathway (NGF-TrkA and p75NTR/MAPK signaling) in only the dorsal striatum of sensitive rats after a month of abstinence. In contrast, oxytocin mRNA expression was increased in only the nucleus accumbens of resistant rats compared to sensitive rats euthanized after that time. These results indicate that footshocks can differentiate two behavioral phenotypes with differential biochemical and epigenetic consequences in the ventral and dorsal striatum.

Nicotine improves probabilistic reward learning in wildtype but not alpha7 nAChR null mutants, yet alpha7 nAChR agonists do not improve probabilistic learning

Cognitive impairments, e.g., reward learning, are present in various psychiatric disorders and warrant treatment. Improving reward-related learning could synergistically enhance psychosocial treatments and cognition generally. A critical first step is to understand the mechanisms underlying reward learning. The dopamine system has been implicated in such learning, but less known is how indirect activation of this system may affect reward learning. We determined the role of alpha7 nicotinic acetylcholine receptors (nAChR) on a probabilistic reversal learning task (PRLT) in mice that includes reward and punishment. Male alpha7 knockout (KO), heterozygous (HT), and wildtype (WT) littermate mice (n = 84) were treated with vehicle, 0.03, or 0.3 mg/kg nicotine. Two cohorts of C57BL/6NJ male mice were treated with various alpha7 nAChR ligands, including the full agonists PNU282877 and AR-R-17779, the positive allosteric modulator CCMI, the partial agonist SSR180711, and the antagonist methyllycaconitine. All mice were then tested in the PRLT. Nicotine (0.3 mg/kg) significantly improved initial reward learning in alpha7 WT and HT mice but did not improve learning in KO mice, suggesting an involvement of the alpha7 nAChR in the pro-learning effects of nicotine. Neither alpha7 nAChR treatments (PNU282987, AR-R-17779, CCMI, SSR180711, nor methyllycaconitine) affected mouse PRLT performance however. Nicotine improved reward learning via a mechanism that may include alpha7 nAChRs. This improvement unlikely relied solely on alpha7 nAChRs however, since no alpha7 nAChR ligand improved reward learning in normal mice. Future assessments of the effects of other nAChR subtypes on reward learning are needed.

Medium spiny neurons of the anterior dorsomedial striatum mediate reversal learning in a cell-type-dependent manner

The striatum has been implicated in the regulation of cognitive flexibility. Abnormalities in the anterior dorsomedial striatum (aDMS) are revealed in many mental disorders in which cognitive inflexibility is commonly observed. However, it remains poorly understood whether the aDMS plays a special role in flexible cognitive control and what the regulation pattern is in different neuronal populations. Based on the reversal learning task in mice, we showed that optogenetic activation in dopamine receptor 1-expressing medium spiny neurons (D1R-MSNs) of the aDMS impaired flexibility; meanwhile, suppressing these neurons facilitated behavioral performance. Conversely, D2R-MSN activation accelerated reversal learning, but it induced no change through neuronal suppression. The acquisition and retention of discrimination learning were unaffected by the manipulation of any type of MSN. Through bi-direct optogenetic modulation in D1R-MSNs of the same subject in a serial reversal learning task, we further revealed the function of D1R-MSNs during different stages of reversal learning, where inhibiting and exciting the same group of neurons reduced perseverative errors and increased regressive errors. Following D1R- and D2R-MSN activation in the aDMS, neuronal activity of the mediodorsal thalamus decreased and increased, respectively, in parallel with behavioral impairment and facilitation, but not as a direct result of the activation of the striatal MSNs. We propose that D1R- and D2R-MSN sub-populations in the aDMS exert opposing functions in cognitive flexibility regulation, with more important and complex roles of D1R-MSNs involved. Mental disorders with cognitive flexibility problems may feature an underlying functional imbalance in the aDMS' two types of neurons.

Making Sense of Rodent Models of Anhedonia

A markedly reduced interest or pleasure in activities previously considered pleasurable is a main symptom in mood disorder and psychosis and is often present in other psychiatric disorders and neurodegenerative diseases. This condition can be labeled as "anhedonia," although in its most rigorous connotation the term refers to the lost capacity to feel pleasure that is one aspect of the complex phenomenon of processing and responding to reward. The responses to rewarding stimuli are relatively easy to study in rodents, and the experimental conditions that consistently and persistently impair these responses are used to model anhedonia. To this end, long-term exposure to environmental aversive conditions is primarily used, and the resulting deficits in reward responses are often accompanied by other deficits that are mainly reminiscent of clinical depressive symptoms. The different components of impaired reward responses induced by environmental aversive events can be assessed by different tests or protocols that require different degrees of time allocation, technical resources, and equipment. Rodent models of anhedonia are valuable tools in the study of the neurobiological mechanisms underpinning impaired behavioral responses and in the screening and characterization of drugs that may reverse these behavioral deficits. In particular, the antianhedonic or promotivational effects are relevant features in the spectrum of activities of drugs used in mood disorders or psychosis. Thus, more than the model, it is the choice of tests that is crucial since it influences which facets of anhedonia will be detected and should be tuned to the purpose of the study.

Presence of Inhibitory Glycinergic Transmission in Medium Spiny Neurons in the Nucleus Accumbens

It is believed that the rewarding actions of drugs are mediated by dysregulation of the mesolimbic dopaminergic system leading to increased levels of dopamine in the nucleus accumbens (nAc). It is widely recognized that GABAergic transmission is critical for neuronal inhibition within nAc. However, it is currently unknown if medium spiny neurons (MSNs) also receive inhibition by means of glycinergic synaptic inputs. We used a combination of proteomic and electrophysiology studies to characterize the presence of glycinergic input into MSNs from nAc demonstrating the presence of glycine transmission into nAc. In D1 MSNs, we found low frequency glycinergic miniature inhibitory postsynaptic currents (mIPSCs) which were blocked by 1 μM strychnine (STN), insensitive to low (10, 50 mM) and high (100 mM) ethanol (EtOH) concentrations, but sensitive to 30 μM propofol. Optogenetic experiments confirmed the existence of STN-sensitive glycinergic IPSCs and suggest a contribution of GABA and glycine neurotransmitters to the IPSCs in nAc. The study reveals the presence of glycinergic transmission in a non-spinal region and opens the possibility of a novel mechanism for the regulation of the reward pathway.

Chronic inflammatory pain decreases the glutamate vesicles in presynaptic terminals of the nucleus accumbens

Reward system has been proved to be important to nociceptive behavior, and the nucleus accumbens (NAc) is a key node in reward circuitry. It has been further revealed that dopamine system modulates the NAc to influence the pain sensation, whereas the role of glutamatergic projection in the NAc in the modulation of chronic pain is still elusive. In this study, we used a complete Freund’s adjuvant-induced chronic inflammatory pain model to explore the changes of the glutamatergic terminals in the NAc, and we found that following the chronic inflammation, the protein level of vesicular glutamate transporter1 (VGLUT1) was significantly decreased in the NAc. Immunofluorescence staining further showed a reduced expression of VGLUT1-positive terminals in the dopamine receptor 2 (D2R) spiny projection neurons of NAc after chronic inflammatory pain. Furthermore, using a whole-cell recording in double transgenic mice, in which dopamine receptor 1- and D2R-expressing neurons can be visualized, we found that the frequency of spontaneous excitatory postsynaptic currents was significantly decreased and paired-pulse ratio of evoked excitatory postsynaptic currents was increased in D2R neurons, but not in dopamine receptor 1 neurons in NAc of complete Freund’s adjuvant group. Moreover, the abnormal expression of soluble N-ethylmaleimide-sensitive factor attachment protein receptor complex contributed to the reduced formation of glutamate vesicles. Hence, our results demonstrated that decreased glutamate release in the indirect pathway of the NAc may be a critical mechanism for chronic pain and provided a novel evidence for the presynaptic mechanisms in chronic pain regulation.

Dopamine receptors mediate strategy abandoning via modulation of a specific prelimbic cortex-nucleus accumbens pathway in mice

The ability to abandon old strategies and adopt new ones is essential for survival in a constantly changing environment. While previous studies suggest the importance of the prefrontal cortex and some subcortical areas in the generation of strategy-switching flexibility, the fine neural circuitry and receptor mechanisms involved are not fully understood. In this study, we showed that optogenetic excitation and inhibition of the prelimbic cortex-nucleus accumbens (NAc) pathway in the mouse respectively enhances and suppresses strategy-switching ability in a cross-modal spatial-egocentric task. This ability is dependent on an intact dopaminergic tone in the NAc, as local dopamine denervation impaired the performance of the animal in the switching of tasks. In addition, based on a brain-slice preparation obtained from Drd2-EGFP BAC transgenic mice, we demonstrated direct innervation of D2 receptor-expressing medium spiny neurons (D2-MSNs) in the NAc by prelimbic cortical neurons, which is under the regulation by presynaptic dopamine receptors. While presynaptic D1-type receptor activation enhances the glutamatergic transmission from the prelimbic cortex to D2-MSNs, D2-type receptor activation suppresses this synaptic connection. Furthermore, manipulation of this pathway by optogenetic activation or administration of a D1-type agonist or a D2-type antagonist could restore impaired task-switching flexibility in mice with local NAc dopamine depletion; this restoration is consistent with the effects of knocking down the expression of specific dopamine receptors in the pathway. Our results point to a critical role of a specific prelimbic cortex-NAc subpathway in mediating strategy abandoning, allowing the switching from one strategy to another in problem solving.

Altered morphology of the nucleus accumbens in persistent developmental stuttering

PURPOSE

Neuroimaging studies in persistent developmental stuttering repeatedly report altered basal ganglia functions. Together with thalamus and cerebellum, these structures mediate sensorimotor functions and thus represent a plausible link between stuttering and neuroanatomy. However, stuttering is a complex, multifactorial disorder. Besides sensorimotor functions, emotional and social-motivational factors constitute major aspects of the disorder. Here, we investigated cortical and subcortical gray matter regions to study whether persistent developmental stuttering is also linked to alterations of limbic structures.

METHODS

The study included 33 right-handed participants who stutter and 34 right-handed control participants matched for sex, age, and education. Structural images were acquired using magnetic resonance imaging to estimate volumetric characteristics of the nucleus accumbens, hippocampus, amygdala, pallidum, putamen, caudate nucleus, and thalamus.

RESULTS

Volumetric comparisons and vertex-based shape comparisons revealed structural differences. The right nucleus accumbens was larger in participants who stutter compared to controls.

CONCLUSION

Recent theories of basal ganglia functions suggest that the nucleus accumbens is a motivation-to-movement interface. A speaker intends to reach communicative goals, but stuttering can derail these efforts. It is therefore highly plausible to find alterations in the motivation-to-movement interface in stuttering. While behavioral studies of stuttering sought to find links between the limbic and sensorimotor system, we provide the first neuroimaging evidence of alterations in the limbic system. Thus, our findings might initialize a unified neurobiological framework of persistent developmental stuttering that integrates sensorimotor and social-motivational neuroanatomical circuitries.

Nigrostriatal and Mesolimbic D2/3 Receptor Expression in Parkinson's Disease Patients with Compulsive Reward-Driven Behaviors

The nigrostriatal and mesocorticolimbic dopamine networks regulate reward-driven behavior. Regional alterations to mesolimbic dopamine D_2/3 receptor expression are described in drug-seeking and addiction disorders. Parkinson's disease (PD) patients are frequently prescribed D₂-like dopamine agonist (DAgonist) therapy for motor symptoms, yet a proportion develop clinically significant behavioral addictions characterized by impulsive and compulsive behaviors (ICBs). Until now, changes in D_2/3 receptor binding in both striatal and extrastriatal regions have not been concurrently quantified in this population. We identified 35 human PD patients (both male and female) receiving DAgonist therapy, with (n = 17) and without (n = 18) ICBs, matched for age, disease duration, disease severity, and dose of dopamine therapy. In the off-dopamine state, all completed PET imaging with [¹⁸F]fallypride, a high affinity D₂-like receptor ligand that can measure striatal and extrastriatal D_2/3 nondisplaceable binding potential (BP_ND). Striatal differences between ICB+/ICB- patients localized to the ventral striatum and putamen, where ICB+ subjects had reduced BP_ND In this group, self-reported severity of ICB symptoms positively correlated with midbrain D_2/3 receptor BP_ND Group differences in regional D_2/3 BP_ND relationships were also notable: ICB+ (but not ICB-) patients expressed positive correlations between midbrain and caudate, putamen, globus pallidus, and amygdala BP_NDs. These findings support the hypothesis that compulsive behaviors in PD are associated with reduced ventral and dorsal striatal D_2/3 expression, similar to changes in comparable behavioral disorders. The data also suggest that relatively preserved ventral midbrain dopaminergic projections throughout nigrostriatal and mesolimbic networks are characteristic of ICB+ patients, and may account for differential DAgonist therapeutic response.SIGNIFICANCE STATEMENT The biologic determinants of compulsive reward-based behaviors have broad clinical relevance, from addiction to neurodegenerative disorders. Here, we address biomolecular distinctions in Parkinson's disease patients with impulsive compulsive behaviors (ICBs). This is the first study to image a large cohort of ICB+ patients using positron emission tomography with [18F]fallypride, allowing quantification of D_2/3 receptors throughout the mesocorticolimbic network. We demonstrate widespread differences in dopaminergic networks, including (1) D2-like receptor distinctions in the ventral striatum and putamen, and (2) a preservation of widespread dopaminergic projections emerging from the midbrain, which is associated with the severity of compulsive behaviors. This clearly illustrates the roles of D_2/3 receptors and medication effects in maladaptive behaviors, and localizes them specifically to nigrostriatal and extrastriatal regions.

Male-specific deficits in natural reward learning in a mouse model of neurodevelopmental disorders

Neurodevelopmental disorders, including autism spectrum disorders, are highly male biased, but the underpinnings of this are unknown. Striatal dysfunction has been strongly implicated in the pathophysiology of neurodevelopmental disorders, raising the question of whether there are sex differences in how the striatum is impacted by genetic risk factors linked to neurodevelopmental disorders. Here we report male-specific deficits in striatal function important to reward learning in a mouse model of 16p11.2 hemideletion, a genetic mutation that is strongly associated with the risk of neurodevelopmental disorders, particularly autism and attention-deficit hyperactivity disorder. We find that male, but not female, 16p11.2 deletion animals show impairments in reward-directed learning and maintaining motivation to work for rewards. Male, but not female, deletion animals overexpress mRNA for dopamine receptor 2 and adenosine receptor 2a in the striatum, markers of medium spiny neurons signaling via the indirect pathway, associated with behavioral inhibition. Both sexes show a 50% reduction of mRNA levels of the genes located within the 16p11.2 region in the striatum, including the kinase extracellular-signal related kinase 1 (ERK1). However, hemideletion males show increased activation in the striatum for ERK1, both at baseline and in response to sucrose, a signaling change associated with decreased striatal plasticity. This increase in ERK1 phosphorylation is coupled with a decrease in the abundance of the ERK phosphatase striatum-enriched protein-tyrosine phosphatase in hemideletion males. In contrast, females do not show activation of ERK1 in response to sucrose, but notably hemideletion females show elevated protein levels for ERK1 as well as the related kinase ERK2 over what would be predicted by mRNA levels. These data indicate profound sex differences in the impact of a genetic lesion linked with neurodevelopmental disorders, including mechanisms of male-specific vulnerability and female-specific resilience impacting intracellular signaling in the brain.

Reward from bugs to bipeds: a comparative approach to understanding how reward circuits function

In a complex environment, animals learn from their responses to stimuli and events. Appropriate response to reward and punishment can promote survival, reproduction and increase evolutionary fitness. Interestingly, the neural processes underlying these responses are remarkably similar across phyla. In all species, dopamine is central to encoding reward and directing motivated behaviors, however, a comprehensive understanding of how circuits encode reward and direct motivated behaviors is still lacking. In part, this is a result of the sheer diversity of neurons, the heterogeneity of their responses and the complexity of neural circuits within which they are found. We argue that general features of reward circuitry are common across model organisms, and thus principles learned from invertebrate model organisms can inform research across species. In particular, we discuss circuit motifs that appear to be functionally equivalent from flies to primates. We argue that a comparative approach to studying and understanding reward circuit function provides a more comprehensive understanding of reward circuitry, and informs disorders that affect the brain’s reward circuitry.

Cannabis and cocaine decrease cognitive impulse control and functional corticostriatal connectivity in drug users with low activity DBH genotypes

The dopamine β-hydroxylase (DβH) enzyme transforms dopamine into noradrenaline. We hypothesized that individuals with low activity DBH genotypes (rs1611115 CT/TT) are more sensitive to the influence of cannabis and cocaine on cognitive impulse control and functional connectivity in the limbic ‘reward’ circuit because they experience a drug induced hyperdopaminergic state compared to individuals with high activity DBH genotypes (rs1611115 CC). Regular drug users (N = 122) received acute doses of cannabis (450 μg/kg THC), cocaine HCl 300 mg and placebo. Cognitive impulse control was assessed by means of the Matching Familiar Figures Test (MFFT). Resting state fMRI was measured in a subset of participants to determine functional connectivity between the nucleus accumbens (NAc) and (sub)cortical areas. The influence of cannabis and cocaine on impulsivity and functional connectivity significantly interacted with DBH genotype. Both drugs increased cognitive impulsivity in participants with CT/TT genotypes but not in CC participants. Both drugs also reduced functional connectivity between the NAc and the limbic lobe, prefrontal cortex, striatum and thalamus and primarily in individuals with CT/TT genotypes. Correlational analysis indicated a significant negative association between cognitive impulsivity and functional connectivity in subcortical areas of the brain. It is concluded that interference of cannabis and cocaine with cognitive impulse control and functional corticostriatal connectivity depends on DBH genotype. The present data provide a neural substrate and behavioral mechanism by which drug users can progress to drug seeking and may also offer a rationale for targeted pharmacotherapy in chronic drug users with high risk DBH genotypes.

Mechanisms of automaticity and anticipatory control in fluid intelligence

The constructs of fluid (Gf) and crystalized (Gc) intelligence represent an early attempt to describe the mechanisms of problem solving in the vertebrate brain. Modern neuroscience demonstrates that problem solving involves interplay between the mechanisms of automaticity and anticipatory control, enabling nature's elegant solution to the challenges animals face in their environment. Studies of neural functioning are making clear the primary role of cortical-subcortical interactions in the manifestation of intelligent behavior in humans and other vertebrates. A tridimensional model of intelligent problem solving is explored, wherein the basal ganglia system (BGS) and cerebrocerebellar system (CCS) interact within large scale brain networks. The BGS and CCS work together to enable automaticity to occur. The BGS enables the organism to learn what to do through a powerful instrumental learning system. The BGS also regulates when behavior is released through an inhibitory system which is incredibly sensitive to context. The CCS enables the organism to learn how to perform adaptive behaviors. Internal cerebellar models enable gradual improvements in the quality of behavioral output. The BGS and CCS interact within large scale brain networks, including the dorsal attention network (DAN), ventral attention network (VAN), default mode network (DMN) and frontoparietal network (FPN). The interactions of these systems enable vertebrate organisms to develop a vast array of complex adaptive behaviors. The benefits and importance of developing clinical tests to measure the integrity of these systems is considered.

Striatal dopamine D1 receptor suppression impairs reward-associative learning

Dopamine (DA) is required for reinforcement learning. Hence, disruptions in DA signaling may contribute to the learning deficits associated with psychiatric disorders. The DA D1 receptor (D1R) has been linked to learning and is a target for cognitive/motivational enhancement in patients with schizophrenia. Separating the striatal D1R contribution to learning vs. motivation, however, has been challenging. We suppressed striatal D1R expression in mice using a D1R-targeting short hairpin RNA (shRNA), delivered locally to the striatum via an adeno-associated virus (AAV). We then assessed reward- and punishment-associative learning using a probabilistic learning task and motivation using a progressive-ratio breakpoint procedure. We confirmed suppression of striatal D1Rs immunohistochemically and by testing locomotor activity after the administration of (+)-doxanthrine, a full D1R agonist, in control mice and those treated with the D1RshRNA. D1RshRNA-treated mice exhibited impaired reward-associative learning, while punishment-associative learning was spared. This deficit was unrelated to general learning impairments or amotivation, because the D1shRNA-treated mice exhibited normal Barnes maze learning and normal motivation in the progressive-ratio breakpoint procedure. Suppression of striatal D1Rs selectively impaired reward-associative learning whereas punishment-associative learning, aversion-motivated learning, and appetitive motivation were spared. Because patients with schizophrenia exhibit similar reward-associative learning deficits, D1R-targeted treatments should be investigated to improve reward learning in these patients.