The Functional Role of Striatal Cholinergic Interneurons in Reinforcement Learning From Computational Perspective

Spatially organized striatum-wide acetylcholine dynamics for the learning and extinction of Pavlovian cues and actions

Striatal acetylcholine (ACh) has been linked to behavioral flexibility. A key component of flexibility is down-regulating responding as valued cues and actions become decoupled from positive outcomes. We used array fiber photometry in mice to investigate how ACh release across the striatum evolves during learning and extinction of Pavlovian associations. Changes in multi-phasic release to cues and consummatory actions were bi-directional and region-specific. Following extinction, increases in cue-evoked ACh release emerged in the anterior dorsal striatum (aDS) which preceded a down-regulation of anticipatory behavior. Silencing ACh release from cholinergic interneurons in the aDS blocked behavioral extinction. Dopamine release dipped below baseline for down-shifted cues, but glutamate input onto cholinergic interneurons did not change, suggesting an intrastriatal mechanism for the emergence of ACh increases. Our large-scale mapping of striatal ACh dynamics during learning pinpoints region-specific elevations in ACh release positioned to down-regulate behavior during extinction, a central feature of flexible behavior.

Dopamine transmission in the tail striatum: Regional variation and contribution of dopamine clearance mechanisms

The striatum can be divided into four anatomically and functionally distinct domains: the dorsolateral, dorsomedial, ventral and the more recently identified caudolateral (tail) striatum. Dopamine transmission in these striatal domains underlies many important behaviours, yet little is known about this phenomenon in the tail striatum. Furthermore, the tail is divided anatomically into four divisions (dorsal, medial, intermediate and lateral) based on the profile of D1 and D2 dopamine receptor-expressing medium spiny neurons, something that is not seen elsewhere in the striatum. Considering this organisation, how dopamine transmission occurs in the tail striatum is of great interest. We recorded evoked dopamine release in the four tail divisions, with comparison to the dorsolateral striatum, using fast-scan cyclic voltammetry in rat brain slices. Contributions of clearance mechanisms were investigated using dopamine transporter knockout (DAT-KO) rats, pharmacological transporter inhibitors and dextran. Evoked dopamine release in all tail divisions was smaller in amplitude than in the dorsolateral striatum and, importantly, regional variation was observed: dorsolateral ≈ lateral > medial > dorsal ≈ intermediate. Release amplitudes in the lateral division were 300% of that in the intermediate division, which also exhibited uniquely slow peak dopamine clearance velocity. Dopamine clearance in the intermediate division was most dependent on DAT, and no alternative dopamine transporters investigated (organic cation transporter-3, norepinephrine transporter and serotonin transporter) contributed significantly to dopamine clearance in any tail division. Our findings confirm that the tail striatum is not only a distinct dopamine domain but also that each tail division has unique dopamine transmission characteristics. This supports that the divisions are not only anatomically but also functionally distinct. How this segregation relates to the overall function of the tail striatum, particularly the processing of multisensory information, is yet to be determined.

A Multiscale, Systems-Level, Neuropharmacological Model of Cortico-Basal Ganglia System for Arm Reaching Under Normal, Parkinsonian, and Levodopa Medication Conditions

In order to understand the link between substantia nigra pars compacta (SNc) cell loss and Parkinson's disease (PD) symptoms, we developed a multiscale computational model that can replicate the symptoms at the behavioural level by incorporating the key cellular and molecular mechanisms underlying PD pathology. There is a modelling tradition that links dopamine to reward and uses reinforcement learning (RL) concepts to model the basal ganglia. In our model, we replace the abstract representations of reward with the realistic variable of extracellular DA released by a network of SNc cells and incorporate it in the RL-based behavioural model, which simulates the arm reaching task. Our results successfully replicated the impact of SNc cell loss and levodopa (L-DOPA) medication on reaching performance. It also shows the side effects of medication, such as wearing off and peak dosage dyskinesias. The model demonstrates how differential dopaminergic axonal degeneration in basal ganglia results in various cardinal symptoms of PD. It was able to predict the optimum L-DOPA medication dosage for varying degrees of cell loss. The proposed model has a potential clinical application where drug dosage can be optimised as per patient characteristics.

Cholinergic Receptor Modulation as a Target for Preventing Dementia in Parkinson's Disease

Parkinson’s disease (PD) is a neurodegenerative condition characterized by the loss of dopaminergic neurons in the substantia nigra pars compacta (SNpc) in the midbrain resulting in progressive impairment in cognitive and motor abilities. The physiological and molecular mechanisms triggering dopaminergic neuronal loss are not entirely defined. PD occurrence is associated with various genetic and environmental factors causing inflammation and mitochondrial dysfunction in the brain, leading to oxidative stress, proteinopathy, and reduced viability of dopaminergic neurons. Oxidative stress affects the conformation and function of ions, proteins, and lipids, provoking mitochondrial DNA (mtDNA) mutation and dysfunction. The disruption of protein homeostasis induces the aggregation of alpha-synuclein (α-SYN) and parkin and a deficit in proteasome degradation. Also, oxidative stress affects dopamine release by activating ATP-sensitive potassium channels. The cholinergic system is essential in modulating the striatal cells regulating cognitive and motor functions. Several muscarinic acetylcholine receptors (mAChR) and nicotinic acetylcholine receptors (nAChRs) are expressed in the striatum. The nAChRs signaling reduces neuroinflammation and facilitates neuronal survival, neurotransmitter release, and synaptic plasticity. Since there is a deficit in the nAChRs in PD, inhibiting nAChRs loss in the striatum may help prevent dopaminergic neurons loss in the striatum and its pathological consequences. The nAChRs can also stimulate other brain cells supporting cognitive and motor functions. This review discusses the cholinergic system as a therapeutic target of cotinine to prevent cognitive symptoms and transition to dementia in PD.

The critical balance between dopamine D2 receptor and RGS for the sensitive detection of a transient decay in dopamine signal

In behavioral learning, reward-related events are encoded into phasic dopamine (DA) signals in the brain. In particular, unexpected reward omission leads to a phasic decrease in DA (DA dip) in the striatum, which triggers long-term potentiation (LTP) in DA D2 receptor (D2R)-expressing spiny-projection neurons (D2 SPNs). While this LTP is required for reward discrimination, it is unclear how such a short DA-dip signal (0.5-2 s) is transferred through intracellular signaling to the coincidence detector, adenylate cyclase (AC). In the present study, we built a computational model of D2 signaling to determine conditions for the DA-dip detection. The DA dip can be detected only if the basal DA signal sufficiently inhibits AC, and the DA-dip signal sufficiently disinhibits AC. We found that those two requirements were simultaneously satisfied only if two key molecules, D2R and regulators of G protein signaling (RGS) were balanced within a certain range; this balance has indeed been observed in experimental studies. We also found that high level of RGS was required for the detection of a 0.5-s short DA dip, and the analytical solutions for these requirements confirmed their universality. The imbalance between D2R and RGS is associated with schizophrenia and DYT1 dystonia, both of which are accompanied by abnormal striatal LTP. Our simulations suggest that D2 SPNs in patients with schizophrenia and DYT1 dystonia cannot detect short DA dips. We finally discussed that such psychiatric and movement disorders can be understood in terms of the imbalance between D2R and RGS.

Neurotransmitters, Cell Types, and Circuit Mechanisms of Motor Skill Learning and Clinical Applications

Animals acquire motor skills to better survive and adapt to a changing environment. The ability to learn novel motor actions without disturbing learned ones is essential to maintaining a broad motor repertoire. During motor learning, the brain makes a series of adjustments to build novel sensory–motor relationships that are stored within specific circuits for long-term retention. The neural mechanism of learning novel motor actions and transforming them into long-term memory still remains unclear. Here we review the latest findings with regard to the contributions of various brain subregions, cell types, and neurotransmitters to motor learning. Aiming to seek therapeutic strategies to restore the motor memory in relative neurodegenerative disorders, we also briefly describe the common experimental tests and manipulations for motor memory in rodents.

Tonically Active Neurons in the Monkey Dorsal Striatum Signal Outcome Feedback during Trial-and-error Search Behavior

An animal's choice behavior is shaped by the outcome feedback from selected actions in a trial-and-error approach. Tonically active neurons (TANs), presumed cholinergic interneurons in the striatum, are thought to be involved in the learning and performance of reward-directed behaviors, but it remains unclear how TANs are involved in shaping reward-directed choice behaviors based on the outcome feedback. To this end, we recorded activity of TANs from the dorsal striatum of two macaque monkeys (Macaca fuscata; 1 male, 1 female) while they performed a multi-step choice task to obtain multiple rewards. In this task, the monkeys first searched for a rewarding target from among three alternatives in a trial-and-error manner and then earned additional rewards by repeatedly choosing the rewarded target. We found that a considerable proportion of TANs selectively responded to either the reward or the no-reward outcome feedback during the trial-and-error search, but these feedback responses were not observed during repeat trials. Moreover, the feedback responses of TANs were similarly observed in any search trials, without distinctions regarding the predicted probability of rewards and the location of chosen targets. Unambiguously, TANs detected reward and no-reward feedback specifically when the monkeys performed trial-and-error searches, in which the monkeys were learning the value of the targets and adjusting their subsequent choice behavior based on the reward and no-reward feedback. These results suggest that striatal cholinergic interneurons signal outcome feedback specifically during search behavior, in circumstances where the choice outcomes cannot be predicted with certainty by the animals.

Striatal Cholinergic Interneurons: How to Elucidate Their Function in Health and Disease

Striatal cholinergic interneurons (CINs) are the main source of acetylcholine in the striatum and are believed to play an important role in basal ganglia physiology and pathophysiology. The role of CINs in striatal function is known mostly from extracellular recordings of tonically active striatal neurons in monkeys, which are believed to correspond to CINs. Because these neurons transiently respond to motivationally cues with brief pauses, flanked by bursts of increased activity, they are classically viewed as key players in reward-related learning. However, CIN modulatory function within the striatal network has been mainly inferred from the action of acetylcholine agonists/antagonists or through CIN activation. These manipulations are far from recapitulating CIN activity in response to behaviorally-relevant stimuli. New technical tools such as optogenetics allow researchers to specifically manipulate this sparse neuronal population and to mimic their typical pause response. For example, it is now possible to investigate how short inhibition of CIN activity shapes striatal properties. Here, we review the most recent literature and show how these new techniques have brought considerable insights into the functional role of CINs in normal and pathological states, raising several interesting and novel questions. To continue moving forward, it is crucial to determine in detail CIN activity changes during behavior, particularly in rodents. We will also discuss how computational approaches combined with optogenetics will contribute to further our understanding of the CIN role in striatal circuits.

The interplay between cerebellum and basal ganglia in motor adaptation: A modeling study

Motor adaptation to perturbations is provided by learning mechanisms operating in the cerebellum and basal ganglia. The cerebellum normally performs motor adaptation through supervised learning using information about movement error provided by visual feedback. However, if visual feedback is critically distorted, the system may disengage cerebellar error-based learning and switch to reinforcement learning mechanisms mediated by basal ganglia. Yet, the exact conditions and mechanisms of cerebellum and basal ganglia involvement in motor adaptation remain unknown. We use mathematical modeling to simulate control of planar reaching movements that relies on both error-based and non-error-based learning mechanisms. We show that for learning to be efficient only one of these mechanisms should be active at a time. We suggest that switching between the mechanisms is provided by a special circuit that effectively suppresses the learning process in one structure and enables it in the other. To do so, this circuit modulates learning rate in the cerebellum and dopamine release in basal ganglia depending on error-based learning efficiency. We use the model to explain and interpret experimental data on error- and non-error-based motor adaptation under different conditions.