A neurocomputational model of tonic and phasic dopamine in action selection: a comparison with cognitive deficits in Parkinson's disease

Simulated Dopamine Modulation of a Neurorobotic Model of the Basal Ganglia

The vertebrate basal ganglia play an important role in action selection-the resolution of conflicts between alternative motor programs. The effective operation of basal ganglia circuitry is also known to rely on appropriate levels of the neurotransmitter dopamine. We investigated reducing or increasing the tonic level of simulated dopamine in a prior model of the basal ganglia integrated into a robot control architecture engaged in a foraging task inspired by animal behaviour. The main findings were that progressive reductions in the levels of simulated dopamine caused slowed behaviour and, at low levels, an inability to initiate movement. These states were partially relieved by increased salience levels (stronger sensory/motivational input). Conversely, increased simulated dopamine caused distortion of the robot's motor acts through partially expressed motor activity relating to losing actions. This could also lead to an increased frequency of behaviour switching. Levels of simulated dopamine that were either significantly lower or higher than baseline could cause a loss of behavioural integration, sometimes leaving the robot in a 'behavioral trap'. That some analogous traits are observed in animals and humans affected by dopamine dysregulation suggests that robotic models could prove useful in understanding the role of dopamine neurotransmission in basal ganglia function and dysfunction.

Software for near-real-time voltammetric tracking of tonic neurotransmitter levels in vivo

Tonic extracellular neurotransmitter concentrations are important modulators of central network homeostasis. Disruptions in these tonic levels are thought to play a role in neurologic and psychiatric disease. Therefore, ways to improve their quantification are actively being investigated. Previously published voltammetric software packages have implemented FSCV, which is not capable of measuring tonic concentrations of neurotransmitters in vivo. In this paper, custom software was developed for near-real-time tracking (scans every 10 s) of neurotransmitters’ tonic concentrations with high sensitivity and spatiotemporal resolution both in vitro and in vivo using cyclic voltammetry combined with dynamic background subtraction (M-CSWV and FSCAV). This software was designed with flexibility, speed, and user-friendliness in mind. This software enables near-real-time measurement by reducing data analysis time through an optimized modeling algorithm, and efficient memory handling makes long-term measurement possible. The software permits customization of the cyclic voltammetric waveform shape, enabling experiments to detect a specific analyte of interest. Finally, flexibility considerations allow the user to alter the fitting parameters, filtering characteristics, and size and shape of the analyte kernel, based on data obtained live during the experiment to obtain accurate measurements as experimental conditions change. Herein, the design and advantages of this near-real-time voltammetric software are described, and its use is demonstrated in in vivo experiments.

A Neuro-Computational Model for Discrete-Continuous Dual-Task Process

Studies on dual-task (DT) procedures in human behavior are important, as they can offer great insight into the cognitive control system. Accordingly, a discrete-continuous auditory-tracking DT experiment was conducted in this study with different difficulty conditions, including a continuous mouse-tracking task concurrent with a discrete auditory task (AT). Behavioral results of 25 participants were investigated via different factors, such as response time (RT), errors, and hesitations (pauses in tracking tasks). In DT, synchronization of different target neuron units was observed in corresponding brain regions; consequently, a computational model of the stimulus process was proposed to investigate the DT interference procedure during the stimulus process. This generally relates to the bottom-up attention system that a neural resource allocates for various ongoing stimuli. We proposed a black-box model based on interactions and mesoscopic behaviors of neural units. Model structure was implemented based on neurological studies and oscillator units to represent neural activities. Each unit represents one stimulus feature of task concept. Comparing the model's output behavior with the experiment results (RT) validates the model. Evaluation of the proposed model and data on RT implies that the stimulus of the AT affects the DT procedure in the model output (84% correlation). However, the continuous task is not significantly changed (26% correlation). The continuous task simulation results were inconsistent with the experiment, suggesting that continuous interference occurs in higher cognitive processing regions and is controlled by the top-down attentional system. However, this is consistent with the psychological research finding of DT interference occurring in response preparation rather than the stimulus process stage. Furthermore, we developed the proposed model by adding qualitative interpretation and saving the model's generality to address various types of discrete continuous DT procedures. The model predicts a justification method for brain rhythm interactions by synchronization, and manipulating parameters would produce different behaviors. The decrement of coupling parameter and strength factor would predict a similar pattern as in Parkinson's disease and ADHD disorder, respectively. Also, by increasing the similarity factor among the features, the model's result shows automatic task performance in each task.

Modeling nucleus accumbens : A Computational Model from Single Cell to Circuit Level

Nucleus accumbens is part of the neural structures required for reward based learning and cognitive processing of motivation. Understanding its cellular dynamics and its role in basal ganglia circuits is important not only in diagnosing behavioral disorders and psychiatric problems as addiction and depression but also for developing therapeutic treatments for them. Building a computational model would expand our comprehension of nucleus accumbens. In this work, we are focusing on establishing a model of nucleus accumbens which has not been considered as much as dorsal striatum in computational neuroscience. We will begin by modeling the behavior of single cells and then build a holistic model of nucleus accumbens considering the effect of synaptic currents. We will verify the validity of the model by showing the consistency of simulation results with the empirical data. Furthermore, the simulation results reveal the joint effect of cortical stimulation and dopaminergic modulation on the activity of medium spiny neurons. This effect differentiates with the type of dopamine receptors.

An integrative model of Parkinson's disease treatment including levodopa pharmacokinetics, dopamine kinetics, basal ganglia neurotransmission and motor action throughout disease progression

Levodopa is considered the gold standard treatment of Parkinson's disease. Although very effective in alleviating symptoms at their onset, its chronic use with the progressive neuronal denervation in the basal ganglia leads to a decrease in levodopa's effect duration and to the appearance of motor complications. This evolution challenges the establishment of optimal regimens to manage the symptoms as the disease progresses. Based on up-to-date pathophysiological and pharmacological knowledge, we developed an integrative model for Parkinson's disease to evaluate motor function in response to levodopa treatment as the disease progresses. We combined a pharmacokinetic model of levodopa to a model of dopamine's kinetics and a neurocomputational model of basal ganglia. The parameter values were either measured directly or estimated from human and animal data. The concentrations and behaviors predicted by our model were compared to available information and data. Using this model, we were able to predict levodopa plasma concentration, its related dopamine concentration in the brain and the response performance of a motor task for different stages of disease.

Age-Related Decline in Learning Deterministic Judgment-Based Sequences

OBJECTIVES

Because sequence learning is integral to cognitive functions across the life span, the present study examined the effect of healthy aging on deterministic judgment-based sequence learning.

METHODS

College-aged, younger-old (YO), and older-old (OO) adults completed a judgment-based sequence learning task which required them to learn a full sequence by chaining together single stimulus-response associations in a step-by-step fashion.

RESULTS

Results showed that younger adults outperformed YO and OO adults; older adults were less able to acquire the full sequence and committed significantly more errors during learning. Additionally, higher sequence learning errors were associated with advancing age among older adults, even when controlling for other factors known to contribute to sequence learning abilities. Such impairments were selective to learning sequential information, because adults of all ages performed equivalently on postlearning probe trials, as well as on learning simple stimulus-response associations.

DISCUSSION

This pattern of age deficits during deterministic sequence learning challenges past reports of age preservation. Though the neural processes underlying learning cannot be determined here, our patterns of age deficits and preservation may reflect different brain regions that are involved in the task phases, adding behavioral evidence to the emerging hypothesis of frontostriatal declines despite spared hippocampal function with age.

Attentional blink and putative noninvasive dopamine markers: Two experiments to consolidate possible associations

Adaptive behavioral control involves a balance between top-down persistence and flexible updating of goals under changing demands. According to the metacontrol state model (MSM), this balance emerges from the interaction between the frontal and the striatal dopaminergic system. The attentional blink (AB) task has been argued to tap into the interaction between persistence and flexibility, as it reflects overpersistence—the too-exclusive allocation of attentional resources to the processing of the first of two consecutive targets. Notably, previous studies are inconclusive about the association between the AB and noninvasive proxies of dopamine including the spontaneous eye blink rate (sEBR), which allegedly assesses striatal dopamine levels. We aimed to substantiate and extend previous attempts to predict individual sizes of the AB in two separate experiments with larger sample sizes (N = 71 & N = 65) by means of noninvasive behavioral and physiological proxies of dopamine (DA), such as sEBR and mood measures, which are likely to reflect striatal dopamine levels, and color discrimination, which has been argued to tap into the frontal dopamine levels. Our findings did not confirm the prediction that AB size covaries with sEBR, mood, or color discrimination. The implications of this inconsistency with previous observations are discussed.

Dyslexia-related impairments in sequence learning predict linguistic abilities

Dyslexia is often characterized by disordered word recognition and spelling, though dysfunction on various non-linguistic tasks suggests a more pervasive deficit may underlie reading and spelling abilities. The serial-order learning impairment in dyslexia (SOLID) hypothesis proposes that sequence learning impairments fundamentally disrupt cognitive abilities, including linguistic processes, among individuals with dyslexia; yet only some studies report sequence learning deficits in people with dyslexia relative to controls. Evidence may be mixed because traditional sequence learning tasks often require strong motor demands, working memory processes and/or executive functions, wherein people with dyslexia can show impairments. Thus, observed sequence learning deficits in dyslexia may only appear to the extent that comorbid motor-based processes, memory capacity, or executive processes are involved. The present study measured sequence learning in college-aged students with and without dyslexia using a single task that evaluates sequencing and non-sequencing components but without strong motor, executive, or memory demands. During sequencing, each additional link in a sequence of stimuli leading to a reward is trained step-by-step, until a complete sequence is acquired. People with dyslexia made significantly more sequencing errors than controls, despite equivalent performance on non-sequencing components. Mediation analyses further revealed that sequence learning accounted for a large portion of the variance between dyslexia status and linguistic abilities, particularly pseudo-word reading. These findings extend the SOLID hypothesis by showing difficulties in the ability to acquire sequences that may play an underlying role in literacy acquisition.

Different Dopaminergic Dysfunctions Underlying Parkinsonian Akinesia and Tremor

Although the occurrence of Parkinsonian akinesia and tremor is traditionally associated to dopaminergic degeneration, the multifaceted neural processes that cause these impairments are not fully understood. As a consequence, current dopamine medications cannot be tailored to the specific dysfunctions of patients with the result that generic drug therapies produce different effects on akinesia and tremor. This article proposes a computational model focusing on the role of dopamine impairments in the occurrence of akinesia and resting tremor. The model has three key features, to date never integrated in a single computational system: (a) an architecture constrained on the basis of the relevant known system-level anatomy of the basal ganglia-thalamo-cortical loops; (b) spiking neurons with physiologically-constrained parameters; (c) a detailed simulation of the effects of both phasic and tonic dopamine release. The model exhibits a neural dynamics compatible with that recorded in the brain of primates and humans. Moreover, it suggests that akinesia might involve both tonic and phasic dopamine dysregulations whereas resting tremor might be primarily caused by impairments involving tonic dopamine release and the responsiveness of dopamine receptors. These results could lead to develop new therapies based on a system-level view of the Parkinson's disease and targeting phasic and tonic dopamine in differential ways.

Basal Ganglia dysfunctions in movement disorders: What can be learned from computational simulations

The basal ganglia are a complex neuronal system that is impaired in several movement disorders, including Parkinson's disease, Huntington's disease, and dystonia. Empirical studies have provided valuable insights into the brain dysfunctions underlying these disorders. The systems-level perspective, however, of how patients' motor, cognitive, and emotional impairments originate from known brain dysfunctions has been a challenge to empirical investigations. These causal relations have been analyzed via computational modeling, a method that describes the simulation of interacting brain processes in a computer system. In this article, we review computational insights into the brain dysfunctions underlying Parkinson's disease, Huntington's disease, and dystonia, with particular foci on dysfunctions of the dopamine system, basal ganglia pathways, and neuronal oscillations. © 2016 International Parkinson and Movement Disorder Society.

A Biologically Inspired Computational Model of Basal Ganglia in Action Selection

The basal ganglia (BG) are a subcortical structure implicated in action selection. The aim of this work is to present a new cognitive neuroscience model of the BG, which aspires to represent a parsimonious balance between simplicity and completeness. The model includes the 3 main pathways operating in the BG circuitry, that is, the direct (Go), indirect (NoGo), and hyperdirect pathways. The main original aspects, compared with previous models, are the use of a two-term Hebb rule to train synapses in the striatum, based exclusively on neuronal activity changes caused by dopamine peaks or dips, and the role of the cholinergic interneurons (affected by dopamine themselves) during learning. Some examples are displayed, concerning a few paradigmatic cases: action selection in basal conditions, action selection in the presence of a strong conflict (where the role of the hyperdirect pathway emerges), synapse changes induced by phasic dopamine, and learning new actions based on a previous history of rewards and punishments. Finally, some simulations show model working in conditions of altered dopamine levels, to illustrate pathological cases (dopamine depletion in parkinsonian subjects or dopamine hypermedication). Due to its parsimonious approach, the model may represent a straightforward tool to analyze BG functionality in behavioral experiments.

A review of presented mathematical models in Parkinson's disease: black- and gray-box models

Parkinson's disease (PD), one of the most common movement disorders, is caused by damage to the central nervous system. Despite all of the studies on PD, the formation mechanism of its symptoms remained unknown. It is still not obvious why damage only to the substantia nigra pars compacta, a small part of the brain, causes a wide range of symptoms. Moreover, the causes of brain damages remain to be fully elucidated. Exact understanding of the brain function seems to be impossible. On the other hand, some engineering tools are trying to understand the behavior and performance of complex systems. Modeling is one of the most important tools in this regard. Developing quantitative models for this disease has begun in recent decades. They are very effective not only in better understanding of the disease, offering new therapies, and its prediction and control, but also in its early diagnosis. Modeling studies include two main groups: black-box models and gray-box models. Generally, in the black-box modeling, regardless of the system information, the symptom is only considered as the output. Such models, besides the quantitative analysis studies, increase our knowledge of the disorders behavior and the disease symptoms. The gray-box models consider the involved structures in the symptoms appearance as well as the final disease symptoms. These models can effectively save time and be cost-effective for the researchers and help them select appropriate treatment mechanisms among all possible options. In this review paper, first, efforts are made to investigate some studies on PD quantitative analysis. Then, PD quantitative models will be reviewed. Finally, the results of using such models are presented to some extent.

Human substantia nigra neurons encode decision outcome and are modulated by categorization uncertainty in an auditory categorization task

The ability to categorize stimuli – predator or prey, friend or foe – is an essential feature of the decision-making process. Underlying that ability is the development of an internally generated category boundary to generate decision outcomes. While classic temporal difference reinforcement models assume midbrain dopaminergic neurons underlie the prediction error required to learn boundary location, these neurons also demonstrate a robust response to nonreward incentive stimuli. More recent models suggest that this may reflect a motivational aspect to performing a task which should be accounted for when modeling dopaminergic neuronal behavior. To clarify the role of substantia nigra dopamine neurons in uncertain perceptual decision making, we investigated their behavior using single neuron extracellular recordings in patients with Parkinson's disease undergoing deep brain stimulation. Subjects underwent a simple auditory categorical decision-making task in which they had to classify a tone as either low- or high-pitched relative to an explicit threshold tone and received feedback but no reward. We demonstrate that the activity of human SN dopaminergic neurons is predictive of perceptual categorical decision outcome and is modulated by uncertainty. Neuronal activity was highest during difficult (uncertain) decisions that resulted in correct responses and lowest during easy decisions that resulted in incorrect responses. This pattern of results is more consistent with a “motivational” role with regards to perceptual categorization and suggests that dopamine neurons are most active when critical information – as represented by uncertainty – is available for learning decision boundaries.

Impulse control disorders in Parkinson's disease: an overview from neurobiology to treatment

Parkinson's disease (PD) is the second most common neurodegenerative brain disorder and is characterized by motor symptoms such as tremor, bradykinesia, rigidity and postural instability. A majority of the patients also develop non-motor symptoms. Impulse control disorders (ICD) are behavioural changes that often fail to be detected in clinical practice. The prevalence of ICD in PD varies widely from 6.1 to 31.2 % and treatment with dopaminergic medication is considered to be the greatest risk factor. Management consists mainly of reducing dopaminergic medication. In our experience, ICD has a tremendous impact on the quality of life of the patients and their families and should therefore not be disregarded. Studies addressing the role of ICD in PD caregiver strain are imperative. We attempt to give a comprehensive overview of the literature on the complicated neurobiology of ICD and discuss risk factors, genetic susceptibility, screening modalities and management.

Computational models of basal-ganglia pathway functions: focus on functional neuroanatomy

Over the past 15 years, computational models have had a considerable impact on basal-ganglia research. Most of these models implement multiple distinct basal-ganglia pathways and assume them to fulfill different functions. As there is now a multitude of different models, it has become complex to keep track of their various, sometimes just marginally different assumptions on pathway functions. Moreover, it has become a challenge to oversee to what extent individual assumptions are corroborated or challenged by empirical data. Focusing on computational, but also considering non-computational models, we review influential concepts of pathway functions and show to what extent they are compatible with or contradict each other. Moreover, we outline how empirical evidence favors or challenges specific model assumptions and propose experiments that allow testing assumptions against each other.

Phasic-like stimulation of the medial forebrain bundle augments striatal gene expression despite methamphetamine-induced partial dopamine denervation

Methamphetamine-induced partial dopamine depletions are associated with impaired basal ganglia function, including decreased preprotachykinin mRNA expression and impaired transcriptional activation of activity-regulated, cytoskeleton-associated (Arc) gene in striatum. Recent work implicates deficits in phasic dopamine signaling as a potential mechanism linking methamphetamine-induced dopamine loss to impaired basal ganglia function. This study thus sought to establish a causal link between phasic dopamine transmission and altered basal ganglia function by determining whether the deficits in striatal neuron gene expression could be restored by increasing phasic dopamine release. Three weeks after pretreatment with saline or a neurotoxic regimen of methamphetamine, rats underwent phasic- or tonic-like stimulation of ascending dopamine neurons. Striatal gene expression was examined using in situ hybridization histochemistry. Phasic-like, but not tonic-like, stimulation induced immediate-early genes Arc and zif268 in both groups, despite the partial striatal dopamine denervation in methamphetamine-pretreated rats, with the Arc expression occurring in presumed striatonigral efferent neurons. Phasic-like stimulation also restored preprotachykinin mRNA expression. These results suggest that disruption of phasic dopamine signaling likely underlies methamphetamine-induced impairments in basal ganglia function, and that restoring phasic dopamine signaling may be a viable approach to manage long-term consequences of methamphetamine-induced dopamine loss on basal ganglia functions.

From occasional choices to inevitable musts: a computational model of nicotine addiction

Although, there are considerable works on the neural mechanisms of reward-based learning and decision making, and most of them mention that addiction can be explained by malfunctioning in these cognitive processes, there are very few computational models. This paper focuses on nicotine addiction, and a computational model for nicotine addiction is proposed based on the neurophysiological basis of addiction. The model compromises different levels ranging from molecular basis to systems level, and it demonstrates three different possible behavioral patterns which are addict, nonaddict, and indecisive. The dynamical behavior of the proposed model is investigated with tools used in analyzing nonlinear dynamical systems, and the relation between the behavioral patterns and the dynamics of the system is discussed.

Dissociating the cognitive effects of levodopa versus dopamine agonists in a neurocomputational model of learning in Parkinson's disease

BACKGROUND/AIMS

Levodopa and dopamine agonists have different effects on the motor, cognitive, and psychiatric aspects of Parkinson's disease (PD).

METHODS

Using a computational model of basal ganglia (BG) and prefrontal cortex (PFC) dopamine, we provide a theoretical synthesis of the dissociable effects of these dopaminergic medications on brain and cognition. Our model incorporates the findings that levodopa is converted by dopamine cells into dopamine, and thus activates prefrontal and striatal D(1) and D(2) dopamine receptors, whereas antiparkinsonian dopamine agonists directly stimulate D(2) receptors in the BG and PFC (although some have weak affinity to D(1) receptors).

RESULTS

In agreement with prior neuropsychological studies, our model explains how levodopa enhances, but dopamine agonists impair or have no effect on, stimulus-response learning and working memory.

CONCLUSION

Our model explains how levodopa and dopamine agonists have differential effects on motor and cognitive processes in PD.

The risky business of dopamine agonists in Parkinson disease and impulse control disorders

Risk-taking behavior is characterized by pursuit of reward in spite of potential negative consequences. Dopamine neurotransmission along the mesocorticolimbic pathway is a potential modulator of risk behavior. In patients with Parkinson's disease (PD), impulse control disorder (ICD) can result from dopaminergic medication use, particularly dopamine agonists (DAA). Behaviors associated with ICD include hypersexuality as well as compulsive gambling, shopping, and eating, and these behaviors are potentially linked to alterations to risk processing. Using the Balloon Analogue Risk Task, we assessed the role of agonist therapy on risk-taking behavior in PD patients with (n = 22) and without (n = 19) active ICD symptoms. Patients performed the task both "on" and "off" DAA. DAA increased risk-taking in PD patients with active ICD symptoms, but it did not affect risk behavior of PD controls. DAA dose was also important in explaining risk behavior. Both groups similarly reduced their risk-taking in high compared to low risk conditions and following the occurrence of a negative consequence, suggesting that ICD patients do not necessarily differ in their abilities to process and adjust to some aspects of negative consequences. Our findings suggest dopaminergic augmentation of risk-taking behavior as a potential contributing mechanism for the emergence of ICD in PD patients.

Levodopa and the feedback process on set-shifting in Parkinson's disease

OBJECTIVE

To study the interaction between levodopa and the feedback process on set-shifting in Parkinson's disease (PD).

METHODS

Functional magnetic resonance imaging (fMRI) studies were performed on 13 PD subjects and 17 age-matched healthy controls while they performed a modified card-sorting task. Experimental time periods were defined based on the types of feedback provided. PD subjects underwent the fMRI experiment twice, once during "off" medication (PDoff) and again after levodopa replacement (PDon).

RESULTS

Compared with normal subjects, the cognitive processing times were prolonged in PDoff but not in PDon subjects during learning through positive outcomes. The ability to set-shift through negative outcomes was not affected in PD subjects, even when "off" medication. Intergroup comparisons showed the lateral prefrontal cortex was deactivated in PDoff subjects during positive feedback learning, especially following internal feedback cues. The cortical activations were increased in the posterior brain regions in PDoff subjects following external feedback learning, especially when negative feedback cues were provided. Levodopa replacement did not completely restore the activation patterns in PD subjects to normal although activations in the corticostriatal loops were restored.

CONCLUSION

PD subjects showed differential ability to set-shift, depending on the dopamine status as well as the types of feedback cues provided. PD subjects had difficulty performing set-shift tasks through positive outcomes when "off" medication, and showed improvement after levodopa replacement. The ability to set-shift through negative feedback was not affected in PD subjects even when "off" medication, possibly due to compensatory changes outside the nigrostriatal dopaminergic pathway.

Computational cognitive models of prefrontal-striatal-hippocampal interactions in Parkinson's disease and schizophrenia

Disruption to different components of the prefrontal cortex, basal ganglia, and hippocampal circuits leads to various psychiatric and neurological disorders including Parkinson's disease (PD) and schizophrenia. Medications used to treat these disorders (such as levodopa, dopamine agonists, antipsychotics, among others) affect the prefrontal-striatal-hippocampal circuits in a complex fashion. We have built models of prefrontal-striatal and striatal-hippocampal interactions which simulate cognitive dysfunction in PD and schizophrenia. In these models, we argue that the basal ganglia is key for stimulus-response learning, the hippocampus for stimulus-stimulus representational learning, and the prefrontal cortex for stimulus selection during learning about multidimensional stimuli. In our models, PD is associated with reduced dopamine levels in the basal ganglia and prefrontal cortex. In contrast, the cognitive deficits in schizophrenia are associated primarily with hippocampal dysfunction, while the occurrence of negative symptoms is associated with frontostriatal deficits in a subset of patients. In this paper, we review our past models and provide new simulation results for both PD and schizophrenia. We also describe an extended model that includes simulation of the different functional role of D1 and D2 dopamine receptors in the basal ganglia and prefrontal cortex, a dissociation we argue is essential for understanding the non-uniform effects of levodopa, dopamine agonists, and antipsychotics on cognition. Motivated by clinical and physiological data, we discuss model limitations and challenges to be addressed in future models of these brain disorders.

A neurocomputational model of dopamine and prefrontal-striatal interactions during multicue category learning by Parkinson patients

Most existing models of dopamine and learning in Parkinson disease (PD) focus on simulating the role of basal ganglia dopamine in reinforcement learning. Much data argue, however, for a critical role for prefrontal cortex (PFC) dopamine in stimulus selection in attentional learning. Here, we present a new computational model that simulates performance in multicue category learning, such as the "weather prediction" task. The model addresses how PD and dopamine medications affect stimulus selection processes, which mediate reinforcement learning. In this model, PFC dopamine is key for attentional learning, whereas basal ganglia dopamine, consistent with other models, is key for reinforcement and motor learning. The model assumes that competitive dynamics among PFC neurons is the neural mechanism underlying stimulus selection with limited attentional resources, whereas competitive dynamics among striatal neurons is the neural mechanism underlying action selection. According to our model, PD is associated with decreased phasic and tonic dopamine levels in both PFC and basal ganglia. We assume that dopamine medications increase dopamine levels in both the basal ganglia and PFC, which, in turn, increase tonic dopamine levels but decrease the magnitude of phasic dopamine signaling in these brain structures. Increase of tonic dopamine levels in the simulated PFC enhances attentional shifting performance. The model provides a mechanistic account for several phenomena, including (a) medicated PD patients are more impaired at multicue probabilistic category learning than unmedicated patients and (b) medicated PD patients opt out of reversal when there are alternative and redundant cue dimensions.

A neural model of hippocampal-striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients

Building on our previous neurocomputational models of basal ganglia and hippocampal region function (and their modulation by dopamine and acetylcholine, respectively), we show here how an integration of these models can inform our understanding of the interaction between the basal ganglia and hippocampal region in associative learning and transfer generalization across various patient populations. As a common test bed for exploring interactions between these brain regions and neuromodulators, we focus on the acquired equivalence task, an associative learning paradigm in which stimuli that have been associated with the same outcome acquire a functional similarity such that subsequent generalization between these stimuli increases. This task has been used to test cognitive dysfunction in various patient populations with damages to the hippocampal region and basal ganglia, including studies of patients with Parkinson's disease (PD), schizophrenia, basal forebrain amnesia, and hippocampal atrophy. Simulation results show that damage to the hippocampal region-as in patients with hippocampal atrophy (HA), hypoxia, mild Alzheimer's (AD), or schizophrenia-leads to intact associative learning but impaired transfer generalization performance. Moreover, the model demonstrates how PD and anterior communicating artery (ACoA) aneurysm-two very different brain disorders that affect different neural mechanisms-can have similar effects on acquired equivalence performance. In particular, the model shows that simulating a loss of dopamine function in the basal ganglia module (as in PD) leads to slow acquisition learning but intact transfer generalization. Similarly, the model shows that simulating the loss of acetylcholine in the hippocampal region (as in ACoA aneurysm) also results in slower acquisition learning. We argue from this that changes in associative learning of stimulus-action pathways (in the basal ganglia) or changes in the learning of stimulus representations (in the hippocampal region) can have similar functional effects.

Neurocomputational models of motor and cognitive deficits in Parkinson's disease

We review the contributions of biologically constrained computational models to our understanding of motor and cognitive deficits in Parkinson's disease (PD). The loss of dopaminergic neurons innervating the striatum in PD, and the well-established role of dopamine (DA) in reinforcement learning (RL), enable neural network models of the basal ganglia (BG) to derive concrete and testable predictions. We focus in this review on one simple underlying principle - the notion that reduced DA increases activity and causes long-term potentiation in the indirect pathway of the BG. We show how this theory can provide a unified account of diverse and seemingly unrelated phenomena in PD including progressive motor degeneration as well as cognitive deficits in RL, decision making and working memory. DA replacement therapy and deep brain stimulation can alleviate some aspects of these impairments, but can actually introduce negative effects such as motor dyskinesias and cognitive impulsivity. We discuss these treatment effects in terms of modulation of specific mechanisms within the computational framework. In addition, we review neurocomputational interpretations of increased impulsivity in the face of response conflict in patients with deep-brain-stimulation.

Initial clinical manifestations of Parkinson's disease: features and pathophysiological mechanisms

A dopaminergic deficiency in patients with Parkinson's disease (PD) causes abnormalities of movement, behaviour, learning, and emotions. The main motor features (ie, tremor, rigidity, and akinesia) are associated with a deficiency of dopamine in the posterior putamen and the motor circuit. Hypokinesia and bradykinesia might have a dual anatomo-functional basis: hypokinesia mediated by brainstem mechanisms and bradykinesia by cortical mechanisms. The classic pathophysiological model for PD (ie, hyperactivity in the globus pallidus pars interna and substantia nigra pars reticulata) does not explain rigidity and tremor, which might be caused by changes in primary motor cortex activity. Executive functions (ie, planning and problem solving) are also impaired in early PD, but are usually not clinically noticed. These impairments are associated with dopamine deficiency in the caudate nucleus and with dysfunction of the associative and other non-motor circuits. Apathy, anxiety, and depression are the main psychiatric manifestations in untreated PD, which might be caused by ventral striatum dopaminergic deficit and depletion of serotonin and norepinephrine. In this Review we discuss the motor, cognitive, and psychiatric manifestations associated with the dopaminergic deficiency in the early phase of the parkinsonian state and the different circuits implicated, and we propose distinct mechanisms to explain the wide clinical range of PD symptoms at the time of diagnosis.

Selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in Parkinson's disease patients: evidence from a model-based fMRI study

Animal studies have found that the phasic activity of dopamine neurons during reward-related learning resembles a "prediction error" (PE) signal derived from a class of computational models called reinforcement learning (RL). An apparently similar signal can be measured using fMRI in the human striatum, a primary dopaminergic target. However, the fMRI signal does not measure dopamine per se, and therefore further evidence is needed to determine if these signals are related to each other. Parkinson's disease (PD) involves the neurodegeneration of the dopamine system and is accompanied by deficits in reward-related decision-making tasks. In the current study we used a computational RL model to assess striatal error signals in PD patients performing an RL task during fMRI scanning. Results show that error signals were preserved in ventral striatum of PD patients, but impaired in dorsolateral striatum, relative to healthy controls, a pattern reflecting the known selective anatomical degeneration of dopamine nuclei in PD. These findings support the notion that PE signals measured in the human striatum by the BOLD signal may reflect phasic DA activity. These results also provide evidence for a deficiency in PE signaling in the dorsolateral striatum of PD patients that may offer an explanation for their deficits observed in other reward learning tasks.