Striatal mechanisms in Parkinson's disease: new insights from computer modeling

Decision making under uncertainty in a spiking neural network model of the basal ganglia

The mechanisms of decision-making and action selection are generally thought to be under the control of parallel cortico-subcortical loops connecting back to distinct areas of cortex through the basal ganglia and processing motor, cognitive and limbic modalities of decision-making. We have used these properties to develop and extend a connectionist model at a spiking neuron level based on a previous rate model approach. This model is demonstrated on decision-making tasks that have been studied in primates and the electrophysiology interpreted to show that the decision is made in two steps. To model this, we have used two parallel loops, each of which performs decision-making based on interactions between positive and negative feedback pathways. This model is able to perform two-level decision-making as in primates. We show here that, before learning, synaptic noise is sufficient to drive the decision-making process and that, after learning, the decision is based on the choice that has proven most likely to be rewarded. The model is then submitted to lesion tests, reversal learning and extinction protocols. We show that, under these conditions, it behaves in a consistent manner and provides predictions in accordance with observed experimental data.

A network model of basal ganglia for understanding the roles of dopamine and serotonin in reward-punishment-risk based decision making

There is significant evidence that in addition to reward-punishment based decision making, the Basal Ganglia (BG) contributes to risk-based decision making (Balasubramani et al., 2014). Despite this evidence, little is known about the computational principles and neural correlates of risk computation in this subcortical system. We have previously proposed a reinforcement learning (RL)-based model of the BG that simulates the interactions between dopamine (DA) and serotonin (5HT) in a diverse set of experimental studies including reward, punishment and risk based decision making (Balasubramani et al., 2014). Starting with the classical idea that the activity of mesencephalic DA represents reward prediction error, the model posits that serotoninergic activity in the striatum controls risk-prediction error. Our prior model of the BG was an abstract model that did not incorporate anatomical and cellular-level data. In this work, we expand the earlier model into a detailed network model of the BG and demonstrate the joint contributions of DA-5HT in risk and reward-punishment sensitivity. At the core of the proposed network model is the following insight regarding cellular correlates of value and risk computation. Just as DA D1 receptor (D1R) expressing medium spiny neurons (MSNs) of the striatum were thought to be the neural substrates for value computation, we propose that DA D1R and D2R co-expressing MSNs are capable of computing risk. Though the existence of MSNs that co-express D1R and D2R are reported by various experimental studies, prior existing computational models did not include them. Ours is the first model that accounts for the computational possibilities of these co-expressing D1R-D2R MSNs, and describes how DA and 5HT mediate activity in these classes of neurons (D1R-, D2R-, D1R-D2R- MSNs). Starting from the assumption that 5HT modulates all MSNs, our study predicts significant modulatory effects of 5HT on D2R and co-expressing D1R-D2R MSNs which in turn explains the multifarious functions of 5HT in the BG. The experiments simulated in the present study relates 5HT to risk sensitivity and reward-punishment learning. Furthermore, our model is shown to capture reward-punishment and risk based decision making impairment in Parkinson's Disease (PD). The model predicts that optimizing 5HT levels along with DA medications might be essential for improving the patients' reward-punishment learning deficits.

An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning

Although empirical and neural studies show that serotonin (5HT) plays many functional roles in the brain, prior computational models mostly focus on its role in behavioral inhibition. In this study, we present a model of risk based decision making in a modified Reinforcement Learning (RL)-framework. The model depicts the roles of dopamine (DA) and serotonin (5HT) in Basal Ganglia (BG). In this model, the DA signal is represented by the temporal difference error (δ), while the 5HT signal is represented by a parameter (α) that controls risk prediction error. This formulation that accommodates both 5HT and DA reconciles some of the diverse roles of 5HT particularly in connection with the BG system. We apply the model to different experimental paradigms used to study the role of 5HT: (1) Risk-sensitive decision making, where 5HT controls risk assessment, (2) Temporal reward prediction, where 5HT controls time-scale of reward prediction, and (3) Reward/Punishment sensitivity, in which the punishment prediction error depends on 5HT levels. Thus the proposed integrated RL model reconciles several existing theories of 5HT and DA in the BG.

Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study

In a previous modeling study, Leblois et al. (2006) demonstrated an action selection mechanism in cortico-basal ganglia loops based on competition between the positive feedback, direct pathway through the striatum and the negative feedback, hyperdirect pathway through the subthalamic nucleus. The present study investigates how multiple level action selection could be performed by the basal ganglia. To do this, the model is extended in a manner consistent with known anatomy and electrophysiology in three main areas. First, two-level decision making has been incorporated, with a cognitive level selecting based on cue shape and a motor level selecting based on cue position. We show that the decision made at the cognitive level can be used to bias the decision at the motor level. We then demonstrate that, for accurate transmission of information between decision-making levels, low excitability of striatal projection neurons is necessary, a generally observed electrophysiological finding. Second, instead of providing a biasing signal between cue choices as an external input to the network, we show that the action selection process can be driven by reasonable levels of noise. Finally, we incorporate dopamine modulated learning at corticostriatal synapses. As learning progresses, the action selection becomes based on learned visual cue values and is not interfered with by the noise that was necessary before learning.

A neurocomputational model of tonic and phasic dopamine in action selection: a comparison with cognitive deficits in Parkinson's disease

The striatal dopamine signal has multiple facets; tonic level, phasic rise and fall, and variation of the phasic rise/fall depending on the expectation of reward/punishment. We have developed a network model of the striatal direct pathway using an ionic current level model of the medium spiny neuron that incorporates currents sensitive to changes in the tonic level of dopamine. The model neurons in the network learn action selection based on a novel set of mathematical rules that incorporate the phasic change in the dopamine signal. This network model is capable of learning to perform a sequence learning task that in humans is thought to be dependent on the basal ganglia. When both tonic and phasic levels of dopamine are decreased, as would be expected in unmedicated Parkinson's disease (PD), the model reproduces the deficits seen in a human PD group off medication. When the tonic level is increased to normal, but with reduced phasic increases and decreases in response to reward and punishment, respectively, as would be expected in PD medicated with L-Dopa, the model again reproduces the human data. These findings support the view that the cognitive dysfunctions seen in Parkinson's disease are not solely either due to the decreased tonic level of dopamine or to the decreased responsiveness of the phasic dopamine signal to reward and punishment, but to a combination of the two factors that varies dependent on disease stage and medication status.

A scale-free systems theory of motivation and addiction

Scale-free organizations, characterized by uneven distributions of linkages between nodal elements, describe the structure and function of many life-based complex systems developing under evolutionary pressures. We explore motivated behavior as a scale-free map toward a comprehensive translational theory of addiction. Motivational and behavioral repertoires are reframed as link and nodal element sets, respectively, comprising a scale-free structure. These sets are generated by semi-independent information-processing streams within cortical-striatal circuits that cooperatively provide decision-making and sequential processing functions necessary for traversing maps of motivational links connecting behavioral nodes. Dopamine modulation of cortical-striatal plasticity serves a central-hierarchical mechanism for survival-adaptive sculpting and development of motivational-behavioral repertoires by guiding a scale-free design. Drug-induced dopamine activity promotes drug taking as a highly connected behavioral hub at the expense of natural-adaptive motivational links and behavioral nodes. Conceptualizing addiction as pathological alteration of scale-free motivational-behavioral repertoires unifies neurobiological, neurocomputational and behavioral research while addressing addiction vulnerability in adolescence and psychiatric illness. This model may inform integrative research in defining more effective prevention and treatment strategies for addiction.

Computational models simulating electrophysiological activity in the basal ganglia

Modeling of the basal ganglia has played a substantial role in gaining insight into the mechanisms involved in the computational processes performed by this elusive group of nuclei. Models of the basal ganglia have undergone revolutionary changes over the last twenty years due to the rapid accumulation of neuroscientific data. In this chapter, we present distinct modeling approaches that can be used to enhance our understanding of the functional dynamics of information processing within the basal ganglia, and their interactions with the rest of the brain. Specific examples of recently developed models dealing with the analysis of computational processing issues at different structural levels of the basal ganglia are discussed.

Simulation of GABA function in the basal ganglia: computational models of GABAergic mechanisms in basal ganglia function

This chapter outlines current interpretation of computational aspects of GABAergic circuits of the striatum. Recent hypotheses and controversial matters are reviewed. Quantitative aspects of striatal synaptology relevant to computational models are considered, with estimates of the connectivity of the spiny projection neurons and fast-spiking interneurons. Against this background, insights into the computational properties of inhibitory circuits based on analysis and simulation of simple models are discussed. The paper concludes with suggestions for further theoretical and experimental studies.

Neuroengineering models of brain disease

The techniques of computational simulation have begun to be applied to modeling neurological disease and mental illness. Such neuroengineering models provide a conceptual bridge between molecular/cellular pathology and cognitive performance. We consider models of Alzheimer's disease, Parkinson's disease, and schizophrenia. Each of these diseases involves a disorder of neuromodulation coupled with underlying neuronal pathology. Parallels arising between these models suggests that a common set of computational mechanisms may account for functional loss across a spectrum of brain diseases. In particular, we focus on attractor-based network dynamics and how they arise from neural architectures, on mechanisms for linking sequences of attractor states and their role in cognition, and on the role of neuromodulation in controlling these processes. These studies suggest new approaches to understanding the forebrain circuits underlying cognition, and point toward a new tool for dissecting the pathophysiology of brain disease.