How the Basal Ganglia Make Decisions. In: Damasio AR, Damasio H, Christen Y, editors. Neurobiology of Decision-Making. Berlin: Springer Berlin Heidelberg; 1996. pp. 101-13. [DOI: 10.1007/978-3-642-79928-0_6]

Distinct cortico-striatal compartments drive competition between adaptive and automatized behavior

Cortical and basal ganglia circuits play a crucial role in the formation of goal-directed and habitual behaviors. In this study, we investigate the cortico-striatal circuitry involved in learning and the role of this circuitry in the emergence of inflexible behaviors such as those observed in addiction. Specifically, we develop a computational model of cortico-striatal interactions that performs concurrent goal-directed and habit learning. The model accomplishes this by distinguishing learning processes in the dorsomedial striatum (DMS) that rely on reward prediction error signals as distinct from the dorsolateral striatum (DLS) where learning is supported by salience signals. These striatal subregions each operate on unique cortical input: the DMS receives input from the prefrontal cortex (PFC) which represents outcomes, and the DLS receives input from the premotor cortex which determines action selection. Following an initial learning of a two-alternative forced choice task, we subjected the model to reversal learning, reward devaluation, and learning a punished outcome. Behavior driven by stimulus-response associations in the DLS resisted goal-directed learning of new reward feedback rules despite devaluation or punishment, indicating the expression of habit. We repeated these simulations after the impairment of executive control, which was implemented as poor outcome representation in the PFC. The degraded executive control reduced the efficacy of goal-directed learning, and stimulus-response associations in the DLS were even more resistant to the learning of new reward feedback rules. In summary, this model describes how circuits of the dorsal striatum are dynamically engaged to control behavior and how the impairment of executive control by the PFC enhances inflexible behavior.

A biologically constrained spiking neural network model of the primate basal ganglia with overlapping pathways exhibits action selection

Action selection has been hypothesized to be a key function of the basal ganglia, yet the nuclei involved, their interactions and the importance of the direct/indirect pathway segregation in such process remain debated. Here, we design a spiking computational model of the monkey basal ganglia derived from a previously published population model, initially parameterized to reproduce electrophysiological activity at rest and to embody as much quantitative anatomical data as possible. As a particular feature, both models exhibit the strong overlap between the direct and indirect pathways that has been documented in non-human primates. Here, we first show how the translation from a population to an individual neuron model was achieved, with the addition of a minimal number of parameters. We then show that our model performs action selection, even though it was built without any assumption on the activity carried out during behaviour. We investigate the mechanisms of this selection through circuit disruptions and found an instrumental role of the off-centre/on-surround structure of the MSN-STN-GPi circuit, as well as of the MSN-MSN and FSI-MSN projections. This validates their potency in enabling selection. We finally study the pervasive centromedian and parafascicular thalamic inputs that reach all basal ganglia nuclei and whose influence is therefore difficult to anticipate. Our model predicts that these inputs modulate the responsiveness of action selection, making them a candidate for the regulation of the speed-accuracy trade-off during decision-making.

The caudate nucleus contributes causally to decisions that balance reward and uncertain visual information

Our decisions often balance what we observe and what we desire. A prime candidate for implementing this complex balancing act is the basal ganglia pathway, but its roles have not yet been examined experimentally in detail. Here, we show that a major input station of the basal ganglia, the caudate nucleus, plays a causal role in integrating uncertain visual evidence and reward context to guide adaptive decision-making. In monkeys making saccadic decisions based on motion cues and asymmetric reward-choice associations, single caudate neurons encoded both sources of information. Electrical microstimulation at caudate sites during motion viewing affected the monkeys’ decisions. These microstimulation effects included coordinated changes in multiple computational components of the decision process that mimicked the monkeys’ similarly coordinated voluntary strategies for balancing visual and reward information. These results imply that the caudate nucleus plays causal roles in coordinating decision processes that balance external evidence and internal preferences.

Design of robust adaptive controller and feedback error learning for rehabilitation in Parkinson's disease: a simulation study

Deep brain stimulation (DBS) is an efficient therapy to control movement disorders of Parkinson's tremor. Stimulation of one area of basal ganglia (BG) by DBS with no feedback is the prevalent opinion. Reduction of additional stimulatory signal delivered to the brain is the advantage of using feedback. This results in reduction of side effects caused by the excessive stimulation intensity. In fact, the stimulatory intensity of controllers is decreased proportional to reduction of hand tremor. The objective of this study is to design a new controller structure to decrease three indicators: (i) the hand tremor; (ii) the level of delivered stimulation in disease condition; and (iii) the ratio of the level of delivered stimulation in health condition to disease condition. For this purpose, the authors offer a new closed-loop control structure to stimulate two areas of BG simultaneously. One area (STN: subthalamic nucleus) is stimulated by an adaptive controller with feedback error learning. The other area (GPi: globus pallidus internal) is stimulated by a partial state feedback (PSF) controller. Considering the three indicators, the results show that, stimulating two areas simultaneously leads to better performance compared with stimulating one area only. It is shown that both PSF and adaptive controllers are robust regarding system parameter uncertainties. In addition, a method is proposed to update the parameters of the BG model in real time. As a result, the parameters of the controllers can be updated based on the new parameters of the BG model.

Affine differential geometry and smoothness maximization as tools for identifying geometric movement primitives

Neuroscientific studies of drawing-like movements usually analyze neural representation of either geometric (e.g., direction, shape) or temporal (e.g., speed) parameters of trajectories rather than trajectory's representation as a whole. This work is about identifying geometric building blocks of movements by unifying different empirically supported mathematical descriptions that characterize relationship between geometric and temporal aspects of biological motion. Movement primitives supposedly facilitate the efficiency of movements' representation in the brain and comply with such criteria for biological movements as kinematic smoothness and geometric constraint. The minimum-jerk model formalizes criterion for trajectories' maximal smoothness of order 3. I derive a class of differential equations obeyed by movement paths whose nth-order maximally smooth trajectories accumulate path measurement with constant rate. Constant rate of accumulating equi-affine arc complies with the 2/3 power-law model. Candidate primitive shapes identified as equations' solutions for arcs in different geometries in plane and in space are presented. Connection between geometric invariance, motion smoothness, compositionality and performance of the compromised motor control system is proposed within single invariance-smoothness framework. The derived class of differential equations is a novel tool for discovering candidates for geometric movement primitives.

Designing a robust backstepping controller for rehabilitation in Parkinson's disease: a simulation study

In this study, a model of basal ganglia (BG) is applied to develop a deep brain stimulation controller to reduce Parkinson's tremor. Conventionally, one area in BG is stimulated, with no feedback, to control Parkinson's tremor. In this study, a new architecture is proposed to develop feedback controller as well as to stimulate two areas of BG simultaneously. To this end, two controllers are designed and implemented in globus pallidus internal (GPi) and subthalamic nucleus (STN) in the brain. A proportional controller and a backstepping controller are designed and implemented in GPi and STN, respectively. The proposed controllers deliver suitable stimulatory control signals to GPi and STN based on hand tremor amplitude (as the feedback). When tremor reduces, these controllers decrease the stimulatory energy intensity proportionally. Therefore, additional stimulatory signal is not delivered to the brain. Subsequently, the side effects from the excessive stimulation intensity become much less. Comparing with one area stimulation, the results show that stimulating two areas of BG results in reduction of the level of the stimulation intensity. It is observed that these two controllers are both robust in terms of changing the system parameters.

Acquired equivalence associative learning in GTC epileptic patients: experimental and computational study

Previous cognitive behavioral studies based on Acquired Equivalence Associative learning Task (AEALT) showed a strong relation between hippocampus and basal ganglia in associative learning. However, experimental behavioral studies of patients with Generalized Tonic Clonic (GTC) epilepsy remained sparse. The aim of the present study is to integrate a classical behavioral cognitive analysis with a computational model approach to investigate cognitive associative learning impairments in patients with GTC epilepsy. We measured the accuracy of associative learning response performance in five GTC epileptic patients and five control subjects by using AEALT, all subjects were matched in age and gender. We ran the task using E-Prime, a neuropsychological software program, and SPSS for data statistical analysis. We tested whether GTC epileptic patients would have different learning performance than normal subjects, based on the degree and the location of impairment either in basal ganglia and/or hippocampus. With the number of patients that was available, our behavioral analysis showed no remarkable differences in learning performance of GTC patients as compared to their control subjects, both in the transfer and acquisition phases. In parallel, our simulation results confirmed strong connection and interaction between hippocampus and basal ganglia in our GTC and their control subjects. Nevertheless, the differences in neural firing rate of the connectionist model and weight update of basal ganglia were not significantly different between GTC and control subjects. Therefore, the behavioral analysis and the simulation data provided the same result, thus indicating that the computational model is likely to predict cognitive outcomes.

Response selection involves executive control: Evidence from the selective interference paradigm

In the present study, we investigated whether response selection involves executive control, using the selective interference paradigm within Baddeley's (1986) working memory framework. The interference from response selection was estimated by comparing the patterns of dual-task interference of simple and choice RT tasks with those of a number of established working memory tasks. In Experiment 1, we compared impairment of forward and backward verbal serial recall from the RT tasks and articulatory suppression. Experiment 2 measured the adverse effects of the RT tasks and matrix tapping on forward and backward visuospatial serial recall. Finally, in Experiment 3, we examined the impairment from the RT tasks with two measures of executive control--namely, letter and category fluency. Altogether, the three experiments demonstrated that response selection interferes with executive control and that the interference is not produced at the level of working memory's slave systems, which supports the assumption of executive involvement in response selection.

The neural representation of time

This review summarizes recent investigations of temporal processing. We focus on motor and perceptual tasks in which crucial events span hundreds of milliseconds. One key question concerns whether the representation of temporal information is dependent on a specialized system, distributed across a network of neural regions, or computed in a local task-dependent manner. Consistent with the specialized system framework, the cerebellum is associated with various tasks that require precise timing. Computational models of timing mechanisms within the cerebellar cortex are beginning to motivate physiological studies. Emphasis has also been placed on the basal ganglia as a specialized timing system, particularly for longer intervals. We outline an alternative hypothesis in which this structure is associated with decision processes.

Disruption of self-organized actions in monkeys with progressive MPTP-induced parkinsonism. I. Effects of task complexity

Parkinson's disease (PD) is characterized by motor symptoms, usually accompanied by cognitive deficits. The question addressed in this study is whether complexity of routine actions can exacerbate parkinsonian disorders that are often considered to be motor symptoms. To examine this question, we trained four vervet monkeys (Cercopithecus aethiops) to perform three multiple-choice retrieval tasks. In order of ascending complexity, rewards were freely available (task 1), covered with transparent sliding plaques (task 2), and covered with opaque sliding plaques cued by symbols (task 3). Thus, from task 1 to task 2 we added a motor difficulty--the recall of context-adapted movement; and from task 2 to task 3 we added a cognitive difficulty: the recall of symbol-reward associations. The more complex the task, the longer it took to learn, but after extensive training the performance was stable in all tasks, with similar retrieval durations. The monkeys then received systemic 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP) injections (0.3-0.4 mg/kg) every 4-7 days, until the first motor symptoms appeared. In the course of MPTP intoxication, the behavioural performance declined while the motor symptoms were absent or mild--the retrieval duration increased, and non-initiated choices and hesitations between choices became frequent. Interestingly, this decline was in proportion to task complexity, and was particularly pronounced with the cognitive difficulty. Furthermore, freezing appeared only with the cognitive difficulty. We therefore suggest that everyday cognitive difficulties may exacerbate hypokinesia (lack of initiation, abnormal slowness) and executive disorders (hesitations, freezing) in the early stages of human PD.

Information processing, dimensionality reduction and reinforcement learning in the basal ganglia

Modeling of the basal ganglia has played a major role in our understanding of this elusive group of nuclei. Models of the basal ganglia have undergone evolutionary and revolutionary changes over the last 20 years, as new research in the fields of anatomy, physiology and biochemistry of these nuclei has yielded new information. Early models dealt with a single pathway through the nuclei and focused on the nature of the processing performed within it, convergence of information versus parallel processing of information. Later, the Albin-DeLong "box-and-arrow" model characterized the inter-nuclei interaction as multiple pathways while maintaining a simplistic scalar representation of the nuclei themselves. This model made a breakthrough by providing key insights into the behavior of these nuclei in hypo- and hyper-kinetic movement disorders. The next generation of models elaborated the intra-nuclei interactions and focused on the role of the basal ganglia in action selection and sequence generation which form the most current consensus regarding basal ganglia function in both normal and pathological conditions. However, new findings challenge these models and point to a different neural network approach to information processing in the basal ganglia. Here, we take an in-depth look at the reinforcement driven dimensionality reduction (RDDR) model which postulates that the basal ganglia compress cortical information according to a reinforcement signal using optimal extraction methods. The model provides new insights and experimental predictions on the computational capacity of the basal ganglia and their role in health and disease.

Impairment of context-adapted movement selection in a primate model of presymptomatic Parkinson's disease

The MPTP model allows the presymptomatic stage of parkinsonism to be studied in primates and hence specific behavioural manifestations of moderate nigrostriatal denervation to be identified. On the basis of the physiological literature, we hypothesized that depletion of striatal dopamine could impair the selection of context-relevant habits. To examine this hypothesis, we trained three African green monkeys to perform a simple reach-and-grasp task, including three contexts differing only in terms of the presence and position of transparent obstacles. At the end of training, the analysis of reaching trajectories showed that intact monkeys had built a repertoire of movements, from which they could select the appropriate one depending on the context. In the course of MPTP intoxication (0.3-0.4 mg/kg every 4-5 days) and before parkinsonian motor symptoms appeared, the reaction time (RT), movement time (MT) and variability of reaching trajectories increased in all monkeys. Frequently, the initial direction was not adapted to the context, and consequently the movement was either corrected online or restarted under visual assistance. These non-adapted trajectories appeared to be the main reason for the increase in both RT (because of difficulty in selecting) and MT (because of the need to make corrections). These observations indicate that moderate MPTP-induced dopamine depletion results in a deficit in the selection of context-adapted movement, which is compensated by corrections using either proprioceptive or visual feedback. Similar behavioural disorders might therefore occur in the presymptomatic stage of human Parkinson's disease.

Modeling functions of striatal dopamine modulation in learning and planning

The activity of midbrain dopamine neurons is strikingly similar to the reward prediction error of temporal difference reinforcement learning models. Experimental evidence and simulation studies suggest that dopamine neuron activity serves as an effective reinforcement signal for learning of sensorimotor associations in striatal matrisomes. In the current study, we simulate dopamine neuron activity with the extended temporal difference model of Pavlovian learning and examine the influences of this signal on medium spiny neurons in striatal matrisomes. The modeled influences include transient membrane effects of dopamine D(1) receptor activation, dopamine-dependent long-term adaptations of corticostriatal transmission, and effects of dopamine on rhythmic fluctuations of the membrane potential between an elevated "up-state" and a hyperpolarized "down-state". The most dominant activity in the striatal matrisomes is assumed to elicit behaviors via projections from the basal ganglia to the thalamus and the cortex. This "standard model" performs successfully when tested for sensorimotor learning and goal-directed behavior (planning). To investigate the contributions of our model assumptions to learning and planning, we test the performance of several model variants that lack one of these mechanisms. These simulations show that the adaptation of the dopamine-like signal is necessary for sensorimotor learning and planning. Sensorimotor learning requires dopamine-dependent long-term adaptation of corticostriatal transmission. Lack of dopamine-like novelty responses decreases the number of exploratory acts, which impairs planning capabilities. The model loses its planning capabilities if the dopamine-like signal is simulated with the original temporal difference model, because the original temporal difference model does not form novel associative chains. Transient membrane effects of the dopamine-like signal on striatal firing substantially shorten the reaction time in the planning task. The capability for planning is improved by influences of dopamine on the durations of membrane potential fluctuations and by manipulations that prolong the reaction time of the model. These results suggest that responses of dopamine neurons to conditioned stimuli contribute to sensorimotor reward learning, novelty responses of dopamine neurons stimulate exploration, and transient dopamine membrane effects are important for planning.

The effect of dopamine depletion from the caudate nucleus of the common marmoset (Callithrix jacchus) on tests of prefrontal cognitive function

This study examined the effects of depletion of dopamine from the caudate nucleus of the common marmoset (Callithrix jacchus), on tasks sensitive to prefrontal damage (attentional set-shifting and spatial delayed response). There was a marked impairment in performance on the spatial delayed response task, but performance on the attentional set-shifting task was relatively preserved except for an impairment in re-engagement of a previously relevant perceptual dimension. This pattern of impairment is distinct from that seen after excitotoxic lesions of the prefrontal cortex and in patients with Parkinson's disease. Though it is not possible to identify specific cognitive functions that are independent of dopaminergic modulation of the caudate nucleus, due to the partial nature of the lesion, the results do provide insight into those cognitive processes that appear most dependent on caudate dopamine.

The adaptive nature of the human neurocognitive architecture: an alternative model

The model of the human neurocognitive architecture proposed by evolutionary psychologists is based on the presumption that the demands of hunter-gatherer life generated a vast array of cognitive adaptations. Here we present an alternative model. We argue that the problems inherent in the biological markets of ancestral hominids and their mammalian predecessors would have required an adaptively flexible, on-line information-processing system, and would have driven the evolution of a functionally plastic neural substrate, the neocortex, rather than a confederation of evolutionarily prespecified social cognitive adaptations. In alignment with recent neuroscientific evidence, we suggest that human cognitive processes result from the activation of constructed cortical representational networks, which reflect probabilistic relationships between sensory inputs, behavioral responses, and adaptive outcomes. The developmental construction and experiential modification of these networks are mediated by subcortical circuitries that are responsive to the life history regulatory system. As a consequence, these networks are intrinsically adaptively constrained. The theoretical and research implications of this alternative evolutionary model are discussed.

A neural substrate of prediction and reward

The capacity to predict future events permits a creature to detect, model, and manipulate the causal structure of its interactions with its environment. Behavioral experiments suggest that learning is driven by changes in the expectations about future salient events such as rewards and punishments. Physiological work has recently complemented these studies by identifying dopaminergic neurons in the primate whose fluctuating output apparently signals changes or errors in the predictions of future salient and rewarding events. Taken together, these findings can be understood through quantitative theories of adaptive optimizing control.