Basal ganglia neurons dynamically facilitate exploration during associative learning

Pallidal neuromodulation of the explore/exploit trade-off in decision-making

Every decision that we make involves a conflict between exploiting our current knowledge of an action's value or exploring alternative courses of action that might lead to a better, or worse outcome. The sub-cortical nuclei that make up the basal ganglia have been proposed as a neural circuit that may contribute to resolving this explore-exploit 'dilemma'. To test this hypothesis, we examined the effects of neuromodulating the basal ganglia's output nucleus, the globus pallidus interna, in patients who had undergone deep brain stimulation (DBS) for isolated dystonia. Neuromodulation enhanced the number of exploratory choices to the lower value option in a two-armed bandit probabilistic reversal-learning task. Enhanced exploration was explained by a reduction in the rate of evidence accumulation (drift rate) in a reinforcement learning drift diffusion model. We estimated the functional connectivity profile between the stimulating DBS electrode and the rest of the brain using a normative functional connectome derived from heathy controls. Variation in the extent of neuromodulation induced exploration between patients was associated with functional connectivity from the stimulation electrode site to a distributed brain functional network. We conclude that the basal ganglia's output nucleus, the globus pallidus interna, can adaptively modify decision choice when faced with the dilemma to explore or exploit.

Effects of low-dose methylcyclopentadienyl manganese tricarbonyl-derived manganese on the development of diencephalic dopaminergic neurons in zebrafish

Fuel additive methylcyclopentadienyl manganese tricarbonyl (MMT) is counted as an organic manganese (Mn)-derived compound. The toxic effects of Mn (alone and complexed) on dopaminergic (DA) neurotransmission have been investigated in both cellular and animal models. However, the impact of environmentally relevant Mn exposure on DA neurodevelopment is rather poorly understood. In the present study, the MMT dose of 100 μM (about 5 mg Mn/L) caused up-regulation of DA-related genes in association with cell body swelling and increase in the number of DA neurons of the ventral diencephalon subpopulation DC2. Furthermore, our analysis identified significant brain Mn bioaccumulation and enhancement of total dopamine levels in association with locomotor hyperactivity. Although DA levels were restored at adulthood, we observed a deficit in the acquisition and consolidation of memory. Collectively, these findings suggest that developmental exposure to low-level MMT-derived Mn is responsible for the selective alteration of diencephalic DA neurons and with long-lasting effects on fish explorative behaviour in adulthood.

Tonic dopamine, uncertainty and basal ganglia action selection

To make optimal decisions in uncertain circumstances flexible adaption of behaviour is required; exploring alternatives when the best choice is unknown, exploiting what is known when that is best. Using a computational model of the basal ganglia, we propose that switches between exploratory and exploitative decisions are mediated by the interaction between tonic dopamine and cortical input to the basal ganglia. We show that a biologically detailed action selection circuit model, endowed with dopamine dependant striatal plasticity, can optimally solve the explore-exploit problem, estimating the true underlying state of a noisy Gaussian diffusion process. Critical to the model's performance was a fluctuating level of tonic dopamine which increased under conditions of uncertainty. With an optimal range of tonic dopamine, explore-exploit decisions were mediated by the effects of tonic dopamine on the precision of the model action selection mechanism. Under conditions of uncertain reward pay-out, the model's reduced selectivity allowed disinhibition of multiple alternative actions to be explored at random. Conversely, when uncertainly about reward pay-out was low, enhanced selectivity of the action selection circuit facilitated exploitation of the high value choice. Model performance was at the level of a Kalman filter which provides an optimal solution for the task. These simulations support the idea that this subcortical neural circuit may have evolved to facilitate decision making in non-stationary reward environments. The model generates several experimental predictions with relevance to abnormal decision making in neuropsychiatric and neurological disease.

Mechanisms of Network Interactions for Flexible Cortico-Basal Ganglia-Mediated Action Control

In humans, finely tuned γ synchronization (60-90 Hz) rapidly appears at movement onset in a motor control network involving primary motor cortex, the basal ganglia and motor thalamus. Yet the functional consequences of brief movement-related synchronization are still unclear. Distinct synchronization phenomena have also been linked to different forms of motor inhibition, including relaxing antagonist muscles, rapid movement interruption and stabilizing network dynamics for sustained contractions. Here, I will introduce detailed hypotheses about how intrasite and intersite synchronization could interact with firing rate changes in different parts of the network to enable flexible action control. The here proposed cause-and-effect relationships shine a spotlight on potential key mechanisms of cortico-basal ganglia-thalamo-cortical (CBGTC) communication. Confirming or revising these hypotheses will be critical in understanding the neuronal basis of flexible movement initiation, invigoration and inhibition. Ultimately, the study of more complex cognitive phenomena will also become more tractable once we understand the neuronal mechanisms underlying behavioral readouts.

Patterns of white matter hyperintensities associated with cognition in middle-aged cognitively healthy individuals

White matter hyperintensities (WMH) are commonly detected in the brain of elderly individuals and have been associated with a negative impact on multiple cognitive domains. We aim to investigate the impact of global and regional distribution of WMH on episodic memory and executive function in middle-aged cognitively unimpaired participants [N = 561 (45–75 years)] enriched for Alzheimer’s disease risk factors. WMH were automatically segmented from FLAIR, T1 and FSE MR images. WMH load was calculated both globally and regionally. At each cerebral lobe, regional WMH load was measured at four equidistant layers extending from the lateral ventricles to juxtacortical areas. Cognition was measured by The Memory Binding Test (MBT) and WAIS-IV subtests. Global composite z-scores were calculated for the two cognitive domains. Association between global and regional WMH measurements were sought against cognitive measures, both in global composite scores and in individual subtests. We adjusted cognition and WMH burden for the main sociodemographic (age, sex and education) and genetic factors (APOE-ε4). Memory and executive function were significantly associated with global WMH load. Regionally, lower executive performance was mainly associated with higher deep WMH load in frontal areas and, to a lower degree, in occipital, parietal and temporal regions. Lower episodic memory performance was correlated with higher WMH burden in deep frontal and occipital areas. Our novel methodological approach of regional analysis allowed us to reveal the association between cognition and WMH in strategic brain locations. Our results suggest that, even a small WMH load can impact cognition in cognitively unimpaired middle-aged subjects.

Generative Models for Active Vision

The active visual system comprises the visual cortices, cerebral attention networks, and oculomotor system. While fascinating in its own right, it is also an important model for sensorimotor networks in general. A prominent approach to studying this system is active inference-which assumes the brain makes use of an internal (generative) model to predict proprioceptive and visual input. This approach treats action as ensuring sensations conform to predictions (i.e., by moving the eyes) and posits that visual percepts are the consequence of updating predictions to conform to sensations. Under active inference, the challenge is to identify the form of the generative model that makes these predictions-and thus directs behavior. In this paper, we provide an overview of the generative models that the brain must employ to engage in active vision. This means specifying the processes that explain retinal cell activity and proprioceptive information from oculomotor muscle fibers. In addition to the mechanics of the eyes and retina, these processes include our choices about where to move our eyes. These decisions rest upon beliefs about salient locations, or the potential for information gain and belief-updating. A key theme of this paper is the relationship between "looking" and "seeing" under the brain's implicit generative model of the visual world.

Neural activity during a simple reaching task in macaques is counter to gating and rebound in basal ganglia-thalamic communication

Task-related activity in the ventral thalamus, a major target of basal ganglia output, is often assumed to be permitted or triggered by changes in basal ganglia activity through gating- or rebound-like mechanisms. To test those hypotheses, we sampled single-unit activity from connected basal ganglia output and thalamic nuclei (globus pallidus-internus [GPi] and ventrolateral anterior nucleus [VLa]) in monkeys performing a reaching task. Rate increases were the most common peri-movement change in both nuclei. Moreover, peri-movement changes generally began earlier in VLa than in GPi. Simultaneously recorded GPi-VLa pairs rarely showed short-time-scale spike-to-spike correlations or slow across-trials covariations, and both were equally positive and negative. Finally, spontaneous GPi bursts and pauses were both followed by small, slow reductions in VLa rate. These results appear incompatible with standard gating and rebound models. Still, gating or rebound may be possible in other physiological situations: simulations show how GPi-VLa communication can scale with GPi synchrony and GPi-to-VLa convergence, illuminating how synchrony of basal ganglia output during motor learning or in pathological conditions may render this pathway effective. Thus, in the healthy state, basal ganglia-thalamic communication during learned movement is more subtle than expected, with changes in firing rates possibly being dominated by a common external source.

Brain Functional Mechanisms in Attentional Processing Following Modified Conflict Stroop Task

BACKGROUND

Cognitive control of brain regions can be determined by the tasks involving the cognitive control such as the color word Stroop task. Stroop task define the reduction in function in incongruent condition, which requires more attention and control of competitive responses.

OBJECTIVE

The purpose of this study was to evaluate the activity of brain using the Modified Conflict Stroop Task in Military Personnel.

MATERIAL AND METHODS

In this applied experimental study, to specify the activity of different regions of brain in response to conflict Persian color-word Stroop task, 20 healthy persons participated in this study. To evaluate selective attention, the traditional color-word Stroop Task Model was modified, and the Stroop test was designed in high- and low-threat zones. We used functional magnetic resonance imaging (fMRI) to evaluate the brain activation during the Stroop task performance. The color-word Stroop task consists of incongruent, congruent, and neutral conditions, and the subjects were requested to carefully choose the correct answer.

RESULTS

The mean response time was longer in incongruent condition (867.6±193.5ms) compared to congruent and neutral conditions. Analysis of neuroimaging data revealed that the brain conflict-related regions are activated by the Stroop interference. In incongruent trial, the superior frontal gyrus (SFG) and inferior frontal gyrus (IFG) showed the most active and stronger BOLD responses. In congruent trials, the activation in the brain was less and had difference compared with incongruent trials.

CONCLUSION

Our result offers that the frontal cortex and the anterior cingulate cortex are sensitive to different trials of Persian Stroop task. Using modified Stroop task, we determined the brain responses to the selective attention test.

Neural Interactions Underlying Visuomotor Associations in the Human Brain

Rapid and flexible learning during behavioral choices is critical to our daily endeavors and constitutes a hallmark of dynamic reasoning. An important paradigm to examine flexible behavior involves learning new arbitrary associations mapping visual inputs to motor outputs. We conjectured that visuomotor rules are instantiated by translating visual signals into actions through dynamic interactions between visual, frontal and motor cortex. We evaluated the neural representation of such visuomotor rules by performing intracranial field potential recordings in epilepsy subjects during a rule-learning delayed match-to-behavior task. Learning new visuomotor mappings led to the emergence of specific responses associating visual signals with motor outputs in 3 anatomical clusters in frontal, anteroventral temporal and posterior parietal cortex. After learning, mapping selective signals during the delay period showed interactions with visual and motor signals. These observations provide initial steps towards elucidating the dynamic circuits underlying flexible behavior and how communication between subregions of frontal, temporal, and parietal cortex leads to rapid learning of task-relevant choices.

Accelerated redevelopment of vocal skills is preceded by lasting reorganization of the song motor circuitry

Complex motor skills take considerable time and practice to learn. Without continued practice the level of skill performance quickly degrades, posing a problem for the timely utilization of skilled motor behaviors. Here we quantified the recurring development of vocal motor skills and the accompanying changes in synaptic connectivity in the brain of a songbird, while manipulating skill performance by consecutively administrating and withdrawing testosterone. We demonstrate that a songbird with prior singing experience can significantly accelerate the re-acquisition of vocal performance. We further demonstrate that an increase in vocal performance is accompanied by a pronounced synaptic pruning in the forebrain vocal motor area HVC, a reduction that is not reversed when birds stop singing. These results provide evidence that lasting synaptic changes in the motor circuitry are associated with the savings of motor skills, enabling a rapid recovery of motor performance under environmental time constraints.

Environmental harshness and unpredictability, life history, and social and academic behavior of adolescents in nine countries

Safety is essential for life. To survive, humans and other animals have developed sets of psychological and physiological adaptations known as life history (LH) tradeoff strategies in response to various safety constraints. Evolutionarily selected LH strategies in turn regulate development and behavior to optimize survival under prevailing safety conditions. The present study tested LH hypotheses concerning safety based on a 6-year longitudinal sample of 1,245 adolescents and their parents from 9 countries. The results revealed that, invariant across countries, environmental harshness, and unpredictability (lack of safety) was negatively associated with slow LH behavioral profile, measured 2 years later, and slow LH behavioral profile was negatively and positively associated with externalizing behavior and academic performance, respectively, as measured an additional 2 years later. These results support the evolutionary conception that human development responds to environmental safety cues through LH regulation of social and learning behaviors. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Deep Brain Stimulation for Amelioration of Cognitive Impairment in Neurological Disorders: Neurogenesis and Circuit Reanimation

Acute (e.g., traumatic brain injury or stroke) and chronic (e.g., dementia or Parkinson’s disease dementia) neurological disorders that involve cognitive impairment and dysfunctional neural circuits always lead to a dreadful and costly experience for patients and their families. The application of deep brain stimulation for the treatment of neuropsychiatric disorders has shown great potential to modulate pathological neural circuits and trigger endogenous neurogenesis. We summarize several important clinical and translational studies that utilize deep brain stimulation to improve cognition based on the potentiation of neural plasticity and neurogenesis. In addition, we discuss the neuroanatomy and cerebral circuits implicated in such studies as well as the potential mechanisms underlying therapeutic benefits.

The Avian Basal Ganglia Are a Source of Rapid Behavioral Variation That Enables Vocal Motor Exploration

The basal ganglia (BG) participate in aspects of reinforcement learning that require evaluation and selection of motor programs associated with improved performance. However, whether the BG additionally contribute to behavioral variation ("motor exploration") that forms the substrate for such learning remains unclear. In songbirds, a tractable system for studying BG-dependent skill learning, a role for the BG in generating exploratory variability, has been challenged by the finding that lesions of Area X, the song-specific component of the BG, have no lasting effects on several forms of vocal variability that have been studied. Here we demonstrate that lesions of Area X in adult male zebra finches (Taeniopygia gutatta) permanently eliminate rapid within-syllable variation in fundamental frequency (FF), which can act as motor exploration to enable reinforcement-driven song learning. In addition, we found that this within-syllable variation is elevated in juveniles and in adults singing alone, conditions that have been linked to enhanced song plasticity and elevated neural variability in Area X. Consistent with a model that variability is relayed from Area X, via its cortical target, the lateral magnocellular nucleus of the anterior nidopallium (LMAN), to influence song motor circuitry, we found that lesions of LMAN also eliminate within-syllable variability. Moreover, we found that electrical perturbation of LMAN can drive fluctuations in FF that mimic naturally occurring within-syllable variability. Together, these results demonstrate that the BG are a central source of rapid behavioral variation that can serve as motor exploration for vocal learning.SIGNIFICANCE STATEMENT Many complex motor skills, such as speech, are not innately programmed but are learned gradually through trial and error. Learning involves generating exploratory variability in action ("motor exploration") and evaluating subsequent performance to acquire motor programs that lead to improved performance. Although it is well established that the basal ganglia (BG) process signals relating to action evaluation and selection, whether and how the BG promote exploratory motor variability remain unclear. We investigated this question in songbirds, which learn to produce complex vocalizations through trial and error. In contrast with previous studies that did not find effects of BG lesions on vocal motor variability, we demonstrate that the BG are an essential source of rapid behavioral variation linked to vocal learning.

Non-motor Characterization of the Basal Ganglia: Evidence From Human and Non-human Primate Electrophysiology

Although the basal ganglia have been implicated in a growing list of human behaviors, they include some of the least understood nuclei in the brain. For several decades studies have employed numerous methodologies to uncover evidence pointing to the basal ganglia as a hub of both motor and non-motor function. Recently, new electrophysiological characterization of the basal ganglia in humans has become possible through direct access to these deep structures as part of routine neurosurgery. Electrophysiological approaches for identifying non-motor function have the potential to unlock a deeper understanding of pathways that may inform clinical interventions and particularly neuromodulation. Various electrophysiological modalities can also be combined to reveal functional connections between the basal ganglia and traditional structures throughout the neocortex that have been linked to non-motor behavior. Several reviews have previously summarized evidence for non-motor function in the basal ganglia stemming from behavioral, clinical, computational, imaging, and non-primate animal studies; in this review, instead we turn to electrophysiological studies of non-human primates and humans. We begin by introducing common electrophysiological methodologies for basal ganglia investigation, and then we discuss studies across numerous non-motor domains–emotion, response inhibition, conflict, decision-making, error-detection and surprise, reward processing, language, and time processing. We discuss the limitations of current approaches and highlight the current state of the information.

Circuit Mechanisms of Sensorimotor Learning

The relationship between the brain and the environment is flexible, forming the foundation for our ability to learn. Here we review the current state of our understanding of the modifications in the sensorimotor pathway related to sensorimotor learning. We divide the process into three hierarchical levels with distinct goals: (1) sensory perceptual learning, (2) sensorimotor associative learning, and (3) motor skill learning. Perceptual learning optimizes the representations of important sensory stimuli. Associative learning and the initial phase of motor skill learning are ensured by feedback-based mechanisms that permit trial-and-error learning. The later phase of motor skill learning may primarily involve feedback-independent mechanisms operating under the classic Hebbian rule. With these changes under distinct constraints and mechanisms, sensorimotor learning establishes dedicated circuitry for the reproduction of stereotyped neural activity patterns and behavior.

Cerebellar Roles in Self-Timing for Sub- and Supra-Second Intervals

Previous studies suggest that the cerebellum and basal ganglia are involved in sub-second and supra-second timing, respectively. To test this hypothesis at the cellular level, we examined the activity of single neurons in the cerebellar dentate nucleus in monkeys performing the oculomotor version of the self-timing task. Animals were trained to report the passage of time of 400, 600, 1200, or 2400 ms following a visual cue by making self-initiated memory-guided saccades. We found a sizeable preparatory neuronal activity before self-timed saccades across delay intervals, while the time course of activity correlated with the trial-by-trial variation of saccade latency in different ways depending on the length of the delay intervals. For the shorter delay intervals, the ramping up of neuronal firing rate started just after the visual cue and the rate of rise of neuronal activity correlated with saccade timing. In contrast, for the longest delay (2400 ms), the preparatory activity started late during the delay period, and its onset time correlated with self-timed saccade latency. Because electrical microstimulation applied to the recording sites during saccade preparation advanced self-timed but not reactive saccades, regardless of their directions, the signals in the cerebellum may have a causal role in self-timing. We suggest that the cerebellum may regulate timing in both sub-second and supra-second ranges, although its relative contribution might be greater for sub-second than for supra-second time intervals.SIGNIFICANCE STATEMENT How we decide the timing of self-initiated movement is a fundamental question. According to the prevailing hypothesis, the cerebellum plays a role in monitoring sub-second timing, whereas the basal ganglia are important for supra-second timing. To verify this, we explored neuronal signals in the monkey cerebellum while animals reported the passage of time in the range 400-2400 ms by making eye movements. Contrary to our expectations, we found that neurons in the cerebellar dentate nucleus exhibited a similar preparatory activity for both sub-second and supra-second intervals, and that electrical simulation advanced self-timed saccades in both conditions. We suggest that the cerebellum plays a causal role in the fine adjustment of self-timing in a larger time range than previously thought.

Neural and neurochemical basis of reinforcement-guided decision making

Decision making is an adaptive behavior that takes into account several internal and external input variables and leads to the choice of a course of action over other available and often competing alternatives. While it has been studied in diverse fields ranging from mathematics, economics, ecology, and ethology to psychology and neuroscience, recent cross talk among perspectives from different fields has yielded novel descriptions of decision processes. Reinforcement-guided decision making models are based on economic and reinforcement learning theories, and their focus is on the maximization of acquired benefit over a defined period of time. Studies based on reinforcement-guided decision making have implicated a large network of neural circuits across the brain. This network includes a wide range of cortical (e.g., orbitofrontal cortex and anterior cingulate cortex) and subcortical (e.g., nucleus accumbens and subthalamic nucleus) brain areas and uses several neurotransmitter systems (e.g., dopaminergic and serotonergic systems) to communicate and process decision-related information. This review discusses distinct as well as overlapping contributions of these networks and neurotransmitter systems to the processing of decision making. We end the review by touching on neural circuitry and neuromodulatory regulation of exploratory decision making.

Differential contributions of the globus pallidus and ventral thalamus to stimulus-response learning in humans

The ability to learn associations between stimuli, responses and rewards is a prerequisite for survival. Models of reinforcement learning suggest that the striatum, a basal ganglia input nucleus, vitally contributes to these learning processes. Our recently presented computational model predicts, first, that not only the striatum, but also the globus pallidus contributes to the learning (i.e., exploration) of stimulus-response associations based on rewards. Secondly, it predicts that the stable execution (i.e., exploitation) of well-learned associations involves further learning in the thalamus. To test these predictions, we postoperatively recorded local field potentials (LFPs) from patients that had undergone surgery for deep brain stimulation to treat severe movement disorders. Macroelectrodes were placed either in the globus pallidus or in the ventral thalamus. During recordings, patients performed a reward-based stimulus-response learning task that comprised periods of exploration and exploitation. We analyzed correlations between patients' LFP amplitudes and model-based estimates of their reward expectations and reward prediction errors. In line with our first prediction, pallidal LFP amplitudes during the presentation of rewards and reward omissions correlated with patients' reward prediction errors, suggesting pallidal access to reward-based teaching signals. Unexpectedly, the same was true for the thalamus. In further support of this prediction, pallidal LFP amplitudes during stimulus presentation correlated with patients' reward expectations during phases of low reward certainty - suggesting pallidal participation in the learning of stimulus-response associations. In line with our second prediction, correlations between thalamic stimulus-related LFP amplitudes and patients' reward expectations were significant within phases of already high reward certainty, suggesting thalamic participation in exploitation.

Structure and function of large-scale brain systems

This article introduces the functional neuroanatomy of large-scale brain systems. Both the structure and functions of these brain networks are presented. All human behavior is the result of interactions within and between these brain systems. This system of brain function completely changes our understanding of how cognition and behavior are organized within the brain, replacing the traditional lesion model. Understanding behavior within the context of brain network interactions has profound implications for modifying abstract constructs such as attention, learning, and memory. These constructs also must be understood within the framework of a paradigm shift, which emphasizes ongoing interactions within a dynamically changing environment.

Origins of basal ganglia output signals in singing juvenile birds

Across species, complex circuits inside the basal ganglia (BG) converge on pallidal output neurons that exhibit movement-locked firing patterns. Yet the origins of these firing patterns remain poorly understood. In songbirds during vocal babbling, BG output neurons homologous to those found in the primate internal pallidal segment are uniformly activated in the tens of milliseconds prior to syllable onsets. To test the origins of this remarkably homogenous BG output signal, we recorded from diverse upstream BG cell types during babbling. Prior to syllable onsets, at the same time that internal pallidal segment-like neurons were activated, putative medium spiny neurons, fast spiking and tonically active interneurons also exhibited transient rate increases. In contrast, pallidal neurons homologous to those found in primate external pallidal segment exhibited transient rate decreases. To test origins of these signals, we performed recordings following lesion of corticostriatal inputs from premotor nucleus HVC. HVC lesions largely abolished these syllable-locked signals. Altogether, these findings indicate a striking homogeneity of syllable timing signals in the songbird BG during babbling and are consistent with a role for the indirect and hyperdirect pathways in transforming cortical inputs into BG outputs during an exploratory behavior.

Resting-state fMRI activity predicts unsupervised learning and memory in an immersive virtual reality environment

In the real world, learning often proceeds in an unsupervised manner without explicit instructions or feedback. In this study, we employed an experimental paradigm in which subjects explored an immersive virtual reality environment on each of two days. On day 1, subjects implicitly learned the location of 39 objects in an unsupervised fashion. On day 2, the locations of some of the objects were changed, and object location recall performance was assessed and found to vary across subjects. As prior work had shown that functional magnetic resonance imaging (fMRI) measures of resting-state brain activity can predict various measures of brain performance across individuals, we examined whether resting-state fMRI measures could be used to predict object location recall performance. We found a significant correlation between performance and the variability of the resting-state fMRI signal in the basal ganglia, hippocampus, amygdala, thalamus, insula, and regions in the frontal and temporal lobes, regions important for spatial exploration, learning, memory, and decision making. In addition, performance was significantly correlated with resting-state fMRI connectivity between the left caudate and the right fusiform gyrus, lateral occipital complex, and superior temporal gyrus. Given the basal ganglia's role in exploration, these findings suggest that tighter integration of the brain systems responsible for exploration and visuospatial processing may be critical for learning in a complex environment.

The organization and dynamics of corticostriatal pathways link the medial orbitofrontal cortex to future behavioral responses

Accurately making a decision in the face of incongruent options increases the efficiency of making similar congruency decisions in the future. Contextual factors like reward can modulate this adaptive process, suggesting that networks associated with monitoring previous success and failure outcomes might contribute to this form of behavioral updating. To evaluate this possibility, a group of healthy adults (n = 30) were tested with functional MRI (fMRI) while they performed a color-word Stroop task. In a conflict-related region of the medial orbitofrontal cortex (mOFC), stronger BOLD responses predicted faster response times (RTs) on the next trial. More importantly, the degree of behavioral adaptation of RTs was correlated with the magnitude of mOFC-RT associations on the previous trial, but only after accounting for network-level interactions with prefrontal and striatal regions. This suggests that congruency sequencing effects may rely on interactions between distributed corticostriatal circuits. This possibility was evaluated by measuring the convergence of white matter projections from frontal areas into the striatum with diffusion-weighted imaging. In these pathways, greater convergence of corticostriatal projections correlated with stronger functional mOFC-RT associations that, in turn, provided an indirect pathway linking anatomical structure to behavior. Thus distributed corticostriatal processing may mediate the orbitofrontal cortex's influence on behavioral updating, even in the absence of explicit rewards.

Enhanced corticobulbar excitability in chronic smokers during visual exposure to cigarette smoking cues

BACKGROUND

Neuroimaging studies of chronic smokers report altered activity of several neural regions involved in the processing of rewarding outcomes. Neuroanatomical evidence suggests that these regions are directly connected to the tongue muscle through the corticobulbar pathways. Accordingly, we examined whether corticobulbar excitability might be considered a somatic marker for nicotine craving.

METHODS

We compared motor-evoked potential (MEP) amplitudes recorded from the tongue and the extensor carpi radialis (control muscle) of chronic smokers under drug withdrawal and intake conditions as well as a nonsmoker group. All participants were tested during passive exposure to pictures showing a smoking cue or a meaningless stimulus. In the intake condition, chronic smokers were asked to smoke a real cigarette (CSn: group 1) or a placebo (CSp: group 2).

RESULTS

Results show that MEP amplitudes recorded from the tongues of participants in the CSn and CSp groups under the withdrawal condition were selectively enhanced during exposure to a smoking cue. However, this effect on tongue MEP amplitudes disappeared in the intake condition for both the CSn and CSp groups.

LIMITATIONS

Limitations include the fact that the study was conducted in 2 different laboratories, the small sample size, the absence of data on chronic smoker craving strength and the different tastes of the real and placebo cigarettes.

CONCLUSION

These results suggest that, in chronic smokers, tongue muscle MEP amplitudes are sensitive to neural processes active under the physiological status of nicotine craving. This finding implicates a possible functional link between neural excitability of the corticobulbar pathway and the reward system in chronic smokers.

Reward and reinforcement activity in the nucleus accumbens during learning

The nucleus accumbens core (NAcc) has been implicated in learning associations between sensory cues and profitable motor responses. However, the precise mechanisms that underlie these functions remain unclear. We recorded single-neuron activity from the NAcc of primates trained to perform a visual-motor associative learning task. During learning, we found two distinct classes of NAcc neurons. The first class demonstrated progressive increases in firing rates at the go-cue, feedback/tone and reward epochs of the task, as novel associations were learned. This suggests that these neurons may play a role in the exploitation of rewarding behaviors. In contrast, the second class exhibited attenuated firing rates, but only at the reward epoch of the task. These findings suggest that some NAcc neurons play a role in reward-based reinforcement during learning.

Optogenetic stimulation in a computational model of the basal ganglia biases action selection and reward prediction error

Optogenetic stimulation of specific types of medium spiny neurons (MSNs) in the striatum has been shown to bias the selection of mice in a two choices task. This shift is dependent on the localisation and on the intensity of the stimulation but also on the recent reward history. We have implemented a way to simulate this increased activity produced by the optical flash in our computational model of the basal ganglia (BG). This abstract model features the direct and indirect pathways commonly described in biology, and a reward prediction pathway (RP). The framework is similar to Actor-Critic methods and to the ventral/dorsal distinction in the striatum. We thus investigated the impact on the selection caused by an added stimulation in each of the three pathways. We were able to reproduce in our model the bias in action selection observed in mice. Our results also showed that biasing the reward prediction is sufficient to create a modification in the action selection. However, we had to increase the percentage of trials with stimulation relative to that in experiments in order to impact the selection. We found that increasing only the reward prediction had a different effect if the stimulation in RP was action dependent (only for a specific action) or not. We further looked at the evolution of the change in the weights depending on the stage of learning within a block. A bias in RP impacts the plasticity differently depending on that stage but also on the outcome. It remains to experimentally test how the dopaminergic neurons are affected by specific stimulations of neurons in the striatum and to relate data to predictions of our model.

Electrophysiological characterization of entopeduncular nucleus neurons in anesthetized and freely moving rats

The EntoPeduncular nucleus (EP), which is homologous to the internal segment of the Globus Pallidus (GPi) in primates, is one of the two basal ganglia (BG) output nuclei. Despite their importance in cortico-BG information processing, EP neurons have rarely been investigated in rats and there is no available electrophysiological characterization of EP neurons in vivo. We recorded and analyzed the activity of EP neurons in freely moving as well as anesthetized rats, and compared their activity patterns. Examination of neuronal firing statistics during wakefulness suggested that similar to neurons recorded in the primate GPi, EP neurons are a single population characterized by Poisson-like firing. Under isoflurane anesthesia the firing rate of EP neurons decreased substantially and their coefficient of variation and relative duration of quiescence periods increased. Investigation of the relationship between firing rate and depth of anesthesia revealed two distinct neuronal groups: one that decreased its firing rate with the increase in anesthesia level, and a second group where the firing rate was independent of anesthesia level. Post-hoc examination of the firing properties of the two groups showed that they were statistically distinct. These results may thus help reconcile in vitro studies in rats and primates which have reported two distinct neuronal populations, and in vivo studies in behaving primates indicating one homogeneous population. Our data support the existence of two distinct neuronal populations in the rat EP that can be distinguished by their characteristic firing response to anesthesia.

Basal ganglia output to the thalamus: still a paradox

The basal ganglia (BG)-recipient thalamus controls motor output but it remains unclear how its activity is regulated. Several studies report that thalamic activation occurs via disinhibition during pauses in the firing of inhibitory pallidal inputs from the BG. Other studies indicate that thalamic spiking is triggered by pallidal inputs via post-inhibitory 'rebound' calcium spikes. Finally excitatory cortical inputs can drive thalamic activity, which becomes entrained, or time-locked, to pallidal spikes. We present a unifying framework where these seemingly distinct results arise from a continuum of thalamic firing 'modes' controlled by excitatory inputs. We provide a mechanistic explanation for paradoxical pallidothalamic coactivations observed during behavior that raises new questions about what information is integrated in the thalamus to control behavior.

Categorization = decision making + generalization

We rarely, if ever, repeatedly encounter exactly the same situation. This makes generalization crucial for real world decision making. We argue that categorization, the study of generalizable representations, is a type of decision making, and that categorization learning research would benefit from approaches developed to study the neuroscience of decision making. Similarly, methods developed to examine generalization and learning within the field of categorization may enhance decision making research. We first discuss perceptual information processing and integration, with an emphasis on accumulator models. We then examine learning the value of different decision making choices via experience, emphasizing reinforcement learning modeling approaches. Next we discuss how value is combined with other factors in decision making, emphasizing the effects of uncertainty. Finally, we describe how a final decision is selected via thresholding processes implemented by the basal ganglia and related regions. We also consider how memory related functions in the hippocampus may be integrated with decision making mechanisms and contribute to categorization.

Similar circuits but different connectivity patterns between the cerebellum, basal ganglia, and supplementary motor area in early Parkinson's disease patients and controls during predictive motor timing

BACKGROUND AND PURPOSE

The cerebellum, basal ganglia (BG), and other cortical regions, such as supplementary motor area (SMA) have emerged as important structures dealing with various aspects of timing, yet the modulation of functional connectivity between them during motor timing tasks remains unexplored.

METHODS

We used dynamic causal modeling to investigate the differences in effective connectivity (EC) between these regions and its modulation by behavioral outcome during a motor timing prediction task in a group of 16 patients with early Parkinson's disease (PD) and 17 healthy controls. Behavioral events (hits and errors) constituted the driving input connected to the cerebellum, and the modulation in connectivity was assessed relative to the hit condition (successful interception of target).

RESULTS

The driving input elicited response in the target area, while modulatory input changed the specific connection strength. The neuroimaging data revealed similar structure of intrinsic connectivity in both groups with unidirectional connections from cerebellum to both sides of the BG, from BG to the SMA, and then from SMA to the cerebellum. However, the type of intrinsic connection was different between two groups. In the PD group, the connection between the SMA and cerebellum was inhibitory in comparison to the HC group, where the connection was activated. Furthermore, the modulation of connectivity by the performance in the task was different between the two groups, with decreased connectivity between the cerebellum and left BG and SMA and a more pronounced symmetry of these connections in controls. In the same time, there was an increased EC between the cerebellum and both sides of BG with more pronounced asymmetry (stronger connection with left BG) in patients. In addition, in the PD group the modulatory input strengthened inhibitory connectivity between the SMA and the cerebellum, while in the HC group the excitatory connection was slightly strengthened.

CONCLUSIONS

Our findings indicate that although early PD subjects and controls use similar functional circuits to maintain a successful outcome in predictive motor timing behavior, the type and strength of EC and its modulation by behavioral performance differ between these two groups. These functional differences might represent the first step of cortical reorganization aimed at maintaining a normal performance in the brain affected by early Parkinson's disease and may have implications for the neuro-rehabilitation field.

From movement to thought: the development of executive function

This article presents a very simple definition of executive functioning (EF). Although EF is traditionally understood as a cognitive function dependent upon top-down cortical control, we challenge this model. We propose that the functional architecture of the brain evolved to meet the needs of interactive behavior and that cognition develops to control the motor system, which is of paramount importance in adaptation, essentially a manifestation of EF. We propose that traditional models of cognition are incomplete characterizations of EF and that procedural learning and "automatic" behaviors are the most basic, bottom-up functions that support all EF. We propose that motor development in children demonstrates how all knowledge is grounded in sensorimotor interaction and how interactive behavior generates both procedural and declarative knowledge, which later interact to generate EF. This model emphasizes the critical importance of motor behavior in children and stresses the importance of the pediatric motor examination in understanding the development of EF. This model also has implications for why traditional tests of EF have little predictive validity in both children and adults.

The basal ganglia

Through its connections with widespread cortical areas and with dopaminergic midbrain areas, the basal ganglia are well situated to integrate patterns of cortical input with the dopaminergic reward signal originating in the midbrain. In this review, we consider the functions of the basal ganglia in relation to its gross and cellular anatomy, and discuss how these mechanisms subserve the thresholding and selection of motor and cognitive processes. We also discuss how the dopaminergic reward signal enables flexible task learning through modulation of striatal plasticity, and how reinforcement learning models have been used to account for various aspects of basal ganglia activity. Specifically, we will discuss the important role of the basal ganglia in instrumental learning, cognitive control, sequence learning, and categorization tasks. Finally, we will discuss the neurobiological and cognitive characteristics of Parkinson's disease, Huntington's disease and addiction to illustrate the relationship between the basal ganglia and cognitive function. WIREs Cogn Sci 2013, 4:135-148. doi: 10.1002/wcs.1217 For further resources related to this article, please visit the WIREs website.

Sensory integration, sensory processing, and sensory modulation disorders: putative functional neuroanatomic underpinnings

This paper examines conditions that have variously been called sensory integration disorder, sensory processing disorder, and sensory modulation disorder (SID/SPD/SMD). As these conditions lack readily and consistently agreed-upon operational definitions, there has been confusion as to how these disorders are conceptualized. Rather than addressing various diagnostic controversies, we will instead focus upon explaining the symptoms that are believed to characterize these disorders. First, to clarify the overall context within which to view symptoms, we summarize a paradigm of adaptation characterized by continuous sensorimotor interaction with the environment. Next, we review a dual-tiered, integrated model of brain function in order to establish neuroanatomic underpinnings with which to conceptualize the symptom presentations. Generally accepted functions of the neocortex, basal ganglia, and cerebellum are described to illustrate how interactions between these brain regions generate both adaptive and pathological symptoms and behaviors. We then examine the symptoms of SID/SPD/SMD within this interactive model and in relation to their impact upon the development of inhibitory control, working memory, academic skill development, and behavioral automation. We present likely etiologies for these symptoms, not only as they drive neurodevelopmental pathologies but also as they can be understood as variations in the development of neural networks.

A cortical motor nucleus drives the basal ganglia-recipient thalamus in singing birds

The pallido-recipient thalamus transmits information from the basal ganglia (BG) to the cortex and plays a critical role motor initiation and learning. Thalamic activity is strongly inhibited by pallidal inputs from the BG, but the role of non-pallidal inputs, such as excitatory inputs from cortex, is unclear. We have recorded simultaneously from presynaptic pallidal axon terminals and postsynaptic thalamocortical neurons in a BG-recipient thalamic nucleus necessary for vocal variability and learning in zebra finches. We found that song-locked rate modulations in the thalamus could not be explained by pallidal inputs alone, and persisted following pallidal lesion. Instead, thalamic activity was likely driven by inputs from a motor ‘cortical’ nucleus also necessary for singing. These findings suggest a role for cortical inputs to the pallido-recipient thalamus in driving premotor signals important for exploratory behavior and learning.

Synchrony of corticostriatal-midbrain activation enables normal inhibitory control and conflict processing in recovering alcoholic men

BACKGROUND

Alcohol dependence is associated with inhibitory control deficits, possibly related to abnormalities in frontoparietal cortical and midbrain function and connectivity.

METHODS

We examined functional connectivity and microstructural fiber integrity between frontoparietal and midbrain structures using a Stroop Match-to-Sample task with functional magnetic resonance imaging and diffusion tensor imaging in 18 alcoholic and 17 control subjects. Manipulation of color cues and response repetition sequences modulated cognitive demands during Stroop conflict.

RESULTS

Despite similar lateral frontoparietal activity and functional connectivity in alcoholic and control subjects when processing conflict, control subjects deactivated the posterior cingulate cortex (PCC), whereas alcoholic subjects did not. Posterior cingulum fiber integrity predicted the degree of PCC deactivation in control but not alcoholic subjects. Also, PCC activity was modulated by executive control demands: activated during response switching and deactivated during response repetition. Alcoholics showed the opposite pattern: activation during repetition and deactivation during switching. Here, in alcoholic subjects, greater deviations from the normal PCC activity correlated with higher amounts of lifetime alcohol consumption. A functional dissociation of brain network connectivity between the groups further showed that control subjects exhibited greater corticocortical connectivity among middle cingulate, posterior cingulate, and medial prefrontal cortices than alcoholic subjects. In contrast, alcoholic subjects exhibited greater midbrain-orbitofrontal cortical network connectivity than control subjects. Degree of microstructural fiber integrity predicted robustness of functional connectivity.

CONCLUSIONS

Thus, even subtle compromise of microstructural connectivity in alcoholism can influence modulation of functional connectivity and underlie alcohol-related cognitive impairment.

A hypothesis for basal ganglia-dependent reinforcement learning in the songbird

Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.