Habits, rituals, and the evaluative brain

The origin of extracellular fields and currents--EEG, ECoG, LFP and spikes

Neuronal activity in the brain gives rise to transmembrane currents that can be measured in the extracellular medium. Although the major contributor of the extracellular signal is the synaptic transmembrane current, other sources--including Na(+) and Ca(2+) spikes, ionic fluxes through voltage- and ligand-gated channels, and intrinsic membrane oscillations--can substantially shape the extracellular field. High-density recordings of field activity in animals and subdural grid recordings in humans, combined with recently developed data processing tools and computational modelling, can provide insight into the cooperative behaviour of neurons, their average synaptic input and their spiking output, and can increase our understanding of how these processes contribute to the extracellular signal.

Statistical analysis of single-trial Granger causality spectra

Granger causality analysis is becoming central for the analysis of interactions between neural populations and oscillatory networks. However, it is currently unclear whether single-trial estimates of Granger causality spectra can be used reliably to assess directional influence. We addressed this issue by combining single-trial Granger causality spectra with statistical inference based on general linear models. The approach was assessed on synthetic and neurophysiological data. Synthetic bivariate data was generated using two autoregressive processes with unidirectional coupling. We simulated two hypothetical experimental conditions: the first mimicked a constant and unidirectional coupling, whereas the second modelled a linear increase in coupling across trials. The statistical analysis of single-trial Granger causality spectra, based on t-tests and linear regression, successfully recovered the underlying pattern of directional influence. In addition, we characterised the minimum number of trials and coupling strengths required for significant detection of directionality. Finally, we demonstrated the relevance for neurophysiology by analysing two local field potentials (LFPs) simultaneously recorded from the prefrontal and premotor cortices of a macaque monkey performing a conditional visuomotor task. Our results suggest that the combination of single-trial Granger causality spectra and statistical inference provides a valuable tool for the analysis of large-scale cortical networks and brain connectivity.

Thalamic POm projections to the dorsolateral striatum of rats: potential pathway for mediating stimulus-response associations for sensorimotor habits

The dorsolateral part of the striatum (DLS) represents the initial stage for processing sensorimotor information in the basal ganglia. Although the DLS receives much of its input from the primary somatosensory (SI) cortex, peripheral somesthetic stimulation activates the DLS at latencies that are shorter than the response latencies recorded in the SI cortex. To identify the subcortical regions that transmit somesthetic information directly to the DLS, we deposited small quantities of retrograde tracers at DLS sites that displayed consistent time-locked responses to controlled whisker stimulation. The neurons that were retrogradely labeled by these injections were located mainly in the sensorimotor cortex and, to a lesser degree, in the amygdala and thalamus. Quantitative analysis of neuronal labeling in the thalamus indicated that the strongest thalamic input to the whisker-sensitive part of the DLS originates from the medial posterior nucleus (POm), a somesthetic-related region that receives inputs from the spinal trigeminal nucleus. Anterograde tracer injections in POm confirmed that this thalamic region projects to the DLS neuropil. In subsequent experiments, simultaneous recordings from POm and the DLS during whisker stimulation showed that POm consistently responds before the DLS. These results suggest that POm could transmit somesthetic information to the DLS, and this modality-specific thalamostriatal pathway may cooperate with the thalamostriatal projections that originate from the intralaminar nuclei.

Age Trends in Visual Exploration of Social and Nonsocial Information in Children with Autism

Because previous studies of attention in autism spectrum disorders (ASD) have been restricted in age range examined, little is known about how these processes develop over the course of childhood. In this study we examined cross-sectional age effects on patterns of visual attention to social and nonsocial information in 43 typically developing children and 51 children with ASD ranging in age from 2 to 18. Results indicated a sharp increase in visual exploration with age and a decrease in perseverative and detail-focused attention for both groups of children. However, increased age was associated with greater increases in visual exploration for typically developing children than for those children with ASD. The developmental differences were most pronounced for attention to certain nonsocial stimuli as children with ASD demonstrated a disproportionate attentional bias for these stimuli from very early in life. Disproportionate visual attention to certain nonsocial objects relative to social stimuli in ASD spanned from early to late childhood, and thus may represent both an early and a persistent characteristic of the disorder.

Acquisition of "Start" and "Stop" response thresholds in peak-interval timing is differentially sensitive to protein synthesis inhibition in the dorsal and ventral striatum

Time-based decision-making in peak-interval timing procedures involves the setting of response thresholds for the initiation (“Start”) and termination (“Stop”) of a response sequence that is centered on a target duration. Using intracerebral infusions of the protein synthesis inhibitor anisomycin, we report that the acquisition of the “Start” response depends on normal functioning (including protein synthesis) in the dorsal striatum (DS), but not the ventral striatum (VS). Conversely, disruption of the VS, but not the DS, impairs the acquisition of the “Stop” response. We hypothesize that the dorsal and ventral regions of the striatum function as a competitive neural network that encodes the temporal boundaries marking the beginning and end of a timed response sequence.

Large-scale brain networks in board game experts: insights from a domain-related task and task-free resting state

Cognitive performance relies on the coordination of large-scale networks of brain regions that are not only temporally correlated during different tasks, but also networks that show highly correlated spontaneous activity during a task-free state. Both task-related and task-free network activity has been associated with individual differences in cognitive performance. Therefore, we aimed to examine the influence of cognitive expertise on four networks associated with cognitive task performance: the default mode network (DMN) and three other cognitive networks (central-executive network, dorsal attention network, and salience network). During fMRI scanning, fifteen grandmaster and master level Chinese chess players (GM/M) and fifteen novice players carried out a Chinese chess task and a task-free resting state. Modulations of network activity during task were assessed, as well as resting-state functional connectivity of those networks. Relative to novices, GM/Ms showed a broader task-induced deactivation of DMN in the chess problem-solving task, and intrinsic functional connectivity of DMN was increased with a connectivity pattern associated with the caudate nucleus in GM/Ms. The three other cognitive networks did not exhibit any difference in task-evoked activation or intrinsic functional connectivity between the two groups. These findings demonstrate the effect of long-term learning and practice in cognitive expertise on large-scale brain networks, suggesting the important role of DMN deactivation in expert performance and enhanced functional integration of spontaneous activity within widely distributed DMN-caudate circuitry, which might better support high-level cognitive control of behavior.

Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits

The role of the amygdala central nucleus (CeN) in habit learning was assessed in two experiments. First, we examined the effects of bilateral lesions of the anterior CeN on an overtraining-induced lever press habit evaluated using an outcome devaluation protocol. Overtraining generated habitual performance and rendered sham lesioned rats insensitive to outcome devaluation, an effect that was also found in rats given control lesions of the posterior CeN. In contrast, rats with lesions of the anterior CeN did not show normal habit acquisition and their performance remained goal-directed and sensitive to outcome devaluation. Nevertheless, lesions of either the posterior or the anterior CeN abolished the general excitatory influence of a Pavlovian conditioned stimulus on instrumental performance. Second, we assessed the functional interaction between the CeN and dorsolateral striatum (DLS), a region previously implicated in the acquisition of habits, using asymmetrical lesions to disconnect these structures. Rats were given a unilateral lesion of anterior CeN and a unilateral lesion of the DLS, made either ipsilateral (control) or contralateral (disconnection) to the CeN lesion, and given overtraining followed by outcome devaluation. Although the ipsilateral lesioned rats were insensitive to devaluation, the contralateral CeN-DLS lesion impaired habit acquisition, rendering performance sensitive to the devaluation treatment. These results are the first to implicate the CeN and its connection with a circuit involving DLS in habit learning. They imply that, in instrumental conditioning, regions of amygdala parse the instrumental outcome into the reward and reinforcement signals mediating goal-directed and habitual actions, respectively.

Habitual Creativity: Revising Habit, Reconceptualizing Creativity

Current psychological scholarship is based on a dichotomy between habit, associated with automatic reflex behavior, and creativity, which involves deliberation, purpose and heuristic procedures. However, this account is problematic and contradicts everyday experience where mastery, for instance, is one of the highest levels of creative performance achieved within a habitual practice. This article argues that such a separation misrepresents both habit and creativity with important theoretical and practical consequences. A first step toward reconciling the two terms is made by revisiting a series of foundational strands of theory from psychology and related disciplines. In light of these sources, habit is reformulated as a social, situated, and open system, and habitual creativity defined as the intrinsically creative nature of customary action, reflected in the way habits adjust to dynamic contexts, the way they are used, combined, and ultimately perfected. Further distinctions are then made between habit, improvisation, and innovation. Both improvisational and innovative creativity are embedded in habitual forms and this is well illustrated by craftwork: a practiced type of activity on the basis of which artisans improvise, whenever obstacles or difficulties are encountered, and even get to innovate when their intention is to generate novel artifacts or work techniques.

Neuronal activity in the primate dorsomedial prefrontal cortex contributes to strategic selection of response tactics

The functional roles of the primate posterior medial prefrontal cortex have remained largely unknown. Here, we show that this region participates in the regulation of actions in the presence of multiple response tactics. Monkeys performed a forelimb task in which a visual cue required prompt decision of reaching to a left or a right target. The location of the cue was either ipsilateral (concordant) or contralateral (discordant) to the target. As a result of extensive training, the reaction times for the concordant and discordant trials were indistinguishable, indicating that the monkeys developed tactics to overcome the cue-response conflict. Prefrontal neurons exhibited prominent activity when the concordant and discordant trials were randomly presented, requiring rapid selection of a response tactic (reach toward or away from the cue). The following findings indicate that these neurons are involved in the selection of tactics, rather than the selection of action or monitoring of response conflict: (i) The response period activity of neurons in this region disappeared when the monkeys performed the task under the behavioral condition that required a single tactic alone, whereas the action varied across trials. (ii) The neuronal activity was found in the dorsomedial prefrontal cortex but not in the anterior cingulate cortex that has been implicated for the response conflict monitoring. These results suggest that the medial prefrontal cortex participates in the selection of a response tactic that determines an appropriate action. Furthermore, the observation of dynamic, task-dependent neuronal activity necessitates reconsideration of the conventional concept of cortical motor representation.

Specialized cortical subnetworks differentially connect frontal cortex to parahippocampal areas

How information is manipulated and segregated within local circuits in the frontal cortex remains mysterious, in part because of inadequate knowledge regarding the connectivity of diverse pyramidal cell subtypes. The frontal cortex participates in the formation and retrieval of declarative memories through projections to the perirhinal cortex, and in procedural learning through projections to the striatum/pontine nuclei. In rat frontal cortex, we identified two pyramidal cell subtypes selectively projecting to distinct subregions of perirhinal cortex (PRC). PRC-projecting cells in upper layer 2/3 (L2/3) of the frontal cortex projected to perirhinal area 35, while neurons in L5 innervated perirhinal area 36. L2/3 PRC-projecting cells partially overlapped with those projecting to the basolateral amygdala. L5 PRC-projecting cells partially overlapped with crossed corticostriatal cells, but were distinct from neighboring corticothalamic (CTh)/corticopontine cells. L5 PRC-projecting and CTh cells were different in their electrophysiological properties and dendritic/axonal morphologies. Within the frontal cortex, L2/3 PRC-projecting cells innervated L5 PRC-projecting and CTh cells with similar probabilities, but received feedback excitation only from PRC-projecting cells. These data suggest that specific neuron subtypes in different cortical layers are reciprocally excited via interlaminar loops. Thus, two interacting output channels send information from the frontal cortex to different hierarchical stages of the parahippocampal network, areas 35 and 36, with additional collaterals selectively targeting the amygdala or basal ganglia, respectively. Combined with the hierarchical connectivity of PRC-projecting and CTh cells, these observations demonstrate an exquisite diversification of frontal projection neurons selectively connected according to their participation in distinct memory subsystems.

Ventral striatal volume is associated with cognitive decline in older people: a population based MR-study

Striatal degeneration may contribute to cognitive impairment in older people. Here, we examine the relation of degeneration of the striatum and substructures to cognitive decline and dementia in subjects with a wide range of cognitive function. Data are from the prospective community-based Honolulu Asia Aging Study of Japanese American men born 1900-1919. Brain magnetic resonance imaging (MRI) (1.5 T) was acquired on a stratified subsample (n = 477) that included four groups defined by cognitive status relative to the scan date: subjects without dementia (n = 347), subjects identified as demented 2-3 years before brain scanning (n = 30), at the time of scanning (n = 58), and 3-5 years after scanning (n = 42). Volumes of the striatum, including the accumbens, putamen, and caudate nucleus were automatically estimated from T1 MR images. Global cognitive function was measured with the cognitive ability screening instrument (CASI), at four examinations spanning an 8-year interval. Trajectories of cognitive decline were estimated for each quartile of striatal volume using mixed models, controlling for demographic variables, measures of cerebro-vascular damage, global brain atrophy, and hippocampal volume. Diagnosis of dementia before, during, and after brain scanning was associated with smaller volumes of n. accumbens and putamen, but not with caudate nucleus volume. Subjects in the lowest quartile of n. accumbens volume, both in the total sample and in the subjects not diagnosed with dementia during the study, had a significantly (p < 0.0001) steeper decline in cognitive performance compared with those in the highest quartile. In conclusion, volumes of the n. accumbens and putamen are closely associated with the occurrence of dementia and n. accumbens volume predicts cognitive decline in older people. These associations were found independent of the magnitude of other pivotal markers of cognitive decline, i.e. cerebro-vascular damage and hippocampal volume. The present study suggests a role for the ventral striatum in the development of clinical dementia.

Striatum processes reward differently in adolescents versus adults

Adolescents often respond differently than adults to the same salient motivating contexts, such as peer interactions and pleasurable stimuli. Delineating the neural processing differences of adolescents is critical to understanding this phenomenon, as well as the bases of serious behavioral and psychiatric vulnerabilities, such as drug abuse, mood disorders, and schizophrenia. We believe that age-related changes in the ways salient stimuli are processed in key brain regions could underlie the unique predilections and vulnerabilities of adolescence. Because motivated behavior is the central issue, it is critical that age-related comparisons of brain activity be undertaken during motivational contexts. We compared single-unit activity and local field potentials in the nucleus accumbens (NAc) and dorsal striatum (DS) of adolescent and adult rats during a reward-motivated instrumental task. These regions are involved in motivated learning, reward processing, and action selection. We report adolescent neural processing differences in the DS, a region generally associated more with learning than reward processing in adults. Specifically, adolescents, but not adults, had a large proportion of neurons in the DS that activated in anticipation of reward. More similar response patterns were observed in NAc of the two age groups. DS single-unit activity differences were found despite similar local field potential oscillations. This study demonstrates that in adolescents, a region critically involved in learning and habit formation is highly responsive to reward. It thus suggests a mechanism for how rewards might shape adolescent behavior differently, and for their increased vulnerabilities to affective disorders.

Reduced caudate volume and enhanced striatal-DMN integration in chess experts

The superior capability of chess experts largely depends on quick automatic processing skills which are considered to be mediated by the caudate nucleus. We asked whether continued practice or rehearsal of the skill over a long period of time can lead to structural changes in this region. We found that, comparing to novice controls, grandmaster and master level Chinese chess players (GM/Ms), who had a mean period of over 10years of tournament and training practice, exhibited significant smaller gray-matter volume in the bilateral caudate nuclei. When these regions were used as seeds in functional connectivity analysis in resting-state fMRI, significantly enhanced integration was found in GM/Ms between the caudate and the default mode network (DMN), a constellation of brain areas important for goal-directed cognitive performance and theory of mind. These findings demonstrate the structural changes in the caudate nucleus in response to its extensive engagement in chess problem solving, and its enhanced functional integration with widely distributed circuitry to better support high-level cognitive control of behavior.

Transgenic expression of ZBP1 in neurons suppresses cocaine-associated conditioning

To directly address whether regulating mRNA localization can influence animal behavior, we created transgenic mice that conditionally express Zipcode Binding Protein 1 (ZBP1) in a subset of neurons in the brain. ZBP1 is an RNA-binding protein that regulates the localization, as well as translation and stability of target mRNAs in the cytoplasm. We took advantage of the absence of ZBP1 expression in the mature brain to examine the effect of expressing ZBP1 on animal behavior. We constructed a transgene conditionally expressing a GFP-ZBP1 fusion protein in a subset of forebrain neurons and compared cocaine-cued place conditioning in these mice versus noninduced littermates. Transgenic ZBP1 expression resulted in impaired place conditioning relative to nonexpressing littermates, and acutely repressing expression of the transgene restored normal cocaine conditioning. To gain insight into the molecular changes that accounted for this change in behavior, we identified mRNAs that specifically immunoprecipitated with transgenic ZBP1 protein from the brains of these mice. These data suggest that RNA-binding proteins can be used as a tool to identify the post-transcriptional regulation of gene expression in the establishment and function of neural circuits involved in addiction behaviors.

Transitive inference in adults with autism spectrum disorders

Individuals with autism spectrum disorders (ASDs) exhibit intact rote learning with impaired generalization. A transitive inference paradigm, involving training on four sequentially presented stimulus pairs containing overlapping items, with subsequent testing on two novel pairs, was used to investigate this pattern of learning in 27 young adults with ASDs and 31 matched neurotypical individuals (TYPs). On the basis of findings about memory and neuropathology, we hypothesized that individuals with ASDs would use a relational flexibility/conjunctive strategy reliant on an intact hippocampus, versus an associative strength/value transfer strategy requiring intact interactions between the prefrontal cortex and the striatum. Hypotheses were largely confirmed. ASDs demonstrated reduced interference from intervening pairs in early training; only TYPs formed a serial position curve by test; and ASDs exhibited impairments on the novel test pair consisting of end items with intact performance on the inner test pair. However, comparable serial position curves formed for both groups by the end of the first block.

Satiating effects of cocaine are controlled by dopamine actions in the nucleus accumbens core

Intravenous cocaine intake in laboratory animals is characterized by periods of apparent drug satiety between regularly spaced earned injections. The reinforcing properties of cocaine are linked primarily to dopaminergic neurotransmission in the shell and not the core of nucleus accumbens. To determine whether the satiating effects of cocaine are similarly mediated, we perfused dopamine receptor agonists into the core or the shell during intravenous cocaine self-administrations by rats. Neither D1-type (SKF38393) nor D2-type (quinpirole) agonist was effective when given alone. However, a combination of the two agonists perfused into the core but not the shell significantly increased the time between cocaine self-injections, decreasing the amount of earned intake. Together with previous findings, the current data suggest that the satiating and reinforcing effects of cocaine are mediated by different ventral striatal output neurons.

A neurocomputational approach to obsessive-compulsive disorder

A recent article shows that a change in a single parameter in a neural-network model of brain dynamics leads to repetitive behaviors that resist termination and towards which the network tends. These findings may have implications for obsessive-compulsive disorder and are consistent with evidence of glutamatergic hyperactivity in this disorder.

Natural rewards, neuroplasticity, and non-drug addictions

There is a high degree of overlap between brain regions involved in processing natural rewards and drugs of abuse. "Non-drug" or "behavioral" addictions have become increasingly documented in the clinic, and pathologies include compulsive activities such as shopping, eating, exercising, sexual behavior, and gambling. Like drug addiction, non-drug addictions manifest in symptoms including craving, impaired control over the behavior, tolerance, withdrawal, and high rates of relapse. These alterations in behavior suggest that plasticity may be occurring in brain regions associated with drug addiction. In this review, I summarize data demonstrating that exposure to non-drug rewards can alter neural plasticity in regions of the brain that are affected by drugs of abuse. Research suggests that there are several similarities between neuroplasticity induced by natural and drug rewards and that, depending on the reward, repeated exposure to natural rewards might induce neuroplasticity that either promotes or counteracts addictive behavior.

Bidirectional plasticity in striatonigral synapses: a switch to balance direct and indirect basal ganglia pathways

There is no hypothesis to explain how direct and indirect basal ganglia (BG) pathways interact to reach a balance during the learning of motor procedures. Both pathways converge in the substantia nigra pars reticulata (SNr) carrying the result of striatal processing. Unfortunately, the mechanisms that regulate synaptic plasticity in striatonigral (direct pathway) synapses are not known. Here, we used electrophysiological techniques to describe dopamine D(1)-receptor-mediated facilitation in striatonigral synapses in the context of its interaction with glutamatergic inputs, probably coming from the subthalamic nucleus (STN) (indirect pathway) and describe a striatonigral cannabinoid-dependent long-term synaptic depression (LTD). It is shown that striatonigral afferents exhibit D(1)-receptor-mediated facilitation of synaptic transmission when NMDA receptors are inactive, a phenomenon that changes to cannabinoid-dependent LTD when NMDA receptors are active. This interaction makes SNr neurons become coincidence-detector switching ports: When inactive, NMDA receptors lead to a dopamine-dependent enhancement of direct pathway output, theoretically facilitating movement. When active, NMDA receptors result in LTD of the same synapses, thus decreasing movement. We propose that SNr neurons, working as logical gates, tune the motor system to establish a balance between both BG pathways, enabling the system to choose appropriate synergies for movement learning and postural support.

Differential regulation of motor control and response to dopaminergic drugs by D1R and D2R neurons in distinct dorsal striatum subregions

The dorsal striatum is critically involved in a variety of motor behaviours, including regulation of motor activity, motor skill learning and motor response to psychostimulant and neuroleptic drugs, but contribution of D(2)R-striatopallidal and D(1)R-striatonigral neurons in the dorsomedial (DMS, associative) and dorsolateral (DLS, sensorimotor) striatum to distinct functions remains elusive. To delineate cell type-specific motor functions of the DMS or the DLS, we selectively ablated D(2)R- and D(1)R-expressing striatal neurons with spatial resolution. We found that associative striatum exerts a population-selective control over locomotion and reactivity to novelty, striatopallidal and striatonigral neurons inhibiting and stimulating exploration, respectively. Further, DMS-striatopallidal neurons are involved only in early motor learning whereas gradual motor skill acquisition depends on striatonigral neurons in the sensorimotor striatum. Finally, associative striatum D(2)R neurons are required for the cataleptic effect of the typical neuroleptic drug haloperidol and for amphetamine motor response sensitization. Altogether, these data provide direct experimental evidence for cell-specific topographic functional organization of the dorsal striatum.

Impaired formation of stimulus-response, but not action-outcome, associations in rats with methamphetamine-induced neurotoxicity

Methamphetamine (METH) induces neurotoxic changes, including partial striatal dopamine depletions, which are thought to contribute to cognitive dysfunction in rodents and humans. The dorsal striatum is implicated in action-outcome (A-O) and stimulus-response (S-R) associations underlying instrumental learning. Thus, the present study examined the long-term consequences of METH-induced neurotoxicity on A-O and S-R associations underlying appetitive instrumental behavior. Rats were pretreated with saline or a neurotoxic regimen of METH (4 × 7.5-10 mg/kg). Rats trained on random ratio (RR) or random interval (RI) schedules of reinforcement were then subjected to outcome devaluation or contingency degradation, followed by an extinction test. All rats then were killed, and brains removed for determination of striatal dopamine loss. The results show that: (1) METH pretreatment induced a partial 45-50% decrease in striatal dopamine tissue content in dorsomedial and dorsolateral striatum; (2) METH-induced neurotoxicity did not alter acquisition of instrumental behavior on either RR or RI schedules; (3) outcome devaluation and contingency degradation similarly decreased responding in saline- and METH-pretreated rats trained on the RR schedule, suggesting intact A-O associations guiding behavior; (4) outcome devaluation after training on the RI schedule decreased extinction responding only in METH-pretreated rats, suggesting impaired S-R associations. Overall, these data suggest that METH-induced neurotoxicity, possibly due to impairment of the function of dorsolateral striatal circuitry, may decrease cognitive flexibility by impairing the ability to automatize behavioral patterns.

A review of the relation of aerobic fitness and physical activity to brain structure and function in children

A growing number of schools have increasingly de-emphasized the importance of providing physical activity opportunities during the school day, despite emerging research that illustrates the deleterious relationship between low levels of aerobic fitness and neurocognition in children. Accordingly, a brief review of studies that link fitness-related differences in brain structure and brain function to cognitive abilities is provided herein. Overall, the extant literature suggests that childhood aerobic fitness is associated with higher levels of cognition and differences in regional brain structure and function. Indeed, it has recently been found that aerobic fitness level even predicts cognition over time. Given the paucity of work in this area, several avenues for future investigations are also highlighted.

Automated detection of stereotypical motor movements

To overcome problems with traditional methods for measuring stereotypical motor movements in persons with Autism Spectrum Disorders (ASD), we evaluated the use of wireless three-axis accelerometers and pattern recognition algorithms to automatically detect body rocking and hand flapping in children with ASD. Findings revealed that, on average, pattern recognition algorithms correctly identified approximately 90% of stereotypical motor movements repeatedly observed in both laboratory and classroom settings. Precise and efficient recording of stereotypical motor movements could enable researchers and clinicians to systematically study what functional relations exist between these behaviors and specific antecedents and consequences. These measures could also facilitate efficacy studies of behavioral and pharmacologic interventions intended to replace or decrease the incidence or severity of stereotypical motor movements.

A hypothesis for basal ganglia-dependent reinforcement learning in the songbird

Most of our motor skills are not innately programmed, but are learned by a combination of motor exploration and performance evaluation, suggesting that they proceed through a reinforcement learning (RL) mechanism. Songbirds have emerged as a model system to study how a complex behavioral sequence can be learned through an RL-like strategy. Interestingly, like motor sequence learning in mammals, song learning in birds requires a basal ganglia (BG)-thalamocortical loop, suggesting common neural mechanisms. Here, we outline a specific working hypothesis for how BG-forebrain circuits could utilize an internally computed reinforcement signal to direct song learning. Our model includes a number of general concepts borrowed from the mammalian BG literature, including a dopaminergic reward prediction error and dopamine-mediated plasticity at corticostriatal synapses. We also invoke a number of conceptual advances arising from recent observations in the songbird. Specifically, there is evidence for a specialized cortical circuit that adds trial-to-trial variability to stereotyped cortical motor programs, and a role for the BG in "biasing" this variability to improve behavioral performance. This BG-dependent "premotor bias" may in turn guide plasticity in downstream cortical synapses to consolidate recently learned song changes. Given the similarity between mammalian and songbird BG-thalamocortical circuits, our model for the role of the BG in this process may have broader relevance to mammalian BG function.

Extensive training and hippocampus or striatum lesions: effect on place and response strategies

The hippocampus has been linked to spatial navigation and the striatum to response learning. The current study focuses on how these brain regions continue to interact when an animal is very familiar with the task and the environment and must continuously switch between navigation strategies. Rats were trained to solve a plus maze using a place or a response strategy on different trials within a testing session. A room cue (illumination) was used to indicate which strategy should be used on a given trial. After extensive training, animals underwent dorsal hippocampus, dorsal lateral striatum or sham lesions. As expected hippocampal lesions predominantly caused impairment on place but not response trials. Striatal lesions increased errors on both place and response trials. Competition between systems was assessed by determining error type. Pre-lesion and sham animals primarily made errors to arms associated with the wrong (alternative) strategy, this was not found after lesions. The data suggest a qualitative change in the relationship between hippocampal and striatal systems as a task is well learned. During acquisition the two systems work in parallel, competing with each other. After task acquisition, the two systems become more integrated and interdependent. The fact that with extensive training (as something becomes a "habit"), behaviors become dependent upon the dorsal lateral striatum has been previously shown. The current findings indicate that dorsal lateral striatum involvement occurs even when the behavior is spatial and continues to require hippocampal processing.

Parallel contributions of cerebellar, striatal and M1 mechanisms to motor sequence learning

When learning a new motor sequence, we must execute the correct order of movements while simultaneously optimizing sensorimotor parameters such as trajectory, timing, velocity and force. Neurophysiological studies in animals and humans have identified the major brain regions involved in sequence learning, including the motor cortex (M1), basal ganglia (BG) and cerebellum. Current models link these regions to different stages of learning (early vs. late) or different components of performance (spatial vs. sensorimotor). At the same time, research in motor control has given rise to the concept that internal models at different levels of the motor system may contribute to learning. The goal of this review is to develop a new framework for motor sequence learning that combines stage and component models within the context of internal models. To do this, we review behavioral and neuroimaging studies in humans and neurophysiological studies in animals. Based on this evidence, we present a model proposing that sequence learning is underwritten by parallel, interacting processes, including internal model formation and sequence representation, that are instantiated in specific cerebellar, BG or M1 mechanisms depending on task demands and the stage of learning. The striatal system learns predictive stimulus-response associations and is critical for motor chunking. The role of the cerebellum is to acquire the optimal internal model for sequence performance in a particular context, and to contribute to error correction and control of on-going movement. M1 acts to store the representation of a learned sequence, likely as part of a distributed network including the parietal lobe and premotor cortex.

Stress increases behavioral resistance to extinction

Behavioral persistence is required to reach a goal but may impede adaptations to changing environments. Given the well-documented effects of stress on learning and memory processes, we asked here whether stress affects the persistence of behavior. Participants were exposed to stress or a control condition before they learned an instrumental action to gain a food reward. During learning, we presented several extinction blocks in which the food reward was not presented. Stress rendered participants' responding shortly after initial learning insensitive to the extinction procedure. Overall learning curves remained unaffected. Thus, the present findings suggest that stress increases the resistance of behavior to extinction. The cause of the behavioral persistence after stress may be its habitual form.

The neurocognitive connection between physical activity and eating behaviour

As obesity rates increase worldwide, healthcare providers require methods to instill the lifestyle behaviours necessary for sustainable weight loss. Designing effective weight-loss interventions requires an understanding of how these behaviours are elicited, how they relate to each other and whether they are supported by common neurocognitive mechanisms. This may provide valuable insights to optimize existing interventions and develop novel approaches to weight control. Researchers have begun to investigate the neurocognitive underpinnings of eating behaviour and the impact of physical activity on cognition and the brain. This review attempts to bring these somewhat disparate, yet interrelated lines of literature together in order to examine a hypothesis that eating behaviour and physical activity share a common neurocognitive link. The link pertains to executive functions, which rely on brain circuits located in the prefrontal cortex. These advanced cognitive processes are of limited capacity and undergo relentless strain in the current obesogenic environment. The increased demand on these neurocognitive resources as well as their overuse and/or impairment may facilitate impulses to over-eat, contributing to weight gain and obesity. This impulsive eating drive may be counteracted by physical activity due to its enhancement of neurocognitive resources for executive functions and goal-oriented behaviour. By enhancing the resources that facilitate 'top-down' inhibitory control, increased physical activity may help compensate and suppress the hedonic drive to over-eat. Understanding how physical activity and eating behaviours interact on a neurocognitive level may help to maintain a healthy lifestyle in an obesogenic environment.

Preservation of function in Parkinson's disease: what's learning got to do with it?

Dopamine denervation gives rise to abnormal corticostriatal plasticity; however, its role in the symptoms and progression of Parkinson's disease (PD) has not been articulated or incorporated into current clinical models. The 'integrative selective gain' framework proposed here integrates dopaminergic mechanisms known to modulate basal ganglia throughput into a single conceptual framework: (1) synaptic weights, the neural instantiation of accumulated experience and skill modulated by dopamine-dependent plasticity and (2) system gain, the operating parameters of the basal ganglia, modulated by dopamine's on-line effects on cell excitability, glutamatergic transmission and the balance between facilitatory and inhibitory pathways. Within this framework and based on recent work, a hypothesis is presented that prior synaptic weights and established skills can facilitate motor performance and preserve function despite diminished dopamine; however, dopamine denervation induces aberrant corticostriatal plasticity that degrades established synaptic weights and replaces them with inappropriate, inhibitory learning that inverts the function of the basal ganglia resulting in 'anti-optimization' of motor performance. Consequently, mitigating aberrant corticostriatal plasticity represents an important therapeutic objective, as reflected in the long-duration response to levodopa, reinterpreted here as the correction of aberrant learning. It is proposed that viewing aberrant corticostriatal plasticity and learning as a provisional endophenotype of PD would facilitate investigation of this hypothesis.

Sequential learning and rule abstraction in Bengalese finches

The Bengalese finch (Lonchura striata var. domestica) is a species of songbird. Males sing courtship songs with complex note-to-note transition rules, while females discriminate these songs when choosing their mate. The present study uses serial reaction time (RT) to examine the characteristics of the Bengalese finches’ sequential behaviours beyond song production. The birds were trained to produce the sequence with an “A–B–A” structure. After the RT to each key position was determined to be stable, we tested the acquisition of the trained sequential response by presenting novel and random three-term sequences (random test). We also examined whether they could abstract the embedded rule in the trained sequence and apply it to the novel test sequence (abstract test). Additionally, we examined rule abstraction through example training by increasing the number of examples in baseline training from 1 to 5. When considered as (gender) groups, training with 5 examples resulted in no statistically significant differences in the abstract tests, while statistically significant differences were observed in the random tests, suggesting that the male birds learned the trained sequences and transferred the abstract structure they had learned during the training trials. Individual data indicated that males, as opposed to females, were likely to learn the motor pattern of the sequence. The results are consistent with observations that males learn to produce songs with complex sequential rules, whereas females do not.

Neural systems analysis of decision making during goal-directed navigation

The ability to make adaptive decisions during goal-directed navigation is a fundamental and highly evolved behavior that requires continual coordination of perceptions, learning and memory processes, and the planning of behaviors. Here, a neurobiological account for such coordination is provided by integrating current literatures on spatial context analysis and decision-making. This integration includes discussions of our current understanding of the role of the hippocampal system in experience-dependent navigation, how hippocampal information comes to impact midbrain and striatal decision making systems, and finally the role of the striatum in the implementation of behaviors based on recent decisions. These discussions extend across cellular to neural systems levels of analysis. Not only are key findings described, but also fundamental organizing principles within and across neural systems, as well as between neural systems functions and behavior, are emphasized. It is suggested that studying decision making during goal-directed navigation is a powerful model for studying interactive brain systems and their mediation of complex behaviors.

Molecular substrates of action control in cortico-striatal circuits

The purpose of this review is to describe the molecular mechanisms in the striatum that mediate reward-based learning and action control during instrumental conditioning. Experiments assessing the neural bases of instrumental conditioning have uncovered functional circuits in the striatum, including dorsal and ventral striatal sub-regions, involved in action-outcome learning, stimulus-response learning, and the motivational control of action by reward-associated cues. Integration of dopamine (DA) and glutamate neurotransmission within these striatal sub-regions is hypothesized to enable learning and action control through its role in shaping synaptic plasticity and cellular excitability. The extracellular signal regulated kinase (ERK) appears to be particularly important for reward-based learning and action control due to its sensitivity to combined DA and glutamate receptor activation and its involvement in a range of cellular functions. ERK activation in striatal neurons is proposed to have a dual role in both the learning and performance factors that contribute to instrumental conditioning through its regulation of plasticity-related transcription factors and its modulation of intrinsic cellular excitability. Furthermore, perturbation of ERK activation by drugs of abuse may give rise to behavioral disorders such as addiction.

Basal Ganglia disorders associated with imbalances in the striatal striosome and matrix compartments

The striatum is composed principally of GABAergic, medium spiny striatal projection neurons (MSNs) that can be categorized based on their gene expression, electrophysiological profiles, and input–output circuits. Major subdivisions of MSN populations include (1) those in ventromedial and dorsolateral striatal regions, (2) those giving rise to the direct and indirect pathways, and (3) those that lie in the striosome and matrix compartments. The first two classificatory schemes have enabled advances in understanding of how basal ganglia circuits contribute to disease. However, despite the large number of molecules that are differentially expressed in the striosomes or the extra-striosomal matrix, and the evidence that these compartments have different input–output connections, our understanding of how this compartmentalization contributes to striatal function is still not clear. A broad view is that the matrix contains the direct and indirect pathway MSNs that form parts of sensorimotor and associative circuits, whereas striosomes contain MSNs that receive input from parts of limbic cortex and project directly or indirectly to the dopamine-containing neurons of the substantia nigra, pars compacta. Striosomes are widely distributed within the striatum and are thought to exert global, as well as local, influences on striatal processing by exchanging information with the surrounding matrix, including through interneurons that send processes into both compartments. It has been suggested that striosomes exert and maintain limbic control over behaviors driven by surrounding sensorimotor and associative parts of the striatal matrix. Consistent with this possibility, imbalances between striosome and matrix functions have been reported in relation to neurological disorders, including Huntington’s disease, L-DOPA-induced dyskinesias, dystonia, and drug addiction. Here, we consider how signaling imbalances between the striosomes and matrix might relate to symptomatology in these disorders.

A critical review of habit learning and the Basal Ganglia

The current paper briefly outlines the historical development of the concept of habit learning and discusses its relationship to the basal ganglia. Habit learning has been studied in many different fields of neuroscience using different species, tasks, and methodologies, and as a result it has taken on a wide range of definitions from these various perspectives. We identify five common but not universal, definitional features of habit learning: that it is inflexible, slow or incremental, unconscious, automatic, and insensitive to reinforcer devaluation. We critically evaluate for each of these how it has been defined, its utility for research in both humans and non-human animals, and the evidence that it serves as an accurate description of basal ganglia function. In conclusion, we propose a multi-faceted approach to habit learning and its relationship to the basal ganglia, emphasizing the need for formal definitions that will provide directions for future research.

Dopaminergic modulation of striatal neurons, circuits, and assemblies

In recent years, there has been a great deal of progress toward understanding the role of the striatum and dopamine in action selection. The advent of new animal models and the development of optical techniques for imaging and stimulating select neuronal populations have provided the means by which identified synapses, cells, and circuits can be reliably studied. This review attempts to summarize some of the key advances in this broad area, focusing on dopaminergic modulation of intrinsic excitability and synaptic plasticity in canonical microcircuits in the striatum as well as recent work suggesting that there are neuronal assemblies within the striatum devoted to particular types of computation and possibly action selection.

The Pull of the Past

To identify the factors that disrupt and maintain habit performance, two field experiments tested the conditions under which people eat out of habit, leading them to resist motivational influences. Habitual popcorn eaters at a cinema were minimally influenced by their hunger or how much they liked the food, and they ate equal amounts of stale and fresh popcorn. Yet, mechanisms of automaticity influenced habit performance: Participants ate out of habit, regardless of freshness, only when currently in the context associated with past performance (i.e., a cinema; Study 1) and only when eating in a way that allowed them to automatically execute the response cued by that context (i.e., eating with their dominant hand; Study 2). Across all conditions, participants with weaker cinema-popcorn-eating habits ate because of motivations such as liking for the popcorn. The findings reveal how habits resist conflicting motives and provide insight into promising mechanisms of habit change.

Deep Brain Stimulation in Area LC Controllably Triggers Auditory Phantom Percepts

BACKGROUND:

Tinnitus is predominantly viewed as the consequence of dysfunctional hyperactivity, plastic change, or synchronized oscillations in the central auditory system. An alternative to the current auditory-centric view of auditory phantom perception is the basal ganglia-centric view. Recent electrical stimulation experiments in area LC, a locus of the caudate nucleus positioned at its anterior body, has shown loudness modulation of existing tinnitus percepts.

OBJECTIVE:

To demonstrate that auditory phantoms are gated by the dorsal striatum.

METHODS:

Electrical stimulation in area LC via a deep brain stimulation lead was performed in 6 interactive adult subjects (3 with and 3 without chronic tinnitus) undergoing surgery to treat movement disorders. Tinnitus loudness was rated on a 0 to 10 scale, sound quality was described, and localization was referenced to 1 or both ears.

RESULTS:

Short-term area LC stimulation triggered new phantom tones, clicks, and frequency modulated sounds in 5 subjects and altered sound quality of an existing tinnitus percept in 1 subject. The results of this study indicate that perceptual awareness of auditory phantoms is contingent on satisfying a permission condition controlled by the dorsal striatum. Potential auditory phantoms are not automatically gated to reach perceptual awareness. A phantom percept gate control model is proposed.

CONCLUSION:

Neuromodulation of area LC can trigger temporary gate dysfunction and reversibly release new phantoms for conscious awareness. Restoration of restrictive dorsal striatal gate function to treat problematic phantom percepts may be realized by adopting long-term area LC neuromodulation and choosing optimal stimulation parameters.

Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex

Knowledge about hypothetical outcomes from unchosen actions is beneficial only when such outcomes can be correctly attributed to specific actions. Here we show that during a simulated rock-paper-scissors game, rhesus monkeys can adjust their choice behaviors according to both actual and hypothetical outcomes from their chosen and unchosen actions, respectively. In addition, neurons in both dorsolateral prefrontal cortex and orbitofrontal cortex encoded the signals related to actual and hypothetical outcomes immediately after they were revealed to the animal. Moreover, compared to the neurons in the orbitofrontal cortex, those in the dorsolateral prefrontal cortex were more likely to change their activity according to the hypothetical outcomes from specific actions. Conjunctive and parallel coding of multiple actions and their outcomes in the prefrontal cortex might enhance the efficiency of reinforcement learning and also contribute to their context-dependent memory.

Targeting neuronal populations of the striatum

The striatum is critically involved in motor and motivational functions. The dorsal striatum, caudate–putamen, is primarily implicated in motor control and the learning of habits and skills, whereas the ventral striatum, the nucleus accumbens, is essential for motivation and drug reinforcement. The GABA medium-sized spiny neurons (MSNs, about 95% of striatal neurons), which are targets of the cerebral cortex and the midbrain dopaminergic neurons, form two pathways. The dopamine D₁ receptor-positive (D₁R) striatonigral MSNs project to the medial globus pallidus and substantia nigra pars reticulata (direct pathway) and co-express D₁R and substance P, whereas dopamine D₂ receptor-positive (D₂R) striatopallidal MSNs project to the lateral globus pallidus (indirect pathway) and co-express D₂R, adenosine A_2A receptor (A_2AR) and enkephalin (Enk). The specific role of the two efferent pathways in motor and motivational control remained poorly understood until recently. Indeed, D₁R striatonigral and D₂R striatopallidal neurons, are intermingled and morphologically indistinguishable, and, hence, cannot be functionally dissociated with techniques such as chemical lesions or surgery. In view of the still debated respective functions of projection D₂R striatopallidal and D₁R striatonigral neurons and striatal interneurons, both in motor control and learning but also in more cognitive processes such as motivation, the present review sum up the development of new models and techniques (bacterial artificial chromosome transgenesis, optogenetic, viral transgenesis) allowing the selective targeting of these striatal neuronal populations in adult animal brain to understand their specific roles.

Stress effects on memory: an update and integration

It is well known that stressful experiences may affect learning and memory processes. Less clear is the exact nature of these stress effects on memory: both enhancing and impairing effects have been reported. These opposite effects may be explained if the different time courses of stress hormone, in particular catecholamine and glucocorticoid, actions are taken into account. Integrating two popular models, we argue here that rapid catecholamine and non-genomic glucocorticoid actions interact in the basolateral amygdala to shift the organism into a 'memory formation mode' that facilitates the consolidation of stressful experiences into long-term memory. The undisturbed consolidation of these experiences is then promoted by genomic glucocorticoid actions that induce a 'memory storage mode', which suppresses competing cognitive processes and thus reduces interference by unrelated material. Highlighting some current trends in the field, we further argue that stress affects learning and memory processes beyond the basolateral amygdala and hippocampus and that stress may pre-program subsequent memory performance when it is experienced during critical periods of brain development.

Past, present, and future of the pathophysiological model of the Basal Ganglia

The current model of basal ganglia (BG) was introduced two decades ago and has settled most of our current understanding of BG function and dysfunction. Extensive research efforts have been carried out in recent years leading to further refinement and understanding of the normal and diseased BG. Several questions, however, are yet to be resolved. This short review provides a synopsis of the evolution of thought regarding the pathophysiological model of the BG and summarizes the main recent findings and additions to this field of research. We have also tried to identify major challenges that need to be addressed and resolved in the near future. Detailed accounts and state-of-the-art developments concerning research on the BG are provided in the articles that make up this Special Issue.

Quantitative mapping of cocaine-induced ΔFosB expression in the striatum of male and female rats

ΔFosB plays a critical role in drug-induced long-term changes in the brain. In the current study, we evaluated locomotor activity in male and female rats treated with saline or cocaine for 2 weeks and quantitatively mapped ΔFosB expression in the dorsal striatum and nucleus accumbens of each animal by using an anti-FosB antibody that recognizes ΔFosB isoforms preferentially. Behavioral analysis showed that while there was little difference between males and females that sensitized to cocaine, nonsensitizing rats showed a large sex difference. Nonsensitizing males showed low behavioral activation in response to cocaine on the first day of treatment, and their activity remained low. In contrast, nonsensitizing females showed high activation on the first day of treatment and their activity remained high. Western blot and immunohistochemical analyses indicated that basal levels of ΔFosB were higher in the nucleus accumbens than the dorsal striatum, but that the effect of cocaine on ΔFosB was greater in the dorsal striatum. Immunostaining showed that the effect of cocaine in both the dorsal striatum and nucleus accumbens was primarily to increase the intensity of ΔFosB immunoreactivity in individual neurons, rather than to increase the number of cells that express ΔFosB. Detailed mapping of ΔFosB-labeled nuclei showed that basal ΔFosB levels were highest in the medial portion of the dorsal striatum and dorsomedial accumbens, particularly adjacent to the lateral ventricle, whereas the cocaine-induced increase in ΔFosB was most pronounced in the lateral dorsal striatum, where basal ΔFosB expression was lowest. Sex differences in ΔFosB expression were small and independent of cocaine treatment. We discuss implications of the sex difference in locomotor activation and regionally-specific ΔFosB induction by cocaine.