The role of efference copy in striatal learning

Dynamics of striatal action selection and reinforcement learning

Spiny projection neurons (SPNs) in dorsal striatum are often proposed as a locus of reinforcement learning in the basal ganglia. Here, we identify and resolve a fundamental inconsistency between striatal reinforcement learning models and known SPN synaptic plasticity rules. Direct-pathway (dSPN) and indirect-pathway (iSPN) neurons, which promote and suppress actions, respectively, exhibit synaptic plasticity that reinforces activity associated with elevated or suppressed dopamine release. We show that iSPN plasticity prevents successful learning, as it reinforces activity patterns associated with negative outcomes. However, this pathological behavior is reversed if functionally opponent dSPNs and iSPNs, which promote and suppress the current behavior, are simultaneously activated by efferent input following action selection. This prediction is supported by striatal recordings and contrasts with prior models of SPN representations. In our model, learning and action selection signals can be multiplexed without interference, enabling learning algorithms beyond those of standard temporal difference models.

Exploration biases forelimb reaching strategies

The brain can generate actions, such as reaching to a target, using different movement strategies. We investigate how such strategies are learned in a task where perched head-fixed mice learn to reach to an invisible target area from a set start position using a joystick. This can be achieved by learning to move in a specific direction or to a specific endpoint location. As mice learn to reach the target, they refine their variable joystick trajectories into controlled reaches, which depend on the sensorimotor cortex. We show that individual mice learned strategies biased to either direction- or endpoint-based movements. This endpoint/direction bias correlates with spatial directional variability with which the workspace was explored during training. Model-free reinforcement learning agents can generate both strategies with similar correlation between variability during training and learning bias. These results provide evidence that reinforcement of individual exploratory behavior during training biases the reaching strategies that mice learn.

Dopamine lesions alter the striatal encoding of single-limb gait

The striatum serves an important role in motor control, and neurons in this area encode the body's initiation, cessation, and speed of locomotion. However, it remains unclear whether the same neurons also encode the step-by-step rhythmic motor patterns of individual limbs that characterize gait. By combining high-speed video tracking, electrophysiology, and optogenetic tagging, we found that a sizable population of both D1 and D2 receptor expressing medium spiny projection neurons (MSNs) were phase-locked to the gait cycle of individual limbs in mice. Healthy animals showed balanced limb phase-locking between D1 and D2 MSNs, while dopamine depletion led to stronger phase-locking in D2 MSNs. These findings indicate that striatal neurons represent gait on a single-limb and step basis, and suggest that elevated limb phase-locking of D2 MSNs may underlie some of the gait impairments associated with dopamine loss.

Layer 5 Intratelencephalic Neurons in the Motor Cortex Stably Encode Skilled Movement

The primary motor cortex (M1) and the dorsal striatum play a critical role in motor learning and the retention of learned behaviors. Motor representations of corticostriatal ensembles emerge during motor learning. In the coordinated reorganization of M1 and the dorsal striatum for motor learning, layer 5a (L5a) which connects M1 to the ipsilateral and contralateral dorsal striatum, should be a key layer. Although M1 L5a neurons represent movement-related activity in the late stage of learning, it is unclear whether the activity is retained as a memory engram. Here, using Tlx3-Cre male transgenic mice, we conducted two-photon calcium imaging of striatum-projecting L5a intratelencephalic (IT) neurons in forelimb M1 during late sessions of a self-initiated lever-pull task and in sessions after 6 d of nontraining following the late sessions. We found that trained male animals exhibited stable motor performance before and after the nontraining days. At the same time, we found that M1 L5a IT neurons strongly represented the well-learned forelimb movement but not uninstructed orofacial movements. A subset of M1 L5a IT neurons consistently coded the well-learned forelimb movement before and after the nontraining days. Inactivation of M1 IT neurons after learning impaired task performance when the lever was made heavier or when the target range of the pull distance was narrowed. These results suggest that a subset of M1 L5a IT neurons continuously represent skilled movement after learning and serve to fine-tune the kinematics of well-learned movement.SIGNIFICANCE STATEMENT Motor memory persists even when it is not used for a while. IT neurons in L5a of the M1 gradually come to represent skilled forelimb movements during motor learning. However, it remains to be determined whether these changes persist over a long period and how these neurons contribute to skilled movements. Here, we show that a subset of M1 L5a IT neurons retain information for skilled forelimb movements even after nontraining days. Furthermore, suppressing the activity of these neurons during skilled forelimb movements impaired behavioral stability and adaptability. Our results suggest the importance of M1 L5a IT neurons for tuning skilled forelimb movements over a long period.

Motor cortex is required for flexible but not automatic motor sequences

How motor cortex contributes to motor sequence execution is much debated, with studies supporting disparate views. Here we probe the degree to which motor cortex's engagement depends on task demands, specifically whether its role differs for highly practiced, or 'automatic', sequences versus flexible sequences informed by external events. To test this, we trained rats to generate three-element motor sequences either by overtraining them on a single sequence or by having them follow instructive visual cues. Lesioning motor cortex revealed that it is necessary for flexible cue-driven motor sequences but dispensable for single automatic behaviors trained in isolation. However, when an automatic motor sequence was practiced alongside the flexible task, it became motor cortex-dependent, suggesting that subcortical consolidation of an automatic motor sequence is delayed or prevented when the same sequence is produced also in a flexible context. A simple neural network model recapitulated these results and explained the underlying circuit mechanisms. Our results critically delineate the role of motor cortex in motor sequence execution, describing the condition under which it is engaged and the functions it fulfills, thus reconciling seemingly conflicting views about motor cortex's role in motor sequence generation.

Exploration biases how forelimb reaches to a spatial target are learned

The brain can learn to generate actions, such as reaching to a target, using different movement strategies. Understanding how different variables bias which strategies are learned to produce such a reach is important for our understanding of the neural bases of movement. Here we introduce a novel spatial forelimb target task in which perched head-fixed mice learn to reach to a circular target area from a set start position using a joystick. These reaches can be achieved by learning to move into a specific direction or to a specific endpoint location. We find that mice gradually learn to successfully reach the covert target. With time, they refine their initially exploratory complex joystick trajectories into controlled targeted reaches. The execution of these controlled reaches depends on the sensorimotor cortex. Using a probe test with shifting start positions, we show that individual mice learned to use strategies biased to either direction or endpoint-based movements. The degree of endpoint learning bias was correlated with the spatial directional variability with which the workspace was explored early in training. Furthermore, we demonstrate that reinforcement learning model agents exhibit a similar correlation between directional variability during training and learned strategy. These results provide evidence that individual exploratory behavior during training biases the control strategies that mice use to perform forelimb covert target reaches.

Generative models of birdsong learning link circadian fluctuations in song variability to changes in performance

Learning skilled behaviors requires intensive practice over days, months, or years. Behavioral hallmarks of practice include exploratory variation and long-term improvements, both of which can be impacted by circadian processes. During weeks of vocal practice, the juvenile male zebra finch transforms highly variable and simple song into a stable and precise copy of an adult tutor's complex song. Song variability and performance in juvenile finches also exhibit circadian structure that could influence this long-term learning process. In fact, one influential study reported juvenile song regresses towards immature performance overnight, while another suggested a more complex pattern of overnight change. However, neither of these studies thoroughly examined how circadian patterns of variability may structure the production of more or less mature songs. Here we relate the circadian dynamics of song maturation to circadian patterns of song variation, leveraging a combination of data-driven approaches. In particular we analyze juvenile singing in learned feature space that supports both data-driven measures of song maturity and generative developmental models of song production. These models reveal that circadian fluctuations in variability lead to especially regressive morning variants even without overall overnight regression, and highlight the utility of data-driven generative models for untangling these contributions.

Distributed processing for value-based choice by prelimbic circuits targeting anterior-posterior dorsal striatal subregions in male mice

Fronto-striatal circuits have been implicated in cognitive control of behavioral output for social and appetitive rewards. The functional diversity of prefrontal cortical populations is strongly dependent on their synaptic targets, with control of motor output mediated by connectivity to dorsal striatum. Despite evidence for functional diversity along the anterior-posterior striatal axis, it is unclear how distinct fronto-striatal sub-circuits support value-based choice. Here we found segregated prefrontal populations defined by anterior/posterior dorsomedial striatal target. During a feedback-based 2-alternative choice task, single-photon imaging revealed circuit-specific representations of task-relevant information with prelimbic neurons targeting anterior DMS (PL::A-DMS) robustly modulated during choices and negative outcomes, while prelimbic neurons targeting posterior DMS (PL::P-DMS) encoded internal representations of value and positive outcomes contingent on prior choice. Consistent with this distributed coding, optogenetic inhibition of PL::A-DMS circuits strongly impacted choice monitoring and responses to negative outcomes while inhibition of PL::P-DMS impaired task engagement and strategies following positive outcomes. Together our data uncover PL populations engaged in distributed processing for value-based choice.

Models of psychedelic drug action: modulation of cortical-subcortical circuits

Classic psychedelic drugs such as psilocybin and lysergic acid diethylamide (LSD) have recaptured the imagination of both science and popular culture, and may have efficacy in treating a wide range of psychiatric disorders. Human and animal studies of psychedelic drug action in the brain have demonstrated the involvement of the serotonin 2A (5-HT2A) receptor and the cerebral cortex in acute psychedelic drug action, but different models have evolved to try to explain the impact of 5-HT2A activation on neural systems. Two prominent models of psychedelic drug action (the cortico-striatal thalamo-cortical, or CSTC, model and relaxed beliefs under psychedelics, or REBUS, model) have emphasized the role of different subcortical structures as crucial in mediating psychedelic drug effects. We describe these models and discuss gaps in knowledge, inconsistencies in the literature and extensions of both models. We then introduce a third circuit-level model involving the claustrum, a thin strip of grey matter between the insula and the external capsule that densely expresses 5-HT2A receptors (the cortico-claustro-cortical, or CCC, model). In this model, we propose that the claustrum entrains canonical cortical network states, and that psychedelic drugs disrupt 5-HT2A-mediated network coupling between the claustrum and the cortex, leading to attenuation of canonical cortical networks during psychedelic drug effects. Together, these three models may explain many phenomena of the psychedelic experience, and using this framework, future research may help to delineate the functional specificity of each circuit to the action of both serotonergic and non-serotonergic hallucinogens.

Investigation of motor self-monitoring deficits in schizophrenia with passivity experiences using a novel modified joint position matching paradigm

Numerous studies have identified deficits in the self-monitoring system that are associated with schizophrenia. However, the tasks used in the few previous studies generally involved complex cognitive processes and rarely compared between patients with and without passivity experiences (PE). Here, we examined the deficits in internal motor predictive representation in patients with and without PE, and in healthy controls using a novel paradigm which involved minimal cognitive processes. All participants completed a modified joint position matching (mJPM) task, in which they were required to replicate a voluntary, a passive verbally-cued, and a passive tactile-cued movement under blinded conditions. The absolute difference between the target spot and replicated spot was measured and compared. We hypothesised that if there was a failure in the internal motor predictive representation, patients with PEs would replicate less accurately in the voluntary condition, relative to passive conditions while the healthy controls would be more accurate, and, therefore, significant interactions between groups and conditions would be revealed. Both healthy controls and patients without PEs replicated more accurately in the voluntary condition compared with the passive conditions. The patients with PEs were less accurate in the voluntary condition compared with the passive tactile condition. A significant interaction was observed between patients with vs. without PEs × voluntary vs. passive tactile conditions. The findings suggested the relationship between deficits in motor self-monitoring in the prediction process and PEs, thus showing the need to highlight the link between motor performance and PEs.

Inhibition of miR-128 Enhances Vocal Sequence Organization in Juvenile Songbirds

The molecular mechanisms underlying learned vocal communication are not well characterized. This is a major barrier for developing treatments for conditions affecting social communication, such as autism spectrum disorder (ASD). Our group previously generated an activity-dependent gene expression network in the striatopallidal song control nucleus, Area X, in adult zebra finches to identify master regulators of learned vocal behavior. This dataset revealed that the two host genes for microRNA-128, ARPP21 and R3HDM1, are among the top genes whose expression correlates to how much birds sing. Here we examined whether miR-128 itself is behaviorally regulated in Area X and found that its levels decline with singing. We hypothesized that reducing miR-128 during the critical period for vocal plasticity would enhance vocal learning. To test this, we bilaterally injected an antisense miR-128 construct (AS miR-128) or a control scrambled sequence into Area X at post-hatch day 30 (30 d) using sibling-matched experimental and control pupils. The juveniles were then returned to their home cage and raised with their tutors. Strikingly, inhibition of miR-128 in young birds enhanced the organization of learned vocal sequences. Tutor and pupil stereotypy scores were positively correlated, though the correlation was stronger between tutors and control pupils compared to tutors and AS miR-128 pupils. This difference was driven by AS miR-128 pupils achieving higher stereotypy scores despite their tutors’ lower syntax scores. AS miR-128 birds with tutors on the higher end of the stereotypy spectrum were more likely to produce songs with faster tempos relative to sibling controls. Our results suggest that low levels of miR-128 facilitate vocal sequence stereotypy. By analogy, reducing miR-128 could enhance the capacity to learn to speak in patients with non-verbal ASD. To our knowledge, this study is the first to directly link miR-128 to learned vocal communication and provides support for miR-128 as a potential therapeutic target for ASD.

Nigrostriatal dopamine signals sequence-specific action-outcome prediction errors

Dopamine has been suggested to encode cue-reward prediction errors during Pavlovian conditioning, signaling discrepancies between actual versus expected reward predicted by the cues.^1-5 While this theory has been widely applied to reinforcement learning concerning instrumental actions, whether dopamine represents action-outcome prediction errors and how it controls sequential behavior remain largely unknown. The vast majority of previous studies examining dopamine responses primarily have used discrete reward-predictive stimuli,^1-15 whether Pavlovian conditioned stimuli for which no action is required to earn reward or explicit discriminative stimuli that essentially instruct an animal how and when to respond for reward. Here, by training mice to perform optogenetic intracranial self-stimulation, we examined how self-initiated goal-directed behavior influences nigrostriatal dopamine transmission during single and sequential instrumental actions, in behavioral contexts with minimal overt changes in the animal's external environment. We found that dopamine release evoked by direct optogenetic stimulation was dramatically reduced when delivered as the consequence of the animal's own action, relative to non-contingent passive stimulation. This dopamine suppression generalized to food rewards was specific to the reinforced action, was temporally restricted to counteract the expected outcome, and exhibited sequence-selectivity consistent with hierarchical control of sequential behavior. These findings demonstrate that nigrostriatal dopamine signals sequence-specific prediction errors in action-outcome associations, with fundamental implications for reinforcement learning and instrumental behavior in health and disease.

Frequency-Specific Effects of Galvanic Vestibular Stimulation on Response-Time Performance in Parkinson's Disease

Background: Galvanic vestibular stimulation (GVS) is being increasingly explored as a non-invasive brain stimulation technique to treat symptoms in Parkinson's disease (PD). To date, behavioral GVS effects in PD have been explored with only two stimulus types, direct current and random noise (RN). The interaction between GVS effects and anti-parkinsonian medication is unknown. In the present study, we designed multisine (ms) stimuli and investigated the effects of ms and RN GVS on motor response time. In comparison to the RN stimulus, the ms stimuli contained sinusoidal components only at a set of desired frequencies and the phases were optimized to improve participants' comfort. We hypothesized GVS motor effects were a function of stimulation frequency, and specifically, that band-limited ms-GVS would result in better motor performance than conventionally used broadband RN-GVS.

Materials and Methods: Eighteen PD patients (PDMOFF/PDMON: off-/on-levodopa medication) and 20 healthy controls (HC) performed a simple reaction time task while receiving sub-threshold GVS. Each participant underwent nine stimulation conditions: off-stimulation, RN (4–200 Hz), ms-θ (4–8 Hz), ms-α (8–13 Hz), ms-β (13–30 Hz), ms-γ (30–50 Hz), ms-h1 (50–100 Hz), ms-h2 (100–150 Hz), and ms-h3 (150–200 Hz).

Results: The ms-γ resulted in shorter response time (RPT) in both PDMOFF and HC groups compared with the RN. In addition, the RPT of the PDMOFF group decreased during the ms-β while the RPT of the HC group decreased during the ms-α, ms-h1, ms-h2, and ms-h3. There was considerable inter-subject variability in the optimum stimulus type, although the frequency range tended to fall within 8–100 Hz. Levodopa medication significantly reduced the baseline RPT of the PD patients. In contrast to the off-medication state, GVS did not significantly change RPT of the PD patients in the on-medication state.

Conclusions: Using band-limited ms-GVS, we demonstrated that the GVS frequency for the best RPT varied considerably across participants and was >30 Hz for half of the PDMOFF patients. Moreover, dopaminergic medication was found to influence GVS effects in PD patients. Our results indicate the common “one-size-fits-all” RN approach is suboptimal for PD, and therefore personalized stimuli aiming to address this variability is warranted to improve GVS effects.

Plasticity in Cervical Motor Circuits following Spinal Cord Injury and Rehabilitation

Simple Summary

Spinal cord injury results in a decreased quality of life and impacts hundreds of thousands of people in the US alone. This review discusses the underlying cellular mechanisms of injury and the concurrent therapeutic hurdles that impede recovery. It then describes the phenomena of neural plasticity—the nervous system’s ability to change. The primary focus of the review is on the impact of cervical spinal cord injury on control of the upper limbs. The neural plasticity that occurs without intervention is discussed, which shows new connections growing around the injury site and the involvement of compensatory movements. Rehabilitation-driven neural plasticity is shown to have the ability to guide connections to create more normal functions. Various novel stimulation and recording technologies are outlined for their role in further improving rehabilitative outcomes and gains in independence. Finally, the importance of sensory input, an often-overlooked aspect of motor control, is shown in driving neural plasticity. Overall, this review seeks to delineate the historical and contemporary research into neural plasticity following injury and rehabilitation to guide future studies.

Abstract

Neuroplasticity is a robust mechanism by which the central nervous system attempts to adapt to a structural or chemical disruption of functional connections between neurons. Mechanical damage from spinal cord injury potentiates via neuroinflammation and can cause aberrant changes in neural circuitry known as maladaptive plasticity. Together, these alterations greatly diminish function and quality of life. This review discusses contemporary efforts to harness neuroplasticity through rehabilitation and neuromodulation to restore function with a focus on motor recovery following cervical spinal cord injury. Background information on the general mechanisms of plasticity and long-term potentiation of the nervous system, most well studied in the learning and memory fields, will be reviewed. Spontaneous plasticity of the nervous system, both maladaptive and during natural recovery following spinal cord injury is outlined to provide a baseline from which rehabilitation builds. Previous research has focused on the impact of descending motor commands in driving spinal plasticity. However, this review focuses on the influence of physical therapy and primary afferent input and interneuron modulation in driving plasticity within the spinal cord. Finally, future directions into previously untargeted primary afferent populations are presented.

Computational models of the "active self" and its disturbances in schizophrenia

The notion that self-disorders are at the root of the emergence of schizophrenia rather than a symptom of the disease, is getting more traction in the cognitive sciences. This is in line with philosophical approaches that consider an enactive self, constituted through action and interaction with the environment. We thereby analyze different definitions of the self and evaluate various computational theories lending to these ideas. Bayesian and predictive processing are promising approaches for computational modeling of the "active self". We evaluate their implementation and challenges in computational psychiatry and cognitive developmental robotics. We describe how and why embodied robotic systems provide a valuable tool in psychiatry to assess, validate, and simulate mechanisms of self-disorders. Specifically, mechanisms involving sensorimotor learning, prediction, and self-other distinction, can be assessed with artificial agents. This link can provide essential insights to the formation of the self and new avenues in the treatment of psychiatric disorders.

Mesoscale cortical dynamics reflect the interaction of sensory evidence and temporal expectation during perceptual decision-making

How sensory evidence is transformed across multiple brain regions to influence behavior remains poorly understood. We trained mice in a visual change detection task designed to separate the covert antecedents of choices from activity associated with their execution. Wide-field calcium imaging across the dorsal cortex revealed fundamentally different dynamics of activity underlying these processes. Although signals related to execution of choice were widespread, fluctuations in sensory evidence in the absence of overt motor responses triggered a confined activity cascade, beginning with transient modulation of visual cortex and followed by sustained recruitment of the secondary and primary motor cortex. Activation of the motor cortex by sensory evidence was modulated by animals' expectation of when the stimulus was likely to change. These results reveal distinct activation timescales of specific cortical areas by sensory evidence during decision-making and show that recruitment of the motor cortex depends on the interaction of sensory evidence and temporal expectation.

Efference copy in kinesthetic perception: a copy of what is it?

A number of notions in the fields of motor control and kinesthetic perception have been used without clear definitions. In this review, we consider definitions for efference copy, percept, and sense of effort based on recent studies within the physical approach, which assumes that the neural control of movement is based on principles of parametric control and involves defining time-varying profiles of spatial referent coordinates for the effectors. The apparent redundancy in both motor and perceptual processes is reconsidered based on the principle of abundance. Abundance of efferent and afferent signals is viewed as the means of stabilizing both salient action characteristics and salient percepts formalized as stable manifolds in high-dimensional spaces of relevant elemental variables. This theoretical scheme has led recently to a number of novel predictions and findings. These include, in particular, lower accuracy in perception of variables produced by elements involved in a multielement task compared with the same elements in single-element tasks, dissociation between motor and perceptual effects of muscle coactivation, force illusions induced by muscle vibration, and errors in perception of unintentional drifts in performance. Taken together, these results suggest that participation of efferent signals in perception frequently involves distorted copies of actual neural commands, particularly those to antagonist muscles. Sense of effort is associated with such distorted efferent signals. Distortions in efference copy happen spontaneously and can also be caused by changes in sensory signals, e.g., those produced by muscle vibration.

Distortions of the Efferent Copy during Force Perception: A Study of Force Drifts and Effects of Muscle Vibration

We used a finger force matching task to explore the role of efferent signals in force perception. Healthy, young participants performed accurate force production tasks at different force levels with the index and middle fingers of one hand (task-hand). They received visual feedback during an early part of each trial only. After the feedback was turned off, the force drifted toward lower magnitudes. After 5 s of the drift, the participants matched the force with the same finger pair of the other hand (match-hand). The match-hand consistently overshot the task-hand force by a magnitude invariant over the initial force levels. During force matching, both hands were lifted and lowered smoothly to estimate their referent coordinate (RC) and apparent stiffness values. These trials were performed without muscle vibration and under vibration applied to the finger/hand flexors or extensors of the task-hand or match-hand. Effects of vibration were seen in the match-hand only; they were the same during vibration of flexors and extensors. We interpret the vibration-induced effects as consequences of using distorted copies of the central commands to the task-hand during force matching. In particular, using distorted copies of the RC for the antagonist muscle group could account for the differences between the task-hand and match-hand. We conclude that efferent signals may be distorted before their participation in the perceptual process. Such distortions emerge spontaneously and may be amplified by the response of sensory endings to muscle vibration combined over both agonist and antagonist muscle groups.

Identification of FoxP circuits involved in locomotion and object fixation in Drosophila

The FoxP family of transcription factors is necessary for operant self-learning, an evolutionary conserved form of motor learning. The expression pattern, molecular function and mechanisms of action of the Drosophila FoxP orthologue remain to be elucidated. By editing the genomic locus of FoxP with CRISPR/Cas9, we find that the three different FoxP isoforms are expressed in neurons, but not in glia and that not all neurons express all isoforms. Furthermore, we detect FoxP expression in, e.g. the protocerebral bridge, the fan-shaped body and in motor neurons, but not in the mushroom bodies. Finally, we discover that FoxP expression during development, but not adulthood, is required for normal locomotion and landmark fixation in walking flies. While FoxP expression in the protocerebral bridge and motor neurons is involved in locomotion and landmark fixation, the FoxP gene can be excised from dorsal cluster neurons and mushroom-body Kenyon cells without affecting these behaviours.

BOLD differences normally attributed to inhibitory control predict symptoms, not task-directed inhibitory control in ADHD

Background

Altered brain activity that has been observed in attention deficit hyperactivity disorder (ADHD) while performing cognitive control tasks like the stop signal task (SST) has generally been interpreted as reflecting either weak (under-active) or compensatory (over-active) versions of the same functions as in healthy controls. If so, then regional activities that correlate with the efficiency of inhibitory control (i.e. stop signal reaction time, SSRT) in healthy subjects should also correlate with SSRT in ADHD. Here we test the alternate hypothesis that BOLD (blood-oxygen-level-dependent) differences might instead reflect the redirection of neural processing resources normally used for task-directed inhibitory control, towards actively managing symptomatic behaviour. If so, then activities that correlate with SSRT in TD should instead correlate with inattentive and hyperactive symptoms in ADHD.

Methods

We used fMRI (functional magnetic resonance imaging) in 14 typically developing (TD) and 14 ADHD adolescents performing the SST, and in a replication sample of 14 healthy adults. First, we identified significant group BOLD differences during all phases of activity in the SST (i.e. warning, response, reactive inhibition, error detection and post-error slowing). Next, we correlated these phases of activity with SSRT in TD and with SSRT, inattentive and hyperactive symptom scores in ADHD. We then identified whole brain significant correlations in regions of significant group difference in activity.

Results

Only three regions of significant group difference were correlated with SSRT in TD and replication groups (left and right inferior frontal gyri (IFG) during error detection and hypothalamus during post-error slowing). Consistent with regions of altered activity managing symptomatic behaviour instead of task-directed behaviour, left IFG correlated with greater inattentive score, right IFG correlated with lower hyperactive score and hypothalamus correlated with greater inattentive score and oppositely correlated with SSRT compared to TD.

Conclusions

Stimuli that elicit task-directed integration of neural processing in healthy subjects instead appear to be directing integrated function towards managing symptomatic behaviour in ADHD. The ability of the current approach to determine whether altered neural activities reflect comparable functions in ADHD and control groups has broad implications for the development and monitoring of therapeutic interventions.

Contributions of the basal ganglia to action sequence learning and performance

Animals engage in intricately woven and choreographed action sequences that are constructed from trial-and-error learning. The mechanisms by which the brain links together individual actions which are later recalled as fluid chains of behavior are not fully understood, but there is broad consensus that the basal ganglia play a crucial role in this process. This paper presents a comprehensive review of the role of the basal ganglia in action sequencing, with a focus on whether the computational framework of reinforcement learning can capture key behavioral features of sequencing and the neural mechanisms that underlie them. While a simple neurocomputational model of reinforcement learning can capture key features of action sequence learning, this model is not sufficient to capture goal-directed control of sequences or their hierarchical representation. The hierarchical structure of action sequences, in particular, poses a challenge for building better models of action sequencing, and it is in this regard that further investigations into basal ganglia information processing may be informative.

A Reservoir Computing Model of Reward-Modulated Motor Learning and Automaticity

Reservoir computing is a biologically inspired class of learning algorithms in which the intrinsic dynamics of a recurrent neural network are mined to produce target time series. Most existing reservoir computing algorithms rely on fully supervised learning rules, which require access to an exact copy of the target response, greatly reducing the utility of the system. Reinforcement learning rules have been developed for reservoir computing, but we find that they fail to converge on complex motor tasks. Current theories of biological motor learning pose that early learning is controlled by dopamine-modulated plasticity in the basal ganglia that trains parallel cortical pathways through unsupervised plasticity as a motor task becomes well learned. We developed a novel learning algorithm for reservoir computing that models the interaction between reinforcement and unsupervised learning observed in experiments. This novel learning algorithm converges on simulated motor tasks on which previous reservoir computing algorithms fail and reproduces experimental findings that relate Parkinson's disease and its treatments to motor learning. Hence, incorporating biological theories of motor learning improves the effectiveness and biological relevance of reservoir computing models.

Schizophrenia, Dopamine and the Striatum: From Biology to Symptoms

The mesolimbic hypothesis has been a central dogma of schizophrenia for decades, positing that aberrant functioning of midbrain dopamine projections to limbic regions causes psychotic symptoms. Recently, however, advances in neuroimaging techniques have led to the unanticipated finding that dopaminergic dysfunction in schizophrenia is greatest within nigrostriatal pathways, implicating the dorsal striatum in the pathophysiology and calling into question the mesolimbic theory. At the same time our knowledge of striatal anatomy and function has progressed, suggesting new mechanisms via which striatal dysfunction may contribute to the symptoms of schizophrenia. This Review draws together these developments, to explore what they mean for our understanding of the pathophysiology, clinical manifestations, and treatment of the disorder.

Techniques for characterising the mesostriatal dopamine system, both in humans and animal models, have advanced significantly over the past decade.

In vivo imaging studies in schizophrenia patients demonstrate that dopaminergic dysfunction in schizophrenia is greatest in nigrostriatal as opposed to mesolimbic pathways.

Better understanding of striatal structure and function has enhanced our insight into the neurobiological basis of psychotic symptoms.

The role of other neurotransmitters in modulating striatal dopamine function merits further exploration, and modulating these neurotransmitter systems has potential to offer new therapeutic strategies.

Building a state space for song learning

The songbird system has shed light on how the brain produces precisely timed behavioral sequences, and how the brain implements reinforcement learning (RL). RL is a powerful strategy for learning what action to produce in each state, but requires a unique representation of the states involved in the task. Songbird RL circuitry is thought to operate using a representation of each moment within song syllables, consistent with the sparse sequential bursting of neurons in premotor cortical nucleus HVC. However, such sparse sequences are not present in very young birds, which sing highly variable syllables of random lengths. Here, we review and expand upon a model for how the songbird brain could construct latent sequences to support RL, in light of new data elucidating connections between HVC and auditory cortical areas. We hypothesize that learning occurs via four distinct plasticity processes: 1) formation of 'tutor memory' sequences in auditory areas; 2) formation of appropriately-timed latent HVC sequences, seeded by inputs from auditory areas spontaneously replaying the tutor song; 3) strengthening, during spontaneous replay, of connections from HVC to auditory neurons of corresponding timing in the 'tutor memory' sequence, aligning auditory and motor representations for subsequent song evaluation; and 4) strengthening of connections from premotor neurons to motor output neurons that produce the desired sounds, via well-described song RL circuitry.

Adult zebra finches rehearse highly variable song patterns during sleep

Brain activity during sleep is fairly ubiquitous and the best studied possible function is a role in memory consolidation, including motor memory. One suggested mechanism of how neural activity effects these benefits is through reactivation of neurons in patterns resembling those of the preceding experience. The specific patterns of motor activation replayed during sleep are largely unknown for any system. Brain areas devoted to song production in the songbird brain exhibit spontaneous song-like activity during sleep, but single cell neural recordings did not permit detection of the specific song patterns. We have now discovered that this sleep activation can be detected in the muscles of the vocal organ, thus providing a unique window into song-related brain activity at night. We show that male zebra finches (Taeniopygia guttata) frequently exhibit spontaneous song-like activity during the night, but that the fictive song patterns are highly variable and uncoordinated compared to the highly stereotyped day-time song production. This substantial variability is not consistent with the idea that night-time activity replays day-time experiences for consolidation. Although the function of this frequent activation is unknown, it may represent a mechanism for exploring motor space or serve to generate internal error signals that help maintain the high stereotypy of day-time song. In any case, the described activity supports the emerging insight that brain activity during sleep may serve a variety of functions.

Functional Relevance of Different Basal Ganglia Pathways Investigated in a Spiking Model with Reward Dependent Plasticity

The brain enables animals to behaviorally adapt in order to survive in a complex and dynamic environment, but how reward-oriented behaviors are achieved and computed by its underlying neural circuitry is an open question. To address this concern, we have developed a spiking model of the basal ganglia (BG) that learns to dis-inhibit the action leading to a reward despite ongoing changes in the reward schedule. The architecture of the network features the two pathways commonly described in BG, the direct (denoted D1) and the indirect (denoted D2) pathway, as well as a loop involving striatum and the dopaminergic system. The activity of these dopaminergic neurons conveys the reward prediction error (RPE), which determines the magnitude of synaptic plasticity within the different pathways. All plastic connections implement a versatile four-factor learning rule derived from Bayesian inference that depends upon pre- and post-synaptic activity, receptor type, and dopamine level. Synaptic weight updates occur in the D1 or D2 pathways depending on the sign of the RPE, and an efference copy informs upstream nuclei about the action selected. We demonstrate successful performance of the system in a multiple-choice learning task with a transiently changing reward schedule. We simulate lesioning of the various pathways and show that a condition without the D2 pathway fares worse than one without D1. Additionally, we simulate the degeneration observed in Parkinson's disease (PD) by decreasing the number of dopaminergic neurons during learning. The results suggest that the D1 pathway impairment in PD might have been overlooked. Furthermore, an analysis of the alterations in the synaptic weights shows that using the absolute reward value instead of the RPE leads to a larger change in D1.

The functional logic of corticostriatal connections

Unidirectional connections from the cortex to the matrix of the corpus striatum initiate the cortico-basal ganglia (BG)-thalamocortical loop, thought to be important in momentary action selection and in longer-term fine tuning of behavioural repertoire; a discrete set of striatal compartments, striosomes, has the complementary role of registering or anticipating reward that shapes corticostriatal plasticity. Re-entrant signals traversing the cortico-BG loop impact predominantly frontal cortices, conveyed through topographically ordered output channels; by contrast, striatal input signals originate from a far broader span of cortex, and are far more divergent in their termination. The term 'disclosed loop' is introduced to describe this organisation: a closed circuit that is open to outside influence at the initial stage of cortical input. The closed circuit component of corticostriatal afferents is newly dubbed 'operative', as it is proposed to establish the bid for action selection on the part of an incipient cortical action plan; the broader set of converging corticostriatal afferents is described as contextual. A corollary of this proposal is that every unit of the striatal volume, including the long, C-shaped tail of the caudate nucleus, should receive a mandatory component of operative input, and hence include at least one area of BG-recipient cortex amongst the sources of its corticostriatal afferents. Individual operative afferents contact twin classes of GABAergic striatal projection neuron (SPN), distinguished by their neurochemical character, and onward circuitry. This is the basis of the classic direct and indirect pathway model of the cortico-BG loop. Each pathway utilises a serial chain of inhibition, with two such links, or three, providing positive and negative feedback, respectively. Operative co-activation of direct and indirect SPNs is, therefore, pictured to simultaneously promote action, and to restrain it. The balance of this rival activity is determined by the contextual inputs, which summarise the external and internal sensory environment, and the state of ongoing behavioural priorities. Notably, the distributed sources of contextual convergence upon a striatal locus mirror the transcortical network harnessed by the origin of the operative input to that locus, thereby capturing a similar set of contingencies relevant to determining action. The disclosed loop formulation of corticostriatal and subsequent BG loop circuitry, as advanced here, refines the operating rationale of the classic model and allows the integration of more recent anatomical and physiological data, some of which can appear at variance with the classic model. Equally, it provides a lucid functional context for continuing cellular studies of SPN biophysics and mechanisms of synaptic plasticity.

Lack of reliability in the disruption of cognitive performance following exposure to protons

A series of three replications were run to determine the reliability with which exposure to protons produces a disruption of cognitive performance, using a novel object recognition task and operant responding on an ascending fixed-ratio task. For the first two replications, rats were exposed to head-only exposures to 1000 MeV/n protons at the NASA Space Radiation Laboratory. For the third replication, subjects were given head-only or whole-body exposures to both 1000 and 150 MeV/n protons. The results were characterized by a lack of consistency in the effects of exposure to protons on the performance of these cognitive tasks, both within and between replications. The factors that might influence the lack of consistency and the implications for exploratory class missions are discussed.

The neostriatum: two entities, one structure?

The striosome (or patch) was first identified with anatomical techniques as neurons organized in a three-dimensional labyrinth inserted in and interdigitating the rest of neostriatum: the matrix. Striosome and matrix rapidly became known as two neuronal compartments expressing different biochemical markers, embryonic development and afferent and efferent connectivity. In spite of extensive intrinsic neuronal axonal and dendritic extensions supposed to exchange information between matrix and striosomes, evidence suggested the presence of independent areas. Here, we report that indeed these two areas do not exchange synaptic information. We used genetic expression of channel rhodopsin 2 carried by adeno-associated virus serotype 10 (AAVrh10) that only expresses in neurons of the matrix compartment. Whole-cell patch-clamp recordings of matrix neurons activated by light pulses consistently produced inhibitory postsynaptic currents (IPSCs), but the same manipulation did not evoke IPSCs in striosome neurons. The matrix contains both direct and indirect striatal output pathways. By targeting striatal matrix expression of designer receptors exclusively activated by a designer drug (DREADD) hM3di carried by AAVrh10, we were able to inhibit the matrix neuronal compartment of the dorsolateral striatum during performance of a learned single-pellet reach-to-grasp task. As expected, inhibition of matrix neurons by systemic administration of DREADD agonist clozapine-n-oxide interfered with performance of the learned task.

Childhood cognitive development as a skill

Theories view childhood development as being either driven by structural maturation of the brain or being driven by skill-learning. It is hypothesized here that working memory (WM) development during childhood is partly driven by training effects in the environment, and that similar neural mechanisms underlie training-induced plasticity and childhood development. In particular, the functional connectivity of a fronto-parietal network is suggested to be associated with WM capacity. The striatum, dopamine receptor D2 (DRD2) activity, and corticostriatal white-matter tracts, on the other hand, seem to be more important for plasticity and change of WM capacity during both training and development. In this view, the development of WM capacity during childhood partly involves the same mechanisms as skill-learning.

Drosophila FoxP mutants are deficient in operant self-learning

Intact function of the Forkhead Box P2 (FOXP2) gene is necessary for normal development of speech and language. This important role has recently been extended, first to other forms of vocal learning in animals and then also to other forms of motor learning. The homology in structure and in function among the FoxP gene members raises the possibility that the ancestral FoxP gene may have evolved as a crucial component of the neural circuitry mediating motor learning. Here we report that genetic manipulations of the single Drosophila orthologue, dFoxP, disrupt operant self-learning, a form of motor learning sharing several conceptually analogous features with language acquisition. Structural alterations of the dFoxP locus uncovered the role of dFoxP in operant self-learning and habit formation, as well as the dispensability of dFoxP for operant world-learning, in which no motor learning occurs. These manipulations also led to subtle alterations in the brain anatomy, including a reduced volume of the optic glomeruli. RNAi-mediated interference with dFoxP expression levels copied the behavioral phenotype of the mutant flies, even in the absence of mRNA degradation. Our results provide evidence that motor learning and language acquisition share a common ancestral trait still present in extant invertebrates, manifest in operant self-learning. This 'deep' homology probably traces back to before the split between vertebrate and invertebrate animals.

The receptive field is dead. Long live the receptive field?

Advances in experimental techniques, including behavioral paradigms using rich stimuli under closed loop conditions and the interfacing of neural systems with external inputs and outputs, reveal complex dynamics in the neural code and require a revisiting of standard concepts of representation. High-throughput recording and imaging methods along with the ability to observe and control neuronal subpopulations allow increasingly detailed access to the neural circuitry that subserves neural representations and the computations they support. How do we harness theory to build biologically grounded models of complex neural function?

Computational neuroscience: beyond the local circuit

Computational neuroscience has focused largely on the dynamics and function of local circuits of neuronal populations dedicated to a common task, such as processing a common sensory input, storing its features in working memory, choosing between a set of options dictated by controlled experimental settings or generating the appropriate actions. Most of current circuit models suggest mechanisms for computations that can be captured by networks of simplified neurons connected via simple synaptic weights. In this article I review the progress of this approach and its limitations. It is argued that new experimental techniques will yield data that might challenge the present paradigms in that they will (1) demonstrate the computational importance of microscopic structural and physiological complexity and specificity; (2) highlight the importance of models of large brain structures engaged in a variety of tasks; and (3) reveal the necessity of coupling the neuronal networks to chemical and environmental variables.