Motor sequence learning in primate: role of the D2 receptor in movement chunking during consolidation

Reward positivity affects temporal interval production in a continuous timing task

The neural circuits of reward processing and interval timing (including the perception and production of temporal intervals) are functionally intertwined, suggesting that it might be possible for momentary reward processing to influence subsequent timing behavior. Previous animal and human studies have mainly focused on the effect of reward on interval perception, whereas its impact on interval production is less clear. In this study, we examined whether feedback, as an example of performance-contingent reward, biases interval production. We recorded EEG from 20 participants while they engaged in a continuous drumming task with different realistic tempos (1728 trials per participant). Participants received color-coded feedback after each beat about whether they were correct (on time) or incorrect (early or late). Regression-based EEG analysis was used to unmix the rapid occurrence of a feedback response called the reward positivity (RewP), which is traditionally observed in more slow-paced tasks. Using linear mixed modeling, we found that RewP amplitude predicted timing behavior for the upcoming beat. This performance-biasing effect of the RewP was interpreted as reflecting the impact of fluctuations in reward-related anterior cingulate cortex activity on timing, and the necessity of continuous paradigms to make such observations was highlighted.

Computational insights on asymmetrical D1 and D2 receptor-mediated chunking: implications for OCD and Schizophrenia

Repetitive thoughts and motor programs including perseveration are bridge symptoms characteristic of obsessive compulsive disorder (OCD), schizophrenia and in the co-morbid overlap of these conditions. The above pathologies are sensitive to altered activation and kinetics of dopamine D 1 and D 2 receptors that differently influence sequence learning and recall. Recognizing start and stop elements of motor and cognitive behaviors has crucial importance. During chunking, frequent components of temporal strings are concatenated into single units. We extended a published computational model (Asabuki et al. 2018), where two populations of neurons are connected and simulated in a reservoir computing framework. These neural pools were adopted to represent D1 and D2 striatal neuronal populations. We investigated how specific neural and striatal circuit parameters can influence start/stop signaling and found that asymmetric intra-network connection probabilities, synaptic weights and differential time constants may contribute to signaling of start/stop elements within learned sequences. Asymmetric coupling between the striatal D 1 and D 2 neural populations was also demonstrated to be beneficial. Our modeling results predict that dynamical differences between the two dopaminergic striatal populations and the interaction between them may play complementary roles in chunk boundary signaling. Start and stop dichotomies can arise from the larger circuit dynamics as well, since neural and intra-striatal connections only partially support a clear division of labor.

Primates chunk simultaneously-presented memoranda

Though much research has characterized both the behavior and electrophysiology of spatial memory for single targets in non-human primates, we know much less about how multiple memoranda are handled. Multiple memoranda may interact in the brain, affecting the underlying representations. Mnemonic resources are famously limited, so items may compete for "space" in memory or may be encoded cooperatively or in a combined fashion. Understanding the mode of interaction will inform future neural studies. As a first step, we quantified interactions during a multi-item spatial memory task. Two monkeys were shown 1-4 target locations. After a delay, the targets reappeared with a novel target and the animal was rewarded for fixating the novel target. Targets could appear either all at once (simultaneous) or with intervening delays (sequential). We quantified the degree of interaction with memory rate correlations. We found that simultaneously presented targets were stored cooperatively while sequentially presented targets were stored independently. These findings demonstrate how interaction between concurrently memorized items depends on task context. Future studies of multi-item memory would be served by designing experiments to either control or measure the mode of this interaction.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

Understanding Sensitive Period Effects in Musical Training

Adult ability in complex cognitive domains, including music, is commonly thought of as the product of gene-environment interactions, where genetic predispositions influence and are modulated by experience, resulting in the final phenotypic expression. Recently, however, the important contribution of maturation to gene-environment interactions has become better understood. Thus, the timing of exposure to specific experience, such as music training, has been shown to produce long-term impacts on adult behaviour and the brain. Work from our lab and others shows that musical training before the ages of 7-9 enhances performance on musical tasks and modifies brain structure and function, sometimes in unexpected ways. The goal of this paper is to present current evidence for sensitive period effects for musical training in the context of what is known about brain maturation and to present a framework that integrates genetic, environmental and maturational influences on the development of musical skill. We believe that this framework can also be applied more broadly to understanding how predispositions, brain development and experience interact.

Explicit motor sequence learning after stroke: a neuropsychological study

Motor learning interacts with and shapes experience-dependent cerebral plasticity. In stroke patients with paresis of the upper limb, motor recovery was proposed to reflect a process of re-learning the lost/impaired skill, which interacts with rehabilitation. However, to what extent stroke patients with hemiparesis may retain the ability of learning with their affected limb remains an unsolved issue, that was addressed by this study. Nineteen patients, with a cerebrovascular lesion affecting the right or the left hemisphere, underwent an explicit motor learning task (finger tapping task, FTT), which was performed with the paretic hand. Eighteen age-matched healthy participants served as controls. Motor performance was assessed during the learning phase (i.e., online learning), as well as immediately at the end of practice, and after 90 min and 24 h (i.e., retention). Results show that overall, as compared to the control group, stroke patients, regardless of the side (left/right) of the hemispheric lesion, do not show a reliable practice-dependent improvement; consequently, no retention could be detected in the long-term (after 90 min and 24 h). The motor learning impairment was associated with subcortical damage, predominantly affecting the basal ganglia; conversely, it was not associated with age, time elapsed from stroke, severity of upper-limb motor and sensory deficits, and the general neurological condition. This evidence expands our understanding regarding the potential of post-stroke motor recovery through motor practice, suggesting a potential key role of basal ganglia, not only in implicit motor learning as previously pointed out, but also in explicit finger tapping motor tasks.

Multiscale Computation and Dynamic Attention in Biological and Artificial Intelligence

Biological and artificial intelligence (AI) are often defined by their capacity to achieve a hierarchy of short-term and long-term goals that require incorporating information over time and space at both local and global scales. More advanced forms of this capacity involve the adaptive modulation of integration across scales, which resolve computational inefficiency and explore-exploit dilemmas at the same time. Research in neuroscience and AI have both made progress towards understanding architectures that achieve this. Insight into biological computations come from phenomena such as decision inertia, habit formation, information search, risky choices and foraging. Across these domains, the brain is equipped with mechanisms (such as the dorsal anterior cingulate and dorsolateral prefrontal cortex) that can represent and modulate across scales, both with top-down control processes and by local to global consolidation as information progresses from sensory to prefrontal areas. Paralleling these biological architectures, progress in AI is marked by innovations in dynamic multiscale modulation, moving from recurrent and convolutional neural networks-with fixed scalings-to attention, transformers, dynamic convolutions, and consciousness priors-which modulate scale to input and increase scale breadth. The use and development of these multiscale innovations in robotic agents, game AI, and natural language processing (NLP) are pushing the boundaries of AI achievements. By juxtaposing biological and artificial intelligence, the present work underscores the critical importance of multiscale processing to general intelligence, as well as highlighting innovations and differences between the future of biological and artificial intelligence.

Resolving the Electroencephalographic Correlates of Rapid Goal-Directed Chunking in the Frontal-Parietal Network

Previous studies have revealed a specific role of the prefrontal-parietal network in rapid goal-directed chunking (RGDC), which dissociates prefrontal activity related to chunking from parietal working memory demands. However, it remains unknown how the prefrontal and parietal cortices collaborate to accomplish RGDC. To this end, a novel experimental design was used that presented Chinese characters in a chunking task, testing eighteen undergraduate students (9 females, mean age = 22.4 years) while recoding the electroencephalogram (EEG). In the experiment, radical-level chunking was accomplished in a timely stringent way (RT = 1485 ms, SD = 371 ms), whereas the stroke-level chunking was accomplished less coherently (RT = 3278 ms, SD = 1083 ms). By comparing the differences between radical-level chunking vs. stroke-level chunking, we were able to dissociate the chunking processes in the radical-level chunking condition within the analyzed time window (-200 to 1300 ms). The chunking processes resulted in an early increase of gamma band synchronization over parietal and occipital cortices, followed by enhanced power in the beta-gamma band (25-38 Hz) over frontal areas. We suggest that the posterior rhythmic activities in the gamma band may underlie the processes that are directly associated with perceptual manipulations of chunking, while the subsequent beta-gamma activation over frontal areas appears to reflect a post-evaluation process such as reinforcement of the selected rules over alternative solutions, which may be an important characteristic of goal-directed chunking.

Reduced striatal dopamine release during motor skill acquisition in Parkinson's disease

Background

Striatal dopamine is functionally important for the acquisition of motor skills. However, it remains controversial as to whether intrinsic processing of motor learning is impaired in patients with Parkinson’s disease (PD), and if yes, whether the impairment is associated with altered striatal dopamine release. Additionally, most neuro-imaging studies of patients with PD have focused on motor sequence learning. In contrast, skill acquisition, specifically, the reconstruction of muscle control of isolated movements, has barely been studied.

Method

In this study, we used a repetitive skill training task to measure the peak acceleration of left thumb movement during a process to achieve fine tuning of motor skill. Using ¹¹C-raclopride (RAC) positron emission tomography, we investigated changes in striatal dopamine levels in two conditions of a skill acquisition task: initial skill training (Day 1) and acquired condition (Day 2) with eight patients with PD and age-matched healthy subjects (HS).

Result

In HS, the mean acceleration of each session improved through repeated training sessions on Day 1. However, in patients with PD, the training-associated increase was less than that for HS, and this suggests that repetitive skill training does not result in the effective improvement of motor performance. The regions of interest (ROI) analysis revealed that the RAC-binding potential (BP) was significantly reduced in the right putamen on Day 1 compared with Day 2 in HS. In patients with PD, BP within the right putamen was unchanged. Further, we found that patients with PD had increased dopamine levels within the right ventral striatum (VST) and right caudate (CAU) on Day 2, which was greater than that in HS. These results suggest the impaired activation of the putamen during skill acquisition in patients with PD and compensated hyperactivation of the VST and CAU for the reduced dopamine release within the dorsal putamen (DPU).

Conclusion

Our findings suggest that patients with PD had insufficiency in the process to improve motor skills. Different patterns of striatal dopamine release are relevant to the impairment of these motor functions in patients with PD, at the early stage of the disease.

A system-level mathematical model of Basal Ganglia motor-circuit for kinematic planning of arm movements

In this paper, a novel system-level mathematical model of the Basal Ganglia (BG) for kinematic planning, is proposed. An arm composed of several segments presents a geometric redundancy. Thus, selecting one trajectory among an infinite number of possible ones requires overcoming redundancy, according to some kinds of optimization. Solving this optimization is assumed to be the function of BG in planning. In the proposed model, first, a mathematical solution of kinematic planning is proposed for movements of a redundant arm in a plane, based on minimizing energy consumption. Next, the function of each part in the model is interpreted as a possible role of a nucleus of BG. Since the kinematic variables are considered as vectors, the proposed model is presented based on the vector calculus. This vector model predicts different neuronal populations in BG which is in accordance with some recent experimental studies. According to the proposed model, the function of the direct pathway is to calculate the necessary rotation of each joint, and the function of the indirect pathway is to control each joint rotation considering the movement of the other joints. In the proposed model, the local feedback loop between Subthalamic Nucleus and Globus Pallidus externus is interpreted as a local memory to store the previous amounts of movements of the other joints, which are utilized by the indirect pathway. In this model, activities of dopaminergic neurons would encode, at short-term, the error between the desired and actual positions of the end-effector. The short-term modulating effect of dopamine on Striatum is also modeled as cross product. The model is simulated to generate the commands of a redundant manipulator. The performance of the model is studied for different reaching movements between 8 points in a plane. Finally, some symptoms of Parkinson's disease such as bradykinesia and akinesia are simulated by modifying the model parameters, inspired by the dopamine depletion.

The effects of inter-trial interval on implicit learning of sequential visual isometric pinch task

Sequential visual isometric pinch task (SVIPT) has been recently used as a visuomotor sequence task in clinical research. The influence of varying intervals between sequenced trials on the acquisition of implicit sequence learning is not yet determined for SVIPT. The aim of this study was to investigate the effects of inter-trial interval (ITI) on implicit motor sequence learning using SVIPT. A total of 32 healthy participants with mean age 31.3 ± 4.5 years participated in this study. Participants were randomly assigned to one of four ITI groups; (1, 2, 3 and 4 s). They were instructed to control their force on a force transducer to reach a number of targets which appeared on the computer screen by changing the pinch force exerted onto the transducer. In this study, outcome measures were movement time, error rate and skill, which were measured before and after training. Our results indicated that motor sequence learning similarly affected various ITIs. Indeed, all participants exhibited same improvement in implicit learning of SVIPT even though the ITIs varied from 1 to 4 s. Our findings suggest that implicit learning of SVIPT is independent of ITI within this range in healthy individuals.

Levodopa medication improves incidental sequence learning in Parkinson's disease

Empirical evidence suggests that levodopa medication used to treat the motor symptoms of Parkinson's disease (PD) may either improve, impair or not affect specific cognitive processes. This evidence led to the 'dopamine overdose' hypothesis that levodopa medication impairs performance on cognitive tasks if they recruit fronto-striatal circuits which are not yet dopamine-depleted in early PD and as a result the medication leads to an excess of dopamine. This hypothesis has been supported for various learning tasks including conditional associative learning, reversal learning, classification learning and intentional deterministic sequence learning, on all of which PD patients demonstrated significantly worse performance when tested on relative to off dopamine medication. Incidental sequence learning is impaired in PD, but how such learning is affected by dopaminergic therapy remains undetermined. The aim of the current study was to investigate the effect of dopaminergic medication on incidental sequence learning in PD. We used a probabilistic serial reaction time task (SRTT), a sequence learning paradigm considered to make the sequence less apparent and more likely to be learned incidentally rather than intentionally. We compared learning by the same group of PD patients (n=15) on two separate occasions following oral administration of levodopa medication (on state) and after overnight withdrawal of medication (off state). Our results demonstrate for the first time that levodopa medication enhances incidental learning of a probabilistic sequence on the serial reaction time task in PD. However, neither group significantly differed from performance of a control group without a neurological disease, which indicates the importance of within group comparisons for identifying deficits. Levodopa medication enhanced incidental learning by patients with PD on a probabilistic sequence learning paradigm even though the patients were not aware of the existence of the sequence or their acquired knowledge. The results suggest a role in acquiring incidental motor sequence learning for dorsal striatal areas strongly affected by dopamine depletion in early PD.

Differentiating Visual from Response Sequencing during Long-term Skill Learning

The dual-system model of sequence learning posits that during early learning there is an advantage for encoding sequences in sensory frames; however, it remains unclear whether this advantage extends to long-term consolidation. Using the serial RT task, we set out to distinguish the dynamics of learning sequential orders of visual cues from learning sequential responses. On each day, most participants learned a new mapping between a set of symbolic cues and responses made with one of four fingers, after which they were exposed to trial blocks of either randomly ordered cues or deterministic ordered cues (12-item sequence). Participants were randomly assigned to one of four groups (n = 15 per group): Visual sequences (same sequence of visual cues across training days), Response sequences (same order of key presses across training days), Combined (same serial order of cues and responses on all training days), and a Control group (a novel sequence each training day). Across 5 days of training, sequence-specific measures of response speed and accuracy improved faster in the Visual group than any of the other three groups, despite no group differences in explicit awareness of the sequence. The two groups that were exposed to the same visual sequence across days showed a marginal improvement in response binding that was not found in the other groups. These results indicate that there is an advantage, in terms of rate of consolidation across multiple days of training, for learning sequences of actions in a sensory representational space, rather than as motoric representations.

The external globus pallidus: progress and perspectives

The external globus pallidus (GPe) of the basal ganglia is in a unique and powerful position to influence processing of motor information by virtue of its widespread projections to all basal ganglia nuclei. Despite the clinical importance of the GPe in common motor disorders such as Parkinson's disease, there is only limited information about its cellular composition and organizational principles. In this review, recent advances in the understanding of the diversity in the molecular profile, anatomy, physiology and corresponding behaviour during movement of GPe neurons are described. Importantly, this study attempts to build consensus and highlight commonalities of the cellular classification based on existing but contentious literature. Additionally, an analysis of the literature concerning the intricate reciprocal loops formed between the GPe and major synaptic partners, including both the striatum and the subthalamic nucleus, is provided. In conclusion, the GPe has emerged as a crucial node in the basal ganglia macrocircuit. While subtleties in the cellular makeup and synaptic connection of the GPe create new challenges, modern research tools have shown promise in untangling such complexity, and will provide better understanding of the roles of the GPe in encoding movements and their associated pathologies.

Novel Analogues of (R)-5-(Methylamino)-5,6-dihydro-4H-imidazo[4,5,1-ij]quinolin-2(1H)-one (Sumanirole) Provide Clues to Dopamine D2/D3 Receptor Agonist Selectivity

Novel 1-, 5-, and 8-substituted analogues of sumanirole (1), a dopamine D2/D3 receptor (D2R/D3R) agonist, were synthesized. Binding affinities at both D2R and D3R were higher when determined in competition with the agonist radioligand [(3)H]7-hydroxy-N,N-dipropyl-2-aminotetralin (7-OH-DPAT) than with the antagonist radioligand [(3)H]N-methylspiperone. Although 1 was confirmed as a D2R-preferential agonist, its selectivity in binding and functional studies was lower than previously reported. All analogues were determined to be D2R/D3R agonists in both GoBRET and mitogenesis functional assays. Loss of efficacy was detected for the N-1-substituted analogues at D3R. In contrast, the N-5-alkyl-substituted analogues, and notably the n-butyl-arylamides (22b and 22c), all showed improved affinity at D2R over 1 with neither a loss of efficacy nor an increase in selectivity. Computational modeling provided a structural basis for the D2R selectivity of 1, illustrating how subtle differences in the highly homologous orthosteric binding site (OBS) differentially affect D2R/D3R affinity and functional efficacy.

Learning of Chunking Sequences in Cognition and Behavior

We often learn and recall long sequences in smaller segments, such as a phone number 858 534 22 30 memorized as four segments. Behavioral experiments suggest that humans and some animals employ this strategy of breaking down cognitive or behavioral sequences into chunks in a wide variety of tasks, but the dynamical principles of how this is achieved remains unknown. Here, we study the temporal dynamics of chunking for learning cognitive sequences in a chunking representation using a dynamical model of competing modes arranged to evoke hierarchical Winnerless Competition (WLC) dynamics. Sequential memory is represented as trajectories along a chain of metastable fixed points at each level of the hierarchy, and bistable Hebbian dynamics enables the learning of such trajectories in an unsupervised fashion. Using computer simulations, we demonstrate the learning of a chunking representation of sequences and their robust recall. During learning, the dynamics associates a set of modes to each information-carrying item in the sequence and encodes their relative order. During recall, hierarchical WLC guarantees the robustness of the sequence order when the sequence is not too long. The resulting patterns of activities share several features observed in behavioral experiments, such as the pauses between boundaries of chunks, their size and their duration. Failures in learning chunking sequences provide new insights into the dynamical causes of neurological disorders such as Parkinson’s disease and Schizophrenia.

Because chunking is a hallmark of the brain’s organization, efforts to understand its dynamics can provide valuable insights into the brain and its disorders. For identifying the dynamical principles of chunking learning, we hypothesize that perceptual sequences can be learned and stored as a chain of metastable fixed points in a low-dimensional dynamical system, similar to the trajectory of a ball rolling down a pinball machine. During a learning phase, the interactions in the network evolve such that the network learns a chunking representation of the sequence, as when memorizing a phone number in segments. In the example of the pinball machine, learning can be identified with the gradual placement of the pins. After learning, the pins are placed in a way that, at each run, the ball follows the same trajectory (recall of the same sequence) that encodes the perceptual sequence. Simulations show that the dynamics are endowed with the hallmarks of chunking observed in behavioral experiments, such as increased delays observed before loading new chunks.

Contribution of the basal ganglia to spoken language: is speech production like the other motor skills?

Two of the roles assigned to the basal ganglia in spoken language parallel very well their contribution to motor behaviour: (1) their role in sequence processing, resulting in syntax deficits, and (2) their role in movement "vigor," leading to "hypokinetic dysarthria" or "hypophonia." This is an additional example of how the motor system has served the emergence of high-level cognitive functions, such as language.

The striatum: where skills and habits meet

After more than a century of work concentrating on the motor functions of the basal ganglia, new ideas have emerged, suggesting that the basal ganglia also have major functions in relation to learning habits and acquiring motor skills. We review the evidence supporting the role of the striatum in optimizing behavior by refining action selection and in shaping habits and skills as a modulator of motor repertoires. These findings challenge the notion that striatal learning processes are limited to the motor domain. The learning mechanisms supported by striatal circuitry generalize to other domains, including cognitive skills and emotion-related patterns of action.

Basal ganglia and cortical networks for sequential ordering and rhythm of complex movements

Voluntary actions require the concurrent engagement and coordinated control of complex temporal (e.g., rhythm) and ordinal motor processes. Using high-resolution functional magnetic resonance imaging (fMRI) and multi-voxel pattern analysis (MVPA), we sought to determine the degree to which these complex motor processes are dissociable in basal ganglia and cortical networks. We employed three different finger-tapping tasks that differed in the demand on the sequential temporal rhythm or sequential ordering of submovements. Our results demonstrate that sequential rhythm and sequential order tasks were partially dissociable based on activation differences. The sequential rhythm task activated a widespread network centered around the supplementary motor area (SMA) and basal-ganglia regions including the dorsomedial putamen and caudate nucleus, while the sequential order task preferentially activated a fronto-parietal network. There was also extensive overlap between sequential rhythm and sequential order tasks, with both tasks commonly activating bilateral premotor, supplementary motor, and superior/inferior parietal cortical regions, as well as regions of the caudate/putamen of the basal ganglia and the ventro-lateral thalamus. Importantly, within the cortical regions that were active for both complex movements, MVPA could accurately classify different patterns of activation for the sequential rhythm and sequential order tasks. In the basal ganglia, however, overlapping activation for the sequential rhythm and sequential order tasks, which was found in classic motor circuits of the putamen and ventro-lateral thalamus, could not be accurately differentiated by MVPA. Overall, our results highlight the convergent architecture of the motor system, where complex motor information that is spatially distributed in the cortex converges into a more compact representation in the basal ganglia.

Habits as action sequences: hierarchical action control and changes in outcome value

Goal-directed action involves making high-level choices that are implemented using previously acquired action sequences to attain desired goals. Such a hierarchical schema is necessary for goal-directed actions to be scalable to real-life situations, but results in decision-making that is less flexible than when action sequences are unfolded and the decision-maker deliberates step-by-step over the outcome of each individual action. In particular, from this perspective, the offline revaluation of any outcomes that fall within action sequence boundaries will be invisible to the high-level planner resulting in decisions that are insensitive to such changes. Here, within the context of a two-stage decision-making task, we demonstrate that this property can explain the emergence of habits. Next, we show how this hierarchical account explains the insensitivity of over-trained actions to changes in outcome value. Finally, we provide new data that show that, under extended extinction conditions, habitual behaviour can revert to goal-directed control, presumably as a consequence of decomposing action sequences into single actions. This hierarchical view suggests that the development of action sequences and the insensitivity of actions to changes in outcome value are essentially two sides of the same coin, explaining why these two aspects of automatic behaviour involve a shared neural structure.

Striatal and hippocampal involvement in motor sequence chunking depends on the learning strategy

Motor sequences can be learned using an incremental approach by starting with a few elements and then adding more as training evolves (e.g., learning a piano piece); conversely, one can use a global approach and practice the whole sequence in every training session (e.g., shifting gears in an automobile). Yet, the neural correlates associated with such learning strategies in motor sequence learning remain largely unexplored to date. Here we used functional magnetic resonance imaging to measure the cerebral activity of individuals executing the same 8-element sequence after they completed a 4-days training regimen (2 sessions each day) following either a global or incremental strategy. A network comprised of striatal and fronto-parietal regions was engaged significantly regardless of the learning strategy, whereas the global training regimen led to additional cerebellar and temporal lobe recruitment. Analysis of chunking/grouping of sequence elements revealed a common prefrontal network in both conditions during the chunk initiation phase, whereas execution of chunk cores led to higher mediotemporal activity (involving the hippocampus) after global than incremental training. The novelty of our results relate to the recruitment of mediotemporal regions conditional of the learning strategy. Thus, the present findings may have clinical implications suggesting that the ability of patients with lesions to the medial temporal lobe to learn and consolidate new motor sequences may benefit from using an incremental strategy.

Opposing effects of dopamine antagonism in a motor sequence task-tiapride increases cortical excitability and impairs motor learning

The dopaminergic system is involved in learning and participates in the modulation of cortical excitability (CE). CE has been suggested as a marker of learning and use-dependent plasticity. However, results from separate studies on either motor CE or motor learning challenge this notion, suggesting opposing effects of dopaminergic modulation upon these parameters: while agonists decrease and antagonists increase CE, motor learning is enhanced by agonists and disturbed by antagonists. To examine whether this discrepancy persists when complex motor learning and motor CE are measured in the same experimental setup, we investigated the effects of dopaminergic (DA) antagonism upon both parameters and upon task-associated brain activation. Our results demonstrate that DA-antagonism has opposing effects upon motor CE and motor sequence learning. Tiapride did not alter baseline CE, but increased CE post training of a complex motor sequence while simultaneously impairing motor learning. Moreover, tiapride reduced activation in several brain regions associated with motor sequence performance, i.e., dorsolateral PFC (dlPFC), supplementary motor area (SMA), Broca's area, cingulate and caudate body. Blood-oxygenation-level-dependent (BOLD) intensity in anterior cingulate and caudate body, but not CE, correlated with performance across groups. In summary, our results do not support a concept of CE as a general marker of motor learning, since they demonstrate that a straightforward relation of increased CE and higher learning success does not apply to all instances of motor learning. At least for complex motor tasks that recruit a network of brain regions outside motor cortex, CE in primary motor cortex is probably no central determinant for learning success.

Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer

BACKGROUND

Recent theories addressing mesolimbic dopamine's role in reward processing emphasize two apparently distinct functions, one in reinforcement learning (i.e., prediction error) and another in incentive motivation (i.e., the invigoration of reward seeking elicited by reward-paired cues). Here, we evaluate the latter.

METHODS

Using fast-scan cyclic voltammetry, we monitored, in real time, dopamine release in the nucleus accumbens core of rats (n = 9) during a Pavlovian-to-instrumental transfer task in which the effects of a reward-predictive cue on an independently trained instrumental action were assessed. Voltammetric data were parsed into slow and phasic components to determine whether these forms of dopamine signaling were differentially related to task performance.

RESULTS

We found that a reward-paired cue, which increased reward-seeking actions, induced an increase in phasic mesolimbic dopamine release and produced slower elevations in extracellular dopamine. Interestingly, phasic dopamine release was temporally related to and positively correlated with lever-press activity generally, while slow dopamine changes were not significantly related to such activity. Importantly, the propensity of the reward-paired cue to increase lever pressing was predicted by the amplitude of phasic dopamine release events, indicating a possible mechanism through which cues initiate reward-seeking actions.

CONCLUSIONS

These data suggest that those phasic mesolimbic dopamine release events thought to signal reward prediction error may also be related to the incentive motivational impact of reward-paired cues on reward-seeking actions.

Early-stage chunking of finger tapping sequences by persons who stutter and fluent speakers

This research note explored the hypothesis that chunking differences underlie the slow finger-tap sequencing performance reported in the literature for persons who stutter (PWS) relative to fluent speakers (PNS). Early-stage chunking was defined as an immediate and spontaneous tendency to organize a long sequence into pauses, for motor planning, and chunks of fluent motor performance. A previously published study in which 12 PWS and 12 matched PNS practised a 10-item finger tapping sequence 30 times was examined. Both groups significantly decreased the duration of between-chunk intervals (BCIs) and within-chunk intervals (WCIs) over practice. PNS had significantly shorter WCIs relative to PWS, but minimal differences between groups were found for the number of, or duration of, BCI. Results imply that sequencing differences found between PNS and PWS may be due to differences in automatizing movements within chunks or retrieving chunks from memory rather than chunking per se.

Differential recruitment of the sensorimotor putamen and frontoparietal cortex during motor chunking in humans

Motor chunking facilitates movement production by combining motor elements into integrated units of behavior. Previous research suggests that chunking involves two processes: concatenation, aimed at the formation of motor-motor associations between elements or sets of elements, and segmentation, aimed at the parsing of multiple contiguous elements into shorter action sets. We used fMRI to measure the trial-wise recruitment of brain regions associated with these chunking processes as healthy subjects performed a cued-sequence production task. A dynamic network analysis identified chunking structure for a set of motor sequences acquired during fMRI and collected over 3 days of training. Activity in the bilateral sensorimotor putamen positively correlated with chunk concatenation, whereas a left-hemisphere frontoparietal network was correlated with chunk segmentation. Across subjects, there was an aggregate increase in chunk strength (concatenation) with training, suggesting that subcortical circuits play a direct role in the creation of fluid transitions across chunks.

Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task

BACKGROUND

Sequential reward-seeking actions are readily learned despite the temporal gap between the earliest (distal) action in the sequence and the reward delivery. Fast dopamine signaling is hypothesized to mediate this form of learning by reporting errors in reward prediction. However, such a role for dopamine release in voluntarily initiated action sequences remains to be demonstrated.

METHODS

Using fast-scan cyclic voltammetry, we monitored phasic mesolimbic dopamine release, in real time, as rats performed a self-initiated sequence of lever presses to earn sucrose rewards. Before testing, rats received either 0 (n = 11), 5 (n = 11), or 10 (n = 8) days of action sequence training.

RESULTS

For rats acquiring the action sequence task at test, dopamine release was strongly elicited by response-contingent (but unexpected) rewards. With learning, a significant elevation in dopamine release preceded performance of the proximal action and subsequently came to precede the distal action. This predistal dopamine release response was also observed in rats previously trained on the action sequence task, and the amplitude of this signal predicted the latency with which rats completed the action sequence. Importantly, the dopamine response to contingent reward delivery was not observed in rats given extensive pretraining. Pharmacological analysis confirmed that task performance was dopamine-dependent.

CONCLUSIONS

These data suggest that phasic mesolimbic dopamine release mediates the influence that rewards exert over the performance of self-paced, sequentially-organized behavior and sheds light on how dopamine signaling abnormalities may contribute to disorders of behavioral control.

Parallel contributions of cerebellar, striatal and M1 mechanisms to motor sequence learning

When learning a new motor sequence, we must execute the correct order of movements while simultaneously optimizing sensorimotor parameters such as trajectory, timing, velocity and force. Neurophysiological studies in animals and humans have identified the major brain regions involved in sequence learning, including the motor cortex (M1), basal ganglia (BG) and cerebellum. Current models link these regions to different stages of learning (early vs. late) or different components of performance (spatial vs. sensorimotor). At the same time, research in motor control has given rise to the concept that internal models at different levels of the motor system may contribute to learning. The goal of this review is to develop a new framework for motor sequence learning that combines stage and component models within the context of internal models. To do this, we review behavioral and neuroimaging studies in humans and neurophysiological studies in animals. Based on this evidence, we present a model proposing that sequence learning is underwritten by parallel, interacting processes, including internal model formation and sequence representation, that are instantiated in specific cerebellar, BG or M1 mechanisms depending on task demands and the stage of learning. The striatal system learns predictive stimulus-response associations and is critical for motor chunking. The role of the cerebellum is to acquire the optimal internal model for sequence performance in a particular context, and to contribute to error correction and control of on-going movement. M1 acts to store the representation of a learned sequence, likely as part of a distributed network including the parietal lobe and premotor cortex.

Neural systems analysis of decision making during goal-directed navigation

The ability to make adaptive decisions during goal-directed navigation is a fundamental and highly evolved behavior that requires continual coordination of perceptions, learning and memory processes, and the planning of behaviors. Here, a neurobiological account for such coordination is provided by integrating current literatures on spatial context analysis and decision-making. This integration includes discussions of our current understanding of the role of the hippocampal system in experience-dependent navigation, how hippocampal information comes to impact midbrain and striatal decision making systems, and finally the role of the striatum in the implementation of behaviors based on recent decisions. These discussions extend across cellular to neural systems levels of analysis. Not only are key findings described, but also fundamental organizing principles within and across neural systems, as well as between neural systems functions and behavior, are emphasized. It is suggested that studying decision making during goal-directed navigation is a powerful model for studying interactive brain systems and their mediation of complex behaviors.

Computational cognitive models of prefrontal-striatal-hippocampal interactions in Parkinson's disease and schizophrenia

Disruption to different components of the prefrontal cortex, basal ganglia, and hippocampal circuits leads to various psychiatric and neurological disorders including Parkinson's disease (PD) and schizophrenia. Medications used to treat these disorders (such as levodopa, dopamine agonists, antipsychotics, among others) affect the prefrontal-striatal-hippocampal circuits in a complex fashion. We have built models of prefrontal-striatal and striatal-hippocampal interactions which simulate cognitive dysfunction in PD and schizophrenia. In these models, we argue that the basal ganglia is key for stimulus-response learning, the hippocampus for stimulus-stimulus representational learning, and the prefrontal cortex for stimulus selection during learning about multidimensional stimuli. In our models, PD is associated with reduced dopamine levels in the basal ganglia and prefrontal cortex. In contrast, the cognitive deficits in schizophrenia are associated primarily with hippocampal dysfunction, while the occurrence of negative symptoms is associated with frontostriatal deficits in a subset of patients. In this paper, we review our past models and provide new simulation results for both PD and schizophrenia. We also describe an extended model that includes simulation of the different functional role of D1 and D2 dopamine receptors in the basal ganglia and prefrontal cortex, a dissociation we argue is essential for understanding the non-uniform effects of levodopa, dopamine agonists, and antipsychotics on cognition. Motivated by clinical and physiological data, we discuss model limitations and challenges to be addressed in future models of these brain disorders.

Advance cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum

One of the most characteristic features of habitual behaviors is that they can be evoked by a single cue. In the experiments reported here, we tested for the effects of such advance cueing on the firing patterns of striatal neurons in the sensorimotor striatum. Rats ran in a T-maze with instruction cues about the location of reward given at the start of the runs. This advance cueing about reward produced a highly augmented task-bracketing pattern of activity at the beginning and end of procedural task performance relative to the patterns found previously with midtask cueing. Remarkably, the largest increase in activity early during the T-maze runs was not associated with the instruction cues themselves, the earliest predictors of reward; instead, the highest peak of early activity was associated with the beginning of the motor period of the task. We suggest that the advance cueing, reducing midrun demands for decision making but adding a working-memory load, facilitated chunking of the maze runs as executable scripts anchored to sensorimotor aspects of the task and unencumbered by midtask decision-making demands. Our findings suggest that the acquisition of stronger task-bracketing patterns of striatal activity in the sensorimotor striatum could reflect this enhancement of behavioral chunking. Deficits in such representations of learned sequential behaviors could contribute to motor and cognitive problems in a range of neurological disorders affecting the basal ganglia, including Parkinson's disease.

Movement chunking during sequence learning is a dopamine-dependant process: a study conducted in Parkinson's disease

Chunking of single movements into integrated sequences has been described during motor learning, and we have recently demonstrated that this process involves a dopamine-dependant mechanism in animal (Levesque et al. in Exp Brain Res 182:499-508, 2007; Tremblay et al. in Behav Brain Res 198:231-239, 2009). However, there is no such evidence in human. The aim of the present study was to assess this question in Parkinson's disease (PD), a neurological condition known for its dopamine depletion in the striatum. Eleven PD patients were tested under their usual levodopa medication (ON state), and following a 12-h levodopa withdrawal (OFF state). Patients were compared with 12 healthy participants on a motor learning sequencing task, requiring pressing fourteen buttons in the correct order, which was determined by visual stimuli presented on a computer screen. Learning was assessed from three blocks of 20 trials administered successively. Chunks of movements were intrinsically created by each participant during this learning period. Then, the sequence was shuffled according to the participant's own chunks, generating two new sequences, with either preserved or broken chunks. Those new motor sequences had to be performed separately in a fourth and fifth blocks of 20 trials. Results showed that execution time improved in every group during the learning period (from blocks 1 to 3). However, while motor chunking occurred in healthy controls and ON-PD patients, it did not in OFF-PD patients. In the shuffling conditions, a significant difference was seen between the preserved and the broken chunks conditions for both healthy participants and ON-PD patients, but not for OFF-PD patients. These results suggest that movement chunking during motor sequence learning is a dopamine-dependent process in human.

Motor sequences and the basal ganglia: kinematics, not habits

Despite a lack of definitive evidence, it is frequently proposed that the basal ganglia (BG) motor circuit plays a critical role in the storage and execution of movement sequences (or motor habits). To test this hypothesis directly, we inactivated the sensorimotor territory of the globus pallidus internus (sGPi, the main BG motor output) in two monkeys trained to perform overlearned and random sequences of four out-and-back reaching movements directed to visual targets. Infusion of muscimol (a GABA(A) agonist) into sGPi caused dysmetria and slowing of individual movements, but these impairments were virtually identical for overlearned and random sequences. The fluid predictive execution of learned sequences and the animals' tendency to reproduce the sequence pattern in random trials was preserved following pallidal blockade. These results suggest that the BG motor circuit contributes to motor execution, but not to motor sequencing or the storage of overlearned serial skills.