Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives

Foundations of human spatial problem solving

Despite great strides in both machine learning and neuroscience, we do not know how the human brain solves problems in the general sense. We approach this question by drawing on the framework of engineering control theory. We demonstrate a computational neural model with only localist learning laws that is able to find solutions to arbitrary problems. The model and humans perform a multi-step task with arbitrary and changing starting and desired ending states. Using a combination of computational neural modeling, human fMRI, and representational similarity analysis, we show here that the roles of a number of brain regions can be reinterpreted as interacting mechanisms of a control theoretic system. The results suggest a new set of functional perspectives on the orbitofrontal cortex, hippocampus, basal ganglia, anterior temporal lobe, lateral prefrontal cortex, and visual cortex, as well as a new path toward artificial general intelligence.

Learning-induced changes in the neural circuits underlying motor sequence execution

As the old adage goes: practice makes perfect. Yet, the neural mechanisms by which rote repetition transforms a halting behavior into a fluid, effortless, and "automatic" action are not well understood. Here we consider the possibility that well-practiced motor sequences, which initially rely on higher-level decision-making circuits, become wholly specified in lower-level control circuits. We review studies informing this idea, discuss the constraints on such shift in control, and suggest approaches to pinpoint circuit-level changes associated with motor sequence learning.

Hierarchical Reinforcement Learning, Sequential Behavior, and the Dorsal Frontostriatal System

To effectively behave within ever-changing environments, biological agents must learn and act at varying hierarchical levels such that a complex task may be broken down into more tractable subtasks. Hierarchical reinforcement learning (HRL) is a computational framework that provides an understanding of this process by combining sequential actions into one temporally extended unit called an option. However, there are still open questions within the HRL framework, including how options are formed and how HRL mechanisms might be realized within the brain. In this review, we propose that the existing human motor sequence literature can aid in understanding both of these questions. We give specific emphasis to visuomotor sequence learning tasks such as the discrete sequence production task and the M × N (M steps × N sets) task to understand how hierarchical learning and behavior manifest across sequential action tasks as well as how the dorsal cortical-subcortical circuitry could support this kind of behavior. This review highlights how motor chunks within a motor sequence can function as HRL options. Furthermore, we aim to merge findings from motor sequence literature with reinforcement learning perspectives to inform experimental design in each respective subfield.

Reinforcement-learning in fronto-striatal circuits

We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.

A dynamic neural field model of continuous input integration

The ability of neural systems to turn transient inputs into persistent changes in activity is thought to be a fundamental requirement for higher cognitive functions. In continuous attractor networks frequently used to model working memory or decision making tasks, the persistent activity settles to a stable pattern with the stereotyped shape of a "bump" independent of integration time or input strength. Here, we investigate a new bump attractor model in which the bump width and amplitude not only reflect qualitative and quantitative characteristics of a preceding input but also the continuous integration of evidence over longer timescales. The model is formalized by two coupled dynamic field equations of Amari-type which combine recurrent interactions mediated by a Mexican-hat connectivity with local feedback mechanisms that balance excitation and inhibition. We analyze the existence, stability and bifurcation structure of single and multi-bump solutions and discuss the relevance of their input dependence to modeling cognitive functions. We then systematically compare the pattern formation process of the two-field model with the classical Amari model. The results reveal that the balanced local feedback mechanisms facilitate the encoding and maintenance of multi-item memories. The existence of stable subthreshold bumps suggests that different to the Amari model, the suppression effect of neighboring bumps in the range of lateral competition may not lead to a complete loss of information. Moreover, bumps with larger amplitude are less vulnerable to noise-induced drifts and distance-dependent interaction effects resulting in more faithful memory representations over time.

Modeling Post-Scratching Locomotion with Two Rhythm Generators and a Shared Pattern Formation

This study aimed to present a model of post-scratching locomotion with two intermixed central pattern generator (CPG) networks, one for scratching and another for locomotion. We hypothesized that the rhythm generator layers for each CPG are different, with the condition that both CPGs share their supraspinal circuits and their motor outputs at the level of their pattern formation networks. We show that the model reproduces the post-scratching locomotion latency of 6.2 ± 3.5 s, and the mean cycle durations for scratching and post-scratching locomotion of 0.3 ± 0.09 s and 1.7 ± 0.6 s, respectively, which were observed in a previous experimental study. Our findings show how the transition of two rhythmic movements could be mediated by information exchanged between their CPG circuits through routes converging in a common pattern formation layer. This integrated organization may provide flexible and effective connectivity despite the rigidity of the anatomical connections in the spinal cord circuitry.

Competitive state of movements during planning predicts sequence performance

Humans can learn and produce skilled movement sequences from memory, yet the nature of sequence planning is not well understood. Previous computational and neurophysiological work suggests that movements in a sequence are planned as parallel graded activations and selected for output through competition. However, the relevance of this planning pattern to sequence production fluency and accuracy, as opposed to the temporal structure of sequences, is unclear. To resolve this question, we assessed the relative availability of constituent movements behaviorally during the preparation of motor sequences from memory. In three separate multisession experiments, healthy participants were trained to retrieve and produce four-element finger press sequences with particular timing according to an abstract sequence cue. We evaluated reaction time (RT) and error rate as markers of movement availability to constituent movement probes. Our results demonstrate that longer preparation time produces more pronounced differences in availability between adjacent sequence elements, whereas no effect was found for sequence speed or temporal grouping. Further, participants with larger position-dependent differences in movement availability tended to initiate correct sequences faster and with a higher temporal accuracy. Our results suggest that competitive preactivation is established gradually during sequence planning and predicts sequence skill, rather than the temporal structure of the motor sequence.NEW & NOTEWORTHY Sequence planning is an integral part of motor sequence control. Here, we demonstrate that the competitive state of sequential movements during sequence planning can be read out behaviorally through movement probes. We show that position-dependent differences in movement availability during planning reflect sequence preparedness and skill but not the timing of the planned sequence. Behavioral access to the preparatory state of movements may serve as a marker of sequence planning capacity.

Isoluminant stimuli in a familiar discrete keying sequence task can be ignored

Motor sequencing models suggest that when with extensive practice sequence representations have developed, stimuli indicating the individual sequence elements may no longer be used for sequence execution. However, it is not clear whether participants can at all refrain from processing these stimuli. Two experiments were performed in which participants practiced two 7-keypress sequences by responding to isoluminant key-specific stimuli. In the mixed condition of the ensuing test phase, the stimuli were displayed only occasionally, and the question was whether this would make participants stop processing these stimuli. In Experiment 1, the benefit of displaying stimuli was assessed after substantial practice, while Experiment 2 examined development of this benefit across practice. The results of Experiment 1 showed that participants rely a little less on these stimuli when they are displayed only occasionally, but Experiment 2 revealed that participants quickly developed high awareness, and that they ignored these stimuli already after limited practice. These findings confirm that participants can choose to ignore these isoluminant stimuli but tend to use them when they are displayed. These and other findings show in some detail how various cognitive systems interact to produce familiar keying sequences.

The Planning Horizon for Movement Sequences

When performing a long chain of actions in rapid sequence, future movements need to be planned concurrently with ongoing action. However, how far ahead we plan, and whether this ability improves with practice, is currently unknown. Here, we designed an experiment in which healthy volunteers produced sequences of 14 finger presses quickly and accurately on a keyboard in response to numerical stimuli. On every trial, participants were only shown a fixed number of stimuli ahead of the current keypress. The size of this viewing window varied between 1 (next digit revealed with the pressing of the current key) and 14 (full view of the sequence). Participants practiced the task for 5 days, and their performance was continuously assessed on random sequences. Our results indicate that participants used the available visual information to plan multiple actions into the future, but that the planning horizon was limited: receiving information about more than three movements ahead did not result in faster sequence production. Over the course of practice, we found larger performance improvements for larger viewing windows and an expansion of the planning horizon. These findings suggest that the ability to plan future responses during ongoing movement constitutes an important aspect of skillful movement. Based on the results, we propose a framework to investigate the neuronal processes underlying simultaneous planning and execution.

The Varying Effects of Dual Tasks on the Performance of Motor Skills across Practice

Numerous previous studies using the dual-task methodology have indicated that the effect of attentional direction on the performance of motor skill differs as a function of skill levels. Whereas previous studies relied mostly on inter-individual comparisons, this study focused on how the effects of different attentional conditions change within individuals with practice. Participants were instructed to learn a short and a long keying sequence (three versus six keys) and then practiced under two block-wise alternating dual-task conditions. In each trial, a tone, either low- or high-pitched, was presented at one of the three/six keys and participants had to indicate either the pitch of the tone (extraneous dual task) or the key with which the tone was presented (skill-focused dual task) after finishing the execution of the keying sequence. Motor task performance was assessed by reaction time (RT) and movement duration (MD), and the concurrent cognitive task performance was assessed by the error rate. RT was faster in the skill-focused dual-task condition at the beginning of practice, whereas a generally shorter MD was found in the extraneous dual-task condition. The error rate in the extraneous dual task decreased with practice, whereas in the skill-focused dual task, it increased with practice. These results show that the effects of attentional direction differ not only as a function of the amount of practice but also as a function of the stage of information processing. Furthermore, our results indicate that the direction of attention alone does not explain the different patterns of performance at different skill levels seen across dual-task studies; rather, the skill levels, the nature of cognitive demands, the difficulty level of dual tasks, and the complexity of the motor skill could all drive performance differences.

Neural Trajectories in the Supplementary Motor Area and Motor Cortex Exhibit Distinct Geometries, Compatible with Different Classes of Computation

The supplementary motor area (SMA) is believed to contribute to higher order aspects of motor control. We considered a key higher order role: tracking progress throughout an action. We propose that doing so requires population activity to display low "trajectory divergence": situations with different future motor outputs should be distinct, even when present motor output is identical. We examined neural activity in SMA and primary motor cortex (M1) as monkeys cycled various distances through a virtual environment. SMA exhibited multiple response features that were absent in M1. At the single-neuron level, these included ramping firing rates and cycle-specific responses. At the population level, they included a helical population-trajectory geometry with shifts in the occupied subspace as movement unfolded. These diverse features all served to reduce trajectory divergence, which was much lower in SMA versus M1. Analogous population-trajectory geometry, also with low divergence, naturally arose in networks trained to internally guide multi-cycle movement.

The Simon effect in a discrete sequence production task: Key-specific stimuli cannot be ignored due to attentional capture

Two experiments examined whether practicing discrete key pressing sequences eventually leads to a disregard of the key-specific stimuli, as suggested by sequence learning models, or whether these stimuli continue to be relied upon because the associated luminance increase attracts visuospatial attention. Participants practiced two sequences by reacting to two fixed series of seven letter stimuli, each displayed at a location that did or did not correspond with the required response location. Stimulus use was indicated by a Simon effect in that key presses were slowed when stimulus and key locations did not correspond. Experiment 1 demonstrated that letter stimuli continued to be used as the Simon effect occurred with each sequence element, and this remained quite stable across practice and did not differ for familiar and unfamiliar sequences. Experiment 2 showed that the Simon effect remained present even with meaningless stimuli that were often even harmful. These findings suggest that even in motor sequences that can be executed without element-specific stimuli attention attraction enforces stimulus use. The data further supported the assumptions that S-R translation and sequencing systems are racing to trigger individual responses, and that explicit sequence representations include spatial and verbal knowledge.

Task-Modulated Corticocortical Synchrony in the Cognitive-Motor Network Supporting Handwriting

Both motor and cognitive aspects of behavior depend on dynamic, accurately timed neural processes in large-scale brain networks. Here, we studied synchronous interplay between cortical regions during production of cognitive-motor sequences in humans. Specifically, variants of handwriting that differed in motor variability, linguistic content, and memorization of movement cues were contrasted to unveil functional sensitivity of corticocortical connections. Data-driven magnetoencephalography mapping (n = 10) uncovered modulation of mostly left-hemispheric corticocortical interactions, as quantified by relative changes in phase synchronization. At low frequencies (~2–13 Hz), enhanced frontoparietal synchrony was related to regular handwriting, whereas premotor cortical regions synchronized for simple loop production and temporo-occipital areas for a writing task substituting normal script with loop patterns. At the beta-to-gamma band (~13–45 Hz), enhanced synchrony was observed for regular handwriting in the central and frontoparietal regions, including connections between the sensorimotor and supplementary motor cortices and between the parietal and dorsal premotor/precentral cortices. Interpreted within a modular framework, these modulations of synchrony mainly highlighted interactions of the putative pericentral subsystem of hand coordination and the frontoparietal subsystem mediating working memory operations. As part of cortical dynamics, interregional phase synchrony varies depending on task demands in production of cognitive-motor sequences.

Statistical learning for vocal sequence acquisition in a songbird

Birdsong is a learned communicative behavior that consists of discrete acoustic elements (“syllables”) that are sequenced in a controlled manner. While the learning of the acoustic structure of syllables has been extensively studied, relatively little is known about sequence learning in songbirds. Statistical learning could contribute to the acquisition of vocal sequences, and we investigated the nature and extent of sequence learning at various levels of song organization in the Bengalese finch, Lonchura striata var. domestica. We found that, under semi-natural conditions, pupils (sons) significantly reproduced the sequence statistics of their tutor’s (father’s) songs at multiple levels of organization (e.g., syllable repertoire, prevalence, and transitions). For example, the probability of syllable transitions at “branch points” (relatively complex sequences that are followed by multiple types of transitions) were significantly correlated between the songs of tutors and pupils. We confirmed the contribution of learning to sequence similarities between fathers and sons by experimentally tutoring juvenile Bengalese finches with the songs of unrelated tutors. We also discovered that the extent and fidelity of sequence similarities between tutors and pupils were significantly predicted by the prevalence of sequences in the tutor’s song and that distinct types of sequence modifications (e.g., syllable additions or deletions) followed distinct patterns. Taken together, these data provide compelling support for the role of statistical learning in vocal production learning and identify factors that could modulate the extent of vocal sequence learning.

Learning spatiotemporal signals using a recurrent spiking network that discretizes time

Learning to produce spatiotemporal sequences is a common task that the brain has to solve. The same neurons may be used to produce different sequential behaviours. The way the brain learns and encodes such tasks remains unknown as current computational models do not typically use realistic biologically-plausible learning. Here, we propose a model where a spiking recurrent network of excitatory and inhibitory spiking neurons drives a read-out layer: the dynamics of the driver recurrent network is trained to encode time which is then mapped through the read-out neurons to encode another dimension, such as space or a phase. Different spatiotemporal patterns can be learned and encoded through the synaptic weights to the read-out neurons that follow common Hebbian learning rules. We demonstrate that the model is able to learn spatiotemporal dynamics on time scales that are behaviourally relevant and we show that the learned sequences are robustly replayed during a regime of spontaneous activity.

A Mobile Phone App for the Generation and Characterization of Motor Habits

Habits are a powerful route to efficiency; the ability to constantly shift between goal-directed and habitual strategies, as well as integrate them into behavioral output, is key to optimal performance in everyday life. When such ability is impaired, it may lead to loss of control and to compulsive behavior. Habits have successfully been induced and investigated in rats using methods such as overtraining stimulus-response associations and outcome devaluation, respectively. However, such methods have ineffectively measured habits in humans because (1) human habits usually involve more complex sequences of actions than in rats and (2) of pragmatic impediments posed by the extensive time (weeks or even months), it may take for routine habits to develop. We present here a novel behavioral paradigm-a mobile-phone app methodology-for inducing and measuring habits in humans during their everyday schedule and environment. It assumes that practice is key to achieve automaticity and proficiency and that the use of a hierarchical sequence of actions is the best strategy for capturing the cognitive mechanisms involved in habit formation (including "chunking") and consolidation. The task is a gamified self-instructed and self-paced app on a mobile phone that enables subjects to learn and practice two sequences of finger movements, composed of chords and single presses. It involves a step-wise learning procedure in which subjects begin responding to a visual and auditory cued sequence by generating responses on the screen using four fingers. Such cues progressively disappear throughout 1 month of training, enabling the subject ultimately to master the motor skill involved. We present preliminary data for the acquisition of motor sequence learning in 29 healthy individuals, each trained over a month period. We demonstrate an asymptotic improvement in performance, as well as its automatic nature. We also report how people integrate the task into their daily routine, the development of motor precision throughout training, and the effect of intermittent reinforcement and reward extinction in habit preservation. The findings help to validate this "real world" app for measuring human habits.

Movement Repetition Facilitates Response Preparation

Our sensorimotor system appears to be influenced by the recent history of our movements. Repeating movements toward a particular direction is known to have a dramatic effect on involuntary movements elicited by cortical stimulation-a phenomenon that has been termed use-dependent plasticity. However, analogous effects of repetition on behavior have proven elusive. Here, we show that movement repetition enhances the generation of similar movements in the future by reducing the time required to select and prepare the repeated movement. We further show that this reaction time advantage for repeated movements is attributable to more rapid, but still flexible, preparation of the repeated movement rather than anticipation and covert advance preparation of the previously repeated movement. Our findings demonstrate a powerful and beneficial effect of movement repetition on response preparation, which may represent a behavioral counterpart to use-dependent plasticity effects in primary motor cortex.

Fluid intelligence and working memory support dissociable aspects of learning by physical but not observational practice

Humans have a remarkable ability to learn by watching others, whether learning to tie an elaborate knot or play the piano. However, the mechanisms that translate visual input into motor skill execution remain unclear. It has been proposed that common cognitive and neural mechanisms underpin learning motor skills by physical and observational practice. Here we provide a novel test of the common mechanism hypothesis by testing the extent to which certain individual differences predict observational as well as physical learning. Participants (N = 92 per group) either physically practiced a five-element key-press sequence or watched videos of similar sequences before physically performing trained and untrained sequences in a test phase. We also measured cognitive abilities across participants that have previously been associated with rates of learning, including working memory and fluid intelligence. Our findings show that individual differences in working memory and fluid intelligence predict improvements in dissociable aspects of motor learning following physical practice, but not observational practice. Working memory predicts general learning gains from pre- to post-test that generalise to untrained sequences, whereas fluid intelligence predicts sequence-specific gains that are tied to trained sequences. However, neither working memory nor fluid intelligence predict training gains following observational learning. Therefore, these results suggest limits to the shared mechanism hypothesis of physical and observational learning. Indeed, models of observational learning need updating to reflect the extent to which such learning is based on shared as well as distinct processes compared to physical learning. We suggest that such differences could reflect the more intentional nature of learning during physical compared to observational practice, which relies to a greater extent on higher-order cognitive resources such as working memory and fluid intelligence.

Effects of auditory rhythm on movement accuracy in dance performance

The present study addresses the impact of the rhythmic complexity of music on the accuracy of dance performance. This study examined the effects of different levels of auditory syncopation on the execution of a dance sequence by trained dancers and exercisers (i.e., nondancers). It was hypothesized that nondancers would make more errors in synchronizing movements with moderately and highly syncopated rhythms while no performance degradation would manifest among trained dancers. Participants performed a dance sequence synchronized with three different rhythm tracks that were regular, moderately syncopated, and highly syncopated. We found significant performance degradation when comparing conditions of no syncopation vs. high syncopation for both trained dancers (p = .002) and nondancers (p = .001). Dancers and nondancers did not differ in how they managed to execute the task with increasing levels of syncopation (p = .384). The pattern of difference between trained dancers and nondancers was similar across the No Syncop and Highly Syncop conditions. The present findings may have marked implications for practitioners given that the tasks employed were analogous to those frequently observed in real-life dance settings.

Sequence learning is driven by improvements in motor planning

The ability to perform complex sequences of movements quickly and accurately is critical for many motor skills. Although training improves performance in a large variety of motor sequence tasks, the precise mechanisms behind such improvements are poorly understood. Here we investigated the contribution of single-action selection, sequence preplanning, online planning, and motor execution to performance in a discrete sequence production task. Five visually presented numbers cued a sequence of five finger presses, which had to be executed as quickly and accurately as possible. To study how sequence planning influenced sequence production, we manipulated the amount of time that participants were given to prepare each sequence by using a forced-response paradigm. Over 4 days, participants were trained on 10 sequences and tested on 80 novel sequences. Our results revealed that participants became faster in selecting individual finger presses. They also preplanned three or four sequence items into the future, and the speed of preplanning improved for trained, but not for untrained, sequences. Because preplanning capacity remained limited, the remaining sequence elements had to be planned online during sequence execution, a process that also improved with sequence-specific training. Overall, our results support the view that motor sequence learning effects are best characterized by improvements in planning processes that occur both before and concurrently with motor execution. NEW & NOTEWORTHY Complex skills often require the production of sequential movements. Although practice improves performance, it remains unclear how these improvements are achieved. Our findings show that learning effects in a sequence production task can be attributed to an enhanced ability to plan upcoming movements. These results shed new light on planning processes in the context of movement sequences and have important implications for our understanding of the neural mechanisms that underlie skill acquisition.

Skill in discrete keying sequences is execution rate specific

The present study tested the hypothesis that in motor sequences, the interval between successive movements is critical for the type of representation that develops. Participants practiced two 7-key sequences in the context of a discrete sequence production (DSP) task. The 0-RSI group practiced these sequences with response stimulus intervals (RSIs) of 0, which is typical for the DSP task, while the long-RSI group practiced the same sequences with unpredictable RSIs between 500 and 2000 ms. The ensuing test phase examined performance of these familiar and of unfamiliar sequences for both groups under both RSI regimes. The results support our hypothesis that the motor chunks that 0-RSI participants developed could not be used with long RSIs, whereas the long-RSI participants developed sequence representations that cannot be used with 0 RSIs. A new, computerized, sequence awareness task showed that long-RSI participants had limited sequence knowledge. The sequencing skill developed by long-RSI participants can, therefore, not have been based on explicit knowledge.

The Role of Motor Activity in Insight Problem Solving (the Case of the Nine-Dot Problem)

Attempts to estimate the contribution made by motor activity to insight problem solving is hindered by a lack of detailed description of motor behavior. The goal of this study was to develop and put to the test a novel method for studying the dynamics of insight problem solving based on a quantitative analysis of ongoing motor activity. As a proper problem model, we chose the nine-dot problem (Maier, 1930), in which solvers had to draw a sequence of connected line segments. Instead of using the traditional pen-and-paper way of solving the nine-dot problem we asked participants to use their index finger to draw line segments on the surface of a tablet computer. We are arguing that successful studying of the role of motor activity during problem solving requires the distinction between its instrumental and functional role. We considered the functional role on the motor activity as closely related to the on-line mode of motor planning. The goal of Experiment 1 was to explore the potential power of the method and, at the same time, to assay the patterns of motor activity related to on-line and off-line modes of motor planning. Experiments 2 and 3 were designed to uncover the potential impact of preliminary motor training on the motor output of successful and unsuccessful problem solvers. In these experiments, we tested hypotheses on how preliminary motor training, which presumably played a functional role in Experiment 2 and an instrumental role in Experiment 3, affects the motor activity of a problem solver and hence their effectiveness in solving the problem. The three experiments showed consistent results. They suggest that successful solving of the nine-dot problem relies upon the functional role of motor activity and requires both off-line and on-line modes of motor planning, with the latter helping to overcome the perceptual constraints imposed by a spatial arrangement of the nine dots. The method that we applied allows for systematic comparison between successful and unsuccessful problem solvers based on the quantitative parameters of their motor activity. Through it, we found new specific patterns of motor activity that differentiate successful and unsuccessful solvers.

Skill acquisition as a function of age, hand and task difficulty: Interactions between cognition and action

Some activities can be meaningfully dichotomised as 'cognitive' or 'sensorimotor' in nature-but many cannot. This has radical implications for understanding activity limitation in disability. For example, older adults take longer to learn the serial order of a complex sequence but also exhibit slower, more variable and inaccurate motor performance. So is their impaired skill acquisition a cognitive or motor deficit? We modelled sequence learning as a process involving a limited capacity buffer (working memory), where reduced performance restricts the number of elements that can be stored. To test this model, we examined the relationship between motor performance and sequence learning. Experiment 1 established that older adults were worse at learning the serial order of a complex sequence. Experiment 2 found that participants showed impaired sequence learning when the non-preferred hand was used. Experiment 3 confirmed that serial order learning is impaired when motor demands increase (as the model predicted). These results can be captured by reinforcement learning frameworks which suggest sequence learning will be constrained both by an individual's sensorimotor ability and cognitive capacity.

Neural Competitive Queuing of Ordinal Structure Underlies Skilled Sequential Action

Fluent retrieval and execution of movement sequences is essential for daily activities, but the neural mechanisms underlying sequence planning remain elusive. Here participants learned finger press sequences with different orders and timings and reproduced them in a magneto-encephalography (MEG) scanner. We classified the MEG patterns for each press in the sequence and examined pattern dynamics during preparation and production. Our results demonstrate the “competitive queuing” (CQ) of upcoming action representations, extending previous computational and non-human primate recording studies to non-invasive measures in humans. In addition, we show that CQ reflects an ordinal template that generalizes across specific motor actions at each position. Finally, we demonstrate that CQ predicts participants’ production accuracy and originates from parahippocampal and cerebellar sources. These results suggest that the brain learns and controls multiple sequences by flexibly combining representations of specific actions and interval timing with high-level, parallel representations of sequence position.

•

Non-invasive recordings in humans show “competitive queuing” of upcoming actions

•

Queuing gradient mainly reflects a high-level template of sequence position

•

Queuing gradient originates in ipsilateral parahippocampal and cerebellar areas

•

Strength of queuing gradient predicts production accuracy

The neuroanatomy of speech sequencing at the syllable level

Correctly ordering a sequence of speech sounds is a crucial aspect of speech production. Although studies have yielded a rich body of data on the neural substrates of visuomotor sequencing and sequence learning, research on brain regions and their functions involving speech sequence production hasn’t attracted much attention until recently. Previous functional MRI studies manipulating the complexity of sequences at the phonemic, syllabic, and suprasyllabic levels have revealed a network of motor-related cortical and sub-cortical speech regions. In this study, we directly compared human brain activity measured with functional MRI during processing of a sequence of syllables compared with the same syllables processed individually. Among a network of regions independently identified as being part of the sensorimotor circuits for speech production, only the left posterior inferior frontal gyrus (pars opercularis, lIFG), the supplementary motor area (SMA), and the left inferior parietal lobe (lIPL) responded more during the production of syllable sequences compared to producing the same syllables articulated one at a time.

Incorporation of a working memory strategy in GMT to facilitate serial-order behaviour in brain-injured patients

Goal Management Training (GMT) is an effective method for improving disorganised behaviour in multistep real-life tasks after brain damage. In the present study we incorporated Working Memory Training (WMT) in GMT to explore their combined efficacy in facilitating the serial-order maintenance of the steps that had to be learned. GMT+WMT was compared to a control WMT designed for other purposes. For this purpose 18 brain-injured patients (aged 20-54), who were at least 4 months post-onset, were randomly assigned to either the GMT+WMT or the WMT treatment. Inclusion was based on a baseline score of less than six correct steps on each of two multistep everyday tasks. Alternative versions of these tasks were used as primary outcome tasks. Pre-treatment and post-treatment comparisons of scores on these primary tasks and on several secondary neuropsychological measures were collected. The results show that post-treatment the GMT+WMT group performed significantly better than the WMT group on the primary outcome measures and on several ecologically valid executive tests that demanded a step-by-step maintenance of multiple actions. Time effects were found for both groups on the secondary measures. Other measures showed no significant differences. We conclude that our results support the efficacy of the combined GMT+WMT in facilitating performance in everyday multistep tasks.

Songbirds work around computational complexity by learning song vocabulary independently of sequence

While acquiring motor skills, animals transform their plastic motor sequences to match desired targets. However, because both the structure and temporal position of individual gestures are adjustable, the number of possible motor transformations increases exponentially with sequence length. Identifying the optimal transformation towards a given target is therefore a computationally intractable problem. Here we show an evolutionary workaround for reducing the computational complexity of song learning in zebra finches. We prompt juveniles to modify syllable phonology and sequence in a learned song to match a newly introduced target song. Surprisingly, juveniles match each syllable to the most spectrally similar sound in the target, regardless of its temporal position, resulting in unnecessary sequence errors, that they later try to correct. Thus, zebra finches prioritize efficient learning of syllable vocabulary, at the cost of inefficient syntax learning. This strategy provides a non-optimal but computationally manageable solution to the task of vocal sequence learning.

Bimanual tapping of a syncopated rhythm reveals hemispheric preferences for relative movement frequencies

In bimanual multifrequency tapping, right-handers commonly use the right hand to tap the relatively higher rate and the left hand to tap the relatively lower rate. This could be due to hemispheric specializations for the processing of relative frequencies. An extension of the double-filtering-by-frequency theory to motor control proposes a left hemispheric specialization for the control of relatively high and a right hemispheric specialization for the control of relatively low tapping rates. We investigated timing variability and rhythmic accentuation in right handers tapping mono- and multifrequent bimanual rhythms to test the predictions of the double-filtering-by-frequency theory. Yet, hemispheric specializations for the processing of relative tapping rates could be masked by a left hemispheric dominance for the control of known sequences. Tapping was thus either performed in an overlearned quadruple meter (tap of the slow rhythm on the first auditory beat) or in a syncopated quadruple meter (tap of the slow rhythm on the fourth auditory beat). Independent of syncopation, the right hand outperformed the left hand in timing accuracy for fast tapping. A left hand timing benefit for slow tapping rates as predicted by the double-filtering-by-frequency theory was only found in the syncopated tapping group. This suggests a right hemisphere preference for the control of slow tapping rates when rhythms are not overlearned. Error rates indicate that overlearned rhythms represent hierarchically structured meters that are controlled by a single timer that could potentially reside in the left hemisphere.

Structural Pathways Supporting Swift Acquisition of New Visuomotor Skills

Human skill learning requires fine-scale coordination of distributed networks of brain regions linked by white matter tracts to allow for effective information transmission. Yet how individual differences in these anatomical pathways may impact individual differences in learning remains far from understood. Here, we test the hypothesis that individual differences in structural organization of networks supporting task performance predict individual differences in the rate at which humans learn a visuomotor skill. Over the course of 6 weeks, 20 healthy adult subjects practiced a discrete sequence production task, learning a sequence of finger movements based on discrete visual cues. We collected structural imaging data, and using deterministic tractography generated structural networks for each participant to identify streamlines connecting cortical and subcortical brain regions. We observed that increased white matter connectivity linking early visual regions was associated with a faster learning rate. Moreover, the strength of multiedge paths between motor and visual modules was also correlated with learning rate, supporting the potential role of extended sets of polysynaptic connections in successful skill acquisition. Our results demonstrate that estimates of anatomical connectivity from white matter microstructure can be used to predict future individual differences in the capacity to learn a new motor-visual skill, and that these predictions are supported both by direct connectivity in visual cortex and indirect connectivity between visual cortex and motor cortex.

Motor Sequence Learning in Healthy Older Adults Is Not Necessarily Facilitated by Transcranial Direct Current Stimulation (tDCS)

Background: Transcranial Direct Current Stimulation (tDCS) of the primary motor cortex (M1) can modulate neuronal activity, and improve performance of basic motor tasks. The possibility that tDCS could assist in rehabilitation (e.g., for paresis post-stroke) offers hope but the evidence base is incomplete, with some behavioural studies reporting no effect of tDCS on complex motor learning. Older adults who show age-related decline in movement and learning (skills which tDCS could potentially facilitate), are also under-represented within tDCS literature. To address these issues, we examined whether tDCS would improve motor sequence learning in healthy young and older adults. Methods: In Experiment One, young participants learned 32 aiming movements using their preferred (right) hand whilst receiving: (i) 30 min Anodal Stimulation of left M1; (ii) 30 min Cathodal Stimulation of right M1; or (iii) 30 min Sham. Experiment Two used a similar task, but with older adults receiving Anodal Stimulation or Sham. Results: Whilst motor learning occurred in all participants, tDCS did not improve the rate or accuracy of motor learning for either age group. Conclusion: Our results suggest that the effects of tDCS may be limited to motor performance with no clear beneficial effects for motor learning.

Unitization of route knowledge

There are many theories that explain how route knowledge is acquired. We examined here if the sequence of elements that are part of a route can become integrated into a single unit, to the extent that the processing of individual transitions may only be relevant in the context of this entire unit. In Experiments 1 and 2, participants learned a route for ten blocks. Subsequently, at test they were intermittently exposed to the same training route along with a novel route which contained partial overlap with the original training route. Results show that the very same stimulus, appearing in the very same location, requiring the very same response (e.g., left turn), was responded to significantly faster in the context of the original training route than in the novel route. In Experiment 3, we employed a modified paradigm containing landmarks and two matched routes which were both substantially longer and contained a greater degree of overlap than the routes in Experiments 1 and 2. Results were replicated, namely, the same overlapping route segment, common to both routes, was performed significantly slower when appearing in the context of a novel than the original route. Furthermore, the difference between the overlapping segments was similar to the difference observed for the non-overlapping segments, i.e., an old route segment in the context of a novel route was processed as if it were an entirely novel segment. We discuss the results in relation to binding, chunking, and transfer effects, as well as potential practical implications.

Age-related effects of increasing postural challenge on eye movement onset latencies to visual targets

When a single light cue is given in the visual field, our eyes orient towards it with an average latency of 200 ms. If a second cue is presented at or around the time of the response to the first, a secondary eye movement occurs that represents a reorientation to the new target. While studies have shown that eye movement latencies to 'single-step' targets may or may not be lengthened with age, secondary eye movements (during 'double-step' displacements) are significantly delayed with increasing age. The aim of this study was to investigate whether the postural challenge posed simply by standing (as opposed to sitting) results in significantly longer eye movement latencies in older adults compared to the young. Ten young (<35 years) and 10 older healthy adults (>65 years) participated in the study. They were required to fixate upon a central target and move their eyes in response to 2 types of stimuli: (1) a single-step perturbation of target position either 15° to the right or left and (2) a double-step target displacement incorporating an initial target jump to the right or left by 15°, followed after 200 ms, by a shift of target position to the opposite side (e.g. +15° then -15°). All target displacement conditions were executed in sit and stand positions with the participant at the same distance from the targets. Eye movements were recorded using electro-oculography. Older adults did not show significantly longer eye movement latencies than the younger adults for single-step target displacements, and postural configuration (stand compared to sit) had no effect upon latencies for either group. We categorised double-step trials into those during which the second light changed after or before the onset of the eye shift to the first light. For the former category, young participants showed faster secondary eye shifts to the second light in the standing position, while the older adults did not. For the latter category of double-step trial, young participants showed no significant difference between sit and stand secondary eye movement latencies, but older adults were significantly longer standing compared to sitting. The older adults were significantly longer than the younger adults across both postural conditions, regardless of when the second light change occurred during the eye shift to the first light. We suggest that older adults require greater time and perhaps attentional processes to execute eye movements to unexpected changes in target position when faced with the need to maintain standing balance.

Disruption of Broca's Area Alters Higher-order Chunking Processing during Perceptual Sequence Learning

Because Broca's area is known to be involved in many cognitive functions, including language, music, and action processing, several attempts have been made to propose a unifying theory of its role that emphasizes a possible contribution to syntactic processing. Recently, we have postulated that Broca's area might be involved in higher-order chunk processing during implicit learning of a motor sequence. Chunking is an information-processing mechanism that consists of grouping consecutive items in a sequence and is likely to be involved in all of the aforementioned cognitive processes. Demonstrating a contribution of Broca's area to chunking during the learning of a nonmotor sequence that does not involve language could shed new light on its function. To address this issue, we used offline MRI-guided TMS in healthy volunteers to disrupt the activity of either the posterior part of Broca's area (left Brodmann's area [BA] 44) or a control site just before participants learned a perceptual sequence structured in distinct hierarchical levels. We found that disruption of the left BA 44 increased the processing time of stimuli representing the boundaries of higher-order chunks and modified the chunking strategy. The current results highlight the possible role of the left BA 44 in building up effector-independent representations of higher-order events in structured sequences. This might clarify the contribution of Broca's area in processing hierarchical structures, a key mechanism in many cognitive functions, such as language and composite actions.

Allowing time to consolidate knowledge gained through random practice facilitates later novel motor sequence acquisition

Two experiments were conducted to examine the efficacy of random (RP) and blocked practice (BP) for enhancing later motor learning. Each experiment involved practicing three unique seven key serial reaction time (SRT) tasks in either a blocked or random format followed by practice of a novel SRT task either 2-min (Experiment 1) or 24-h (Experiment 2) later. While the expected benefit of RP for retention was present in both experiments, in Experiment 1 there was no advantage from prior RP for new learning. Experiment 2 explored the possibility that increasing the interval, from 2-min to 24-h, between BP or RP and practice of the novel motor task might allow consolidation of sequence knowledge acquired during BP or RP which in turn might facilitate new learning. As a result of the additional time between training bouts RP facilitated the rate at which the novel motor task was acquired. Interestingly, when this additional time was provided, both BP and RP supported (a) a performance saving for the first trial with the novel task, and (b) an offline improvement in performance across a 24-h interval not present when only the novel motor task was practiced. The latter benefits for new learning may have resulted from exposure to prior physical practice per se. or practice variability. These data are discussed with respect to (a) future learning benefits from prior experience training with greater CI, and (b) the importance of memory consolidation for motor learning.

Off-line simulation inspires insight: A neurodynamics approach to efficient robot task learning

There is currently an increasing demand for robots able to acquire the sequential organization of tasks from social learning interactions with ordinary people. Interactive learning-by-demonstration and communication is a promising research topic in current robotics research. However, the efficient acquisition of generalized task representations that allow the robot to adapt to different users and contexts is a major challenge. In this paper, we present a dynamic neural field (DNF) model that is inspired by the hypothesis that the nervous system uses the off-line re-activation of initial memory traces to incrementally incorporate new information into structured knowledge. To achieve this, the model combines fast activation-based learning to robustly represent sequential information from single task demonstrations with slower, weight-based learning during internal simulations to establish longer-term associations between neural populations representing individual subtasks. The efficiency of the learning process is tested in an assembly paradigm in which the humanoid robot ARoS learns to construct a toy vehicle from its parts. User demonstrations with different serial orders together with the correction of initial prediction errors allow the robot to acquire generalized task knowledge about possible serial orders and the longer term dependencies between subgoals in very few social learning interactions. This success is shown in a joint action scenario in which ARoS uses the newly acquired assembly plan to construct the toy together with a human partner.

An Adapting Auditory-motor Feedback Loop Can Contribute to Generating Vocal Repetition

Consecutive repetition of actions is common in behavioral sequences. Although integration of sensory feedback with internal motor programs is important for sequence generation, if and how feedback contributes to repetitive actions is poorly understood. Here we study how auditory feedback contributes to generating repetitive syllable sequences in songbirds. We propose that auditory signals provide positive feedback to ongoing motor commands, but this influence decays as feedback weakens from response adaptation during syllable repetitions. Computational models show that this mechanism explains repeat distributions observed in Bengalese finch song. We experimentally confirmed two predictions of this mechanism in Bengalese finches: removal of auditory feedback by deafening reduces syllable repetitions; and neural responses to auditory playback of repeated syllable sequences gradually adapt in sensory-motor nucleus HVC. Together, our results implicate a positive auditory-feedback loop with adaptation in generating repetitive vocalizations, and suggest sensory adaptation is important for feedback control of motor sequences.

Repetitions are common in animal vocalizations. Songs of many songbirds contain syllables that repeat a variable number of times, with non-Markovian distributions of repeat counts. The neural mechanism underlying such syllable repetitions is unknown. In this work, we show that auditory feedback plays an important role in sustaining syllable repetitions in the Bengalese finch. Deafening reduces syllable repetitions and skews the repeat number distribution towards short repeats. These effects are explained with our computational model, which suggests that syllable repeats are initially sustained by auditory feedback to the neural networks that drive the syllable production. The feedback strength weakens as the syllable repeats, increasing the likelihood that the syllable repetition stops. Neural recordings confirm such adaptation of auditory feedback to the auditory-motor circuit in the Bengalese finch. Our results suggests that sensory feedback can directly impact repetitions in motor sequences, and may provide insights into neural mechanisms of speech disorders such as stuttering.

Role of the site of synaptic competition and the balance of learning forces for Hebbian encoding of probabilistic Markov sequences

The majority of distinct sensory and motor events occur as temporally ordered sequences with rich probabilistic structure. Sequences can be characterized by the probability of transitioning from the current state to upcoming states (forward probability), as well as the probability of having transitioned to the current state from previous states (backward probability). Despite the prevalence of probabilistic sequencing of both sensory and motor events, the Hebbian mechanisms that mold synapses to reflect the statistics of experienced probabilistic sequences are not well understood. Here, we show through analytic calculations and numerical simulations that Hebbian plasticity (correlation, covariance, and STDP) with pre-synaptic competition can develop synaptic weights equal to the conditional forward transition probabilities present in the input sequence. In contrast, post-synaptic competition can develop synaptic weights proportional to the conditional backward probabilities of the same input sequence. We demonstrate that to stably reflect the conditional probability of a neuron's inputs and outputs, local Hebbian plasticity requires balance between competitive learning forces that promote synaptic differentiation and homogenizing learning forces that promote synaptic stabilization. The balance between these forces dictates a prior over the distribution of learned synaptic weights, strongly influencing both the rate at which structure emerges and the entropy of the final distribution of synaptic weights. Together, these results demonstrate a simple correspondence between the biophysical organization of neurons, the site of synaptic competition, and the temporal flow of information encoded in synaptic weights by Hebbian plasticity while highlighting the utility of balancing learning forces to accurately encode probability distributions, and prior expectations over such probability distributions.

Scale-Free Navigational Planning by Neuronal Traveling Waves

Spatial navigation and planning is assumed to involve a cognitive map for evaluating trajectories towards a goal. How such a map is realized in neuronal terms, however, remains elusive. Here we describe a simple and noise-robust neuronal implementation of a path finding algorithm in complex environments. We consider a neuronal map of the environment that supports a traveling wave spreading out from the goal location opposite to direction of the physical movement. At each position of the map, the smallest firing phase between adjacent neurons indicate the shortest direction towards the goal. In contrast to diffusion or single-wave-fronts, local phase differences build up in time at arbitrary distances from the goal, providing a minimal and robust directional information throughout the map. The time needed to reach the steady state represents an estimate of an agent’s waiting time before it heads off to the goal. Given typical waiting times we estimate the minimal number of neurons involved in the cognitive map. In the context of the planning model, forward and backward spread of neuronal activity, oscillatory waves, and phase precession get a functional interpretation, allowing for speculations about the biological counterpart.

Habits as action sequences: hierarchical action control and changes in outcome value

Goal-directed action involves making high-level choices that are implemented using previously acquired action sequences to attain desired goals. Such a hierarchical schema is necessary for goal-directed actions to be scalable to real-life situations, but results in decision-making that is less flexible than when action sequences are unfolded and the decision-maker deliberates step-by-step over the outcome of each individual action. In particular, from this perspective, the offline revaluation of any outcomes that fall within action sequence boundaries will be invisible to the high-level planner resulting in decisions that are insensitive to such changes. Here, within the context of a two-stage decision-making task, we demonstrate that this property can explain the emergence of habits. Next, we show how this hierarchical account explains the insensitivity of over-trained actions to changes in outcome value. Finally, we provide new data that show that, under extended extinction conditions, habitual behaviour can revert to goal-directed control, presumably as a consequence of decomposing action sequences into single actions. This hierarchical view suggests that the development of action sequences and the insensitivity of actions to changes in outcome value are essentially two sides of the same coin, explaining why these two aspects of automatic behaviour involve a shared neural structure.