Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives

Contributions from associative and explicit sequence knowledge to the execution of discrete keying sequences

Research has provided many indications that highly practiced 6-key sequences are carried out in a chunking mode in which key-specific stimuli past the first are largely ignored. When in such sequences a deviating stimulus occasionally occurs at an unpredictable location, participants fall back to responding to individual stimuli (Verwey & Abrahamse, 2012). The observation that in such a situation execution still benefits from prior practice has been attributed to the possibility to operate in an associative mode. To better understand the contribution to the execution of keying sequences of motor chunks, associative sequence knowledge and also of explicit sequence knowledge, the present study tested three alternative accounts for the earlier finding of an execution rate increase at the end of 6-key sequences performed in the associative mode. The results provide evidence that the earlier observed execution rate increase can be attributed to the use of explicit sequence knowledge. In the present experiment this benefit was limited to sequences that are executed at the moderately fast rates of the associative mode, and occurred at both the earlier and final elements of the sequences.

Control of spoken vowel acoustics and the influence of phonetic context in human speech sensorimotor cortex

Speech production requires the precise control of vocal tract movements to generate individual speech sounds (phonemes) which, in turn, are rapidly organized into complex sequences. Multiple productions of the same phoneme can exhibit substantial variability, some of which is inherent to control of the vocal tract and its biomechanics, and some of which reflects the contextual effects of surrounding phonemes ("coarticulation"). The role of the CNS in these aspects of speech motor control is not well understood. To address these issues, we recorded multielectrode cortical activity directly from human ventral sensory-motor cortex (vSMC) during the production of consonant-vowel syllables. We analyzed the relationship between the acoustic parameters of vowels (pitch and formants) and cortical activity on a single-trial level. We found that vSMC activity robustly predicted acoustic parameters across vowel categories (up to 80% of variance), as well as different renditions of the same vowel (up to 25% of variance). Furthermore, we observed significant contextual effects on vSMC representations of produced phonemes that suggest active control of coarticulation: vSMC representations for vowels were biased toward the representations of the preceding consonant, and conversely, representations for consonants were biased toward upcoming vowels. These results reveal that vSMC activity for phonemes are not invariant and provide insight into the cortical mechanisms of coarticulation.

On the importance of being first: serial order effects in the interaction between action plans and ongoing actions

When we plan sequences of actions, we must hold some elements of the sequence in working memory (WM) while we execute others. Research shows that execution of an action can be delayed if it partly overlaps (vs. does not overlap) with another action plan maintained in WM (partial repetition cost). However, it is not known whether all features of the action maintained in WM interfere equally with current actions. Most serial models of memory and action assume that interference will be equal, because all action features in the sequence should be activated to an equal degree in parallel; others assume that action features earlier in the sequence will interfere more than those later in the sequence, because earlier features will be more active. Using a partial repetition paradigm, this study examined whether serial position of action features in action sequences maintained in WM have an influence on current actions. Two stimulus events occurred in a sequence, and participants planned and maintained an action sequence to the first event (action A) in WM while executing a speeded response to the second event (action B). Results showed delayed execution of action B when it matched the first feature in the action A sequence (partial repetition cost), but not when it matched the last feature. These findings suggest that serial order is represented in the action plan prior to response execution, consistent with models that assume that serial order is represented by a primacy gradient of parallel feature activation prior to action execution.

Learning a keying sequence you never executed: evidence for independent associative and motor chunk learning

A substantial amount of research has addressed how people learn and control movement sequences. Recent results suggested that practice with discrete key pressing sequences results in two types of sequence learning: associative learning and motor chunk development (Verwey & Abrahamse, 2012). In the present study, we addressed whether in keying sequences of limited length associative learning develops also when the use of the chunking mode is prevented by introducing during practice random deviants. In line with the notion of two different learning mechanisms, the present results indicate that associative sequence learning develops when motor chunks cannot be developed during practice. This confirms the notion that motor chunks do not rely on these associations. In addition, experience with a particular execution mode during the practice phase seems to benefit subsequent use of that mode with unfamiliar and random sequences. Also, participants with substantial video-gaming experience were faster in executing discrete keying sequences in the chunking mode. These last two results may point to the development of a general ability to produce movement sequences in the chunking mode.

A suppression hierarchy among competing motor programs drives sequential grooming in Drosophila

Motor sequences are formed through the serial execution of different movements, but how nervous systems implement this process remains largely unknown. We determined the organizational principles governing how dirty fruit flies groom their bodies with sequential movements. Using genetically targeted activation of neural subsets, we drove distinct motor programs that clean individual body parts. This enabled competition experiments revealing that the motor programs are organized into a suppression hierarchy; motor programs that occur first suppress those that occur later. Cleaning one body part reduces the sensory drive to its motor program, which relieves suppression of the next movement, allowing the grooming sequence to progress down the hierarchy. A model featuring independently evoked cleaning movements activated in parallel, but selected serially through hierarchical suppression, was successful in reproducing the grooming sequence. This provides the first example of an innate motor sequence implemented by the prevailing model for generating human action sequences.

DOI:http://dx.doi.org/10.7554/eLife.02951.001

Anyone who has ever lived with a cat is familiar with its grooming behavior. This innate behavior follows a particular sequence as the cat methodically cleans its body parts one-by-one. Many animals also have grooming habits, even insects such as fruit flies. The fact that grooming sequences are seen across such different species suggests that this behavior is important for survival. Nevertheless, how the brain organizes grooming sequences, or other behaviors that involve a sequence of tasks, is not well understood.

Fruit flies make a good model for studying grooming behavior for a couple of reasons. First, they are fastidious cleaners. When coated with dust they will faithfully carry out a series of cleaning tasks to clean each body part. Second, there are many genetic tools and techniques that researchers can use to manipulate the fruit flies' behaviors. One technique allows specific brain cells to be targeted and activated to trigger particular behaviors.

Seeds et al. used these sophisticated techniques, computer modeling, and behavioral observations to uncover how the brains of fruit flies orchestrate a grooming sequence. Dust-covered flies follow a predictable sequence of cleaning tasks: beginning by using their front legs to clean their eyes, they then clean their antennae and head. This likely helps to protect their sensory organs. Next, they move on to the abdomen, possibly to ensure that dust doesn't interfere with their ability to breathe. Wings and thorax follow last. Periodically, the flies stop to rub their legs together to remove any accumulated dust before resuming the cleaning sequence.

Seeds et al. activated different sets of brain cells one-by-one to see if they could trigger a particular grooming task and found that individual cleaning tasks could be triggered, in the absence of dust, by stimulating a specific group of brain cells. This suggests each cleaning task is a discrete behavior controlled by a subset of cells. Then Seeds et al. tried to stimulate more than one cleaning behavior at a time; they discovered that wing-cleaning suppressed thorax-cleaning, abdomen-cleaning suppressed both of these, and head-cleaning suppressed all the others. This suggests that a ‘hierarchy’ exists in the brain that exactly matches the sequence that flies normally follow as they clean their body parts.

By learning more about how the brain coordinates grooming sequences, the findings of Seeds et al. may also provide insights into other behaviors that involve a sequence of tasks, such as nest building in animals or typing in humans. Following on from this work, one of the next challenges will be to see if such behaviors also use a ‘suppression hierarchy’ to ensure that individual tasks are carried out in the right order.

DOI:http://dx.doi.org/10.7554/eLife.02951.002

The coding of repetitions and alternations in action sequences: spatial or relational?

We used variants of the Simon task to investigate whether repetitions and alternations in short keypress sequences are represented by spatial or relational codes. With spatial coding, either absolute or relative location would be used for coding the second response in a sequence. With relational coding, the second response would be coded in terms of a non-spatial relation to the first one (e.g., left response-same response, for a repetition). In three experiments with different imperative stimuli, we compared Simon effects across three experimental conditions, a single-response condition, a response-repetition condition (each stimulus required two keypresses on the same side, e.g., left-left), and a response-alternation condition (each stimulus required a keypress on each side, e.g., left-right). When compared to the single-response condition, spatial coding of the second response should modulate the Simon effect (i.e., response conflict) in selecting the first response because spatial coding of the second response produces additional dimensional overlap of stimulus and the second-response code. We observed Simon effects in the response times of first responses in each condition, and they increased in the response-alternation condition, but not in the response-repetition condition. The findings suggest relational coding of response repetitions, and spatial coding of response alternations.

Effects of direction and index of difficulty on aiming movements after stroke

BACKGROUND

Brain hemispheres play different roles in the control of aiming movements that are impaired after unilateral stroke. It is not clear whether those roles are influenced by the direction and the difficulty of the task.

OBJECTIVE

To evaluate the influence of direction and index of difficulty (ID) of the task on performance of ipsilesional aiming movements after unilateral stroke.

METHODS

Ten individuals with right hemisphere stroke, ten with left hemisphere stroke, and ten age- and gender-matched controls performed the aiming movements on a digitizing tablet as fast as possible. Stroke individuals used their ipsilesional arm. The direction (ipsilateral or contralateral), size (0.8 or 1.6 cm), and distance (9 or 18 cm) of the targets, presented on a monitor, were manipulated and determined to be of different ID (3.5, 4.5, and 5.5). Results. Individuals with right hemisphere lesion were more sensitive to ID of the task, affecting planning and final position accuracy. Left hemisphere lesion generated slower and less smooth movements and was more influenced by target distance. Contralateral movements and higher ID increased planning demands and hindered movement execution.

CONCLUSION

Right and left hemisphere damages are differentially influenced by task constraints which suggest their complementary roles in the control of aiming movements.

Sleep-related offline learning in a complex arm movement sequence

Sleep is known to elicit off-line improvements of newly learned procedural skills, a phenomenon attributed to enhancement consolidation of an internal skill representation. In the motor domain, enhancement consolidation has been reported almost exclusively for sequential-finger-tapping skills. The aim of the present study was to extend the notion of sleep-related enhancement consolidation to tasks closer to everyday motor skills. This was achieved by employing a sequence of unrestrained reaching-movements with the non-dominant arm. Fifteen reaching-movements had to be executed as fast as possible, following a spatial pattern in the horizontal plane. Terminating each movement, a peg had to be fitted into a hole on an electronic pegboard. Two experimental groups received initial training, one in the evening, the other one in the morning. Subsequently, performance in both groups was retested twelve, and again 24 hrs later. Thus, during retention each individual experienced a night of sleep, either followed or preceded by a wake interval. Performance error remained low throughout training and retests. Yet mean total execution time, indicative of task execution-speed, significantly decreased for all individuals throughout initial training (no group differences), and significantly decreased again in either group following nocturnal sleep, but not following wake. This finding does not appear to result merely from additional practice afforded at the time of retests, because only after a night of sleep individuals of both experimental groups also revealed performance improvement beyond that estimated from their initial training performance.

Learned spatiotemporal sequence recognition and prediction in primary visual cortex

Learning to recognize and predict temporal sequences is fundamental to sensory perception, and is impaired in several neuropsychiatric disorders, but little is known about where and how this occurs in the brain. We discovered that repeated presentations of a visual sequence over a course of days causes evoked response potentiation in mouse V1 that is highly specific for stimulus order and timing. Remarkably, after V1 is trained to recognize a sequence, cortical activity regenerates the full sequence even when individual stimulus elements are omitted. This novel neurophysiological report of sequence learning advances the understanding of how the brain makes “intelligent guesses” based on limited information to form visual percepts and suggests that it is possible to study the mechanistic basis of this high–level cognitive ability by studying low–level sensory systems.

Intracortical microstimulation of supplementary eye field impairs ability of monkeys to make serially ordered saccades

Neurons in the supplementary eye field (SEF) of the macaque monkey exhibit rank selectivity, firing differentially as a function of the phase attained during the performance of a task requiring the execution of saccades to a series of objects in fixed order. The activity of these neurons is commonly thought to represent ordinal position in the service of serial-order performance. However, there is little evidence causally linking neuronal activity in the SEF to sequential behavior. To explore the role of the SEF in serial-order performance, we delivered intracortical microstimulation while monkeys performed a task requiring them to make saccades to three objects in a fixed order on each trial. Microstimulation, considered on average across all SEF sites and all phases of the trial, affected saccadic kinematics. In particular, it prolonged the reaction time, increased the peak velocity, and slightly increased the amplitude of saccades. In addition, it interfered with the monkeys' ability to select the target appropriate to a given phase of the trial. The pattern of the errors was such as would be expected if microstimulation shifted the neural representation of ordinal position toward a later phase of the trial.

Modality-specific organization in the representation of sensorimotor sequences

Sensorimotor representations of movement sequences are hierarchically organized. Here we test the effects of different stimulus modalities on such organizations. In the visual group, participants responded to a repeated sequence of visually presented stimuli by depressing spatially compatible keys on a response pad. In the auditory group, learners were required to respond to auditorily presented stimuli, which had no direct spatial correspondence with the response keys: the lowest pitch corresponded to the leftmost key and the highest pitch to the rightmost key. We demonstrate that hierarchically and auto-organized sensorimotor representations are developed through practice, which are specific both to individuals and stimulus modalities. These findings highlight the dynamic and sensory-specific modulation of chunk processing during sensorimotor learning - sensorimotor chunking - and provide evidence that modality-specific mechanisms underlie the hierarchical organization of sequence representations.

Sequence complexity effects on speech production in healthy speakers and speakers with hypokinetic or ataxic dysarthria

The present study investigated the effects of sequence complexity, defined in terms of phonemic similarity and phonotoactic probability, on the timing and accuracy of serial ordering for speech production in healthy speakers and speakers with either hypokinetic or ataxic dysarthria. Sequences were comprised of strings of consonant-vowel (CV) syllables with each syllable containing the same vowel, /a/, paired with a different consonant. High complexity sequences contained phonemically similar consonants, and sounds and syllables that had low phonotactic probabilities; low complexity sequences contained phonemically dissimilar consonants and high probability sounds and syllables. Sequence complexity effects were evaluated by analyzing speech error rates and within-syllable vowel and pause durations. This analysis revealed that speech error rates were significantly higher and speech duration measures were significantly longer during production of high complexity sequences than during production of low complexity sequences. Although speakers with dysarthria produced longer overall speech durations than healthy speakers, the effects of sequence complexity on error rates and speech durations were comparable across all groups. These findings indicate that the duration and accuracy of processes for selecting items in a speech sequence is influenced by their phonemic similarity and/or phonotactic probability. Moreover, this robust complexity effect is present even in speakers with damage to subcortical circuits involved in serial control for speech.

Are all changes equal? comparing early and late changes in sequence learning

Although it is known that a change in a learned motor sequence slows performance down, it is yet unknown if this impairment varies depending on whether the changed element is early or late in the sequence. In Experiment 1, we showed greater impairment in performance when changing the third vs. the sixth element in a 7-element sequence. The impairment was greater for the deviant and the following elements than for the preceding ones. In Experiment 2, we replicated the results of Experiment 1 and expanded them by showing that a change in the third element of a 4-element sequence produced similar results to those of the late change condition in the long 7-element sequence. It is proposed that during practice, associative relations between the sequence elements are formed together with the representation of the whole chunk. Following the change in sequence, the chunk representation is impaired and performance mainly reflects the associative links between the elements. An early change hampers these associative relations to a greater extent than a late change, and as a consequence slows performance down more than a late change does. The implications and advantages of such a mechanism are discussed.

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device

Animal behavior often involves a temporally ordered sequence of actions learned from experience. Here we describe simulations of interconnected networks of spiking neurons that learn to generate patterns of activity in correct temporal order. The simulation consists of large-scale networks of thousands of excitatory and inhibitory neurons that exhibit short-term synaptic plasticity and spike-timing dependent synaptic plasticity. The neural architecture within each area is arranged to evoke winner-take-all (WTA) patterns of neural activity that persist for tens of milliseconds. In order to generate and switch between consecutive firing patterns in correct temporal order, a reentrant exchange of signals between these areas was necessary. To demonstrate the capacity of this arrangement, we used the simulation to train a brain-based device responding to visual input by autonomously generating temporal sequences of motor actions.

Control of automated behavior: insights from the discrete sequence production task

Work with the discrete sequence production (DSP) task has provided a substantial literature on discrete sequencing skill over the last decades. The purpose of the current article is to provide a comprehensive overview of this literature and of the theoretical progress that it has prompted. We start with a description of the DSP task and the phenomena that are typically observed with it. Then we propose a cognitive model, the dual processor model (DPM), which explains performance of (skilled) discrete key-press sequences. Key features of this model are the distinction between a cognitive processor and a motor system (i.e., motor buffer and motor processor), the interplay between these two processing systems, and the possibility to execute familiar sequences in two different execution modes. We further discuss how this model relates to several related sequence skill research paradigms and models, and we outline outstanding questions for future research throughout the paper. We conclude by sketching a tentative neural implementation of the DPM.

Evidence for graded central processing resources in a sequential movement task

In the present experiment, we examined slowing of the individual key presses of a familiar keying sequence by four different versions of a concurrent tone counting task. This was done to determine whether the same cognitive processor that has previously been assumed by the dual processor model (DPM) to initiate familiar keying sequences and assist in their execution, is involved also in the central processes of a very different task (viz. identifying tones and counting target tones). The present results confirm this hypothesis. They also suggest that in this particular situation the central processing resources underlying the cognitive processor can be distributed across the central processes of different tasks in a graded manner, rather than that they continue to behave like a single, central processor that serially switches between the central processes of the concurrently performed tasks. We argue that the production of highly practiced movement sequences can be considered automatic in the sense that execution of familiar movement sequences can continue without cognitive control once they have been initiated.

Development of temporal structure in zebra finch song

Zebra finch song has provided an excellent case study in the neural basis of sequence learning, with a high degree of temporal precision and tight links with precisely timed bursting in forebrain neurons. To examine the development of song timing, we measured the following four aspects of song temporal structure at four age ranges between 65 and 375 days posthatch: the mean durations of song syllables and the silent gaps between them, timing variability linked to song tempo, timing variability expressed independently across syllables and gaps, and transition probabilities between consecutive syllable pairs. We found substantial increases in song tempo between 65 and 85 days posthatch, due almost entirely to a shortening of gaps. We also found a decrease in tempo variability, also specific to gaps. Both the magnitude of the increase in tempo and the decrease in tempo variability were correlated on gap-by-gap basis with increases in the reliability of corresponding syllable transitions. Syllables had no systematic increase in tempo or decrease in tempo variability. In contrast to tempo parameters, both syllables and gaps showed an early sharp reduction in independent variability followed by continued reductions over the first year. The data suggest that links between syllable-based representations are strengthened during the later parts of the traditional period of song learning and that song rhythm continues to become more regular throughout the first year of life. Similar learning patterns have been identified in human sequence learning, suggesting a potentially rich area of comparative research.

Contributions from the left PMd and the SMA during sequence retrieval as determined by depth of training

The dorsal premotor cortex (PMd) and the supplementary motor area (SMA) are critical for the acquisition and expression of sequential behavior, but little is known regarding how these regions are recruited when we must simultaneously acquire multiple sequences under different amounts of training. We hypothesized that these regions contribute to the retrieval of sequences at different familiarity levels, with the left PMd supporting sequences of moderate familiarity and the SMA supporting sequences of greater familiarity. Double-pulse transcranial magnetic stimulation (TMS) was applied during the retrieval of six sequences previously learned under three different amounts of exposure during 30 days of training using a discrete sequence production task. TMS led to a significant interaction of sequence error between depth of training and stimulation location. Stimulation of the left PMd increased error during moderate sequence retrieval, whereas stimulation of the SMA increased error during the retrieval of both moderately and extensively trained sequences. The lack of a double dissociation fails to support a direct correspondence between brain region and putative behavioral learning stage. Instead, the interaction suggests that SMA and PMd support the expression of sequences over different, albeit overlapping, time scales. Separate analysis of sequence initiation time did not demonstrate any significant difference between moderately and extensively trained sequences. Instead, stimulation to either region quickened sequence initiation for these sequences, but not for those sequences with poor retrieval performance. This supports the general role of these premotor regions in the maintenance of specific sequence knowledge prior to movement onset.

Differential recruitment of the sensorimotor putamen and frontoparietal cortex during motor chunking in humans

Motor chunking facilitates movement production by combining motor elements into integrated units of behavior. Previous research suggests that chunking involves two processes: concatenation, aimed at the formation of motor-motor associations between elements or sets of elements, and segmentation, aimed at the parsing of multiple contiguous elements into shorter action sets. We used fMRI to measure the trial-wise recruitment of brain regions associated with these chunking processes as healthy subjects performed a cued-sequence production task. A dynamic network analysis identified chunking structure for a set of motor sequences acquired during fMRI and collected over 3 days of training. Activity in the bilateral sensorimotor putamen positively correlated with chunk concatenation, whereas a left-hemisphere frontoparietal network was correlated with chunk segmentation. Across subjects, there was an aggregate increase in chunk strength (concatenation) with training, suggesting that subcortical circuits play a direct role in the creation of fluid transitions across chunks.

Mapping sensorimotor sequences to word sequences: a connectionist model of language acquisition and sentence generation

In this article we present a neural network model of sentence generation. The network has both technical and conceptual innovations. Its main technical novelty is in its semantic representations: the messages which form the input to the network are structured as sequences, so that message elements are delivered to the network one at a time. Rather than learning to linearise a static semantic representation as a sequence of words, our network rehearses a sequence of semantic signals, and learns to generate words from selected signals. Conceptually, the network's use of rehearsed sequences of semantic signals is motivated by work in embodied cognition, which posits that the structure of semantic representations has its origin in the serial structure of sensorimotor processing. The rich sequential structure of the network's semantic inputs also allows it to incorporate certain Chomskyan ideas about innate syntactic knowledge and parameter-setting, as well as a more empiricist account of the acquisition of idiomatic syntactic constructions.

Distinct modes of executing movement sequences: reacting, associating, and chunking

Responding to individual key-specific stimuli in entirely unfamiliar keying sequences is said to involve a reaction mode. With practice, short keying sequences can be executed in the chunking mode. This is indicated by the first stimulus sufficing for rapid execution of the entire sequence. The present study explored whether an associative mode develops also in participants who practice short keying sequences. This associative mode would involve priming by earlier events of responses to external stimuli, and is believed to be responsible for skill in the Serial Reaction Time task. In the present study participants practiced two discrete 6-key sequences. In the ensuing test phase, participants were prevented from using the chunking mode by including two deviant stimuli in most sequences. The results from the remaining - unchanged - familiar sequences confirmed that participants no longer used the chunking mode, but as predicted by associative learning these sequences were executed faster than unfamiliar sequences.

Sequence Learning Under Uncertainty in Children: Self-Reflection vs. Self-Assertion

We know that stochastic feedback impairs children’s associative stimulus–response (S–R) learning (Crone et al., 2004a; Eppinger et al., 2009), but the impact of stochastic feedback on sequence learning that involves deductive reasoning has not been not tested so far. In the current study, 8- to 11-year-old children (N = 171) learned a sequence of four left and right button presses, LLRR, RRLL, LRLR, RLRL, LRRL, and RLLR, which needed to be deduced from feedback because no directional cues were given. One group of children experienced consistent feedback only (deterministic feedback, 100% correct). In this condition, green feedback on the screen indicated that the children had been right when they were right, and red feedback indicated that the children had been wrong when they were wrong. Another group of children experienced inconsistent feedback (stochastic feedback, 85% correct, 15% false), where in some trials, green feedback on the screen could signal that children were right when in fact they were wrong, and red feedback could indicate that they were wrong when in fact they had been right. Independently of age, children’s sequence learning in the stochastic condition was initially much lower than in the deterministic condition, but increased gradually and improved with practice. Responses toward positive vs. negative feedback varied with age. Children were increasingly able to understand that they could have been wrong when feedback indicated they were right (self-reflection), but they remained unable to understand that they could have been right when feedback indicated they were wrong (self-assertion).

Influence of cueing on the preparation and execution of untrained and trained complex motor responses

This study investigated the influence of cueing on the performance of untrained and trained complex motor responses. Healthy adults responded to a visual target by performing four sequential movements (complex response) or a single movement (simple response) of their middle finger. A visual cue preceded the target by an interval of 300, 1000, or 2000 ms. In Experiment 1, the complex and simple responses were not previously trained. During the testing session, the complex response pattern varied on a trial-by-trial basis following the indication provided by the visual cue. In Experiment 2, the complex response and the simple response were extensively trained beforehand. During the testing session, the trained complex response pattern was performed in all trials. The latency of the untrained and trained complex responses decreased from the short to the medium and long cue-target intervals. The latency of the complex response was longer than that of the simple response, except in the case of the trained responses and the long cue-target interval. These results suggest that the preparation of untrained complex responses cannot be completed in advance, this being possible, however, for trained complex responses when enough time is available. The duration of the 1st submovement, 1st pause and 2nd submovement of the untrained and the trained complex responses increased from the short to the long cue-target interval, suggesting that there is an increase of online programming of the response possibly related to the degree of certainty about the moment of target appearance.

Cell phones change the way we walk

Cell phone use among pedestrians leads to increased cognitive distraction, reduced situation awareness and increases in unsafe behavior. Performing a dual-task, such as talking or texting with a cell phone while walking, may interfere with working memory and result in walking errors. At baseline, thirty-three participants visually located a target 8m ahead; then vision was occluded and they were instructed to walk to the remembered target. One week later participants were assigned to either walk, walk while talking on a cell phone, or walk while texting on a cell phone toward the target with vision occluded. Duration and final location of the heel were noted. Linear distance traveled, lateral angular deviation from the start line, and gait velocity were derived. Changes from baseline to testing were analyzed with paired t-tests. Participants engaged in cell phone use presented with significant reductions in gait velocity (texting: 33% reduction, p=0.01; talking: 16% reduction, p=0.02). Moreover, participants who were texting while walking demonstrated a 61% increase in lateral deviation (p=0.04) and 13% increase in linear distance traveled (p=0.03). These results suggest that the dual-task of walking while using a cell phone impacts executive function and working memory and influences gait to such a degree that it may compromise safety. Importantly, comparison of the two cell phone conditions demonstrates texting creates a significantly greater interference effect on walking than talking on a cell phone.

Time scale hierarchies in the functional organization of complex behaviors

Traditional approaches to cognitive modelling generally portray cognitive events in terms of ‘discrete’ states (point attractor dynamics) rather than in terms of processes, thereby neglecting the time structure of cognition. In contrast, more recent approaches explicitly address this temporal dimension, but typically provide no entry points into cognitive categorization of events and experiences. With the aim to incorporate both these aspects, we propose a framework for functional architectures. Our approach is grounded in the notion that arbitrary complex (human) behaviour is decomposable into functional modes (elementary units), which we conceptualize as low-dimensional dynamical objects (structured flows on manifolds). The ensemble of modes at an agent’s disposal constitutes his/her functional repertoire. The modes may be subjected to additional dynamics (termed operational signals), in particular, instantaneous inputs, and a mechanism that sequentially selects a mode so that it temporarily dominates the functional dynamics. The inputs and selection mechanisms act on faster and slower time scales then that inherent to the modes, respectively. The dynamics across the three time scales are coupled via feedback, rendering the entire architecture autonomous. We illustrate the functional architecture in the context of serial behaviour, namely cursive handwriting. Subsequently, we investigate the possibility of recovering the contributions of functional modes and operational signals from the output, which appears to be possible only when examining the output phase flow (i.e., not from trajectories in phase space or time).

In most established approaches to cognitive modelling, cognitive events are treated as ‘discrete’ states, thus passing by the continuous nature of cognitive processes. In contrast, some novel approaches explicitly acknowledge cognition’s temporal structure but provides no entry points into cognitive categorization of events and experiences. We attempt to incorporate both aspects in a new framework, which departs from the established idea that complex (human) behaviour is made up of elementary functional ‘building blocks’, referred to as modes. We model these as mathematical objects that are inherently dynamic (i.e., account for change over time). A mechanism sequentially selects the modes required and binds them together to compose complex behaviours. These modes may be subjected to brief inputs. The ensemble of these three ingredients, which influence one another and operate on different time scales, constitutes a functional architecture. We illustrate the architecture via cursive handwriting simulations, and investigate the possibility of recovering the contributions of the architecture from the written word. This appears possible only when focussing on the dynamic modes.

Independence of serial position and response conflict in pre-planned manual response sequences

The present study investigates the effect of spatial stimulus-response correspondence (i.e. Simon effect) in pre-planned manual response sequences. Participants performed pre-cued response sequences consisting of three (Experiment 1) or four (Experiments 2 and 3) key-presses at different locations. Importantly, participants performed each response to a visual go signal, which appeared at a location corresponding to one response in the sequence. This task allowed investigating interference gradients across spatially noncorresponding conditions. We observed a Simon effect at each serial position, that is, RT for the corresponding condition was always shorter than RT for each noncorresponding condition. However, we failed to observe interference gradients from both preceding and subsequent responses in the sequence. These results are inconsistent with (1) a primacy gradient of activations representing serial order and (2) the temporary suppression of an executed response as a mechanism for preventing response repetitions. However, results provide indirect evidence for positional models of serial order.

Motor skill learning in the middle-aged: limited development of motor chunks and explicit sequence knowledge

The present study examined whether middle-aged participants, like young adults, learn movement patterns by preparing and executing integrated sequence representations (i.e., motor chunks) that eliminate the need for external guidance of individual movements. Twenty-four middle-aged participants (aged 55-62) practiced two fixed key press sequences, one including three and one including six key presses in the discrete sequence production task. Their performance was compared with that of 24 young adults (aged 18-28). In the middle-aged participants motor chunks as well as explicit sequence knowledge appeared to be less developed than in the young adults. This held especially with respect to the unstructured 6-key sequences in which most middle-aged did not develop independence of the key-specific stimuli and learning seems to have been based on associative learning. These results are in line with the notion that sequence learning involves several mechanisms and that aging affects the relative contribution of these mechanisms.

Role of Broca's area in implicit motor skill learning: evidence from continuous theta-burst magnetic stimulation

Complex actions can be regarded as a concatenation of simple motor acts, arranged according to specific rules. Because the caudal part of the Broca's region (left Brodmann's area 44, BA 44) is involved in processing hierarchically organized behaviors, we aimed to test the hypothesis that this area may also play a role in learning structured motor sequences. To address this issue, we investigated the inhibitory effects of a continuous theta-burst TMS (cTBS) applied over left BA 44 in healthy subjects, just before they performed a serial RT task (SRTT). SRTT has been widely used to study motor skill learning and is also of interest because, for complex structured sequences, subjects spontaneously organize them into smaller subsequences, referred to as chunks. As a control, cTBS was applied over the vertex in another group, which underwent the same experiment. Control subjects showed both a general practice learning effect, evidenced by a progressive decrease in RT across blocks and a sequence-specific learning effect, demonstrated by a significant RT increase in a pseudorandom sequence. In contrast, when cTBS was applied over left BA 44, subjects lacked both the general practice and sequence-specific learning effects. However, surprisingly, their chunking pattern was preserved and remained indistinguishable from controls. The present study indicates that left BA 44 plays a role in motor sequence learning, but without being involved in elementary chunking. This dissociation between chunking and sequence learning could be explained if we postulate that left BA 44 intervenes in high hierarchical level processing, possibly to integrate elementary chunks together.

Representing serial action and perception

This article presents a review on the representational base of sequence learning in the serial reaction time task. The first part of the article addresses the major questions and challenges that underlie the debate on implicit and explicit learning. In the second part, the informational content that underlies sequence representations is reviewed. The latter issue has produced a rich and equivocal literature. A taxonomy illustrates that substantial support exists for associations between successive stimulus features, between successive response features, and between successive response-to-stimulus compounds. We suggest that sequence learning is not predetermined with respect to one particular type of information but, rather, develops according to an overall principle of activation contingent on task characteristics. Moreover, substantiating such an integrative approach is proposed by a synthesis with the dual-system model (Keele, Ivry, Mayr, Hazeltine, & Heuer, 2003).

Complex processes from dynamical architectures with time-scale hierarchy

The idea that complex motor, perceptual, and cognitive behaviors are composed of smaller units, which are somehow brought into a meaningful relation, permeates the biological and life sciences. However, no principled framework defining the constituent elementary processes has been developed to this date. Consequently, functional configurations (or architectures) relating elementary processes and external influences are mostly piecemeal formulations suitable to particular instances only. Here, we develop a general dynamical framework for distinct functional architectures characterized by the time-scale separation of their constituents and evaluate their efficiency. Thereto, we build on the (phase) flow of a system, which prescribes the temporal evolution of its state variables. The phase flow topology allows for the unambiguous classification of qualitatively distinct processes, which we consider to represent the functional units or modes within the dynamical architecture. Using the example of a composite movement we illustrate how different architectures can be characterized by their degree of time scale separation between the internal elements of the architecture (i.e. the functional modes) and external interventions. We reveal a tradeoff of the interactions between internal and external influences, which offers a theoretical justification for the efficient composition of complex processes out of non-trivial elementary processes or functional modes.

Learning deferred imitation of long spatial sequences

Sequence learning has been the subject of research in various paradigms but has not been investigated for learning deferred imitation of long spatial sequences. In this task no guiding stimuli support the sequence reproduction and all sequence information has to be recalled from memory. We investigate this kind of imitation learning with a task in which a long sequence of spatial positions has to be reproduced without guiding stimuli, either by manual pointing or by ocular fixations. Sequences consisting of 20 positions were acquired after only 25 training trials. The persistence of learned sequences over several days showed that the sequence was retained in long-term memory. A transfer test revealed that the learned sequence was independent of the effector. A detailed analysis of the error distributions of pointing and ocular fixations was performed to characterize the guiding control signal. The independence of the variable position errors from the movement directions as well as the lack of error propagation between successive targets suggest that the reproduction in this learning task was guided by sequential positions rather than sequential displacements.

Cognitive processing in new and practiced discrete keying sequences

This study addresses the role of cognitive control in the initiation and execution of familiar and unfamiliar movement sequences. To become familiar with two movement sequences participants first practiced two discrete key press sequences by responding to two fixed series of 6-key specific stimuli. In the ensuing test phase they executed these two familiar and also two unfamiliar keying sequences while there was a two-third chance a tone was presented together with one randomly selected key specific stimulus in each sequence. In the counting condition of the test phase participants counted the low pitched (i.e., target) tones. By and large the results support the dual processor model in which the prime role of the cognitive processor shifts from executing to initiating sequences while the gradual development of motor chunks allows a motor processor to execute the sequences. Yet, the results extend this simple model by suggesting that with little practice sequence execution is based also on some non-cognitive (perhaps associative) learning mechanism and, for some participants, on the use of explicit sequence knowledge. Also, after extensive practice the cognitive processor appears to still contribute to slower responses. The occurrence of long interkey intervals was replicated suggesting that fixed 6-key sequences include several motor chunks. Yet, no indication was found that the cognitive processor is responsible for concatenating these chunks.

Neural representations and mechanisms for the performance of simple speech sequences

Speakers plan the phonological content of their utterances before their release as speech motor acts. Using a finite alphabet of learned phonemes and a relatively small number of syllable structures, speakers are able to rapidly plan and produce arbitrary syllable sequences that fall within the rules of their language. The class of computational models of sequence planning and performance termed competitive queuing models have followed K. S. Lashley [The problem of serial order in behavior. In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. 112-136). New York: Wiley, 1951] in assuming that inherently parallel neural representations underlie serial action, and this idea is increasingly supported by experimental evidence. In this article, we developed a neural model that extends the existing DIVA model of speech production in two complementary ways. The new model includes paired structure and content subsystems [cf. MacNeilage, P. F. The frame/content theory of evolution of speech production. Behavioral and Brain Sciences, 21, 499-511, 1998 ] that provide parallel representations of a forthcoming speech plan as well as mechanisms for interfacing these phonological planning representations with learned sensorimotor programs to enable stepping through multisyllabic speech plans. On the basis of previous reports, the model's components are hypothesized to be localized to specific cortical and subcortical structures, including the left inferior frontal sulcus, the medial premotor cortex, the basal ganglia, and the thalamus. The new model, called gradient order DIVA, thus fills a void in current speech research by providing formal mechanistic hypotheses about both phonological and phonetic processes that are grounded by neuroanatomy and physiology. This framework also generates predictions that can be tested in future neuroimaging and clinical case studies.

Diminished motor skill development in elderly: indications for limited motor chunk use

The present study examined whether elderly use motor chunks after practicing discrete keying sequences, just like young adults, or whether they perhaps learn these movement patterns in a different way. To that end, elderly (75-88) and young adults (18-28) practiced as part of the discrete sequence production (DSP) task two fixed series of three and six key presses. The results demonstrate that elderly did improve with practice but this improvement was largely sequence-unspecific. Detailed analyses showed that, in contrast to young adults, most elderly did not use motor chunks, had little explicit sequence knowledge, and remained highly dependent on external stimuli. Still, elderly did show sequence-specific learning with a 6-key sequence that can be explained by an associative learning mechanism.

Rank signals in four areas of macaque frontal cortex during selection of actions and objects in serial order

Neurons in several areas of monkey frontal cortex exhibit ordinal position (rank) selectivity during the performance of serial order tasks. It has been unclear whether rank selectivity or the dependence of rank selectivity on task context varies across the areas of frontal cortex. To resolve this issue, we recorded from neurons in the supplementary motor area (SMA), presupplementary motor area (pre-SMA), supplementary eye field (SEF), and dorsolateral prefrontal cortex (dlPFC) as monkeys performed two oculomotor tasks, one requiring the selection of three actions in sequence and the other requiring the selection of three objects in sequence. We found that neurons representing all ranks were present in all areas. Only to a moderate degree did the prevalence and nature of rank selectivity vary from area to area. The two most prominent inter-area differences involved a lower prevalence of rank selectivity in the dlPFC than in the other areas and a higher proportion of neurons preferring late ranks in the SMA and SEF than in the other areas. Neurons in all four areas are rank generalists in the sense of favoring the same rank in both the serial action task and the serial object task.

How long does it take to describe what one sees? The first step using picture description tasks

The study explored the dependences between quantifiable features of a picture and the time it takes to describe it. Six native English speakers and six bilinguals watched pictures presented on the monitor and described them "as quickly and accurately as possible". The bilingual participants performed the test twice, in English and in their native language. The pictures could contain one to six objects. There were four series of trials that differed in the number of characteristics of the objects the participants were instructed to describe. Reaction time showed a modest, close to linear scaling with the number of objects. Both reaction time and speech time were significantly longer for the bilingual participants performing in English as compared to their performance in the native language and to the English speaking participants. The difference in reaction time did not depend on the number of objects. Speech time showed a close to linear scaling with the number of objects within each of the four series. The linear regression coefficient in this relationship increased linearly with the number of characteristics of the objects across all series. The results are discussed in relation to speed-accuracy trade-off and different strategies of picture description.

Neural correlates of sequence encoding in visuomotor learning

To examine the neural basis of sequence learning, a fundamental but poorly understood human ability, we recorded event-related potentials (ERPs) while subjects viewed and memorized randomly directed sequences of motions for later imitation. Previously, we found that the amplitude of ERPs elicited by successive motion segments decreased as a function of each segment's serial position. This happened when subjects were required to remember the sequence, but not when they were performing a perceptual task. Here, to study the functional significance of this amplitude gradient in sequence learning, we presented each sequence several times in succession and examined changes in ERP amplitude as subjects learned the sequence through repeated observation and imitation. Behaviorally, with each repetition subjects grew more accurate in reproducing what they had seen. At the same time, ERPs grew smaller with each successive presentation, replicating and extending previous demonstrations of repetition suppression. Importantly, a comparison of ERPs to segments occupying different serial positions within a sequence revealed a decreasing amplitude gradient that grew steeper with sequence repetition. This sharpening of the amplitude gradient may reflect an explicit encoding process that relies on a magnitude code for serial order.