Oculomotor evidence of sequence learning on the serial reaction time task

Early Eye Disengagement Is Regulated by Task Complexity and Task Repetition in Visual Tracking Task

Understanding human actions often requires in-depth detection and interpretation of bio-signals. Early eye disengagement from the target (EEDT) represents a significant eye behavior that involves the proactive disengagement of the gazes from the target to gather information on the anticipated pathway, thereby enabling rapid reactions to the environment. It remains unknown how task difficulty and task repetition affect EEDT. We aim to provide direct evidence of how these factors influence EEDT. We developed a visual tracking task in which participants viewed arrow movement videos while their eye movements were tracked. The task complexity was increased by increasing movement steps. Every movement pattern was performed twice to assess the effect of repetition on eye movement. Participants were required to recall the movement patterns for recall accuracy evaluation and complete cognitive load assessment. EEDT was quantified by the fixation duration and frequency within the areas of eye before arrow. When task difficulty increased, we found the recall accuracy score decreased, the cognitive load increased, and EEDT decreased significantly. The EEDT was higher in the second trial, but significance only existed in tasks with lower complexity. EEDT was positively correlated with recall accuracy and negatively correlated with cognitive load. Performing EEDT was reduced by task complexity and increased by task repetition. EEDT may be a promising sensory measure for assessing task performance and cognitive load and can be used for the future development of eye-tracking-based sensors.

Across-task binding: The development of a representation in learning a continuous movement sequence

Across-task binding is defined as the stimulus/response of one task being linked to the response of another task. The purpose of the present experiment was to determine across-task binding in a continuous movement sequence task with an auditory task of high and low pitch tones and the development of a movement sequence representation. According to the two systems theory of sequence learning, we expected that the developed representation in the across-task binding context relies on the multi-dimensional system rather than on the unidimensional system which is restricted to a set of modules where each module processed information along one task/dimension. An inter-manual transfer design was used to disentangle the sequence representations. The mirror transfer test required the same pattern of muscle activation and joint angles (motor coordinates) in the contralateral limb as experienced during the acquisition phase, while in the non-mirror transfer test, the visual-spatial locations (spatial coordinates) of the target waveform were reinstated. The main finding was that consistently combining visual-spatial positions in a sequence and auditory dimensions such as the tone pitch does not rely on a multidimensional system as predicted by the two-systems theory.

Concurrent visual sequence learning

Many researchers in the field of implicit statistical learning agree that there does not exist one general implicit learning mechanism, but rather, that implicit learning takes place in highly specialized encapsulated modules. However, the exact representational content of these modules is still under debate. While there is ample evidence for a distinction between modalities (e.g., visual, auditory perception), the representational content of the modules might even be distinguished by features within the same modalities (e.g., location, color, and shape within the visual modality). In implicit sequence learning, there is evidence for the latter hypothesis, as a stimulus-color sequence can be learned concurrently with a stimulus-location sequence. Our aim was to test whether this also holds true for non-spatial features within the visual modality. This has been shown in artificial grammar learning, but not yet in implicit sequence learning. Hence, in Experiment 1, we replicated an artificial grammar learning experiment of Conway and Christiansen (2006) in which participants were supposed to learn color and shape grammars concurrently. In Experiment 2, we investigated concurrent learning of sequences with an implicit sequence learning paradigm: the serial reaction time task. Here, we found evidence for concurrent learning of two sequences, a color and shape sequence. Overall, the findings converge to the assumption that implicit learning might be based on features.

Neural basis of implicit motor sequence learning: Modulation of cortical power

Implicit sequence learning describes the acquisition of serially ordered movements and sequentially structured cognitive information, that occurs without awareness. Theta, alpha and beta cortical oscillations are present during implicit motor sequence learning, but their role in this process is unclear. The current study addressed this gap in the literature. A total of 50 healthy adults aged between 19 and 37 years participated in the study. Implicit motor sequence learning was examined using the Serial Reaction Time task where participants unknowingly repeat a sequence of finger movements in response to a visual stimulus. Sequence learning was examined by comparing reaction times and oscillatory power between sequence trials and a set of control trials comprising random stimulus presentations. Electroencephalography was recorded as participants completed the task. Analyses of the behavioral data revealed participants learnt the sequence. Analyses of oscillatory activity, using permutation testing, revealed sequence learning was associated with a decrease in theta band (4-7 Hz) power recorded over frontal and central electrode sites. Sequence learning effects were not observed in the alpha (7-12 Hz) or beta bands (12-20 Hz). Even though alpha and beta power modulations have long been associated with executing a motor response, it seems theta power is a correlate of sequence learning in the manual domain. Theta power modulations on the serial reaction time task may reflect disengagement of attentional resources, either promoting or occurring as a consequence of implicit motor sequence learning.

Time course of learning sequence representations in action imagery practice

Action imagery practice (AIP) is effective to improve motor performance in a variety of tasks, though it is often less effective than action execution practice (AEP). In sequence learning, AIP and AEP result in the acquisition of effector-independent representations. However, it is unresolved whether effector-dependent representations can be acquired in AIP. In the present study, we investigated the acquisition of effector-independent representations and effector-dependent representations in AEP and AIP in an implicit sequence learning task (a visual serial-reaction-time task, involving a twelve-element sequence). Participants performed six sessions, each starting with tests. A practice sequence, a mirror sequence, and a different sequence were tested with the practice and transfer hand. In the first four sessions, after the tests, two groups performed either AIP (N = 50) or AEP (N = 54). Improvement in the different sequence indicated sequence-unspecific learning in both AEP and AIP. Importantly, reaction times of the practice hand became shorter in the practice sequence than in the other sequences, indicating implicit sequence learning in both, AEP and AIP. This effect was stronger in the practice hand than in the transfer hand, indicating effector-dependent sequence representations in both AEP and AIP. However, effector-dependent sequence representations were stronger in AEP than in AIP. No significant differences between groups were observed in the transfer hand, although effector-independent sequence representations were observed in AEP only. In conclusion, AIP promotes not only sequence-unspecific stimulus-response coupling and anticipations of the subsequent stimuli, but also anticipations of the subsequent responses.

Effects of visuospatial implicit sequence learning on visual stimulus processing: Evidence from event-related potentials and neural synchrony

A previous study reported that reaction times (RTs) and the amplitude of the P1 component of event-related potentials (ERPs) elicited by visual stimuli decreased during visuospatial implicit sequence learning in the serial reaction time task, suggesting that sequence learning reduces attentional demands on visual stimulus processing. In the present study, to evaluate the replicability of the previous finding and to obtain a better understanding of how visual stimulus processing is affected by visuospatial implicit sequence learning, we measured ERPs and neural synchrony from 44 participants during a modified serial reaction time task which controlled for a possible confounding factor in the previous study (i.e., arousal). The results indicated that RTs and neural synchrony of the lower frequency band (22-34 Hz) decreased for a learned sequence, whereas no significant effects on the amplitudes of P1, N1, and P3 components of ERPs were observed. These results suggest that attentional demands on visual stimulus processing can be reduced by visuospatial implicit sequence learning, as suggested by the previous study, but stimulus-locked ERPs may not be sensitive enough to reflect such learning effects.

Measuring statistical learning by eye-tracking

Statistical learning—the skill to pick up probability-based regularities of the environment—plays a crucial role in adapting to the environment and learning perceptual, motor, and language skills in healthy and clinical populations. Here, we developed a new method to measure statistical learning without any manual responses. We used the Alternating Serial Reaction Time (ASRT) task, adapted to eye-tracker, which, besides measuring reaction times (RTs), enabled us to track learning-dependent anticipatory eye movements. We found robust, interference-resistant learning on RT; moreover, learning-dependent anticipatory eye movements were even more sensitive measures of statistical learning on this task. Our method provides a way to apply the widely used ASRT task to operationalize statistical learning in clinical populations where the use of manual tasks is hindered, such as in Parkinson’s disease. Furthermore, it also enables future basic research to use a more sensitive version of this task to measure predictive processing.

Inhibition of Eye Movements Disrupts Spatial Sequence Learning

Implicit sequence learning is an integral part of human experience, yet the nature of the mechanisms underlying this type of learning remains a matter of debate. In the current study, we provide a test for two accounts of implicit sequence learning, that is, one that highlights sequence learning in the absence of any motor responses (with suppressed eye movements) and one that highlights the relative contribution of the motor processes (i.e., eye movements) to learning. To adjudicate between these accounts and determine whether a motor response is a requisite process in sequence learning, we used anticipation measures to compare performance on the standard oculomotor serial reaction time (SRT) task and on a version of the SRT task where the eye movements were restricted during the learning phase. our results demonstrated an increased proportion of correct anticipations in the standard SRT task compared to the restricted-movement task.

Occipital sleep spindles predict sequence learning in a visuo-motor task

STUDY OBJECTIVES

The brain appears to use internal models to successfully interact with its environment via active predictions of future events. Both internal models and the predictions derived from them are based on previous experience. However, it remains unclear how previously encoded information is maintained to support this function, especially in the visual domain. In the present study, we hypothesized that sleep consolidates newly encoded spatio-temporal regularities to improve predictions afterwards.

METHODS

We tested this hypothesis using a novel sequence-learning paradigm that aimed to dissociate perceptual from motor learning. We recorded behavioral performance and high-density electroencephalography (EEG) in male human participants during initial training and during testing two days later, following an experimental night of sleep (n = 16, including high-density EEG recordings) or wakefulness (n = 17).

RESULTS

Our results show sleep-dependent behavioral improvements correlated with sleep-spindle activity specifically over occipital cortices. Moreover, event-related potential (ERP) responses indicate a shift of attention away from predictable to unpredictable sequences after sleep, consistent with enhanced automaticity in the processing of predictable sequences.

CONCLUSIONS

These findings suggest a sleep-dependent improvement in the prediction of visual sequences, likely related to visual cortex reactivation during sleep spindles. Considering that controls in our experiments did not fully exclude oculomotor contributions, future studies will need to address the extent to which these effects depend on purely perceptual versus oculomotor sequence learning.

Order, please! Explicit sequence learning in hybrid search in younger and older age

Sequence learning effects in simple perceptual and motor tasks are largely unaffected by normal aging. However, less is known about sequence learning in more complex cognitive tasks that involve attention and memory processes and how this changes with age. In this study, we examined whether incidental and intentional sequence learning would facilitate hybrid visual and memory search in younger and older adults. Observers performed a hybrid search task, in which they memorized four or 16 target objects and searched for any of those target objects in displays with four or 16 objects. The memorized targets appeared either in a repeating sequential order or in random order. In the first experiment, observers were not told about the sequence before the experiment. Only a subset of younger adults and none of the older adults incidentally learned the sequence. The "learners" acquired explicit knowledge about the sequence and searched faster in the sequence compared to random condition. In the second experiment, observers were told about the sequence before the search task. Both younger and older adults searched faster in sequence blocks than random blocks. Older adults, however, showed this sequence-learning effect only in blocks with smaller target sets. Our findings indicate that explicit sequence knowledge can facilitate hybrid search, as it allows observers to predict the next target and restrict their visual and memory search. In older age, the sequence-learning effect is constrained by load, presumably due to age-related decline in executive functions.

Implicit manual and oculomotor sequence learning in developmental language disorder

Procedural memory functioning in developmental language disorder (DLD) has largely been investigated by examining implicit sequence learning by the manual motor system. This study examined whether poor sequence learning in DLD is present in the oculomotor domain. Twenty children with DLD and 20 age-matched typically developing (TD) children were presented with a serial reaction time (SRT) task. On the task, a visual stimulus repeatedly appears in different positions on a computer display which prompts a manual response. The children were unaware that on the first three blocks and final block of trials, the visual stimulus followed a sequence. On the fourth block, the stimulus appeared in random positions. Manual reaction times (RT) and saccadic amplitudes were recorded, which assessed sequence learning in the manual and oculomotor domains, respectively. Manual RT were sensitive to sequence learning for the TD group, but not the DLD group. For the TD group, manual RT increased when the random block was presented. This was not the case for the DLD group. In the oculomotor domain, sequence learning was present in both groups. Specifically, sequence learning was found to modulate saccadic amplitudes resulting in both DLD and TD children being able to anticipate the location of the visual stimulus. Overall, the study indicates that not all aspects of the procedural memory system are equally impaired in DLD.

Day versus night consolidation of implicit sequence learning using manual and oculomotor activation versions of the serial reaction time task: reaction time and anticipation measures

This study presents two experiments that explored consolidation of implicit sequence learning based on two dependent variables-reaction time (RT) and correct anticipations to clarify the role of sleep, and whether the manual component is necessary for consolidation processes. Experiment 1 (n = 37) explored the performance of adults using an ocular variant of the serial reaction time task (O-SRT) with manual activation (MA), and Experiment 2 (n = 37) used the ocular activation (OA) version of the task. Each experiment consisted of a Day and a Night group that performed two sessions of the O-SRT with an intervening 12-h offline period (morning/evening in Day group, evening/following morning in Night group). Night offline had an advantage only when manual response was required and when correct anticipations (i.e., accuracy) but not RT (i.e., speed) were measured. We associated this finding with the dual-learning processes required in the MA O-SRT that led to increased sequence specific learning overnight. When using the OA O-SRT, both groups demonstrated similar rates after offline in RT and correct anticipations. We interpreted this finding to reflect stabilization, which confirmed our hypothesis. As expected, all the groups demonstrated reduced performance when another sequence was introduced, thus reflecting sequence-specific learning. This study used a powerful procedure that allows measurement of implicit sequence learning in several ways: by evaluating two different measures (RT, correct anticipations) and by isolating different aspects of the task (i.e., with/without the manual learning component, more/less general skill learning), which are known to affect learning and consolidation.

Oculomotor anticipation reveals a multitude of learning processes underlying the serial reaction time task

Sequence learning is the cognitive faculty enabling everyday skill acquisition. In the lab, it is typically measured in speed of response to sequential stimuli, whereby faster responses are taken to indicate improved anticipation. However, response speed is an indirect measure of anticipation, that can provide only limited information on underlying processes. As a result, little is known about what is learned during sequence learning, and how that unfolds over time. In this work, eye movements that occurred before targets appeared on screen in an ocular serial reaction time (O-SRT) task provided an online indication of where participants anticipated upcoming targets. When analyzed in the context of the stimuli preceding them, oculomotor anticipations revealed several simultaneous learning processes. These processes influenced each other, as learning the task grammar facilitated acquisition of the target sequence. However, they were dissociable, as the grammar was similarly learned whether a repeating sequence inhabited the task or not. Individual differences were found in how the different learning processes progressed, allowing for similar performance to be produced for different latent reasons. This study provides new insights into the processes subserving sequence learning, and a new method for high-resolution study of it.

Implicit sequence learning in individuals with Parkinson's disease: The added value of using an ocular version of the serial reaction time (O-SRT) task

INTRODUCTION

Though the majority of studies reported impaired sequence learning in individuals with Parkinson's disease (PD) tested with the Serial Reaction Time (SRT) task, findings are inconclusive. To elucidate this point, we used an eye tracker in an ocular SRT task version (O-SRT) that in addition to RT, enables extraction of two measures reflecting different cognitive processes, namely, Correct Anticipation (CA) and number of Stucks.

METHODS

Individuals with PD (n = 29) and matched controls (n = 31) were tested with the O-SRT task, consisting of a repeated sequence of six blocks, then a block with an interference sequence followed by an original sequence block.

RESULTS

Unlike controls, patients with PD did not improve in CA rate across learning trials, did not show an increase in RT when presented with the interference sequence, and showed a significantly higher rate of Stucks.

CONCLUSIONS

Low CA rate and high Stucks rate emerge as the cardinal deficits leading to impaired sequence learning following PD. These are viewed as reflecting difficulty in exploration for an efficient learning strategy. This study highlights the advantage in using the O-SRT task, which enables the generation of several informative measures of learning, allowing better characterization of the PD effect on sequence learning.

The impact of implicit and explicit suggestions that 'there is nothing to learn' on implicit sequence learning

We can sometimes efficiently pick up statistical regularities in our environment in the absence of clear intentions or awareness, a process typically referred to as implicit sequence learning. In the current study, we tried to address the question whether suggesting participants that there is nothing to learn can impact this form of learning. If a priori predictions or intentions to learn are important in guiding implicit learning, we reasoned that suggesting participants that there is nothing to learn in a given context should hamper implicit learning. We introduced participants to random contexts that indicated that there was nothing to learn, either implicitly (i.e., by presenting blocks of random trials in "Experiment 1"), or explicitly (i.e., by explicitly instructing them in "Experiment 2"). Next, in a subsequent learning phase, participants performed an implicit sequence learning task. We found that these implicit or explicit suggestions that 'there was nothing to learn' did not influence the emergence of implicit knowledge in the subsequent learning phase. Although these findings seem consistent with simple associative or Hebbian learning accounts of implicit sequence learning (i.e., not steered by predictions), we discuss potential limitations that should inform future studies on the role of a priori predictions in implicit learning.

How sequence learning unfolds: Insights from anticipatory eye movements

The acquisition of sequential knowledge is pivotal in forming skilled behavior. Despite extensive research of sequence learning, much remains unknown regarding what knowledge participants learn in such studies, and how that knowledge takes form over time. By tracking eye-movements made before stimuli appear on screen during a serial reaction time (SRT) task, we devised a method for assessing learning at the individual participant level in an item-based resolution. Our method enables uncovering what participants actually learn about the sequence presented to them, and when. Results demonstrate that learning is more heterogeneous than previously thought, driven by learning both of chunks and of statistics embedded in the sequence. Also, learning develops rapidly, but in a fragmented and non-sequential manner, eventually encompassing only a subset of available regularities. The tools developed in this work may aid in further dissociating processes and mechanisms underlying sequence learning and its impairments, in normal and in clinical populations.

Sequential modulation of across-task congruency in the serial reaction time task

While detrimental effects of dual-tasking on the acquisition and usage of sequence knowledge in the Serial Reaction Time Task (SRTT) have been attributed to the integration of regularly and randomly sequenced events, direct evidence for across-task integration has been scarce. In two experiments, we paired two spatial tasks: A visual-manual SRTT (serial reaction time task) of length 4 and a two-choice task with random order of stimuli. We reasoned that across-task integration might result in SRTT- and two-choice task events being stored and retrieved together. Therefore, RT might be influenced by current congruency as well as by whether it is repeated or altered from the congruency level encountered for this SRTT event in the last loop of the sequence. We observed such a modulation in two experiments, suggesting that across-task integration indeed takes place and that the detrimental effect of dual-tasking on sequence learning is, at least in parts, based on across-task integration.

Working memory affects anticipatory behavior during implicit pattern learning

We investigated the relation between implicit sequence learning and individual differences in working memory (WM) capacity. Participants performed an oculomotor version of the serial reaction time (SRT) task and three computerized WM tasks. Implicit learning was measured using anticipation measures only, as they represent strong indicators of learning. Our results demonstrate that anticipatory behavior in the SRT task changes as a function of WM capacity, such that it increases with decreased WM capacity. On the other hand, WM capacity did not affect the overall number of correct anticipations in the task. In addition, we report a positive relation between WM capacity and the number of consecutive correct anticipations (or chunks), and a negative relation between WM capacity and the overall number of errors, indicating different learning strategies during implicit sequence learning. The results of the current study are theoretically important, because they demonstrate that individual differences in WM capacity could account for differences in learning processes, and ultimately change individuals' anticipatory behavior, even when learning is implicit, without intention and awareness.

Sequence Knowledge on When and What Supports Dual-Tasking

The constraints in overlapping response selection have been established in dual-tasking studies with random sequence of stimuli and responses as well as random stimulus onset asynchrony (SOA). While this approach makes it possible to control for advance activation of upcoming stimuli or responses, it leaves open whether such preparatory processing can indeed influence dual-task performance. We investigated whether and how the sequence of stimuli and responses and the sequence of SOAs can be learned and used under dual-tasking. In each trial, participants (N = 28 in Experiment 1 and N = 30 in Experiment 2) were first presented with a random two-choice task followed by a four-choice Serial Reaction Time Task (SRTT), presented in a sequence of length four (position sequence). The SOA (timing) sequence also had length four. In test phases, one or both of the sequences were randomized. Results showed that both position and timing sequences were learned and supported dual-task performance, suggesting that predictive processing with respect to timing and identity of stimuli and responses can help to circumvent the response selection bottleneck constraints. Furthermore, in contrast to previous work on acquisition of interval sequences in single tasking, we found that the sequence of what (i.e. stimulus) and the sequence of when (i.e. interval between two tasks) contributed independently to performance.

Translating visual information into action predictions: Statistical learning in action and nonaction contexts

Humans are sensitive to the statistical regularities in action sequences carried out by others. In the present eyetracking study, we investigated whether this sensitivity can support the prediction of upcoming actions when observing unfamiliar action sequences. In two between-subjects conditions, we examined whether observers would be more sensitive to statistical regularities in sequences performed by a human agent versus self-propelled 'ghost' events. Secondly, we investigated whether regularities are learned better when they are associated with contingent effects. Both implicit and explicit measures of learning were compared between agent and ghost conditions. Implicit learning was measured via predictive eye movements to upcoming actions or events, and explicit learning was measured via both uninstructed reproduction of the action sequences and verbal reports of the regularities. The findings revealed that participants, regardless of condition, readily learned the regularities and made correct predictive eye movements to upcoming events during online observation. However, different patterns of explicit-learning outcomes emerged following observation: Participants were most likely to re-create the sequence regularities and to verbally report them when they had observed an actor create a contingent effect. These results suggest that the shift from implicit predictions to explicit knowledge of what has been learned is facilitated when observers perceive another agent's actions and when these actions cause effects. These findings are discussed with respect to the potential role of the motor system in modulating how statistical regularities are learned and used to modify behavior.

Differential effects of cue-based and sequence knowledge-based predictability on multitasking performance

Everyday multitasking often is characterized by predictable sequences. While such sequential regularities are present in setups using the Serial Reaction Time Task (SRTT), many laboratory studies on dual-tasking performance use random sequences of stimuli in either of the two tasks. In the current study, following single-task training on the SRTT, participants completed trials where they were confronted with an additional visual-manual task with either a random (Experiment 1) or a partially predictable (Experiment 2) stimulus sequence. In the SRTT, we cued participants with respect to which of the four stimulus options were yet to occur (before a new round with all four options would start). We randomly mixed a sequence to be practiced with random sequences of the same length and with the same constraint. Thus, we were able to vary predictability of upcoming stimuli (from chance to 100%) as well as sequence knowledge (practiced vs. random sequence) in order to assess how cueing and sequence knowledge, as two potential bases of prediction, would affect performance in single- and dual-tasking. Results suggest that both cueing and sequence knowledge-based prediction can lead to shorter RTs in dual-tasking. In previous studies, the disruption of sequence learning by adding a task with a random stimulus sequence has been linked to the effects of automatic prediction between events in the two tasks. In line with these studies, dual-task performance did not impede usage of sequence knowledge when a task with a predictable (rather than random) sequence of stimuli was added to the SRTT.

Visuospatial sequence learning on the serial reaction time task modulates the P1 event-related potential

This study examined whether the P1, N1, and P3 ERP components would be sensitive to sequence learning effects on the serial reaction time task. On this task, participants implicitly learn a visuospatial sequence. Participants in this study were 35 healthy adults. Reaction time (RT) data revealed that, at the group level, participants learned the sequence. Specifically, RT became faster following repeated exposure to the visuospatial sequence and then slowed down in a control condition. Analyses of ERP data revealed no evidence for sequence learning effects for the N1 or P3 component. However, sequence learning effects were observed for the P1 component. Mean P1 amplitude mirrored the RT data. The analyses showed that P1 amplitude significantly decreased as participants were exposed to the sequence but then significantly increased in the control condition. This suggests that visuospatial sequence learning can modulate visual attention levels. Specifically, it seems that, as sequence knowledge is acquired, fewer demands are placed on visual attention resources.

The "Motor" in Implicit Motor Sequence Learning: A Foot-stepping Serial Reaction Time Task

This protocol describes a modified serial reaction time (SRT) task used to study implicit motor sequence learning. Unlike the classic SRT task that involves finger-pressing movements while sitting, the modified SRT task requires participants to step with both feet while maintaining a standing posture. This stepping task necessitates whole body actions that impose postural challenges. The foot-stepping task complements the classic SRT task in several ways. The foot-stepping SRT task is a better proxy for the daily activities that require ongoing postural control, and thus may help us better understand sequence learning in real-life situations. In addition, response time serves as an indicator of sequence learning in the classic SRT task, but it is unclear whether response time, reaction time (RT) representing mental process, or movement time (MT) reflecting the movement itself, is a key player in motor sequence learning. The foot-stepping SRT task allows researchers to disentangle response time into RT and MT, which may clarify how motor planning and movement execution are involved in sequence learning. Lastly, postural control and cognition are interactively related, but little is known about how postural control interacts with learning motor sequences. With a motion capture system, the movement of the whole body (e.g., the center of mass (COM)) can be recorded. Such measures allow us to reveal the dynamic processes underlying discrete responses measured by RT and MT, and may aid in elucidating the relationship between postural control and the explicit and implicit processes involved in sequence learning. Details of the experimental set-up, procedure, and data processing are described. The representative data are adopted from one of our previous studies. Results are related to response time, RT, and MT, as well as the relationship between the anticipatory postural response and the explicit processes involved in implicit motor sequence learning.

Predictive Movements and Human Reinforcement Learning of Sequential Action

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time‐course of motion, including predictive movements. This paper describes a mouse movement trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement—before the next cue appears—increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10‐item sequence within 10 repetitions. Comparing the high‐ and low‐performers’ detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model‐free reinforcement learning models fit the high‐performing participants, while the four low‐performing participants provide better fit with a simple negative recency bias model.

The influence of eye-movements on the development of a movement sequence representation during observational and physical practice

An experiment was conducted to examine the development of a movement sequence representation and the role of eye-movements during observational and physical practice. The task was to reproduce a 1300ms spatial-temporal pattern of a sequence of elbow flexions and extensions. An inter-manual transfer design with a retention and two effector transfer tests (contralateral limb) was used. The mirror transfer test required the same pattern of homologous muscle activation and a sequence of joint angles as experienced during the acquisition phase, and the non-mirror transfer test required the same visual-spatial pattern as performed or observed during acquisition. Participants were randomly assigned to one of four groups differing in eye-movements (free to use their eyes vs. instruction to fixate) and the practice type (observational practice vs. physical practice). The results indicated that permitting to use eye-movements facilitates sequence learning. This advantage was found on both practice types. The results of the transfer tests indicated that participants of the physical practice group who were permitted to use their eyes demonstrated superior transfer performance in the mirror transfer test, while participants in the observational practice group demonstrated better performance on the non-mirror transfer test. These findings indicated that eye-movements enhanced the development of a visual-spatial representation during observational practice as well as a motor representation during physical practice.

Effects of model types in observational learning on implicit sequential learning

This study investigated whether implicit learning of sequence by observation occurred in a serial reaction time task and whether the learning effects were modulated by model behavioral type. In Experiment 1, we let 20 participants perform a sequence for 12 blocks and chose the best and worst performance models based on reaction time and errors. In Experiment 2, new observers viewed a movie clip chosen from the following three: the best model performing the sequential task in the first (the first six blocks) or second session (the last six blocks), or the worst model performing the task in the first session. Then, the observers performed the observed sequence, a test sequence and awareness test. We found that (1) implicit sequential learning occurred by observation regardless of model behavior type, (2) the learning effects were not susceptible to model behavior type and (3) speed index reflecting reaction time became larger even in the test session when the observers viewed the best model performing the second session. Overall, observers developed general motor representations through action-observation. In addition, their responses were also contagious; if the model performed the sequence faster, the observer might be able to perform the sequence faster.

Anticipation Measures of Sequence Learning: Manual versus Oculomotor Versions of the Serial Reaction Time Task

The serial reaction time (SRT) task has generated a very large amount of research. Nevertheless the debate continues as to the exact cognitive processes underlying implicit sequence learning. Thus, the first goal of this study is to elucidate the underlying cognitive processes enabling sequence acquisition. We therefore compared reaction time (RT) in sequence learning in a standard manual activated (MA) to that in an ocular activated (OA) version of the task, within a single experimental setting. The second goal is to use eye movement measures to compare anticipation, as an additional indication of sequence learning, between the two versions of the SRT. Performance of the group given the MA version of the task ( n = 29) was compared with that of the group given the OA version ( n = 30). The results showed that although overall, RT was faster for the OA group, the rate of sequence learning was similar to that of the MA group performing the standard version of the SRT. Because the stimulus-response association is automatic and exists prior to training in the OA task, the decreased reaction time in this version of the task reflects a purer measure of the sequence learning that occurs in the SRT task. The results of this study show that eye tracking anticipation can be measured directly and can serve as a direct measure of sequence learning. Finally, using the OA version of the SRT to study sequence learning presents a significant methodological contribution by making sequence learning studies possible among populations that struggle to perform manual responses.

Tied to expectations: Predicting features speeds processing even under adverse circumstances

When participants predict the upcoming stimulus in a randomized choice reaction task, a match between prediction and stimulus increases processing speed at a level similar to that observed in cueing studies with highly valid cues. This might be taken to suggest that people cannot help but fully use their self-generated, verbalized predictions for preparing task processing. Thus, we tested how flexibly participants can control formation and implementation of predictions. In Experiment 1, we varied validity and response-relevance of predictions. We observed that prediction effects on RT can be boosted by increasing validity, but prevail under adverse circumstances. This was not the case in a control group who read rather than predicted the feature words, suggesting that the effect was specific to predictions as such. Experiment 2 provided further evidence for limited control of participants over implementing and forming predictions. Participants were provided with practice on stimuli occurring with varying frequency, but neither learned to strategically choose predictions to maximize the number of match trials, nor did they reduce the amount of prediction-based preparation when predicting an infrequent stimulus. As sequential aftereffects of prediction match did not vary with validity, they were identified as an independent effect of verbalizing a response-relevant stimulus feature. The results are consistent with the view that the predicted stimulus feature is represented in the focus of attention in working memory and that the amount of implementation can be subject to weighting.

The Things You Do: Internal Models of Others' Expected Behaviour Guide Action Observation

Predictions allow humans to manage uncertainties within social interactions. Here, we investigate how explicit and implicit person models-how different people behave in different situations-shape these predictions. In a novel action identification task, participants judged whether actors interacted with or withdrew from objects. In two experiments, we manipulated, unbeknownst to participants, the two actors action likelihoods across situations, such that one actor typically interacted with one object and withdrew from the other, while the other actor showed the opposite behaviour. In Experiment 2, participants additionally received explicit information about the two individuals that either matched or mismatched their actual behaviours. The data revealed direct but dissociable effects of both kinds of person information on action identification. Implicit action likelihoods affected response times, speeding up the identification of typical relative to atypical actions, irrespective of the explicit knowledge about the individual's behaviour. Explicit person knowledge, in contrast, affected error rates, causing participants to respond according to expectations instead of observed behaviour, even when they were aware that the explicit information might not be valid. Together, the data show that internal models of others' behaviour are routinely re-activated during action observation. They provide first evidence of a person-specific social anticipation system, which predicts forthcoming actions from both explicit information and an individuals' prior behaviour in a situation. These data link action observation to recent models of predictive coding in the non-social domain where similar dissociations between implicit effects on stimulus identification and explicit behavioural wagers have been reported.

Magnifying visual target information and the role of eye movements in motor sequence learning

An experiment investigated the influence of eye movements on learning a simple motor sequence task when the visual display was magnified. The task was to reproduce a 1300 ms spatial-temporal pattern of elbow flexions and extensions. The spatial-temporal pattern was displayed in front of the participants. Participants were randomly assigned to four groups differing on eye movements (free to use their eyes/instructed to fixate) and the visual display (small/magnified). All participants had to perform a pre-test, an acquisition phase, a delayed retention test, and a transfer test. The results indicated that participants in each practice condition increased their performance during acquisition. The participants who were permitted to use their eyes in the magnified visual display outperformed those who were instructed to fixate on the magnified visual display. When a small visual display was used, the instruction to fixate induced no performance decrements compared to participants who were permitted to use their eyes during acquisition. The findings demonstrated that a spatial-temporal pattern can be learned without eye movements, but being permitting to use eye movements facilitates the response production when the visual angle is increased.

Expectation mismatch: differences between self-generated and cue-induced expectations

Expectation of upcoming stimuli and tasks can lead to improved performance, if the anticipated situation occurs, while expectation mismatch can lead to less efficient processing. Researchers have used methodological approaches that rely on either self-generated expectations (predictions) or cue-induced expectations to investigate expectation mismatch effects. Differentiating these two types of expectations for different contents of expectation such as stimuli, responses, task sets and conflict level, we review evidence suggesting that self-generated expectations lead to larger facilitating effects and conflict effects on the behavioral and neural level - as compared to cue-based expectations. On a methodological level, we suggest that self-generated as compared to cue-induced expectations allow for a higher amount of experimental control in many experimental designs on expectation effects. On a theoretical level, we argue for qualitative differences in how cues vs. self-generated expectations influence performance. While self-generated expectations might generally involve representing the expected event in the focus of attention in working memory, cues might only lead to such representations under supportive circumstances (i.e., cue of high validity and attended).

Right fronto-parietal dysfunction underlying spatial attention in bipolar disorder

Although the neural underpinning of bipolar disorder (BD) is still unknown, recent research suggests that the right fronto-parietal cortex is particularly affected in BD patients. If this were true, we would expect atypical functional cerebral asymmetries in allocation of visuospatial attention. To test this hypothesis, euthymic BD patients and age- and gender-matched healthy controls were compared on the visual line-bisection task, a reliable measure of visuospatial attention, associated with right parietal function. Line bisection performance (i.e. absolute and directional bias) was compared between groups as a function of response hand and line position. The results showed a typical hand-use effect in healthy controls involving a larger leftward bias (i.e. pseudoneglect) with the left hand than with the right hand. Although euthymic BD patients did not differ from healthy controls in the overall accuracy (i.e. absolute bias), they differed significantly in the directional line bisection bias. In contrast to healthy controls, BD patients did not significantly deviate from the veridical center, regardless of which hand was used to bisect horizontal lines. This finding indicates an atypical functional cerebral asymmetry in visuospatial attention in euthymic BD patients, supporting the idea of a dysfunction especially in the right fronto-parietal cortex.

The study of sequence learning in individuals with schizophrenia: a critical review of the literature

The serial reaction time task (SRTT) has been used extensively to study implicit sequence learning. A number of studies have used the SRTT to examine sequence learning in schizophrenia patients. Despite these studies, it remains unclear whether sequence learning is impaired in patients, whether antipsychotic medications affect sequence learning, and what types of sequential information patients might have difficulty learning. Methodological limitations have made it difficult to obtain good answers to these questions. Methodological innovations from the general SRTT literature that have not yet been adopted in the schizophrenia literature could provide better answers.

Generalisation of new sequence knowledge depends on response modality

New visuomotor skills can guide behaviour in novel situations. Prior studies indicate that learning a visuospatial sequence via responses based on manual key presses leads to effector- and response-independent knowledge. Little is known, however, about the extent to which new sequence knowledge can generalise, and, thereby guide behaviour, outside of the manual response modality. Here, we examined whether learning a visuospatial sequence either via manual (key presses, without eye movements), oculomotor (obligatory eye movements), or perceptual (covert reorienting of visuospatial attention) responses supported generalisation to direct and indirect tests administered either in the same (baseline conditions) or a novel response modality (transfer conditions) with respect to initial study. Direct tests measured the use of conscious knowledge about the studied sequence, whereas the indirect tests did not ostensibly draw on the study phase and measured response priming. Oculomotor learning supported the use of conscious knowledge on the manual direct tests, whereas manual learning supported generalisation to the oculomotor direct tests but did not support the conscious use of knowledge. Sequence knowledge acquired via perceptual responses did not generalise onto any of the manual tests. Manual, oculomotor, and perceptual sequence learning all supported generalisation in the baseline conditions. Notably, the manual baseline condition and the manual to oculomotor transfer condition differed in the magnitude of general skill acquired during the study phase; however, general skill did not predict performance on the post-study tests. The results demonstrated that generalisation was only affected by the responses used to initially code the visuospatial sequence when new knowledge was applied to a novel response modality. We interpret these results in terms of response-effect distinctiveness, the availability of integrated effector- and motor-plan based information, and discuss their implications for neurocognitive accounts of sequence learning.

Prediction during statistical learning, and implications for the implicit/explicit divide

Accounts of statistical learning, both implicit and explicit, often invoke predictive processes as central to learning, yet practically all experiments employ non-predictive measures during training. We argue that the common theoretical assumption of anticipation and prediction needs clearer, more direct evidence for it during learning. We offer a novel experimental context to explore prediction, and report results from a simple sequential learning task designed to promote predictive behaviors in participants as they responded to a short sequence of simple stimulus events. Predictive tendencies in participants were measured using their computer mouse, the trajectories of which served as a means of tapping into predictive behavior while participants were exposed to very short and simple sequences of events. A total of 143 participants were randomly assigned to stimulus sequences along a continuum of regularity. Analysis of computer-mouse trajectories revealed that (a) participants almost always anticipate events in some manner, (b) participants exhibit two stable patterns of behavior, either reacting to vs. predicting future events, (c) the extent to which participants predict relates to performance on a recall test, and (d) explicit reports of perceiving patterns in the brief sequence correlates with extent of prediction. We end with a discussion of implicit and explicit statistical learning and of the role prediction may play in both kinds of learning.

Visuospatial Perceptual Sequence Learning and Eye Movements

We examined perceptual sequence learning by means of an adapted serial reaction time task in which eye movements were unnecessary for performing the sequence learning task. Participants had to respond to the identity of a target letter pair (“OX” or “XO”) appearing in one of four locations. On the other locations, similar distractor letter pairs (“QY” or “YQ”) were shown. While target identity changed randomly, target location was structured according to a deterministic sequence. To render eye movements superfluous, (1) stimulus letter pairs appeared around a fixation cross with a visual angle of 0.63°, which means that they appeared within the foveal visual area and (2) the letter pairs were presented for only 100 ms, a period too short to allow proper eye movements. Reliable sequence knowledge was acquired under these conditions, as responses were both slower and less accurate when the trained sequence was replaced by an untrained sequence. These results support the notion that perceptual sequence learning can be based on shifts of attention without overt oculomotor movements.

Importance of the temporal structure of movement sequences on the ability of monkeys to use serial order information

The capacity to acquire motor skills through repeated practice of a sequence of movements underlies many everyday activities. Extensive research in humans has dealt with the importance of spatial and temporal factors on motor sequence learning, standing in contrast to the few studies available in animals, particularly in nonhuman primates. In the present experiments, we studied the effect of the serial order of stimuli and associated movements in macaque monkeys overtrained to make arm-reaching movements in response to spatially distinct visual targets. Under different conditions, the temporal structure of the motor sequence was varied by changing the duration of the interval between successive target stimuli or by adding a cue that reliably signaled the onset time of the forthcoming target stimulus. In each condition, the extent to which the monkeys are sensitive to the spatial regularities was assessed by comparing performance when stimulus locations follow a repeating sequence, as opposed to a random sequence. We observed no improvement in task performance on repeated sequence blocks, compared to random sequence blocks, when target stimuli are relatively distant from each other in time. On the other hand, the shortening of the time interval between successive target stimuli or, more efficiently, the addition of a temporal cue before the target stimulus yielded a performance advantage under repeated sequence, reflected in a decrease in the latency of arm and saccadic eye movements accompanied by an increased tendency for eye movements to occur in an anticipatory manner. Contrary to the effects on movement initiation, the serial order of stimuli and movements did not markedly affect the execution of movement. Moreover, the location of a given target in the random sequence influenced task performance based on the location of the preceding target, monkeys being faster in responding as a result of familiarity caused by extensive practice with some target transitions also used in the repeated sequence. This performance advantage was most prominently detectable when temporal prediction of forthcoming target stimuli was optimized. Taken together, the present findings demonstrate that the monkey's capacity to make use of serial order information to speed task performance was dependent on the temporal structure of the motor sequence.

Eye movements and imitation learning: intentional disruption of expectation

Over repeated viewings of motion along a quasi-random path, ability to reproduce that path from memory improves. To assess the role of expectations and sequence context on such learning, subjects eye movements were measured while trajectories were viewed for subsequent reproduction. As a sequence of motions was repeated, subjects' eye movements became anticipatory, leading the stimulus' motions. To investigate how prediction errors affected eye movements and imitation learning, we injected an occasional deviant motion into a well-learned stimulus sequence, violating subjects' expectation about the motion that would be seen. This unexpected direction of motion in the stimulus sequence did not impair reproduction of the sequence. The externally induced prediction errors promoted one-shot learning: During the very next stimulus presentation, their eye movements showed that subjects now expected the new sequence item to reappear. A second experiment showed that an associative mismatch can facilitate accurate reproduction of an unexpected stimulus. After a deviant sequence item was presented, imitation accuracy for sequences that contained the deviant direction of motion was reduced relative to sequences that restored the original direction of motions. These findings demonstrate that in the context of a familiar sequence, unexpected events can play an important role in learning the sequence.

Visuospatial sequence learning without seeing

BACKGROUND

The ability to detect and integrate associations between unrelated items that are close in space and time is a key feature of human learning and memory. Learning sequential associations between non-adjacent visual stimuli (higher-order visuospatial dependencies) can occur either with or without awareness (explicit vs. implicit learning) of the products of learning. Existing behavioural and neurocognitive studies of explicit and implicit sequence learning, however, are based on conscious access to the sequence of target locations and, typically, on conditions where the locations for orienting, or motor, responses coincide with the locations of the target sequence.

METHODOLOGY/PRINCIPAL FINDINGS

Dichoptic stimuli were presented on a novel sequence learning task using a mirror stereoscope to mask the eye-of-origin of visual input from conscious awareness. We demonstrate that conscious access to the sequence of target locations and responses that coincide with structure of the target sequence are dispensable features when learning higher-order visuospatial associations. Sequence knowledge was expressed in the ability of participants to identify the trained higher-order visuospatial sequence on a recognition test, even though the trained and untrained recognition sequences were identical when viewed at a conscious binocular level, and differed only at the level of the masked sequential associations.

CONCLUSIONS/SIGNIFICANCE

These results demonstrate that unconscious processing can support perceptual learning of higher-order sequential associations through interocular integration of retinotopic-based codes stemming from monocular eye-of-origin information. Furthermore, unlike other forms of perceptual associative learning, visuospatial attention did not need to be directed to the locations of the target sequence. More generally, the results pose a challenge to neural models of learning to account for a previously unknown capacity of the human visual system to support the detection, learning and recognition of higher-order sequential associations under conditions where observers are unable to see the target sequence or perform responses that coincide with structure of the target sequence.

Rank signals in four areas of macaque frontal cortex during selection of actions and objects in serial order

Neurons in several areas of monkey frontal cortex exhibit ordinal position (rank) selectivity during the performance of serial order tasks. It has been unclear whether rank selectivity or the dependence of rank selectivity on task context varies across the areas of frontal cortex. To resolve this issue, we recorded from neurons in the supplementary motor area (SMA), presupplementary motor area (pre-SMA), supplementary eye field (SEF), and dorsolateral prefrontal cortex (dlPFC) as monkeys performed two oculomotor tasks, one requiring the selection of three actions in sequence and the other requiring the selection of three objects in sequence. We found that neurons representing all ranks were present in all areas. Only to a moderate degree did the prevalence and nature of rank selectivity vary from area to area. The two most prominent inter-area differences involved a lower prevalence of rank selectivity in the dlPFC than in the other areas and a higher proportion of neurons preferring late ranks in the SMA and SEF than in the other areas. Neurons in all four areas are rank generalists in the sense of favoring the same rank in both the serial action task and the serial object task.

Incidental and intentional sequence learning in youth-onset psychosis and Attention-Deficit/Hyperactivity Disorder (ADHD)

The goal was to compare incidental and intentional spatial sequence learning in youth-onset psychosis and ADHD. The authors tested 8- to 19-year-olds with psychosis or ADHD and healthy controls on a serial reaction time (RT) task and used manual and oculomotor measures to examine learning. Participants were also administered a block in which they were explicitly instructed to learn a sequence. As in our previous studies with healthy adults and children, oculomotor anticipations and RTs showed learning effects similar to those in the manual modality. Results showed intact sequence-specific learning but fewer oculomotor anticipations in both clinical groups during incidental learning. In intentional learning, only the psychosis group showed impairments compared to controls. There were no interactions between age and diagnosis. Thus, the psychosis group showed relatively preserved incidental learning despite impairments in intentional learning. Additionally, both clinical groups showed impairments in the ability to search for, extract, and anticipate regularities (whether the regularities were there or not), but not in the ability to respond to these regularities when they were there.

Disentangling perceptual from motor implicit sequence learning with a serial color-matching task

This paper contributes to the domain of implicit sequence learning by presenting a new version of the serial reaction time (SRT) task that allows unambiguously separating perceptual from motor learning. Participants matched the colors of three small squares with the color of a subsequently presented large target square. An identical sequential structure was tied to the colors of the target square (perceptual version, Experiment 1) or to the manual responses (motor version, Experiment 2). Short blocks of sequenced and randomized trials alternated and hence provided a continuous monitoring of the learning process. Reaction time measurements demonstrated clear evidence of independently learning perceptual and motor serial information, though revealed different time courses between both learning processes. No explicit awareness of the serial structure was needed for either of the two types of learning to occur. The paradigm introduced in this paper evidenced that perceptual learning can occur with SRT measurements and opens important perspectives for future imaging studies to answer the ongoing question, which brain areas are involved in the implicit learning of modality specific (motor vs. perceptual) or general serial order.