Sensory information in perceptual-motor sequence learning: visual and/or tactile stimuli

Stimulus-locked auditory information facilitates real-time visuo-motor sequence learning

Prior research investigating whether and how multisensory information facilitates skill learning is quite mixed; whereas some research points to congruent information improving learning, other work suggests that people become reliant on the redundant information, such that its removal ultimately detracts from the ability to perform a unisensory task. We examined this question using the Serial Interception Sequence Learning (SISL) task, a visuo-motor paradigm in which participants implicitly learn a sequence embedded in noise. We investigated whether adding auditory information in different ways would enhance real time sequence learning and whether any benefits of multisensory learning would persist with visual-only testing. Auditory information was used either as feedback on the visuo-motor task (Experiments 1 and 2) or was presented synchronously with visual information during learning (Experiment 3). Robust sequence-specific performance advantages occurred across conditions and experiments; however, auditory information enhanced real-time performance only when it was synchronized with visual information. Participants were significantly more accurate, faster, and more precise with stimulus-locked auditory information during training. Notably, these benefits did not generalize to the visual-only context, suggesting that the benefits of stimulus-locked auditory information are primarily useful only when the perceptual information is present.

Tones slow down visuomotor responses in a visual-spatial task

The current study examined how simple tones affect speeded visuomotor responses in a visual-spatial sequence learning task. Across the three reported experiments, participants were presented with a visual target that appeared in different locations on a touchscreen monitor and they were instructed to touch the visual targets as quickly as possible. Visual sequences were either paired with sounds that correlated with the location of the target, paired with sounds that did not correlate with the location of the target, or the sequences were presented in silence (baseline). Response times decreased across training and participants were slower to respond to the visual stimuli when the sequences were paired with tones. Moreover, these interference effects were more pronounced early in training and explicit instructions directing attention to the visual modality had little effect on eliminating auditory interference, suggesting that these interference effects may stem from bottom-up factors and do not appear to be under attentional control. These findings have implications on tasks that require the processing of simultaneously presented auditory and visual information and provide support for a proposed mechanism underlying auditory dominance on a task that is typically better suited for the visual modality.

Consistent Shifts of Stimulus Modality Induce Chunking in Sequence Learning

The ability to learn event sequences is crucial to human behavior because it enables us to interact adaptively with our environment. The sensory environment is essential in guiding the acquisition of these sequences, so the role of the stimulus modality in sequence learning is of high relevance. The present study examined structured stimulus modality shifts in sequence learning using the serial reaction time task (SRT). Participants had to respond to numbers that were presented either in the visual or in the auditory modality. Importantly, the numbers, as well as the stimulus modality, followed a fixed pattern. We found better performance in sequenced than in random blocks, indicating sequence learning. Moreover, the performance was better when the stimulus modality remained the same than when the modality changed between successive trials (the modality shift effect, MSE). However, sequence learning facilitated performance primarily in modality repetitions, so that the MSE became progressively larger in the sequenced blocks, while it was small in the random blocks, and this pattern was particularly pronounced for the participants who showed a high recall level for the response sequence. To account for this effect, we assume that consistent modality shifts induce parsing of the sequence into chunks. Because chunk retrieval at chunk boundaries incurs RT costs, the acquired sequence knowledge is more efficiently expressed in modality repetitions (i.e., within chunks). Together, the data suggest that the formation of explicit knowledge enhances chunking in sequence learning.

Learning and recognition of tactile temporal sequences by mice and humans

The world around us is replete with stimuli that unfold over time. When we hear an auditory stream like music or speech or scan a texture with our fingertip, physical features in the stimulus are concatenated in a particular order. This temporal patterning is critical to interpreting the stimulus. To explore the capacity of mice and humans to learn tactile sequences, we developed a task in which subjects had to recognise a continuous modulated noise sequence delivered to whiskers or fingertips, defined by its temporal patterning over hundreds of milliseconds. GO and NO-GO sequences differed only in that the order of their constituent noise modulation segments was temporally scrambled. Both mice and humans efficiently learned tactile sequences. Mouse sequence recognition depended on detecting transitions in noise amplitude; animals could base their decision on the earliest information available. Humans appeared to use additional cues, including the duration of noise modulation segments.

DOI:http://dx.doi.org/10.7554/eLife.27333.001

Effects of an Additional Sequence of Color Stimuli on Visuomotor Sequence Learning

Through practice, people are able to integrate a secondary sequence (e.g., a stimulus-based sequence) into a primary sequence (e.g., a response-based sequence), but it is still controversial whether the integrated sequences lead to better learning than only the primary sequence. In the present study, we aimed to investigate the effects of a sequence that integrated space and color sequences on early and late learning phases (corresponding to effector-independent and effector-dependent learning, respectively) and how the effects differed in the integrated and primary sequences in each learning phase. In the task, the participants were required to learn a sequence of button presses using trial-and-error and to perform the sequence successfully for 20 trials (m × n task). First, in the baseline task, all participants learned a non-colored sequence, in which the response button always turned red. Then, in the learning task, the participants were assigned to two groups: a colored sequence group (i.e., space and color) or a non-colored sequence group (i.e., space). In the colored sequence, the response button turned a pre-determined color and the participants were instructed to attend to the sequences of both location and color as much as they could. The results showed that the participants who performed the colored sequence acquired the correct button presses of the sequence earlier, but showed a slower mean performance time than those who performed the non-colored sequence. Moreover, the slower performance time in the colored sequence group remained in a subsequent transfer task in which the spatial configurations of the buttons were vertically mirrored from the learning task. These results indicated that if participants explicitly attended to both the spatial response sequence and color stimulus sequence at the same time, they could develop their spatial representations of the sequence earlier (i.e., early development of the effector-independent learning), but might not be able to enhance their motor representations of the sequence (i.e., late development of the effector-dependent learning). Thus, the undeveloped effector-dependent representations in the colored sequence group directly led to a long performance time in the transfer sequence.

Multisensory Congruency Enhances Explicit Awareness in a Sequence Learning Task

We examined the effect of audiovisual training on learning a repeated sequence of motor responses. Participants were trained with either congruent or incongruent audiovisual cues to produce motor responses. Learning was tested by comparing reaction times to untrained sequences and by asking participants to recreate the trained sequence. A strong association was found between the two measures and the majority of high-scoring participants belonged to the congruent audiovisual condition. Because the second measure requires explicit knowledge of the trained sequence, we conclude that audiovisual congruency facilitates explicit learning.

Exposing sequence learning in a double-step task

Is it possible to learn to perform a motor sequence without awareness of the sequence? In two experiments, we presented participants with the most elementary sequence: an alternation between two options. We used a double-step pointing task in which the final position of the target alternated between two quite similar values. The task forced participants to start moving before the final target was visible, allowing us to determine participants’ expectations about the final target position without explicitly asking them. We tracked participants’ expectations (and thus motor sequence learning) by measuring the direction of the initial part of the movement, before any response to the final step. We found that participants learnt to anticipate the average size of the final step, but that they did not learn the sequence. In a second experiment, we extended the duration of the learning period and increased the difference in size between the target position changes. Some participants started anticipating the step size in accordance with the sequence at some time during the experiment. These participants reported having noticed the simple sequence. The participants who had not noticed the sequence did not move in anticipation of the sequence. This suggests that participants who did not learn this very simple sequence explicitly also did not learn it implicitly.

Mental representations of magnitude and order: a dissociation by sensorimotor learning

Numbers and spatially directed actions share cognitive representations. This assertion is derived from studies that have demonstrated that the processing of small- and large-magnitude numbers facilitates motor behaviors that are directed to the left and right, respectively. However, little is known about the role of sensorimotor learning for such number-action associations. In this study, we show that sensorimotor learning in a serial reaction-time task can modify the associations between number magnitudes and spatially directed movements. Experiments 1 and 3 revealed that this effect is present only for the learned sequence and does not transfer to a novel unpracticed sequence. Experiments 2 and 4 showed that the modification of stimulus-action associations by sensorimotor learning does not occur for other sets of ordered stimuli such as letters of the alphabet. These results strongly suggest that numbers and actions share a common magnitude representation that differs from the common order representation shared by letters and spatially directed actions. Only the magnitude representation, but not the order representation, can be modified episodically by sensorimotor learning.

Modality-specific organization in the representation of sensorimotor sequences

Sensorimotor representations of movement sequences are hierarchically organized. Here we test the effects of different stimulus modalities on such organizations. In the visual group, participants responded to a repeated sequence of visually presented stimuli by depressing spatially compatible keys on a response pad. In the auditory group, learners were required to respond to auditorily presented stimuli, which had no direct spatial correspondence with the response keys: the lowest pitch corresponded to the leftmost key and the highest pitch to the rightmost key. We demonstrate that hierarchically and auto-organized sensorimotor representations are developed through practice, which are specific both to individuals and stimulus modalities. These findings highlight the dynamic and sensory-specific modulation of chunk processing during sensorimotor learning - sensorimotor chunking - and provide evidence that modality-specific mechanisms underlie the hierarchical organization of sequence representations.

Perceptual sequence learning is more severely impaired than motor sequence learning in patients with chronic cerebellar stroke

Patients with cerebellar stroke are impaired in procedural learning. Several different learning mechanisms contribute to procedural learning in healthy individuals. The aim was to compare the relative share of different learning mechanisms in patients and healthy controls. Ten patients with cerebellar stroke and 12 healthy controls practiced a visuomotor serial reaction time task. Learning blocks with high stimulus-response compatibility were exercised repeatedly; in between these, participants performed test blocks with the same or a different (mirror-inverted or unrelated) stimulus sequence and/or the same or a different (mirror-inverted) stimulus-response allocation. This design allowed to measure the impact of motor learning and perceptual learning independently and to separate both mechanisms from the learning of stimulus-response pairs. Analysis of the learning blocks showed that, as expected, both patients and controls improved their performance over time, although patients remained significantly slower. Analysis of the test blocks revealed that controls showed significant motor learning as well as significant visual perceptual learning, whereas cerebellar patients showed only significant motor learning. Healthy participants were able to use perceptual information for procedural learning even when the rule linking stimuli and responses had been changed, whereas patients with cerebellar lesions could not recruit this perception-based mechanism. Therefore, the cerebellum appears involved in the accurate processing of perceptual information independent from prelearned stimulus-response mappings.

The study of sequence learning in individuals with schizophrenia: a critical review of the literature

The serial reaction time task (SRTT) has been used extensively to study implicit sequence learning. A number of studies have used the SRTT to examine sequence learning in schizophrenia patients. Despite these studies, it remains unclear whether sequence learning is impaired in patients, whether antipsychotic medications affect sequence learning, and what types of sequential information patients might have difficulty learning. Methodological limitations have made it difficult to obtain good answers to these questions. Methodological innovations from the general SRTT literature that have not yet been adopted in the schizophrenia literature could provide better answers.

Only correlated sequences that are actively processed contribute to implicit sequence learning

The purpose of the study was to investigate how implicit sequence learning is affected by the presence of secondary information that is correlated with the primary sequence but not necessarily relevant to performance. In a previous work, we have shown that correlation plays an important role but other prerequisites may also be involved. In Experiments 1 and 2, using a task sequence learning paradigm, we found that primary sequence learning was not affected by secondary information that was sequenced but irrelevant to performance, even though the two streams of information were correlated. In contrast, in Experiment 3, we found that sensitivity to the main sequence was greater with the provision of extra sequenced information that was relevant to performance in addition to being correlated. This suggests that sequence learning was enhanced through the integration of information. We conclude that information in secondary as well as primary sequences must be actively processed if it is to have a beneficial impact. By actively processed we mean information that is selectively attended and necessary for carrying out the tasks.

Redundant sensory information does not enhance sequence learning in the serial reaction time task

In daily life we encounter multiple sources of sensory information at any given moment. Unknown is whether such sensory redundancy in some way affects implicit learning of a sequence of events. In the current paper we explored this issue in a serial reaction time task. Our results indicate that redundant sensory information does not enhance sequence learning when all sensory information is presented at the same location (responding to the position and/or color of the stimuli; Experiment 1), even when the distinct sensory sources provide more or less similar baseline response latencies (responding to the shape and/or color of the stimuli; Experiment 2). These findings support the claim that sequence learning does not (necessarily) benefit from sensory redundancy. Moreover, transfer was observed between various sets of stimuli, indicating that learning was predominantly response-based.

Functional dissociation of hippocampal mechanism during implicit learning based on the domain of associations

Traditionally, the medial temporal lobe (MTL) was linked to explicit or declarative memory in associative learning. However, recent studies have reported MTL involvement even when volunteers are not consciously aware of the learned contingencies. Therefore, the mechanism of the MTL-related learning process cannot be described sufficiently by the explicit/implicit distinction, and the underlying process in the MTL for associative learning needs a more functional characterization. A possible feature that would allow a functional specification also for implicit learning is the nature of the material that is learned. Given that implicit memory tasks often comprise a combination of perceptual and motor learning, we hypothesized that implicit learning of the perceptual but not the motor component entails MTL activation in these studies. To directly test this hypothesis, we designed a purely perceptual and a purely motor variant of the serial reaction time task. In two groups of human volunteers, behavioral results clearly showed that both variants were learned without awareness. Neuronal recordings using fMRI revealed that bilateral hippocampal activation was observed only for implicit learning of the perceptual sequence, not for the motor sequence. This dissociation clearly shows that the functional role of the hippocampus for learning is determined by the domain of the learned association and that the function of the medial temporal lobe system is the processing of contingencies between perceptual features regardless of the explicit or implicit nature of the ensuing memory.

Representing serial action and perception

This article presents a review on the representational base of sequence learning in the serial reaction time task. The first part of the article addresses the major questions and challenges that underlie the debate on implicit and explicit learning. In the second part, the informational content that underlies sequence representations is reviewed. The latter issue has produced a rich and equivocal literature. A taxonomy illustrates that substantial support exists for associations between successive stimulus features, between successive response features, and between successive response-to-stimulus compounds. We suggest that sequence learning is not predetermined with respect to one particular type of information but, rather, develops according to an overall principle of activation contingent on task characteristics. Moreover, substantiating such an integrative approach is proposed by a synthesis with the dual-system model (Keele, Ivry, Mayr, Hazeltine, & Heuer, 2003).