The planning and execution of short auditory sequences

Repetition costs in sequence chunking

We examined how flexibly we plan sequences of actions when we switch between multiple action sequences. Mastering a sequential skill is assumed to involve integrating successive actions into groups known as chunks that can be efficiently planned and smoothly executed. Chunking is suggested by gains in planning efficiency for long compared to short action sequences following practice and learning associations between actions and perceptual outcomes. Less is understood about how efficiently we plan sequential chunks when we switch between multiple action sequences. Do we plan learned chunks less efficiently when we switch to a different action sequence? We examined this question by comparing the initiation and execution latencies of long versus short action sequences, performed from memory, when sequences switched or repeated across trials. Additionally, each action within the sequences generated predictable perceptual outcomes that were either spatially compatible or spatially incompatible with the action sequences. Results suggested repetition costs (instead of benefits) when performing long sequences. Repetition, as opposed to switching, prolonged initiation and increased the error rate of long compared to short sequences. We attribute these results to the flexible coordination of chunk planning and execution. Repetition may prolong advanced planning of long sequences in order to resolve conflict between multiple chunks, and switching may allow the planning of later chunks to be postponed until execution. We propose that the chunking of action sequences can both facilitate and interfere with action-switching performance.

Same, but different: Binding effects in auditory, but not visual detection performance

Responding to a stimulus leads to the integration of response and stimulus' features into an event file. Upon repetition of any of its features, the previous event file is retrieved, thereby affecting ongoing performance. Such integration-retrieval explanations exist for a number of sequential tasks (that measure these processes as 'binding effects') and are thought to underlie all actions. However, based on attentional orienting literature, Schöpper, Hilchey, et al. (2020) could show that binding effects are absent when participants detect visual targets in a sequence: In visual detection performance, there is simply a benefit for target location changes (inhibition of return). In contrast, Mondor and Leboe (2008) had participants detect auditory targets in a sequence, and found a benefit for frequency repetition - presumably reflecting a binding effect in auditory detection performance. In the current study, we conducted two experiments, that only differed in the modality of the target: Participants signaled the detection of a sound (N = 40) or of a visual target (N = 40). Whereas visual detection performance showed a pattern incongruent with binding assumptions, auditory detection performance revealed a non-spatial feature repetition benefit, suggesting that frequency was bound to the response. Cumulative reaction time distributions indicated that the absence of a binding effect in visual detection performance was not caused by overall faster responding. The current results show a clear limitation to binding accounts in action control: Binding effects are not only limited by task demands, but can entirely depend on target modality.

Simultaneous self-other integration and segregation support real-time interpersonal coordination in a musical joint action task

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

The role of action effects in motor sequence planning and execution: exploring the influence of temporal and spatial effect anticipation

Actions we perform every day generate perceivable outcomes with both spatial and temporal features. According to the ideomotor principle, we plan our actions by anticipating the outcomes, but this principle does not directly address how sequential movements are influenced by different outcomes. We examined how sequential action planning is influenced by the anticipation of temporal and spatial features of action outcomes. We further explored the influence of action sequence switching. Participants performed cued sequences of button presses that generated visual effects which were either spatially compatible or incompatible with the sequences, and the spatial effects appeared after a short or long delay. The sequence cues switched or repeated across trials, and the predictability of action sequence switches was varied across groups. The results showed a delay-anticipation effect for sequential action, whereby a shorter anticipated delay between action sequences and their outcomes speeded initiation and execution of the cued action sequences. Delay anticipation was increased by predictable action switching, but it was not strongly modified by the spatial compatibility of the action outcomes. The results extend previous demonstrations of delay anticipation to the context of sequential action. The temporal delay between actions and their outcomes appears to be retrieved for sequential planning and influences both the initiation and the execution of actions.

Music predictability and liking enhance pupil dilation and promote motor learning in non-musicians

Humans can anticipate music and derive pleasure from it. Expectations facilitate the learning of movements associated with anticipated events, and they are also linked with reward, which may further facilitate learning of the anticipated rewarding events. The present study investigates the synergistic effects of predictability and hedonic responses to music on arousal and motor-learning in a naïve population. Novel melodies were manipulated in their overall predictability (predictable/unpredictable) as objectively defined by a model of music expectation, and ranked as high/medium/low liked based on participants' self-reports collected during an initial listening session. During this session, we also recorded ocular pupil size as an implicit measure of listeners' arousal. During the following motor task, participants learned to play target notes of the melodies on a keyboard (notes were of similar motor and musical complexity across melodies). Pupil dilation was greater for liked melodies, particularly when predictable. Motor performance was facilitated in predictable rather than unpredictable melodies, but liked melodies were learned even in the unpredictable condition. Low-liked melodies also showed learning but mostly in participants with higher scores of task perceived competence. Taken together, these results highlight  the effects of stimuli predictability on learning, which can be however overshadowed by the effects of stimulus liking or task-related intrinsic motivation.

Priming of complex action via movement contingent sensory effects

OBJECTIVES

The aim of the present study was to examine whether movement contingent sensory effects could be used to prime and facilitate motor performance on a ball-tossing task.

DESIGN

The ball-tossing task was performed across two consecutive days, and consisted of an acquisition phase and a test phase. During the acquisition phase, participants (N = 30) practiced an underhanded ball tossing task to a near and far target (N = 360 total, n = 180 each distance). Tosses that landed near the target immediately produced an auditory feedback tone upon landing, with unique tones for both the near and far target. In the test phase, the auditory tones preceded the toss and served as imperative stimuli for the tossing task.

METHOD

The test phase consisted of three tossing conditions (corresponding, non-corresponding, and control) in which the participants responded to the tones by tossing the ball to either the corresponding or non-corresponding target associated with the tones during learning.

RESULTS

Findings indicated that both accuracy and consistency of ball tossing improved when the toss was preceded by the corresponding auditory feedback associated with the successful execution of the action during learning.

CONCLUSIONS

The present study extends previous research by showing that complex actions consisting of multiple degrees of freedom can be primed via movement contingent sensory effects. Furthermore, this study demonstrates that movement-effect priming can impact distal measures of motor performance (e.g., accuracy of tossing), as opposed to the features of movement production (e.g., response selection, initiation, and execution).

Contributions of expected sensory and affective action effects to action selection and performance: Evidence from forced- and free-choice tasks

Whereas ideomotor approaches to action control emphasize the importance of sensory action effects for action selection, motivational approaches emphasize the role of affective action effects. We used a game-like experimental setup to directly compare the roles of sensory and affective action effects in selecting and performing reaching actions in forced- and free-choice tasks. The two kinds of action effects did not interact. Action selection and execution in the forced-choice task were strongly impacted by the spatial compatibility between actions and the expected sensory action effects, whereas the free-choice task was hardly affected. In contrast, action execution, but not selection, in both tasks was strongly impacted by the spatial compatibility between actions and highly valued action effects. This pattern suggests that sensory and affective action effects serve different purposes: The former seem to dominate rule-based action selection, whereas the latter might serve to reduce any remaining action uncertainty.

A conceptual review on action-perception coupling in the musicians' brain: what is it good for?

Experience with a sensorimotor task, such as practicing a piano piece, leads to strong coupling of sensory (visual or auditory) and motor cortices. Here we review behavioral and neurophysiological (M/EEG, TMS and fMRI) research exploring this topic using the brain of musicians as a model system. Our review focuses on a recent body of evidence suggesting that this form of coupling might have (at least) two cognitive functions. First, it leads to the generation of equivalent predictions (concerning both when and what event is more likely to occur) during both perception and production of music. Second, it underpins the common coding of perception and action that supports the integration of the motor output of multiple musicians’ in the context of joint musical tasks. Essentially, training-based coupling of perception and action might scaffold the human ability to represent complex (structured) actions and to entrain multiple agents—via reciprocal prediction and adaptation—in the pursuit of shared goals.

Modulation of isochronous movements in a flexible environment: links between motion and auditory experience

The ability to perform isochronous movements while listening to a rhythmic auditory stimulus requires a flexible process that integrates timing information with movement. Here, we explored how non-temporal and temporal characteristics of an auditory stimulus (presence, interval occupancy, and tempo) affect motor performance. These characteristics were chosen on the basis of their ability to modulate the precision and accuracy of synchronized movements. Subjects have participated in sessions in which they performed sets of repeated isochronous wrist's flexion-extensions under various conditions. The conditions were chosen on the basis of the defined characteristics. Kinematic parameters were evaluated during each session, and temporal parameters were analyzed. In order to study the effects of the auditory stimulus, we have minimized all other sensory information that could interfere with its perception or affect the performance of repeated isochronous movements. The present study shows that the distinct characteristics of an auditory stimulus significantly influence isochronous movements by altering their duration. Results provide evidence for an adaptable control of timing in the audio-motor coupling for isochronous movements. This flexibility would make plausible the use of different encoding strategies to adapt audio-motor coupling for specific tasks.

Mental practice promotes motor anticipation: evidence from skilled music performance

Mental practice (MP) has been shown to improve movement accuracy and velocity, but it is not known whether MP can also optimize movement timing. We addressed this question by studying two groups of expert pianists who performed challenging music sequences after either MP or physical practice (PP). Performance and motion-capture data were collected along with responses to imagery questionnaires. The results showed that MP produced performance improvements, although to a lower degree than PP did. MP and PP induced changes in both movement velocity and movement timing, promoting the emergence of movement anticipatory patterns. Furthermore, motor imagery was associated with greater changes in movement velocity, while auditory imagery was associated with greater movement anticipation. Data from a control group that was not allowed to practice confirmed that the changes in accuracy and kinematics were not due to mere repetition of the sequence during testing. This study provides the first evidence of an anticipatory control following MP and extends the present knowledge on the effectiveness of MP to a task of unparalleled motor complexity. The practical implications of MP in the motor domain are discussed.

Monitoring Individual and Joint Action Outcomes in Duet Music Performance

We investigated whether people monitor the outcomes of their own and their partners' individual actions as well as the outcome of their combined actions when performing joint actions together. Pairs of pianists memorized both parts of a piano duet. Each pianist then performed one part while their partner performed the other; EEG was recorded from both. Auditory outcomes (pitches) associated with keystrokes produced by the pianists were occasionally altered in a way that either did or did not affect the joint auditory outcome (i.e., the harmony of a chord produced by the two pianists' combined pitches). Altered auditory outcomes elicited a feedback-related negativity whether they occurred in the pianist's own part or the partner's part, and whether they affected individual or joint action outcomes. Altered auditory outcomes also elicited a P300 whose amplitude was larger when the alteration affected the joint outcome compared with individual outcomes and when the alteration affected the pianist's own part compared with the partner's part. Thus, musicians engaged in joint actions monitor their own and their partner's actions as well as their combined action outcomes, while at the same time maintaining a distinction between their own and others' actions and between individual and joint outcomes.

Mental imagery in music performance: underlying mechanisms and potential benefits

This paper examines the role of mental imagery in music performance. Self-reports by musicians, and various other sources of anecdotal evidence, suggest that covert auditory, motor, and/or visual imagery facilitate multiple aspects of music performance. The cognitive and motor mechanisms that underlie such imagery include working memory, action simulation, and internal models. Together these mechanisms support the generation of anticipatory images that enable thorough action planning and movement execution that is characterized by efficiency, temporal precision, and biomechanical economy. In ensemble performance, anticipatory imagery may facilitate interpersonal coordination by enhancing online predictions about others' action timing. Overlap in brain regions subserving auditory imagery and temporal prediction is consistent with this view. It is concluded that individual differences in anticipatory imagery may be a source of variation in expressive performance excellence and the quality of ensemble cohesion. Engaging in effortful musical imagery is therefore justified when artistic perfection is the goal.

Adaptive control of ideomotor effect anticipations

According to ideomotor theory, voluntary actions are selected and initiated by means of anticipated action effects. Prior experiments yielded evidence for these effect anticipations with response-effect (R-E) compatibility phenomena using blocked R-E relations. Daily actions, however, typically evoke different effects depending on the situational context. In the present study, we accounted for this natural variability and investigated R-E compatibility effects by a trial-by-trial variation of R-E compatibility relations. In line with recent observations regarding ideomotor learning, R-E compatibility influenced responding only when participants responded in free choice trials assuming that participants then adopted an intention-based action control mode. In contrast, R-E compatibility had no impact when participants responded according to imperative stimuli throughout the experiment, thus when participants adopted a stimulus-based action control mode. Interestingly, once an intention-based mode was established because of free choice trials within an experimental block, we observed response compatibility effects in free as well as forced choice trials. These findings extend and refine theoretical assumptions on different action control modes in goal-directed behavior and the specific contribution of ideomotor processes to intention-based action control.

Implicit chord processing and motor representation in pianists

The aim of this paper is to assess the relevance of pitch dimension in auditory-motor interaction. Several behavioural and brain imaging studies have shown that auditory processing of sounds can activate motor representations, an effect which is however elicited only by action-related sounds, i.e., sounds linked to a specific motor repertoire. Music provides an appropriate framework for further exploration of this issue. Three groups of participants (pianists, non-pianist musicians and non-musicians) were tested with a shape decision task where left-hand and right-hand responses were required; each visual stimulus was paired with an auditory task-irrelevant stimulus (high-pitched or low-pitched piano-timbre chords). Of the three groups, only pianists had longer reaction times for left-hand/high-pitched chords and right-hand/low-pitched chords associations. These findings are consistent with an auditory-motor effect elicited by pitch dimension, as only pianists show an interaction between motor responses and implicit pitch processing. This interaction is consistent with the canonical mapping of hand gestures and pitch dimension on the piano keyboard. The results are discussed within the ideo-motor theoretical framework offered by the Theory of Event Coding (Hommel et al. in Behav Brain Sci 24:849-937, 2001).

Multilevel coordination stability: integrated goal representations in simultaneous intra-personal and inter-agent coordination

The influence of integrated goal representations on multilevel coordination stability was investigated in a task that required finger tapping in antiphase with metronomic tone sequences (inter-agent coordination) while alternating between the two hands (intra-personal coordination). The maximum rate at which musicians could perform this task was measured when taps did or did not trigger feedback tones. Tones produced by the two hands (very low, low, medium, high, and very high) could be the same as, or different from, one another and the (medium-pitched) metronome tones. The benefits of feedback tones were greatest when they were close in pitch to the metronome and the left hand triggered low tones, while the right hand triggered high tones. Thus, multilevel coordination was facilitated by tones that were easy to integrate with, but perceptually distinct from, the metronome, and by compatibility of movement patterns and feedback pitches.

Action Planning in Sequential Skills: Relations to Music Performance

The hypothesis that planning music-like sequential actions involves anticipating their auditory effects was investigated in a series of experiments. Participants with varying levels of musical experience responded to each of four colour-patch stimuli by producing a unique sequence of three taps on three vertically aligned keys. Each tap triggered a tone in most experimental conditions. Response–effect (key-to-tone) mapping was either compatible—taps on the top, middle, and bottom keys triggered high, medium, and low pitched tones, respectively—or incompatible—key-to-tone mapping was scrambled, reversed, or neutral (taps on different keys triggered the same tone). The results suggest that action planning was faster with compatible than with incompatible mappings (and faster than with no tones). Furthermore, the size of this compatibility effect grew with increasing musical experience, which suggests that improvements in auditory imagery ability that typically accompany musical training may augment the role of anticipatory auditory-effect representations during planning.

Prospective coding in event representation

A perceived event such as a visual stimulus in the external world and a to-be-produced event such as an intentional action are subserved by event representations. Event representations do not only contain information about present states but also about past and future states. Here we focus on the role of representing future states in event perception and generation (i.e., prospective coding). Relevant theoretical issues and paradigms are discussed. We suggest that the predictive power of the motor system may be exploited for prospective coding not only in producing but also in perceiving events. Predicting is more advantageous than simply reacting. Perceptual prediction allows us to select appropriate responses ahead of the realization of an (anticipated) event and therefore, it is indispensable to flexibly and timely adapt to new situations and thus, successfully interact with our physical and social environment.

Anticipatory response control in motor sequence learning: Evidence from stimulus–response compatibility

Three experiments using a serial four-choice reaction-time (RT) task explored the interaction of sequence learning and stimulus-based response conflict. In Experiment 1, the spatial stimulus-response (S-R) mapping was manipulated between participants. Incompatible S-R mappings produced much higher RTs than the compatible mapping, but sequence learning decreased this S-R compatibility effect. In Experiment 2, the spatial stimulus feature was made task-irrelevant by assigning responses to symbols that were presented at unpredictable locations. The data indicated a Simon effect (i.e., increased RT when irrelevant stimulus location is spatially incompatible with response location) that was reduced by sequence learning. However, this effect was observed only in participants who developed an explicit sequence representation. Experiment 3 replicated this learning-based modulation of the Simon effect using explicit sequence-learning instructions. Taken together, the data support the notion that explicit sequence learning can lead to motor 'chunking', so that pre-planned response sequences are shielded from conflicting stimulus information.