A grammar of action generates predictions in skilled musicians

Lateral Prefrontal Cortex Is a Hub for Music Production from Structural Rules to Movements

Complex sequential behaviors, such as speaking or playing music, entail flexible rule-based chaining of single acts. However, it remains unclear how the brain translates abstract structural rules into movements. We combined music production with multimodal neuroimaging to dissociate high-level structural and low-level motor planning. Pianists played novel musical chord sequences on a muted MR-compatible piano by imitating a model hand on screen. Chord sequences were manipulated in terms of musical harmony and context length to assess structural planning, and in terms of fingers used for playing to assess motor planning. A model of probabilistic sequence processing confirmed temporally extended dependencies between chords, as opposed to local dependencies between movements. Violations of structural plans activated the left inferior frontal and middle temporal gyrus, and the fractional anisotropy of the ventral pathway connecting these two regions positively predicted behavioral measures of structural planning. A bilateral frontoparietal network was instead activated by violations of motor plans. Both structural and motor networks converged in lateral prefrontal cortex, with anterior regions contributing to musical structure building, and posterior areas to movement planning. These results establish a promising approach to study sequence production at different levels of action representation.

The effects of dual-task interference in predicting turn-ends in speech and music

Determining when a partner's spoken or musical turn will end requires well-honed predictive abilities. Evidence suggests that our motor systems are activated during perception of both speech and music, and it has been argued that motor simulation is used to predict turn-ends across domains. Here we used a dual-task interference paradigm to investigate whether motor simulation of our partner's action underlies our ability to make accurate turn-end predictions in speech and in music. Furthermore, we explored how specific this simulation is to the action being predicted. We conducted two experiments, one investigating speech turn-ends, and one investigating music turn-ends. In each, 34 proficient pianists predicted turn-endings while (1) passively listening, (2) producing an effector-specific motor activity (mouth/hand movement), or (3) producing a task- and effector-specific motor activity (mouthing words/fingering a piano melody). In the speech experiment, any movement during speech perception disrupted predictions of spoken turn-ends, whether the movement was task-specific or not. In the music experiment, only task-specific movement (i.e., fingering a piano melody) disrupted predictions of musical turn-ends. These findings support the use of motor simulation to make turn-end predictions in both speech and music but suggest that the specificity of this simulation may differ between domains.

Musical novices perform with equal accuracy when learning to drum alone or with a peer

The capacity of expert musicians to coordinate with each other when playing in ensembles or rehearsing has been widely investigated. However, little is known about the ability of novices to achieve satisfactory coordinated behaviour when making music together. We tested whether performance accuracy differs when novices play a newly learned drumming pattern with another musically untrained individual (duo group) or alone (solo group). A comparison between musical outcomes of the two groups revealed no significant differences concerning performative accuracy. An additional, exploratory examination of the degree of mutual influence between members of the duos suggested that they reciprocally affected each other when playing together. These findings indicate that a responsive auditory feedback involving surprises introduced by human errors could be part of pedagogical settings that employ repetition or imitation, thereby facilitating coordination among novices in a less prescribed fashion.

Do children learn from their prediction mistakes? A registered report evaluating error-based theories of language acquisition

Error-based theories of language acquisition suggest that children, like adults, continuously make and evaluate predictions in order to reach an adult-like state of language use. However, while these theories have become extremely influential, their central claim-that unpredictable input leads to higher rates of lasting change in linguistic representations-has scarcely been tested. We designed a prime surprisal-based intervention study to assess this claim. As predicted, both 5- to 6-year-old children (n = 72) and adults (n = 72) showed a pre- to post-test shift towards producing the dative syntactic structure they were exposed to in surprising sentences. The effect was significant in both age groups together, and in the child group separately when participants with ceiling performance in the pre-test were excluded. Secondary predictions were not upheld: we found no verb-based learning effects and there was only reliable evidence for immediate prime surprisal effects in the adult, but not in the child group. To our knowledge, this is the first published study demonstrating enhanced learning rates for the same syntactic structure when it appeared in surprising as opposed to predictable contexts, thus providing crucial support for error-based theories of language acquisition.

Music predictability and liking enhance pupil dilation and promote motor learning in non-musicians

Humans can anticipate music and derive pleasure from it. Expectations facilitate the learning of movements associated with anticipated events, and they are also linked with reward, which may further facilitate learning of the anticipated rewarding events. The present study investigates the synergistic effects of predictability and hedonic responses to music on arousal and motor-learning in a naïve population. Novel melodies were manipulated in their overall predictability (predictable/unpredictable) as objectively defined by a model of music expectation, and ranked as high/medium/low liked based on participants' self-reports collected during an initial listening session. During this session, we also recorded ocular pupil size as an implicit measure of listeners' arousal. During the following motor task, participants learned to play target notes of the melodies on a keyboard (notes were of similar motor and musical complexity across melodies). Pupil dilation was greater for liked melodies, particularly when predictable. Motor performance was facilitated in predictable rather than unpredictable melodies, but liked melodies were learned even in the unpredictable condition. Low-liked melodies also showed learning but mostly in participants with higher scores of task perceived competence. Taken together, these results highlight  the effects of stimuli predictability on learning, which can be however overshadowed by the effects of stimulus liking or task-related intrinsic motivation.

Interaction between Perceived Action and Music Sequences in the Left Prefrontal Area

Observing another person's piano play and listening to a melody interact with the observer's execution of piano play. This interaction is thought to occur because the execution of musical-action and the perception of both musical-action and musical-sound share a common representation in which the frontoparietal network is involved. However, it is unclear whether the perceptions of observed piano play and listened musical sound use a common neural resource. The present study used near-infrared spectroscopy to determine whether the interaction between the perception of musical-action and musical-sound sequences appear in the left prefrontal area. Measurements were obtained while participants watched videos that featured hands playing familiar melodies on a piano keyboard. Hand movements were paired with either a congruent or an incongruent melody. Two groups of participants (nine well-trained and nine less-trained) were instructed to identify the melody according to hand movements and to ignore the accompanying auditory track. Increased cortical activation was detected in the well-trained participants when hand movements were paired with incongruent melodies. Therefore, an interference effect was detected regarding the processing of action and sound sequences, indicating that musical-action sequences may be perceived with a representation that is also used for the perception of musical-sound sequences. However, in less-trained participants, such a contrast was not detected between conditions despite both groups featuring comparable key-touch reading abilities. Therefore, the current results imply that the left prefrontal area is involved in translating temporally structured sequences between domains. Additionally, expertise may be a crucial factor underlying this translation.

Behavioral Quantification of Audiomotor Transformations in Improvising and Score-Dependent Musicians

The historically developed practice of learning to play a music instrument from notes instead of by imitation or improvisation makes it possible to contrast two types of skilled musicians characterized not only by dissimilar performance practices, but also disparate methods of audiomotor learning. In a recent fMRI study comparing these two groups of musicians while they either imagined playing along with a recording or covertly assessed the quality of the performance, we observed activation of a right-hemisphere network of posterior superior parietal and dorsal premotor cortices in improvising musicians, indicating more efficient audiomotor transformation. In the present study, we investigated the detailed performance characteristics underlying the ability of both groups of musicians to replicate music on the basis of aural perception alone. Twenty-two classically-trained improvising and score-dependent musicians listened to short, unfamiliar two-part excerpts presented with headphones. They played along or replicated the excerpts by ear on a digital piano, either with or without aural feedback. In addition, they were asked to harmonize or transpose some of the excerpts either to a different key or to the relative minor. MIDI recordings of their performances were compared with recordings of the aural model. Concordance was expressed in an audiomotor alignment score computed with the help of music information retrieval algorithms. Significantly higher alignment scores were found when contrasting groups, voices, and tasks. The present study demonstrates the superior ability of improvising musicians to replicate both the pitch and rhythm of aurally perceived music at the keyboard, not only in the original key, but also in other tonalities. Taken together with the enhanced activation of the right dorsal frontoparietal network found in our previous fMRI study, these results underscore the conclusion that the practice of improvising music can be associated with enhanced audiomotor transformation in response to aurally perceived music.

Syntax in Action Has Priority over Movement Selection in Piano Playing: An ERP Study

Complex human behavior is hierarchically organized. Whether or not syntax plays a role in this organization is currently under debate. The present ERP study uses piano performance to isolate syntactic operations in action planning and to demonstrate their priority over nonsyntactic levels of movement selection. Expert pianists were asked to execute chord progressions on a mute keyboard by copying the posture of a performing model hand shown in sequences of photos. We manipulated the final chord of each sequence in terms of Syntax (congruent/incongruent keys) and Manner (conventional/unconventional fingering), as well as the strength of its predictability by varying the length of the Context (five-chord/two-chord progressions). The production of syntactically incongruent compared to congruent chords showed a response delay that was larger in the long compared to the short context. This behavioral effect was accompanied by a centroparietal negativity in the long but not in the short context, suggesting that a syntax-based motor plan was prepared ahead. Conversely, the execution of the unconventional manner was not delayed as a function of Context and elicited an opposite electrophysiological pattern (a posterior positivity). The current data support the hypothesis that motor plans operate at the level of musical syntax and are incrementally translated to lower levels of movement selection.

Improvisation and the self-organization of multiple musical bodies

Understanding everyday behavior relies heavily upon understanding our ability to improvise, how we are able to continuously anticipate and adapt in order to coordinate with our environment and others. Here we consider the ability of musicians to improvise, where they must spontaneously coordinate their actions with co-performers in order to produce novel musical expressions. Investigations of this behavior have traditionally focused on describing the organization of cognitive structures. The focus, here, however, is on the ability of the time-evolving patterns of inter-musician movement coordination as revealed by the mathematical tools of complex dynamical systems to provide a new understanding of what potentiates the novelty of spontaneous musical action. We demonstrate this approach through the application of cross wavelet spectral analysis, which isolates the strength and patterning of the behavioral coordination that occurs between improvising musicians across a range of nested time-scales. Revealing the sophistication of the previously unexplored dynamics of movement coordination between improvising musicians is an important step toward understanding how creative musical expressions emerge from the spontaneous coordination of multiple musical bodies.

A conceptual review on action-perception coupling in the musicians' brain: what is it good for?

Experience with a sensorimotor task, such as practicing a piano piece, leads to strong coupling of sensory (visual or auditory) and motor cortices. Here we review behavioral and neurophysiological (M/EEG, TMS and fMRI) research exploring this topic using the brain of musicians as a model system. Our review focuses on a recent body of evidence suggesting that this form of coupling might have (at least) two cognitive functions. First, it leads to the generation of equivalent predictions (concerning both when and what event is more likely to occur) during both perception and production of music. Second, it underpins the common coding of perception and action that supports the integration of the motor output of multiple musicians’ in the context of joint musical tasks. Essentially, training-based coupling of perception and action might scaffold the human ability to represent complex (structured) actions and to entrain multiple agents—via reciprocal prediction and adaptation—in the pursuit of shared goals.

Sensorimotor event: an approach to the dynamic, embodied, and embedded nature of sensorimotor cognition

In this paper, I explore the notion of sensorimotor event as the building block of sensorimotor cognition. A sensorimotor event is presented here as a neurally controlled event that recruits those processes and elements that are necessary to address the demands of the situation in which the individual is involved. The notion of sensorimotor event is intended to subsume the dynamic, embodied, and embedded nature of sensorimotor cognition, in agreement with the satisficing and bricoleur approach to sensorimotor cognition presented elsewhere (Vilarroya, 2012). In particular, the notion of sensorimotor event encompasses those relevant neural processes, but also those bodily and environmental elements, that are necessary to deal with the situation in which the individual is involved. This continuum of neural processes as well as bodily and environmental elements can be characterized, and this characterization is considered the basis for the identification of the particular sensorimotor event. Among other consequences, the notion of sensorimotor event suggests a different approach to the classical account of sensory-input mapping onto a motor output. Instead of characterizing how a neural system responds to an external input, the idea defended here is to characterize how system-in-an-environment responds to its antecedent situation.

Motor simulation and the coordination of self and other in real-time joint action

Joint actions require the integration of simultaneous self- and other-related behaviour. Here, we investigated whether this function is underpinned by motor simulation, that is the capacity to represent a perceived action in terms of the neural resources required to execute it. This was tested in a music performance experiment wherein on-line brain stimulation (double-pulse transcranial magnetic stimulation, dTMS) was employed to interfere with motor simulation. Pianists played the right-hand part of piano pieces in synchrony with a recording of the left-hand part, which had (Trained) or had not (Untrained) been practiced beforehand. Training was assumed to enhance motor simulation. The task required adaptation to tempo changes in the left-hand part that, in critical conditions, were preceded by dTMS delivered over the right primary motor cortex. Accuracy of tempo adaptation following dTMS or sham stimulations was compared across Trained and Untrained conditions. Results indicate that dTMS impaired tempo adaptation accuracy only during the perception of trained actions. The magnitude of this interference was greater in empathic individuals possessing a strong tendency to adopt others' perspectives. These findings suggest that motor simulation provides a functional resource for the temporal coordination of one's own behaviour with others in dynamic social contexts.

Distinguishing self and other in joint action. Evidence from a musical paradigm

The capacity to distinguish between one's own and others' behavior is a cognitive prerequisite for successful joint action. We employed a musical joint action task to investigate how the brain achieves this distinction. Pianists performed the right-hand part of piano pieces, previously learned bimanually, while the complementary left-hand part either was not executed or was (believed to be) played by a co-performer. This experimental setting served to induce a co-representation of the left-hand part reflecting either the self or the co-performer. Single-pulse transcranial magnetic stimulation was applied to the right primary motor cortex and motor-evoked potentials (MEPs) were recorded from the resting left forearm. Results show that corticospinal excitability was modulated by whether the representation of the left hand was associated with the self or the other, with the MEP amplitude being low and high, respectively. This result remained unchanged in a separate session where participants could neither see nor hear the other but still infer his presence by means of contextual information. Furthermore, the amplitude of MEPs associated with co-performer presence increased with pianists' self-reported empathy. Thus, the sociality of the context modulates action attribution at the level of the motor control system.