Speed, accuracy, and serial order in sequence production

Event-related brain potentials to typing errors in transparent and intransparent German words

Error detection in typing is crucial for assessing the adequacy of ongoing actions, leveraging both predictive mechanisms for early detection and sensory feedback for late detection. Neurophysiological studies have supported the anticipation of errors through predictive models. This research extends the understanding of error detection in typing, focusing on the neurocognitive mechanisms underlying errors in transparent and intransparent German words. Thirty-six volunteer students typed out aurally presented words, classified as either orthographically transparent or intransparent, on a computer keyboard without the possibility of correction. Because of poor spelling or excessive artifacts, the final sample comprised 27 participants. Event-related potentials (ERPs) were obtained time-locked to key presses, and behavioral data on typing correctness and speed were collected. A higher error rate and longer latency for intransparent words compared to transparent ones were found, suggesting the complexity of spelling impacts typing correctness. Post-error slowing was observed, aligning with increased cognitive control following errors. ERPs revealed a negative component akin to the error-related negativity (ERN) for typing errors, with a pronounced ERN-like negativity preceding erroneous key-presses, particularly for intransparent words. The study provides evidence of the cognitive and neural mechanisms underlying typing errors, highlighting the impact of orthographic transparency. The detection of an ERN-like negativity before erroneous key-presses, especially in typing intransparent words, underscores the brain's use of predictive mechanisms for error detection.

Temporal grouping effects in verbal and musical short-term memory: Is serial order representation domain-general?

The question of the domain-general versus domain-specific nature of the serial order mechanisms involved in short-term memory is currently under debate. The present study aimed at addressing this question through the study of temporal grouping effects in short-term memory tasks with musical material, a domain which has received little interest so far. The goal was to determine whether positional coding-currently the best account of grouping effect in verbal short-term memory-represents a viable mechanism to explain grouping effects in the musical domain. In a first experiment, non-musicians performed serial reconstruction of 6-tone sequences, where half of the sequences was grouped by groups of three items and the other half presented at a regular pace. The overall data pattern suggests that temporal grouping exerts on tone sequences reconstruction the same effects as in the verbal domain, except for ordering errors which were not characterised by the typical increase of interpositions. This pattern has been replicated in two additional experiments with verbal material, using the same grouping structure as in the musical experiment. The findings support that verbal and musical short-term memory domains are characterised by similar temporal grouping effects for the recall of 6-item lists grouped by three, but it also suggests the existence of boundary condition to observe an increase in interposition errors predicted by positional theories.

Not all errors are alike: modulation of error-related neural responses in musical joint action

During joint action, the sense of agency enables interaction partners to implement corrective and adaptive behaviour in response to performance errors. When agency becomes ambiguous (e.g. when action similarity encourages perceptual self-other overlap), confusion as to who produced what may disrupt this process. The current experiment investigated how ambiguity of agency affects behavioural and neural responses to errors in a joint action domain where self-other overlap is common: musical duos. Pairs of pianists performed piano pieces in synchrony, playing either the same pitches (ambiguous agency) or different pitches (unambiguous agency) while electroencephalography (EEG) was recorded for each individual. Behavioural and event-related potential results showed no effects of the agency manipulation but revealed differences in how distinct error types are processed. Self-produced 'wrong note' errors (substitutions) were left uncorrected, showed post-error slowing and elicited an error-related negativity (ERN) peaking before erroneous keystrokes (pre-ERN). In contrast, self-produced 'extra note' errors (additions) exhibited pre-error slowing, error and post-error speeding, were rapidly corrected and elicited the ERN. Other-produced errors evoked a feedback-related negativity but no behavioural effects. Overall findings shed light upon how the nervous system supports fluent interpersonal coordination in real-time joint action by employing distinct mechanisms to manage different types of errors.

Music predictability and liking enhance pupil dilation and promote motor learning in non-musicians

Humans can anticipate music and derive pleasure from it. Expectations facilitate the learning of movements associated with anticipated events, and they are also linked with reward, which may further facilitate learning of the anticipated rewarding events. The present study investigates the synergistic effects of predictability and hedonic responses to music on arousal and motor-learning in a naïve population. Novel melodies were manipulated in their overall predictability (predictable/unpredictable) as objectively defined by a model of music expectation, and ranked as high/medium/low liked based on participants' self-reports collected during an initial listening session. During this session, we also recorded ocular pupil size as an implicit measure of listeners' arousal. During the following motor task, participants learned to play target notes of the melodies on a keyboard (notes were of similar motor and musical complexity across melodies). Pupil dilation was greater for liked melodies, particularly when predictable. Motor performance was facilitated in predictable rather than unpredictable melodies, but liked melodies were learned even in the unpredictable condition. Low-liked melodies also showed learning but mostly in participants with higher scores of task perceived competence. Taken together, these results highlight  the effects of stimuli predictability on learning, which can be however overshadowed by the effects of stimulus liking or task-related intrinsic motivation.

Individual differences in musical training and executive functions: A latent variable approach

Learning and performing music draw on a host of cognitive abilities, and previous research has postulated that musicians might have advantages in related cognitive processes. One such aspect of cognition that may be related to musical training is executive functions (EFs), a set of top-down processes that regulate behavior and cognition according to task demands. Previous studies investigating the link between musical training and EFs have yielded mixed results and are difficult to compare. In part, this is because most studies have looked at only one specific cognitive process, and even studies looking at the same process have used different experimental tasks. Furthermore, most correlational studies have used different "musician" and "non-musician" categorizations for their comparisons, so generalizing the findings is difficult. The present study provides a more comprehensive assessment of how individual differences in musical training relate to latent measures of three separable aspects of EFs. We administered a well-validated EF battery containing multiple tasks tapping the EF components of inhibition, shifting, and working memory updating (Friedman et al. in Journal of Experimental Psychology: General, 137, 201-225, 2008), as well as a comprehensive, continuous measure of musical training and sophistication (Müllensiefen et al., in PLoS ONE, 9, e89642, 2014). Musical training correlated with some individual EF tasks involving inhibition and working memory updating, but not with individual tasks involving shifting. However, musical training only predicted the latent variable of working memory updating, but not the latent variables of inhibition or shifting after controlling for IQ, socioeconomic status, and handedness. Although these data are correlational, they nonetheless suggest that musical experience places particularly strong demands specifically on working memory updating processes.

Verbal and Musical Short-Term Memory: Evidence for Shared Serial Order Processes?

This study explored the validity of an integrative framework for verbal and musical short-term memory (STM). Following this framework, access to domain-specific long-term knowledge bases supports the processing of musical and verbal item information in STM, while domain-general ordering processes support the representation of serial order information in the two domains. We exposed participants to verbal and musical STM tasks assessing either item information, order information, or both item and order information. Using an interindividual differences approach, we observed that performance in item-based STM tasks was not strongly associated between musical and verbal domains. In contrast, strong between-domain associations were observed for STM tasks assessing processing of verbal order and musical rhythm information. These preliminary results are overall in agreement with an integrative approach of verbal and musical STM. At the same time, the results highlight the difficulty of measuring serial order processing in the musical STM domain in a direct and specific manner.

Tonal and textural influences on musical sight-reading

Two experiments investigated the impact of two structural factors-musical tonality and musical texture-on pianists' ability to play by sight without prior preparation, known as musical sight-reading. Tonality refers to the cognitive organization of tones around a central reference pitch, whereas texture refers to the organization of music in terms of the simultaneous versus successive onsets of tones as well as the number of hands (unimanual versus bimanual) involved in performance. Both experiments demonstrated that tonality and texture influenced sight-reading. For tonality, both studies found that errors in performance increased for passages with lesser perceived psychological stability (i.e., minor and atonal passages) relative to greater perceived stability (i.e., major passages). For texture, both studies found that errors in performance increased for passages that were more texturally complex, requiring two-handed versus one-handed performance, with some additional evidence that the relative simultaneity of note onsets (primarily simultaneous versus primarily successive) also influenced errors. These experiments are interpreted within a perception-action framework of music performance, highlighting influences of both top-down cognitive factors and bottom-up motoric processes on sight-reading behavior.

Electrical Brain Responses Reveal Sequential Constraints on Planning during Music Performance

Elements in speech and music unfold sequentially over time. To produce sentences and melodies quickly and accurately, individuals must plan upcoming sequence events, as well as monitor outcomes via auditory feedback. We investigated the neural correlates of sequential planning and monitoring processes by manipulating auditory feedback during music performance. Pianists performed isochronous melodies from memory at an initially cued rate while their electroencephalogram was recorded. Pitch feedback was occasionally altered to match either an immediately upcoming Near-Future pitch (next sequence event) or a more distant Far-Future pitch (two events ahead of the current event). Near-Future, but not Far-Future altered feedback perturbed the timing of pianists' performances, suggesting greater interference of Near-Future sequential events with current planning processes. Near-Future feedback triggered a greater reduction in auditory sensory suppression (enhanced response) than Far-Future feedback, reflected in the P2 component elicited by the pitch event following the unexpected pitch change. Greater timing perturbations were associated with enhanced cortical sensory processing of the pitch event following the Near-Future altered feedback. Both types of feedback alterations elicited feedback-related negativity (FRN) and P3a potentials and amplified spectral power in the theta frequency range. These findings suggest similar constraints on producers' sequential planning to those reported in speech production.

Domain-Generality of Timing-Based Serial Order Processes in Short-Term Memory: New Insights from Musical and Verbal Domains

Several models in the verbal domain of short-term memory (STM) consider a dissociation between item and order processing. This view is supported by data demonstrating that different types of time-based interference have a greater effect on memory for the order of to-be-remembered items than on memory for the items themselves. The present study investigated the domain-generality of the item versus serial order dissociation by comparing the differential effects of time-based interfering tasks, such as rhythmic interference and articulatory suppression, on item and order processing in verbal and musical STM domains. In Experiment 1, participants had to maintain sequences of verbal or musical information in STM, followed by a probe sequence, this under different conditions of interference (no-interference, rhythmic interference, articulatory suppression). They were required to decide whether all items of the probe list matched those of the memory list (item condition) or whether the order of the items in the probe sequence matched the order in the memory list (order condition). In Experiment 2, participants performed a serial order probe recognition task for verbal and musical sequences ensuring sequential maintenance processes, under no-interference or rhythmic interference conditions. For Experiment 1, serial order recognition was not significantly more impacted by interfering tasks than was item recognition, this for both verbal and musical domains. For Experiment 2, we observed selective interference of the rhythmic interference condition on both musical and verbal order STM tasks. Overall, the results suggest a similar and selective sensitivity to time-based interference for serial order STM in verbal and musical domains, but only when the STM tasks ensure sequential maintenance processes.

Auditory N1 reveals planning and monitoring processes during music performance

The current study investigated the relationship between planning processes and feedback monitoring during music performance, a complex task in which performers prepare upcoming events while monitoring their sensory outcomes. Theories of action planning in auditory-motor production tasks propose that the planning of future events co-occurs with the perception of auditory feedback. This study investigated the neural correlates of planning and feedback monitoring by manipulating the contents of auditory feedback during music performance. Pianists memorized and performed melodies at a cued tempo in a synchronization-continuation task while the EEG was recorded. During performance, auditory feedback associated with single melody tones was occasionally substituted with tones corresponding to future (next), present (current), or past (previous) melody tones. Only future-oriented altered feedback disrupted behavior: Future-oriented feedback caused pianists to slow down on the subsequent tone more than past-oriented feedback, and amplitudes of the auditory N1 potential elicited by the tone immediately following the altered feedback were larger for future-oriented than for past-oriented or noncontextual (unrelated) altered feedback; larger N1 amplitudes were associated with greater slowing following altered feedback in the future condition only. Feedback-related negativities were elicited in all altered feedback conditions. In sum, behavioral and neural evidence suggests that future-oriented feedback disrupts performance more than past-oriented feedback, consistent with planning theories that posit similarity-based interference between feedback and planning contents. Neural sensory processing of auditory feedback, reflected in the N1 ERP, may serve as a marker for temporal disruption caused by altered auditory feedback in auditory-motor production tasks.

Context and meter enhance long-range planning in music performance

Neural responses demonstrate evidence of resonance, or oscillation, during the production of periodic auditory events. Music contains periodic auditory events that give rise to a sense of beat, which in turn generates a sense of meter on the basis of multiple periodicities. Metrical hierarchies may aid memory for music by facilitating similarity-based associations among sequence events at different periodic distances that unfold in longer contexts. A fundamental question is how metrical associations arising from a musical context influence memory during music performance. Longer contexts may facilitate metrical associations at higher hierarchical levels more than shorter contexts, a prediction of the range model, a formal model of planning processes in music performance (Palmer and Pfordresher, 2003; Pfordresher et al., 2007). Serial ordering errors, in which intended sequence events are produced in incorrect sequence positions, were measured as skilled pianists performed musical pieces that contained excerpts embedded in long or short musical contexts. Pitch errors arose from metrically similar positions and further sequential distances more often when the excerpt was embedded in long contexts compared to short contexts. Musicians' keystroke intensities and error rates also revealed influences of metrical hierarchies, which differed for performances in long and short contexts. The range model accounted for contextual effects and provided better fits to empirical findings when metrical associations between sequence events were included. Longer sequence contexts may facilitate planning during sequence production by increasing conceptual similarity between hierarchically associated events. These findings are consistent with the notion that neural oscillations at multiple periodicities may strengthen metrical associations across sequence events during planning.

Making and monitoring errors based on altered auditory feedback

Previous research has demonstrated that altered auditory feedback (AAF) disrupts music performance and causes disruptions in both action planning and the perception of feedback events. It has been proposed that this disruption occurs because of interference within a shared representation for perception and action (Pfordresher, 2006). Studies reported here address this claim from the standpoint of error monitoring. In Experiment 1 participants performed short melodies on a keyboard while hearing no auditory feedback, normal auditory feedback, or alterations to feedback pitch on some subset of events. Participants overestimated error frequency when AAF was present but not for normal feedback. Experiment 2 introduced a concurrent load task to determine whether error monitoring requires executive resources. Although the concurrent task enhanced the effect of AAF, it did not alter participants' tendency to overestimate errors when AAF was present. A third correlational study addressed whether effects of AAF are reduced for a subset of the population who may lack the kind of perception/action associations that lead to AAF disruption: poor-pitch singers. Effects of manipulations similar to those presented in Experiments 1 and 2 were reduced for these individuals. We propose that these results are consistent with the notion that AAF interference is based on associations between perception and action within a forward internal model of auditory-motor relationships.

Perceptual pitch deficits coexist with pitch production difficulties in music but not Mandarin speech

Congenital amusia is a musical disorder that mainly affects pitch perception. Among Mandarin speakers, some amusics also have difficulties in processing lexical tones (tone agnosics). To examine to what extent these perceptual deficits may be related to pitch production impairments in music and Mandarin speech, eight amusics, eight tone agnosics, and 12 age- and IQ-matched normal native Mandarin speakers were asked to imitate music note sequences and Mandarin words of comparable lengths. The results indicated that both the amusics and tone agnosics underperformed the controls on musical pitch production. However, tone agnosics performed no worse than the amusics, suggesting that lexical tone perception deficits may not aggravate musical pitch production difficulties. Moreover, these three groups were all able to imitate lexical tones with perfect intelligibility. Taken together, the current study shows that perceptual musical pitch and lexical tone deficits might coexist with musical pitch production difficulties. But at the same time these perceptual pitch deficits might not affect lexical tone production or the intelligibility of the speech words that were produced. The perception-production relationship for pitch among individuals with perceptual pitch deficits may be, therefore, domain-dependent.

The role of pitch and temporal diversity in the perception and production of musical sequences

In two experiments we explored how the dimensions of pitch and time contribute to the perception and production of musical sequences. We tested how dimensional diversity (the number of unique categories in each dimension) affects how pitch and time combine. In Experiment 1, 18 musically trained participants rated the complexity of sequences varying only in their diversity in pitch or time; a separate group of 18 pianists reproduced these sequences after listening to them without practice. Overall, sequences with more diversity were perceived as more complex, but pitch diversity influenced ratings more strongly than temporal diversity. Further, although participants perceived sequences with high levels of pitch diversity as more complex, errors were more common in the sequences with higher diversity in time. Sequences in Experiment 2 exhibited diversity in both pitch and time; diversity levels were a subset of those tested in Experiment 1. Again diversity affected complexity ratings and errors, but there were no statistical interactions between dimensions. Nonetheless, pitch diversity was the primary factor in determining perceived complexity, and again temporal errors occurred more often than pitch errors. Additionally, diversity in one dimension influenced error rates in the other dimension in that both error types were more frequent relative to Experiment 1. These results suggest that although pitch and time do not interact directly, they are nevertheless not processed in an informationally encapsulated manner. The findings also align with a dimensional salience hypothesis, in which pitch is prioritised in the processing of typical Western musical sequences.

Rate effects on timing, key velocity, and finger kinematics in piano performance

We examined the effect of rate on finger kinematics in goal-directed actions of pianists. In addition, we evaluated whether movement kinematics can be treated as an indicator of personal identity. Pianists' finger movements were recorded with a motion capture system while they performed melodies from memory at different rates. Pianists' peak finger heights above the keys preceding keystrokes increased as tempo increased, and were attained about one tone before keypress. These rate effects were not simply due to a strategy to increase key velocity (associated with tone intensity) of the corresponding keystroke. Greater finger heights may compensate via greater tactile feedback for a speed-accuracy tradeoff that underlies the tendency toward larger temporal variability at faster tempi. This would allow pianists to maintain high temporal accuracy when playing at fast rates. In addition, finger velocity and accelerations as pianists' fingers approached keys were sufficiently unique to allow pianists' identification with a neural-network classifier. Classification success was higher in pianists with more extensive musical training. Pianists' movement “signatures” may reflect unique goal-directed movement kinematic patterns, leading to individualistic sound.

Prediction, cognition and the brain

The term “predictive brain” depicts one of the most relevant concepts in cognitive neuroscience which emphasizes the importance of “looking into the future”, namely prediction, preparation, anticipation, prospection or expectations in various cognitive domains. Analogously, it has been suggested that predictive processing represents one of the fundamental principles of neural computations and that errors of prediction may be crucial for driving neural and cognitive processes as well as behavior. This review discusses research areas which have recognized the importance of prediction and introduces the relevant terminology and leading theories in the field in an attempt to abstract some generative mechanisms of predictive processing. Furthermore, we discuss the process of testing the validity of postulated expectations by matching these to the realized events and compare the subsequent processing of events which confirm to those which violate the initial predictions. We conclude by suggesting that, although a lot is known about this type of processing, there are still many open issues which need to be resolved before a unified theory of predictive processing can be postulated with regard to both cognitive and neural functioning.