Short-term effects of processing musical syntax: an ERP study

Cognitive and sensory expectations independently shape musical expectancy and pleasure

Expectation is crucial for our enjoyment of music, yet the underlying generative mechanisms remain unclear. While sensory models derive predictions based on local acoustic information in the auditory signal, cognitive models assume abstract knowledge of music structure acquired over the long term. To evaluate these two contrasting mechanisms, we compared simulations from four computational models of musical expectancy against subjective expectancy and pleasantness ratings of over 1000 chords sampled from 739 US Billboard pop songs. Bayesian model comparison revealed that listeners' expectancy and pleasantness ratings were predicted by the independent, non-overlapping, contributions of cognitive and sensory expectations. Furthermore, cognitive expectations explained over twice the variance in listeners' perceived surprise compared to sensory expectations, suggesting a larger relative importance of long-term representations of music structure over short-term sensory-acoustic information in musical expectancy. Our results thus emphasize the distinct, albeit complementary, roles of cognitive and sensory expectations in shaping musical pleasure, and suggest that this expectancy-driven mechanism depends on musical information represented at different levels of abstraction along the neural hierarchy. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Sequences in harmony: Cognitive interactions between musical and visual narrative structure

From film and television to graphic storytelling, tonal music can accompany visual narratives in a variety of contexts. The apprehension of both musical and narrative sequences involves temporal categories in ordered patterning, which raises an interesting question: Do musical progressions and visual narratives rely on shared sequence processing mechanisms? If this is the case, then cues from music and sequential static images, when presented simultaneously, should interact during audiovisual online processing. We tested this question by measuring reaction times to target picture panels appearing in visual narrative (comic strip) sequences, which were presented panel by panel and synchronized with musical chord progressions. Image sequences were either coherent narratives or incoherent (random) panels, and they were aligned with musical accompaniment consisting of coherent tonal chord progressions or non-tonal (unrelated) chords. Reaction times were faster for target images in coherent sequences than incoherent sequences, and even faster for coherent images with tonal accompaniment than non-tonal chords. This indicated an interaction between sequential structures across domains. We take these results as evidence for a shared, domain-general sequence processing mechanism operating across music and visual narrative.

Explicit processing of melodic structure in congenital amusia can be improved by redescription-associate learning

Congenital amusia is a neurodevelopmental disorder of musical processing. Previous research demonstrates that although explicit musical processing is impaired in congenital amusia, implicit musical processing can be intact. However, little is known about whether implicit knowledge could improve explicit musical processing in individuals with congenital amusia. To this end, we developed a training method utilizing redescription-associate learning, aiming at transferring implicit representations of perceptual states into explicit forms through verbal description and then establishing the associations between the perceptual states reported and responses via feedback, to investigate whether the explicit processing of melodic structure could be improved in individuals with congenital amusia. Sixteen amusics and 11 controls rated the degree of expectedness of melodies during EEG recording before and after training. In the interim, half of the amusics received nine training sessions on melodic structure, while the other half received no training. Results, based on effect size estimation, showed that at pretest, amusics but not controls failed to explicitly distinguish the regular from the irregular melodies and to exhibit an ERAN in response to the irregular endings. At posttest, trained but not untrained amusics performed as well as controls at both the behavioral and neural levels. At the 3-month follow-up, the training effects still maintained. These findings present novel electrophysiological evidence of neural plasticity in the amusic brain, suggesting that redescription-associate learning may be an effective method to remediate impaired explicit processes for individuals with other neurodevelopmental disorders who have intact implicit knowledge.

Modulation of theta and gamma oscillations during familiarization with previously unknown music

Repeated listening to unknown music leads to gradual familiarization with musical sequences. Passively listening to musical sequences could involve an array of dynamic neural responses in reaching familiarization with the musical excerpts. This study elucidates the dynamic brain response and its variation over time by investigating the electrophysiological changes during the familiarization with initially unknown music. Twenty subjects were asked to familiarize themselves with previously unknown 10 s classical music excerpts over three repetitions while their electroencephalogram was recorded. Dynamic spectral changes in neural oscillations are monitored by time-frequency analyses for all frequency bands (theta: 5-9 Hz, alpha: 9-13 Hz, low-beta: 13-21 Hz, high beta: 21-32 Hz, and gamma: 32-50 Hz). Time-frequency analyses reveal sustained theta event-related desynchronization (ERD) in the frontal-midline and the left pre-frontal electrodes which decreased gradually from 1st to 3rd time repetition of the same excerpts (frontal-midline: 57.90 %, left-prefrontal: 75.93 %). Similarly, sustained gamma ERD decreased in the frontal-midline and bilaterally frontal/temporal areas (frontal-midline: 61.47 %, left-frontal: 90.88 %, right-frontal: 87.74 %). During familiarization, the decrease of theta ERD is superior in the first part (1-5 s) whereas the decrease of gamma ERD is superior in the second part (5-9 s) of music excerpts. The results suggest that decreased theta ERD is associated with successfully identifying familiar sequences, whereas decreased gamma ERD is related to forming unfamiliar sequences.

Relationship between early neural responses to syntactic and acoustic irregularities in music

Humans can detect various anomalies in a sound sequence without attending to each dimension explicitly. Event-related potentials (ERPs) have been used to examine the processes of auditory deviance detection. Previous research has shown that music-syntactic anomalies elicit early right anterior negativity (ERAN), whereas more general acoustic irregularities elicit mismatch negativity (MMN). Although these ERP components occur in a similar latency range with a similar scalp topography, the relationship between the detection processes they reflect remains unclear. This study compared these components by manipulating music-syntactic (chord progression) and acoustic (intensity) irregularities orthogonally in two experiments. Non-musicians (Experiment 1: N = 39; Experiment 2: N = 24) were asked to listen to chord sequences, each consisting of 5 four-voice chords, as they watched a silent video clip. Standard, harmonic-deviant, intensity-deviant and double-deviant chords occurred at the final position in each sequence. Deviant stimuli were presented infrequently (p = .10) in Experiment 1 and equiprobably (p = .25) in Experiment 2. Regardless of deviance probability, both harmonic and intensity deviants elicited similar negativities, which were indistinguishable in terms of latency or scalp distribution. When the two deviant types occurred simultaneously, the negativity increased in an additive manner; that is, the amplitude of the double-deviant ERP was as large as the sum of the single-deviant ERPs. These findings suggest that the detection of music-syntactic and acoustic irregularities works independently, based on different regularity representations.

Neural correlates of acoustic dissonance in music: The role of musicianship, schematic and veridical expectations

In western music, harmonic expectations can be fulfilled or broken by unexpected chords. Musical irregularities in the absence of auditory deviance elicit well-studied neural responses (e.g. ERAN, P3, N5). These responses are sensitive to schematic expectations (induced by syntactic rules of chord succession) and veridical expectations about predictability (induced by experimental regularities). However, the cognitive and sensory contributions to these responses and their plasticity as a result of musical training remains under debate. In the present study, we explored whether the neural processing of pure acoustic violations is affected by schematic and veridical expectations. Moreover, we investigated whether these two factors interact with long-term musical training. In Experiment 1, we registered the ERPs elicited by dissonant clusters placed either at the middle or the ending position of chord cadences. In Experiment 2, we presented to the listeners with a high proportion of cadences ending in a dissonant chord. In both experiments, we compared the ERPs of musicians and non-musicians. Dissonant clusters elicited distinctive neural responses (an early negativity, the P3 and the N5). While the EN was not affected by syntactic rules, the P3a and P3b were larger for dissonant closures than for middle dissonant chords. Interestingly, these components were larger in musicians than in non-musicians, while the N5 was the opposite. Finally, the predictability of dissonant closures in our experiment did not modulate any of the ERPs. Our study suggests that, at early time windows, dissonance is processed based on acoustic deviance independently of syntactic rules. However, at longer latencies, listeners may be able to engage integration mechanisms and further processes of attentional and structural analysis dependent on musical hierarchies, which are enhanced in musicians.

Musical Expertise Is Associated with Improved Neural Statistical Learning in the Auditory Domain

 

It is poorly known whether musical training is associated with improvements in general cognitive abilities, such as statistical learning (SL). In standard SL paradigms, musicians have shown better performances than nonmusicians. However, this advantage could be due to differences in auditory discrimination, in memory or truly in the ability to learn sequence statistics. Unfortunately, these different hypotheses make similar predictions in terms of expected results. To dissociate them, we developed a Bayesian model and recorded electroencephalography (EEG). Our results confirm that musicians perform approximately 15% better than nonmusicians at predicting items in auditory sequences that embed either low or high-order statistics. These higher performances are explained in the model by parameters governing the learning of high-order statistics and the selection stage noise. EEG recordings reveal a neural underpinning of the musician's advantage: the P300 amplitude correlates with the surprise elicited by each item, and so, more strongly for musicians. Finally, early EEG components correlate with the surprise elicited by low-order statistics, as opposed to late EEG components that correlate with the surprise elicited by high-order statistics and this effect is stronger for musicians. Overall, our results demonstrate that musical expertise is associated with improved neural SL in the auditory domain.

SIGNIFICANCE STATEMENT

It is poorly known whether musical training leads to improvements in general cognitive skills. One fundamental cognitive ability, SL, is thought to be enhanced in musicians, but previous studies have reported mixed results. This is because such musician's advantage can embrace very different explanations, such as improvement in auditory discrimination or in memory. To solve this problem, we developed a Bayesian model and recorded EEG to dissociate these explanations. Our results reveal that musical expertise is truly associated with an improved ability to learn sequence statistics, especially high-order statistics. This advantage is reflected in the electroencephalographic recordings, where the P300 amplitude is more sensitive to surprising items in musicians than in nonmusicians.

Mapping Tonal Hierarchy in the Brain

In Western tonal music, pitches are organized hierarchically based on their perceived fit in a specific tonal context. This hierarchy forms scales that are commonly used in Western tonal music. The hierarchical nature of tonal structure is well established behaviourally; however, the neural underpinnings are largely unknown. In this study, EEG data and goodness-of-fit ratings were collected from 34 participants who listened to an arpeggio followed by a probe tone, where the probe tone could be any chromatic scale degree and the context any of the major keys. Goodness-of-fit ratings corresponded to the classic tonal hierarchy. N1, P2 and the Early Right Anterior Negativity (ERAN) were significantly modulated by scale degree. Furthermore, neural marker amplitudes and latencies were significantly correlated with similar magnitude to both pitch height and goodness-of-fit ratings. This is different from the clearer divide between pitch height correlating with early neural markers (100-200 ms) and tonal hierarchy correlating with late neural markers (200-1000 ms) reported by Sankaran et al. (2020) and Quiroga-Martinez et al. (2019). Finally, individual differences were greater than any main effects detected when pooling participants and brain-behavior correlations vary widely (i.e. r = -0.8 to 0.8).

Manipulation of low-level features modulates grouping strength of auditory objects

A central challenge of auditory processing involves the segregation, analysis, and integration of acoustic information into auditory perceptual objects for processing by higher order cognitive operations. This study explores the influence of low-level features on auditory object perception. Participants provided perceived musicality ratings in response to randomly generated pure tone sequences. Previous work has shown that music perception relies on the integration of discrete sounds into a holistic structure. Hence, high (versus low) ratings were viewed as indicative of strong (versus weak) object formation. Additionally, participants rated sequences in which random subsets of tones were manipulated along one of three low-level dimensions (timbre, amplitude, or fade-in) at one of three strengths (low, medium, or high). Our primary findings demonstrate how low-level acoustic features modulate the perception of auditory objects, as measured by changes in musicality ratings for manipulated sequences. Secondarily, we used principal component analysis to categorize participants into subgroups based on differential sensitivities to low-level auditory dimensions, thereby highlighting the importance of individual differences in auditory perception. Finally, we report asymmetries regarding the effects of low-level dimensions; specifically, the perceptual significance of timbre. Together, these data contribute to our understanding of how low-level auditory features modulate auditory object perception.

Increased fronto-temporal connectivity by modified melody in real music

In real music, the original melody may appear intact, with little elaboration only, or significantly modified. Since a melody is most easily perceived in music, hearing significantly modified melody may change a brain connectivity. Mozart KV 265 is comprised of a theme with an original melody of “Twinkle Twinkle Little Star” and its significant variations. We studied whether effective connectivity changes with significantly modified melody, between bilateral inferior frontal gyri (IFGs) and Heschl’s gyri (HGs) using magnetoencephalography (MEG). Among the 12 connectivities, the connectivity from the left IFG to the right HG was consistently increased with significantly modified melody compared to the original melody in 2 separate sets of the same rhythmic pattern with different melody (p = 0.005 and 0.034, Bonferroni corrected). Our findings show that the modification of an original melody in a real music changes the brain connectivity.

The human brain processes hierarchical structures of meter and harmony differently: Evidence from musicians and nonmusicians

The processing of temporal structure has been widely investigated, but evidence on how the brain processes temporal and nontemporal structures simultaneously is sparse. Using event-related potentials (ERPs), we examined how the brain responds to temporal (metric) and nontemporal (harmonic) structures in music simultaneously, and whether these processes are impacted by musical expertise. Fifteen musicians and 15 nonmusicians rated the degree of completeness of musical sequences with or without violations in metric or harmonic structures. In the single violation conditions, the ERP results showed that both musicians and nonmusicians exhibited an early right anterior negativity (ERAN) as well as an N5 to temporal violations ("when"), and only an N5-like response to nontemporal violations ("what"), which were consistent with the behavioral results. In the double violation condition, however, only the ERP results, but not the behavioral results, revealed a significant interaction between temporal and nontemporal violations at a later integrative stage, as manifested by an enlarged N5 effect compared to the single violation conditions. These findings provide the first evidence that the human brain uses different neural mechanisms in processing metric and harmonic structures in music, which may shed light on how the brain generates predictions for "what" and "when" events in the natural environment.

Cortical encoding of melodic expectations in human temporal cortex

Humans engagement in music rests on underlying elements such as the listeners' cultural background and interest in music. These factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing. Here we recorded cortical signals as participants listened to Bach melodies. We assessed the relative contributions of acoustic versus melodic components of the music to the neural signal. Melodic features included information on pitch progressions and their tempo, which were extracted from a predictive model of musical structure based on Markov chains. We related the music to brain activity with temporal response functions demonstrating, for the first time, distinct cortical encoding of pitch and note-onset expectations during naturalistic music listening. This encoding was most pronounced at response latencies up to 350 ms, and in both planum temporale and Heschl's gyrus.

Change in left inferior frontal connectivity with less unexpected harmonic cadence by musical expertise

In terms of harmonic expectancy, compared to an expected dominant-to-tonic and an unexpected dominant-to-supertonic, a dominant-to-submediant is a less unexpected cadence, the perception of which may depend on the subject’s musical expertise. The present study investigated how aforementioned 3 different cadences are processed in the networks of bilateral inferior frontal gyri (IFGs) and superior temporal gyri (STGs) with magnetoencephalography. We compared the correct rate and brain connectivity in 9 music-majors (mean age, 23.5 ± 3.4 years; musical training period, 18.7 ± 4.0 years) and 10 non-music-majors (mean age, 25.2 ± 2.6 years; musical training period, 4.2 ± 1.5 years). For the brain connectivity, we computed the summation of partial directed coherence (PDC) values for inflows/outflows to/from each area (sPDC_i/sPDC_o) in bilateral IFGs and STGs. In the behavioral responses, music-majors were better than non-music-majors for all 3 cadences (p < 0.05). However, sPDC_i/sPDC_o was prominent only for the dominant-to-submediant in the left IFG. The sPDC_i was more strongly enhanced in music-majors than in non-music-majors (p = 0.002, Bonferroni corrected), while the sPDC_o was vice versa (p = 0.005, Bonferroni corrected). Our data show that music-majors, with higher musical expertise, are better in identifying a less unexpected cadence than non-music-majors, with connectivity changes centered on the left IFG.

Musical training modulates the early but not the late stage of rhythmic syntactic processing

Syntactic processing is essential for musical understanding. Although the processing of harmonic syntax has been well studied, very little is known about the neural mechanisms underlying rhythmic syntactic processing. The present study investigated the neural processing of rhythmic syntax and whether and to what extent long-term musical training impacts such processing. Fourteen musicians and 14 nonmusicians listened to syntactic-regular or syntactic-irregular rhythmic sequences and judged the completeness of these sequences. Nonmusicians, as well as musicians, showed a P600 effect to syntactic-irregular endings, indicating that musical exposure and perceptual learning of music are sufficient to enable nonmusicians to process rhythmic syntax at the late stage. However, musicians, but not nonmusicians, also exhibited an early right anterior negativity (ERAN) response to syntactic-irregular endings, which suggests that musical training only modulates the early but not the late stage of rhythmic syntactic processing. These findings revealed for the first time the neural mechanisms underlying the processing of rhythmic syntax in music, which has important implications for theories of hierarchically organized music cognition and comparative studies of syntactic processing in music and language.

Effects of veridical expectations on syntax processing in music: Event-related potential evidence

Numerous past studies have investigated neurophysiological correlates of music-syntactic processing. However, only little is known about how prior knowledge about an upcoming syntactically irregular event modulates brain correlates of music-syntactic processing. Two versions of a short chord sequence were presented repeatedly to non-musicians (n = 20) and musicians (n = 20). One sequence version ended on a syntactically regular chord, and the other one ended on a syntactically irregular chord. Participants were either informed (cued condition), or not informed (non-cued condition) about whether the sequence would end on the regular or the irregular chord. Results indicate that in the cued condition (compared to the non-cued condition) the peak latency of the early right anterior negativity (ERAN), elicited by irregular chords, was earlier in both non-musicians and musicians. However, the expectations due to the knowledge about the upcoming event (veridical expectations) did not influence the amplitude of the ERAN. These results suggest that veridical expectations modulate only the speed, but not the principle mechanisms, of music-syntactic processing.

Event-related brain responses while listening to entire pieces of music

Brain responses to discrete short sounds have been studied intensively using the event-related potential (ERP) method, in which the electroencephalogram (EEG) signal is divided into epochs time-locked to stimuli of interest. Here we introduce and apply a novel technique which enables one to isolate ERPs in human elicited by continuous music. The ERPs were recorded during listening to a Tango Nuevo piece, a deep techno track and an acoustic lullaby. Acoustic features related to timbre, harmony, and dynamics of the audio signal were computationally extracted from the musical pieces. Negative deflation occurring around 100 milliseconds after the stimulus onset (N100) and positive deflation occurring around 200 milliseconds after the stimulus onset (P200) ERP responses to peak changes in the acoustic features were distinguishable and were often largest for Tango Nuevo. In addition to large changes in these musical features, long phases of low values that precede a rapid increase - and that we will call Preceding Low-Feature Phases - followed by a rapid increase enhanced the amplitudes of N100 and P200 responses. These ERP responses resembled those to simpler sounds, making it possible to utilize the tradition of ERP research with naturalistic paradigms.

Prior experience biases subcortical sensitivity to sound patterns

To make sense of our ever-changing world, our brains search out patterns. This drive can be so strong that the brain imposes patterns when there are none. The opposite can also occur: The brain can overlook patterns because they do not conform to expectations. In this study, we examined this neural sensitivity to patterns within the auditory brainstem, an evolutionarily ancient part of the brain that can be fine-tuned by experience and is integral to an array of cognitive functions. We have recently shown that this auditory hub is sensitive to patterns embedded within a novel sound stream, and we established a link between neural sensitivity and behavioral indices of learning [Skoe, E., Krizman, J., Spitzer, E., & Kraus, N. The auditory brainstem is a barometer of rapid auditory learning. Neuroscience, 243, 104-114, 2013]. We now ask whether this sensitivity to stimulus statistics is biased by prior experience and the expectations arising from this experience. To address this question, we recorded complex auditory brainstem responses (cABRs) to two patterned sound sequences formed from a set of eight repeating tones. For both patterned sequences, the eight tones were presented such that the transitional probability (TP) between neighboring tones was either 33% (low predictability) or 100% (high predictability). Although both sequences were novel to the healthy young adult listener and had similar TP distributions, one was perceived to be more musical than the other. For the more musical sequence, participants performed above chance when tested on their recognition of the most predictable two-tone combinations within the sequence (TP of 100%); in this case, the cABR differed from a baseline condition where the sound sequence had no predictable structure. In contrast, for the less musical sequence, learning was at chance, suggesting that listeners were "deaf" to the highly predictable repeating two-tone combinations in the sequence. For this condition, the cABR also did not differ from baseline. From this, we posit that the brainstem acts as a Bayesian sound processor, such that it factors in prior knowledge about the environment to index the probability of particular events within ever-changing sensory conditions.

The effects of supervised learning on event-related potential correlates of music-syntactic processing

Humans process music even without conscious effort according to implicit knowledge about syntactic regularities. Whether such automatic and implicit processing is modulated by veridical knowledge has remained unknown in previous neurophysiological studies. This study investigates this issue by testing whether the acquisition of veridical knowledge of a music-syntactic irregularity (acquired through supervised learning) modulates early, partly automatic, music-syntactic processes (as reflected in the early right anterior negativity, ERAN), and/or late controlled processes (as reflected in the late positive component, LPC). Excerpts of piano sonatas with syntactically regular and less regular chords were presented repeatedly (10 times) to non-musicians and amateur musicians. Participants were informed by a cue as to whether the following excerpt contained a regular or less regular chord. Results showed that the repeated exposure to several presentations of regular and less regular excerpts did not influence the ERAN elicited by less regular chords. By contrast, amplitudes of the LPC (as well as of the P3a evoked by less regular chords) decreased systematically across learning trials. These results reveal that late controlled, but not early (partly automatic), neural mechanisms of music-syntactic processing are modulated by repeated exposure to a musical piece. This article is part of a Special Issue entitled SI: Prediction and Attention.

Implicit and explicit statistical learning of tone sequences across spectral shifts

We investigated how the statistical learning of auditory sequences is reflected in neuromagnetic responses in implicit and explicit learning conditions. Complex tones with fundamental frequencies (F0s) in a five-tone equal temperament were generated by a formant synthesizer. The tones were subsequently ordered with the constraint that the probability of the forthcoming tone was statistically defined (80% for one tone; 5% for the other four) by the latest two successive tones (second-order Markov chains). The tone sequence consisted of 500 tones and 250 successive tones with a relative shift of F0s based on the same Markov transitional matrix. In explicit and implicit learning conditions, neuromagnetic responses to the tone sequence were recorded from fourteen right-handed participants. The temporal profiles of the N1m responses to the tones with higher and lower transitional probabilities were compared. In the explicit learning condition, the N1m responses to tones with higher transitional probability were significantly decreased compared with responses to tones with lower transitional probability in the latter half of the 500-tone sequence. Furthermore, this difference was retained even after the F0s were relatively shifted. In the implicit learning condition, N1m responses to tones with higher transitional probability were significantly decreased only for the 250 tones following the relative shift of F0s. The delayed detection of learning effects across the sound-spectral shift in the implicit condition may imply that learning may progress earlier in explicit learning conditions than in implicit learning conditions. The finding that the learning effects were retained across spectral shifts regardless of the learning modality indicates that relative pitch processing may be an essential ability for humans.

Empirical evidence for musical syntax processing? Computer simulations reveal the contribution of auditory short-term memory

During the last decade, it has been argued that (1) music processing involves syntactic representations similar to those observed in language, and (2) that music and language share similar syntactic-like processes and neural resources. This claim is important for understanding the origin of music and language abilities and, furthermore, it has clinical implications. The Western musical system, however, is rooted in psychoacoustic properties of sound, and this is not the case for linguistic syntax. Accordingly, musical syntax processing could be parsimoniously understood as an emergent property of auditory memory rather than a property of abstract processing similar to linguistic processing. To support this view, we simulated numerous empirical studies that investigated the processing of harmonic structures, using a model based on the accumulation of sensory information in auditory memory. The simulations revealed that most of the musical syntax manipulations used with behavioral and neurophysiological methods as well as with developmental and cross-cultural approaches can be accounted for by the auditory memory model. This led us to question whether current research on musical syntax can really be compared with linguistic processing. Our simulation also raises methodological and theoretical challenges to study musical syntax while disentangling the confounded low-level sensory influences. In order to investigate syntactic abilities in music comparable to language, research should preferentially use musical material with structures that circumvent the tonal effect exerted by psychoacoustic properties of sounds.

Neural correlates of music-syntactic processing in two-year old children

•

We observed neurophysiological correlates of music-syntactic processing in 30-month-olds.

•

This indicates that children of that age process harmonic sequences according to complex syntactic regularities.

•

These representations of music-syntactic regularities must have been acquired before and stored in long-term memory.

•

Similar to syntax processing in language, these processes are highly automatic and do not require attention.

Music is a basic and ubiquitous socio-cognitive domain. However, our understanding of the time course of the development of music perception, particularly regarding implicit knowledge of music-syntactic regularities, remains contradictory and incomplete. Some authors assume that the acquisition of knowledge about these regularities lasts until late childhood, but there is also evidence for the presence of such knowledge in four- and five-year-olds. To explore whether such knowledge is already present in younger children, we tested whether 30-month-olds (N = 62) show neurophysiological responses to music-syntactically irregular harmonies. We observed an early right anterior negativity in response to both irregular in-key and out-of-key chords. The N5, a brain response usually present in older children and adults, was not observed, indicating that processes of harmonic integration (as reflected in the N5) are still in development in this age group. In conclusion, our results indicate that 30-month-olds already have acquired implicit knowledge of complex harmonic music-syntactic regularities and process musical information according to this knowledge.

Melody effects on ERANm elicited by harmonic irregularity in musical syntax

Recent studies have reported that early right anterior negativity (ERAN) and its magnetic counterpart (ERANm) are evoked by harmonic irregularity in Western tonal music; however, those studies did not control for differences of melody. Because melody and harmony have an interdependent relationship and because melody (in this study melody is represented by the highest voice part) in a chord sequence may dominate, there is controversy over whether ERAN (or ERANm) changes arise from melody or harmony differences. To separate the effects of melody differences and harmonic irregularity on ERANm responses, we designed two magnetoencephalography experiments and behavioral test. Participants were presented with three types of chord progression sequences (Expected, Intermediate, and Unexpected) with different harmonic regularities in which melody differences were or were not controlled. In the uncontrolled melody difference experiment, the unexpected chord elicited a significantly largest ERANm, but in the controlled melody difference experiment, the amplitude of the ERANm peak did not differ among the three conditions. However, ERANm peak latency was delayed more than that in the uncontrolled melody difference experiment. The behavioral results show the difference between the two experiments even if harmonic irregularity was discriminated in the uncontrolled melody difference experiment. In conclusion, our analysis reveals that there is a relationship between the effects of harmony and melody on ERANm. Hence, we suggest that a melody difference in a chord progression is largely responsible for the observed changes in ERANm, reaffirming that melody plays an important role in the processing of musical syntax.

Passive sound exposure induces rapid perceptual learning in musicians: event-related potential evidence

Musicians show enhanced auditory processing compared to nonmusicians. However, the neural basis underlying the effects of musical training on rapid plasticity in auditory processing has not been systematically studied. Here, the rapid (one session) learning-related plastic changes in event-related potential (ERP) responses for pitch and duration deviants between passive blocks were compared between musicians and nonmusicians. Passive blocks were interleaved with an active discrimination task. In addition to musicians having faster and stronger overall source activation for deviating sounds, source analysis revealed rapid plastic changes in the left and right temporal and left frontal sources that were present only in musicians. Source activation decreased in these areas even without focused attention. Furthermore, deviant-related ERP responses above the parietal areas decreased after the active task in both musicians and nonmusicians. Taken together, the results indicate enhanced rapid plasticity in sound change discrimination and perceptual learning in musicians when compared with nonmusicians.

Musical expertise modulates early processing of syntactic violations in language

Syntactic violations in speech and music have been shown to elicit an anterior negativity (AN) as early as 100 ms after violation onset and a posterior positivity that peaks at roughly 600 ms (P600/LPC). The language AN is typically reported as left-lateralized (LAN), whereas the music AN is typically reported as right-lateralized (RAN). However, several lines of evidence suggest syntactic processing of language and music rely on overlapping neural systems. The current study tested the hypothesis that syntactic processing of speech and music share neural resources by examining whether musical proficiency modulates ERP indices of linguistic syntactic processing. ERPs were measured in response to syntactic violations in sentences and chord progressions in musicians and non-musicians. Violations in speech were insertion errors in normal and semantically impoverished English sentences. Violations in music were out-of-key chord substitutions from distantly and closely related keys. Phrase-structure violations elicited an AN and P600 in both groups. Harmonic violations elicited an LPC in both groups, blatant harmonic violations also elicited a RAN in musicians only. Cross-domain effects of musical proficiency were similar to previously reported within-domain effects of linguistic proficiency on the distribution of the language AN; syntactic violations in normal English sentences elicited a LAN in musicians and a bilateral AN in non-musicians. The late positivities elicited by violations differed in latency and distribution between domains. These results suggest that initial processing of syntactic violations in language and music relies on shared neural resources in the general population, and that musical expertise results in more specialized cortical organization of syntactic processing in both domains.

Evoked magnetoencephalographic responses to omission of a tone in a musical scale

The musical scale is a basis for melodies and can be a simple melody by itself. The present study investigated magnetoencephalographic (MEG) responses to omissions of one tone out of the C major scale. The tone preceding the omitted "target" tone was either prolonged or repeated. In another series, the tone after the target tone was repeated. In "normal" oddball experiments, the complete C major scale was presented more frequently than an incomplete scale lacking one tone, and in "reverse" oddball experiments, the roles were exchanged. In the normal oddball experiments, omission of any tone produced a response significantly different in amplitude from the standard response in the group of non-musicians, although the responses differed depending on the types of omission. The leading tone (B in the C major scale) was shown to elicit a large response when omitted and also when its presence was emphasized. The Reverse oddball experiments showed that repeated presentation of an incomplete scale lacking one tone temporarily reduced the influence of the complete scale but could not even temporarily replace it working as "standard." In addition, an auxiliary study was done to see possible influence of rhythmic variations.

Musical training modulates encoding of higher-order regularities in the auditory cortex

We investigated the effect of long-term musical training on the time course of development of neuronal representations within the auditory cortex by means of magnetoencephalography. In musicians but not in nonmusicians, pre-attentive encoding of a complex regularity within a tone sequence was evident by a constant increase of the pattern mismatch negativity within < 10 min. The group difference was more pronounced in the left hemisphere, indicating stronger plastic changes in its structures supporting temporal analysis and sound pattern encoding. The results suggest an effect of long-term musical training on short-term auditory learning processes. This has implications not only for cognitive neuroscience in showing how short-term and long-term neuronal plasticity can interact within the auditory cortex, but also for educational and clinical applications of implicit auditory learning where beneficial effects of (musical) experience might be exploited.

Detecting scale violations in absence of mismatch requires music-syntactic analysis: a further look at the early right anterior negativity (ERAN)

The purpose of this study was to determine whether infrequent scale violations in a sequence of in-key notes are detected when the deviants are matched for frequency of occurrence and preceding intervals with the control notes. We further investigated whether the detectability of scale violations is modulated by the presence of melodic context and by the level of musical training. Event related potentials were recorded from 14 musicians and 13 non-musicians. In non-musicians, the out-of-key notes elicited an early right anterior negativity (ERAN), which appeared prominently over right frontal sites only when presented within structured sequences; no effects were found when the out-of-key notes were presented within scrambled sequences. In musicians, the out-of-key notes elicited a similar bilateral ERAN in structured and scrambled sequences. Our findings suggest that scale information is processed at the level of music-syntactic analysis, and that the detection of deviants does not require activation of auditory sensory memory by mismatch effects. Scales are perceived as a broader context, not just as online interval relations. Additional melodic context information appears necessary to support the representation of scale deviants in non-musicians, but not in musically-trained individuals, likely as a consequence of stronger pre-existing representations.

Implicit memory in music and language

Research on music and language in recent decades has focused on their overlapping neurophysiological, perceptual, and cognitive underpinnings, ranging from the mechanism for encoding basic auditory cues to the mechanism for detecting violations in phrase structure. These overlaps have most often been identified in musicians with musical knowledge that was acquired explicitly, through formal training. In this paper, we review independent bodies of work in music and language that suggest an important role for implicitly acquired knowledge, implicit memory, and their associated neural structures in the acquisition of linguistic or musical grammar. These findings motivate potential new work that examines music and language comparatively in the context of the implicit memory system.

Effects of Selective Attention on Syntax Processing in Music and Language

The present study investigated the effects of auditory selective attention on the processing of syntactic information in music and speech using event-related potentials. Spoken sentences or musical chord sequences were either presented in isolation, or simultaneously. When presented simultaneously, participants had to focus their attention either on speech, or on music. Final words of sentences and final harmonies of chord sequences were syntactically either correct or incorrect. Irregular chords elicited an early right anterior negativity (ERAN), whose amplitude was decreased when music was simultaneously presented with speech, compared to when only music was presented. However, the amplitude of the ERAN-like waveform elicited when music was ignored did not differ from the conditions in which participants attended the chord sequences. Irregular sentences elicited an early left anterior negativity (ELAN), regardless of whether speech was presented in isolation, was attended, or was to be ignored. These findings suggest that the neural mechanisms underlying the processing of syntactic structure of music and speech operate partially automatically, and, in the case of music, are influenced by different attentional conditions. Moreover, the ERAN was slightly reduced when irregular sentences were presented, but only when music was ignored. Therefore, these findings provide no clear support for an interaction of neural resources for syntactic processing already at these early stages.

The effect of conditional probability of chord progression on brain response: an MEG study

Background

Recent electrophysiological and neuroimaging studies have explored how and where musical syntax in Western music is processed in the human brain. An inappropriate chord progression elicits an event-related potential (ERP) component called an early right anterior negativity (ERAN) or simply an early anterior negativity (EAN) in an early stage of processing the musical syntax. Though the possible underlying mechanism of the EAN is assumed to be probabilistic learning, the effect of the probability of chord progressions on the EAN response has not been previously explored explicitly.

Methodology/Principal Findings

In the present study, the empirical conditional probabilities in a Western music corpus were employed as an approximation of the frequencies in previous exposure of participants. Three types of chord progression were presented to musicians and non-musicians in order to examine the correlation between the probability of chord progression and the neuromagnetic response using magnetoencephalography (MEG). Chord progressions were found to elicit early responses in a negatively correlating fashion with the conditional probability. Observed EANm (as a magnetic counterpart of the EAN component) responses were consistent with the previously reported EAN responses in terms of latency and location. The effect of conditional probability interacted with the effect of musical training. In addition, the neural response also correlated with the behavioral measures in the non-musicians.

Conclusions/Significance

Our study is the first to reveal the correlation between the probability of chord progression and the corresponding neuromagnetic response. The current results suggest that the physiological response is a reflection of the probabilistic representations of the musical syntax. Moreover, the results indicate that the probabilistic representation is related to the musical training as well as the sensitivity of an individual.

Hearing it again and again: on-line subcortical plasticity in humans

Background

Human brainstem activity is sensitive to local sound statistics, as reflected in an enhanced response in repetitive compared to pseudo-random stimulus conditions [1]. Here we probed the short-term time course of this enhancement using a paradigm that assessed how the local sound statistics (i.e., repetition within a five-note melody) interact with more global statistics (i.e., repetition of the melody).

Methodology/Principal Findings

To test the hypothesis that subcortical repetition enhancement builds over time, we recorded auditory brainstem responses in young adults to a five-note melody containing a repeated note, and monitored how the response changed over the course of 1.5 hrs. By comparing response amplitudes over time, we found a robust time-dependent enhancement to the locally repeating note that was superimposed on a weaker enhancement of the globally repeating pattern.

Conclusions/Significance

We provide the first demonstration of on-line subcortical plasticity in humans. This complements previous findings that experience-dependent subcortical plasticity can occur on a number of time scales, including life-long experiences with music and language, and short-term auditory training. Our results suggest that the incoming stimulus stream is constantly being monitored, even when the stimulus is physically invariant and attention is directed elsewhere, to augment the neural response to the most statistically salient features of the ongoing stimulus stream. These real-time transformations, which may subserve humans' strong disposition for grouping auditory objects, likely reflect a mix of local processes and corticofugal modulation arising from statistical regularities and the influences of expectation. Our results contribute to our understanding of the biological basis of statistical learning and initiate a new investigational approach relating to the time-course of subcortical plasticity. Although the reported time-dependent enhancements are believed to reflect universal neurophysiological processes, future experiments utilizing a larger array of stimuli are needed to establish the generalizability of our findings.

Differences in Electric Brain Responses to Melodies and Chords

The music we usually listen to in everyday life consists of either single melodies or harmonized melodies (i.e., of melodies “accompanied” by chords). However, differences in the neural mechanisms underlying melodic and harmonic processing have remained largely unknown. Using EEG, this study compared effects of music-syntactic processing between chords and melodies. In melody blocks, sequences consisted of five tones, the final tone being either regular or irregular (p = .5). Analogously, in chord blocks, sequences consisted of five chords, the final chord function being either regular or irregular. Melodies were derived from the top voice of chord sequences, allowing a proper comparison between melodic and harmonic processing. Music-syntactic incongruities elicited an early anterior negativity with a latency of approximately 125 msec in both the melody and the chord conditions. This effect was followed in the chord condition, but not in the melody condition, by an additional negative effect that was maximal at approximately 180 msec. Both effects were maximal at frontal electrodes, but the later effect was more broadly distributed over the scalp than the earlier effect. These findings indicate that melodic information (which is also contained in the top voice of chords) is processed earlier and with partly different neural mechanisms than harmonic information of chords.

On the processing of Japanese wh-questions: an ERP study

The processing of Japanese wh-questions was investigated using event-related brain potentials (ERPs). Unlike in English or German, a wh-element in Japanese need not be displaced from its canonical position, but instead needs a corresponding Q(uestion)-particle to indicate its interrogative scope. We tested to see if there were any processing correlates specific to these features of Japanese wh-questions. Both mono-clausal and bi-clausal Japanese wh-questions elicited right-lateralized anterior negativity (RAN) between wh-words and corresponding Q-particles, relative to structurally-equivalent yes/no-question control conditions. These results suggest a reliable neural processing correlate of the dependency between wh-elements and Q-particles in Japanese, similar to effects of (left) anterior negativity between wh-fillers and gaps in English and German, but with a right- rather than left-lateralized distribution. It is suggested that wh-in-situ questions in Japanese are processed by the incremental formation of a long-distance dependency between wh-elements and their Q-particles, resulting in a working memory load for keeping track of scopeless wh-elements.

Decrease in early right alpha band phase synchronization and late gamma band oscillations in processing syntax in music

The present study investigated the neural correlates associated with the processing of music-syntactical irregularities as compared with regular syntactic structures in music. Previous studies reported an early ( approximately 200 ms) right anterior negative component (ERAN) by traditional event-related-potential analysis during music-syntactical irregularities, yet little is known about the underlying oscillatory and synchronization properties of brain responses which are supposed to play a crucial role in general cognition including music perception. First we showed that the ERAN was primarily represented by low frequency (<8 Hz) brain oscillations. Further, we found that music-syntactical irregularities as compared with music-syntactical regularities, were associated with (i) an early decrease in the alpha band (9-10 Hz) phase synchronization between right fronto-central and left temporal brain regions, and (ii) a late ( approximately 500 ms) decrease in gamma band (38-50 Hz) oscillations over fronto-central brain regions. These results indicate a weaker degree of long-range integration when the musical expectancy is violated. In summary, our results reveal neural mechanisms of music-syntactic processing that operate at different levels of cortical integration, ranging from early decrease in long-range alpha phase synchronization to late local gamma oscillations.

Effects of unexpected chords and of performer's expression on brain responses and electrodermal activity

Background

There is lack of neuroscientific studies investigating music processing with naturalistic stimuli, and brain responses to real music are, thus, largely unknown.

Methodology/Principal Findings

This study investigates event-related brain potentials (ERPs), skin conductance responses (SCRs) and heart rate (HR) elicited by unexpected chords of piano sonatas as they were originally arranged by composers, and as they were played by professional pianists. From the musical excerpts played by the pianists (with emotional expression), we also created versions without variations in tempo and loudness (without musical expression) to investigate effects of musical expression on ERPs and SCRs. Compared to expected chords, unexpected chords elicited an early right anterior negativity (ERAN, reflecting music-syntactic processing) and an N5 (reflecting processing of meaning information) in the ERPs, as well as clear changes in the SCRs (reflecting that unexpected chords also elicited emotional responses). The ERAN was not influenced by emotional expression, whereas N5 potentials elicited by chords in general (regardless of their chord function) differed between the expressive and the non-expressive condition.

Conclusions/Significance

These results show that the neural mechanisms of music-syntactic processing operate independently of the emotional qualities of a stimulus, justifying the use of stimuli without emotional expression to investigate the cognitive processing of musical structure. Moreover, the data indicate that musical expression affects the neural mechanisms underlying the processing of musical meaning. Our data are the first to reveal influences of musical performance on ERPs and SCRs, and to show physiological responses to unexpected chords in naturalistic music.

Untangling syntactic and sensory processing: an ERP study of music perception

The present study investigated music-syntactic processing with chord sequences that ended on either regular or irregular chord functions. Sequences were composed such that perceived differences in the cognitive processing between syntactically regular and irregular chords could not be due to the sensory processing of acoustic factors like pitch repetition, pitch commonality (the major component of "sensory dissonance"), or roughness. Three experiments with independent groups of subjects were conducted: a behavioral experiment and two experiments using electroencephalography. Irregular chords elicited an early right anterior negativity (ERAN) in the event-related brain potentials (ERPs) under both task-relevant and task-irrelevant conditions. Behaviorally, participants detected around 75% of the irregular chords, indicating that these chords were only moderately salient. Nevertheless, the irregular chords reliably elicited clear ERP effects. Amateur musicians were slightly more sensitive to musical irregularities than nonmusicians, supporting previous studies demonstrating effects of musical training on music-syntactic processing. The findings indicate that the ERAN is an index of music-syntactic processing and that the ERAN can be elicited even when irregular chords are not detectable based on acoustical factors such as pitch repetition, sensory dissonance, or roughness.