The effect of conditional probability of chord progression on brain response: an MEG study

Statistical Learning of Chord-Transition Regularities in a Novel Equitempered Scale: An MMN Study

In music and language domains, it has been suggested that patterned transitions of sounds can be acquired implicitly through statistical learning. Previous studies have investigated the statistical learning of auditory regularities by recording early neural responses to a sequence of tones presented at high or low transition probabilities. However, it remains unclear whether the statistical learning of musical chord transitions is reflected in endogenous, regularity-dependent components of the event-related potential (ERP). The present study aimed to record the mismatch negativity (MMN) elicited by chord transitions that deviated from newly learned transitional regularities. Chords were generated in a novel 18 equal temperament pitch class scale to avoid interference from the existing tonal representations of the 12 equal temperament pitch class system. Thirty-six adults without professional musical training listened to a sequence of randomly inverted chords in which certain chords were presented with high (standard) or low (deviant) transition probabilities. An irrelevant timbre change detection task was assigned to make them attend to the sequence during the ERP recording. After that, a familiarity test was administered in which the participants were asked to choose the more familiar chord sequence out of two successive sequences. The results showed that deviant transitions elicited the MMN, although the participants could not recognize the standard transition beyond the level of chance. These findings suggest that humans can statistically learn new transitional regularities of chords in a novel musical scale, even though they did not recognize them explicitly. This study provides further evidence that music-syntactic regularities can be acquired implicitly through statistical learning.

On the encoding of natural music in computational models and human brains

This article discusses recent developments and advances in the neuroscience of music to understand the nature of musical emotion. In particular, it highlights how system identification techniques and computational models of music have advanced our understanding of how the human brain processes the textures and structures of music and how the processed information evokes emotions. Musical models relate physical properties of stimuli to internal representations called features, and predictive models relate features to neural or behavioral responses and test their predictions against independent unseen data. The new frameworks do not require orthogonalized stimuli in controlled experiments to establish reproducible knowledge, which has opened up a new wave of naturalistic neuroscience. The current review focuses on how this trend has transformed the domain of the neuroscience of music.

Linking the neural basis of distributional statistical learning with transitional statistical learning: The paradox of attention

Statistical learning, the process of tracking distributional information and discovering embedded patterns, is traditionally regarded as a form of implicit learning. However, recent studies proposed that both implicit (attention-independent) and explicit (attention-dependent) learning systems are involved in statistical learning. To understand the role of attention in statistical learning, the current study investigates the cortical processing of distributional patterns in speech across local and global contexts. We then ask how these cortical responses relate to statistical learning behavior in a word segmentation task. We found Event-Related Potential (ERP) evidence of pre-attentive processing of both the local (mismatching negativity) and global distributional information (late discriminative negativity). However, as speech elements became less frequent and more surprising, some participants showed an involuntary attentional shift, reflected in a P3a response. Individuals who displayed attentive neural tracking of distributional information showed faster learning in a speech statistical learning task. These results suggest that an involuntary attentional shift might play a facilitatory, but not essential, role in statistical learning.

Dissociation of Connectivity for Syntactic Irregularity and Perceptual Ambiguity in Musical Chord Stimuli

Musical syntax has been studied mainly in terms of “syntactic irregularity” in harmonic/melodic sequences. However, “perceptual ambiguity” referring to the uncertainty of judgment/classification of presented stimuli can in addition be involved in our musical stimuli using three different chord sequences. The present study addresses how “syntactic irregularity” and “perceptual ambiguity” on musical syntax are dissociated, in terms of effective connectivity between the bilateral inferior frontal gyrus (IFGs) and superior temporal gyrus (STGs) by linearized time-delayed mutual information (LTDMI). Three conditions were of five-chord sequences with endings of dominant to tonic, dominant to submediant, and dominant to supertonic. The dominant to supertonic is most irregular, compared with the regular dominant to tonic. The dominant to submediant of the less irregular condition is the most ambiguous condition. In the LTDMI results, connectivity from the right to the left IFG (IFG-LTDMI) was enhanced for the most irregular condition, whereas that from the right to the left STG (STG-LTDMI) was enhanced for the most ambiguous condition (p = 0.024 in IFG-LTDMI, p < 0.001 in STG-LTDMI, false discovery rate (FDR) corrected). Correct rate was negatively correlated with STG-LTDMI, further reflecting perceptual ambiguity (p = 0.026). We found for the first time that syntactic irregularity and perceptual ambiguity coexist in chord stimulus testing musical syntax and that the two processes are dissociated in interhemispheric connectivities in the IFG and STG, respectively.

Musical Expertise Is Associated with Improved Neural Statistical Learning in the Auditory Domain

 

It is poorly known whether musical training is associated with improvements in general cognitive abilities, such as statistical learning (SL). In standard SL paradigms, musicians have shown better performances than nonmusicians. However, this advantage could be due to differences in auditory discrimination, in memory or truly in the ability to learn sequence statistics. Unfortunately, these different hypotheses make similar predictions in terms of expected results. To dissociate them, we developed a Bayesian model and recorded electroencephalography (EEG). Our results confirm that musicians perform approximately 15% better than nonmusicians at predicting items in auditory sequences that embed either low or high-order statistics. These higher performances are explained in the model by parameters governing the learning of high-order statistics and the selection stage noise. EEG recordings reveal a neural underpinning of the musician's advantage: the P300 amplitude correlates with the surprise elicited by each item, and so, more strongly for musicians. Finally, early EEG components correlate with the surprise elicited by low-order statistics, as opposed to late EEG components that correlate with the surprise elicited by high-order statistics and this effect is stronger for musicians. Overall, our results demonstrate that musical expertise is associated with improved neural SL in the auditory domain.

SIGNIFICANCE STATEMENT

It is poorly known whether musical training leads to improvements in general cognitive skills. One fundamental cognitive ability, SL, is thought to be enhanced in musicians, but previous studies have reported mixed results. This is because such musician's advantage can embrace very different explanations, such as improvement in auditory discrimination or in memory. To solve this problem, we developed a Bayesian model and recorded EEG to dissociate these explanations. Our results reveal that musical expertise is truly associated with an improved ability to learn sequence statistics, especially high-order statistics. This advantage is reflected in the electroencephalographic recordings, where the P300 amplitude is more sensitive to surprising items in musicians than in nonmusicians.

Auditory but Not Audiovisual Cues Lead to Higher Neural Sensitivity to the Statistical Regularities of an Unfamiliar Musical Style

It is still a matter of debate whether visual aids improve learning of music. In a multisession study, we investigated the neural signatures of novel music sequence learning with or without aids (auditory-only: AO, audiovisual: AV). During three training sessions on three separate days, participants (nonmusicians) reproduced (note by note on a keyboard) melodic sequences generated by an artificial musical grammar. The AV group (n = 20) had each note color-coded on screen, whereas the AO group (n = 20) had no color indication. We evaluated learning of the statistical regularities of the novel music grammar before and after training by presenting melodies ending on correct or incorrect notes and by asking participants to judge the correctness and surprisal of the final note, while EEG was recorded. We found that participants successfully learned the new grammar. Although the AV group, as compared to the AO group, reproduced longer sequences during training, there was no significant difference in learning between groups. At the neural level, after training, the AO group showed a larger N100 response to low-probability compared with high-probability notes, suggesting an increased neural sensitivity to statistical properties of the grammar; this effect was not observed in the AV group. Our findings indicate that visual aids might improve sequence reproduction while not necessarily promoting better learning, indicating a potential dissociation between sequence reproduction and learning. We suggest that the difficulty induced by auditory-only input during music training might enhance cognitive engagement, thereby improving neural sensitivity to the underlying statistical properties of the learned material.

Brain networks underlying aesthetic appreciation as modulated by interaction of the spectral and temporal organisations of music

Music is organised both spectrally and temporally, determining musical structures such as musical scale, harmony, and sequential rules in chord progressions. A number of human neuroimaging studies investigated neural processes associated with emotional responses to music investigating the influence of musical valence (pleasantness/unpleasantness) comparing the response to music and unpleasantly manipulated counterparts where harmony and sequential rules were varied. Interactions between the previously applied alterations to harmony and sequential rules of the music in terms of emotional experience and corresponding neural activities have not been systematically studied although such interactions are at the core of how music affects the listener. The current study investigates the interaction between such alterations in harmony and sequential rules by using data sets from two functional magnetic resonance imaging (fMRI) experiments. While replicating the previous findings, we found a significant interaction between the spectral and temporal alterations in the fronto-limbic system, including the ventromedial prefrontal cortex (vmPFC), nucleus accumbens, caudate nucleus, and putamen. We further revealed that the functional connectivity between the vmPFC and the right inferior frontal gyrus (IFG) was reduced when listening to excerpts with alterations in both domains compared to the original music. As it has been suggested that the vmPFC operates as a pivotal point that mediates between the limbic system and the frontal cortex in reward-related processing, we propose that this fronto-limbic interaction might be related to the involvement of cognitive processes in the emotional appreciation of music.

Change in left inferior frontal connectivity with less unexpected harmonic cadence by musical expertise

In terms of harmonic expectancy, compared to an expected dominant-to-tonic and an unexpected dominant-to-supertonic, a dominant-to-submediant is a less unexpected cadence, the perception of which may depend on the subject’s musical expertise. The present study investigated how aforementioned 3 different cadences are processed in the networks of bilateral inferior frontal gyri (IFGs) and superior temporal gyri (STGs) with magnetoencephalography. We compared the correct rate and brain connectivity in 9 music-majors (mean age, 23.5 ± 3.4 years; musical training period, 18.7 ± 4.0 years) and 10 non-music-majors (mean age, 25.2 ± 2.6 years; musical training period, 4.2 ± 1.5 years). For the brain connectivity, we computed the summation of partial directed coherence (PDC) values for inflows/outflows to/from each area (sPDC_i/sPDC_o) in bilateral IFGs and STGs. In the behavioral responses, music-majors were better than non-music-majors for all 3 cadences (p < 0.05). However, sPDC_i/sPDC_o was prominent only for the dominant-to-submediant in the left IFG. The sPDC_i was more strongly enhanced in music-majors than in non-music-majors (p = 0.002, Bonferroni corrected), while the sPDC_o was vice versa (p = 0.005, Bonferroni corrected). Our data show that music-majors, with higher musical expertise, are better in identifying a less unexpected cadence than non-music-majors, with connectivity changes centered on the left IFG.

Tonality Tunes the Statistical Characteristics in Music: Computational Approaches on Statistical Learning

Statistical learning is a learning mechanism based on transition probability in sequences such as music and language. Recent computational and neurophysiological studies suggest that the statistical learning contributes to production, action, and musical creativity as well as prediction and perception. The present study investigated how statistical structure interacts with tonalities in music based on various-order statistical models. To verify this in all 24 major and minor keys, the transition probabilities of the sequences containing the highest pitches in Bach's Well-Tempered Clavier, which is a collection of two series (No. 1 and No. 2) of preludes and fugues in all of the 24 major and minor keys, were calculated based on nth-order Markov models. The transition probabilities of each sequence were compared among tonalities (major and minor), two series (No. 1 and No. 2), and music types (prelude and fugue). The differences in statistical characteristics between major and minor keys were detected in lower- but not higher-order models. The results also showed that statistical knowledge in music might be modulated by tonalities and composition periods. Furthermore, the principal component analysis detected the shared components of related keys, suggesting that the tonalities modulate statistical characteristics in music. The present study may suggest that there are at least two types of statistical knowledge in music that are interdependent on and independent of tonality, respectively.

Intracranial Recordings and Computational Modeling of Music Reveal the Time Course of Prediction Error Signaling in Frontal and Temporal Cortices

Prediction is held to be a fundamental process underpinning perception, action, and cognition. To examine the time course of prediction error signaling, we recorded intracranial EEG activity from nine presurgical epileptic patients while they listened to melodies whose information theoretical predictability had been characterized using a computational model. We examined oscillatory activity in the superior temporal gyrus (STG), the middle temporal gyrus (MTG), and the pars orbitalis of the inferior frontal gyrus, lateral cortical areas previously implicated in auditory predictive processing. We also examined activity in anterior cingulate gyrus (ACG), insula, and amygdala to determine whether signatures of prediction error signaling may also be observable in these subcortical areas. Our results demonstrate that the information content (a measure of unexpectedness) of musical notes modulates the amplitude of low-frequency oscillatory activity (theta to beta power) in bilateral STG and right MTG from within 100 and 200 msec of note onset, respectively. Our results also show this cortical activity to be accompanied by low-frequency oscillatory modulation in ACG and insula-areas previously associated with mediating physiological arousal. Finally, we showed that modulation of low-frequency activity is followed by that of high-frequency (gamma) power from approximately 200 msec in the STG, between 300 and 400 msec in the left insula, and between 400 and 500 msec in the ACG. We discuss these results with respect to models of neural processing that emphasize gamma activity as an index of prediction error signaling and highlight the usefulness of musical stimuli in revealing the wide-reaching neural consequences of predictive processing.

Neurophysiological Markers of Statistical Learning in Music and Language: Hierarchy, Entropy, and Uncertainty

Statistical learning (SL) is a method of learning based on the transitional probabilities embedded in sequential phenomena such as music and language. It has been considered an implicit and domain-general mechanism that is innate in the human brain and that functions independently of intention to learn and awareness of what has been learned. SL is an interdisciplinary notion that incorporates information technology, artificial intelligence, musicology, and linguistics, as well as psychology and neuroscience. A body of recent study has suggested that SL can be reflected in neurophysiological responses based on the framework of information theory. This paper reviews a range of work on SL in adults and children that suggests overlapping and independent neural correlations in music and language, and that indicates disability of SL. Furthermore, this article discusses the relationships between the order of transitional probabilities (TPs) (i.e., hierarchy of local statistics) and entropy (i.e., global statistics) regarding SL strategies in human's brains; claims importance of information-theoretical approaches to understand domain-general, higher-order, and global SL covering both real-world music and language; and proposes promising approaches for the application of therapy and pedagogy from various perspectives of psychology, neuroscience, computational studies, musicology, and linguistics.

Single, but not dual, attention facilitates statistical learning of two concurrent auditory sequences

When we are exposed to a novel stimulus sequence, we can learn the sequence by extracting a statistical structure that is potentially embedded in the sequence. This mechanism is called statistical learning, and is considered a fundamental and domain-general process that is innate in humans. In the real-world environment, humans are inevitably exposed to auditory sequences that often overlap with one another, such as speech sound streams from multiple speakers or entangled melody lines generated by multiple instruments. The present study investigated how single and dual attention modulates brain activity, reflecting statistical learning when two auditory sequences were presented simultaneously. The results demonstrated that the effect of statistical learning had more pronounced neural activity when listeners paid attention to only one sequence and ignored the other, rather than paying attention to both sequences. Biased attention may thus be an essential strategy when learners are exposed to multiple information streams.

The effects of supervised learning on event-related potential correlates of music-syntactic processing

Humans process music even without conscious effort according to implicit knowledge about syntactic regularities. Whether such automatic and implicit processing is modulated by veridical knowledge has remained unknown in previous neurophysiological studies. This study investigates this issue by testing whether the acquisition of veridical knowledge of a music-syntactic irregularity (acquired through supervised learning) modulates early, partly automatic, music-syntactic processes (as reflected in the early right anterior negativity, ERAN), and/or late controlled processes (as reflected in the late positive component, LPC). Excerpts of piano sonatas with syntactically regular and less regular chords were presented repeatedly (10 times) to non-musicians and amateur musicians. Participants were informed by a cue as to whether the following excerpt contained a regular or less regular chord. Results showed that the repeated exposure to several presentations of regular and less regular excerpts did not influence the ERAN elicited by less regular chords. By contrast, amplitudes of the LPC (as well as of the P3a evoked by less regular chords) decreased systematically across learning trials. These results reveal that late controlled, but not early (partly automatic), neural mechanisms of music-syntactic processing are modulated by repeated exposure to a musical piece. This article is part of a Special Issue entitled SI: Prediction and Attention.

Neural correlates of intentional switching from ternary to binary meter in a musical hemiola pattern

Musical rhythms are often perceived and interpreted within a metrical framework that integrates timing information hierarchically based on interval ratios. Endogenous timing processes facilitate this metrical integration and allow us using the sensory context for predicting when an expected sensory event will happen (“predictive timing”). Previously, we showed that listening to metronomes and subjectively imagining the two different meters of march and waltz modulated the resulting auditory evoked responses in the temporal lobe and motor-related brain areas such as the motor cortex, basal ganglia, and cerebellum. Here we further explored the intentional transitions between the two metrical contexts, known as hemiola in the Western classical music dating back to the sixteenth century. We examined MEG from 12 musicians while they repeatedly listened to a sequence of 12 unaccented clicks with an interval of 390 ms, and tapped to them with the right hand according to a 3 + 3 + 2 + 2 + 2 hemiola accent pattern. While participants listened to the same metronome sequence and imagined the accents, their pattern of brain responses significantly changed just before the “pivot” point of metric transition from ternary to binary meter. Until 100 ms before the pivot point, brain activities were more similar to those in the simple ternary meter than those in the simple binary meter, but the pattern was reversed afterwards. A similar transition was also observed at the downbeat after the pivot. Brain areas related to the metric transition were identified from source reconstruction of the MEG using a beamformer and included auditory cortices, sensorimotor and premotor cortices, cerebellum, inferior/middle frontal gyrus, parahippocampal gyrus, inferior parietal lobule, cingulate cortex, and precuneus. The results strongly support that predictive timing processes related to auditory-motor, fronto-parietal, and medial limbic systems underlie metrical representation and its transitions.

Predictive uncertainty in auditory sequence processing

Previous studies of auditory expectation have focused on the expectedness perceived by listeners retrospectively in response to events. In contrast, this research examines predictive uncertainty—a property of listeners' prospective state of expectation prior to the onset of an event. We examine the information-theoretic concept of Shannon entropy as a model of predictive uncertainty in music cognition. This is motivated by the Statistical Learning Hypothesis, which proposes that schematic expectations reflect probabilistic relationships between sensory events learned implicitly through exposure. Using probability estimates from an unsupervised, variable-order Markov model, 12 melodic contexts high in entropy and 12 melodic contexts low in entropy were selected from two musical repertoires differing in structural complexity (simple and complex). Musicians and non-musicians listened to the stimuli and provided explicit judgments of perceived uncertainty (explicit uncertainty). We also examined an indirect measure of uncertainty computed as the entropy of expectedness distributions obtained using a classical probe-tone paradigm where listeners rated the perceived expectedness of the final note in a melodic sequence (inferred uncertainty). Finally, we simulate listeners' perception of expectedness and uncertainty using computational models of auditory expectation. A detailed model comparison indicates which model parameters maximize fit to the data and how they compare to existing models in the literature. The results show that listeners experience greater uncertainty in high-entropy musical contexts than low-entropy contexts. This effect is particularly apparent for inferred uncertainty and is stronger in musicians than non-musicians. Consistent with the Statistical Learning Hypothesis, the results suggest that increased domain-relevant training is associated with an increasingly accurate cognitive model of probabilistic structure in music.

Statistical learning of serial visual transitions by neurons in monkey inferotemporal cortex

If monkeys repeatedly, over the course of weeks, view displays in which two images appear in fixed sequence, then neurons of inferotemporal cortex (ITC) come to exhibit prediction suppression. The response to the trailing image is weaker if it follows the leading image with which it was paired during training than if it follows some other leading image. Prediction suppression is a plausible neural mechanism for statistical learning of visual transitions such as has been demonstrated in behavioral studies of human infants and adults. However, in the human studies, subjects are exposed to continuous sequences in which the same image can be both predicted and predicting and statistical dependency can exist between nonadjacent items. The aim of the present study was to investigate whether prediction suppression in ITC develops under such circumstances. To resolve this issue, we exposed monkeys repeatedly to triplets of images presented in fixed order. The results indicate that prediction suppression can be induced by training not only with pairs of images but also with longer sequences.

Neural correlates of music-syntactic processing in two-year old children

•

We observed neurophysiological correlates of music-syntactic processing in 30-month-olds.

•

This indicates that children of that age process harmonic sequences according to complex syntactic regularities.

•

These representations of music-syntactic regularities must have been acquired before and stored in long-term memory.

•

Similar to syntax processing in language, these processes are highly automatic and do not require attention.

Music is a basic and ubiquitous socio-cognitive domain. However, our understanding of the time course of the development of music perception, particularly regarding implicit knowledge of music-syntactic regularities, remains contradictory and incomplete. Some authors assume that the acquisition of knowledge about these regularities lasts until late childhood, but there is also evidence for the presence of such knowledge in four- and five-year-olds. To explore whether such knowledge is already present in younger children, we tested whether 30-month-olds (N = 62) show neurophysiological responses to music-syntactically irregular harmonies. We observed an early right anterior negativity in response to both irregular in-key and out-of-key chords. The N5, a brain response usually present in older children and adults, was not observed, indicating that processes of harmonic integration (as reflected in the N5) are still in development in this age group. In conclusion, our results indicate that 30-month-olds already have acquired implicit knowledge of complex harmonic music-syntactic regularities and process musical information according to this knowledge.

Melody effects on ERANm elicited by harmonic irregularity in musical syntax

Recent studies have reported that early right anterior negativity (ERAN) and its magnetic counterpart (ERANm) are evoked by harmonic irregularity in Western tonal music; however, those studies did not control for differences of melody. Because melody and harmony have an interdependent relationship and because melody (in this study melody is represented by the highest voice part) in a chord sequence may dominate, there is controversy over whether ERAN (or ERANm) changes arise from melody or harmony differences. To separate the effects of melody differences and harmonic irregularity on ERANm responses, we designed two magnetoencephalography experiments and behavioral test. Participants were presented with three types of chord progression sequences (Expected, Intermediate, and Unexpected) with different harmonic regularities in which melody differences were or were not controlled. In the uncontrolled melody difference experiment, the unexpected chord elicited a significantly largest ERANm, but in the controlled melody difference experiment, the amplitude of the ERANm peak did not differ among the three conditions. However, ERANm peak latency was delayed more than that in the uncontrolled melody difference experiment. The behavioral results show the difference between the two experiments even if harmonic irregularity was discriminated in the uncontrolled melody difference experiment. In conclusion, our analysis reveals that there is a relationship between the effects of harmony and melody on ERANm. Hence, we suggest that a melody difference in a chord progression is largely responsible for the observed changes in ERANm, reaffirming that melody plays an important role in the processing of musical syntax.

Co-localizing linguistic and musical syntax with intracranial EEG

Despite general agreement on shared syntactic resources in music and language, the neuroanatomical underpinnings of this overlap remain largely unexplored. While previous studies mainly considered frontal areas as supramodal grammar processors, the domain-general syntactic role of temporal areas has been so far neglected. Here we capitalized on the excellent spatial and temporal resolution of subdural EEG recordings to co-localize low-level syntactic processes in music and language in the temporal lobe in a within-subject design. We used Brain Surface Current Density mapping to localize and compare neural generators of the early negativities evoked by violations of phrase structure grammar in both music and spoken language. The results show that the processing of syntactic violations relies in both domains on bilateral temporo-fronto-parietal neural networks. We found considerable overlap of these networks in the superior temporal lobe, but also differences in the hemispheric timing and relative weighting of their fronto-temporal constituents. While alluding to the dissimilarity in how shared neural resources may be configured depending on the musical or linguistic nature of the perceived stimulus, the combined data lend support for a co-localization of early musical and linguistic syntax processing in the temporal lobe.

Impaired learning of event frequencies in tone deafness

Musical knowledge is ubiquitous, effortless, and implicitly acquired all over the world via exposure to musical materials in one's culture. In contrast, one group of individuals who show insensitivity to music, specifically the inability to discriminate pitches and melodies, is the tone-deaf. In this study, we asked whether difficulties in pitch and melody discrimination among the tone-deaf could be related to learning difficulties, and, if so, what processes of learning might be affected in the tone-deaf. We investigated the learning of frequency information in a new musical system in tone-deaf individuals and matched controls. Results showed significantly impaired learning abilities in frequency matching in the tone-deaf. This impairment was positively correlated with the severity of tone deafness as assessed by the Montreal Battery for Evaluation of Amusia. Taken together, the results suggest that tone deafness is characterized by an impaired ability to acquire frequency information from pitched materials in the sound environment.