Differential Processing of Consonance and Dissonance within the Human Superior Temporal Gyrus

Resting-state brain connectivity correlates of musical sophistication

Introduction

A growing body of research has investigated how performing arts training, and more specifically, music training, impacts the brain. Recent meta-analytic work has identified multiple brain areas where activity varies as a function of levels of musical expertise gained through music training. However, research has also shown that musical sophistication may be high even without music training. Thus, we aim to extend previous work by investigating whether the functional connectivity of these areas relates to interindividual differences in musical sophistication, and to characterize differences in connectivity attributed to performing arts training.

Methods

We analyzed resting-state functional magnetic resonance imaging from n = 74 participants, of whom 37 received performing arts training, that is, including a musical instrument, singing, and/or acting, at university level. We used a validated, continuous measure of musical sophistication to further characterize our sample. Following standard pre-processing, fifteen brain areas were identified a priori based on meta-analytic work and used as seeds in separate seed-to-voxel analyses to examine the effect of musical sophistication across the sample, and between-group analyses to examine the effects of performing arts training.

Results

Connectivity of bilateral superior temporal gyrus, bilateral precentral gyrus and cerebellum, and bilateral putamen, left insula, and left thalamus varied with different aspects of musical sophistication. By including these measures of these aspects as covariates in post hoc analyses, we found that connectivity of the right superior temporal gyrus and left precentral gyrus relate to effects of performing arts training beyond effects of individual musical sophistication.

Discussion

Our results highlight the potential role of sensory areas in active engagement with music, the potential role of motor areas in emotion processing, and the potential role of connectivity between putamen and lingual gyrus in general musical sophistication.

Consonance and dissonance perception. A critical review of the historical sources, multidisciplinary findings, and main hypotheses

Revealed more than two millennia ago by Pythagoras, consonance and dissonance (C/D) are foundational concepts in music theory, perception, and aesthetics. The search for the biological, acoustical, and cultural factors that affect C/D perception has resulted in descriptive accounts inspired by arithmetic, musicological, psychoacoustical or neurobiological frameworks without reaching a consensus. Here, we review the key historical sources and modern multidisciplinary findings on C/D and integrate them into three main hypotheses: the vocal similarity hypothesis (VSH), the psychocultural hypothesis (PH), and the sensorimotor hypothesis (SH). By illustrating the hypotheses-related findings, we highlight their major conceptual, methodological, and terminological shortcomings. Trying to provide a unitary framework for C/D understanding, we put together multidisciplinary research on human and animal vocalizations, which converges to suggest that auditory roughness is associated with distress/danger and, therefore, elicits defensive behavioral reactions and neural responses that indicate aversion. We therefore stress the primacy of vocality and roughness as key factors in the explanation of C/D phenomenon, and we explore the (neuro)biological underpinnings of the attraction-aversion mechanisms that are triggered by C/D stimuli. Based on the reviewed evidence, while the aversive nature of dissonance appears as solidly rooted in the multidisciplinary findings, the attractive nature of consonance remains a somewhat speculative claim that needs further investigation. Finally, we outline future directions for empirical research in C/D, especially regarding cross-modal and cross-cultural approaches.

Speech-related auditory salience detection in the posterior superior temporal region

Processing auditory human speech requires both detection (early and transient) and analysis (sustained). We analyzed high gamma (70-110 Hz) activity of intracranial electroencephalography waveforms acquired during an auditory task that paired forward speech, reverse speech, and signal correlated noise. We identified widespread superior temporal sites with sustained activity responding only to forward and reverse speech regardless of paired order. More localized superior temporal auditory onset sites responded to all stimulus types when presented first in a pair and responded in recurrent fashion to the second paired stimulus in select conditions even in the absence of interstimulus silence; a novel finding. Auditory onset activity to a second paired sound recurred according to relative salience, with evidence of partial suppression during linguistic processing. We propose that temporal lobe auditory onset sites facilitate a salience detector function with hysteresis of 200 ms and are influenced by cortico-cortical feedback loops involving linguistic processing and articulation.

The Rapid Emergence of Musical Pitch Structure in Human Cortex

In tonal music, continuous acoustic waveforms are mapped onto discrete, hierarchically arranged, internal representations of pitch. To examine the neural dynamics underlying this transformation, we presented male and female human listeners with tones embedded within a Western tonal context while recording their cortical activity using magnetoencephalography. Machine learning classifiers were then trained to decode different tones from their underlying neural activation patterns at each peristimulus time sample, providing a dynamic measure of their dissimilarity in cortex. Comparing the time-varying dissimilarity between tones with the predictions of acoustic and perceptual models, we observed a temporal evolution in the brain's representational structure. Whereas initial dissimilarities mirrored their fundamental-frequency separation, dissimilarities beyond 200 ms reflected the perceptual status of each tone within the tonal hierarchy of Western music. These effects occurred regardless of stimulus regularities within the context or whether listeners were engaged in a task requiring explicit pitch analysis. Lastly, patterns of cortical activity that discriminated between tones became increasingly stable in time as the information coded by those patterns transitioned from low-to-high level properties. Current results reveal the dynamics with which the complex perceptual structure of Western tonal music emerges in cortex at the timescale of an individual tone.SIGNIFICANCE STATEMENT Little is understood about how the brain transforms an acoustic waveform into the complex perceptual structure of musical pitch. Applying neural decoding techniques to the cortical activity of human subjects engaged in music listening, we measured the dynamics of information processing in the brain on a moment-to-moment basis as subjects heard each tone. In the first 200 ms after onset, transient patterns of neural activity coded the fundamental frequency of tones. Subsequently, a period emerged during which more temporally stable activation patterns coded the perceptual status of each tone within the "tonal hierarchy" of Western music. Our results provide a crucial link between the complex perceptual structure of tonal music and the underlying neural dynamics from which it emerges.

Dynamic intersubject neural synchronization reflects affective responses to sad music

Psychological theories of emotion often highlight the dynamic quality of the affective experience, yet neuroimaging studies of affect have traditionally relied on static stimuli that lack ecological validity. Consequently, the brain regions that represent emotions and feelings as they unfold remain unclear. Recently, dynamic, model-free analytical techniques have been employed with naturalistic stimuli to better capture time-varying patterns of activity in the brain; yet, few studies have focused on relating these patterns to changes in subjective feelings. Here, we address this gap, using intersubject correlation and phase synchronization to assess how stimulus-driven changes in brain activity and connectivity are related to two aspects of emotional experience: emotional intensity and enjoyment. During fMRI scanning, healthy volunteers listened to a full-length piece of music selected to induce sadness. After scanning, participants listened to the piece twice while simultaneously rating the intensity of felt sadness or felt enjoyment. Activity in the auditory cortex, insula, and inferior frontal gyrus was significantly synchronized across participants. Synchronization in auditory, visual, and prefrontal regions was significantly greater in participants with higher measures of a subscale of trait empathy related to feeling emotions in response to music. When assessed dynamically, continuous enjoyment ratings positively predicted a moment-to-moment measure of intersubject synchronization in auditory, default mode, and striatal networks, as well as the orbitofrontal cortex, whereas sadness predicted intersubject synchronization in limbic and striatal networks. The results suggest that stimulus-driven patterns of neural communication in emotional processing and high-level cortical regions carry meaningful information with regards to our feeling in response to a naturalistic stimulus.

The pleasantness of sensory dissonance is mediated by musical style and expertise

Western musical styles use a large variety of chords and vertical sonorities. Based on objective acoustical properties, chords can be situated on a dissonant-consonant continuum. While this might to some extent converge with the unpleasant-pleasant continuum, subjective liking might diverge for various chord forms from music across different styles. Our study aimed to investigate how well appraisals of the roughness and pleasantness dimensions of isolated chords taken from real-world music are predicted by Parncutt’s established model of sensory dissonance. Furthermore, we related these subjective ratings to style of origin and acoustical features of the chords as well as musical sophistication of the raters. Ratings were obtained for chords deemed representative of the harmonic language of three different musical styles (classical, jazz and avant-garde music), plus randomly generated chords. Results indicate that pleasantness and roughness ratings were, on average, mirror opposites; however, their relative distribution differed greatly across styles, reflecting different underlying aesthetic ideals. Parncutt’s model only weakly predicted ratings for all but Classical chords, suggesting that listeners’ appraisal of the dissonance and pleasantness of chords bears not only on stimulus-side but also on listener-side factors. Indeed, we found that levels of musical sophistication negatively predicted listeners’ tendency to rate the consonance and pleasantness of any one chord as coupled measures, suggesting that musical education and expertise may serve to individuate how these musical dimensions are apprehended.

Spectral Imprints of Working Memory for Everyday Associations in the Frontoparietal Network

How does the human brain rapidly process incoming information in working memory? In growing divergence from a single-region focus on the prefrontal cortex (PFC), recent work argues for emphasis on how distributed neural networks are rapidly coordinated in support of this central neurocognitive function. Previously, we showed that working memory for everyday “what,” “where,” and “when” associations depends on multiplexed oscillatory systems, in which signals of different frequencies simultaneously link the PFC to parieto-occipital and medial temporal regions, pointing to a complex web of sub-second, bidirectional interactions. Here, we used direct brain recordings to delineate the frontoparietal oscillatory correlates of working memory with high spatiotemporal precision. Seven intracranial patients with electrodes simultaneously localized to prefrontal and parietal cortices performed a visuospatial working memory task that operationalizes the types of identity and spatiotemporal information we encounter every day. First, task-induced oscillations in the same delta-theta (2–7 Hz) and alpha-beta (9–24 Hz) frequency ranges previously identified using scalp electroencephalography (EEG) carried information about the contents of working memory. Second, maintenance was linked to directional connectivity from the parietal cortex to the PFC. However, presentation of the test prompt to cue identity, spatial, or temporal information changed delta-theta coordination from a unidirectional, parietal-led system to a bidirectional, frontoparietal system. Third, the processing of spatiotemporal information was more bidirectional in the delta-theta range than was the processing of identity information, where alpha-beta connectivity did not exhibit sensitivity to the contents of working memory. These findings implicate a bidirectional delta-theta mechanism for frontoparietal control over the contents of working memory.

Early neural responses underlie advantages for consonance over dissonance

Consonant musical intervals tend to be more readily processed than dissonant intervals. In the present study, we explore the neural basis for this difference by registering how the brain responds after changes in consonance and dissonance, and how formal musical training modulates these responses. Event-related brain potentials (ERPs) were registered while participants were presented with sequences of consonant intervals interrupted by a dissonant interval, or sequences of dissonant intervals interrupted by a consonant interval. Participants were musicians and non-musicians. Our results show that brain responses triggered by changes in a consonant context differ from those triggered in a dissonant context. Changes in a sequence of consonant intervals are rapidly processed independently of musical expertise, as revealed by a change-related mismatch negativity (MMN, a component of the ERPs triggered by an odd stimulus in a sequence of stimuli) elicited in both musicians and non-musicians. In contrast, changes in a sequence of dissonant intervals elicited a late MMN only in participants with prolonged musical training. These different neural responses might form the basis for the processing advantages observed for consonance over dissonance and provide information about how formal musical training modulates them.

•

We registered ERPs after deviant intervals in consonant and dissonant sequences.

•

Violations of consonant sequences are detected independently of musical expertise.

•

Musical training modulates responses to violations of dissonant sequences.

•

These neural responses might form the basis for a processing advantage of consonance.

Computational Approach to Musical Consonance and Dissonance

In sixth century BC, Pythagoras discovered the mathematical foundation of musical consonance and dissonance. When auditory frequencies in small-integer ratios are combined, the result is a harmonious perception. In contrast, most frequency combinations result in audible, off-centered by-products labeled “beating” or “roughness;” these are reported by most listeners to sound dissonant. In this paper, we consider second-order beats, a kind of beating recognized as a product of neural processing, and demonstrate that the data-driven approach of Recurrence Quantification Analysis (RQA) allows for the reconstruction of the order in which interval ratios are ranked in music theory and harmony. We take advantage of computer-generated sounds containing all intervals over the span of an octave. To visualize second-order beats, we use a glissando from the unison to the octave. This procedure produces a profile of recurrence values that correspond to subsequent epochs along the original signal. We find that the higher recurrence peaks exactly match the epochs corresponding to just intonation frequency ratios. This result indicates a link between consonance and the dynamical features of the signal. Our findings integrate a new element into the existing theoretical models of consonance, thus providing a computational account of consonance in terms of dynamical systems theory. Finally, as it considers general features of acoustic signals, the present approach demonstrates a universal aspect of consonance and dissonance perception and provides a simple mathematical tool that could serve as a common framework for further neuro-psychological and music theory research.

Decoding the dynamic representation of musical pitch from human brain activity

In music, the perception of pitch is governed largely by its tonal function given the preceding harmonic structure of the music. While behavioral research has advanced our understanding of the perceptual representation of musical pitch, relatively little is known about its representational structure in the brain. Using Magnetoencephalography (MEG), we recorded evoked neural responses to different tones presented within a tonal context. Multivariate Pattern Analysis (MVPA) was applied to “decode” the stimulus that listeners heard based on the underlying neural activity. We then characterized the structure of the brain’s representation using decoding accuracy as a proxy for representational distance, and compared this structure to several well established perceptual and acoustic models. The observed neural representation was best accounted for by a model based on the Standard Tonal Hierarchy, whereby differences in the neural encoding of musical pitches correspond to their differences in perceived stability. By confirming that perceptual differences honor those in the underlying neuronal population coding, our results provide a crucial link in understanding the cognitive foundations of musical pitch across psychological and neural domains.

Auditory enhancement of visual memory encoding is driven by emotional content of the auditory material and mediated by superior frontal cortex

BACKGROUND

The aim of the present study was to investigate how auditory background interacts with learning and memory. Both facilitatory (e.g., "Mozart effect") and interfering effects of background have been reported, depending on the type of auditory stimulation and of concurrent cognitive tasks.

METHOD

Here we recorded event related potentials (ERPs) during face encoding followed by an old/new memory test to investigate the effect of listening to classical music (Čajkovskij, dramatic), environmental sounds (rain) or silence on learning. Participants were 15 healthy non-musician university students. Almost 400 (previously unknown) faces of women and men of various age were presented.

RESULTS

Listening to music during study led to a better encoding of faces as indexed by an increased Anterior Negativity. The FN400 response recorded during the memory test showed a gradient in its amplitude reflecting face familiarity. FN400 was larger to new than old faces, and to faces studied during rain sound listening and silence than music listening.

CONCLUSION

The results indicate that listening to music enhances memory recollection of faces by merging with visual information. A swLORETA analysis showed the main involvement of Superior Temporal Gyrus (STG) and medial frontal gyrus in the integration of audio-visual information.