The perception of harmonic triads: an fMRI study

Dissonant music engages early visual processing

The neuroscientific examination of music processing in audio-visual contexts offers a valuable framework to assess how auditory information influences the emotional encoding of visual information. Using fMRI during naturalistic film viewing, we investigated the neural mechanisms underlying the effect of music on valence inferences during mental state attribution. Thirty-eight participants watched the same short-film accompanied by systematically controlled consonant or dissonant music. Subjects were instructed to think about the main character's intentions. The results revealed that increasing levels of dissonance led to more negatively valenced inferences, displaying the profound emotional impact of musical dissonance. Crucially, at the neuroscientific level and despite music being the sole manipulation, dissonance evoked the response of the primary visual cortex (V1). Functional/effective connectivity analysis showed a stronger coupling between the auditory ventral stream (AVS) and V1 in response to tonal dissonance and demonstrated the modulation of early visual processing via top-down feedback inputs from the AVS to V1. These V1 signal changes indicate the influence of high-level contextual representations associated with tonal dissonance on early visual cortices, serving to facilitate the emotional interpretation of visual information. Our results highlight the significance of employing systematically controlled music, which can isolate emotional valence from the arousal dimension, to elucidate the brain's sound-to-meaning interface and its distributive crossmodal effects on early visual encoding during naturalistic film viewing.

Prediction errors arising from switches between major and minor modes in music: An fMRI study

The major and minor modes in Western music have positive and negative connotations, respectively. The present fMRI study examined listeners' neural responses to switches between major and minor modes. We manipulated the final chords of J. S. Bach's keyboard pieces so that each major-mode passage ended with either the major (Major-Major) or minor (Major-Minor) tonic chord, and each minor-mode passage ended with either the minor (Minor-Minor) or major (Minor-Major) tonic chord. If the final major and minor chords have positive and negative reward values respectively, the Major-Minor and Minor-Major stimuli would cause negative and positive reward prediction errors (RPEs) respectively in a listener's brain. We found that activity in a frontoparietal network was significantly higher for Major-Minor than for Major-Major. Based on previous research, these results support the idea that a major-to-minor switch causes negative RPE. The contrast of Minor-Major minus Minor-Minor yielded activation in the ventral insula and visual cortex, speaking against the idea that a minor-to-major switch causes positive RPE. We discuss our results in relation to executive functions and the emotional connotations of major versus minor modes.

Hierarchical control as a shared neurocognitive mechanism for language and music

Although comparative research has made substantial progress in clarifying the relationship between language and music as neurocognitive systems from both a theoretical and empirical perspective, there is still no consensus about which mechanisms, if any, are shared and how they bring about different neurocognitive systems. In this paper, we tackle these two questions by focusing on hierarchical control as a neurocognitive mechanism underlying syntax in language and music. We put forward the Coordinated Hierarchical Control (CHC) hypothesis: linguistic and musical syntax rely on hierarchical control, but engage this shared mechanism differently depending on the current control demand. While linguistic syntax preferably engages the abstract rule-based control circuit, musical syntax rather employs the coordination of the abstract rule-based and the more concrete motor-based control circuits. We provide evidence for our hypothesis by reviewing neuroimaging as well as neuropsychological studies on linguistic and musical syntax. The CHC hypothesis makes a set of novel testable predictions to guide future work on the relationship between language and music.

The Triadic Roots of Human Cognition: "Mind" Is the Ability to go Beyond Dyadic Associations

Empirical evidence is reviewed indicating that the extraordinary aspects of the human mind are due to our species' ability to go beyond simple "dyadic associations" and to process the relations among three items of information simultaneously. Classic explanations of the "triadic" nature of human skills have been advocated by various scholars in the context of the evolution of human cognition. Here I summarize the core processes as found in (i) the syntax of language, (ii) tool-usage, and (iii) joint attention. I then review the triadic foundations of two perceptual phenomena of great importance in human aesthetics: (iv) harmony perception and (v) pictorial depth perception. In all five subfields of human psychology, most previous work has emphasized the recursive, hierarchical complexity of such "higher cognition," but a strongly reductionist approach indicates that the core mechanisms are triadic. It is concluded that the cognitive skills traditionally considered to be "uniquely" human require three-way associational processing that most non-Primate animal species find difficult or impossible, but all members of Homo sapiens - regardless of small cultural differences - find easy and inherently intriguing.

Music models aberrant rule decoding and reward valuation in dementia

Aberrant rule- and reward-based processes underpin abnormalities of socio-emotional behaviour in major dementias. However, these processes remain poorly characterized. Here we used music to probe rule decoding and reward valuation in patients with frontotemporal dementia (FTD) syndromes and Alzheimer's disease (AD) relative to healthy age-matched individuals. We created short melodies that were either harmonically resolved ('finished') or unresolved ('unfinished'); the task was to classify each melody as finished or unfinished (rule processing) and rate its subjective pleasantness (reward valuation). Results were adjusted for elementary pitch and executive processing; neuroanatomical correlates were assessed using voxel-based morphometry. Relative to healthy older controls, patients with behavioural variant FTD showed impairments of both musical rule decoding and reward valuation, while patients with semantic dementia showed impaired reward valuation but intact rule decoding, patients with AD showed impaired rule decoding but intact reward valuation and patients with progressive non-fluent aphasia performed comparably to healthy controls. Grey matter associations with task performance were identified in anterior temporal, medial and lateral orbitofrontal cortices, previously implicated in computing diverse biological and non-biological rules and rewards. The processing of musical rules and reward distils cognitive and neuroanatomical mechanisms relevant to complex socio-emotional dysfunction in major dementias.

Vocal similarity predicts the relative attraction of musical chords

The foundations of human music have long puzzled philosophers, mathematicians, psychologists, and neuroscientists. Although virtually all cultures uses combinations of tones as a basis for musical expression, why humans favor some tone combinations over others has been debated for millennia. Here we show that our attraction to specific tone combinations played simultaneously (chords) is predicted by their spectral similarity to voiced speech sounds. This connection between auditory aesthetics and a primary characteristic of vocalization adds to other evidence that tonal preferences arise from the biological advantages of social communication mediated by speech and language.

Musical chords are combinations of two or more tones played together. While many different chords are used in music, some are heard as more attractive (consonant) than others. We have previously suggested that, for reasons of biological advantage, human tonal preferences can be understood in terms of the spectral similarity of tone combinations to harmonic human vocalizations. Using the chromatic scale, we tested this theory further by assessing the perceived consonance of all possible dyads, triads, and tetrads within a single octave. Our results show that the consonance of chords is predicted by their relative similarity to voiced speech sounds. These observations support the hypothesis that the relative attraction of musical tone combinations is due, at least in part, to the biological advantages that accrue from recognizing and responding to conspecific vocal stimuli.

Calculation of the acoustical properties of triadic harmonies

The author reports that the harmonic "tension" and major/minor "valence" of pitch combinations can be calculated directly from acoustical properties without relying on concepts from traditional harmony theory. The capability to compute the well-known types of harmonic triads means that their perception is not simply a consequence of learning an arbitrary cultural "idiom" handed down from the Italian Renaissance. On the contrary, for typical listeners familiar with diatonic music, attention to certain, definable, acoustical features underlies the perception of the valence (modality) and the inherent tension (instability) of three-tone harmonies.

Neural mechanisms underlying valence inferences to sound: The role of the right angular gyrus

We frequently infer others' intentions based on non-verbal auditory cues. Although the brain underpinnings of social cognition have been extensively studied, no empirical work has yet examined the impact of musical structure manipulation on the neural processing of emotional valence during mental state inferences. We used a novel sound-based theory-of-mind paradigm in which participants categorized stimuli of different sensory dissonance level in terms of positive/negative valence. Whilst consistent with previous studies which propose facilitated encoding of consonances, our results demonstrated that distinct levels of consonance/dissonance elicited differential influences on the right angular gyrus, an area implicated in mental state attribution and attention reorienting processes. Functional and effective connectivity analyses further showed that consonances modulated a specific inhibitory interaction from associative memory to mental state attribution substrates. Following evidence suggesting that individuals with autism may process social affective cues differently, we assessed the relationship between participants' task performance and self-reported autistic traits in clinically typical adults. Higher scores on the social cognition scales of the AQ were associated with deficits in recognising positive valence in consonant sound cues. These findings are discussed with respect to Bayesian perspectives on autistic perception, which highlight a functional failure to optimize precision in relation to prior beliefs.

Heidelberg Neuro-Music Therapy Enhances Task-Negative Activity in Tinnitus Patients

Background: Suffering from tinnitus causes mental distress in most patients. Recent findings point toward a diminished activity of the brain's default-mode network (DMN) in subjects with mental disorders including depression or anxiety and also recently in subjects with tinnitus-related distress. We recently developed a therapeutic intervention, namely the Heidelberg Neuro-Music Therapy (HNMT), which shows an effective reduction of tinnitus-related distress following a 1-week short-term treatment. This approach offers the possibility to evaluate the neural changes associated with the improvements in tinnitus distress. We previously reported gray matter (GM) reorganization in DMN regions and in primary auditory areas following HNMT in cases of recent-onset tinnitus. Here we evaluate on the same patient group, using functional MRI (fMRI), the activity of the DMN following the improvements tinnitus-related distress related to the HNMT intervention.

Methods: The DMN activity was estimated by the task-negative activation (TNA) during long inter-trial intervals in a word recognition task. The level of TNA was evaluated twice, before and after the 1-week study period, in 18 treated tinnitus patients (“treatment group,” TG), 21 passive tinnitus controls (PTC), and 22 active healthy controls (AC). During the study, the participants in TG and AC groups were treated with HNMT, whereas PTC patients did not receive any tinnitus-specific treatment. Therapy-related effects on DMN activity were assessed by comparing the pairs of fMRI records from the TG and PTC groups.

Results: Treatment of the TG group with HNMT resulted in an augmented DMN activity in the PCC by 2.5% whereas no change was found in AC and PTC groups. This enhancement of PCC activity correlated with a reduction in tinnitus distress (Spearman Rho: −0.5; p < 0.005).

Conclusion: Our findings show that an increased DMN activity, especially in the PCC, underlies the improvements in tinnitus-related distress triggered by HNMT and identify the DMN as an important network involved in therapeutic improvements.

Sensory cortical response to uncertainty and low salience during recognition of affective cues in musical intervals

Previous neuroimaging studies have shown an increased sensory cortical response (i.e., heightened weight on sensory evidence) under higher levels of predictive uncertainty. The signal enhancement theory proposes that attention improves the quality of the stimulus representation, and therefore reduces uncertainty by increasing the gain of the sensory signal. The present study employed functional magnetic resonance imaging (fMRI) to investigate the neural correlates for ambiguous valence inferences signaled by auditory information within an emotion recognition paradigm. Participants categorized sound stimuli of three distinct levels of consonance/dissonance controlled by interval content. Separate behavioural and neuroscientific experiments were conducted. Behavioural results revealed that, compared with the consonance condition (perfect fourths, fifths and octaves) and the strong dissonance condition (minor/major seconds and tritones), the intermediate dissonance condition (minor thirds) was the most ambiguous, least salient and more cognitively demanding category (slowest reaction times). The neuroscientific findings were consistent with a heightened weight on sensory evidence whilst participants were evaluating intermediate dissonances, which was reflected in an increased neural response of the right Heschl’s gyrus. The results support previous studies that have observed enhanced precision of sensory evidence whilst participants attempted to represent and respond to higher degrees of uncertainty, and converge with evidence showing preferential processing of complex spectral information in the right primary auditory cortex. These findings are discussed with respect to music-theoretical concepts and recent Bayesian models of perception, which have proposed that attention may heighten the weight of information coming from sensory channels to stimulate learning about unknown predictive relationships.

Review on Neural Correlates of Emotion Regulation and Music: Implications for Emotion Dysregulation

Previous studies have examined the neural correlates of emotion regulation and the neural changes that are evoked by music exposure. However, the link between music and emotion regulation is poorly understood. The objectives of this review are to (1) synthesize what is known about the neural correlates of emotion regulation and music-evoked emotions, and (2) consider the possibility of therapeutic effects of music on emotion dysregulation. Music-evoked emotions can modulate activities in both cortical and subcortical systems, and across cortical-subcortical networks. Functions within these networks are integral to generation and regulation of emotions. Since dysfunction in these networks are observed in numerous psychiatric disorders, a better understanding of neural correlates of music exposure may lead to more systematic and effective use of music therapy in emotion dysregulation.

Functional MRI of music emotion processing in frontotemporal dementia

Frontotemporal dementia is an important neurodegenerative disorder of younger life led by profound emotional and social dysfunction. Here we used fMRI to assess brain mechanisms of music emotion processing in a cohort of patients with frontotemporal dementia (n = 15) in relation to healthy age-matched individuals (n = 11). In a passive-listening paradigm, we manipulated levels of emotion processing in simple arpeggio chords (mode versus dissonance) and emotion modality (music versus human emotional vocalizations). A complex profile of disease-associated functional alterations was identified with separable signatures of musical mode, emotion level, and emotion modality within a common, distributed brain network, including posterior and anterior superior temporal and inferior frontal cortices and dorsal brainstem effector nuclei. Separable functional signatures were identified post-hoc in patients with and without abnormal craving for music (musicophilia): a model for specific abnormal emotional behaviors in frontotemporal dementia. Our findings indicate the potential of music to delineate neural mechanisms of altered emotion processing in dementias, with implications for future disease tracking and therapeutic strategies.

Human brain basis of musical rhythm perception: common and distinct neural substrates for meter, tempo, and pattern

Rhythm as the time structure of music is composed of distinct temporal components such as pattern, meter, and tempo. Each feature requires different computational processes: meter involves representing repeating cycles of strong and weak beats; pattern involves representing intervals at each local time point which vary in length across segments and are linked hierarchically; and tempo requires representing frequency rates of underlying pulse structures. We explored whether distinct rhythmic elements engage different neural mechanisms by recording brain activity of adult musicians and non-musicians with positron emission tomography (PET) as they made covert same-different discriminations of (a) pairs of rhythmic, monotonic tone sequences representing changes in pattern, tempo, and meter, and (b) pairs of isochronous melodies. Common to pattern, meter, and tempo tasks were focal activities in right, or bilateral, areas of frontal, cingulate, parietal, prefrontal, temporal, and cerebellar cortices. Meter processing alone activated areas in right prefrontal and inferior frontal cortex associated with more cognitive and abstract representations. Pattern processing alone recruited right cortical areas involved in different kinds of auditory processing. Tempo processing alone engaged mechanisms subserving somatosensory and premotor information (e.g., posterior insula, postcentral gyrus). Melody produced activity different from the rhythm conditions (e.g., right anterior insula and various cerebellar areas). These exploratory findings suggest the outlines of some distinct neural components underlying the components of rhythmic structure.

Anatomical differences in the human inferior colliculus relate to the perceived valence of musical consonance and dissonance

Helmholtz himself speculated about a role of the cochlea in the perception of musical dissonance. Here we indirectly investigated this issue, assessing the valence judgment of musical stimuli with variable consonance/dissonance and presented diotically (exactly the same dissonant signal was presented to both ears) or dichotically (a consonant signal was presented to each ear--both consonant signals were rhythmically identical but differed by a semitone in pitch). Differences in brain organisation underlying inter-subject differences in the percept of dichotically presented dissonance were determined with voxel-based morphometry. Behavioral results showed that diotic dissonant stimuli were perceived as more unpleasant than dichotically presented dissonance, indicating that interactions within the cochlea modulated the valence percept during dissonance. However, the behavioral data also suggested that the dissonance percept did not depend crucially on the cochlea, but also occurred as a result of binaural integration when listening to dichotic dissonance. These results also showed substantial between-participant variations in the valence response to dichotic dissonance. These differences were in a voxel-based morphometry analysis related to differences in gray matter density in the inferior colliculus, which strongly substantiated a key role of the inferior colliculus in consonance/dissonance representation in humans.

The role of the auditory brainstem in processing musically relevant pitch

Neuroimaging work has shed light on the cerebral architecture involved in processing the melodic and harmonic aspects of music. Here, recent evidence is reviewed illustrating that subcortical auditory structures contribute to the early formation and processing of musically relevant pitch. Electrophysiological recordings from the human brainstem and population responses from the auditory nerve reveal that nascent features of tonal music (e.g., consonance/dissonance, pitch salience, harmonic sonority) are evident at early, subcortical levels of the auditory pathway. The salience and harmonicity of brainstem activity is strongly correlated with listeners' perceptual preferences and perceived consonance for the tonal relationships of music. Moreover, the hierarchical ordering of pitch intervals/chords described by the Western music practice and their perceptual consonance is well-predicted by the salience with which pitch combinations are encoded in subcortical auditory structures. While the neural correlates of consonance can be tuned and exaggerated with musical training, they persist even in the absence of musicianship or long-term enculturation. As such, it is posited that the structural foundations of musical pitch might result from innate processing performed by the central auditory system. A neurobiological predisposition for consonant, pleasant sounding pitch relationships may be one reason why these pitch combinations have been favored by composers and listeners for centuries. It is suggested that important perceptual dimensions of music emerge well before the auditory signal reaches cerebral cortex and prior to attentional engagement. While cortical mechanisms are no doubt critical to the perception, production, and enjoyment of music, the contribution of subcortical structures implicates a more integrated, hierarchically organized network underlying music processing within the brain.

Corticostriatal contributions to musical expectancy perception

This study investigates the functional neuroanatomy of harmonic music perception with fMRI. We presented short pieces of Western classical music to nonmusicians. The ending of each piece was systematically manipulated in the following four ways: Standard Cadence (expected resolution), Deceptive Cadence (moderate deviation from expectation), Modulated Cadence (strong deviation from expectation but remaining within the harmonic structure of Western tonal music), and Atonal Cadence (strongest deviation from expectation by leaving the harmonic structure of Western tonal music). Music compared with baseline broadly recruited regions of the bilateral superior temporal gyrus (STG) and the right inferior frontal gyrus (IFG). Parametric regressors scaled to the degree of deviation from harmonic expectancy identified regions sensitive to expectancy violation. Areas within the BG were significantly modulated by expectancy violation, indicating a previously unappreciated role in harmonic processing. Expectancy violation also recruited bilateral cortical regions in the IFG and anterior STG, previously associated with syntactic processing in other domains. The posterior STG was not significantly modulated by expectancy. Granger causality mapping found functional connectivity between IFG, anterior STG, posterior STG, and the BG during music perception. Our results imply the IFG, anterior STG, and the BG are recruited for higher-order harmonic processing, whereas the posterior STG is recruited for basic pitch and melodic processing.