A Free Energy Formulation of Music Generation and Perception: Helmholtz Revisited

Cognitive and sensory expectations independently shape musical expectancy and pleasure

Expectation is crucial for our enjoyment of music, yet the underlying generative mechanisms remain unclear. While sensory models derive predictions based on local acoustic information in the auditory signal, cognitive models assume abstract knowledge of music structure acquired over the long term. To evaluate these two contrasting mechanisms, we compared simulations from four computational models of musical expectancy against subjective expectancy and pleasantness ratings of over 1000 chords sampled from 739 US Billboard pop songs. Bayesian model comparison revealed that listeners' expectancy and pleasantness ratings were predicted by the independent, non-overlapping, contributions of cognitive and sensory expectations. Furthermore, cognitive expectations explained over twice the variance in listeners' perceived surprise compared to sensory expectations, suggesting a larger relative importance of long-term representations of music structure over short-term sensory-acoustic information in musical expectancy. Our results thus emphasize the distinct, albeit complementary, roles of cognitive and sensory expectations in shaping musical pleasure, and suggest that this expectancy-driven mechanism depends on musical information represented at different levels of abstraction along the neural hierarchy. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

From Generative Models to Generative Passages: A Computational Approach to (Neuro) Phenomenology

This paper presents a version of neurophenomenology based on generative modelling techniques developed in computational neuroscience and biology. Our approach can be described as computational phenomenology because it applies methods originally developed in computational modelling to provide a formal model of the descriptions of lived experience in the phenomenological tradition of philosophy (e.g., the work of Edmund Husserl, Maurice Merleau-Ponty, etc.). The first section presents a brief review of the overall project to naturalize phenomenology. The second section presents and evaluates philosophical objections to that project and situates our version of computational phenomenology with respect to these projects. The third section reviews the generative modelling framework. The final section presents our approach in detail. We conclude by discussing how our approach differs from previous attempts to use generative modelling to help understand consciousness. In summary, we describe a version of computational phenomenology which uses generative modelling to construct a computational model of the inferential or interpretive processes that best explain this or that kind of lived experience.

Musicality as a predictive process

Savage et al. argue for musicality as having evolved for the overarching purpose of social bonding. By way of contrast, we highlight contemporary predictive processing models of human cognitive functioning in which the production and enjoyment of music follows directly from the principle of prediction error minimization.

Motor and Predictive Processes in Auditory Beat and Rhythm Perception

In this article, we review recent advances in research on rhythm and musical beat perception, focusing on the role of predictive processes in auditory motor interactions. We suggest that experimental evidence of the motor system's role in beat perception, including in passive listening, may be explained by the generation and maintenance of internal predictive models, concordant with the Active Inference framework of sensory processing. We highlight two complementary hypotheses for the neural underpinnings of rhythm perception: The Action Simulation for Auditory Prediction hypothesis (Patel and Iversen, 2014) and the Gradual Audiomotor Evolution hypothesis (Merchant and Honing, 2014) and review recent experimental progress supporting each of these hypotheses. While initial formulations of ASAP and GAE explain different aspects of beat-based timing-the involvement of motor structures in the absence of movement, and physical entrainment to an auditory beat respectively-we suggest that work under both hypotheses provide converging evidence toward understanding the predictive role of the motor system in the perception of rhythm, and the specific neural mechanisms involved. We discuss future experimental work necessary to further evaluate the causal neural mechanisms underlying beat and rhythm perception.

TTOM in action: Refining the variational approach to cognition and culture

The target article "Thinking Through Other Minds" (TTOM) offered an account of the distinctively human capacity to acquire cultural knowledge, norms, and practices. To this end, we leveraged recent ideas from theoretical neurobiology to understand the human mind in social and cultural contexts. Our aim was both synthetic - building an integrative model adequate to account for key features of cultural learning and adaptation; and prescriptive - showing how the tools developed to explain brain dynamics can be applied to the emergence of social and cultural ecologies of mind. In this reply to commentators, we address key issues, including: (1) refining the concept of culture to show how TTOM and the free-energy principle (FEP) can capture essential elements of human adaptation and functioning; (2) addressing cognition as an embodied, enactive, affective process involving cultural affordances; (3) clarifying the significance of the FEP formalism related to entropy minimization, Bayesian inference, Markov blankets, and enactivist views; (4) developing empirical tests and applications of the TTOM model; (5) incorporating cultural diversity and context at the level of intra-cultural variation, individual differences, and the transition to digital niches; and (6) considering some implications for psychiatry. The commentators' critiques and suggestions point to useful refinements and applications of the model. In ongoing collaborations, we are exploring how to augment the theory with affective valence, take into account individual differences and historicity, and apply the model to specific domains including epistemic bias.

The hierarchically mechanistic mind: an evolutionary systems theory of the human brain, cognition, and behavior

The purpose of this review was to integrate leading paradigms in psychology and neuroscience with a theory of the embodied, situated human brain, called the Hierarchically Mechanistic Mind (HMM). The HMM describes the brain as a complex adaptive system that functions to minimize the entropy of our sensory and physical states via action-perception cycles generated by hierarchical neural dynamics. First, we review the extant literature on the hierarchical structure of the brain. Next, we derive the HMM from a broader evolutionary systems theory that explains neural structure and function in terms of dynamic interactions across four nested levels of biological causation (i.e., adaptation, phylogeny, ontogeny, and mechanism). We then describe how the HMM aligns with a global brain theory in neuroscience called the free-energy principle, leveraging this theory to mathematically formulate neural dynamics across hierarchical spatiotemporal scales. We conclude by exploring the implications of the HMM for psychological inquiry.

Musical reward prediction errors engage the nucleus accumbens and motivate learning

Enjoying music reliably ranks among life's greatest pleasures. Like many hedonic experiences, it engages several reward-related brain areas, with activity in the nucleus accumbens (NAc) most consistently reflecting the listener's subjective response. Converging evidence suggests that this activity arises from musical "reward prediction errors" (RPEs) that signal the difference between expected and perceived musical events, but this hypothesis has not been directly tested. In the present fMRI experiment, we assessed whether music could elicit formally modeled RPEs in the NAc by applying a well-established decision-making protocol designed and validated for studying RPEs. In the scanner, participants chose between arbitrary cues that probabilistically led to dissonant or consonant music, and learned to make choices associated with the consonance, which they preferred. We modeled regressors of trial-by-trial RPEs, finding that NAc activity tracked musically elicited RPEs, to an extent that explained variance in the individual learning rates. These results demonstrate that music can act as a reward, driving learning and eliciting RPEs in the NAc, a hub of reward- and music enjoyment-related activity.

Now you hear it: a predictive coding model for understanding rhythmic incongruity

Rhythmic incongruity in the form of syncopation is a prominent feature of many contemporary musical styles. Syncopations afford incongruity between rhythmic patterns and the meter, giving rise to mental models of differently accented isochronous beats. Syncopations occur either in isolation or as part of rhythmic patterns, so-called grooves. On the basis of the predictive coding framework, we discuss how brain processing of rhythm can be seen as a special case of predictive coding. We present a simple, yet powerful model for how the brain processes rhythmic incongruity: the model for predictive coding of rhythmic incongruity. Our model proposes that a given rhythm's syncopation and its metrical uncertainty (precision) is at the heart of how the brain models rhythm and meter based on priors, predictions, and prediction error. Our minimal model can explain prominent features of brain processing of syncopation: why isolated syncopations lead to stronger prediction error in the brains of musicians, as evidenced by larger event-related potentials to rhythmic incongruity, and why we all experience a stronger urge to move to grooves with a medium level of syncopation compared with low and high levels of syncopation.

Music-Evoked Emotions-Current Studies

The present study is focused on a review of the current state of investigating music-evoked emotions experimentally, theoretically and with respect to their therapeutic potentials. After a concise historical overview and a schematic of the hearing mechanisms, experimental studies on music listeners and on music performers are discussed, starting with the presentation of characteristic musical stimuli and the basic features of tomographic imaging of emotional activation in the brain, such as functional magnetic resonance imaging (fMRI) and positron emission tomography (PET), which offer high spatial resolution in the millimeter range. The progress in correlating activation imaging in the brain to the psychological understanding of music-evoked emotion is demonstrated and some prospects for future research are outlined. Research in psychoneuroendocrinology and molecular markers is reviewed in the context of music-evoked emotions and the results indicate that the research in this area should be intensified. An assessment of studies involving measuring techniques with high temporal resolution down to the 10 ms range, as, e.g., electroencephalography (EEG), event-related brain potentials (ERP), magnetoencephalography (MEG), skin conductance response (SCR), finger temperature, and goose bump development (piloerection) can yield information on the dynamics and kinetics of emotion. Genetic investigations reviewed suggest the heredity transmission of a predilection for music. Theoretical approaches to musical emotion are directed to a unified model for experimental neurological evidence and aesthetic judgment. Finally, the reports on musical therapy are briefly outlined. The study concludes with an outlook on emerging technologies and future research fields.

Music-evoked emotions: principles, brain correlates, and implications for therapy

This paper describes principles underlying the evocation of emotion with music: evaluation, resonance, memory, expectancy/tension, imagination, understanding, and social functions. Each of these principles includes several subprinciples, and the framework on music-evoked emotions emerging from these principles and subprinciples is supposed to provide a starting point for a systematic, coherent, and comprehensive theory on music-evoked emotions that considers both reception and production of music, as well as the relevance of emotion-evoking principles for music therapy.

Brain correlates of music-evoked emotions

Music is a universal feature of human societies, partly owing to its power to evoke strong emotions and influence moods. During the past decade, the investigation of the neural correlates of music-evoked emotions has been invaluable for the understanding of human emotion. Functional neuroimaging studies on music and emotion show that music can modulate activity in brain structures that are known to be crucially involved in emotion, such as the amygdala, nucleus accumbens, hypothalamus, hippocampus, insula, cingulate cortex and orbitofrontal cortex. The potential of music to modulate activity in these structures has important implications for the use of music in the treatment of psychiatric and neurological disorders.