The neuroscience of music – towards ecological validity

Audio-visual concert performances synchronize audience's heart rates

People enjoy engaging with music. Live music concerts provide an excellent option to investigate real-world music experiences, and at the same time, use neurophysiological synchrony to assess dynamic engagement. In the current study, we assessed engagement in a live concert setting using synchrony of cardiorespiratory measures, comparing inter-subject, stimulus-response, correlation, and phase coherence. As engagement might be enhanced in a concert setting by seeing musicians perform, we presented audiences with audio-only (AO) and audio-visual (AV) piano performances. Only correlation synchrony measures were above chance level. In comparing time-averaged synchrony across conditions, AV performances evoked a higher inter-subject correlation of heart rate (ISC-HR). However, synchrony averaged across music pieces did not correspond to self-reported engagement. On the other hand, time-resolved analyses show that synchronized deceleration-acceleration heart rate (HR) patterns, typical of an "orienting response" (an index of directed attention), occurred within music pieces at salient events of section boundaries. That is, seeing musicians perform heightened audience engagement at structurally important moments in Western classical music. Overall, we could show that multisensory information shapes dynamic engagement. By comparing different synchrony measures, we further highlight the advantages of time series analysis, specifically ISC-HR, as a robust measure of holistic musical listening experiences in naturalistic concert settings.

Beauty and the brain - Investigating the neural and musical attributes of beauty during naturalistic music listening

Beauty judgments are common in daily life, but rarely studied in cognitive neuroscience. Here, in three studies, we searched for the neural mechanisms of musical beauty using a naturalistic free-listening paradigm applied to behavioral and neuroimaging recordings and validated by experts' judgments. In Study 1, 30 adults continuously rated the perceived beauty of three musical pieces using a motion sensor. This served to identify the musical passages that were inter-subjectively judged as more or less beautiful ('beautiful' vs. 'not-beautiful' passages). For identifying the consistent neural determinants of the perception of musical beauty, we utilized these ratings in Study 2, where 36 adults were recorded with functional magnetic resonance imaging (fMRI) while they listened attentively to the same pieces of Study 1. In Study 3, to identify the musicological features characterizing the beautiful and not-beautiful passages of Studies 1-2, we collected post-hoc questionnaires from 12 music-composition experts. Results from Study 2 evidenced focal activity in the orbitofrontal cortex when listening to beautiful passages whereas the not-beautiful passages were associated with bilateral supratemporal activity. Effective connectivity analysis discovered inhibition of auditory activation and neural communication with the right orbitofrontal cortex for listening to beautiful passages vs. intrinsic activation of auditory cortices and decreased coupling to orbitofrontal cortex for not-beautiful passages. Experts' questionnaires indicated that the beautiful passages were more melodic, calm, sad, slow, tonal, traditional, and simple than the ones rated negatively. In sum, we identified neural and psychological underpinnings of musical beauty, irrespectively of individual taste and listening biography.

Encoding models for developmental cognitive computational neuroscience: Promise, challenges, and potential

Cognitive computational neuroscience has received broad attention in recent years as an emerging area integrating cognitive science, neuroscience, and artificial intelligence. At the heart of this field, approaches using encoding models allow for explaining brain activity from latent and high-dimensional features, including artificial neural networks. With the notable exception of temporal response function models that are applied to electroencephalography, most prior studies have focused on adult subjects, making it difficult to capture how brain representations change with learning and development. Here, we argue that future developmental cognitive neuroscience studies would benefit from approaches relying on encoding models. We provide an overview of encoding models used in adult functional magnetic resonance imaging research. This research has notably used data with a small number of subjects, but with a large number of samples per subject. Studies using encoding models also generally require task-based neuroimaging data. Though these represent challenges for developmental studies, we argue that these challenges may be overcome by using functional alignment techniques and naturalistic paradigms. These methods would facilitate encoding model analysis in developmental neuroimaging research, which may lead to important theoretical advances.

Lights on music cognition: A systematic and critical review of fNIRS applications and future perspectives

Research investigating the neural processes related to music perception and production constitutes a well-established field within the cognitive neurosciences. While most neuroimaging tools have limitations in studying the complexity of musical experiences, functional Near-Infrared Spectroscopy (fNIRS) represents a promising, relatively new tool for studying music processes in both laboratory and ecological settings, which is also suitable for both typical and pathological populations across development. Here we systematically review fNIRS studies on music cognition, highlighting prospects and potentialities. We also include an overview of fNIRS basic theory, together with a brief comparison to characteristics of other neuroimaging tools. Fifty-nine studies meeting inclusion criteria (i.e., using fNIRS with music as the primary stimulus) are presented across five thematic sections. Critical discussion of methodology leads us to propose guidelines of good practices aiming for robust signal analyses and reproducibility. A continuously updated world map is proposed, including basic information from studies meeting the inclusion criteria. It provides an organized, accessible, and updatable reference database, which could serve as a catalyst for future collaborations within the community. In conclusion, fNIRS shows potential for investigating cognitive processes in music, particularly in ecological contexts and with special populations, aligning with current research priorities in music cognition.

Overcoming boundaries: Interdisciplinary challenges and opportunities in cognitive neuroscience

Cognitive neuroscience has considerable untapped potential to translate our understanding of brain function into applications that maintain, restore, or enhance human cognition. Complex, real-world phenomena encountered in daily life, professional contexts, and in the arts, can also be a rich source of information for better understanding cognition, which in turn can lead to advances in knowledge and health outcomes. Interdisciplinary work is needed for these bi-directional benefits to be realized. Our cognitive neuroscience team has been collaborating on several interdisciplinary projects: hardware and software development for brain stimulation, measuring human operator state in safety-critical robotics environments, and exploring emotional regulation in actors who perform traumatic narratives. Our approach is to study research questions of mutual interest in the contexts of domain-specific applications, using (and sometimes improving) the experimental tools and techniques of cognitive neuroscience. These interdisciplinary attempts are described as case studies in the present work to illustrate non-trivial challenges that come from working across traditional disciplinary boundaries. We reflect on how obstacles to interdisciplinary work can be overcome, with the goals of enriching our understanding of human cognition and amplifying the positive effects cognitive neuroscientists have on society and innovation.

A Concert-Based Study on Melodic Contour Identification among Varied Hearing Profiles-A Preliminary Report

Background: This study investigated how different hearing profiles influenced melodic contour identification (MCI) in a real-world concert setting with a live band including drums, bass, and a lead instrument. We aimed to determine the impact of various auditory assistive technologies on music perception in an ecologically valid environment. Methods: The study involved 43 participants with varying hearing capabilities: normal hearing, bilateral hearing aids, bimodal hearing, single-sided cochlear implants, and bilateral cochlear implants. Participants were exposed to melodies played on a piano or accordion, with and without an electric bass as a masker, accompanied by a basic drum rhythm. Bayesian logistic mixed-effects models were utilized to analyze the data. Results: The introduction of an electric bass as a masker did not significantly affect MCI performance for any hearing group when melodies were played on the piano, contrary to its effect on accordion melodies and previous studies. Greater challenges were observed with accordion melodies, especially when accompanied by an electric bass. Conclusions: MCI performance among hearing aid users was comparable to other hearing-impaired profiles, challenging the hypothesis that they would outperform cochlear implant users. A cohort of short melodies inspired by Western music styles was developed for future contour identification tasks.

Frontal and cerebellar contributions to pitch and rhythm processing: a TMS study

Music represents a salient stimulus for the brain with two key features: pitch and rhythm. Few data are available on cognitive analysis of music listening in musically naïve healthy participants. Beyond auditory cortices, neuroimaging data showed the involvement of prefrontal cortex in pitch and of cerebellum in rhythm. The present study is aimed at investigating the role of prefrontal and cerebellar cortices in both pitch and rhythm processing. The performance of fifteen participants without musical expertise was investigated in a listening discrimination task. The task required to decide whether two eight-element melodic sequences were equal or different according to pitch or rhythm characteristics. Before the task, we applied a protocol of continuous theta burst transcranial magnetic stimulation interfering with the activity of the left cerebellar hemisphere (lCb), right inferior frontal gyrus (rIFG), or vertex (Cz-control site), in a within cross-over design. Our results showed that participants were more accurate in pitch than rhythm tasks. Importantly, the reaction times were slower following rIFG or lCb stimulations in both tasks. Notably, frontal and cerebellar stimulations did not induce any motor effect in right and left hand. The present findings point to the role of the fronto-cerebellar network in music processing with a single mechanism for both pitch and rhythm patterns.

Making sense of music: Insights from neurophysiology and connectivity analyses in naturalistic listening conditions

According to the latest frameworks, auditory perception and memory involve the constant prediction of future sound events by the brain, based on the continuous extraction of feature regularities from the environment. The neural hierarchical mechanisms for predictive processes in perception and memory for sounds are typically studied in relation to simple acoustic features in isolated sounds or sound patterns inserted in highly certain contexts. Such studies have identified reliable prediction formation and error signals, e.g., the N100 or the mismatch negativity (MMN) evoked responses. In real life, though, individuals often face situations in which uncertainty prevails and where making sense of sounds becomes a hard challenge. In music, not only deviations from predictions are masterly set up by composers to induce emotions but sometimes the sheer uncertainty of sound scenes is exploited for aesthetic purposes, especially in compositional styles such as Western atonal classical music. In very recent magnetoencephalography (MEG) and electroencephalography (EEG) studies, experimental and technical advances in stimulation paradigms and analysis approaches have permitted the identification of prediction-error responses from highly uncertain, atonal contexts and the extraction of prediction-related responses from real, continuous music. Moreover, functional connectivity analyses revealed the emergence of cortico-hippocampal interactions during the formation of auditory memories for more predictable vs. less predictable patterns. These findings contribute to understanding the general brain mechanisms that enable us to predict even highly uncertain sound environments and to possibly make sense of and appreciate even atonal music.

Early social communication through music: State of the art and future perspectives

A growing body of research shows that the universal capacity for music perception and production emerges early in development. Possibly building on this predisposition, caregivers around the world often communicate with infants using songs or speech entailing song-like characteristics. This suggests that music might be one of the earliest developing and most accessible forms of interpersonal communication, providing a platform for studying early communicative behavior. However, little research has examined music in truly communicative contexts. The current work aims to facilitate the development of experimental approaches that rely on dynamic and naturalistic social interactions. We first review two longstanding lines of research that examine musical interactions by focusing either on the caregiver or the infant. These include defining the acoustic and non-acoustic features that characterize infant-directed (ID) music, as well as behavioral and neurophysiological research examining infants' processing of musical timing and pitch. Next, we review recent studies looking at early musical interactions holistically. This research focuses on how caregivers and infants interact using music to achieve co-regulation, mutual engagement, and increase affiliation and prosocial behavior. We conclude by discussing methodological, technological, and analytical advances that might empower a comprehensive study of musical communication in early childhood.

Physiological synchrony and shared flow state in Javanese gamelan: positively associated while improvising, but not for traditional performance

The experience of shared flow refers to the optimal balance between challenge and ability for a given task, resulting from interpersonal action in a group situation. The performance of Javanese gamelan is an ideal setting to investigate shared flow, due to the requirement that all performers on varying instrumental parts work harmoniously, allowing for shared flow and its native equivalent, ngeli. To minimise the disruption of flow, while still measuring it continuously, one way to assess a person's state is by measuring physiological responses of the sympathetic (i.e., fight-or-flight) system, namely heart rate and skin conductance. Flow has been related to physiological signatures, and shared actions in music-making have been related to synchronised physiology. However, to our knowledge, no study yet has directly investigated the links between shared physiology and shared flow. Therefore, this study aimed to assess the associations between flow states, physiological synchrony, and Javanese gamelan playing. Subsequently, we tested for differences between advanced and beginner groups playing traditional gamelan pieces and improvising. Firstly, a factor analysis revealed a two-factor solution of Awareness and Absorption for self-reported shared flow. Next, using inter-subject correlation to assess synchrony and circular shuffling to infer significance, we found a greater proportion of significance in traditional playing compared to improvised playing for the experienced group, and the opposite for the beginner group. Lastly, linear mixed models revealed largely positive associations between synchronised physiology and shared flow during improvised playing, and negative associations during traditional playing, regardless of experience levels. This study demonstrates methodological possibilities for the quantitative study of shared flow in music-making contexts, and potential differences in shared flow experience in improvised and traditional, or prescribed, playing.

Influence of music on the hearing and mental health of adolescents and countermeasures

This review elaborates on the influence of music on the psychological well-being of adolescents, covering the potential advantages, drawbacks, and necessary strategic interventions associated with music exposure. Initially, we characterize music and delve into a multifaceted classification system. Music, as a pervasive art form, is categorized based on regional and national parameters, and it also distinguishes through the assorted genres and traits. The mental well-being of adolescents is significantly influenced by music through mechanisms such as the facilitation of emotional expression and regulation, fortification of social bonds and the sense of belonging, as well as the fostering of creativity and cognitive development. Nonetheless, music, if misused or associated with inappropriate content, could elicit a spectrum of issues ranging from auditory impairment, diversion of attention, addiction tendencies, to the induction of negative emotions. To counteract these potential hazards, we propose several mitigation strategies including the selection of appropriate music styles, the establishment of a wholesome music environment, the promotion of the constructive role of music education, and fostering active participation in music activities among the youth. In conclusion, we underscore the necessity of a collaborative endeavor from all sectors of society to ensure a healthy music environment for the youth, which in turn would enhance the positive influence of music on the mental health development of this population.