Statistical learning and Gestalt-like principles predict melodic expectations

Predictive processing of music and language in autism: Evidence from Mandarin and English speakers

Atypical predictive processing has been associated with autism across multiple domains, based mainly on artificial antecedents and consequents. As structured sequences where expectations derive from implicit learning of combinatorial principles, language and music provide naturalistic stimuli for investigating predictive processing. In this study, we matched melodic and sentence stimuli in cloze probabilities and examined musical and linguistic prediction in Mandarin- (Experiment 1) and English-speaking (Experiment 2) autistic and non-autistic individuals using both production and perception tasks. In the production tasks, participants listened to unfinished melodies/sentences and then produced the final notes/words to complete these items. In the perception tasks, participants provided expectedness ratings of the completed melodies/sentences based on the most frequent notes/words in the norms. While Experiment 1 showed intact musical prediction but atypical linguistic prediction in autism in the Mandarin sample that demonstrated imbalanced musical training experience and receptive vocabulary skills between groups, the group difference disappeared in a more closely matched sample of English speakers in Experiment 2. These findings suggest the importance of taking an individual differences approach when investigating predictive processing in music and language in autism, as the difficulty in prediction in autism may not be due to generalized problems with prediction in any type of complex sequence processing.

thebeat: A Python package for working with rhythms and other temporal sequences

thebeat is a Python package for working with temporal sequences and rhythms in the behavioral and cognitive sciences, as well as in bioacoustics. It provides functionality for creating experimental stimuli, and for visualizing and analyzing temporal data. Sequences, sounds, and experimental trials can be generated using single lines of code. thebeat contains functions for calculating common rhythmic measures, such as interval ratios, and for producing plots, such as circular histograms. thebeat saves researchers time when creating experiments, and provides the first steps in collecting widely accepted methods for use in timing research. thebeat is an open-source, on-going, and collaborative project, and can be extended for use in specialized subfields. thebeat integrates easily with the existing Python ecosystem, allowing one to combine our tested code with custom-made scripts. The package was specifically designed to be useful for both skilled and novice programmers. thebeat provides a foundation for working with temporal sequences onto which additional functionality can be built. This combination of specificity and plasticity should facilitate research in multiple research contexts and fields of study.

Does animacy affect visual statistical learning? Revisiting the effects of selective attention and animacy on visual statistical learning

Animates receive preferential attentional processing over inanimates because, from an evolutionary perspective, animates are important to human survival. We investigated whether animacy affects visual statistical learning-the detection and extraction of regularities in visual information from our rich, dynamic, and complex environment. Participants completed a selective-attention task, in which regularities were embedded in two visual streams, an attended and an unattended visual stream. The attended visual stream always consisted of line-drawings of non-objects, while the unattended visual stream consisted of line-drawings of either animates or inanimates. Participants then completed a triplet-discrimination task, which assessed their ability to extract regularities from the attended and unattended visual streams. We also assessed participants' awareness of regularities in the visual statistical learning task, and asked if any learning strategies were used. We were specifically interested in whether the animacy status of line-drawings in the unattended visual stream would affect visual statistical learning. There were four key findings. First, selective attention modulates visual statistical learning, with greater visual statistical learning for attended than for unattended information. Second, animacy does not affect visual statistical learning, with no differences found in visual statistical learning performance between the animate and inanimate condition. Third, awareness of regularities was associated with visual statistical learning of attended information. Fourth, participants used strategies (e.g., naming or labelling stimuli) during the visual statistical learning task. Further research is required to understand whether visual statistical learning is one of the adaptive functions that evolved from ancestral environments.

Hebbian learning can explain rhythmic neural entrainment to statistical regularities

In many domains, learners extract recurring units from continuous sequences. For example, in unknown languages, fluent speech is perceived as a continuous signal. Learners need to extract the underlying words from this continuous signal and then memorize them. One prominent candidate mechanism is statistical learning, whereby learners track how predictive syllables (or other items) are of one another. Syllables within the same word predict each other better than syllables straddling word boundaries. But does statistical learning lead to memories of the underlying words-or just to pairwise associations among syllables? Electrophysiological results provide the strongest evidence for the memory view. Electrophysiological responses can be time-locked to statistical word boundaries (e.g., N400s) and show rhythmic activity with a periodicity of word durations. Here, I reproduce such results with a simple Hebbian network. When exposed to statistically structured syllable sequences (and when the underlying words are not excessively long), the network activation is rhythmic with the periodicity of a word duration and activation maxima on word-final syllables. This is because word-final syllables receive more excitation from earlier syllables with which they are associated than less predictable syllables that occur earlier in words. The network is also sensitive to information whose electrophysiological correlates were used to support the encoding of ordinal positions within words. Hebbian learning can thus explain rhythmic neural activity in statistical learning tasks without any memory representations of words. Learners might thus need to rely on cues beyond statistical associations to learn the words of their native language. RESEARCH HIGHLIGHTS: Statistical learning may be utilized to identify recurring units in continuous sequences (e.g., words in fluent speech) but may not generate explicit memory for words. Exposure to statistically structured sequences leads to rhythmic activity with a period of the duration of the underlying units (e.g., words). I show that a memory-less Hebbian network model can reproduce this rhythmic neural activity as well as putative encodings of ordinal positions observed in earlier research. Direct tests are needed to establish whether statistical learning leads to declarative memories for words.

Cognitive and sensory expectations independently shape musical expectancy and pleasure

Expectation is crucial for our enjoyment of music, yet the underlying generative mechanisms remain unclear. While sensory models derive predictions based on local acoustic information in the auditory signal, cognitive models assume abstract knowledge of music structure acquired over the long term. To evaluate these two contrasting mechanisms, we compared simulations from four computational models of musical expectancy against subjective expectancy and pleasantness ratings of over 1000 chords sampled from 739 US Billboard pop songs. Bayesian model comparison revealed that listeners' expectancy and pleasantness ratings were predicted by the independent, non-overlapping, contributions of cognitive and sensory expectations. Furthermore, cognitive expectations explained over twice the variance in listeners' perceived surprise compared to sensory expectations, suggesting a larger relative importance of long-term representations of music structure over short-term sensory-acoustic information in musical expectancy. Our results thus emphasize the distinct, albeit complementary, roles of cognitive and sensory expectations in shaping musical pleasure, and suggest that this expectancy-driven mechanism depends on musical information represented at different levels of abstraction along the neural hierarchy. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

Expectation adaptation for rare cadences in music: Item order matters in repetition priming

Humans make predictions about future events in many domains, including when they listen to music. Previous accounts of harmonic expectation in music have emphasised the role of implicit musical knowledge acquired in the long term through the mechanism of statistical learning. However, it is not known whether listeners can adapt their expectations for unusual harmonies in the short term through repetition priming, and whether the extent of any short-term adaptation depends on the unfolding statistical structure of the music. To explore these possibilities, we presented 150 participants with phrases from Bach chorales that ended with a cadence that was either a priori likely or unlikely based on the long-term statistical structure of the corpus of chorales. While holding the 50-50 incidence of likely vs. unlikely cadences constant, we manipulated the order in which these phrases were presented such that the local probability of hearing an unlikely cadence changed throughout the experiment. For each phrase, participants provided two judgements: (a) a prospective rating of how confident they were in their expectations for the cadence, and (b) a retrospective rating of how well the presented cadence matched their expectations. While confidence ratings increased over the course of the experiment, the rate of change decreased as the local probability of an unexpected cadence increased. Participants' expectations favoured likely cadences over unlikely cadences on average, but their expectation ratings for unlikely cadences increased at a faster rate over the course of the experiment than for likely cadences, particularly when the local probability of hearing an unlikely cadence was high. Thus, despite entrenched long-term statistics about cadences, listeners can indeed adapt to unusual musical harmonies and are sensitive to the local statistical structure of the musical environment. We suggest that this adaptation is an instance of Bayesian belief updating, a domain-general process that accounts for expectation adaptation in multiple domains.

Interactionally Embedded Gestalt Principles of Multimodal Human Communication

Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g., parallel or temporally misaligned), must be rapidly processed as a coherent message by the receiver. In this contribution, we introduce the notion of interactionally embedded, affordance-driven gestalt perception as a framework that can explain how this rapid processing of multimodal signals is achieved as efficiently as it is. We discuss empirical evidence showing how basic principles of gestalt perception can explain some aspects of unimodal phenomena such as verbal language processing and visual scene perception but require additional features to explain multimodal human communication. We propose a framework in which high-level gestalt predictions are continuously updated by incoming sensory input, such as unfolding speech and visual signals. We outline the constituent processes that shape high-level gestalt perception and their role in perceiving relevance and prägnanz. Finally, we provide testable predictions that arise from this multimodal interactionally embedded gestalt-perception framework. This review and framework therefore provide a theoretically motivated account of how we may understand the highly complex, multimodal behaviors inherent in natural social interaction.

What a difference a syllable makes-Rhythmic reading of poetry

In reading conventional poems aloud, the rhythmic experience is coupled with the projection of meter, enabling the prediction of subsequent input. However, it is unclear how top-down and bottom-up processes interact. If the rhythmicity in reading loud is governed by the top-down prediction of metric patterns of weak and strong stress, these should be projected also onto a randomly included, lexically meaningless syllable. If bottom-up information such as the phonetic quality of consecutive syllables plays a functional role in establishing a structured rhythm, the occurrence of the lexically meaningless syllable should affect reading and the number of these syllables in a metrical line should modulate this effect. To investigate this, we manipulated poems by replacing regular syllables at random positions with the syllable "tack". Participants were instructed to read the poems aloud and their voice was recorded during the reading. At the syllable level, we calculated the syllable onset interval (SOI) as a measure of articulation duration, as well as the mean syllable intensity. Both measures were supposed to operationalize how strongly a syllable was stressed. Results show that the average articulation duration of metrically strong regular syllables was longer than for weak syllables. This effect disappeared for "tacks". Syllable intensities, on the other hand, captured metrical stress of "tacks" as well, but only for musically active participants. Additionally, we calculated the normalized pairwise variability index (nPVI) for each line as an indicator for rhythmic contrast, i.e., the alternation between long and short, as well as louder and quieter syllables, to estimate the influence of "tacks" on reading rhythm. For SOI the nPVI revealed a clear negative effect: When "tacks" occurred, lines appeared to be read less altering, and this effect was proportional to the number of tacks per line. For intensity, however, the nPVI did not capture significant effects. Results suggests that top-down prediction does not always suffice to maintain a rhythmic gestalt across a series of syllables that carry little bottom-up prosodic information. Instead, the constant integration of sufficiently varying bottom-up information appears necessary to maintain a stable metrical pattern prediction.

Temporal hierarchies in the predictive processing of melody - From pure tones to songs

Listening to musical melodies is a complex task that engages perceptual and memoryrelated processes. The processes underlying melody cognition happen simultaneously on different timescales, ranging from milliseconds to minutes. Although attempts have been made, research on melody perception is yet to produce a unified framework of how melody processing is achieved in the brain. This may in part be due to the difficulty of integrating concepts such as perception, attention and memory, which pertain to different temporal scales. Recent theories on brain processing, which hold prediction as a fundamental principle, offer potential solutions to this problem and may provide a unifying framework for explaining the neural processes that enable melody perception on multiple temporal levels. In this article, we review empirical evidence for predictive coding on the levels of pitch formation, basic pitch-related auditory patterns,more complex regularity processing extracted from basic patterns and long-term expectations related to musical syntax. We also identify areas that would benefit from further inquiry and suggest future directions in research on musical melody perception.

Hebbian, correlational learning provides a memory-less mechanism for Statistical Learning irrespective of implementational choices: Reply to Tovar and Westermann (2022)

Statistical learning relies on detecting the frequency of co-occurrences of items and has been proposed to be crucial for a variety of learning problems, notably to learn and memorize words from fluent speech. Endress and Johnson (2021) (hereafter EJ) recently showed that such results can be explained based on simple memory-less correlational learning mechanisms such as Hebbian Learning. Tovar and Westermann (2022) (hereafter TW) reproduced these results with a different Hebbian model. We show that the main differences between the models are whether temporal decay acts on both the connection weights and the activations (in TW) or only on the activations (in EJ), and whether interference affects weights (in TW) or activations (in EJ). Given that weights and activations are linked through the Hebbian learning rule, the networks behave similarly. However, in contrast to TW, we do not believe that neurophysiological data are relevant to adjudicate between abstract psychological models with little biological detail. Taken together, both models show that different memory-less correlational learning mechanisms provide a parsimonious account of Statistical Learning results. They are consistent with evidence that Statistical Learning might not allow learners to learn and retain words, and Statistical Learning might support predictive processing instead.

Improvisation is a novel tool to study musicality

Humans spontaneously invent songs from an early age. Here, we exploit this natural inclination to probe implicit musical knowledge in 33 untrained and poor singers (amusia). Each sang 28 long improvisations as a response to a verbal prompt or a continuation of a melodic stem. To assess the extent to which each improvisation reflects tonality, which has been proposed to be a core organizational principle of musicality and which is present within most music traditions, we developed a new algorithm that compares a sung excerpt to a probability density function representing the tonal hierarchy of Western music. The results show signatures of tonality in both nonmusicians and individuals with congenital amusia, who have notorious difficulty performing musical tasks that require explicit responses and memory. The findings are a proof of concept that improvisation can serve as a novel, even enjoyable method for systematically measuring hidden aspects of musicality across the spectrum of musical ability.

Final-note expectancy and humor: an empirical investigation

Background

Melodic expectations were manipulated to investigate the nature of tonally incongruent melodic final notes that may elicit humor in listeners. To our knowledge, this is the first experiment aiming at studying humor elicitation in music with the use of empirical, quantitative methods. To this aim, we have based the experiment on the incongruency/resolution theory of humor and the violations of expectations in music. Our goal was to determine the amount of change, that is, the degree of incongruency required to elicit humor.

Methods

We composed two simple, 8-bar long melodies, and changed their final notes so that they could randomly finish on any semitone between an octave upwards and downwards with respect to the original, tonic final note. This resulted in 25 versions for both melodies, including the original final notes, for each semitone. Musician and non-musician participants rated each version of each melody on five 7-point bipolar scales according to goodness of fit, humor, beauty, playfulness, and pleasantness.

Results and conclusions

Our results showed that even a single change of the final note can elicit humor. No strong connection was found between humor elicitation and the level of incongruency (i.e., the amount of violation of expectation). Instead, changes to the major-mode melody were more likely to be found humorous than those to the minor-mode melody, implying that a so-called playful context is necessary for humor elicitation as the major melody was labelled playful by the listeners. Furthermore, final notes below the original tonic end note were also found to be less humorous and less fitting to the melodic context than those above it.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40359-022-00936-z.

Atonal Music as a Model for Investigating Exploratory Behavior

Atonal music is often characterized by low predictability stemming from the absence of tonal or metrical hierarchies. In contrast, Western tonal music exhibits intrinsic predictability due to its hierarchical structure and therefore, offers a directly accessible predictive model to the listener. In consequence, a specific challenge of atonal music is that listeners must generate a variety of new predictive models. Listeners must not only refrain from applying available tonal models to the heard music, but they must also search for statistical regularities and build new rules that may be related to musical properties other than pitch, such as timbre or dynamics. In this article, we propose that the generation of such new predictive models and the aesthetic experience of atonal music are characterized by internal states related to exploration. This is a behavior well characterized in behavioral neuroscience as fulfilling an innate drive to reduce uncertainty but which has received little attention in empirical music research. We support our proposal with emerging evidence that the hedonic value is associated with the recognition of patterns in low-predictability sound sequences and that atonal music elicits distinct behavioral responses in listeners. We end by outlining new research avenues that might both deepen our understanding of the aesthetic experience of atonal music in particular, and reveal core qualities of the aesthetic experience in general.

Understanding Design Features of Music and Language: The Choric/Dialogic Distinction

Music and spoken language share certain characteristics: both consist of sequences of acoustic elements that are combinatorically combined, and these elements partition the same continuous acoustic dimensions (frequency, formant space and duration). However, the resulting categories differ sharply: scale tones and note durations of small integer ratios appear in music, while speech uses phonemes, lexical tone, and non-isochronous durations. Why did music and language diverge into the two systems we have today, differing in these specific features? We propose a framework based on information theory and a reverse-engineering perspective, suggesting that design features of music and language are a response to their differential deployment along three different continuous dimensions. These include the familiar propositional-aesthetic ('goal') and repetitive-novel ('novelty') dimensions, and a dialogic-choric ('interactivity') dimension that is our focus here. Specifically, we hypothesize that music exhibits specializations enhancing coherent production by several individuals concurrently-the 'choric' context. In contrast, language is specialized for exchange in tightly coordinated turn-taking-'dialogic' contexts. We examine the evidence for our framework, both from humans and non-human animals, and conclude that many proposed design features of music and language follow naturally from their use in distinct dialogic and choric communicative contexts. Furthermore, the hybrid nature of intermediate systems like poetry, chant, or solo lament follows from their deployment in the less typical interactive context.

Cortical activity during naturalistic music listening reflects short-range predictions based on long-term experience

Expectations shape our experience of music. However, the internal model upon which listeners form melodic expectations is still debated. Do expectations stem from Gestalt-like principles or statistical learning? If the latter, does long-term experience play an important role, or are short-term regularities sufficient? And finally, what length of context informs contextual expectations? To answer these questions, we presented human listeners with diverse naturalistic compositions from Western classical music, while recording neural activity using MEG. We quantified note-level melodic surprise and uncertainty using various computational models of music, including a state-of-the-art transformer neural network. A time-resolved regression analysis revealed that neural activity over fronto-temporal sensors tracked melodic surprise particularly around 200ms and 300-500ms after note onset. This neural surprise response was dissociated from sensory-acoustic and adaptation effects. Neural surprise was best predicted by computational models that incorporated long-term statistical learning-rather than by simple, Gestalt-like principles. Yet, intriguingly, the surprise reflected primarily short-range musical contexts of less than ten notes. We present a full replication of our novel MEG results in an openly available EEG dataset. Together, these results elucidate the internal model that shapes melodic predictions during naturalistic music listening.

Rhythmic subvocalization: An eye-tracking study on silent poetry reading

The present study investigates effects of conventionally metered and rhymed poetry on eyemovements in silent reading. Readers saw MRRL poems (i.e., metrically regular, rhymed language) in two layouts. In poem layout, verse endings coincided with line breaks. In prose layout verse endings could be mid-line. We also added metrical and rhyme anomalies. We hypothesized that silently reading MRRL results in building up auditive expectations that are based on a rhythmic “audible gestalt” and propose that rhythmicity is generated through subvocalization. Our results revealed that readers were sensitive to rhythmic-gestalt-anomalies but showed differential effects in poem and prose layouts. Metrical anomalies in particular resulted in robust reading disruptions across a variety of eye-movement measures in the poem layout and caused re-reading of the local context. Rhyme anomalies elicited stronger effects in prose layout and resulted in systematic re-reading of pre-rhymes. The presence or absence of rhythmic-gestalt-anomalies, as well as the layout manipulation, also affected reading in general. Effects of syllable number indicated a high degree of subvocalization. The overall pattern of results suggests that eye-movements reflect, and are closely aligned with, the rhythmic subvocalization of MRRL.

This study introduces a two-stage approach to the analysis of long MRRL stimuli and contributes to the discussion of how the processing of rhythm in music and speech may overlap.

Pitches that Wire Together Fire Together: Scale Degree Associations Across Time Predict Melodic Expectations

The ongoing generation of expectations is fundamental to listeners' experience of music, but research into types of statistical information that listeners extract from musical melodies has tended to emphasize transition probabilities and n-grams, with limited consideration given to other types of statistical learning that may be relevant. Temporal associations between scale degrees represent a different type of information present in musical melodies that can be learned from musical corpora using expectation networks, a computationally simple method based on activation and decay. Expectation networks infer the expectation of encountering one scale degree followed in the near (but not necessarily immediate) future by another given scale degree, with previous work suggesting that scale degree associations learned by expectation networks better predict listener ratings of pitch similarity than transition probabilities. The current work outlines how these learned scale degree associations can be combined to predict melodic continuations and tests the resulting predictions on a dataset of listener responses to a musical cloze task previously used to compare two other models of melodic expectation, a variable-order Markov model (IDyOM) and Temperley's music-theoretically motivated model. Under multinomial logistic regression, all three models explain significant unique variance in human melodic expectations, with coefficient estimates highest for expectation networks. These results suggest that generalized scale degree associations informed by both adjacent and nonadjacent relationships between melodic notes influence listeners' melodic predictions above and beyond n-gram context, highlighting the need to consider a broader range of statistical learning processes that may underlie listeners' expectations for upcoming musical events.

Predictive Uncertainty Underlies Auditory Boundary Perception

Anticipating the future is essential for efficient perception and action planning. Yet the role of anticipation in event segmentation is understudied because empirical research has focused on retrospective cues such as surprise. We address this concern in the context of perception of musical-phrase boundaries. A computational model of cognitive sequence processing was used to control the information-dynamic properties of tone sequences. In an implicit, self-paced listening task (N = 38), undergraduates dwelled longer on tones generating high entropy (i.e., high uncertainty) than on those generating low entropy (i.e., low uncertainty). Similarly, sequences that ended on tones generating high entropy were rated as sounding more complete (N = 31 undergraduates). These entropy effects were independent of both the surprise (i.e., information content) and phrase position of target tones in the original musical stimuli. Our results indicate that events generating high entropy prospectively contribute to segmentation processes in auditory sequence perception, independently of the properties of the subsequent event.

Asymmetry in scales enhances learning of new musical structures

Despite the remarkable variability music displays across cultures, certain recurrent musical features motivate the hypothesis that fundamental cognitive principles constrain the way music is produced. One such feature concerns the structure of musical scales. The vast majority of musical cultures use scales that are not uniformly symmetric-that is, scales that contain notes spread unevenly across the octave. Here we present evidence that the structure of musical scales has a substantial impact on how listeners learn new musical systems. Three experiments were conducted to test the hypothesis that nonuniformity facilitates the processing of melodies. Novel melodic stimuli were composed based on artificial grammars using scales with different levels of symmetry. Experiment 1 tested the acquisition of tonal hierarchies and melodic regularities on three different 12-tone equal-tempered scales using a finite-state grammar. Experiments 2 and 3 used more flexible Markov-chain grammars and were designed to generalize the effect to 14-tone and 16-tone equal-tempered scales. The results showed that performance was significantly enhanced by scale structures that specified the tonal space by providing unique intervallic relations between notes. These results suggest that the learning of novel musical systems is modulated by the symmetry of scales, which in turn may explain the prevalence of nonuniform scales across musical cultures.

Higher-Order Musical Temporal Structure in Bird Song

Bird songs often display musical acoustic features such as tonal pitch selection, rhythmicity, and melodic contouring. We investigated higher-order musical temporal structure in bird song using an experimental method called “music scrambling” with human subjects. Recorded songs from a phylogenetically diverse group of 20 avian taxa were split into constituent elements (“notes” or “syllables”) and recombined in original and random order. Human subjects were asked to evaluate which version sounded more “musical” on a per-species basis. Species identity and stimulus treatment were concealed from subjects, and stimulus presentation order was randomized within and between taxa. Two recordings of human music were included as a control for attentiveness. Participants varied in their assessments of individual species musicality, but overall they were significantly more likely to rate bird songs with original temporal sequence as more musical than those with randomized temporal sequence. We discuss alternative hypotheses for the origins of avian musicality, including honest signaling, perceptual bias, and arbitrary aesthetic coevolution.

Early cortical processing of pitch height and the role of adaptation and musicality

Pitch is an important perceptual feature; however, it is poorly understood how its cortical correlates are shaped by absolute vs relative fundamental frequency (f₀), and by neural adaptation. In this study, we assessed transient and sustained auditory evoked fields (AEFs) at the onset, progression, and offset of short pitch height sequences, taking into account the listener's musicality. We show that neuromagnetic activity reflects absolute f₀ at pitch onset and offset, and relative f₀ at transitions within pitch sequences; further, sequences with fixed f₀ lead to larger response suppression than sequences with variable f₀ contour, and to enhanced offset activity. Musical listeners exhibit stronger f₀-related AEFs and larger differences between their responses to fixed vs variable sequences, both within sequences and at pitch offset. The results resemble prominent psychoacoustic phenomena in the perception of pitch contours; moreover, they suggest a strong influence of adaptive mechanisms on cortical pitch processing which, in turn, might be modulated by a listener's musical expertise.

Does unfairness sound wrong? A cross-domain investigation of expectations in music and social decision-making

This study was interested in investigating the existence of a shared psychological mechanism for the processing of expectations across domains. The literature on music and language shows that violations of expectations produce similar neural responses and violating the expectation in one domain may influence the processing of stimuli in the other domain. Like music and language, our social world is governed by a system of inherent rules or norms, such as fairness. The study therefore aimed to draw a parallel to the social domain and investigate whether a manipulation of melodic expectation can influence the processing of higher-level expectations of fairness. Specifically, we aimed to investigate whether the presence of an unexpected melody enhances or reduces participants' sensitivity to the violations of fairness and the behavioural reactions associated with these. We embedded a manipulation of melodic expectation within a social decision-making paradigm, whereby musically expected and unexpected stimuli will be simultaneously presented with fair and unfair divisions in a third-party altruistic punishment game. Behavioural and electroencephalographic responses were recorded. Results from the pre-planned analyses show that participants are less likely to punish when melodies are more unexpected and that violations of fairness norms elicit medial frontal negativity (MFN)-life effects. Because no significant interactions between melodic expectancy and fairness of the division were found, results fail to provide evidence of a shared mechanism for the processing of expectations. Exploratory analyses show two additional effects: (i) unfair divisions elicit an early attentional component (P2), probably associated with stimulus saliency, and (ii) mid-value divisions elicit a late MFN-like component, probably reflecting stimulus ambiguity. Future studies could build on these results to further investigate the effect of the cross-domain influence of music on the processing of social stimuli on these early and late components.

Decomposing neural responses to melodic surprise in musicians and non-musicians: Evidence for a hierarchy of predictions in the auditory system

Neural responses to auditory surprise are typically studied with highly unexpected, disruptive sounds. Consequently, little is known about auditory prediction in everyday contexts that are characterized by fine-grained, non-disruptive fluctuations of auditory surprise. To address this issue, we used IDyOM, a computational model of auditory expectation, to obtain continuous surprise estimates for a set of newly composed melodies. Our main goal was to assess whether the neural correlates of non-disruptive surprising sounds in a musical context are affected by musical expertise. Using magnetoencephalography (MEG), auditory responses were recorded from musicians and non-musicians while they listened to the melodies. Consistent with a previous study, the amplitude of the N1m component increased with higher levels of computationally estimated surprise. This effect, however, was not different between the two groups. Further analyses offered an explanation for this finding: Pitch interval size itself, rather than probabilistic prediction, was responsible for the modulation of the N1m, thus pointing to low-level sensory adaptation as the underlying mechanism. In turn, the formation of auditory regularities and proper probabilistic prediction were reflected in later components: The mismatch negativity (MMNm) and the P3am, respectively. Overall, our findings reveal a hierarchy of expectations in the auditory system and highlight the need to properly account for sensory adaptation in research addressing statistical learning.

Aggregate context effects in music processing

Control of stimulus confounds is an ever-present, and ever-important, aspect of experimental design. Typically, researchers concern themselves with such control on a local level, ensuring that individual stimuli contain only the properties they intend for them to represent. Significantly less attention, however, is paid to stimulus properties in the aggregate, aspects that, although not present in individual stimuli, can nevertheless become emergent properties of the stimulus set when viewed in total. This paper describes two examples of such effects. The first (Case Study 1) focuses on emergent properties of pairs of to-be-performed tones on a piano keyboard, and the second (Case Study 2) focuses on emergent properties of short, atonal melodies in a perception/memory task. In both cases these sets of stimuli induced identifiable tonal influences despite being explicitly created to be devoid of musical tonality. These results highlight the importance of monitoring aggregate stimulus properties in one's research, and are discussed with reference to their implications for interpreting psychological findings quite generally.

Cortical encoding of melodic expectations in human temporal cortex

Humans engagement in music rests on underlying elements such as the listeners' cultural background and interest in music. These factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing. Here we recorded cortical signals as participants listened to Bach melodies. We assessed the relative contributions of acoustic versus melodic components of the music to the neural signal. Melodic features included information on pitch progressions and their tempo, which were extracted from a predictive model of musical structure based on Markov chains. We related the music to brain activity with temporal response functions demonstrating, for the first time, distinct cortical encoding of pitch and note-onset expectations during naturalistic music listening. This encoding was most pronounced at response latencies up to 350 ms, and in both planum temporale and Heschl's gyrus.

The prevalence and importance of statistical learning in human cognition and behavior

Statistical learning, the ability to extract regularities from the environment over time, has become a topic of burgeoning interest. Its influence on behavior, spanning infancy to adulthood, has been demonstrated across a range of tasks, both those labeled as tests of statistical learning and those from other learning domains that predated statistical learning research or that are not typically considered in the context of that literature. Given this pervasive role in human cognition, statistical learning has the potential to reconcile seemingly distinct learning phenomena and may be an under-appreciated but important contributor to a wide range of human behaviors that are studied as unrelated processes, such as episodic memory and spatial navigation.