Brain potentials predict learning, transmission and modification of an artificial symbolic system

Brain responses to a lab-evolved artificial language with space-time metaphors

What is the connection between the cultural evolution of a language and the rapid processing response to that language in the brains of individual learners? In an iterated communication study that was conducted previously, participants were asked to communicate temporal concepts such as "tomorrow," "day after," "year," and "past" using vertical movements recorded on a touch screen. Over time, participants developed simple artificial 'languages' that used space metaphorically to communicate in nuanced ways about time. Some conventions appeared rapidly and universally (e.g., using larger vertical movements to convey greater temporal durations). Other conventions required extensive social interaction and exhibited idiosyncratic variation (e.g., using vertical location to convey past or future). Here we investigate whether the brain's response during acquisition of such a language reflects the process by which the language's conventions originally evolved. We recorded participants' EEG as they learned one of these artificial space-time languages. Overall, the brain response to this artificial communication system was language-like, with, for instance, violations to the system's conventions eliciting an N400-like component. Over the course of learning, participants' brain responses developed in ways that paralleled the process by which the language had originally evolved, with early neural sensitivity to violations of a rapidly-evolving universal convention, and slowly developing neural sensitivity to an idiosyncratic convention that required slow social negotiation to emerge. This study opens up exciting avenues of future work to disentangle how neural biases influence learning and transmission in the emergence of structure in language.

From One Bilingual to the Next: An Iterated Learning Study on Language Evolution in Bilingual Societies

Studies of language evolution in the lab have used the iterated learning paradigm to show how linguistic structure emerges through cultural transmission-repeated cycles of learning and use across generations of speakers . However, agent-based simulations suggest that prior biases crucially impact the outcome of cultural transmission. Here, we explored this notion through an iterated learning study of English-French bilingual adults (mostly sequential bilinguals dominant in English). Each participant learned two unstructured artificial languages in a counterbalanced fashion, one resembling English, another resembling French at the phono-orthographic level. The output of each participant was passed down to the next participant, forming diffusion chains of 10 generations per language. We hypothesized that artificial languages would become easier to learn and exhibit greater structure when they were aligned with participants' bilingual experience (i.e., English languages being easier to learn overall), or as a function of practice (i.e., languages learned second being easier to learn overall). Instead, we found that English-like languages became more structured over generations, but only when they were learned first. In contrast, French-like languages became more structured regardless of the order of learning, suggesting the presence of an asymmetric switch cost during artificial language learning. Moreover, individual differences in language usage modulated the amount of structure produced by the participants. Overall, these data suggest that bilingual experience impacts how novel languages are learned at an individual level, which can then scale up to cultural transmission of novel language at a group level.

High-fidelity transmission of auditory symbolic material is associated with reduced right-left neuroanatomical asymmetry between primary auditory regions

The intergenerational stability of auditory symbolic systems, such as music, is thought to rely on brain processes that allow the faithful transmission of complex sounds. Little is known about the functional and structural aspects of the human brain which support this ability, with a few studies pointing to the bilateral organization of auditory networks as a putative neural substrate. Here, we further tested this hypothesis by examining the role of left-right neuroanatomical asymmetries between auditory cortices. We collected neuroanatomical images from a large sample of participants (nonmusicians) and analyzed them with Freesurfer's surface-based morphometry method. Weeks after scanning, the same individuals participated in a laboratory experiment that simulated music transmission: the signaling games. We found that high accuracy in the intergenerational transmission of an artificial tone system was associated with reduced rightward asymmetry of cortical thickness in Heschl's sulcus. Our study suggests that the high-fidelity copying of melodic material may rely on the extent to which computational neuronal resources are distributed across hemispheres. Our data further support the role of interhemispheric brain organization in the cultural transmission and evolution of auditory symbolic systems.

Signaling games and music as a credible signal

The argument by Mehr et al. that music emerged and evolved culturally as a credible signal is convincing, but it lacks one essential ingredient: a model of signaling behavior that supports the main hypothesis theoretically and empirically. We argue that signaling games can help us explain how musical structures emerge as population-level phenomena, through sender-receiver signaling interactions.

Melodic Universals Emerge or Are Sustained Through Cultural Evolution

To understand why music is structured the way it is, we need an explanation that accounts for both the universality and variability found in musical traditions. Here we test whether statistical universals that have been identified for melodic structures in music can emerge as a result of cultural adaptation to human biases through iterated learning. We use data from an experiment in which artificial whistled systems, where sounds produced with a slide whistle were learned by human participants and transmitted multiple times from person to person. These sets of whistled signals needed to be memorised and recalled and the reproductions of one participant were used as the input set for the next. We tested for the emergence of seven different melodic features, such as discrete pitches, motivic patterns, or phrase repetition, and found some evidence for the presence of most of these statistical universals. We interpret this as promising evidence that, similarly to rhythmic universals, iterated learning experiments can also unearth melodic statistical universals. More, ideally cross-cultural, experiments are nonetheless needed. Simulating the cultural transmission of artificial proto-musical systems can help unravel the origins of universal tendencies in musical structures.

White matter variability in auditory callosal pathways contributes to variation in the cultural transmission of auditory symbolic systems

The cultural transmission of spoken language and music relies on human capacities for encoding and recalling auditory patterns. In this experiment, we show that interindividual differences in this ability are associated with variation in the organization of cross-callosal white matter pathways. First, high-angular resolution diffusion MRI (dMRI) data were analyzed in a large participant sample (N = 51). Subsequently, these participants underwent a behavioral test that models in the laboratory the cultural transmission of auditory symbolic systems: the signaling game. Cross-callosal and intrahemispheric (arcuate fasciculus) pathways were reconstructed and analyzed using conventional diffusion tensor imaging (DTI) as well as a more advanced dMRI technique: fixel-based analysis (FBA). The DTI metric of fractional anisotropy (FA) in auditory callosal pathways predicted-weeks after scanning-the fidelity of transmission of an artificial tone system. The ability to coherently transmit auditory signals in one signaling game, irrespective of the signals learned during the previous game, was predicted by morphological properties of the fiber bundles in the most anterior portions of the corpus callosum. The current study is the first application of dMRI in the field of cultural transmission, and the first to connect individual characteristics of callosal pathways to core behaviors in the transmission of auditory symbolic systems.

Extracting human cortical responses to sound onsets and acoustic feature changes in real music, and their relation to event rate

Evoked cortical responses (ERs) have mainly been studied in controlled experiments using simplified stimuli. Though, an outstanding question is how the human cortex responds to the complex stimuli encountered in realistic situations. Few electroencephalography (EEG) studies have used Music Information Retrieval (MIR) tools to extract cortical P1/N1/P2 to acoustical changes in real music. However, less than ten events per music piece could be detected leading to ERs due to limitations in automatic detection of sound onsets. Also, the factors influencing a successful extraction of the ERs have not been identified. Finally, previous studies did not localize the sources of the cortical generators. This study is based on an EEG/MEG dataset from 48 healthy normal hearing participants listening to three real music pieces. Acoustic features were computed from the audio signal of the music with the MIR Toolbox. To overcome limits in automatic methods, sound onsets were also manually detected. The chance of obtaining detectable ERs based on ten randomly picked onset points was less than 1:10,000. For the first time, we show that naturalistic P1/N1/P2 ERs can be reliably measured across 100 manually identified sound onsets, substantially improving the signal-to-noise level compared to <10 trials. More ERs were measurable in musical sections with slow event rates (0.2 Hz-2.5 Hz) than with fast event rates (>2.5 Hz). Furthermore, during monophonic sections of the music only P1/P2 were measurable, and during polyphonic sections only N1. Finally, MEG source analysis revealed that naturalistic P2 is located in core areas of the auditory cortex.

Perceptual learning of tone patterns changes the effective connectivity between Heschl's gyrus and planum temporale

Learning of complex auditory sequences such as music can be thought of as optimizing an internal model of regularities through unpredicted events (or “prediction errors”). We used dynamic causal modeling (DCM) and parametric empirical Bayes on functional magnetic resonance imaging (fMRI) data to identify modulation of effective brain connectivity that takes place during perceptual learning of complex tone patterns. Our approach differs from previous studies in two aspects. First, we used a complex oddball paradigm based on tone patterns as opposed to simple deviant tones. Second, the use of fMRI allowed us to identify cortical regions with high spatial accuracy. These regions served as empirical regions‐of‐interest for the analysis of effective connectivity. Deviant patterns induced an increased blood oxygenation level‐dependent response, compared to standards, in early auditory (Heschl's gyrus [HG]) and association auditory areas (planum temporale [PT]) bilaterally. Within this network, we found a left‐lateralized increase in feedforward connectivity from HG to PT during deviant responses and an increase in excitation within left HG. In contrast to previous findings, we did not find frontal activity, nor did we find modulations of backward connections in response to oddball sounds. Our results suggest that complex auditory prediction errors are encoded by changes in feedforward and intrinsic connections, confined to superior temporal gyrus.

Inter-subject Similarity of Brain Activity in Expert Musicians After Multimodal Learning: A Behavioral and Neuroimaging Study on Learning to Play a Piano Sonata

Human behavior is inherently multimodal and relies on sensorimotor integration. This is evident when pianists exhibit activity in motor and premotor cortices, as part of a dorsal pathway, while listening to a familiar piece of music, or when naïve participants learn to play simple patterns on the piano. Here we investigated the interaction between multimodal learning and dorsal-stream activity over the course of four weeks in ten skilled pianists by adopting a naturalistic data-driven analysis approach. We presented the pianists with audio-only, video-only and audiovisual recordings of a piano sonata during functional magnetic resonance imaging (fMRI) before and after they had learned to play the sonata by heart for a total of four weeks. We followed the learning process and its outcome with questionnaires administered to the pianists, one piano instructor following their training, and seven external expert judges. The similarity of the pianists' brain activity during stimulus presentations was examined before and after learning by means of inter-subject correlation (ISC) analysis. After learning, an increased ISC was found in the pianists while watching the audiovisual performance, particularly in motor and premotor regions of the dorsal stream. While these brain structures have previously been associated with learning simple audio-motor sequences, our findings are the first to suggest their involvement in learning a complex and demanding audiovisual-motor task. Moreover, the most motivated learners and the best performers of the sonata showed ISC in the dorsal stream and in the reward brain network.

Functional connectivity in human auditory networks and the origins of variation in the transmission of musical systems

Music producers, whether original composers or performers, vary in their ability to acquire and faithfully transmit music. This form of variation may serve as a mechanism for the emergence of new traits in musical systems. In this study, we aim to investigate whether individual differences in the social learning and transmission of music relate to intrinsic neural dynamics of auditory processing systems. We combined auditory and resting-state functional magnetic resonance imaging (fMRI) with an interactive laboratory model of cultural transmission, the signaling game, in an experiment with a large cohort of participants (N=51). We found that the degree of interhemispheric rs-FC within fronto-temporal auditory networks predicts—weeks after scanning—learning, transmission, and structural modification of an artificial tone system. Our study introduces neuroimaging in cultural transmission research and points to specific neural auditory processing mechanisms that constrain and drive variation in the cultural transmission and regularization of musical systems.

From random to regular: neural constraints on the emergence of isochronous rhythm during cultural transmission

A core design feature of human communication systems and expressive behaviours is their temporal organization. The cultural evolutionary origins of this feature remain unclear. Here, we test the hypothesis that regularities in the temporal organization of signalling sequences arise in the course of cultural transmission as adaptations to aspects of cortical function. We conducted two experiments on the transmission of rhythms associated with affective meanings, focusing on one of the most widespread forms of regularity in language and music: isochronicity. In the first experiment, we investigated how isochronous rhythmic regularities emerge and change in multigenerational signalling games, where the receiver (learner) in a game becomes the sender (transmitter) in the next game. We show that signalling sequences tend to become rhythmically more isochronous as they are transmitted across generations. In the second experiment, we combined electroencephalography (EEG) and two-player signalling games over 2 successive days. We show that rhythmic regularization of sequences can be predicted based on the latencies of the mismatch negativity response in a temporal oddball paradigm. These results suggest that forms of isochronicity in communication systems originate in neural constraints on information processing, which may be expressed and amplified in the course of cultural transmission.

Which Melodic Universals Emerge from Repeated Signaling Games? A Note on Lumaca and Baggio (2017) ‡

Music is a peculiar human behavior, yet we still know little as to why and how music emerged. For centuries, the study of music has been the sole prerogative of the humanities. Lately, however, music is being increasingly investigated by psychologists, neuroscientists, biologists, and computer scientists. One approach to studying the origins of music is to empirically test hypotheses about the mechanisms behind this structured behavior. Recent lab experiments show how musical rhythm and melody can emerge via the process of cultural transmission. In particular, Lumaca and Baggio (2017) tested the emergence of a sound system at the boundary between music and language. In this study, participants were given random pairs of signal-meanings; when participants negotiated their meaning and played a "game of telephone" with them, these pairs became more structured and systematic. Over time, the small biases introduced in each artificial transmission step accumulated, displaying quantitative trends, including the emergence, over the course of artificial human generations, of features resembling properties of language and music. In this Note, we highlight the importance of Lumaca and Baggio's experiment, place it in the broader literature on the evolution of language and music, and suggest refinements for future experiments. We conclude that, while psychological evidence for the emergence of proto-musical features is accumulating, complementary work is needed: Mathematical modeling and computer simulations should be used to test the internal consistency of experimentally generated hypotheses and to make new predictions.

Signaling Games and the Evolution of Structure in Language and Music: A Reply to Ravignani and Verhoef (2018) ‡

In their commentary on our work, Ravignani and Verhoef (2018) raise concerns about two methodological aspects of our experimental paradigm (the signaling game): (1) the use of melodic signals of fixed length and duration, and (2) the fact that signals are endowed with meaning. They argue that music is hardly a semantic system and that our methodological choices may limit the capacity of our paradigm to shed light on the emergence and evolution of a number of putative musical universals. We reply that musical systems are semantic systems and that the aim of our research is not to study musical universals as such, but to compare more closely the kinds of principles that organize meaning and structure in linguistic and musical systems.

Music Evolution in the Laboratory: Cultural Transmission Meets Neurophysiology

In recent years, there has been renewed interest in the biological and cultural evolution of music, and specifically in the role played by perceptual and cognitive factors in shaping core features of musical systems, such as melody, harmony, and rhythm. One proposal originates in the language sciences. It holds that aspects of musical systems evolve by adapting gradually, in the course of successive generations, to the structural and functional characteristics of the sensory and memory systems of learners and “users” of music. This hypothesis has found initial support in laboratory experiments on music transmission. In this article, we first review some of the most important theoretical and empirical contributions to the field of music evolution. Next, we identify a major current limitation of these studies, i.e., the lack of direct neural support for the hypothesis of cognitive adaptation. Finally, we discuss a recent experiment in which this issue was addressed by using event-related potentials (ERPs). We suggest that the introduction of neurophysiology in cultural transmission research may provide novel insights on the micro-evolutionary origins of forms of variation observed in cultural systems.

Evolving building blocks of rhythm: how human cognition creates music via cultural transmission

Why does musical rhythm have the structure it does? Musical rhythm, in all its cross-cultural diversity, exhibits commonalities across world cultures. Traditionally, music research has been split into two fields. Some scientists focused on musicality, namely the human biocognitive predispositions for music, with an emphasis on cross-cultural similarities. Other scholars investigated music, seen as a cultural product, focusing on the variation in world musical cultures. Recent experiments found deep connections between music and musicality, reconciling these opposing views. Here, we address the question of how individual cognitive biases affect the process of cultural evolution of music. Data from two experiments are analyzed using two complementary techniques. In the experiments, participants hear drumming patterns and imitate them. These patterns are then given to the same or another participant to imitate. The structure of these initially random patterns is tracked along experimental "generations." Frequentist statistics show how participants' biases are amplified by cultural transmission, making drumming patterns more structured. Structure is achieved faster in transmission within rather than between participants. A Bayesian model approximates the motif structures participants learned and created. Our data and models suggest that individual biases for musicality may shape the cultural transmission of musical rhythm.

Developmental Constraints on Learning Artificial Grammars with Fixed, Flexible and Free Word Order

Human learning, although highly flexible and efficient, is constrained in ways that facilitate or impede the acquisition of certain systems of information. Some such constraints, active during infancy and childhood, have been proposed to account for the apparent ease with which typically developing children acquire language. In a series of experiments, we investigated the role of developmental constraints on learning artificial grammars with a distinction between shorter and relatively frequent words ('function words,' F-words) and longer and less frequent words ('content words,' C-words). We constructed 4 finite-state grammars, in which the order of F-words, relative to C-words, was either fixed (F-words always occupied the same positions in a string), flexible (every F-word always followed a C-word), or free. We exposed adults (N = 84) and kindergarten children (N = 100) to strings from each of these artificial grammars, and we assessed their ability to recognize strings with the same structure, but a different vocabulary. Adults were better at recognizing strings when regularities were available (i.e., fixed and flexible order grammars), while children were better at recognizing strings from the grammars consistent with the attested distribution of function and content words in natural languages (i.e., flexible and free order grammars). These results provide evidence for a link between developmental constraints on learning and linguistic typology.

Cultural Transmission and Evolution of Melodic Structures in Multi-generational Signaling Games

It has been proposed that languages evolve by adapting to the perceptual and cognitive constraints of the human brain, developing, in the course of cultural transmission, structural regularities that maximize or optimize learnability and ease of processing. To what extent would perceptual and cognitive constraints similarly affect the evolution of musical systems? We conducted an experiment on the cultural evolution of artificial melodic systems, using multi-generational signaling games as a laboratory model of cultural transmission. Signaling systems, using five-tone sequences as signals, and basic and compound emotions as meanings, were transmitted from senders to receivers along diffusion chains in which the receiver in each game became the sender in the next game. During transmission, structural regularities accumulated in the signaling systems, following principles of proximity, symmetry, and good continuation. Although the compositionality of signaling systems did not increase significantly across generations, we did observe a significant increase in similarity among signals from the same set. We suggest that our experiment tapped into the cognitive and perceptual constraints operative in the cultural evolution of musical systems, which may differ from the mechanisms at play in language evolution and change.