1
|
Hu M, Bianco R, Hidalgo AR, Chait M. Concurrent Encoding of Sequence Predictability and Event-Evoked Prediction Error in Unfolding Auditory Patterns. J Neurosci 2024; 44:e1894232024. [PMID: 38350998 PMCID: PMC10993036 DOI: 10.1523/jneurosci.1894-23.2024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 02/02/2024] [Accepted: 02/06/2024] [Indexed: 03/26/2024] Open
Abstract
Human listeners possess an innate capacity to discern patterns within rapidly unfolding sensory input. Core questions, guiding ongoing research, focus on the mechanisms through which these representations are acquired and whether the brain prioritizes or suppresses predictable sensory signals. Previous work, using fast auditory sequences (tone-pips presented at a rate of 20 Hz), revealed sustained response effects that appear to track the dynamic predictability of the sequence. Here, we extend the investigation to slower sequences (4 Hz), permitting the isolation of responses to individual tones. Stimuli were 50 ms tone-pips, ordered into random (RND) and regular (REG; a repeating pattern of 10 frequencies) sequences; Two timing profiles were created: in "fast" sequences, tone-pips were presented in direct succession (20 Hz); in "slow" sequences, tone-pips were separated by a 200 ms silent gap (4 Hz). Naive participants (N = 22; both sexes) passively listened to these sequences, while brain responses were recorded using magnetoencephalography (MEG). Results unveiled a heightened magnitude of sustained brain responses in REG when compared to RND patterns. This manifested from three tones after the onset of the pattern repetition, even in the context of slower sequences characterized by extended pattern durations (2,500 ms). This observation underscores the remarkable implicit sensitivity of the auditory brain to acoustic regularities. Importantly, brain responses evoked by single tones exhibited the opposite pattern-stronger responses to tones in RND than REG sequences. The demonstration of simultaneous but opposing sustained and evoked response effects reveals concurrent processes that shape the representation of unfolding auditory patterns.
Collapse
Affiliation(s)
- Mingyue Hu
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| | - Roberta Bianco
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
- Neuroscience of Perception & Action Lab, Italian Institute of Technology (IIT), Rome 00161, Italy
| | | | - Maria Chait
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| |
Collapse
|
2
|
Orekhova EV, Fadeev KA, Goiaeva DE, Obukhova TS, Ovsiannikova TM, Prokofyev AO, Stroganova TA. Different hemispheric lateralization for periodicity and formant structure of vowels in the auditory cortex and its changes between childhood and adulthood. Cortex 2024; 171:287-307. [PMID: 38061210 DOI: 10.1016/j.cortex.2023.10.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 08/31/2023] [Accepted: 10/30/2023] [Indexed: 02/12/2024]
Abstract
The spectral formant structure and periodicity pitch are the major features that determine the identity of vowels and the characteristics of the speaker. However, very little is known about how the processing of these features in the auditory cortex changes during development. To address this question, we independently manipulated the periodicity and formant structure of vowels while measuring auditory cortex responses using magnetoencephalography (MEG) in children aged 7-12 years and adults. We analyzed the sustained negative shift of source current associated with these vowel properties, which was present in the auditory cortex in both age groups despite differences in the transient components of the auditory response. In adults, the sustained activation associated with formant structure was lateralized to the left hemisphere early in the auditory processing stream requiring neither attention nor semantic mapping. This lateralization was not yet established in children, in whom the right hemisphere contribution to formant processing was strong and decreased during or after puberty. In contrast to the formant structure, periodicity was associated with a greater response in the right hemisphere in both children and adults. These findings suggest that left-lateralization for the automatic processing of vowel formant structure emerges relatively late in ontogenesis and pose a serious challenge to current theories of hemispheric specialization for speech processing.
Collapse
Affiliation(s)
- Elena V Orekhova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Kirill A Fadeev
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Dzerassa E Goiaeva
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana S Obukhova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana M Ovsiannikova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Andrey O Prokofyev
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana A Stroganova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| |
Collapse
|
3
|
Háden GP, Bouwer FL, Honing H, Winkler I. Beat processing in newborn infants cannot be explained by statistical learning based on transition probabilities. Cognition 2024; 243:105670. [PMID: 38016227 DOI: 10.1016/j.cognition.2023.105670] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2023] [Revised: 11/08/2023] [Accepted: 11/17/2023] [Indexed: 11/30/2023]
Abstract
Newborn infants have been shown to extract temporal regularities from sound sequences, both in the form of learning regular sequential properties, and extracting periodicity in the input, commonly referred to as a regular pulse or the 'beat'. However, these two types of regularities are often indistinguishable in isochronous sequences, as both statistical learning and beat perception can be elicited by the regular alternation of accented and unaccented sounds. Here, we manipulated the isochrony of sound sequences in order to disentangle statistical learning from beat perception in sleeping newborn infants in an EEG experiment, as previously done in adults and macaque monkeys. We used a binary accented sequence that induces a beat when presented with isochronous timing, but not when presented with randomly jittered timing. We compared mismatch responses to infrequent deviants falling on either accented or unaccented (i.e., odd and even) positions. Results showed a clear difference between metrical positions in the isochronous sequence, but not in the equivalent jittered sequence. This suggests that beat processing is present in newborns. Despite previous evidence for statistical learning in newborns the effects of this ability were not detected in the jittered condition. These results show that statistical learning by itself does not fully explain beat processing in newborn infants.
Collapse
Affiliation(s)
- Gábor P Háden
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Magyar tudósok körútja 2, H-1117 Budapest, Hungary; Department of Telecommunications and Media Informatics, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Magyar tudósok körútja 2, 1117 Budapest, Hungary.
| | - Fleur L Bouwer
- Music Cognition Group, Institute for Logic, Language, and Computation, University of Amsterdam, P.O. Box 94242, 1090 GE Amsterdam, the Netherlands; Amsterdam Brain and Cognition, University of Amsterdam, P.O. Box 15900, 1001 NK Amsterdam, the Netherlands; Department of Psychology, Brain & Cognition, University of Amsterdam, P.O. Box 15900, 1001 NK Amsterdam, the Netherlands; Cognitive Psychology Unit, Institute of Psychology & Leiden Institute for Brain and Cognition, Leiden University, 2333 AK Leiden, the Netherlands.
| | - Henkjan Honing
- Music Cognition Group, Institute for Logic, Language, and Computation, University of Amsterdam, P.O. Box 94242, 1090 GE Amsterdam, the Netherlands; Amsterdam Brain and Cognition, University of Amsterdam, P.O. Box 15900, 1001 NK Amsterdam, the Netherlands.
| | - István Winkler
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Magyar tudósok körútja 2, H-1117 Budapest, Hungary.
| |
Collapse
|
4
|
Bianco R, Zuk NJ, Bigand F, Quarta E, Grasso S, Arnese F, Ravignani A, Battaglia-Mayer A, Novembre G. Neural encoding of musical expectations in a non-human primate. Curr Biol 2024; 34:444-450.e5. [PMID: 38176416 DOI: 10.1016/j.cub.2023.12.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Revised: 10/26/2023] [Accepted: 12/07/2023] [Indexed: 01/06/2024]
Abstract
The appreciation of music is a universal trait of humankind.1,2,3 Evidence supporting this notion includes the ubiquity of music across cultures4,5,6,7 and the natural predisposition toward music that humans display early in development.8,9,10 Are we musical animals because of species-specific predispositions? This question cannot be answered by relying on cross-cultural or developmental studies alone, as these cannot rule out enculturation.11 Instead, it calls for cross-species experiments testing whether homologous neural mechanisms underlying music perception are present in non-human primates. We present music to two rhesus monkeys, reared without musical exposure, while recording electroencephalography (EEG) and pupillometry. Monkeys exhibit higher engagement and neural encoding of expectations based on the previously seeded musical context when passively listening to real music as opposed to shuffled controls. We then compare human and monkey neural responses to the same stimuli and find a species-dependent contribution of two fundamental musical features-pitch and timing12-in generating expectations: while timing- and pitch-based expectations13 are similarly weighted in humans, monkeys rely on timing rather than pitch. Together, these results shed light on the phylogeny of music perception. They highlight monkeys' capacity for processing temporal structures beyond plain acoustic processing, and they identify a species-dependent contribution of time- and pitch-related features to the neural encoding of musical expectations.
Collapse
Affiliation(s)
- Roberta Bianco
- Neuroscience of Perception & Action Lab, Italian Institute of Technology, Viale Regina Elena 291, 00161 Rome, Italy.
| | - Nathaniel J Zuk
- Department of Psychology, Nottingham Trent University, 50 Shakespeare Street, Nottingham NG1 4FQ, UK
| | - Félix Bigand
- Neuroscience of Perception & Action Lab, Italian Institute of Technology, Viale Regina Elena 291, 00161 Rome, Italy
| | - Eros Quarta
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Stefano Grasso
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Flavia Arnese
- Neuroscience of Perception & Action Lab, Italian Institute of Technology, Viale Regina Elena 291, 00161 Rome, Italy
| | - Andrea Ravignani
- Comparative Bioacoustics Group, Max Planck Institute for Psycholinguistics, Wundtlaan 1, 6525 XD Nijmegen, the Netherlands; Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Universitetsbyen 3, 8000 Aarhus, Denmark; Department of Human Neurosciences, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Alexandra Battaglia-Mayer
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Giacomo Novembre
- Neuroscience of Perception & Action Lab, Italian Institute of Technology, Viale Regina Elena 291, 00161 Rome, Italy.
| |
Collapse
|
5
|
Albury AW, Bianco R, Gold BP, Penhune VB. Context changes judgments of liking and predictability for melodies. Front Psychol 2023; 14:1175682. [PMID: 38034280 PMCID: PMC10684779 DOI: 10.3389/fpsyg.2023.1175682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 10/23/2023] [Indexed: 12/02/2023] Open
Abstract
Predictability plays an important role in the experience of musical pleasure. By leveraging expectations, music induces pleasure through tension and surprise. However, musical predictions draw on both prior knowledge and immediate context. Similarly, musical pleasure, which has been shown to depend on predictability, may also vary relative to the individual and context. Although research has demonstrated the influence of both long-term knowledge and stimulus features in influencing expectations, it is unclear how perceptions of a melody are influenced by comparisons to other music pieces heard in the same context. To examine the effects of context we compared how listeners' judgments of two distinct sets of stimuli differed when they were presented alone or in combination. Stimuli were excerpts from a repertoire of Western music and a set of experimenter created melodies. Separate groups of participants rated liking and predictability for each set of stimuli alone and in combination. We found that when heard together, the Repertoire stimuli were more liked and rated as less predictable than if they were heard alone, with the opposite pattern being observed for the Experimental stimuli. This effect was driven by a change in ratings between the Alone and Combined conditions for each stimulus set. These findings demonstrate a context-based shift of predictability ratings and derived pleasure, suggesting that judgments stem not only from the physical properties of the stimulus, but also vary relative to other options available in the immediate context.
Collapse
Affiliation(s)
- Alexander W. Albury
- Department of Psychology, Concordia University, Montreal, QC, Canada
- International Laboratory for Brain, Music and Sound Research (BRAMS) and Center for Research in Brain, Language and Music (CRBLM), Montreal, QC, Canada
| | - Roberta Bianco
- Neuroscience of Perception and Action Laboratory, Italian Institute of Technology, Rome, Italy
| | - Benjamin P. Gold
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, United States
| | - Virginia B. Penhune
- Department of Psychology, Concordia University, Montreal, QC, Canada
- International Laboratory for Brain, Music and Sound Research (BRAMS) and Center for Research in Brain, Language and Music (CRBLM), Montreal, QC, Canada
| |
Collapse
|
6
|
Ringer H, Schröger E, Grimm S. Neural signatures of automatic repetition detection in temporally regular and jittered acoustic sequences. PLoS One 2023; 18:e0284836. [PMID: 37948467 PMCID: PMC10637696 DOI: 10.1371/journal.pone.0284836] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 10/20/2023] [Indexed: 11/12/2023] Open
Abstract
Detection of repeating patterns within continuous sound streams is crucial for efficient auditory perception. Previous studies demonstrated a remarkable sensitivity of the human auditory system to periodic repetitions in unfamiliar, meaningless sounds. Automatic repetition detection was reflected in different EEG markers, including sustained activity, neural synchronisation, and event-related responses to pattern occurrences. The current study investigated how listeners' attention and the temporal regularity of a sound modulate repetition perception, and how this influence is reflected in different EEG markers that were previously suggested to subserve dissociable functions. We reanalysed data of a previous study in which listeners were presented with sequences of unfamiliar artificial sounds that either contained repetitions of a certain sound segment or not. Repeating patterns occurred either regularly or with a temporal jitter within the sequences, and participants' attention was directed either towards the pattern repetitions or away from the auditory stimulation. Across both regular and jittered sequences during both attention and in-attention, pattern repetitions led to increased sustained activity throughout the sequence, evoked a characteristic positivity-negativity complex in the event-related potential, and enhanced inter-trial phase coherence of low-frequency oscillatory activity time-locked to repeating pattern onsets. While regularity only had a minor (if any) influence, attention significantly strengthened pattern repetition perception, which was consistently reflected in all three EEG markers. These findings suggest that the detection of pattern repetitions within continuous sounds relies on a flexible mechanism that is robust against in-attention and temporal irregularity, both of which typically occur in naturalistic listening situations. Yet, attention to the auditory input can enhance processing of repeating patterns and improve repetition detection.
Collapse
Affiliation(s)
- Hanna Ringer
- International Max Planck Research School on Neuroscience of Communication (IMPRS NeuroCom), Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
- Cognitive and Biological Psychology, Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany
- Research Group Neurocognition of Music and Language, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Erich Schröger
- Cognitive and Biological Psychology, Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany
| | - Sabine Grimm
- Physics of Cognition Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
- Cognitive Systems Lab, Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
| |
Collapse
|
7
|
Al Roumi F, Planton S, Wang L, Dehaene S. Brain-imaging evidence for compression of binary sound sequences in human memory. eLife 2023; 12:e84376. [PMID: 37910588 PMCID: PMC10619979 DOI: 10.7554/elife.84376] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 10/14/2023] [Indexed: 11/03/2023] Open
Abstract
According to the language-of-thought hypothesis, regular sequences are compressed in human memory using recursive loops akin to a mental program that predicts future items. We tested this theory by probing memory for 16-item sequences made of two sounds. We recorded brain activity with functional MRI and magneto-encephalography (MEG) while participants listened to a hierarchy of sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants' knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the complexity derived from the minimal description length in our formal language. Furthermore, activity should increase with complexity for learned sequences, and decrease with complexity for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal, and cerebellar cortices. These regions overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures.
Collapse
Affiliation(s)
- Fosca Al Roumi
- Cognitive Neuroimaging Unit, Université Paris-Saclay, INSERM, CEA, CNRS, NeuroSpin centerGif/YvetteFrance
| | - Samuel Planton
- Cognitive Neuroimaging Unit, Université Paris-Saclay, INSERM, CEA, CNRS, NeuroSpin centerGif/YvetteFrance
| | - Liping Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of SciencesShanghaiChina
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, Université Paris-Saclay, INSERM, CEA, CNRS, NeuroSpin centerGif/YvetteFrance
- Collège de France, Université Paris Sciences Lettres (PSL)ParisFrance
| |
Collapse
|
8
|
Tóth B, Velősy PK, Kovács P, Háden GP, Polver S, Sziller I, Winkler I. Auditory learning of recurrent tone sequences is present in the newborn's brain. Neuroimage 2023; 281:120384. [PMID: 37739198 DOI: 10.1016/j.neuroimage.2023.120384] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 08/13/2023] [Accepted: 09/19/2023] [Indexed: 09/24/2023] Open
Abstract
The seemingly effortless ability of our auditory system to rapidly detect new events in a dynamic environment is crucial for survival. Whether the underlying brain processes are innate is unknown. To answer this question, electroencephalography was recorded while regularly patterned (REG) versus random (RAND) tone sequences were presented to sleeping neonates. Regular relative to random sequences elicited differential neural responses after only a single repetition of the pattern indicating the existence of an innate capacity of the auditory system to detect auditory sequential regularities. We show that the newborn auditory system accumulates evidence only somewhat longer than the minimum amount determined by the ideal Bayesian observer model (the prediction from a variable-order Markov chain model) before detecting a repeating pattern. Thus, newborns can quickly form representations for regular features of the sound input, preparing the way for learning the contingencies of the environment.
Collapse
Affiliation(s)
- Brigitta Tóth
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary.
| | - Péter Kristóf Velősy
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary; Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Petra Kovács
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary; Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Gábor Peter Háden
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary; Department of Telecommunications and Media Informatics, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Budapest, Hungary
| | - Silvia Polver
- Department of Developmental Psychology and Socialisation, University of Padova, Padova, Italy
| | - Istvan Sziller
- Division of Obstetrics and Gynecology, DBC - Szent Imre University Teaching Hospital, Budapest, Hungary
| | - István Winkler
- Institute of Cognitive Neuroscience and Psychology, Research Centre for Natural Sciences, Budapest, Hungary
| |
Collapse
|
9
|
Yu CY, Cabildo A, Grahn JA, Vanden Bosch der Nederlanden CM. Perceived rhythmic regularity is greater for song than speech: examining acoustic correlates of rhythmic regularity in speech and song. Front Psychol 2023; 14:1167003. [PMID: 37303916 PMCID: PMC10250601 DOI: 10.3389/fpsyg.2023.1167003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Accepted: 05/09/2023] [Indexed: 06/13/2023] Open
Abstract
Rhythm is a key feature of music and language, but the way rhythm unfolds within each domain differs. Music induces perception of a beat, a regular repeating pulse spaced by roughly equal durations, whereas speech does not have the same isochronous framework. Although rhythmic regularity is a defining feature of music and language, it is difficult to derive acoustic indices of the differences in rhythmic regularity between domains. The current study examined whether participants could provide subjective ratings of rhythmic regularity for acoustically matched (syllable-, tempo-, and contour-matched) and acoustically unmatched (varying in tempo, syllable number, semantics, and contour) exemplars of speech and song. We used subjective ratings to index the presence or absence of an underlying beat and correlated ratings with stimulus features to identify acoustic metrics of regularity. Experiment 1 highlighted that ratings based on the term "rhythmic regularity" did not result in consistent definitions of regularity across participants, with opposite ratings for participants who adopted a beat-based definition (song greater than speech), a normal-prosody definition (speech greater than song), or an unclear definition (no difference). Experiment 2 defined rhythmic regularity as how easy it would be to tap or clap to the utterances. Participants rated song as easier to clap or tap to than speech for both acoustically matched and unmatched datasets. Subjective regularity ratings from Experiment 2 illustrated that stimuli with longer syllable durations and with less spectral flux were rated as more rhythmically regular across domains. Our findings demonstrate that rhythmic regularity distinguishes speech from song and several key acoustic features can be used to predict listeners' perception of rhythmic regularity within and across domains as well.
Collapse
Affiliation(s)
- Chu Yi Yu
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
| | - Anne Cabildo
- Department of Psychology, University of Toronto, Mississauga, ON, Canada
| | - Jessica A. Grahn
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
| | - Christina M. Vanden Bosch der Nederlanden
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
- Department of Psychology, University of Toronto, Mississauga, ON, Canada
| |
Collapse
|
10
|
Gill NK, Francis NA. Repetition plasticity in primary auditory cortex occurs across long timescales for spectrotemporally randomized pure-tones. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.04.26.538446. [PMID: 37162964 PMCID: PMC10168329 DOI: 10.1101/2023.04.26.538446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Repetition plasticity is a ubiquitous property of sensory systems in which repetitive sensation causes either a decrease ("repetition suppression", i.e. "adaptation") or increase ("repetition enhancement", i.e. "facilitation") in the amplitude of neural responses. Timescales of repetition plasticity for sensory neurons typically span milliseconds to tens of seconds, with longer durations for cortical vs subcortical regions. Here, we used 2-photon (2P) imaging to study repetition plasticity in mouse primary auditory cortex (A1) layer 2/3 (L2/3) during the presentation of spectrotemporally randomized pure-tone frequencies. Our study revealed subpopulations of neurons with repetition plasticity for equiprobable frequencies spaced minutes apart over a 20-minute period. We found both repetition suppression and enhancement in individual neurons and on average across populations. Each neuron tended to show repetition plasticity for 1-2 pure-tone frequencies near the neuron's best frequency. Moreover, we found correlated changes in neural response amplitude and latency across stimulus repetitions. Together, our results highlight cortical specialization for pattern recognition over long timescales in complex acoustic sequences.
Collapse
Affiliation(s)
- Nasiru K Gill
- Department of Biology, University of Maryland, College Park, MD, 20742
| | - Nikolas A Francis
- Department of Biology, University of Maryland, College Park, MD, 20742
- Brain and Behavior Institute, University of Maryland, College Park, MD, 20742
| |
Collapse
|
11
|
English G, Ghasemi Nejad N, Sommerfelt M, Yanik MF, von der Behrens W. Bayesian surprise shapes neural responses in somatosensory cortical circuits. Cell Rep 2023; 42:112009. [PMID: 36701237 DOI: 10.1016/j.celrep.2023.112009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 09/16/2022] [Accepted: 12/31/2022] [Indexed: 01/26/2023] Open
Abstract
Numerous psychophysical studies show that Bayesian inference governs sensory decision-making; however, the specific neural circuitry underlying this probabilistic mechanism remains unclear. We record extracellular neural activity along the somatosensory pathway of mice while delivering sensory stimulation paradigms designed to isolate the response to the surprise generated by Bayesian inference. Our results demonstrate that laminar cortical circuits in early sensory areas encode Bayesian surprise. Systematic sensitivity to surprise is not identified in the somatosensory thalamus, rather emerging in the primary (S1) and secondary (S2) somatosensory cortices. Multiunit spiking activity and evoked potentials in layer 6 of these regions exhibit the highest sensitivity to surprise. Gamma power in S1 layer 2/3 exhibits an NMDAR-dependent scaling with surprise, as does alpha power in layers 2/3 and 6 of S2. These results show a precise spatiotemporal neural representation of Bayesian surprise and suggest that Bayesian inference is a fundamental component of cortical processing.
Collapse
Affiliation(s)
- Gwendolyn English
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland.
| | - Newsha Ghasemi Nejad
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Marcel Sommerfelt
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Mehmet Fatih Yanik
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland
| | - Wolfger von der Behrens
- Institute of Neuroinformatics, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland; ZNZ Neuroscience Center Zurich, ETH Zurich & University of Zurich, 8057 Zurich, Switzerland.
| |
Collapse
|
12
|
Ringer H, Schröger E, Grimm S. Within- and between-subject consistency of perceptual segmentation in periodic noise: A combined behavioral tapping and EEG study. Psychophysiology 2023; 60:e14174. [PMID: 36106761 DOI: 10.1111/psyp.14174] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 08/19/2022] [Accepted: 08/22/2022] [Indexed: 01/04/2023]
Abstract
It is remarkable that human listeners can perceive periodicity in noise, as the isochronous repetition of a particular noise segment is not accompanied by salient physical cues in the acoustic signal. Previous research suggested that listeners rely on short temporally local and idiosyncratic features to perceptually segment periodic noise sequences. The present study sought to test this assumption by disentangling consistency of perceptual segmentation within and between listeners. Presented periodic noise sequences either consisted of seamless repetitions of a 500-ms segment or of repetitions of a 200-ms segment that were interleaved with 300-ms portions of random noise. Both within- and between-subject consistency was stronger for interleaved (compared with seamless) periodic sequences. The increased consistency likely resulted from reduced temporal jitter of potential features used for perceptual segmentation when the recurring segment was shorter and occurred interleaved with random noise. These results support the notion that perceptual segmentation of periodic noise relies on subtle temporally local features. However, the finding that some specific noise sequences were segmented more consistently across listeners than others challenges the assumption that the features are necessarily idiosyncratic. Instead, in some specific noise samples, a preference for certain spectral features is shared between individuals.
Collapse
Affiliation(s)
- Hanna Ringer
- International Max Planck Research School NeuroCom, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany
| | - Erich Schröger
- Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany
| | - Sabine Grimm
- Wilhelm Wundt Institute for Psychology, Leipzig University, Leipzig, Germany.,Institute of Physics, Chemnitz University of Technology, Chemnitz, Germany
| |
Collapse
|
13
|
Basiński K, Quiroga-Martinez DR, Vuust P. Temporal hierarchies in the predictive processing of melody - From pure tones to songs. Neurosci Biobehav Rev 2023; 145:105007. [PMID: 36535375 DOI: 10.1016/j.neubiorev.2022.105007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2022] [Revised: 11/30/2022] [Accepted: 12/14/2022] [Indexed: 12/23/2022]
Abstract
Listening to musical melodies is a complex task that engages perceptual and memoryrelated processes. The processes underlying melody cognition happen simultaneously on different timescales, ranging from milliseconds to minutes. Although attempts have been made, research on melody perception is yet to produce a unified framework of how melody processing is achieved in the brain. This may in part be due to the difficulty of integrating concepts such as perception, attention and memory, which pertain to different temporal scales. Recent theories on brain processing, which hold prediction as a fundamental principle, offer potential solutions to this problem and may provide a unifying framework for explaining the neural processes that enable melody perception on multiple temporal levels. In this article, we review empirical evidence for predictive coding on the levels of pitch formation, basic pitch-related auditory patterns,more complex regularity processing extracted from basic patterns and long-term expectations related to musical syntax. We also identify areas that would benefit from further inquiry and suggest future directions in research on musical melody perception.
Collapse
Affiliation(s)
- Krzysztof Basiński
- Division of Quality of Life Research, Medical University of Gdańsk, Poland
| | - David Ricardo Quiroga-Martinez
- Helen Wills Neuroscience Institute & Department of Psychology, University of California Berkeley, USA; Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Denmark
| | - Peter Vuust
- Center for Music in the Brain, Aarhus University & The Royal Academy of Music, Denmark
| |
Collapse
|
14
|
Herrmann B, Maess B, Johnsrude IS. Sustained responses and neural synchronization to amplitude and frequency modulation in sound change with age. Hear Res 2023; 428:108677. [PMID: 36580732 DOI: 10.1016/j.heares.2022.108677] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Revised: 12/09/2022] [Accepted: 12/16/2022] [Indexed: 12/23/2022]
Abstract
Perception of speech requires sensitivity to features, such as amplitude and frequency modulations, that are often temporally regular. Previous work suggests age-related changes in neural responses to temporally regular features, but little work has focused on age differences for different types of modulations. We recorded magnetoencephalography in younger (21-33 years) and older adults (53-73 years) to investigate age differences in neural responses to slow (2-6 Hz sinusoidal and non-sinusoidal) modulations in amplitude, frequency, or combined amplitude and frequency. Audiometric pure-tone average thresholds were elevated in older compared to younger adults, indicating subclinical hearing impairment in the recruited older-adult sample. Neural responses to sound onset (independent of temporal modulations) were increased in magnitude in older compared to younger adults, suggesting hyperresponsivity and a loss of inhibition in the aged auditory system. Analyses of neural activity to modulations revealed greater neural synchronization with amplitude, frequency, and combined amplitude-frequency modulations for older compared to younger adults. This potentiated response generalized across different degrees of temporal regularity (sinusoidal and non-sinusoidal), although neural synchronization was generally lower for non-sinusoidal modulation. Despite greater synchronization, sustained neural activity was reduced in older compared to younger adults for sounds modulated both sinusoidally and non-sinusoidally in frequency. Our results suggest age differences in the sensitivity of the auditory system to features present in speech and other natural sounds.
Collapse
Affiliation(s)
- Björn Herrmann
- Rotman Research Institute, Baycrest, North York, ON M6A 2E1, Canada; Department of Psychology, University of Toronto, Toronto, ON M5S 1A1, Canada; Department of Psychology & Brain and Mind Institute, The University of Western Ontario, London, ON N6A 3K7, Canada.
| | - Burkhard Maess
- Max Planck Institute for Human Cognitive and Brain Sciences, Brain Networks Unit, Leipzig 04103, Germany
| | - Ingrid S Johnsrude
- Department of Psychology & Brain and Mind Institute, The University of Western Ontario, London, ON N6A 3K7, Canada; School of Communication Sciences & Disorders, The University of Western Ontario, London, ON N6A 5B7, Canada
| |
Collapse
|
15
|
Bianco R, Chait M. No Link Between Speech-in-Noise Perception and Auditory Sensory Memory - Evidence From a Large Cohort of Older and Younger Listeners. Trends Hear 2023; 27:23312165231190688. [PMID: 37828868 PMCID: PMC10576936 DOI: 10.1177/23312165231190688] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Revised: 07/06/2023] [Accepted: 07/11/2023] [Indexed: 10/14/2023] Open
Abstract
A growing literature is demonstrating a link between working memory (WM) and speech-in-noise (SiN) perception. However, the nature of this correlation and which components of WM might underlie it, are being debated. We investigated how SiN reception links with auditory sensory memory (aSM) - the low-level processes that support the short-term maintenance of temporally unfolding sounds. A large sample of old (N = 199, 60-79 yo) and young (N = 149, 20-35 yo) participants was recruited online and performed a coordinate response measure-based speech-in-babble task that taps listeners' ability to track a speech target in background noise. We used two tasks to investigate implicit and explicit aSM. Both were based on tone patterns overlapping in processing time scales with speech (presentation rate of tones 20 Hz; of patterns 2 Hz). We hypothesised that a link between SiN and aSM may be particularly apparent in older listeners due to age-related reduction in both SiN reception and aSM. We confirmed impaired SiN reception in the older cohort and demonstrated reduced aSM performance in those listeners. However, SiN and aSM did not share variability. Across the two age groups, SiN performance was predicted by a binaural processing test and age. The results suggest that previously observed links between WM and SiN may relate to the executive components and other cognitive demands of the used tasks. This finding helps to constrain the search for the perceptual and cognitive factors that explain individual variability in SiN performance.
Collapse
Affiliation(s)
- Roberta Bianco
- Ear Institute, University College London, London, UK
- Neuroscience of Perception and Action Lab, Italian Institute of Technology (IIT), Rome, Italy
| | - Maria Chait
- Ear Institute, University College London, London, UK
| |
Collapse
|
16
|
O'Reilly JA. Recurrent Neural Network Model of Human Event-related Potentials in Response to Intensity Oddball Stimulation. Neuroscience 2022; 504:63-74. [DOI: 10.1016/j.neuroscience.2022.10.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 09/27/2022] [Accepted: 10/03/2022] [Indexed: 10/31/2022]
|
17
|
Brain-correlates of processing local dependencies within a statistical learning paradigm. Sci Rep 2022; 12:15296. [PMID: 36097186 PMCID: PMC9468168 DOI: 10.1038/s41598-022-19203-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Accepted: 08/25/2022] [Indexed: 11/09/2022] Open
Abstract
Statistical learning refers to the implicit mechanism of extracting regularities in our environment. Numerous studies have investigated the neural basis of statistical learning. However, how the brain responds to violations of auditory regularities based on prior (implicit) learning requires further investigation. Here, we used functional magnetic resonance imaging (fMRI) to investigate the neural correlates of processing events that are irregular based on learned local dependencies. A stream of consecutive sound triplets was presented. Unbeknown to the subjects, triplets were either (a) standard, namely triplets ending with a high probability sound or, (b) statistical deviants, namely triplets ending with a low probability sound. Participants (n = 33) underwent a learning phase outside the scanner followed by an fMRI session. Processing of statistical deviants activated a set of regions encompassing the superior temporal gyrus bilaterally, the right deep frontal operculum including lateral orbitofrontal cortex, and the right premotor cortex. Our results demonstrate that the violation of local dependencies within a statistical learning paradigm does not only engage sensory processes, but is instead reminiscent of the activation pattern during the processing of local syntactic structures in music and language, reflecting the online adaptations required for predictive coding in the context of statistical learning.
Collapse
|
18
|
Mencke I, Omigie D, Quiroga-Martinez DR, Brattico E. Atonal Music as a Model for Investigating Exploratory Behavior. Front Neurosci 2022; 16:793163. [PMID: 35812236 PMCID: PMC9256982 DOI: 10.3389/fnins.2022.793163] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Accepted: 05/12/2022] [Indexed: 11/13/2022] Open
Abstract
Atonal music is often characterized by low predictability stemming from the absence of tonal or metrical hierarchies. In contrast, Western tonal music exhibits intrinsic predictability due to its hierarchical structure and therefore, offers a directly accessible predictive model to the listener. In consequence, a specific challenge of atonal music is that listeners must generate a variety of new predictive models. Listeners must not only refrain from applying available tonal models to the heard music, but they must also search for statistical regularities and build new rules that may be related to musical properties other than pitch, such as timbre or dynamics. In this article, we propose that the generation of such new predictive models and the aesthetic experience of atonal music are characterized by internal states related to exploration. This is a behavior well characterized in behavioral neuroscience as fulfilling an innate drive to reduce uncertainty but which has received little attention in empirical music research. We support our proposal with emerging evidence that the hedonic value is associated with the recognition of patterns in low-predictability sound sequences and that atonal music elicits distinct behavioral responses in listeners. We end by outlining new research avenues that might both deepen our understanding of the aesthetic experience of atonal music in particular, and reveal core qualities of the aesthetic experience in general.
Collapse
Affiliation(s)
- Iris Mencke
- Department of Music, Max Planck Institute for Empirical Aesthetics, Frankfurt, Germany
- *Correspondence: Iris Mencke,
| | - Diana Omigie
- Department of Psychology, Goldsmiths, University of London, London, United Kingdom
| | - David Ricardo Quiroga-Martinez
- Department of Clinical Medicine, Center for Music in the Brain, Aarhus University and Royal Academy of Music, Aarhus, Denmark
| | - Elvira Brattico
- Department of Clinical Medicine, Center for Music in the Brain, Aarhus University and Royal Academy of Music, Aarhus, Denmark
- Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy
| |
Collapse
|
19
|
Mehra M, Mukesh A, Bandyopadhyay S. Separate Functional Subnetworks of Excitatory Neurons Show Preference to Periodic and Random Sound Structures. J Neurosci 2022; 42:3165-3183. [PMID: 35241488 PMCID: PMC8994540 DOI: 10.1523/jneurosci.0333-21.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2021] [Revised: 11/18/2021] [Accepted: 01/03/2022] [Indexed: 11/21/2022] Open
Abstract
Auditory cortex (ACX) neurons are sensitive to spectro-temporal sound patterns and violations in patterns induced by rare stimuli embedded within streams of sounds. We investigate the auditory cortical representation of repeated presentations of sequences of sounds with standard stimuli (common) with an embedded deviant (rare) stimulus in two conditions, Periodic (Fixed deviant position) or Random (Random deviant position). We used extracellular single-unit and two-photon Ca2+ imaging recordings in layer 2/3 neurons of the mouse (Mus musculus) ACX of either sex. Population single-unit average responses increased over repetitions in the Random condition and were suppressed or did not change in the Periodic condition, showing general irregularity preference. A subset of neurons showed the opposite behavior, indicating regularity preference. Furthermore, pairwise noise correlations were higher in the Random condition than in the Periodic condition, suggesting a role of recurrent connections in the observed differential adaptation. Functional two-photon Ca2+ imaging showed that excitatory (EX), and inhibitory (IN) neurons [parvalbumin-positive (PV) and somatostatin-positive (SOM)] also had different categories of long-term adaptation as observed with single-units. However, examination of functional connectivity between pairs of neurons of different categories showed that EX-PV connected pairs behaved opposite to the EX-EX and EX-SOM pairs, with more connections outside category in Random condition than Periodic condition. Finally, considering Regularity, Irregularity, and no preference of connected pairs of neurons showed that EX-EX and EX-SOM pairs were in largely separate functional subnetworks with different preferences, not EX-PV pairs. Thus, separate subnetworks underlie coding of periodic and random sound sequences.SIGNIFICANCE STATEMENT Studying how the auditory cortex (ACX) neurons respond to streams of sound sequences help us understand the importance of changes in dynamic acoustic noisy scenes around us. Humans and animals are sensitive to regularity and its violations in sound sequences. Psychophysical tasks in humans show that the auditory brain differentially responds to Periodic and Random structures, independent of the listener's attentional states. Here, we show that mouse ACX L2/3 neurons detect changes and respond differently to patterns over long-time scales. The differential functional connectivity profile obtained in response to two different sound contexts suggests the vital role of recurrent connections in the auditory cortical network. Furthermore, the excitatory-inhibitory neuronal interactions can contribute to detecting the changing sound patterns.
Collapse
Affiliation(s)
- Muneshwar Mehra
- Information Processing Laboratory, Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology Kharagpur, 721302, India
- Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, 721302, India
| | - Adarsh Mukesh
- Information Processing Laboratory, Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology Kharagpur, 721302, India
- Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, 721302, India
| | - Sharba Bandyopadhyay
- Information Processing Laboratory, Department of Electronics and Electrical Communication Engineering, Indian Institute of Technology Kharagpur, 721302, India
- Advanced Technology Development Centre, Indian Institute of Technology Kharagpur, 721302, India
| |
Collapse
|
20
|
Mencke I, Quiroga-Martinez DR, Omigie D, Michalareas G, Schwarzacher F, Haumann NT, Vuust P, Brattico E. Prediction under uncertainty: Dissociating sensory from cognitive expectations in highly uncertain musical contexts. Brain Res 2021; 1773:147664. [PMID: 34560052 DOI: 10.1016/j.brainres.2021.147664] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 09/16/2021] [Accepted: 09/17/2021] [Indexed: 10/20/2022]
Abstract
Predictive models in the brain rely on the continuous extraction of regularities from the environment. These models are thought to be updated by novel information, as reflected in prediction error responses such as the mismatch negativity (MMN). However, although in real life individuals often face situations in which uncertainty prevails, it remains unclear whether and how predictive models emerge in high-uncertainty contexts. Recent research suggests that uncertainty affects the magnitude of MMN responses in the context of music listening. However, musical predictions are typically studied with MMN stimulation paradigms based on Western tonal music, which are characterized by relatively high predictability. Hence, we developed an MMN paradigm to investigate how the high uncertainty of atonal music modulates predictive processes as indexed by the MMN and behavior. Using MEG in a group of 20 subjects without musical training, we demonstrate that the magnetic MMN in response to pitch, intensity, timbre, and location deviants is evoked in both tonal and atonal melodies, with no significant differences between conditions. In contrast, in a separate behavioral experiment involving 39 non-musicians, participants detected pitch deviants more accurately and rated confidence higher in the tonal than in the atonal musical context. These results indicate that contextual tonal uncertainty modulates processing stages in which conscious awareness is involved, although deviants robustly elicit low-level pre-attentive responses such as the MMN. The achievement of robust MMN responses, despite high tonal uncertainty, is relevant for future studies comparing groups of listeners' MMN responses to increasingly ecological music stimuli.
Collapse
Affiliation(s)
- Iris Mencke
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322 Frankfurt/Main, Germany; Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark.
| | - David Ricardo Quiroga-Martinez
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark
| | - Diana Omigie
- Department of Psychology, University of London, Goldsmiths, SE14 6NW London, United Kingdom
| | - Georgios Michalareas
- Department of Neuroscience, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322 Frankfurt/Main, Germany
| | - Franz Schwarzacher
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark
| | - Niels Trusbak Haumann
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark
| | - Peter Vuust
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark
| | - Elvira Brattico
- Center for Music in the Brain, Department of Clinical Medicine, Aarhus University & The Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, 8000 Aarhus C, Denmark; Department of Education, Psychology and Communication, University of Bari Aldo Moro, Piazza Umberto I, 70121 Bari, Italy
| |
Collapse
|
21
|
Quiroga-Martinez DR, Hansen NC, Højlund A, Pearce M, Brattico E, Holmes E, Friston K, Vuust P. Musicianship and melodic predictability enhance neural gain in auditory cortex during pitch deviance detection. Hum Brain Mapp 2021; 42:5595-5608. [PMID: 34459062 PMCID: PMC8559476 DOI: 10.1002/hbm.25638] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2021] [Revised: 07/22/2021] [Accepted: 08/07/2021] [Indexed: 11/10/2022] Open
Abstract
When listening to music, pitch deviations are more salient and elicit stronger prediction error responses when the melodic context is predictable and when the listener is a musician. Yet, the neuronal dynamics and changes in connectivity underlying such effects remain unclear. Here, we employed dynamic causal modeling (DCM) to investigate whether the magnetic mismatch negativity response (MMNm)-and its modulation by context predictability and musical expertise-are associated with enhanced neural gain of auditory areas, as a plausible mechanism for encoding precision-weighted prediction errors. Using Bayesian model comparison, we asked whether models with intrinsic connections within primary auditory cortex (A1) and superior temporal gyrus (STG)-typically related to gain control-or extrinsic connections between A1 and STG-typically related to propagation of prediction and error signals-better explained magnetoencephalography responses. We found that, compared to regular sounds, out-of-tune pitch deviations were associated with lower intrinsic (inhibitory) connectivity in A1 and STG, and lower backward (inhibitory) connectivity from STG to A1, consistent with disinhibition and enhanced neural gain in these auditory areas. More predictable melodies were associated with disinhibition in right A1, while musicianship was associated with disinhibition in left A1 and reduced connectivity from STG to left A1. These results indicate that musicianship and melodic predictability, as well as pitch deviations themselves, enhance neural gain in auditory cortex during deviance detection. Our findings are consistent with predictive processing theories suggesting that precise and informative error signals are selected by the brain for subsequent hierarchical processing.
Collapse
Affiliation(s)
- David R Quiroga-Martinez
- Center for Music in the Brain, Aarhus University & Royal Academy of Music Aarhus/Aalborg, Aarhus, Denmark
| | - Niels Christian Hansen
- Center for Music in the Brain, Aarhus University & Royal Academy of Music Aarhus/Aalborg, Aarhus, Denmark.,Aarhus Institute of Advanced Studies, Aarhus University, Aarhus, Denmark
| | - Andreas Højlund
- Center for Functionally Integrative Neuroscience, Aarhus University, Aarhus, Denmark
| | - Marcus Pearce
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Elvira Brattico
- Center for Music in the Brain, Aarhus University & Royal Academy of Music Aarhus/Aalborg, Aarhus, Denmark.,Department of Education, Psychology and Communication, University of Bari Aldo Moro, Bari, Italy
| | - Emma Holmes
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, UCL, London, UK
| | - Karl Friston
- The Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, UCL, London, UK
| | - Peter Vuust
- Center for Music in the Brain, Aarhus University & Royal Academy of Music Aarhus/Aalborg, Aarhus, Denmark
| |
Collapse
|
22
|
Dheerendra P, Barascud N, Kumar S, Overath T, Griffiths TD. Dynamics underlying auditory-object-boundary detection in primary auditory cortex. Eur J Neurosci 2021; 54:7274-7288. [PMID: 34549472 DOI: 10.1111/ejn.15471] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 09/10/2021] [Accepted: 09/16/2021] [Indexed: 11/28/2022]
Abstract
Auditory object analysis requires the fundamental perceptual process of detecting boundaries between auditory objects. However, the dynamics underlying the identification of discontinuities at object boundaries are not well understood. Here, we employed a synthetic stimulus composed of frequency-modulated ramps known as 'acoustic textures', where boundaries were created by changing the underlying spectrotemporal statistics. We collected magnetoencephalographic (MEG) data from human volunteers and observed a slow (<1 Hz) post-boundary drift in the neuromagnetic signal. The response evoking this drift signal was source localised close to Heschl's gyrus (HG) bilaterally, which is in agreement with a previous functional magnetic resonance imaging (fMRI) study that found HG to be involved in the detection of similar auditory object boundaries. Time-frequency analysis demonstrated suppression in alpha and beta bands that occurred after the drift signal.
Collapse
Affiliation(s)
- Pradeep Dheerendra
- Biosciences Institute, Newcastle University, Newcastle upon Tyne, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK.,Institute of Neuroscience & Psychology, University of Glasgow, Glasgow, UK
| | - Nicolas Barascud
- LSCP, Département d'Etudes Cognitives, ENS, EHESS, CNRS, PSL Research University, Paris, France.,Ear Institute, University College London, London, UK
| | - Sukhbinder Kumar
- Biosciences Institute, Newcastle University, Newcastle upon Tyne, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Tobias Overath
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.,Department of Psychology and Neuroscience, Duke University, Durham, North Carolina, USA
| | - Timothy D Griffiths
- Biosciences Institute, Newcastle University, Newcastle upon Tyne, UK.,Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
23
|
Herrmann B, Maess B, Johnsrude IS. A neural signature of regularity in sound is reduced in older adults. Neurobiol Aging 2021; 109:1-10. [PMID: 34634748 DOI: 10.1016/j.neurobiolaging.2021.09.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Revised: 09/03/2021] [Accepted: 09/08/2021] [Indexed: 01/21/2023]
Abstract
Sensitivity to repetitions in sound amplitude and frequency is crucial for sound perception. As with other aspects of sound processing, sensitivity to such patterns may change with age, and may help explain some age-related changes in hearing such as segregating speech from background sound. We recorded magnetoencephalography to characterize differences in the processing of sound patterns between younger and older adults. We presented tone sequences that either contained a pattern (made of a repeated set of tones) or did not contain a pattern. We show that auditory cortex in older, compared to younger, adults is hyperresponsive to sound onsets, but that sustained neural activity in auditory cortex, indexing the processing of a sound pattern, is reduced. Hence, the sensitivity of neural populations in auditory cortex fundamentally differs between younger and older individuals, overresponding to sound onsets, while underresponding to patterns in sounds. This may help to explain some age-related changes in hearing such as increased sensitivity to distracting sounds and difficulties tracking speech in the presence of other sound.
Collapse
Affiliation(s)
- Björn Herrmann
- Department of Psychology & Brain and Mind Institute, The University of Western Ontario, London, ON, Canada; Rotman Research Institute, Baycrest, North York, ON, Canada; Department of Psychology, University of Toronto, Toronto, ON, Canada.
| | - Burkhard Maess
- Brain Networks Unit, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Ingrid S Johnsrude
- Department of Psychology & Brain and Mind Institute, The University of Western Ontario, London, ON, Canada; School of Communication Sciences & Disorders, The University of Western Ontario, London, ON, Canada
| |
Collapse
|
24
|
Benhamou E, Zhao S, Sivasathiaseelan H, Johnson JCS, Requena-Komuro MC, Bond RL, van Leeuwen JEP, Russell LL, Greaves CV, Nelson A, Nicholas JM, Hardy CJD, Rohrer JD, Warren JD. Decoding expectation and surprise in dementia: the paradigm of music. Brain Commun 2021; 3:fcab173. [PMID: 34423301 PMCID: PMC8376684 DOI: 10.1093/braincomms/fcab173] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/31/2021] [Indexed: 01/08/2023] Open
Abstract
Making predictions about the world and responding appropriately to unexpected events are essential functions of the healthy brain. In neurodegenerative disorders, such as frontotemporal dementia and Alzheimer's disease, impaired processing of 'surprise' may underpin a diverse array of symptoms, particularly abnormalities of social and emotional behaviour, but is challenging to characterize. Here, we addressed this issue using a novel paradigm: music. We studied 62 patients (24 female; aged 53-88) representing major syndromes of frontotemporal dementia (behavioural variant, semantic variant primary progressive aphasia, non-fluent-agrammatic variant primary progressive aphasia) and typical amnestic Alzheimer's disease, in relation to 33 healthy controls (18 female; aged 54-78). Participants heard famous melodies containing no deviants or one of three types of deviant note-acoustic (white-noise burst), syntactic (key-violating pitch change) or semantic (key-preserving pitch change). Using a regression model that took elementary perceptual, executive and musical competence into account, we assessed accuracy detecting melodic deviants and simultaneously recorded pupillary responses and related these to deviant surprise value (information-content) and carrier melody predictability (entropy), calculated using an unsupervised machine learning model of music. Neuroanatomical associations of deviant detection accuracy and coupling of detection to deviant surprise value were assessed using voxel-based morphometry of patients' brain MRI. Whereas Alzheimer's disease was associated with normal deviant detection accuracy, behavioural and semantic variant frontotemporal dementia syndromes were associated with strikingly similar profiles of impaired syntactic and semantic deviant detection accuracy and impaired behavioural and autonomic sensitivity to deviant information-content (all P < 0.05). On the other hand, non-fluent-agrammatic primary progressive aphasia was associated with generalized impairment of deviant discriminability (P < 0.05) due to excessive false-alarms, despite retained behavioural and autonomic sensitivity to deviant information-content and melody predictability. Across the patient cohort, grey matter correlates of acoustic deviant detection accuracy were identified in precuneus, mid and mesial temporal regions; correlates of syntactic deviant detection accuracy and information-content processing, in inferior frontal and anterior temporal cortices, putamen and nucleus accumbens; and a common correlate of musical salience coding in supplementary motor area (all P < 0.05, corrected for multiple comparisons in pre-specified regions of interest). Our findings suggest that major dementias have distinct profiles of sensory 'surprise' processing, as instantiated in music. Music may be a useful and informative paradigm for probing the predictive decoding of complex sensory environments in neurodegenerative proteinopathies, with implications for understanding and measuring the core pathophysiology of these diseases.
Collapse
Affiliation(s)
- Elia Benhamou
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Sijia Zhao
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GG, UK
| | - Harri Sivasathiaseelan
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Jeremy C S Johnson
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Maï-Carmen Requena-Komuro
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Rebecca L Bond
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Janneke E P van Leeuwen
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Lucy L Russell
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Caroline V Greaves
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Annabel Nelson
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Jennifer M Nicholas
- Department of Medical Statistics, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
| | - Chris J D Hardy
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Jonathan D Rohrer
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Jason D Warren
- Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| |
Collapse
|
25
|
Milne AE, Zhao S, Tampakaki C, Bury G, Chait M. Sustained Pupil Responses Are Modulated by Predictability of Auditory Sequences. J Neurosci 2021; 41:6116-6127. [PMID: 34083259 PMCID: PMC8276747 DOI: 10.1523/jneurosci.2879-20.2021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 05/05/2021] [Accepted: 05/08/2021] [Indexed: 11/21/2022] Open
Abstract
The brain is highly sensitive to auditory regularities and exploits the predictable order of sounds in many situations, from parsing complex auditory scenes, to the acquisition of language. To understand the impact of stimulus predictability on perception, it is important to determine how the detection of predictable structure influences processing and attention. Here, we use pupillometry to gain insight into the effect of sensory regularity on arousal. Pupillometry is a commonly used measure of salience and processing effort, with more perceptually salient or perceptually demanding stimuli consistently associated with larger pupil diameters. In two experiments we tracked human listeners' pupil dynamics while they listened to sequences of 50-ms tone pips of different frequencies. The order of the tone pips was either random, contained deterministic (fully predictable) regularities (experiment 1, n = 18, 11 female) or had a probabilistic regularity structure (experiment 2, n = 20, 17 female). The sequences were rapid, preventing conscious tracking of sequence structure thus allowing us to focus on the automatic extraction of different types of regularities. We hypothesized that if regularity facilitates processing by reducing processing demands, a smaller pupil diameter would be seen in response to regular relative to random patterns. Conversely, if regularity is associated with heightened arousal and attention (i.e., engages processing resources) the opposite pattern would be expected. In both experiments we observed a smaller sustained (tonic) pupil diameter for regular compared with random sequences, consistent with the former hypothesis and confirming that predictability facilitates sequence processing.SIGNIFICANCE STATEMENT The brain is highly sensitive to auditory regularities. To appreciate the impact that the presence of predictability has on perception, we need to better understand how a predictable structure influences processing and attention. We recorded listeners' pupil responses to sequences of tones that followed either a predictable or unpredictable pattern, as the pupil can be used to implicitly tap into these different cognitive processes. We found that the pupil showed a smaller sustained diameter to predictable sequences, indicating that predictability eased processing rather than boosted attention. The findings suggest that the pupil response can be used to study the automatic extraction of regularities, and that the effects are most consistent with predictability helping the listener to efficiently process upcoming sounds.
Collapse
Affiliation(s)
- Alice E Milne
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| | - Sijia Zhao
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| | | | - Gabriela Bury
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| | - Maria Chait
- Ear Institute, University College London, London WC1X 8EE, United Kingdom
| |
Collapse
|
26
|
Takács Á, Kóbor A, Kardos Z, Janacsek K, Horváth K, Beste C, Nemeth D. Neurophysiological and functional neuroanatomical coding of statistical and deterministic rule information during sequence learning. Hum Brain Mapp 2021; 42:3182-3201. [PMID: 33797825 PMCID: PMC8193527 DOI: 10.1002/hbm.25427] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2020] [Revised: 02/23/2021] [Accepted: 03/15/2021] [Indexed: 12/18/2022] Open
Abstract
Humans are capable of acquiring multiple types of information presented in the same information stream. It has been suggested that at least two parallel learning processes are important during learning of sequential patterns-statistical learning and rule-based learning. Yet, the neurophysiological underpinnings of these parallel learning processes are not fully understood. To differentiate between the simultaneous mechanisms at the single trial level, we apply a temporal EEG signal decomposition approach together with sLORETA source localization method to delineate whether distinct statistical and rule-based learning codes can be distinguished in EEG data and can be related to distinct functional neuroanatomical structures. We demonstrate that concomitant but distinct aspects of information coded in the N2 time window play a role in these mechanisms: mismatch detection and response control underlie statistical learning and rule-based learning, respectively, albeit with different levels of time-sensitivity. Moreover, the effects of the two learning mechanisms in the different temporally decomposed clusters of neural activity also differed from each other in neural sources. Importantly, the right inferior frontal cortex (BA44) was specifically implicated in visuomotor statistical learning, confirming its role in the acquisition of transitional probabilities. In contrast, visuomotor rule-based learning was associated with the prefrontal gyrus (BA6). The results show how simultaneous learning mechanisms operate at the neurophysiological level and are orchestrated by distinct prefrontal cortical areas. The current findings deepen our understanding on the mechanisms of how humans are capable of learning multiple types of information from the same stimulus stream in a parallel fashion.
Collapse
Affiliation(s)
- Ádám Takács
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of MedicineTU DresdenDresdenGermany
| | - Andrea Kóbor
- Brain Imaging CentreResearch Centre for Natural SciencesBudapestHungary
| | - Zsófia Kardos
- Brain Imaging CentreResearch Centre for Natural SciencesBudapestHungary
- Department of Cognitive ScienceBudapest University of Technology and EconomicsBudapestHungary
| | - Karolina Janacsek
- Institute of PsychologyELTE Eötvös Loránd UniversityBudapestHungary
- Brain, Memory and Language Research Group, Institute of Cognitive Neuroscience and PsychologyResearch Centre for Natural SciencesBudapestHungary
- Centre of Thinking and Learning, Institute for Lifecourse Development, School of Human Sciences, Faculty of Education, Health and Human SciencesUniversity of GreenwichLondonUK
| | - Kata Horváth
- Institute of PsychologyELTE Eötvös Loránd UniversityBudapestHungary
- Brain, Memory and Language Research Group, Institute of Cognitive Neuroscience and PsychologyResearch Centre for Natural SciencesBudapestHungary
- Doctoral School of PsychologyELTE Eötvös Loránd UniversityBudapestHungary
| | - Christian Beste
- Cognitive Neurophysiology, Department of Child and Adolescent Psychiatry, Faculty of MedicineTU DresdenDresdenGermany
| | - Dezso Nemeth
- Institute of PsychologyELTE Eötvös Loránd UniversityBudapestHungary
- Brain, Memory and Language Research Group, Institute of Cognitive Neuroscience and PsychologyResearch Centre for Natural SciencesBudapestHungary
- Lyon Neuroscience Research Center (CRNL)Université de LyonLyonFrance
| |
Collapse
|
27
|
Herrmann B, Araz K, Johnsrude IS. Sustained neural activity correlates with rapid perceptual learning of auditory patterns. Neuroimage 2021; 238:118238. [PMID: 34098064 DOI: 10.1016/j.neuroimage.2021.118238] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2020] [Revised: 06/02/2021] [Accepted: 06/03/2021] [Indexed: 11/27/2022] Open
Abstract
Repeating structures forming regular patterns are common in sounds. Learning such patterns may enable accurate perceptual organization. In five experiments, we investigated the behavioral and neural signatures of rapid perceptual learning of regular sound patterns. We show that recurring (compared to novel) patterns are detected more quickly and increase sensitivity to pattern deviations and to the temporal order of pattern onset relative to a visual stimulus. Sustained neural activity reflected perceptual learning in two ways. Firstly, sustained activity increased earlier for recurring than novel patterns when participants attended to sounds, but not when they ignored them; this earlier increase mirrored the rapid perceptual learning we observed behaviorally. Secondly, the magnitude of sustained activity was generally lower for recurring than novel patterns, but only for trials later in the experiment, and independent of whether participants attended to or ignored sounds. The late manifestation of sustained activity reduction suggests that it is not directly related to rapid perceptual learning, but to a mechanism that does not require attention to sound. In sum, we demonstrate that the latency of sustained activity reflects rapid perceptual learning of auditory patterns, while the magnitude may reflect a result of learning, such as better prediction of learned auditory patterns.
Collapse
Affiliation(s)
- Björn Herrmann
- Rotman Research Institute, Baycrest, M6A 2E1, North York, ON, Canada; Department of Psychology, University of Toronto, M5S 1A1, Toronto, ON, Canada; Department of Psychology, University of Western Ontario, N6A 3K7, London, ON, Canada.
| | - Kurdo Araz
- Department of Psychology, University of Western Ontario, N6A 3K7, London, ON, Canada
| | - Ingrid S Johnsrude
- Department of Psychology, University of Western Ontario, N6A 3K7, London, ON, Canada; School of Communication Sciences & Disorders, University of Western Ontario, N6A 5B7 London, ON, Canada
| |
Collapse
|
28
|
Listeners with congenital amusia are sensitive to context uncertainty in melodic sequences. Neuropsychologia 2021; 158:107911. [PMID: 34102187 DOI: 10.1016/j.neuropsychologia.2021.107911] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Revised: 05/26/2021] [Accepted: 06/03/2021] [Indexed: 11/21/2022]
Abstract
In typical listeners, the perceptual salience of a surprising auditory event depends on the uncertainty of its context. For example, in melodies, pitch deviants are more easily detected and generate larger neural responses when the context is highly predictable than when it is less so. However, it is not known whether amusic listeners with abnormal pitch processing are sensitive to the degree of uncertainty of pitch sequences and, if so, whether they are to a different extent than typical non-musician listeners. To answer this question, we manipulated the uncertainty of short melodies while participants with and without congenital amusia underwent EEG recordings in a passive listening task. Uncertainty was manipulated by presenting melodies with different levels of complexity and familiarity, under the assumption that simpler and more familiar patterns would enhance pitch predictability. We recorded mismatch negativity (MMN) responses to pitch, intensity, timbre, location, and rhythm deviants as a measure of auditory surprise. In both participant groups, we observed reduced MMN amplitudes and longer peak latencies for all sound features with increasing levels of complexity, and putative familiarity effects only for intensity deviants. No significant group-by-complexity or group-by-familiarity interactions were detected. However, in contrast to previous studies, pitch MMN responses in amusics were disrupted in high complexity and unfamiliar melodies. The present results thus indicate that amusics are sensitive to the uncertainty of melodic sequences and that preattentive auditory change detection is greatly spared in this population across sound features and levels of predictability. However, our findings also hint at pitch-specific impairments in this population when uncertainty is high, thus suggesting that pitch processing under high uncertainty conditions requires an intact frontotemporal loop.
Collapse
|
29
|
Planton S, van Kerkoerle T, Abbih L, Maheu M, Meyniel F, Sigman M, Wang L, Figueira S, Romano S, Dehaene S. A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans. PLoS Comput Biol 2021; 17:e1008598. [PMID: 33465081 PMCID: PMC7845997 DOI: 10.1371/journal.pcbi.1008598] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2020] [Revised: 01/29/2021] [Accepted: 12/01/2020] [Indexed: 01/29/2023] Open
Abstract
Working memory capacity can be improved by recoding the memorized information in a condensed form. Here, we tested the theory that human adults encode binary sequences of stimuli in memory using an abstract internal language and a recursive compression algorithm. The theory predicts that the psychological complexity of a given sequence should be proportional to the length of its shortest description in the proposed language, which can capture any nested pattern of repetitions and alternations using a limited number of instructions. Five experiments examine the capacity of the theory to predict human adults' memory for a variety of auditory and visual sequences. We probed memory using a sequence violation paradigm in which participants attempted to detect occasional violations in an otherwise fixed sequence. Both subjective complexity ratings and objective violation detection performance were well predicted by our theoretical measure of complexity, which simply reflects a weighted sum of the number of elementary instructions and digits in the shortest formula that captures the sequence in our language. While a simpler transition probability model, when tested as a single predictor in the statistical analyses, accounted for significant variance in the data, the goodness-of-fit with the data significantly improved when the language-based complexity measure was included in the statistical model, while the variance explained by the transition probability model largely decreased. Model comparison also showed that shortest description length in a recursive language provides a better fit than six alternative previously proposed models of sequence encoding. The data support the hypothesis that, beyond the extraction of statistical knowledge, human sequence coding relies on an internal compression using language-like nested structures.
Collapse
Affiliation(s)
- Samuel Planton
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
| | - Timo van Kerkoerle
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
| | - Leïla Abbih
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
| | - Maxime Maheu
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
- Université de Paris, Paris, France
| | - Florent Meyniel
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
| | - Mariano Sigman
- Laboratorio de Neurociencia, Universidad Torcuato Di Tella, Buenos Aires, Argentina
- CONICET (Consejo Nacional de Investigaciones Científicas y Tecnicas), Buenos Aires, Argentina
- Facultad de Lenguas y Educacion, Universidad Nebrija, Madrid, Spain
| | - Liping Wang
- Institute of Neuroscience, Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
| | - Santiago Figueira
- CONICET (Consejo Nacional de Investigaciones Científicas y Tecnicas), Buenos Aires, Argentina
- Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales, Departamento de Computacion, Buenos Aires, Argentina
| | - Sergio Romano
- CONICET (Consejo Nacional de Investigaciones Científicas y Tecnicas), Buenos Aires, Argentina
- Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales, Departamento de Computacion, Buenos Aires, Argentina
| | - Stanislas Dehaene
- Cognitive Neuroimaging Unit, CEA, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin center, Gif/Yvette, France
- Collège de France, Paris, France
| |
Collapse
|
30
|
Aman L, Picken S, Andreou LV, Chait M. Sensitivity to temporal structure facilitates perceptual analysis of complex auditory scenes. Hear Res 2020; 400:108111. [PMID: 33333425 PMCID: PMC7812374 DOI: 10.1016/j.heares.2020.108111] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Revised: 10/13/2020] [Accepted: 11/06/2020] [Indexed: 11/17/2022]
Abstract
Perception relies on sensitivity to predictable structure in the environment. We used artificial acoustic scenes to investigate this in the auditory modality. Listeners track the temporal structure of multiple concurrent acoustic streams. Sensitivity to predictable structure supports auditory scene analysis, even when scenes are complex. Benefit of regularity observed even when listeners are unaware of the predictable structure.
The notion that sensitivity to the statistical structure of the environment is pivotal to perception has recently garnered considerable attention. Here we investigated this issue in the context of hearing. Building on previous work (Sohoglu and Chait, 2016a; elife), stimuli were artificial ‘soundscapes’ populated by multiple (up to 14) simultaneous streams (‘auditory objects’) comprised of tone-pip sequences, each with a distinct frequency and pattern of amplitude modulation. Sequences were either temporally regular or random. We show that listeners’ ability to detect abrupt appearance or disappearance of a stream is facilitated when scene streams were characterized by a temporally regular fluctuation pattern. The regularity of the changing stream as well as that of the background (non-changing) streams contribute independently to this effect. Remarkably, listeners benefit from regularity even when they are not consciously aware of it. These findings establish that perception of complex acoustic scenes relies on the availability of detailed representations of the regularities automatically extracted from multiple concurrent streams.
Collapse
Affiliation(s)
- Lucie Aman
- Ear Institute, University College London, 332 Gray's Inn Road, London WC1X 8EE, UK; Department of Psychiatry, University of Cambridge, Cambridge, UK
| | - Samantha Picken
- Ear Institute, University College London, 332 Gray's Inn Road, London WC1X 8EE, UK
| | - Lefkothea-Vasiliki Andreou
- Ear Institute, University College London, 332 Gray's Inn Road, London WC1X 8EE, UK; Vocational Lyceum of Zakynthos, Ministry of Education, Research and Religious Affairs, Zakynthos, Greece
| | - Maria Chait
- Ear Institute, University College London, 332 Gray's Inn Road, London WC1X 8EE, UK.
| |
Collapse
|
31
|
Harrison PMC, Bianco R, Chait M, Pearce MT. PPM-Decay: A computational model of auditory prediction with memory decay. PLoS Comput Biol 2020; 16:e1008304. [PMID: 33147209 PMCID: PMC7668605 DOI: 10.1371/journal.pcbi.1008304] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2020] [Revised: 11/16/2020] [Accepted: 09/04/2020] [Indexed: 12/19/2022] Open
Abstract
Statistical learning and probabilistic prediction are fundamental processes in auditory cognition. A prominent computational model of these processes is Prediction by Partial Matching (PPM), a variable-order Markov model that learns by internalizing n-grams from training sequences. However, PPM has limitations as a cognitive model: in particular, it has a perfect memory that weights all historic observations equally, which is inconsistent with memory capacity constraints and recency effects observed in human cognition. We address these limitations with PPM-Decay, a new variant of PPM that introduces a customizable memory decay kernel. In three studies-one with artificially generated sequences, one with chord sequences from Western music, and one with new behavioral data from an auditory pattern detection experiment-we show how this decay kernel improves the model's predictive performance for sequences whose underlying statistics change over time, and enables the model to capture effects of memory constraints on auditory pattern detection. The resulting model is available in our new open-source R package, ppm (https://github.com/pmcharrison/ppm).
Collapse
Affiliation(s)
- Peter M. C. Harrison
- Computational Auditory Perception Research Group, Max Planck Institute for Empirical Aesthetics, Frankfurt, Germany
- Cognitive Science Research Group, Queen Mary University of London, London, UK
- * E-mail:
| | - Roberta Bianco
- UCL Ear Institute, University College London, London, UK
| | - Maria Chait
- UCL Ear Institute, University College London, London, UK
| | - Marcus T. Pearce
- Cognitive Science Research Group, Queen Mary University of London, London, UK
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark
| |
Collapse
|
32
|
Andermann M, Günther M, Patterson RD, Rupp A. Early cortical processing of pitch height and the role of adaptation and musicality. Neuroimage 2020; 225:117501. [PMID: 33169697 DOI: 10.1016/j.neuroimage.2020.117501] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 10/19/2020] [Accepted: 10/21/2020] [Indexed: 02/06/2023] Open
Abstract
Pitch is an important perceptual feature; however, it is poorly understood how its cortical correlates are shaped by absolute vs relative fundamental frequency (f0), and by neural adaptation. In this study, we assessed transient and sustained auditory evoked fields (AEFs) at the onset, progression, and offset of short pitch height sequences, taking into account the listener's musicality. We show that neuromagnetic activity reflects absolute f0 at pitch onset and offset, and relative f0 at transitions within pitch sequences; further, sequences with fixed f0 lead to larger response suppression than sequences with variable f0 contour, and to enhanced offset activity. Musical listeners exhibit stronger f0-related AEFs and larger differences between their responses to fixed vs variable sequences, both within sequences and at pitch offset. The results resemble prominent psychoacoustic phenomena in the perception of pitch contours; moreover, they suggest a strong influence of adaptive mechanisms on cortical pitch processing which, in turn, might be modulated by a listener's musical expertise.
Collapse
Affiliation(s)
- Martin Andermann
- Section of Biomagnetism, Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany.
| | - Melanie Günther
- Section of Biomagnetism, Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany
| | - Roy D Patterson
- Department of Physiology, Development and Neuroscience, University of Cambridge, Downing Street, Cambridge, CB2 3EG, United Kingdom
| | - André Rupp
- Section of Biomagnetism, Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany
| |
Collapse
|
33
|
Statistical Learning and Inference Is Impaired in the Nonclinical Continuum of Psychosis. J Neurosci 2020; 40:6759-6769. [PMID: 32690617 DOI: 10.1523/jneurosci.0315-20.2020] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2020] [Revised: 06/23/2020] [Accepted: 06/30/2020] [Indexed: 02/08/2023] Open
Abstract
Our perceptions result from the brain's ability to make inferences, or predictive models, of sensory information. Recently, it has been proposed that psychotic traits may be linked to impaired predictive processes. Here, we examine the brain dynamics underlying statistical learning and inference in stable and volatile environments, in a population of healthy human individuals (N = 75; 36 males, 39 females) with a range of psychotic-like experiences. We measured prediction error responses to sound sequences with electroencephalography, gauged sensory inference explicitly by behaviorally recording sensory statistical learning errors, and used dynamic causal modeling to tap into the underlying neural circuitry. We discuss the findings that were robust to replication across the two experiments (Discovery dataset, N = 31; Validation dataset, N = 44). First, we found that during stable conditions, participants demonstrated greater precision in their predictive model, reflected in a larger prediction error response to unexpected sounds, and decreased statistical learning errors. Moreover, individuals with attenuated prediction errors in stable conditions were found to make greater incorrect predictions about sensory information. Critically, we show that greater errors in statistical learning and inference are related to increased psychotic-like experiences. These findings link neurophysiology to behavior during statistical learning and prediction formation, as well as providing further evidence for the idea of a continuum of psychosis in the healthy, nonclinical population.SIGNIFICANCE STATEMENT While perceiving the world, we make inferences by learning the statistics present in the sensory environment. It has been argued that psychosis may emerge because of a failure to learn sensory statistics, resulting in an impaired representation of the world. Recently, it has been proposed that psychosis exists on a continuum; however, there is conflicting evidence on whether sensory learning deficits align on the nonclinical end of the psychosis continuum. We found that statistical learning of sensory events is associated with the magnitude of mismatch negativity and, critically, is impaired in healthy people who report more psychotic-like experiences. We replicated these findings in an independent sample, demonstrating strengthened credibility to support the continuum of psychosis that extends into the nonclinical population.
Collapse
|
34
|
Dercksen TT, Widmann A, Schröger E, Wetzel N. Omission related brain responses reflect specific and unspecific action-effect couplings. Neuroimage 2020; 215:116840. [PMID: 32289452 DOI: 10.1016/j.neuroimage.2020.116840] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Revised: 03/31/2020] [Accepted: 04/06/2020] [Indexed: 11/29/2022] Open
Abstract
When an auditory stimulus is predicted but unexpectedly omitted, an omission response can be observed in the EEG. This endogenous response to the absence of a stimulus demonstrates the important role of prediction in perception. SanMiguel et al. (2013a) showed that in order to observe an omission response, a specific prediction concerning the identity of an upcoming stimulus is necessary. They used button presses coupled to either a single sound (predictable identity), or a random sound (unpredictable identity). In the event-related potentials, a sequence of omission responses consisting of oN1, oN2, and oP3 was observed in the single condition but not in the random condition. Given the importance of omission studies to understand the role of prediction in perception, we replicated this study. We enhanced statistical power by doubling the sample size and adjusting data pre-processing, and applied temporal principal component analysis and replication Bayes statistics. Results in the single sound condition were successfully replicated. Principal component analysis additionally revealed attenuated oN1 and oP3 omission responses in the random sound condition. These results suggest the existence of both specific and unspecific predictions along the sound processing hierarchy, where precision weighting possibly influences the strength of prediction error. Results are discussed in the framework of predictive coding and are congruent with everyday life, where uncertainty often requires broader or more general predictions.
Collapse
Affiliation(s)
- Tjerk T Dercksen
- Leibniz Institute for Neurobiology, Brenneckestraße 6, 39118, Magdeburg, Germany; Center for Behavioral Brain Sciences, Universitätsplatz 2, D-39106, Magdeburg, Germany.
| | - Andreas Widmann
- Leibniz Institute for Neurobiology, Brenneckestraße 6, 39118, Magdeburg, Germany; Leipzig University, Neumarkt 9-19, D-04109, Leipzig, Germany.
| | - Erich Schröger
- Leipzig University, Neumarkt 9-19, D-04109, Leipzig, Germany.
| | - Nicole Wetzel
- Leibniz Institute for Neurobiology, Brenneckestraße 6, 39118, Magdeburg, Germany; Center for Behavioral Brain Sciences, Universitätsplatz 2, D-39106, Magdeburg, Germany; University of Applied Sciences Magdeburg-Stendal, Osterburgerstraße 25, 39576, Stendal, Germany.
| |
Collapse
|
35
|
Bianco R, Harrison PMC, Hu M, Bolger C, Picken S, Pearce MT, Chait M. Long-term implicit memory for sequential auditory patterns in humans. eLife 2020; 9:e56073. [PMID: 32420868 PMCID: PMC7338054 DOI: 10.7554/elife.56073] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2020] [Accepted: 05/18/2020] [Indexed: 11/17/2022] Open
Abstract
Memory, on multiple timescales, is critical to our ability to discover the structure of our surroundings, and efficiently interact with the environment. We combined behavioural manipulation and modelling to investigate the dynamics of memory formation for rarely reoccurring acoustic patterns. In a series of experiments, participants detected the emergence of regularly repeating patterns within rapid tone-pip sequences. Unbeknownst to them, a few patterns reoccurred every ~3 min. All sequences consisted of the same 20 frequencies and were distinguishable only by the order of tone-pips. Despite this, reoccurring patterns were associated with a rapidly growing detection-time advantage over novel patterns. This effect was implicit, robust to interference, and persisted for 7 weeks. The results implicate an interplay between short (a few seconds) and long-term (over many minutes) integration in memory formation and demonstrate the remarkable sensitivity of the human auditory system to sporadically reoccurring structure within the acoustic environment.
Collapse
Affiliation(s)
- Roberta Bianco
- UCL Ear Institute, University College LondonLondonUnited Kingdom
| | - Peter MC Harrison
- School of Electronic Engineering and Computer Science, Queen Mary University of LondonLondonUnited Kingdom
| | - Mingyue Hu
- UCL Ear Institute, University College LondonLondonUnited Kingdom
| | - Cora Bolger
- UCL Ear Institute, University College LondonLondonUnited Kingdom
| | - Samantha Picken
- UCL Ear Institute, University College LondonLondonUnited Kingdom
| | - Marcus T Pearce
- School of Electronic Engineering and Computer Science, Queen Mary University of LondonLondonUnited Kingdom
- Department of Clinical Medicine, Aarhus UniversityAarhusDenmark
| | - Maria Chait
- UCL Ear Institute, University College LondonLondonUnited Kingdom
| |
Collapse
|
36
|
Szalárdy O, Tóth B, Farkas D, Orosz G, Honbolygó F, Winkler I. Linguistic predictability influences auditory stimulus classification within two concurrent speech streams. Psychophysiology 2020; 57:e13547. [DOI: 10.1111/psyp.13547] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2019] [Revised: 01/20/2020] [Accepted: 01/22/2020] [Indexed: 11/30/2022]
Affiliation(s)
- Orsolya Szalárdy
- Faculty of Medicine Institute of Behavioural Sciences Semmelweis University Budapest Hungary
- Institute of Cognitive Neuroscience and Psychology Research Centre for Natural Sciences Hungarian Academy of Sciences Budapest Hungary
| | - Brigitta Tóth
- Institute of Cognitive Neuroscience and Psychology Research Centre for Natural Sciences Hungarian Academy of Sciences Budapest Hungary
| | - Dávid Farkas
- Analytics Development, Performance Management and Analytics, Business Development, Integrated Supply Chain Management, Nokia Business Services, Nokia Operations, Nokia Budapest Hungary
| | - Gábor Orosz
- Department of Psychology Stanford University Stanford CA USA
| | - Ferenc Honbolygó
- Brain Imaging Centre Research Centre for Natural Sciences Hungarian Academy of Sciences Budapest Hungary
- Institute of Psychology ELTE Eötvös Loránd University Budapest Hungary
| | - István Winkler
- Institute of Cognitive Neuroscience and Psychology Research Centre for Natural Sciences Hungarian Academy of Sciences Budapest Hungary
| |
Collapse
|
37
|
Di Liberto GM, Pelofi C, Bianco R, Patel P, Mehta AD, Herrero JL, de Cheveigné A, Shamma S, Mesgarani N. Cortical encoding of melodic expectations in human temporal cortex. eLife 2020; 9:e51784. [PMID: 32122465 PMCID: PMC7053998 DOI: 10.7554/elife.51784] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Accepted: 01/20/2020] [Indexed: 01/14/2023] Open
Abstract
Humans engagement in music rests on underlying elements such as the listeners' cultural background and interest in music. These factors modulate how listeners anticipate musical events, a process inducing instantaneous neural responses as the music confronts these expectations. Measuring such neural correlates would represent a direct window into high-level brain processing. Here we recorded cortical signals as participants listened to Bach melodies. We assessed the relative contributions of acoustic versus melodic components of the music to the neural signal. Melodic features included information on pitch progressions and their tempo, which were extracted from a predictive model of musical structure based on Markov chains. We related the music to brain activity with temporal response functions demonstrating, for the first time, distinct cortical encoding of pitch and note-onset expectations during naturalistic music listening. This encoding was most pronounced at response latencies up to 350 ms, and in both planum temporale and Heschl's gyrus.
Collapse
Affiliation(s)
- Giovanni M Di Liberto
- Laboratoire des systèmes perceptifs, Département d’études cognitives, École normale supérieure, PSL University, CNRS75005 ParisFrance
| | - Claire Pelofi
- Department of Psychology, New York UniversityNew YorkUnited States
- Institut de Neurosciences des Système, UMR S 1106, INSERM, Aix Marseille UniversitéMarseilleFrance
| | | | - Prachi Patel
- Department of Electrical Engineering, Columbia UniversityNew YorkUnited States
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Ashesh D Mehta
- Department of Neurosurgery, Zucker School of Medicine at Hofstra/NorthwellManhassetUnited States
- Feinstein Institute of Medical Research, Northwell HealthManhassetUnited States
| | - Jose L Herrero
- Department of Neurosurgery, Zucker School of Medicine at Hofstra/NorthwellManhassetUnited States
- Feinstein Institute of Medical Research, Northwell HealthManhassetUnited States
| | - Alain de Cheveigné
- Laboratoire des systèmes perceptifs, Département d’études cognitives, École normale supérieure, PSL University, CNRS75005 ParisFrance
- UCL Ear InstituteLondonUnited Kingdom
| | - Shihab Shamma
- Laboratoire des systèmes perceptifs, Département d’études cognitives, École normale supérieure, PSL University, CNRS75005 ParisFrance
- Institute for Systems Research, Electrical and Computer Engineering, University of MarylandCollege ParkUnited States
| | - Nima Mesgarani
- Department of Electrical Engineering, Columbia UniversityNew YorkUnited States
- Mortimer B Zuckerman Mind Brain Behavior Institute, Columbia UniversityNew YorkUnited States
| |
Collapse
|
38
|
Walsh KS, McGovern DP, Clark A, O'Connell RG. Evaluating the neurophysiological evidence for predictive processing as a model of perception. Ann N Y Acad Sci 2020; 1464:242-268. [PMID: 32147856 PMCID: PMC7187369 DOI: 10.1111/nyas.14321] [Citation(s) in RCA: 109] [Impact Index Per Article: 27.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2019] [Revised: 01/21/2020] [Accepted: 02/03/2020] [Indexed: 12/12/2022]
Abstract
For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.
Collapse
Affiliation(s)
- Kevin S. Walsh
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| | - David P. McGovern
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
- School of PsychologyDublin City UniversityDublinIreland
| | - Andy Clark
- Department of PhilosophyUniversity of SussexBrightonUK
- Department of InformaticsUniversity of SussexBrightonUK
| | - Redmond G. O'Connell
- Trinity College Institute of Neuroscience and School of PsychologyTrinity College DublinDublinIreland
| |
Collapse
|
39
|
Bianco R, Ptasczynski LE, Omigie D. Pupil responses to pitch deviants reflect predictability of melodic sequences. Brain Cogn 2020; 138:103621. [DOI: 10.1016/j.bandc.2019.103621] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Revised: 10/08/2019] [Accepted: 10/15/2019] [Indexed: 10/25/2022]
|
40
|
Quiroga‐Martinez DR, C. Hansen N, Højlund A, Pearce M, Brattico E, Vuust P. Musical prediction error responses similarly reduced by predictive uncertainty in musicians and non‐musicians. Eur J Neurosci 2020; 51:2250-2269. [DOI: 10.1111/ejn.14667] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2019] [Revised: 11/26/2019] [Accepted: 12/23/2019] [Indexed: 12/14/2022]
Affiliation(s)
| | - Niels C. Hansen
- The MARCS Institute for Brain, Behaviour, and Development Western Sydney University Sydney NSW Australia
| | - Andreas Højlund
- Center for Functionally Integrative Neuroscience Aarhus University Aarhus Denmark
| | - Marcus Pearce
- Center for Music in the Brain Aarhus University & The Royal Academy of music Aarhus Denmark
- School of Electronic Engineering and Computer Science Queen Mary University of London London UK
| | - Elvira Brattico
- Center for Music in the Brain Aarhus University & The Royal Academy of music Aarhus Denmark
| | - Peter Vuust
- Center for Music in the Brain Aarhus University & The Royal Academy of music Aarhus Denmark
| |
Collapse
|
41
|
Zhao S, Chait M, Dick F, Dayan P, Furukawa S, Liao HI. Pupil-linked phasic arousal evoked by violation but not emergence of regularity within rapid sound sequences. Nat Commun 2019; 10:4030. [PMID: 31492881 PMCID: PMC6731273 DOI: 10.1038/s41467-019-12048-1] [Citation(s) in RCA: 34] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2018] [Accepted: 08/19/2019] [Indexed: 11/09/2022] Open
Abstract
The ability to track the statistics of our surroundings is a key computational challenge. A prominent theory proposes that the brain monitors for unexpected uncertainty - events which deviate substantially from model predictions, indicating model failure. Norepinephrine is thought to play a key role in this process by serving as an interrupt signal, initiating model-resetting. However, evidence is from paradigms where participants actively monitored stimulus statistics. To determine whether Norepinephrine routinely reports the statistical structure of our surroundings, even when not behaviourally relevant, we used rapid tone-pip sequences that contained salient pattern-changes associated with abrupt structural violations vs. emergence of regular structure. Phasic pupil dilations (PDR) were monitored to assess Norepinephrine. We reveal a remarkable specificity: When not behaviourally relevant, only abrupt structural violations evoke a PDR. The results demonstrate that Norepinephrine tracks unexpected uncertainty on rapid time scales relevant to sensory signals.
Collapse
Affiliation(s)
- Sijia Zhao
- Ear Institute, University College London, London, WC1X 8EE, UK
| | - Maria Chait
- Ear Institute, University College London, London, WC1X 8EE, UK.
| | - Fred Dick
- Department of Psychological Sciences, Birkbeck College, London, WC1E 7HX, UK
- Department of Experimental Psychology, University College London, London, WC1H 0DS, UK
| | - Peter Dayan
- Max Planck Institute for Biological Cybernetics, 72076, Tübingen, Germany
| | - Shigeto Furukawa
- NTT Communication Science Laboratories, NTT Corporation, Atsugi, 243-0198, Japan
| | - Hsin-I Liao
- NTT Communication Science Laboratories, NTT Corporation, Atsugi, 243-0198, Japan
| |
Collapse
|
42
|
Rapid Ocular Responses Are Modulated by Bottom-up-Driven Auditory Salience. J Neurosci 2019; 39:7703-7714. [PMID: 31391262 PMCID: PMC6764203 DOI: 10.1523/jneurosci.0776-19.2019] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2019] [Revised: 06/28/2019] [Accepted: 07/12/2019] [Indexed: 02/03/2023] Open
Abstract
Despite the prevalent use of alerting sounds in alarms and human-machine interface systems and the long-hypothesized role of the auditory system as the brain's "early warning system," we have only a rudimentary understanding of what determines auditory salience-the automatic attraction of attention by sound-and which brain mechanisms underlie this process. A major roadblock has been the lack of a robust, objective means of quantifying sound-driven attentional capture. Here we demonstrate that: (1) a reliable salience scale can be obtained from crowd-sourcing (N = 911), (2) acoustic roughness appears to be a driving feature behind this scaling, consistent with previous reports implicating roughness in the perceptual distinctiveness of sounds, and (3) crowd-sourced auditory salience correlates with objective autonomic measures. Specifically, we show that a salience ranking obtained from online raters correlated robustly with the superior colliculus-mediated ocular freezing response, microsaccadic inhibition (MSI), measured in naive, passively listening human participants (of either sex). More salient sounds evoked earlier and larger MSI, consistent with a faster orienting response. These results are consistent with the hypothesis that MSI reflects a general reorienting response that is evoked by potentially behaviorally important events regardless of their modality.SIGNIFICANCE STATEMENT Microsaccades are small, rapid, fixational eye movements that are measurable with sensitive eye-tracking equipment. We reveal a novel, robust link between microsaccade dynamics and the subjective salience of brief sounds (salience rankings obtained from a large number of participants in an online experiment): Within 300 ms of sound onset, the eyes of naive, passively listening participants demonstrate different microsaccade patterns as a function of the sound's crowd-sourced salience. These results position the superior colliculus (hypothesized to underlie microsaccade generation) as an important brain area to investigate in the context of a putative multimodal salience hub. They also demonstrate an objective means for quantifying auditory salience.
Collapse
|
43
|
Hao Y, Yao L, Sun Q, Gupta D. Interaction of Self-Regulation and Contextual Effects on Pre-attentive Auditory Processing: A Combined EEG/ECG Study. Front Neurosci 2019; 13:638. [PMID: 31275111 PMCID: PMC6593616 DOI: 10.3389/fnins.2019.00638] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2019] [Accepted: 06/03/2019] [Indexed: 11/13/2022] Open
Abstract
Environmental changes are not always within the focus of our attention, and sensitive reactions (i.e., quicker and stronger responses) can be essential for an organism's survival and adaptation. Here we report that neurophysiological responses to sound changes that are not in the focus of attention are related to both ambient acoustic contexts and regulation ability. We assessed electroencephalograph (EEG) mismatch negativity (MMN) latency and amplitude in response to sound changes in two contexts: ascending and descending pitch sequences while participants were instructed to attend to muted videos. Prolonged latency and increased amplitude of MMN at fronto-central region occurred in ascending pitch sequences relative to descending sequences. We also assessed how regulation related to the contextual effects on MMN. Reactions to changes in the ascending sequence were observed with the attention control (frontal EEG theta/beta ratio) indicating speed of reaction, and the autonomous regulation (heart-rate variability) indicating intensity of reaction. Moreover, sound changes in the ascending context were associated with more activation of anterior cingulate cortex and insula, suggesting arousal effects and regulation processes. These findings suggest that the relation between speed and intensity is not fixed and may be modified by contexts and self-regulation ability. Specifically, cortical and cardiovascular indicators of self-regulation may specify different aspects of response sensitivity in terms of speed and intensity.
Collapse
Affiliation(s)
- Yu Hao
- Department of Design and Environmental Analysis, Cornell University, Ithaca, NY, United States
| | - Lin Yao
- School of Electrical and Computer Engineering, Cornell University, Ithaca, NY, United States
| | - Qiuyan Sun
- Department of Nutritional Science, Cornell University, Ithaca, NY, United States
| | - Disha Gupta
- School of Medicine, New York University, New York, NY, United States
| |
Collapse
|
44
|
Quiroga-Martinez DR, Hansen NC, Højlund A, Pearce MT, Brattico E, Vuust P. Reduced prediction error responses in high-as compared to low-uncertainty musical contexts. Cortex 2019; 120:181-200. [PMID: 31323458 DOI: 10.1016/j.cortex.2019.06.010] [Citation(s) in RCA: 29] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2018] [Revised: 04/05/2019] [Accepted: 06/19/2019] [Indexed: 02/05/2023]
Abstract
Theories of predictive processing propose that prediction error responses are modulated by the certainty of the predictive model or precision. While there is some evidence for this phenomenon in the visual and, to a lesser extent, the auditory modality, little is known about whether it operates in the complex auditory contexts of daily life. Here, we examined how prediction error responses behave in a more complex and ecologically valid auditory context than those typically studied. We created musical tone sequences with different degrees of pitch uncertainty to manipulate the precision of participants' auditory expectations. Magnetoencephalography was used to measure the magnetic counterpart of the mismatch negativity (MMNm) as a neural marker of prediction error in a multi-feature paradigm. Pitch, slide, intensity and timbre deviants were included. We compared high-entropy stimuli, consisting of a set of non-repetitive melodies, with low-entropy stimuli consisting of a simple, repetitive pitch pattern. Pitch entropy was quantitatively assessed with an information-theoretic model of auditory expectation. We found a reduction in pitch and slide MMNm amplitudes in the high-entropy as compared to the low-entropy context. No significant differences were found for intensity and timbre MMNm amplitudes. Furthermore, in a separate behavioral experiment investigating the detection of pitch deviants, similar decreases were found for accuracy measures in response to more fine-grained increases in pitch entropy. Our results are consistent with a precision modulation of auditory prediction error in a musical context, and suggest that this effect is specific to features that depend on the manipulated dimension-pitch information, in this case.
Collapse
Affiliation(s)
| | - Niels C Hansen
- The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Australia
| | - Andreas Højlund
- Center for Functionally Integrative Neuroscience, Aarhus University, Denmark
| | - Marcus T Pearce
- Center for Music in the Brain, Aarhus University and The Royal Academy of Music, Denmark; School of Electronic Engineering and Computer Science, Queen Mary University of London, UK
| | - Elvira Brattico
- Center for Music in the Brain, Aarhus University and The Royal Academy of Music, Denmark
| | - Peter Vuust
- Center for Music in the Brain, Aarhus University and The Royal Academy of Music, Denmark
| |
Collapse
|
45
|
Lumaca M, Trusbak Haumann N, Brattico E, Grube M, Vuust P. Weighting of neural prediction error by rhythmic complexity: A predictive coding account using mismatch negativity. Eur J Neurosci 2019; 49:1597-1609. [DOI: 10.1111/ejn.14329] [Citation(s) in RCA: 59] [Impact Index Per Article: 11.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2018] [Revised: 11/28/2018] [Accepted: 12/12/2018] [Indexed: 12/27/2022]
Affiliation(s)
- Massimo Lumaca
- Department of Clinical MedicineCenter for Music in the BrainAarhus University & The Royal Academy of Music Aarhus C Denmark
- SISSA International School for Advanced Studies Trieste Italy
| | - Niels Trusbak Haumann
- Department of Clinical MedicineCenter for Music in the BrainAarhus University & The Royal Academy of Music Aarhus C Denmark
| | - Elvira Brattico
- Department of Clinical MedicineCenter for Music in the BrainAarhus University & The Royal Academy of Music Aarhus C Denmark
| | - Manon Grube
- Department of Clinical MedicineCenter for Music in the BrainAarhus University & The Royal Academy of Music Aarhus C Denmark
| | - Peter Vuust
- Department of Clinical MedicineCenter for Music in the BrainAarhus University & The Royal Academy of Music Aarhus C Denmark
| |
Collapse
|
46
|
Motala A, Caceres LG. Disentangling Neural Synchronization and Sustained Neural Activity in the Processing of Auditory Temporal Patterns. Front Hum Neurosci 2019; 12:497. [PMID: 30618679 PMCID: PMC6305306 DOI: 10.3389/fnhum.2018.00497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2018] [Accepted: 11/26/2018] [Indexed: 11/14/2022] Open
Affiliation(s)
- Aysha Motala
- School of Optometry & Vision Science Graduate Program, Cardiff University, Cardiff, United Kingdom
| | - Lucila Guadalupe Caceres
- Sensorimotor Dynamics Lab, Department of Science and Technology, National University of Quilmes, Buenos Aires, Argentina
| |
Collapse
|