1
|
Madhavan M, Zaid MI, Chaudhary K, Soribe C, Jlal Siddiqui M, Ali N, Rahaman R, Brito Lima H, Nair S, Din N, Marzec M, Wróbel MJ. Memory Load Test - A Concept for Cognitive Reserve Evaluation with Auditory Perception. OTOLARYNGOLOGIA POLSKA 2024; 78:1-9. [PMID: 39267575 DOI: 10.5604/01.3001.0054.5661] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/17/2024]
Abstract
<b>Introduction:</b> Auditory scene analysis refers to the system through which the auditory system distinguishes distinct auditory events and sources to create meaningful auditory information. The exact number of directly perceived auditory stimuli is unknown, studies suggest it may range from 3 to 5. This number differs among individuals, and potentially may indirectly indicate the ability to store and process the complex information, related to the memory load, which is combined with human cognitive processes.<b>Aim:</b> This study aims to further identify and quantify the number of sounds that can be perceived simultaneously in a complex auditory environment.<b>Material and methods:</b> Participants were presented with structured acoustic recordings and were asked to identify the exact number of targeted stimuli heard throughout the test. The experiment was designed to assess the auditory load and determine the maximum number of auditory stimuli that a healthy human can perceive at once.<b>Results:</b> Our study showed that on average, participants could identify up to three sounds at once with accuracy of responses declining progressively for four sounds or more.<b>Conclusions:</b> This study aimed to investigate the human capacity to detect and identify multiple sound signals simultaneously in a noisy environment. By understanding this ability, we sought to assess cognitive reserve in individuals. Our objective was to determine if auditory load could serve as a diagnostic tool for cognitive evaluation. We believe that further research will establish the validity of this approach, and we anticipate that it is only a matter of time before it becomes a viable method for assessing cognitive function.
Collapse
Affiliation(s)
- Maya Madhavan
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Muhammad Ibraheem Zaid
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Khizer Chaudhary
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Chetachi Soribe
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Muhammed Jlal Siddiqui
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Nabiha Ali
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Roisin Rahaman
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Helga Brito Lima
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Sanya Nair
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Nessa Din
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Maria Marzec
- Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| | - Maciej J Wróbel
- ENT Scientific Club, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland, Department of Otolaryngology and Laryngological Oncology, Collegium Medicum, Nicolaus Copernicus University in Bydgoszcz, Poland
| |
Collapse
|
2
|
Kim H, Ratkute V, Epp B. Monaural and binaural masking release with speech-like stimuli. JASA EXPRESS LETTERS 2024; 4:094404. [PMID: 39287502 DOI: 10.1121/10.0028736] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2024] [Accepted: 08/28/2024] [Indexed: 09/19/2024]
Abstract
The relevance of comodulation and interaural phase difference for speech perception is still unclear. We used speech-like stimuli to link spectro-temporal properties of formants with masking release. The stimuli comprised a tone and three masker bands centered at formant frequencies F1, F2, and F3 derived from a consonant-vowel. The target was a diotic or dichotic frequency-modulated tone following F2 trajectories. Results showed a small comodulation masking release, while the binaural masking level difference was comparable to previous findings. The data suggest that factors other than comodulation may play a dominant role in grouping frequency components in speech.
Collapse
Affiliation(s)
- Hyojin Kim
- Auditory Physics Group, Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, ; ;
| | - Viktorija Ratkute
- Auditory Physics Group, Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, ; ;
| | - Bastian Epp
- Auditory Physics Group, Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, ; ;
| |
Collapse
|
3
|
Heller CR, Hamersky GR, David SV. Task-specific invariant representation in auditory cortex. eLife 2024; 12:RP89936. [PMID: 39172655 PMCID: PMC11341091 DOI: 10.7554/elife.89936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/24/2024] Open
Abstract
Categorical sensory representations are critical for many behaviors, including speech perception. In the auditory system, categorical information is thought to arise hierarchically, becoming increasingly prominent in higher-order cortical regions. The neural mechanisms that support this robust and flexible computation remain poorly understood. Here, we studied sound representations in the ferret primary and non-primary auditory cortex while animals engaged in a challenging sound discrimination task. Population-level decoding of simultaneously recorded single neurons revealed that task engagement caused categorical sound representations to emerge in non-primary auditory cortex. In primary auditory cortex, task engagement caused a general enhancement of sound decoding that was not specific to task-relevant categories. These findings are consistent with mixed selectivity models of neural disentanglement, in which early sensory regions build an overcomplete representation of the world and allow neurons in downstream brain regions to flexibly and selectively read out behaviorally relevant, categorical information.
Collapse
Affiliation(s)
- Charles R Heller
- Neuroscience Graduate Program, Oregon Health and Science UniversityPortlandUnited States
| | - Gregory R Hamersky
- Neuroscience Graduate Program, Oregon Health and Science UniversityPortlandUnited States
| | - Stephen V David
- Otolaryngology, Oregon Health & Science UniversityPortlandUnited States
| |
Collapse
|
4
|
Vaziri PA, McDougle SD, Clark DA. Humans use local spectrotemporal correlations to detect rising and falling pitch. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.08.03.606481. [PMID: 39131316 PMCID: PMC11312537 DOI: 10.1101/2024.08.03.606481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/13/2024]
Abstract
To discern speech or appreciate music, the human auditory system detects how pitch increases or decreases over time. However, the algorithms used to detect changes in pitch, or pitch motion, are incompletely understood. Here, using psychophysics, computational modeling, functional neuroimaging, and analysis of recorded speech, we ask if humans detect pitch motion using computations analogous to those used by the visual system. We adapted stimuli from studies of vision to create novel auditory correlated noise stimuli that elicited robust pitch motion percepts. Crucially, these stimuli possess no persistent features across frequency or time, but do possess positive or negative local spectrotemporal correlations in intensity. In psychophysical experiments, we found clear evidence that humans judge pitch direction based on both positive and negative spectrotemporal correlations. The observed sensitivity to negative correlations is a direct analogue of illusory "reverse-phi" motion in vision, and thus constitutes a new auditory illusion. Our behavioral results and computational modeling led us to hypothesize that human auditory processing employs pitch direction opponency. fMRI measurements in auditory cortex supported this hypothesis. To link our psychophysical findings to real-world pitch perception, we analyzed recordings of English and Mandarin speech and discovered that pitch direction was robustly signaled by the same positive and negative spectrotemporal correlations used in our psychophysical tests, suggesting that sensitivity to both positive and negative correlations confers ecological benefits. Overall, this work reveals that motion detection algorithms sensitive to local correlations are deployed by the central nervous system across disparate modalities (vision and audition) and dimensions (space and frequency).
Collapse
Affiliation(s)
| | - Samuel D McDougle
- Dept of Psychology, Yale University, New Haven, CT 06511
- Wu Tsai Institute, Yale University, New Haven, CT 06511
| | - Damon A Clark
- Wu Tsai Institute, Yale University, New Haven, CT 06511
- Dept of Molecular Cellular and Developmental Biology, Yale University, New Haven, CT 06511
- Dept of Physics, Yale University, New Haven, CT 06511
- Dept of Neuroscience, Yale University, New Haven, CT 06511
- Quantitative Biology Institute, Yale University, New Haven, CT 06511
| |
Collapse
|
5
|
Uemura M, Katagiri Y, Imai E, Kawahara Y, Otani Y, Ichinose T, Kondo K, Kowa H. Dorsal Anterior Cingulate Cortex Coordinates Contextual Mental Imagery for Single-Beat Manipulation during Rhythmic Sensorimotor Synchronization. Brain Sci 2024; 14:757. [PMID: 39199452 PMCID: PMC11352649 DOI: 10.3390/brainsci14080757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2024] [Revised: 07/17/2024] [Accepted: 07/23/2024] [Indexed: 09/01/2024] Open
Abstract
Flexible pulse-by-pulse regulation of sensorimotor synchronization is crucial for voluntarily showing rhythmic behaviors synchronously with external cueing; however, the underpinning neurophysiological mechanisms remain unclear. We hypothesized that the dorsal anterior cingulate cortex (dACC) plays a key role by coordinating both proactive and reactive motor outcomes based on contextual mental imagery. To test our hypothesis, a missing-oddball task in finger-tapping paradigms was conducted in 33 healthy young volunteers. The dynamic properties of the dACC were evaluated by event-related deep-brain activity (ER-DBA), supported by event-related potential (ERP) analysis and behavioral evaluation based on signal detection theory. We found that ER-DBA activation/deactivation reflected a strategic choice of motor control modality in accordance with mental imagery. Reverse ERP traces, as omission responses, confirmed that the imagery was contextual. We found that mental imagery was updated only by environmental changes via perceptual evidence and response-based abductive reasoning. Moreover, stable on-pulse tapping was achievable by maintaining proactive control while creating an imagery of syncopated rhythms from simple beat trains, whereas accuracy was degraded with frequent erroneous tapping for missing pulses. We conclude that the dACC voluntarily regulates rhythmic sensorimotor synchronization by utilizing contextual mental imagery based on experience and by creating novel rhythms.
Collapse
Affiliation(s)
- Maho Uemura
- Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe 654-0142, Japan; (Y.O.); (H.K.)
- School of Music, Mukogawa Women’s University, Nishinomiya 663-8558, Japan;
| | - Yoshitada Katagiri
- Department of Bioengineering, School of Engineering, The University of Tokyo, Tokyo 113-8655, Japan;
| | - Emiko Imai
- Department of Biophysics, Kobe University Graduate School of Health Sciences, Kobe 654-0142, Japan;
| | - Yasuhiro Kawahara
- Department of Human life and Health Sciences, Division of Arts and Sciences, The Open University of Japan, Chiba 261-8586, Japan;
| | - Yoshitaka Otani
- Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe 654-0142, Japan; (Y.O.); (H.K.)
- Faculty of Rehabilitation, Kobe International University, Kobe 658-0032, Japan
| | - Tomoko Ichinose
- School of Music, Mukogawa Women’s University, Nishinomiya 663-8558, Japan;
| | | | - Hisatomo Kowa
- Department of Rehabilitation Science, Kobe University Graduate School of Health Sciences, Kobe 654-0142, Japan; (Y.O.); (H.K.)
| |
Collapse
|
6
|
te Rietmolen N, Mercier MR, Trébuchon A, Morillon B, Schön D. Speech and music recruit frequency-specific distributed and overlapping cortical networks. eLife 2024; 13:RP94509. [PMID: 39038076 PMCID: PMC11262799 DOI: 10.7554/elife.94509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/24/2024] Open
Abstract
To what extent does speech and music processing rely on domain-specific and domain-general neural networks? Using whole-brain intracranial EEG recordings in 18 epilepsy patients listening to natural, continuous speech or music, we investigated the presence of frequency-specific and network-level brain activity. We combined it with a statistical approach in which a clear operational distinction is made between shared, preferred, and domain-selective neural responses. We show that the majority of focal and network-level neural activity is shared between speech and music processing. Our data also reveal an absence of anatomical regional selectivity. Instead, domain-selective neural responses are restricted to distributed and frequency-specific coherent oscillations, typical of spectral fingerprints. Our work highlights the importance of considering natural stimuli and brain dynamics in their full complexity to map cognitive and brain functions.
Collapse
Affiliation(s)
- Noémie te Rietmolen
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Manuel R Mercier
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Agnès Trébuchon
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
- APHM, Hôpital de la Timone, Service de Neurophysiologie CliniqueMarseilleFrance
| | - Benjamin Morillon
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Daniele Schön
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| |
Collapse
|
7
|
Di Tullio RW, Wei L, Balasubramanian V. Slow and steady: auditory features for discriminating animal vocalizations. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.20.599962. [PMID: 39005308 PMCID: PMC11244870 DOI: 10.1101/2024.06.20.599962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/16/2024]
Abstract
We propose that listeners can use temporal regularities - spectro-temporal correlations that change smoothly over time - to discriminate animal vocalizations within and between species. To test this idea, we used Slow Feature Analysis (SFA) to find the most temporally regular components of vocalizations from birds (blue jay, house finch, American yellow warbler, and great blue heron), humans (English speakers), and rhesus macaques. We projected vocalizations into the learned feature space and tested intra-class (same speaker/species) and inter-class (different speakers/species) auditory discrimination by a trained classifier. We found that: 1) Vocalization discrimination was excellent (> 95%) in all cases; 2) Performance depended primarily on the ~10 most temporally regular features; 3) Most vocalizations are dominated by ~10 features with high temporal regularity; and 4) These regular features are highly correlated with the most predictable components of animal sounds.
Collapse
Affiliation(s)
- Ronald W Di Tullio
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, USA
- Computational Neuroscience Initiative, University of Pennsylvania, USA
| | - Linran Wei
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylkvania, USA
| | - Vijay Balasubramanian
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, USA
- Computational Neuroscience Initiative, University of Pennsylvania, USA
- Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA
| |
Collapse
|
8
|
Bellmann OT, Asano R. Neural correlates of musical timbre: an ALE meta-analysis of neuroimaging data. Front Neurosci 2024; 18:1373232. [PMID: 38952924 PMCID: PMC11215185 DOI: 10.3389/fnins.2024.1373232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2024] [Accepted: 05/29/2024] [Indexed: 07/03/2024] Open
Abstract
Timbre is a central aspect of music that allows listeners to identify musical sounds and conveys musical emotion, but also allows for the recognition of actions and is an important structuring property of music. The former functions are known to be implemented in a ventral auditory stream in processing musical timbre. While the latter functions are commonly attributed to areas in a dorsal auditory processing stream in other musical domains, its involvement in musical timbre processing is so far unknown. To investigate if musical timbre processing involves both dorsal and ventral auditory pathways, we carried out an activation likelihood estimation (ALE) meta-analysis of 18 experiments from 17 published neuroimaging studies on musical timbre perception. We identified consistent activations in Brodmann areas (BA) 41, 42, and 22 in the bilateral transverse temporal gyri, the posterior superior temporal gyri and planum temporale, in BA 40 of the bilateral inferior parietal lobe, in BA 13 in the bilateral posterior Insula, and in BA 13 and 22 in the right anterior insula and superior temporal gyrus. The vast majority of the identified regions are associated with the dorsal and ventral auditory processing streams. We therefore propose to frame the processing of musical timbre in a dual-stream model. Moreover, the regions activated in processing timbre show similarities to the brain regions involved in processing several other fundamental aspects of music, indicating possible shared neural bases of musical timbre and other musical domains.
Collapse
Affiliation(s)
| | - Rie Asano
- Systematic Musicology, Institute for Musicology, University of Cologne, Cologne, Germany
| |
Collapse
|
9
|
Sharma H, Azouz R. Reliability and stability of tactile perception in the whisker somatosensory system. Front Neurosci 2024; 18:1344758. [PMID: 38872944 PMCID: PMC11169650 DOI: 10.3389/fnins.2024.1344758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2023] [Accepted: 05/14/2024] [Indexed: 06/15/2024] Open
Abstract
Rodents rely on their whiskers as vital sensory tools for tactile perception, enabling them to distinguish textures and shapes. Ensuring the reliability and constancy of tactile perception under varying stimulus conditions remains a fascinating and fundamental inquiry. This study explores the impact of stimulus configurations, including whisker movement velocity and object spatial proximity, on texture discrimination and stability in rats. To address this issue, we employed three distinct approaches for our investigation. Stimulus configurations notably affected tactile inputs, altering whisker vibration's kinetic and kinematic aspects with consistent effects across various textures. Through a texture discrimination task, rats exhibited consistent discrimination performance irrespective of changes in stimulus configuration. However, alterations in stimulus configuration significantly affected the rats' ability to maintain stability in texture perception. Additionally, we investigated the influence of stimulus configurations on cortical neuronal responses by manipulating them experimentally. Notably, cortical neurons demonstrated substantial and intricate changes in firing rates without compromising the ability to discriminate between textures. Nevertheless, these changes resulted in a reduction in texture neuronal response stability. Stimulating multiple whiskers led to improved neuronal texture discrimination and maintained coding stability. These findings emphasize the importance of considering numerous factors and their interactions when studying the impact of stimulus configuration on neuronal responses and behavior.
Collapse
Affiliation(s)
| | - Rony Azouz
- Department of Physiology and Cell Biology, Zlotowski Center for Neuroscience, Ben-Gurion University of the Negev, Be’er Sheva, Israel
| |
Collapse
|
10
|
Lu K, Dutta K, Mohammed A, Elhilali M, Shamma S. Temporal-Coherence Induces Binding of Responses to Sound Sequences in Ferret Auditory Cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.21.595170. [PMID: 38854125 PMCID: PMC11160575 DOI: 10.1101/2024.05.21.595170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2024]
Abstract
Binding the attributes of a sensory source is necessary to perceive it as a unified entity, one that can be attended to and extracted from its surrounding scene. In auditory perception, this is the essence of the cocktail party problem in which a listener segregates one speaker from a mixture of voices, or a musical stream from simultaneous others. It is postulated that coherence of the temporal modulations of a source's features is necessary to bind them. The focus of this study is on the role of temporal-coherence in binding and segregation, and specifically as evidenced by the neural correlates of rapid plasticity that enhance cortical responses among synchronized neurons, while suppressing them among asynchronized ones. In a first experiment, we find that attention to a sound sequence rapidly binds it to other coherent sequences while suppressing nearby incoherent sequences, thus enhancing the contrast between the two groups. In a second experiment, a sequence of synchronized multi-tone complexes, embedded in a cloud of randomly dispersed background of desynchronized tones, perceptually and neurally pops-out after a fraction of a second highlighting the binding among its coherent tones against the incoherent background. These findings demonstrate the role of temporal-coherence in binding and segregation.
Collapse
Affiliation(s)
- Kai Lu
- Emory University Medical School
| | - Kelsey Dutta
- Electrical and Computer Engineering Department & Institute for Systems Research, University of Maryland College Park
| | - Ali Mohammed
- Electrical and Computer Engineering Department & Institute for Systems Research, University of Maryland College Park
| | - Mounya Elhilali
- Electrical and Computer Engineering, The Johns Hopkins University
| | - Shihab Shamma
- Electrical and Computer Engineering Department & Institute for Systems Research, University of Maryland College Park
- Départment d'étude Cognitives, école normale supérieure, PSL
| |
Collapse
|
11
|
van der Heijden K, Patel P, Bickel S, Herrero JL, Mehta AD, Mesgarani N. Joint population coding and temporal coherence link an attended talker's voice and location features in naturalistic multi-talker scenes. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.13.593814. [PMID: 38798551 PMCID: PMC11118436 DOI: 10.1101/2024.05.13.593814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Listeners readily extract multi-dimensional auditory objects such as a 'localized talker' from complex acoustic scenes with multiple talkers. Yet, the neural mechanisms underlying simultaneous encoding and linking of different sound features - for example, a talker's voice and location - are poorly understood. We analyzed invasive intracranial recordings in neurosurgical patients attending to a localized talker in real-life cocktail party scenarios. We found that sensitivity to an individual talker's voice and location features was distributed throughout auditory cortex and that neural sites exhibited a gradient from sensitivity to a single feature to joint sensitivity to both features. On a population level, cortical response patterns of both dual-feature sensitive sites but also single-feature sensitive sites revealed simultaneous encoding of an attended talker's voice and location features. However, for single-feature sensitive sites, the representation of the primary feature was more precise. Further, sites which selective tracked an attended speech stream concurrently encoded an attended talker's voice and location features, indicating that such sites combine selective tracking of an attended auditory object with encoding of the object's features. Finally, we found that attending a localized talker selectively enhanced temporal coherence between single-feature voice sensitive sites and single-feature location sensitive sites, providing an additional mechanism for linking voice and location in multi-talker scenes. These results demonstrate that a talker's voice and location features are linked during multi-dimensional object formation in naturalistic multi-talker scenes by joint population coding as well as by temporal coherence between neural sites. SIGNIFICANCE STATEMENT Listeners effortlessly extract auditory objects from complex acoustic scenes consisting of multiple sound sources in naturalistic, spatial sound scenes. Yet, how the brain links different sound features to form a multi-dimensional auditory object is poorly understood. We investigated how neural responses encode and integrate an attended talker's voice and location features in spatial multi-talker sound scenes to elucidate which neural mechanisms underlie simultaneous encoding and linking of different auditory features. Our results show that joint population coding as well as temporal coherence mechanisms contribute to distributed multi-dimensional auditory object encoding. These findings shed new light on cortical functional specialization and multidimensional auditory object formation in complex, naturalistic listening scenes. HIGHLIGHTS Cortical responses to an single talker exhibit a distributed gradient, ranging from sites that are sensitive to both a talker's voice and location (dual-feature sensitive sites) to sites that are sensitive to either voice or location (single-feature sensitive sites).Population response patterns of dual-feature sensitive sites encode voice and location features of the attended talker in multi-talker scenes jointly and with equal precision.Despite their sensitivity to a single feature at the level of individual cortical sites, population response patterns of single-feature sensitive sites also encode location and voice features of a talker jointly, but with higher precision for the feature they are primarily sensitive to.Neural sites which selectively track an attended speech stream concurrently encode the attended talker's voice and location features.Attention selectively enhances temporal coherence between voice and location selective sites over time.Joint population coding as well as temporal coherence mechanisms underlie distributed multi-dimensional auditory object encoding in auditory cortex.
Collapse
|
12
|
Boncz Á, Szalárdy O, Velősy PK, Béres L, Baumgartner R, Winkler I, Tóth B. The effects of aging and hearing impairment on listening in noise. iScience 2024; 27:109295. [PMID: 38558934 PMCID: PMC10981015 DOI: 10.1016/j.isci.2024.109295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Revised: 09/19/2023] [Accepted: 02/16/2024] [Indexed: 04/04/2024] Open
Abstract
The study investigates age-related decline in listening abilities, particularly in noisy environments, where the challenge lies in extracting meaningful information from variable sensory input (figure-ground segregation). The research focuses on peripheral and central factors contributing to this decline using a tone-cloud-based figure detection task. Results based on behavioral measures and event-related brain potentials (ERPs) indicate that, despite delayed perceptual processes and some deterioration in attention and executive functions with aging, the ability to detect sound sources in noise remains relatively intact. However, even mild hearing impairment significantly hampers the segregation of individual sound sources within a complex auditory scene. The severity of the hearing deficit correlates with an increased susceptibility to masking noise. The study underscores the impact of hearing impairment on auditory scene analysis and highlights the need for personalized interventions based on individual abilities.
Collapse
Affiliation(s)
- Ádám Boncz
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
| | - Orsolya Szalárdy
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
- Institute of Behavioural Sciences, Faculty of Medicine, Semmelweis University, Budapest, Hungary
| | - Péter Kristóf Velősy
- Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Luca Béres
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
- Department of Cognitive Science, Budapest University of Technology and Economics, Budapest, Hungary
| | - Robert Baumgartner
- Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria
| | - István Winkler
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
| | - Brigitta Tóth
- Institute of Cognitive Neuroscience and Psychology, HUN-REN Research Centre for Natural Sciences, Budapest, Hungary
| |
Collapse
|
13
|
Bidelman GM, Bernard F, Skubic K. Hearing in categories aids speech streaming at the "cocktail party". BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.03.587795. [PMID: 38617284 PMCID: PMC11014555 DOI: 10.1101/2024.04.03.587795] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Our perceptual system bins elements of the speech signal into categories to make speech perception manageable. Here, we aimed to test whether hearing speech in categories (as opposed to a continuous/gradient fashion) affords yet another benefit to speech recognition: parsing noisy speech at the "cocktail party." We measured speech recognition in a simulated 3D cocktail party environment. We manipulated task difficulty by varying the number of additional maskers presented at other spatial locations in the horizontal soundfield (1-4 talkers) and via forward vs. time-reversed maskers, promoting more and less informational masking (IM), respectively. In separate tasks, we measured isolated phoneme categorization using two-alternative forced choice (2AFC) and visual analog scaling (VAS) tasks designed to promote more/less categorical hearing and thus test putative links between categorization and real-world speech-in-noise skills. We first show that listeners can only monitor up to ~3 talkers despite up to 5 in the soundscape and streaming is not related to extended high-frequency hearing thresholds (though QuickSIN scores are). We then confirm speech streaming accuracy and speed decline with additional competing talkers and amidst forward compared to reverse maskers with added IM. Dividing listeners into "discrete" vs. "continuous" categorizers based on their VAS labeling (i.e., whether responses were binary or continuous judgments), we then show the degree of IM experienced at the cocktail party is predicted by their degree of categoricity in phoneme labeling; more discrete listeners are less susceptible to IM than their gradient responding peers. Our results establish a link between speech categorization skills and cocktail party processing, with a categorical (rather than gradient) listening strategy benefiting degraded speech perception. These findings imply figure-ground deficits common in many disorders might arise through a surprisingly simple mechanism: a failure to properly bin sounds into categories.
Collapse
Affiliation(s)
- Gavin M. Bidelman
- Department of Speech, Language and Hearing Sciences, Indiana University, Bloomington, IN, USA
- Program in Neuroscience, Indiana University, Bloomington, IN, USA
- Cognitive Science Program, Indiana University, Bloomington, IN, USA
| | - Fallon Bernard
- School of Communication Sciences & Disorders, University of Memphis, Memphis TN, USA
| | - Kimberly Skubic
- School of Communication Sciences & Disorders, University of Memphis, Memphis TN, USA
| |
Collapse
|
14
|
Viswanathan V, Rupp KM, Hect JL, Harford EE, Holt LL, Abel TJ. Intracranial Mapping of Response Latencies and Task Effects for Spoken Syllable Processing in the Human Brain. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.04.05.588349. [PMID: 38617227 PMCID: PMC11014624 DOI: 10.1101/2024.04.05.588349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/16/2024]
Abstract
Prior lesion, noninvasive-imaging, and intracranial-electroencephalography (iEEG) studies have documented hierarchical, parallel, and distributed characteristics of human speech processing. Yet, there have not been direct, intracranial observations of the latency with which regions outside the temporal lobe respond to speech, or how these responses are impacted by task demands. We leveraged human intracranial recordings via stereo-EEG to measure responses from diverse forebrain sites during (i) passive listening to /bi/ and /pi/ syllables, and (ii) active listening requiring /bi/-versus-/pi/ categorization. We find that neural response latency increases from a few tens of ms in Heschl's gyrus (HG) to several tens of ms in superior temporal gyrus (STG), superior temporal sulcus (STS), and early parietal areas, and hundreds of ms in later parietal areas, insula, frontal cortex, hippocampus, and amygdala. These data also suggest parallel flow of speech information dorsally and ventrally, from HG to parietal areas and from HG to STG and STS, respectively. Latency data also reveal areas in parietal cortex, frontal cortex, hippocampus, and amygdala that are not responsive to the stimuli during passive listening but are responsive during categorization. Furthermore, multiple regions-spanning auditory, parietal, frontal, and insular cortices, and hippocampus and amygdala-show greater neural response amplitudes during active versus passive listening (a task-related effect). Overall, these results are consistent with hierarchical processing of speech at a macro level and parallel streams of information flow in temporal and parietal regions. These data also reveal regions where the speech code is stimulus-faithful and those that encode task-relevant representations.
Collapse
Affiliation(s)
- Vibha Viswanathan
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15260
| | - Kyle M. Rupp
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15260
| | - Jasmine L. Hect
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15260
| | - Emily E. Harford
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15260
| | - Lori L. Holt
- Department of Psychology, The University of Texas at Austin, Austin, TX 78712
| | - Taylor J. Abel
- Department of Neurological Surgery, University of Pittsburgh, Pittsburgh, PA 15260
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15238
| |
Collapse
|
15
|
Testard C, Tremblay S, Parodi F, DiTullio RW, Acevedo-Ithier A, Gardiner KL, Kording K, Platt ML. Neural signatures of natural behaviour in socializing macaques. Nature 2024; 628:381-390. [PMID: 38480888 DOI: 10.1038/s41586-024-07178-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 02/07/2024] [Indexed: 03/18/2024]
Abstract
Our understanding of the neurobiology of primate behaviour largely derives from artificial tasks in highly controlled laboratory settings, overlooking most natural behaviours that primate brains evolved to produce1-3. How primates navigate the multidimensional social relationships that structure daily life4 and shape survival and reproductive success5 remains largely unclear at the single-neuron level. Here we combine ethological analysis, computer vision and wireless recording technologies to identify neural signatures of natural behaviour in unrestrained, socially interacting pairs of rhesus macaques. Single-neuron and population activity in the prefrontal and temporal cortex robustly encoded 24 species-typical behaviours, as well as social context. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioural mechanism supporting friendships and alliances6, and neural activity maintained a running account of these social investments. Confronted with an aggressive intruder, behavioural and neural population responses reflected empathy and were buffered by the presence of a partner. Our findings reveal a highly distributed neurophysiological ledger of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.
Collapse
Affiliation(s)
- Camille Testard
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA.
- Department of Molecular and Cellular Biology, Harvard University, Cambridge, MA, USA.
| | - Sébastien Tremblay
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
- Department of Psychiatry & Neuroscience, Université Laval, Québec, Québec, Canada
| | - Felipe Parodi
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
| | - Ron W DiTullio
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
| | | | - Kristin L Gardiner
- Department of Pathobiology, University of Pennsylvania, Philadelphia, PA, USA
| | - Konrad Kording
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael L Platt
- Department of Neuroscience, University of Pennsylvania, Philadelphia, PA, USA
- Department of Marketing, University of Pennsylvania, Philadelphia, PA, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
16
|
McMullin MA, Kumar R, Higgins NC, Gygi B, Elhilali M, Snyder JS. Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception. Open Mind (Camb) 2024; 8:333-365. [PMID: 38571530 PMCID: PMC10990578 DOI: 10.1162/opmi_a_00131] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 02/10/2024] [Indexed: 04/05/2024] Open
Abstract
Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.
Collapse
Affiliation(s)
| | - Rohit Kumar
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Nathan C. Higgins
- Department of Communication Sciences & Disorders, University of South Florida, Tampa, FL, USA
| | - Brian Gygi
- East Bay Institute for Research and Education, Martinez, CA, USA
| | - Mounya Elhilali
- Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD, USA
| | - Joel S. Snyder
- Department of Psychology, University of Nevada, Las Vegas, Las Vegas, NV, USA
| |
Collapse
|
17
|
Noyce AL, Varghese L, Mathias SR, Shinn-Cunningham BG. Perceptual organization and task demands jointly shape auditory working memory capacity. JASA EXPRESS LETTERS 2024; 4:034402. [PMID: 38526127 PMCID: PMC10966505 DOI: 10.1121/10.0025392] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/17/2023] [Accepted: 03/08/2024] [Indexed: 03/26/2024]
Abstract
Listeners performed two different tasks in which they remembered short sequences comprising either complex tones (generally heard as one melody) or everyday sounds (generally heard as separate objects). In one, listeners judged whether a probe item had been present in the preceding sequence. In the other, they judged whether a second sequence of the same items was identical in order to the preceding sequence. Performance on the first task was higher for everyday sounds; performance on the second was higher for complex tones. Perceptual organization strongly shapes listeners' memory for sounds, with implications for real-world communication.
Collapse
Affiliation(s)
- Abigail L Noyce
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, USA
| | - Leonard Varghese
- Department of Biomedical Engineering, Boston University, Boston, Massachusetts 02215, USA
| | - Samuel R Mathias
- Department of Psychiatry, Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts 02115, , , ,
| | | |
Collapse
|
18
|
Alamatsaz N, Rosen MJ, Ihlefeld A. Increased reliance on temporal coding when target sound is softer than the background. Sci Rep 2024; 14:4457. [PMID: 38396044 PMCID: PMC10891139 DOI: 10.1038/s41598-024-54865-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Accepted: 02/17/2024] [Indexed: 02/25/2024] Open
Abstract
Everyday environments often contain multiple concurrent sound sources that fluctuate over time. Normally hearing listeners can benefit from high signal-to-noise ratios (SNRs) in energetic dips of temporally fluctuating background sound, a phenomenon called dip-listening. Specialized mechanisms of dip-listening exist across the entire auditory pathway. Both the instantaneous fluctuating and the long-term overall SNR shape dip-listening. An unresolved issue regarding cortical mechanisms of dip-listening is how target perception remains invariant to overall SNR, specifically, across different tone levels with an ongoing fluctuating masker. Equivalent target detection over both positive and negative overall SNRs (SNR invariance) is reliably achieved in highly-trained listeners. Dip-listening is correlated with the ability to resolve temporal fine structure, which involves temporally-varying spike patterns. Thus the current work tests the hypothesis that at negative SNRs, neuronal readout mechanisms need to increasingly rely on decoding strategies based on temporal spike patterns, as opposed to spike count. Recordings from chronically implanted electrode arrays in core auditory cortex of trained and awake Mongolian gerbils that are engaged in a tone detection task in 10 Hz amplitude-modulated background sound reveal that rate-based decoding is not SNR-invariant, whereas temporal coding is informative at both negative and positive SNRs.
Collapse
Affiliation(s)
- Nima Alamatsaz
- Graduate School of Biomedical Sciences, Rutgers University, Newark, NJ, USA
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, USA
| | - Merri J Rosen
- Northeast Ohio Medical University (NEOMED), Rootstown, OH, USA.
- University Hospitals Hearing Research Center at NEOMED, Rootstown, OH, USA.
- Brain Health Research Institute, Kent State University, Kent, OH, USA.
| | | |
Collapse
|
19
|
Jigo M, Carmel JB, Wang Q, Rodenkirch C. Transcutaneous cervical vagus nerve stimulation improves sensory performance in humans: a randomized controlled crossover pilot study. Sci Rep 2024; 14:3975. [PMID: 38368486 PMCID: PMC10874458 DOI: 10.1038/s41598-024-54026-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Accepted: 02/07/2024] [Indexed: 02/19/2024] Open
Abstract
Accurate senses depend on high-fidelity encoding by sensory receptors and error-free processing in the brain. Progress has been made towards restoring damaged sensory receptors. However, methods for on-demand treatment of impaired central sensory processing are scarce. Prior invasive studies demonstrated that continuous vagus nerve stimulation (VNS) in rodents can activate the locus coeruleus-norepinephrine system to rapidly improve central sensory processing. Here, we investigated whether transcutaneous VNS improves sensory performance in humans. We conducted three sham-controlled experiments, each with 12 neurotypical adults, that measured the effects of transcutaneous VNS on metrics of auditory and visual performance, and heart rate variability (HRV). Continuous stimulation was delivered to cervical (tcVNS) or auricular (taVNS) branches of the vagus nerve while participants performed psychophysics tasks or passively viewed a display. Relative to sham stimulation, tcVNS improved auditory performance by 37% (p = 0.00052) and visual performance by 23% (p = 0.038). Participants with lower performance during sham conditions experienced larger tcVNS-evoked improvements (p = 0.0040). Lastly, tcVNS increased HRV during passive viewing, corroborating vagal engagement. No evidence for an effect of taVNS was observed. These findings validate the effectiveness of tcVNS in humans and position it as a method for on-demand interventions of impairments associated with central sensory processing dysfunction.
Collapse
Affiliation(s)
| | - Jason B Carmel
- Sharper Sense, Inc., New York, NY, USA
- Department of Neurology and Orthopedics, Columbia University Medical Center, New York, NY, USA
| | - Qi Wang
- Sharper Sense, Inc., New York, NY, USA
- Department of Biomedical Engineering, Columbia University, New York, NY, USA
| | - Charles Rodenkirch
- Sharper Sense, Inc., New York, NY, USA.
- The Jacobs Technion-Cornell Institute at Cornell Tech, New York, NY, USA.
| |
Collapse
|
20
|
Orekhova EV, Fadeev KA, Goiaeva DE, Obukhova TS, Ovsiannikova TM, Prokofyev AO, Stroganova TA. Different hemispheric lateralization for periodicity and formant structure of vowels in the auditory cortex and its changes between childhood and adulthood. Cortex 2024; 171:287-307. [PMID: 38061210 DOI: 10.1016/j.cortex.2023.10.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2023] [Revised: 08/31/2023] [Accepted: 10/30/2023] [Indexed: 02/12/2024]
Abstract
The spectral formant structure and periodicity pitch are the major features that determine the identity of vowels and the characteristics of the speaker. However, very little is known about how the processing of these features in the auditory cortex changes during development. To address this question, we independently manipulated the periodicity and formant structure of vowels while measuring auditory cortex responses using magnetoencephalography (MEG) in children aged 7-12 years and adults. We analyzed the sustained negative shift of source current associated with these vowel properties, which was present in the auditory cortex in both age groups despite differences in the transient components of the auditory response. In adults, the sustained activation associated with formant structure was lateralized to the left hemisphere early in the auditory processing stream requiring neither attention nor semantic mapping. This lateralization was not yet established in children, in whom the right hemisphere contribution to formant processing was strong and decreased during or after puberty. In contrast to the formant structure, periodicity was associated with a greater response in the right hemisphere in both children and adults. These findings suggest that left-lateralization for the automatic processing of vowel formant structure emerges relatively late in ontogenesis and pose a serious challenge to current theories of hemispheric specialization for speech processing.
Collapse
Affiliation(s)
- Elena V Orekhova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Kirill A Fadeev
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Dzerassa E Goiaeva
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana S Obukhova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana M Ovsiannikova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Andrey O Prokofyev
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| | - Tatiana A Stroganova
- Center for Neurocognitive Research (MEG Center), Moscow State University of Psychology and Education, Moscow, Russian Federation.
| |
Collapse
|
21
|
Gao J, Chen H, Fang M, Ding N. Original speech and its echo are segregated and separately processed in the human brain. PLoS Biol 2024; 22:e3002498. [PMID: 38358954 PMCID: PMC10868781 DOI: 10.1371/journal.pbio.3002498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 01/15/2024] [Indexed: 02/17/2024] Open
Abstract
Speech recognition crucially relies on slow temporal modulations (<16 Hz) in speech. Recent studies, however, have demonstrated that the long-delay echoes, which are common during online conferencing, can eliminate crucial temporal modulations in speech but do not affect speech intelligibility. Here, we investigated the underlying neural mechanisms. MEG experiments demonstrated that cortical activity can effectively track the temporal modulations eliminated by an echo, which cannot be fully explained by basic neural adaptation mechanisms. Furthermore, cortical responses to echoic speech can be better explained by a model that segregates speech from its echo than by a model that encodes echoic speech as a whole. The speech segregation effect was observed even when attention was diverted but would disappear when segregation cues, i.e., speech fine structure, were removed. These results strongly suggested that, through mechanisms such as stream segregation, the auditory system can build an echo-insensitive representation of speech envelope, which can support reliable speech recognition.
Collapse
Affiliation(s)
- Jiaxin Gao
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
| | - Honghua Chen
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
| | - Mingxuan Fang
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
| | - Nai Ding
- Key Laboratory for Biomedical Engineering of Ministry of Education, College of Biomedical Engineering and Instrument Sciences, Zhejiang University, Hangzhou, China
- Nanhu Brain-computer Interface Institute, Hangzhou, China
- The State key Lab of Brain-Machine Intelligence; The MOE Frontier Science Center for Brain Science & Brain-machine Integration, Zhejiang University, Hangzhou, China
| |
Collapse
|
22
|
Anderson SR, Burg E, Suveg L, Litovsky RY. Review of Binaural Processing With Asymmetrical Hearing Outcomes in Patients With Bilateral Cochlear Implants. Trends Hear 2024; 28:23312165241229880. [PMID: 38545645 PMCID: PMC10976506 DOI: 10.1177/23312165241229880] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Revised: 01/11/2024] [Accepted: 01/16/2024] [Indexed: 04/01/2024] Open
Abstract
Bilateral cochlear implants (BiCIs) result in several benefits, including improvements in speech understanding in noise and sound source localization. However, the benefit bilateral implants provide among recipients varies considerably across individuals. Here we consider one of the reasons for this variability: difference in hearing function between the two ears, that is, interaural asymmetry. Thus far, investigations of interaural asymmetry have been highly specialized within various research areas. The goal of this review is to integrate these studies in one place, motivating future research in the area of interaural asymmetry. We first consider bottom-up processing, where binaural cues are represented using excitation-inhibition of signals from the left ear and right ear, varying with the location of the sound in space, and represented by the lateral superior olive in the auditory brainstem. We then consider top-down processing via predictive coding, which assumes that perception stems from expectations based on context and prior sensory experience, represented by cascading series of cortical circuits. An internal, perceptual model is maintained and updated in light of incoming sensory input. Together, we hope that this amalgamation of physiological, behavioral, and modeling studies will help bridge gaps in the field of binaural hearing and promote a clearer understanding of the implications of interaural asymmetry for future research on optimal patient interventions.
Collapse
Affiliation(s)
- Sean R. Anderson
- Waisman Center, University of Wisconsin-Madison, Madison, WI, USA
- Department of Physiology and Biophysics, University of Colorado Anschutz Medical School, Aurora, CO, USA
| | - Emily Burg
- Waisman Center, University of Wisconsin-Madison, Madison, WI, USA
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Lukas Suveg
- Waisman Center, University of Wisconsin-Madison, Madison, WI, USA
| | - Ruth Y. Litovsky
- Waisman Center, University of Wisconsin-Madison, Madison, WI, USA
- Department of Communication Sciences and Disorders, University of Wisconsin-Madison, Madison, WI, USA
- Department of Surgery, Division of Otolaryngology, University of Wisconsin-Madison, Madison, WI, USA
| |
Collapse
|
23
|
Palaparthi A, Alluri RK, Titze IR. Deep Learning for Neuromuscular Control of Vocal Source for Voice Production. APPLIED SCIENCES (BASEL, SWITZERLAND) 2024; 14:769. [PMID: 39071945 PMCID: PMC11281313 DOI: 10.3390/app14020769] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
A computational neuromuscular control system that generates lung pressure and three intrinsic laryngeal muscle activations (cricothyroid, thyroarytenoid, and lateral cricoarytenoid) to control the vocal source was developed. In the current study, LeTalker, a biophysical computational model of the vocal system was used as the physical plant. In the LeTalker, a three-mass vocal fold model was used to simulate self-sustained vocal fold oscillation. A constant/ǝ/vowel was used for the vocal tract shape. The trachea was modeled after MRI measurements. The neuromuscular control system generates control parameters to achieve four acoustic targets (fundamental frequency, sound pressure level, normalized spectral centroid, and signal-to-noise ratio) and four somatosensory targets (vocal fold length, and longitudinal fiber stress in the three vocal fold layers). The deep-learning-based control system comprises one acoustic feedforward controller and two feedback (acoustic and somatosensory) controllers. Fifty thousand steady speech signals were generated using the LeTalker for training the control system. The results demonstrated that the control system was able to generate the lung pressure and the three muscle activations such that the four acoustic and four somatosensory targets were reached with high accuracy. After training, the motor command corrections from the feedback controllers were minimal compared to the feedforward controller except for thyroarytenoid muscle activation.
Collapse
Affiliation(s)
- Anil Palaparthi
- Utah Center for Vocology, University of Utah, Salt Lake City, UT 84112, USA
| | - Rishi K. Alluri
- School of Biological Sciences, University of Utah, Salt Lake City, UT 84112, USA
| | - Ingo R. Titze
- Utah Center for Vocology, University of Utah, Salt Lake City, UT 84112, USA
| |
Collapse
|
24
|
Turpin T, Uluç I, Kotlarz P, Lankinen K, Mamashli F, Ahveninen J. Comparing auditory and visual aspects of multisensory working memory using bimodally matched feature patterns. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.03.551865. [PMID: 37577481 PMCID: PMC10418174 DOI: 10.1101/2023.08.03.551865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/15/2023]
Abstract
Working memory (WM) reflects the transient maintenance of information in the absence of external input, which can be attained via multiple senses separately or simultaneously. Pertaining to WM, the prevailing literature suggests the dominance of vision over other sensory systems. However, this imbalance may be stemming from challenges in finding comparable stimuli across modalities. Here, we addressed this problem by using a balanced multisensory retro-cue WM design, which employed combinations of auditory (ripple sounds) and visuospatial (Gabor patches) patterns, adjusted relative to each participant's discrimination ability. In three separate experiments, the participant was asked to determine whether the (retro-cued) auditory and/or visual items maintained in WM matched or mismatched the subsequent probe stimulus. In Experiment 1, all stimuli were audiovisual, and the probes were either fully mismatching, only partially mismatching, or fully matching the memorized item. Experiment 2 was otherwise same as Experiment 1, but the probes were unimodal. In Experiment 3, the participant was cued to maintain only the auditory or visual aspect of an audiovisual item pair. In two of the three experiments, the participant matching performance was significantly more accurate for the auditory than visual attributes of probes. When the perceptual and task demands are bimodally equated, auditory attributes can be matched to multisensory items in WM at least as accurately as, if not more precisely than, their visual counterparts.
Collapse
Affiliation(s)
- Tori Turpin
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
| | - Işıl Uluç
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Parker Kotlarz
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
| | - Kaisu Lankinen
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Fahimeh Mamashli
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Jyrki Ahveninen
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
25
|
Tuckute G, Feather J, Boebinger D, McDermott JH. Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions. PLoS Biol 2023; 21:e3002366. [PMID: 38091351 PMCID: PMC10718467 DOI: 10.1371/journal.pbio.3002366] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 10/06/2023] [Indexed: 12/18/2023] Open
Abstract
Models that predict brain responses to stimuli provide one measure of understanding of a sensory system and have many potential applications in science and engineering. Deep artificial neural networks have emerged as the leading such predictive models of the visual system but are less explored in audition. Prior work provided examples of audio-trained neural networks that produced good predictions of auditory cortical fMRI responses and exhibited correspondence between model stages and brain regions, but left it unclear whether these results generalize to other neural network models and, thus, how to further improve models in this domain. We evaluated model-brain correspondence for publicly available audio neural network models along with in-house models trained on 4 different tasks. Most tested models outpredicted standard spectromporal filter-bank models of auditory cortex and exhibited systematic model-brain correspondence: Middle stages best predicted primary auditory cortex, while deep stages best predicted non-primary cortex. However, some state-of-the-art models produced substantially worse brain predictions. Models trained to recognize speech in background noise produced better brain predictions than models trained to recognize speech in quiet, potentially because hearing in noise imposes constraints on biological auditory representations. The training task influenced the prediction quality for specific cortical tuning properties, with best overall predictions resulting from models trained on multiple tasks. The results generally support the promise of deep neural networks as models of audition, though they also indicate that current models do not explain auditory cortical responses in their entirety.
Collapse
Affiliation(s)
- Greta Tuckute
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America
- Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, United States of America
| | - Jenelle Feather
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America
- Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, United States of America
| | - Dana Boebinger
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America
- Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, United States of America
- Program in Speech and Hearing Biosciences and Technology, Harvard, Cambridge, Massachusetts, United States of America
- University of Rochester Medical Center, Rochester, New York, New York, United States of America
| | - Josh H. McDermott
- Department of Brain and Cognitive Sciences, McGovern Institute for Brain Research MIT, Cambridge, Massachusetts, United States of America
- Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, United States of America
- Program in Speech and Hearing Biosciences and Technology, Harvard, Cambridge, Massachusetts, United States of America
| |
Collapse
|
26
|
Chillale RK, Shamma S, Ostojic S, Boubenec Y. Dynamics and maintenance of categorical responses in primary auditory cortex during task engagement. eLife 2023; 12:e85706. [PMID: 37970945 DOI: 10.7554/elife.85706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 11/12/2023] [Indexed: 11/19/2023] Open
Abstract
Grouping sets of sounds into relevant categories is an important cognitive ability that enables the association of stimuli with appropriate goal-directed behavioral responses. In perceptual tasks, the primary auditory cortex (A1) assumes a prominent role by concurrently encoding both sound sensory features and task-related variables. Here, we sought to explore the role of A1 in the initiation of sound categorization, shedding light on its involvement in this cognitive process. We trained ferrets to discriminate click trains of different rates in a Go/No-Go delayed categorization task and recorded neural activity during both active behavior and passive exposure to the same sounds. Purely categorical response components were extracted and analyzed separately from sensory responses to reveal their contributions to the overall population response throughout the trials. We found that categorical activity emerged during sound presentation in the population average and was present in both active behavioral and passive states. However, upon task engagement, categorical responses to the No-Go category became suppressed in the population code, leading to an asymmetrical representation of the Go stimuli relative to the No-Go sounds and pre-stimulus baseline. The population code underwent an abrupt change at stimulus offset, with sustained responses after the Go sounds during the delay period. Notably, the categorical responses observed during the stimulus period exhibited a significant correlation with those extracted from the delay epoch, suggesting an early involvement of A1 in stimulus categorization.
Collapse
Affiliation(s)
- Rupesh K Chillale
- Laboratoire des Systèmes Perceptifs, Département d'Études Cognitives, École Normale Supérieure, PSL University,, Paris, France
- Laboratoire de Neurosciences Cognitives Computationnelle (INSERM U960), Département d'Études Cognitives, École Normale Supérieure, Paris, France
| | - Shihab Shamma
- Laboratoire des Systèmes Perceptifs, Département d'Études Cognitives, École Normale Supérieure, PSL University,, Paris, France
- Institute for System Research, Department of Electrical and Computer Engineering, University of Maryland, College Park, College Park, Maryland, United States
| | - Srdjan Ostojic
- Laboratoire de Neurosciences Cognitives Computationnelle (INSERM U960), Département d'Études Cognitives, École Normale Supérieure, Paris, France
| | - Yves Boubenec
- Laboratoire des Systèmes Perceptifs, Département d'Études Cognitives, École Normale Supérieure, PSL University,, Paris, France
| |
Collapse
|
27
|
Obara K, Ebina T, Terada SI, Uka T, Komatsu M, Takaji M, Watakabe A, Kobayashi K, Masamizu Y, Mizukami H, Yamamori T, Kasai K, Matsuzaki M. Change detection in the primate auditory cortex through feedback of prediction error signals. Nat Commun 2023; 14:6981. [PMID: 37957168 PMCID: PMC10643402 DOI: 10.1038/s41467-023-42553-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2022] [Accepted: 10/13/2023] [Indexed: 11/15/2023] Open
Abstract
Although cortical feedback signals are essential for modulating feedforward processing, no feedback error signal across hierarchical cortical areas has been reported. Here, we observed such a signal in the auditory cortex of awake common marmoset during an oddball paradigm to induce auditory duration mismatch negativity. Prediction errors to a deviant tone presentation were generated as offset calcium responses of layer 2/3 neurons in the rostral parabelt (RPB) of higher-order auditory cortex, while responses to non-deviant tones were strongly suppressed. Within several hundred milliseconds, the error signals propagated broadly into layer 1 of the primary auditory cortex (A1) and accumulated locally on top of incoming auditory signals. Blockade of RPB activity prevented deviance detection in A1. Optogenetic activation of RPB following tone presentation nonlinearly enhanced A1 tone response. Thus, the feedback error signal is critical for automatic detection of unpredicted stimuli in physiological auditory processing and may serve as backpropagation-like learning.
Collapse
Affiliation(s)
- Keitaro Obara
- Department of Physiology, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan
- Brain Functional Dynamics Collaboration Laboratory, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
| | - Teppei Ebina
- Department of Physiology, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Shin-Ichiro Terada
- Department of Physiology, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan
| | - Takanori Uka
- Department of Integrative Physiology, Graduate School of Medicine, University of Yamanashi, Yamanashi, 409-3898, Japan
| | - Misako Komatsu
- Laboratory for Molecular Analysis of Higher Brain Function, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
| | - Masafumi Takaji
- Laboratory for Molecular Analysis of Higher Brain Function, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
- Laboratory for Haptic Perception and Cognitive Physiology, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
| | - Akiya Watakabe
- Laboratory for Molecular Analysis of Higher Brain Function, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
- Laboratory for Haptic Perception and Cognitive Physiology, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
| | - Kenta Kobayashi
- Section of Viral Vector Development, National Institute for Physiological Sciences, Aichi, 444-8585, Japan
| | - Yoshito Masamizu
- Brain Functional Dynamics Collaboration Laboratory, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
| | - Hiroaki Mizukami
- Division of Genetic Therapeutics, Center for Molecular Medicine, Jichi Medical University, Tochigi, 329-0498, Japan
| | - Tetsuo Yamamori
- Laboratory for Molecular Analysis of Higher Brain Function, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
- Laboratory for Haptic Perception and Cognitive Physiology, RIKEN Center for Brain Science, Saitama, 351-0198, Japan
- Central Institute of Experimental Animals, Kanagawa, 210-0821, Japan
| | - Kiyoto Kasai
- Department of Neuropsychiatry, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-8655, Japan
- International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo Institutes for Advanced Study, Tokyo, 113-0033, Japan
| | - Masanori Matsuzaki
- Department of Physiology, Graduate School of Medicine, The University of Tokyo, Tokyo, 113-0033, Japan.
- Brain Functional Dynamics Collaboration Laboratory, RIKEN Center for Brain Science, Saitama, 351-0198, Japan.
- International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo Institutes for Advanced Study, Tokyo, 113-0033, Japan.
| |
Collapse
|
28
|
Zhang H, Xie J, Tao Q, Xiao Y, Cui G, Fang W, Zhu X, Xu G, Li M, Han C. The effect of motion frequency and sound source frequency on steady-state auditory motion evoked potential. Hear Res 2023; 439:108897. [PMID: 37871451 DOI: 10.1016/j.heares.2023.108897] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 08/18/2023] [Accepted: 10/12/2023] [Indexed: 10/25/2023]
Abstract
The ability of humans to perceive motion sound sources is important for accurate response to the living environment. Periodic motion sound sources can elicit steady-state motion auditory evoked potential (SSMAEP). The purpose of this study was to investigate the effects of different motion frequencies and different frequencies of sound source on SSMAEP. The stimulation paradigms for simulating periodic motion of sound sources were designed utilizing head-related transfer function (HRTF) techniques in this study. The motion frequencies of the paradigm are set respectively to 1-10 Hz, 15 Hz, 20 Hz, 30 Hz, 40 Hz, 60 Hz, and 80 Hz. In addition, the frequencies of sound source of the paradigms were set to 500 Hz, 1000 Hz, 2000 Hz, 3000 Hz, and 4000 Hz at motion frequencies of 6 Hz and 40 Hz. Fourteen subjects with normal hearing were recruited for the study. SSMAEP was elicited by 500 Hz pure tone at motion frequencies of 1-10 Hz, 15 Hz, 20 Hz, 30 Hz, 40 Hz, 60 Hz, and 80 Hz. SSMAEP was strongest at motion frequencies of 6 Hz. Moreover, at 6 Hz motion frequency, the SSMAEP amplitude was largest at the tone frequency of 500 Hz and smallest at 4000 Hz. Whilst SSMAEP elicited by 4000 Hz pure tone was significantly the strongest at motion frequency of 40 Hz. SSMAEP can be elicited by periodic motion sound sources at motion frequencies up to 80 Hz. SSMAEP also has a strong response at lower frequency. Low-frequency pure tones are beneficial to enhance SSMAEP at low-frequency sound source motion, whilst high-frequency pure tones help to enhance SSMAEP at high-frequency sound source motion. The study provides new insight into the brain's perception of rhythmic auditory motion.
Collapse
Affiliation(s)
- Huanqing Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China
| | - Jun Xie
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China; School of Mechanical Engineering, Xinjiang University, Urumqi, China; National Key Laboratory of Human Factors Engineering, China Astronauts Research and Training Center, Beijing, China; State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, China.
| | - Qing Tao
- School of Mechanical Engineering, Xinjiang University, Urumqi, China.
| | - Yi Xiao
- National Key Laboratory of Human Factors Engineering, China Astronauts Research and Training Center, Beijing, China
| | - Guiling Cui
- National Key Laboratory of Human Factors Engineering, China Astronauts Research and Training Center, Beijing, China
| | - Wenhu Fang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China
| | - Xinyu Zhu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China
| | - Guanghua Xu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China; State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, China
| | - Min Li
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China; State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, China
| | - Chengcheng Han
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, China; State Key Laboratory for Manufacturing Systems Engineering, Xi'an Jiaotong University, Xi'an, China
| |
Collapse
|
29
|
Berthomieu G, Koehl V, Paquier M. Loudness constancy for noise and speech: How instructions and source information affect loudness of distant sounds. Atten Percept Psychophys 2023; 85:2774-2796. [PMID: 37466907 DOI: 10.3758/s13414-023-02719-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/20/2023] [Indexed: 07/20/2023]
Abstract
The physical properties of a sound evolve when traveling away from its source. As an example, the sound pressure level at the listener's ears will vary according to their respective distance and azimuth. However, several studies have reported loudness to remain constant when varying the distance between the source and the listener. This loudness constancy has been reported to occur when the listener focused attention on the sound as emitted by the source (namely the distal stimulus). Instead, the listener can focus on the sound as reaching the ears (namely the proximal stimulus). The instructions given to the listener when assessing loudness can drive focus toward the proximal or distal stimulus. However, focusing on the distal stimulus requires to have sufficient information about the sound source, which could be provided by either the environment or by the stimulus itself. The present study gathers three experiments designed to assess loudness when driving listeners' focus toward the proximal or distal stimuli. Listeners were provided with different quality and quantity of information about the source depending on the environment (visible or hidden sources, free field or reverberant rooms) and on the stimulus itself (noise or speech). The results show that listeners reported constant loudness when asked to focus on the distal stimulus only, provided enough information about the source was available. These results highlight that loudness relies on the way the listener focuses on the stimuli and emphasize the importance of the instructions that are given in loudness studies.
Collapse
Affiliation(s)
| | - Vincent Koehl
- Univ Brest, Lab-STICC, CNRS, UMR 6285, F-29200, Brest, France
| | - Mathieu Paquier
- Univ Brest, Lab-STICC, CNRS, UMR 6285, F-29200, Brest, France
| |
Collapse
|
30
|
Testard C, Tremblay S, Parodi F, DiTullio RW, Acevedo-Ithier A, Gardiner K, Kording KP, Platt M. Neural signatures of natural behavior in socializing macaques. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.05.547833. [PMID: 37461580 PMCID: PMC10349985 DOI: 10.1101/2023.07.05.547833] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/05/2023]
Abstract
Our understanding of the neurobiology of primate behavior largely derives from artificial tasks in highly-controlled laboratory settings, overlooking most natural behaviors primate brains evolved to produce1. In particular, how primates navigate the multidimensional social relationships that structure daily life and shape survival and reproductive success remains largely unexplored at the single neuron level. Here, we combine ethological analysis with new wireless recording technologies to uncover neural signatures of natural behavior in unrestrained, socially interacting pairs of rhesus macaques within a larger colony. Population decoding of single neuron activity in prefrontal and temporal cortex unveiled robust encoding of 24 species-typical behaviors, which was strongly modulated by the presence and identity of surrounding monkeys. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioral mechanism supporting friendships and alliances, and neural activity maintained a running account of these social investments. When confronted with an aggressive intruder, behavioral and neural population responses reflected empathy and were buffered by the presence of a partner. Surprisingly, neural signatures in prefrontal and temporal cortex were largely indistinguishable and irreducible to visual and motor contingencies. By employing an ethological approach to the study of primate neurobiology, we reveal a highly-distributed neurophysiological record of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.
Collapse
|
31
|
Paraouty N, Yao JD, Varnet L, Chou CN, Chung S, Sanes DH. Sensory cortex plasticity supports auditory social learning. Nat Commun 2023; 14:5828. [PMID: 37730696 PMCID: PMC10511464 DOI: 10.1038/s41467-023-41641-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 09/11/2023] [Indexed: 09/22/2023] Open
Abstract
Social learning (SL) through experience with conspecifics can facilitate the acquisition of many behaviors. Thus, when Mongolian gerbils are exposed to a demonstrator performing an auditory discrimination task, their subsequent task acquisition is facilitated, even in the absence of visual cues. Here, we show that transient inactivation of auditory cortex (AC) during exposure caused a significant delay in task acquisition during the subsequent practice phase, suggesting that AC activity is necessary for SL. Moreover, social exposure induced an improvement in AC neuron sensitivity to auditory task cues. The magnitude of neural change during exposure correlated with task acquisition during practice. In contrast, exposure to only auditory task cues led to poorer neurometric and behavioral outcomes. Finally, social information during exposure was encoded in the AC of observer animals. Together, our results suggest that auditory SL is supported by AC neuron plasticity occurring during social exposure and prior to behavioral performance.
Collapse
Affiliation(s)
- Nihaad Paraouty
- Center for Neural Science New York University, New York, NY, 10003, USA.
| | - Justin D Yao
- Department of Otolaryngology, Rutgers University, New Brunswick, NJ, 08901, USA
| | - Léo Varnet
- Laboratoire des Systèmes Perceptifs, UMR 8248, Ecole Normale Supérieure, PSL University, Paris, 75005, France
| | - Chi-Ning Chou
- Center for Computational Neuroscience, Flatiron Institute, Simons Foundation, New York, NY, USA
- School of Engineering & Applied Sciences, Harvard University, Cambridge, MA, 02138, USA
| | - SueYeon Chung
- Center for Neural Science New York University, New York, NY, 10003, USA
- Center for Computational Neuroscience, Flatiron Institute, Simons Foundation, New York, NY, USA
| | - Dan H Sanes
- Center for Neural Science New York University, New York, NY, 10003, USA
- Department of Psychology, New York University, New York, NY, 10003, USA
- Department of Biology, New York University, New York, NY, 10003, USA
- Neuroscience Institute, NYU Langone Medical Center, New York, NY, 10003, USA
| |
Collapse
|
32
|
Sagar V, Shanahan LK, Zelano CM, Gottfried JA, Kahnt T. High-precision mapping reveals the structure of odor coding in the human brain. Nat Neurosci 2023; 26:1595-1602. [PMID: 37620443 PMCID: PMC10726579 DOI: 10.1038/s41593-023-01414-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2022] [Accepted: 07/18/2023] [Indexed: 08/26/2023]
Abstract
Odor perception is inherently subjective. Previous work has shown that odorous molecules evoke distributed activity patterns in olfactory cortices, but how these patterns map on to subjective odor percepts remains unclear. In the present study, we collected neuroimaging responses to 160 odors from 3 individual subjects (18 h per subject) to probe the neural coding scheme underlying idiosyncratic odor perception. We found that activity in the orbitofrontal cortex (OFC) represents the fine-grained perceptual identity of odors over and above coarsely defined percepts, whereas this difference is less pronounced in the piriform cortex (PirC) and amygdala. Furthermore, the implementation of perceptual encoding models enabled us to predict olfactory functional magnetic resonance imaging responses to new odors, revealing that the dimensionality of the encoded perceptual spaces increases from the PirC to the OFC. Whereas encoding of lower-order dimensions generalizes across subjects, encoding of higher-order dimensions is idiosyncratic. These results provide new insights into cortical mechanisms of odor coding and suggest that subjective olfactory percepts reside in the OFC.
Collapse
Affiliation(s)
- Vivek Sagar
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | | | - Christina M Zelano
- Department of Neurology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Jay A Gottfried
- Department of Neurology, University of Pennsylvania, Philadelphia, PA, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Thorsten Kahnt
- National Institute on Drug Abuse Intramural Research Program, Baltimore, MD, USA.
| |
Collapse
|
33
|
Grifoni J, Pagani M, Persichilli G, Bertoli M, Bevacqua MG, L'Abbate T, Flamini I, Brancucci A, Cerniglia L, Paulon L, Tecchio F. Auditory Personalization of EMDR Treatment to Relieve Trauma Effects: A Feasibility Study [EMDR+]. Brain Sci 2023; 13:1050. [PMID: 37508982 PMCID: PMC10377614 DOI: 10.3390/brainsci13071050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 07/02/2023] [Accepted: 07/04/2023] [Indexed: 07/30/2023] Open
Abstract
According to the WHO (World Health Organization), Eye Movement Desensitization and Reprocessing (EMDR) is an elective therapy to treat people with post-traumatic stress disorders (PTSD). In line with the personalization of therapeutic strategies, through this pilot study, we assessed in people suffering from the effects of trauma the feasibility, safety, acceptance, and efficacy of EMDR enriched with sound stimulation (by administering neutral sounds synchronized with the guided bilateral alternating stimulation of the gaze) and musical reward (musical listening based on the patients' predisposition and personal tastes). Feasibility, quantified by the number of patients who completed the treatment, was excellent as this was the case in 12 out of the 12 enrolled people with psychological trauma. Safety and acceptance, assessed by self-compiled questionnaires, were excellent, with an absence of side effects and high satisfaction. Efficacy, quantified by the number of EMDR treatment sessions required to reach the optimal scores on the Subjective Units of Disturbance (SUD) and Validity of Cognition (VOC) scales typical of EMDR protocols, revealed an average duration of 8.5 (SD 1.2) sessions, which is well below the 12 sessions considered a standard EMDR treatment duration. EMDR+ appears to be a relevant personalization of EMDR, particularly in music-sensitive people, consolidating the therapeutic alliance through a multisensory communicative bond for trauma treatment.
Collapse
Affiliation(s)
- Joy Grifoni
- International Telematic University Uninettuno, 00186 Rome, Italy
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| | - Marco Pagani
- International Telematic University Uninettuno, 00186 Rome, Italy
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| | - Giada Persichilli
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| | - Massimo Bertoli
- International Telematic University Uninettuno, 00186 Rome, Italy
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| | | | - Teresa L'Abbate
- International Telematic University Uninettuno, 00186 Rome, Italy
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| | | | - Alfredo Brancucci
- Dipartimento di Scienze Motorie, Umane e della Salute, Università di Roma 'Foro Italico', 00135 Rome, Italy
| | - Luca Cerniglia
- International Telematic University Uninettuno, 00186 Rome, Italy
| | - Luca Paulon
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
- Luca Paulon, Engineer Freelance, 00159 Rome, Italy
| | - Franca Tecchio
- LET'S and LABSS, Institute of Cognitive Sciences and Technologies ISTC, Consiglio Nazionale delle Ricerche CNR, 00185 Rome, Italy
| |
Collapse
|
34
|
Liu W, Vicario DS. Dynamic encoding of phonetic categories in zebra finch auditory forebrain. Sci Rep 2023; 13:11172. [PMID: 37430030 DOI: 10.1038/s41598-023-37982-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2023] [Accepted: 06/30/2023] [Indexed: 07/12/2023] Open
Abstract
Vocal communication requires the formation of acoustic categories to enable invariant representations of sounds despite superficial variations. Humans form acoustic categories for speech phonemes, enabling the listener to recognize words independent of speakers; animals can also discriminate speech phonemes. We investigated the neural mechanisms of this process using electrophysiological recordings from the zebra finch secondary auditory area, caudomedial nidopallium (NCM), during passive exposure to human speech stimuli consisting of two naturally spoken words produced by multiple speakers. Analysis of neural distance and decoding accuracy showed improvements in neural discrimination between word categories over the course of exposure, and this improved representation transferred to the same words by novel speakers. We conclude that NCM neurons formed generalized representations of word categories independent of speaker-specific variations that became more refined over the course of passive exposure. The discovery of this dynamic encoding process in NCM suggests a general processing mechanism for forming categorical representations of complex acoustic signals that humans share with other animals.
Collapse
Affiliation(s)
- Wanyi Liu
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, 08854, USA.
| | - David S Vicario
- Department of Psychology, Rutgers, The State University of New Jersey, Piscataway, NJ, 08854, USA.
| |
Collapse
|
35
|
Cho H, Fonken YM, Adamek M, Jimenez R, Lin JJ, Schalk G, Knight RT, Brunner P. Unexpected sound omissions are signaled in human posterior superior temporal gyrus: an intracranial study. Cereb Cortex 2023; 33:8837-8848. [PMID: 37280730 PMCID: PMC10350817 DOI: 10.1093/cercor/bhad155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2019] [Revised: 04/11/2023] [Accepted: 04/11/2023] [Indexed: 06/08/2023] Open
Abstract
Context modulates sensory neural activations enhancing perceptual and behavioral performance and reducing prediction errors. However, the mechanism of when and where these high-level expectations act on sensory processing is unclear. Here, we isolate the effect of expectation absent of any auditory evoked activity by assessing the response to omitted expected sounds. Electrocorticographic signals were recorded directly from subdural electrode grids placed over the superior temporal gyrus (STG). Subjects listened to a predictable sequence of syllables, with some infrequently omitted. We found high-frequency band activity (HFA, 70-170 Hz) in response to omissions, which overlapped with a posterior subset of auditory-active electrodes in STG. Heard syllables could be distinguishable reliably from STG, but not the identity of the omitted stimulus. Both omission- and target-detection responses were also observed in the prefrontal cortex. We propose that the posterior STG is central for implementing predictions in the auditory environment. HFA omission responses in this region appear to index mismatch-signaling or salience detection processes.
Collapse
Affiliation(s)
- Hohyun Cho
- Department of Neurosurgery, Washington University School of Medicine in Saint Louis, St. Louis, MO 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO 63110, USA
| | - Yvonne M Fonken
- Department of Psychology and the Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
- TNO Human Factors Research Institute, Soesterberg 3769 DE, Netherlands
| | - Markus Adamek
- Department of Neurosurgery, Washington University School of Medicine in Saint Louis, St. Louis, MO 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO 63110, USA
| | - Richard Jimenez
- Department of Psychology and the Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Jack J Lin
- Department of Neurology and Center for Mind and Brain, University of California, Davis, Davis, CA 95618, USA
| | - Gerwin Schalk
- Frontier Lab for Applied Neurotechnology, Tianqiao and Chrissy Chen Institute, Shanghai 201203, People’s Republic of China
- Department of Neurosurgery, Fudan University/Huashan Hospital, Shanghai 200031, People’s Republic of China
| | - Robert T Knight
- Department of Psychology and the Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Peter Brunner
- Department of Neurosurgery, Washington University School of Medicine in Saint Louis, St. Louis, MO 63110, USA
- National Center for Adaptive Neurotechnologies, St. Louis, MO 63110, USA
- Department of Neurology, Albany Medical College, Albany, NY 12208, USA
| |
Collapse
|
36
|
Rubin JJ, Kawahara AY. A framework for understanding post-detection deception in predator-prey interactions. PeerJ 2023; 11:e15389. [PMID: 37377786 PMCID: PMC10292197 DOI: 10.7717/peerj.15389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2022] [Accepted: 04/19/2023] [Indexed: 06/29/2023] Open
Abstract
Predators and prey exist in persistent conflict that often hinges on deception-the transmission of misleading or manipulative signals-as a means for survival. Deceptive traits are widespread across taxa and sensory systems, representing an evolutionarily successful and common strategy. Moreover, the highly conserved nature of the major sensory systems often extends these traits past single species predator-prey interactions toward a broader set of perceivers. As such, deceptive traits can provide a unique window into the capabilities, constraints and commonalities across divergent and phylogenetically-related perceivers. Researchers have studied deceptive traits for centuries, but a unified framework for categorizing different types of post-detection deception in predator-prey conflict still holds potential to inform future research. We suggest that deceptive traits can be distinguished by their effect on object formation processes. Perceptual objects are composed of physical attributes (what) and spatial (where) information. Deceptive traits that operate after object formation can therefore influence the perception and processing of either or both of these axes. We build upon previous work using a perceiver perspective approach to delineate deceptive traits by whether they closely match the sensory information of another object or create a discrepancy between perception and reality by exploiting the sensory shortcuts and perceptual biases of their perceiver. We then further divide this second category, sensory illusions, into traits that distort object characteristics along either the what or where axes, and those that create the perception of whole novel objects, integrating the what/where axes. Using predator-prey examples, we detail each step in this framework and propose future avenues for research. We suggest that this framework will help organize the many forms of deceptive traits and help generate predictions about selective forces that have driven animal form and behavior across evolutionary time.
Collapse
Affiliation(s)
- Juliette J. Rubin
- McGuire Center for Lepidoptera and Biodiversity, Florida Museum of Natural History, University of Florida, Gainesville, FL, USA
- Department of Biology, University of Florida, Gainesville, FL, USA
| | - Akito Y. Kawahara
- McGuire Center for Lepidoptera and Biodiversity, Florida Museum of Natural History, University of Florida, Gainesville, FL, USA
- Department of Biology, University of Florida, Gainesville, FL, USA
| |
Collapse
|
37
|
Lestang JH, Cai H, Averbeck BB, Cohen YE. Functional network properties of the auditory cortex. Hear Res 2023; 433:108768. [PMID: 37075536 PMCID: PMC10205700 DOI: 10.1016/j.heares.2023.108768] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/28/2022] [Revised: 03/27/2023] [Accepted: 04/11/2023] [Indexed: 04/21/2023]
Abstract
The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.
Collapse
Affiliation(s)
- Jean-Hugues Lestang
- Departments of Otorhinolaryngology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Huaizhen Cai
- Departments of Otorhinolaryngology, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Bruno B Averbeck
- Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, USA.
| | - Yale E Cohen
- Departments of Otorhinolaryngology, University of Pennsylvania, Philadelphia, PA 19104, USA; Neuroscience, University of Pennsylvania, Philadelphia, PA 19104, USA; Bioengineering, University of Pennsylvania, Philadelphia, PA 19104, USA
| |
Collapse
|
38
|
Wang ZQ, Wen HZ, Luo TT, Chen PH, Zhao YD, Wu GY, Xiong Y. Corticostriatal Neurons in the Anterior Auditory Field Regulate Frequency Discrimination Behavior. Neurosci Bull 2023; 39:962-972. [PMID: 36629979 PMCID: PMC10264320 DOI: 10.1007/s12264-022-01015-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2022] [Accepted: 09/24/2022] [Indexed: 01/12/2023] Open
Abstract
The anterior auditory field (AAF) is a core region of the auditory cortex and plays a vital role in discrimination tasks. However, the role of the AAF corticostriatal neurons in frequency discrimination remains unclear. Here, we used c-Fos staining, fiber photometry recording, and pharmacogenetic manipulation to investigate the function of the AAF corticostriatal neurons in a frequency discrimination task. c-Fos staining and fiber photometry recording revealed that the activity of AAF pyramidal neurons was significantly elevated during the frequency discrimination task. Pharmacogenetic inhibition of AAF pyramidal neurons significantly impaired frequency discrimination. In addition, histological results revealed that AAF pyramidal neurons send strong projections to the striatum. Moreover, pharmacogenetic suppression of the striatal projections from pyramidal neurons in the AAF significantly disrupted the frequency discrimination. Collectively, our findings show that AAF pyramidal neurons, particularly the AAF-striatum projections, play a crucial role in frequency discrimination behavior.
Collapse
Affiliation(s)
- Zhao-Qun Wang
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China
| | - Hui-Zhong Wen
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China
| | - Tian-Tian Luo
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China
| | - Peng-Hui Chen
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China
| | - Yan-Dong Zhao
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China
| | - Guang-Yan Wu
- Experimental Center of Basic Medicine, Army Medical University, Chongqing, 400038, China.
| | - Ying Xiong
- Department of Neurobiology, Chongqing Key Laboratory of Neurobiology, Army Medical University, Chongqing, 400038, China.
| |
Collapse
|
39
|
Cheng J, Li J, Wang A, Zhang M. Semantic Bimodal Presentation Differentially Slows Working Memory Retrieval. Brain Sci 2023; 13:brainsci13050811. [PMID: 37239283 DOI: 10.3390/brainsci13050811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2023] [Revised: 05/14/2023] [Accepted: 05/16/2023] [Indexed: 05/28/2023] Open
Abstract
Although evidence has shown that working memory (WM) can be differentially affected by the multisensory congruency of different visual and auditory stimuli, it remains unclear whether different multisensory congruency about concrete and abstract words could impact further WM retrieval. By manipulating the attention focus toward different matching conditions of visual and auditory word characteristics in a 2-back paradigm, the present study revealed that for the characteristically incongruent condition under the auditory retrieval condition, the response to abstract words was faster than that to concrete words, indicating that auditory abstract words are not affected by visual representation, while auditory concrete words are. Alternatively, for concrete words under the visual retrieval condition, WM retrieval was faster in the characteristically incongruent condition than in the characteristically congruent condition, indicating that visual representation formed by auditory concrete words may interfere with WM retrieval of visual concrete words. The present findings demonstrated that concrete words in multisensory conditions may be too aggressively encoded with other visual representations, which would inadvertently slow WM retrieval. However, abstract words seem to suppress interference better, showing better WM performance than concrete words in the multisensory condition.
Collapse
Affiliation(s)
- Jia Cheng
- Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215123, China
| | - Jingjing Li
- Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215123, China
| | - Aijun Wang
- Department of Psychology, Research Center for Psychology and Behavioral Sciences, Soochow University, Suzhou 215123, China
| | - Ming Zhang
- Department of Psychology, Suzhou University of Science and Technology, Suzhou 215009, China
- Faculty of Interdisciplinary Science and Engineering in Health Systems, Okayama University, Okayama 700-0082, Japan
| |
Collapse
|
40
|
Gurariy G, Randall R, Greenberg AS. Neuroimaging evidence for the direct role of auditory scene analysis in object perception. Cereb Cortex 2023; 33:6257-6272. [PMID: 36562994 PMCID: PMC10183742 DOI: 10.1093/cercor/bhac501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 11/29/2022] [Accepted: 11/30/2022] [Indexed: 12/24/2022] Open
Abstract
Auditory Scene Analysis (ASA) refers to the grouping of acoustic signals into auditory objects. Previously, we have shown that perceived musicality of auditory sequences varies with high-level organizational features. Here, we explore the neural mechanisms mediating ASA and auditory object perception. Participants performed musicality judgments on randomly generated pure-tone sequences and manipulated versions of each sequence containing low-level changes (amplitude; timbre). Low-level manipulations affected auditory object perception as evidenced by changes in musicality ratings. fMRI was used to measure neural activation to sequences rated most and least musical, and the altered versions of each sequence. Next, we generated two partially overlapping networks: (i) a music processing network (music localizer) and (ii) an ASA network (base sequences vs. ASA manipulated sequences). Using Representational Similarity Analysis, we correlated the functional profiles of each ROI to a model generated from behavioral musicality ratings as well as models corresponding to low-level feature processing and music perception. Within overlapping regions, areas near primary auditory cortex correlated with low-level ASA models, whereas right IPS was correlated with musicality ratings. Shared neural mechanisms that correlate with behavior and underlie both ASA and music perception suggests that low-level features of auditory stimuli play a role in auditory object perception.
Collapse
Affiliation(s)
- Gennadiy Gurariy
- Department of Biomedical Engineering, Medical College of Wisconsin and Marquette University, 8701 W Watertown Plank Rd, Milwaukee, WI 53233, United States
| | - Richard Randall
- School of Music and Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, United States
| | - Adam S Greenberg
- Department of Biomedical Engineering, Medical College of Wisconsin and Marquette University, 8701 W Watertown Plank Rd, Milwaukee, WI 53233, United States
| |
Collapse
|
41
|
DiTullio RW, Parthiban C, Piasini E, Chaudhari P, Balasubramanian V, Cohen YE. Time as a supervisor: temporal regularity and auditory object learning. Front Comput Neurosci 2023; 17:1150300. [PMID: 37216064 PMCID: PMC10192587 DOI: 10.3389/fncom.2023.1150300] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Accepted: 03/30/2023] [Indexed: 05/24/2023] Open
Abstract
Sensory systems appear to learn to transform incoming sensory information into perceptual representations, or "objects," that can inform and guide behavior with minimal explicit supervision. Here, we propose that the auditory system can achieve this goal by using time as a supervisor, i.e., by learning features of a stimulus that are temporally regular. We will show that this procedure generates a feature space sufficient to support fundamental computations of auditory perception. In detail, we consider the problem of discriminating between instances of a prototypical class of natural auditory objects, i.e., rhesus macaque vocalizations. We test discrimination in two ethologically relevant tasks: discrimination in a cluttered acoustic background and generalization to discriminate between novel exemplars. We show that an algorithm that learns these temporally regular features affords better or equivalent discrimination and generalization than conventional feature-selection algorithms, i.e., principal component analysis and independent component analysis. Our findings suggest that the slow temporal features of auditory stimuli may be sufficient for parsing auditory scenes and that the auditory brain could utilize these slowly changing temporal features.
Collapse
Affiliation(s)
- Ronald W. DiTullio
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, United States
- Neuroscience Graduate Group, University of Pennsylvania, Philadelphia, PA, United States
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, United States
| | - Chetan Parthiban
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, United States
| | - Eugenio Piasini
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, United States
- Scuola Internazionale Superiore di Studi Avanzati (SISSA), Trieste, Italy
| | - Pratik Chaudhari
- Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, United States
| | - Vijay Balasubramanian
- David Rittenhouse Laboratory, Department of Physics and Astronomy, University of Pennsylvania, Philadelphia, PA, United States
- Computational Neuroscience Initiative, University of Pennsylvania, Philadelphia, PA, United States
- Santa Fe Institute, Santa Fe, NM, United States
| | - Yale E. Cohen
- Departments of Otorhinolaryngology, Neuroscience, and Bioengineering, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
42
|
Ngo HVV, Oster H, Andreou C, Obleser J. Circadian rhythms in auditory hallucinations and psychosis. Acta Physiol (Oxf) 2023; 237:e13944. [PMID: 36744985 DOI: 10.1111/apha.13944] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 01/31/2023] [Accepted: 02/01/2023] [Indexed: 02/07/2023]
Abstract
Circadian rhythms are imprinted in all organisms and influence virtually all aspects of physiology and behavior in adaptation to the 24-h day-night cycle. This recognition of a circadian timekeeping system permeating essentially all healthy functioning of body and mind quickly leads to the realization that, in turn, human ailments should be probed for the degree to which they are rooted in or marked by disruptions and dysregulations of circadian clock functions in the human body. In this review, we will focus on psychosis as a key mental illness and foremost one of its cardinal symptoms: auditory hallucinations. We will discuss recent empirical evidence and conceptual advances probing the potential role of circadian disruption in auditory hallucinations. Moreover, a dysbalance in excitation and inhibition within cortical networks, which in turn drive a disinhibition of dopaminergic signaling, will be highlighted as central physiological mechanism. Finally, we will propose two avenues for experimentally intervening on the circadian influences to potentially alleviate hallucinations in psychotic disorders.
Collapse
Affiliation(s)
- Hong-Viet V Ngo
- Department of Psychology, University of Lübeck, Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Lübeck, Germany
| | - Henrik Oster
- Center of Brain, Behavior and Metabolism, University of Lübeck, Lübeck, Germany
- Institute of Neurobiology, University of Lübeck, Lübeck, Germany
| | - Christina Andreou
- Center of Brain, Behavior and Metabolism, University of Lübeck, Lübeck, Germany
- Department of Psychiatry and Psychotherapy, University of Lübeck, Lübeck, Germany
| | - Jonas Obleser
- Department of Psychology, University of Lübeck, Lübeck, Germany
- Center of Brain, Behavior and Metabolism, University of Lübeck, Lübeck, Germany
| |
Collapse
|
43
|
Ahveninen J, Uluç I, Raij T, Nummenmaa A, Mamashli F. Spectrotemporal content of human auditory working memory represented in functional connectivity patterns. Commun Biol 2023; 6:294. [PMID: 36941477 PMCID: PMC10027691 DOI: 10.1038/s42003-023-04675-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 03/07/2023] [Indexed: 03/23/2023] Open
Abstract
Recent research suggests that working memory (WM), the mental sketchpad underlying thinking and communication, is maintained by multiple regions throughout the brain. Whether parts of a stable WM representation could be distributed across these brain regions is, however, an open question. We addressed this question by examining the content-specificity of connectivity-pattern matrices between subparts of cortical regions-of-interest (ROI). These connectivity patterns were calculated from functional MRI obtained during a ripple-sound auditory WM task. Statistical significance was assessed by comparing the decoding results to a null distribution derived from a permutation test considering all comparable two- to four-ROI connectivity patterns. Maintained WM items could be decoded from connectivity patterns across ROIs in frontal, parietal, and superior temporal cortices. All functional connectivity patterns that were specific to maintained sound content extended from early auditory to frontoparietal cortices. Our results demonstrate that WM maintenance is supported by content-specific patterns of functional connectivity across different levels of cortical hierarchy.
Collapse
Affiliation(s)
- Jyrki Ahveninen
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA.
- Department of Radiology, Harvard Medical School, Boston, MA, USA.
| | - Işıl Uluç
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Tommi Raij
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Aapo Nummenmaa
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| | - Fahimeh Mamashli
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
- Department of Radiology, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
44
|
Giordano BL, Esposito M, Valente G, Formisano E. Intermediate acoustic-to-semantic representations link behavioral and neural responses to natural sounds. Nat Neurosci 2023; 26:664-672. [PMID: 36928634 PMCID: PMC10076214 DOI: 10.1038/s41593-023-01285-9] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2022] [Accepted: 02/15/2023] [Indexed: 03/18/2023]
Abstract
Recognizing sounds implicates the cerebral transformation of input waveforms into semantic representations. Although past research identified the superior temporal gyrus (STG) as a crucial cortical region, the computational fingerprint of these cerebral transformations remains poorly characterized. Here, we exploit a model comparison framework and contrasted the ability of acoustic, semantic (continuous and categorical) and sound-to-event deep neural network representation models to predict perceived sound dissimilarity and 7 T human auditory cortex functional magnetic resonance imaging responses. We confirm that spectrotemporal modulations predict early auditory cortex (Heschl's gyrus) responses, and that auditory dimensions (for example, loudness, periodicity) predict STG responses and perceived dissimilarity. Sound-to-event deep neural networks predict Heschl's gyrus responses similar to acoustic models but, notably, they outperform all competing models at predicting both STG responses and perceived dissimilarity. Our findings indicate that STG entails intermediate acoustic-to-semantic sound representations that neither acoustic nor semantic models can account for. These representations are compositional in nature and relevant to behavior.
Collapse
Affiliation(s)
- Bruno L Giordano
- Institut de Neurosciences de La Timone, UMR 7289, CNRS and Université Aix-Marseille, Marseille, France.
| | - Michele Esposito
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, the Netherlands
| | - Giancarlo Valente
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, the Netherlands
| | - Elia Formisano
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, the Netherlands. .,Maastricht Centre for Systems Biology (MaCSBio), Faculty of Science and Engineering, Maastricht University, Maastricht, the Netherlands. .,Brightlands Institute for Smart Society (BISS), Maastricht University, Maastricht, the Netherlands.
| |
Collapse
|
45
|
Robotka H, Thomas L, Yu K, Wood W, Elie JE, Gahr M, Theunissen FE. Sparse ensemble neural code for a complete vocal repertoire. Cell Rep 2023; 42:112034. [PMID: 36696266 PMCID: PMC10363576 DOI: 10.1016/j.celrep.2023.112034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 08/08/2022] [Accepted: 01/09/2023] [Indexed: 01/26/2023] Open
Abstract
The categorization of animal vocalizations into distinct behaviorally relevant groups for communication is an essential operation that must be performed by the auditory system. This auditory object recognition is a difficult task that requires selectivity to the group identifying acoustic features and invariance to renditions within each group. We find that small ensembles of auditory neurons in the forebrain of a social songbird can code the bird's entire vocal repertoire (∼10 call types). Ensemble neural discrimination is not, however, correlated with single unit selectivity, but instead with how well the joint single unit tunings to characteristic spectro-temporal modulations span the acoustic subspace optimized for the discrimination of call types. Thus, akin to face recognition in the visual system, call type recognition in the auditory system is based on a sparse code representing a small number of high-level features and not on highly selective grandmother neurons.
Collapse
Affiliation(s)
- H Robotka
- Max Planck Institute for Ornithology, Seewiesen, Germany
| | - L Thomas
- University of California, Berkeley, Helen Wills Neuroscience Institute, Berkeley, CA, USA
| | - K Yu
- University of California, Berkeley, Helen Wills Neuroscience Institute, Berkeley, CA, USA
| | - W Wood
- University of California, Berkeley, Helen Wills Neuroscience Institute, Berkeley, CA, USA
| | - J E Elie
- University of California, Berkeley, Helen Wills Neuroscience Institute, Berkeley, CA, USA
| | - M Gahr
- Max Planck Institute for Ornithology, Seewiesen, Germany
| | - F E Theunissen
- Max Planck Institute for Ornithology, Seewiesen, Germany; University of California, Berkeley, Helen Wills Neuroscience Institute, Berkeley, CA, USA; Department of Psychology and Integrative Biology, University of California, Berkeley, Berkeley, CA, USA.
| |
Collapse
|
46
|
Reversible Inactivation of Ferret Auditory Cortex Impairs Spatial and Nonspatial Hearing. J Neurosci 2023; 43:749-763. [PMID: 36604168 PMCID: PMC9899081 DOI: 10.1523/jneurosci.1426-22.2022] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Revised: 11/16/2022] [Accepted: 11/29/2022] [Indexed: 01/06/2023] Open
Abstract
A key question in auditory neuroscience is to what extent are brain regions functionally specialized for processing specific sound features, such as location and identity. In auditory cortex, correlations between neural activity and sounds support both the specialization of distinct cortical subfields, and encoding of multiple sound features within individual cortical areas. However, few studies have tested the contribution of auditory cortex to hearing in multiple contexts. Here we determined the role of ferret primary auditory cortex in both spatial and nonspatial hearing by reversibly inactivating the middle ectosylvian gyrus during behavior using cooling (n = 2 females) or optogenetics (n = 1 female). Optogenetic experiments used the mDLx promoter to express Channelrhodopsin-2 in GABAergic interneurons, and we confirmed both viral expression (n = 2 females) and light-driven suppression of spiking activity in auditory cortex, recorded using Neuropixels under anesthesia (n = 465 units from 2 additional untrained female ferrets). Cortical inactivation via cooling or optogenetics impaired vowel discrimination in colocated noise. Ferrets implanted with cooling loops were tested in additional conditions that revealed no deficit when identifying vowels in clean conditions, or when the temporally coincident vowel and noise were spatially separated by 180 degrees. These animals did, however, show impaired sound localization when inactivating the same auditory cortical region implicated in vowel discrimination in noise. Our results demonstrate that, as a brain region showing mixed selectivity for spatial and nonspatial features of sound, primary auditory cortex contributes to multiple forms of hearing.SIGNIFICANCE STATEMENT Neurons in primary auditory cortex are often sensitive to the location and identity of sounds. Here we inactivated auditory cortex during spatial and nonspatial listening tasks using cooling, or optogenetics. Auditory cortical inactivation impaired multiple behaviors, demonstrating a role in both the analysis of sound location and identity and confirming a functional contribution of mixed selectivity observed in neural activity. Parallel optogenetic experiments in two additional untrained ferrets linked behavior to physiology by demonstrating that expression of Channelrhodopsin-2 permitted rapid light-driven suppression of auditory cortical activity recorded under anesthesia.
Collapse
|
47
|
Alho J, Khan S, Mamashli F, Perrachione TK, Losh A, McGuiggan NM, Graham S, Nayal Z, Joseph RM, Hämäläinen MS, Bharadwaj H, Kenet T. Atypical cortical processing of bottom-up speech binding cues in children with autism spectrum disorders. Neuroimage Clin 2023; 37:103336. [PMID: 36724734 PMCID: PMC9898310 DOI: 10.1016/j.nicl.2023.103336] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Revised: 01/10/2023] [Accepted: 01/20/2023] [Indexed: 01/23/2023]
Abstract
Individuals with autism spectrum disorder (ASD) commonly display speech processing abnormalities. Binding of acoustic features of speech distributed across different frequencies into coherent speech objects is fundamental in speech perception. Here, we tested the hypothesis that the cortical processing of bottom-up acoustic cues for speech binding may be anomalous in ASD. We recorded magnetoencephalography while ASD children (ages 7-17) and typically developing peers heard sentences of sine-wave speech (SWS) and modulated SWS (MSS) where binding cues were restored through increased temporal coherence of the acoustic components and the introduction of harmonicity. The ASD group showed increased long-range feedforward functional connectivity from left auditory to parietal cortex with concurrent decreased local functional connectivity within the parietal region during MSS relative to SWS. As the parietal region has been implicated in auditory object binding, our findings support our hypothesis of atypical bottom-up speech binding in ASD. Furthermore, the long-range functional connectivity correlated with behaviorally measured auditory processing abnormalities, confirming the relevance of these atypical cortical signatures to the ASD phenotype. Lastly, the group difference in the local functional connectivity was driven by the youngest participants, suggesting that impaired speech binding in ASD might be ameliorated upon entering adolescence.
Collapse
Affiliation(s)
- Jussi Alho
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA.
| | - Sheraz Khan
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Fahimeh Mamashli
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Tyler K Perrachione
- Department of Speech, Language, and Hearing Sciences, Boston University, 635 Commonwealth Ave, Boston, MA 02215, USA
| | - Ainsley Losh
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Nicole M McGuiggan
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Steven Graham
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Zein Nayal
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Robert M Joseph
- Department of Anatomy and Neurobiology, Boston University School of Medicine, 72 East Concord St, Boston, MA 02118, USA
| | - Matti S Hämäläinen
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA
| | - Hari Bharadwaj
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Department of Speech, Language, and Hearing Sciences, and Weldon School of Biomedical Engineering, Purdue University, 715 Clinic Drive, West Lafayette, IN 47907, USA
| | - Tal Kenet
- Department of Neurology, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, 149 13th St, Boston, MA 02129, USA.
| |
Collapse
|
48
|
Yao JD, Zemlianova KO, Hocker DL, Savin C, Constantinople CM, Chung S, Sanes DH. Transformation of acoustic information to sensory decision variables in the parietal cortex. Proc Natl Acad Sci U S A 2023; 120:e2212120120. [PMID: 36598952 PMCID: PMC9926273 DOI: 10.1073/pnas.2212120120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Accepted: 11/08/2022] [Indexed: 01/05/2023] Open
Abstract
The process by which sensory evidence contributes to perceptual choices requires an understanding of its transformation into decision variables. Here, we address this issue by evaluating the neural representation of acoustic information in the auditory cortex-recipient parietal cortex, while gerbils either performed a two-alternative forced-choice auditory discrimination task or while they passively listened to identical acoustic stimuli. During task engagement, stimulus identity decoding performance from simultaneously recorded parietal neurons significantly correlated with psychometric sensitivity. In contrast, decoding performance during passive listening was significantly reduced. Principal component and geometric analyses revealed the emergence of low-dimensional encoding of linearly separable manifolds with respect to stimulus identity and decision, but only during task engagement. These findings confirm that the parietal cortex mediates a transition of acoustic representations into decision-related variables. Finally, using a clustering analysis, we identified three functionally distinct subpopulations of neurons that each encoded task-relevant information during separate temporal segments of a trial. Taken together, our findings demonstrate how parietal cortex neurons integrate and transform encoded auditory information to guide sound-driven perceptual decisions.
Collapse
Affiliation(s)
- Justin D. Yao
- Center for Neural Science, New York University, New YorkNY 10003
- Department of Otolaryngology, Head and Neck Surgery, Rutgers Robert Wood Johnson Medical School, New Brunswick, NJ08901
- Brain Health Institute, Rutgers University, Piscataway, NJ08854
| | | | - David L. Hocker
- Center for Neural Science, New York University, New YorkNY 10003
| | - Cristina Savin
- Center for Neural Science, New York University, New YorkNY 10003
- Neuroscience Institute, New York University Langone School of Medicine, New York, NY10016
- Center for Data Science, New York University, New YorkNY 10011
| | - Christine M. Constantinople
- Center for Neural Science, New York University, New YorkNY 10003
- Neuroscience Institute, New York University Langone School of Medicine, New York, NY10016
| | - SueYeon Chung
- Center for Neural Science, New York University, New YorkNY 10003
- Flatiron Institute, Simons Foundation, New YorkNY 10010
| | - Dan H. Sanes
- Center for Neural Science, New York University, New YorkNY 10003
- Neuroscience Institute, New York University Langone School of Medicine, New York, NY10016
- Department of Psychology, New York University, New YorkNY 10003
- Department of Biology, New York University, New YorkNY 10003
| |
Collapse
|
49
|
Same, but different: Binding effects in auditory, but not visual detection performance. Atten Percept Psychophys 2023; 85:438-451. [PMID: 35107812 PMCID: PMC9935720 DOI: 10.3758/s13414-021-02436-5] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/27/2021] [Indexed: 11/08/2022]
Abstract
Responding to a stimulus leads to the integration of response and stimulus' features into an event file. Upon repetition of any of its features, the previous event file is retrieved, thereby affecting ongoing performance. Such integration-retrieval explanations exist for a number of sequential tasks (that measure these processes as 'binding effects') and are thought to underlie all actions. However, based on attentional orienting literature, Schöpper, Hilchey, et al. (2020) could show that binding effects are absent when participants detect visual targets in a sequence: In visual detection performance, there is simply a benefit for target location changes (inhibition of return). In contrast, Mondor and Leboe (2008) had participants detect auditory targets in a sequence, and found a benefit for frequency repetition - presumably reflecting a binding effect in auditory detection performance. In the current study, we conducted two experiments, that only differed in the modality of the target: Participants signaled the detection of a sound (N = 40) or of a visual target (N = 40). Whereas visual detection performance showed a pattern incongruent with binding assumptions, auditory detection performance revealed a non-spatial feature repetition benefit, suggesting that frequency was bound to the response. Cumulative reaction time distributions indicated that the absence of a binding effect in visual detection performance was not caused by overall faster responding. The current results show a clear limitation to binding accounts in action control: Binding effects are not only limited by task demands, but can entirely depend on target modality.
Collapse
|
50
|
Shvadron S, Snir A, Maimon A, Yizhar O, Harel S, Poradosu K, Amedi A. Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device. Front Hum Neurosci 2023; 17:1058617. [PMID: 36936618 PMCID: PMC10017858 DOI: 10.3389/fnhum.2023.1058617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Accepted: 01/09/2023] [Indexed: 03/06/2023] Open
Abstract
Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes' identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception.
Collapse
Affiliation(s)
- Shira Shvadron
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
- *Correspondence: Shira Shvadron,
| | - Adi Snir
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
| | - Amber Maimon
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
| | - Or Yizhar
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
- Research Group Adaptive Memory and Decision Making, Max Planck Institute for Human Development, Berlin, Germany
- Max Planck Dahlem Campus of Cognition (MPDCC), Max Planck Institute for Human Development, Berlin, Germany
| | - Sapir Harel
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
| | - Keinan Poradosu
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
- Weizmann Institute of Science, Rehovot, Israel
| | - Amir Amedi
- Baruch Ivcher School of Psychology, The Baruch Ivcher Institute for Brain, Cognition, and Technology, Reichman University, Herzliya, Israel
- The Ruth and Meir Rosenthal, Brain Imaging Center, Reichman University, Herzliya, Israel
| |
Collapse
|