1
|
Oganian Y, Bhaya-Grossman I, Johnson K, Chang EF. Vowel and formant representation in the human auditory speech cortex. Neuron 2023; 111:2105-2118.e4. [PMID: 37105171 PMCID: PMC10330593 DOI: 10.1016/j.neuron.2023.04.004] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 02/08/2023] [Accepted: 04/04/2023] [Indexed: 04/29/2023]
Abstract
Vowels, a fundamental component of human speech across all languages, are cued acoustically by formants, resonance frequencies of the vocal tract shape during speaking. An outstanding question in neurolinguistics is how formants are processed neurally during speech perception. To address this, we collected high-density intracranial recordings from the human speech cortex on the superior temporal gyrus (STG) while participants listened to continuous speech. We found that two-dimensional receptive fields based on the first two formants provided the best characterization of vowel sound representation. Neural activity at single sites was highly selective for zones in this formant space. Furthermore, formant tuning is adjusted dynamically for speaker-specific spectral context. However, the entire population of formant-encoding sites was required to accurately decode single vowels. Overall, our results reveal that complex acoustic tuning in the two-dimensional formant space underlies local vowel representations in STG. As a population code, this gives rise to phonological vowel perception.
Collapse
Affiliation(s)
- Yulia Oganian
- Department of Neurological Surgery, University of California, San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Ilina Bhaya-Grossman
- Department of Neurological Surgery, University of California, San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA; University of California, Berkeley-University of California, San Francisco Graduate Program in Bioengineering, Berkeley, CA 94720, USA
| | - Keith Johnson
- Department of Linguistics, University of California, Berkeley, Berkeley, CA, USA
| | - Edward F Chang
- Department of Neurological Surgery, University of California, San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA.
| |
Collapse
|
2
|
Makov S, Pinto D, Har-Shai Yahav P, Miller LM, Zion Golumbic E. "Unattended, distracting or irrelevant": Theoretical implications of terminological choices in auditory selective attention research. Cognition 2023; 231:105313. [PMID: 36344304 DOI: 10.1016/j.cognition.2022.105313] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 09/30/2022] [Accepted: 10/19/2022] [Indexed: 11/06/2022]
Abstract
For seventy years, auditory selective attention research has focused on studying the cognitive mechanisms of prioritizing the processing a 'main' task-relevant stimulus, in the presence of 'other' stimuli. However, a closer look at this body of literature reveals deep empirical inconsistencies and theoretical confusion regarding the extent to which this 'other' stimulus is processed. We argue that many key debates regarding attention arise, at least in part, from inappropriate terminological choices for experimental variables that may not accurately map onto the cognitive constructs they are meant to describe. Here we critically review the more common or disruptive terminological ambiguities, differentiate between methodology-based and theory-derived terms, and unpack the theoretical assumptions underlying different terminological choices. Particularly, we offer an in-depth analysis of the terms 'unattended' and 'distractor' and demonstrate how their use can lead to conflicting theoretical inferences. We also offer a framework for thinking about terminology in a more productive and precise way, in hope of fostering more productive debates and promoting more nuanced and accurate cognitive models of selective attention.
Collapse
Affiliation(s)
- Shiri Makov
- The Gonda Multidisciplinary Center for Brain Research, Bar Ilan University, Israel
| | - Danna Pinto
- The Gonda Multidisciplinary Center for Brain Research, Bar Ilan University, Israel
| | - Paz Har-Shai Yahav
- The Gonda Multidisciplinary Center for Brain Research, Bar Ilan University, Israel
| | - Lee M Miller
- The Center for Mind and Brain, University of California, Davis, CA, United States of America; Department of Neurobiology, Physiology, & Behavior, University of California, Davis, CA, United States of America; Department of Otolaryngology / Head and Neck Surgery, University of California, Davis, CA, United States of America
| | - Elana Zion Golumbic
- The Gonda Multidisciplinary Center for Brain Research, Bar Ilan University, Israel.
| |
Collapse
|
3
|
Di Dona G, Scaltritti M, Sulpizio S. Formant-invariant voice and pitch representations are pre-attentively formed from constantly varying speech and non-speech stimuli. Eur J Neurosci 2022; 56:4086-4106. [PMID: 35673798 PMCID: PMC9545905 DOI: 10.1111/ejn.15730] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 05/23/2022] [Accepted: 05/24/2022] [Indexed: 11/30/2022]
Abstract
The present study investigated whether listeners can form abstract voice representations while ignoring constantly changing phonological information and if they can use the resulting information to facilitate voice change detection. Further, the study aimed at understanding whether the use of abstraction is restricted to the speech domain or can be deployed also in non‐speech contexts. We ran an electroencephalogram (EEG) experiment including one passive and one active oddball task, each featuring a speech and a rotated speech condition. In the speech condition, participants heard constantly changing vowels uttered by a male speaker (standard stimuli) which were infrequently replaced by vowels uttered by a female speaker with higher pitch (deviant stimuli). In the rotated speech condition, participants heard rotated vowels, in which the natural formant structure of speech was disrupted. In the passive task, the mismatch negativity was elicited after the presentation of the deviant voice in both conditions, indicating that listeners could successfully group together different stimuli into a formant‐invariant voice representation. In the active task, participants showed shorter reaction times (RTs), higher accuracy and a larger P3b in the speech condition with respect to the rotated speech condition. Results showed that whereas at a pre‐attentive level the cognitive system can track pitch regularities while presumably ignoring constantly changing formant information both in speech and in rotated speech, at an attentive level the use of such information is facilitated for speech. This facilitation was also testified by a stronger synchronisation in the theta band (4–7 Hz), potentially pointing towards differences in encoding/retrieval processes.
Collapse
Affiliation(s)
- Giuseppe Di Dona
- Dipartimento di Psicologia e Scienze Cognitive, Università degli Studi di Trento, Trento, Italy
| | - Michele Scaltritti
- Dipartimento di Psicologia e Scienze Cognitive, Università degli Studi di Trento, Trento, Italy
| | - Simone Sulpizio
- Dipartimento di Psicologia, Università degli Studi di Milano-Bicocca, Milano, Italy.,Milan Center for Neuroscience (NeuroMi), Università degli Studi di Milano-Bicocca, Milano, Italy
| |
Collapse
|
4
|
Romanovska L, Bonte M. How Learning to Read Changes the Listening Brain. Front Psychol 2021; 12:726882. [PMID: 34987442 PMCID: PMC8721231 DOI: 10.3389/fpsyg.2021.726882] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2021] [Accepted: 11/23/2021] [Indexed: 01/18/2023] Open
Abstract
Reading acquisition reorganizes existing brain networks for speech and visual processing to form novel audio-visual language representations. This requires substantial cortical plasticity that is reflected in changes in brain activation and functional as well as structural connectivity between brain areas. The extent to which a child's brain can accommodate these changes may underlie the high variability in reading outcome in both typical and dyslexic readers. In this review, we focus on reading-induced functional changes of the dorsal speech network in particular and discuss how its reciprocal interactions with the ventral reading network contributes to reading outcome. We discuss how the dynamic and intertwined development of both reading networks may be best captured by approaching reading from a skill learning perspective, using audio-visual learning paradigms and longitudinal designs to follow neuro-behavioral changes while children's reading skills unfold.
Collapse
Affiliation(s)
| | - Milene Bonte
- *Correspondence: Linda Romanovska, ; Milene Bonte,
| |
Collapse
|
5
|
Di Dona G, Scaltritti M, Sulpizio S. Early differentiation of memory retrieval processes for newly learned voices and phonemes as indexed by the MMN. BRAIN AND LANGUAGE 2021; 220:104981. [PMID: 34166941 DOI: 10.1016/j.bandl.2021.104981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/28/2020] [Revised: 05/31/2021] [Accepted: 06/01/2021] [Indexed: 06/13/2023]
Abstract
Linguistic and vocal information are thought to be differentially processed since the early stages of speech perception, but it remains unclear if this differentiation also concerns automatic processes of memory retrieval. The aim of this ERP study was to compare the automatic retrieval processes for newly learned voices vs phonemes. In a longitudinal experiment, two groups of participants were trained in learning either a new phoneme or a new voice. The MMN elicited by the presentation of the two was measured before and after the training. An enhanced MMN was elicited by the presentation of the learned phoneme, reflecting the activation of an automatic memory retrieval process. Instead, a reduced MMN was elicited by the learned voice, indicating that the voice was perceived as a typical member of the learned voice identity. This suggests that the automatic processes that retrieve linguistic and vocal information are differently affected by experience.
Collapse
Affiliation(s)
- Giuseppe Di Dona
- Dipartimento di Psicologia e Scienze Cognitive, Università degli Studi di Trento, Corso Bettini 84, 38068 Rovereto (TN), Italy.
| | - Michele Scaltritti
- Dipartimento di Psicologia e Scienze Cognitive, Università degli Studi di Trento, Corso Bettini 84, 38068 Rovereto (TN), Italy.
| | - Simone Sulpizio
- Dipartimento di Psicologia, Università degli Studi di Milano-Bicocca, Piazza dell'Ateneo Nuovo 1, 20126 Milano (MI), Italy; Milan Center for Neuroscience (NeuroMi), Università degli Studi di Milano-Bicocca, Piazza dell'Ateneo Nuovo 1, 20126 Milano (MI), Italy.
| |
Collapse
|
6
|
Wikman P, Sahari E, Salmela V, Leminen A, Leminen M, Laine M, Alho K. Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing. Neuroimage 2020; 224:117365. [PMID: 32941985 DOI: 10.1016/j.neuroimage.2020.117365] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2020] [Revised: 08/19/2020] [Accepted: 09/07/2020] [Indexed: 12/20/2022] Open
Abstract
Recent studies utilizing electrophysiological speech envelope reconstruction have sparked renewed interest in the cocktail party effect by showing that auditory neurons entrain to selectively attended speech. Yet, the neural networks of attention to speech in naturalistic audiovisual settings with multiple sound sources remain poorly understood. We collected functional brain imaging data while participants viewed audiovisual video clips of lifelike dialogues with concurrent distracting speech in the background. Dialogues were presented in a full-factorial design, comprising task (listen to the dialogues vs. ignore them), audiovisual quality and semantic predictability. We used univariate analyses in combination with multivariate pattern analysis (MVPA) to study modulations of brain activity related to attentive processing of audiovisual speech. We found attentive speech processing to cause distinct spatiotemporal modulation profiles in distributed cortical areas including sensory and frontal-control networks. Semantic coherence modulated attention-related activation patterns in the earliest stages of auditory cortical processing, suggesting that the auditory cortex is involved in high-level speech processing. Our results corroborate views that emphasize the dynamic nature of attention, with task-specificity and context as cornerstones of the underlying neuro-cognitive mechanisms.
Collapse
Affiliation(s)
- Patrik Wikman
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland.
| | - Elisa Sahari
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland
| | - Viljami Salmela
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland; Advanced Magnetic Imaging Centre, Aalto NeuroImaging, Aalto University, Espoo, Finland
| | - Alina Leminen
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland; Department of Digital Humanities, University of Helsinki, Helsinki, Finland
| | - Miika Leminen
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland; Department of Phoniatrics, Helsinki University Hospital, Helsinki, Finland
| | - Matti Laine
- Department of Psychology, Åbo Akademi University, Turku, Finland
| | - Kimmo Alho
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland; Advanced Magnetic Imaging Centre, Aalto NeuroImaging, Aalto University, Espoo, Finland
| |
Collapse
|
7
|
Notter MP, Hanke M, Murray MM, Geiser E. Encoding of Auditory Temporal Gestalt in the Human Brain. Cereb Cortex 2020; 29:475-484. [PMID: 29365070 DOI: 10.1093/cercor/bhx328] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2017] [Indexed: 12/16/2022] Open
Abstract
The perception of an acoustic rhythm is invariant to the absolute temporal intervals constituting a sound sequence. It is unknown where in the brain temporal Gestalt, the percept emerging from the relative temporal proximity between acoustic events, is encoded. Two different relative temporal patterns, each induced by three experimental conditions with different absolute temporal patterns as sensory basis, were presented to participants. A linear support vector machine classifier was trained to differentiate activation patterns in functional magnetic resonance imaging data to the two different percepts. Across the sensory constituents the classifier decoded which percept was perceived. A searchlight analysis localized activation patterns specific to the temporal Gestalt bilaterally to the temporoparietal junction, including the planum temporale and supramarginal gyrus, and unilaterally to the right inferior frontal gyrus (pars opercularis). We show that auditory areas not only process absolute temporal intervals, but also integrate them into percepts of Gestalt and that encoding of these percepts persists in high-level associative areas. The findings complement existing knowledge regarding the processing of absolute temporal patterns to the processing of relative temporal patterns relevant to the sequential binding of perceptual elements into Gestalt.
Collapse
Affiliation(s)
- Michael P Notter
- Department of Radiology.,Neuropsychology and Neurorehabilitation Service.,EEG Brain Mapping Core, Center for Biomedical Imaging (CIBM), Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Michael Hanke
- Institute of Psychology, Otto-von-Guericke-University.,Center for Behavioral Brain Sciences, Magdeburg, Germany
| | - Micah M Murray
- Department of Radiology.,Neuropsychology and Neurorehabilitation Service.,EEG Brain Mapping Core, Center for Biomedical Imaging (CIBM), Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.,Ophthalmology Department, University of Lausanne and Fondation Asile des Aveugles, Lausanne, Switzerland.,Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN, USA
| | - Eveline Geiser
- Department of Radiology.,Neuropsychology and Neurorehabilitation Service.,McGovern Institute, Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
8
|
Sorati M, Behne DM. Musical Expertise Affects Audiovisual Speech Perception: Findings From Event-Related Potentials and Inter-trial Phase Coherence. Front Psychol 2019; 10:2562. [PMID: 31803107 PMCID: PMC6874039 DOI: 10.3389/fpsyg.2019.02562] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2019] [Accepted: 10/29/2019] [Indexed: 12/03/2022] Open
Abstract
In audiovisual speech perception, visual information from a talker's face during mouth articulation is available before the onset of the corresponding audio speech, and thereby allows the perceiver to use visual information to predict the upcoming audio. This prediction from phonetically congruent visual information modulates audiovisual speech perception and leads to a decrease in N1 and P2 amplitudes and latencies compared to the perception of audio speech alone. Whether audiovisual experience, such as with musical training, influences this prediction is unclear, but if so, may explain some of the variations observed in previous research. The current study addresses whether audiovisual speech perception is affected by musical training, first assessing N1 and P2 event-related potentials (ERPs) and in addition, inter-trial phase coherence (ITPC). Musicians and non-musicians are presented the syllable, /ba/ in audio only (AO), video only (VO), and audiovisual (AV) conditions. With the predictory effect of mouth movement isolated from the AV speech (AV-VO), results showed that, compared to audio speech, both groups have a lower N1 latency and P2 amplitude and latency. Moreover, they also showed lower ITPCs in the delta, theta, and beta bands in audiovisual speech perception. However, musicians showed significant suppression of N1 amplitude and desynchronization in the alpha band in audiovisual speech, not present for non-musicians. Collectively, the current findings indicate that early sensory processing can be modified by musical experience, which in turn can explain some of the variations in previous AV speech perception research.
Collapse
Affiliation(s)
- Marzieh Sorati
- Department of Psychology, Norwegian University of Science and Technology, Trondheim, Norway
| | | |
Collapse
|
9
|
What's what in auditory cortices? Neuroimage 2018; 176:29-40. [DOI: 10.1016/j.neuroimage.2018.04.028] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2018] [Revised: 04/04/2018] [Accepted: 04/12/2018] [Indexed: 11/30/2022] Open
|
10
|
Anurova I, Immonen P. Native vs. second language discrimination in the modulation of the N400/N400m. Eur J Neurosci 2017; 45:1289-1299. [DOI: 10.1111/ejn.13566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2016] [Revised: 03/14/2017] [Accepted: 03/20/2017] [Indexed: 11/29/2022]
Affiliation(s)
- Irina Anurova
- Neuroscience Unit; Department of Physiology; Faculty of Medicine; University of Helsinki; Haartmaninkatu 8 P.O. Box 63 00014 Helsinki Finland
- BioMag Laboratory; HUS Medical Imaging Center; University of Helsinki and Helsinki University Hospital; Helsinki Finland
| | - Perttu Immonen
- Neuroscience Unit; Department of Physiology; Faculty of Medicine; University of Helsinki; Haartmaninkatu 8 P.O. Box 63 00014 Helsinki Finland
| |
Collapse
|
11
|
Fraga González G, Žarić G, Tijms J, Bonte M, Blomert L, Leppänen P, van der Molen MW. Responsivity to dyslexia training indexed by the N170 amplitude of the brain potential elicited by word reading. Brain Cogn 2016; 106:42-54. [PMID: 27200495 DOI: 10.1016/j.bandc.2016.05.001] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2015] [Revised: 04/28/2016] [Accepted: 05/01/2016] [Indexed: 11/18/2022]
Abstract
The present study examined training effects in dyslexic children on reading fluency and the amplitude of N170, a negative brain-potential component elicited by letter and symbol strings. A group of 18 children with dyslexia in 3rd grade (9.05±0.46years old) was tested before and after following a letter-speech sound mapping training. A group of 20 third-grade typical readers (8.78±0.35years old) performed a single time on the same brain potential task. The training was differentially effective in speeding up reading fluency in the dyslexic children. In some children, training had a beneficial effect on reading fluency ('improvers') while a training effect was absent in others ('non-improvers'). Improvers at pre-training showed larger N170 amplitude to words compared to non-improvers. N170 amplitude decreased following training in improvers but not in non-improvers. But the N170 amplitude pattern in improvers continued to differ from the N170 amplitude pattern across hemispheres seen in typical readers. Finally, we observed a positive relation between the decrease in N170 amplitude and gains in reading fluency. Collectively, the results that emerged from the present study indicate the sensitivity of N170 amplitude to reading fluency and its potential as a predictor of reading fluency acquisition.
Collapse
Affiliation(s)
- G Fraga González
- Department of Psychology, University of Amsterdam, The Netherlands; Rudolf Berlin Center, Amsterdam, The Netherlands.
| | - G Žarić
- Department of Cognitive Neuroscience, Maastricht University, The Netherlands; Maastricht Brain Imaging Center, Maastricht University, The Netherlands
| | - J Tijms
- Department of Psychology, University of Amsterdam, The Netherlands; IWAL Institute, Amsterdam, The Netherlands
| | - M Bonte
- Department of Cognitive Neuroscience, Maastricht University, The Netherlands; Maastricht Brain Imaging Center, Maastricht University, The Netherlands
| | - L Blomert
- Department of Cognitive Neuroscience, Maastricht University, The Netherlands
| | - P Leppänen
- Department of Psychology, University of Jyväskylä, Finland
| | - M W van der Molen
- Department of Psychology, University of Amsterdam, The Netherlands; Amsterdam Brain and Cognition, University of Amsterdam, The Netherlands
| |
Collapse
|
12
|
Bonte M, Ley A, Scharke W, Formisano E. Developmental refinement of cortical systems for speech and voice processing. Neuroimage 2016; 128:373-384. [PMID: 26777479 DOI: 10.1016/j.neuroimage.2016.01.015] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2015] [Revised: 12/15/2015] [Accepted: 01/06/2016] [Indexed: 01/31/2023] Open
Affiliation(s)
- Milene Bonte
- Department of Cognitive Neuroscience and Maastricht Brain Imaging Center, Faculty of Psychology and Neuroscience, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands.
| | - Anke Ley
- Department of Cognitive Neuroscience and Maastricht Brain Imaging Center, Faculty of Psychology and Neuroscience, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands
| | - Wolfgang Scharke
- Department of Cognitive Neuroscience and Maastricht Brain Imaging Center, Faculty of Psychology and Neuroscience, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands
| | - Elia Formisano
- Department of Cognitive Neuroscience and Maastricht Brain Imaging Center, Faculty of Psychology and Neuroscience, Maastricht University, P.O. Box 616, 6200 MD, Maastricht, The Netherlands
| |
Collapse
|
13
|
Saarinen T, Jalava A, Kujala J, Stevenson C, Salmelin R. Task-sensitive reconfiguration of corticocortical 6-20 Hz oscillatory coherence in naturalistic human performance. Hum Brain Mapp 2015; 36:2455-69. [PMID: 25760689 PMCID: PMC6680250 DOI: 10.1002/hbm.22784] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2014] [Revised: 02/24/2015] [Accepted: 02/24/2015] [Indexed: 01/01/2023] Open
Abstract
Electrophysiological oscillatory coherence between brain regions has been proposed to facilitate functional long-range connectivity within neurocognitive networks. This notion is supported by intracortical recordings of coherence in singled-out corticocortical connections in the primate cortex. However, the manner in which this operational principle manifests in the task-sensitive connectivity that supports human naturalistic performance remains undercharacterized. Here, we demonstrate task-sensitive reconfiguration of global patterns of coherent connectivity in association with a set of easier and more demanding naturalistic tasks, ranging from picture comparison to speech comprehension and object manipulation. Based on whole-cortex neuromagnetic recording in healthy behaving individuals, the task-sensitive component of long-range corticocortical coherence was mapped at spectrally narrow-band oscillatory frequencies between 6 and 20 Hz (theta to alpha and low-beta bands). This data-driven cortical mapping unveiled markedly distinct and topologically task-relevant spatiospectral connectivity patterns for the different tasks. The results demonstrate semistable oscillatory states relevant for neurocognitive processing. The present findings decisively link human behavior to corticocortical coherence at oscillatory frequencies that are widely thought to convey long-range, feedback-type neural interaction in cortical functional networks.
Collapse
Affiliation(s)
- Timo Saarinen
- Brain Research UnitO.V. Lounasmaa LaboratoryAalto UniversityAALTOFinland
- Aalto NeuroImagingAalto UniversityAALTOFinland
| | - Antti Jalava
- Brain Research UnitO.V. Lounasmaa LaboratoryAalto UniversityAALTOFinland
- Aalto NeuroImagingAalto UniversityAALTOFinland
| | - Jan Kujala
- Brain Research UnitO.V. Lounasmaa LaboratoryAalto UniversityAALTOFinland
| | - Claire Stevenson
- Brain Research UnitO.V. Lounasmaa LaboratoryAalto UniversityAALTOFinland
| | - Riitta Salmelin
- Brain Research UnitO.V. Lounasmaa LaboratoryAalto UniversityAALTOFinland
- Aalto NeuroImagingAalto UniversityAALTOFinland
| |
Collapse
|
14
|
Correia JM, Jansma B, Hausfeld L, Kikkert S, Bonte M. EEG decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations. Front Psychol 2015; 6:71. [PMID: 25705197 PMCID: PMC4319403 DOI: 10.3389/fpsyg.2015.00071] [Citation(s) in RCA: 83] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2014] [Accepted: 01/13/2015] [Indexed: 11/13/2022] Open
Abstract
Spoken word recognition and production require fast transformations between acoustic, phonological, and conceptual neural representations. Bilinguals perform these transformations in native and non-native languages, deriving unified semantic concepts from equivalent, but acoustically different words. Here we exploit this capacity of bilinguals to investigate input invariant semantic representations in the brain. We acquired EEG data while Dutch subjects, highly proficient in English listened to four monosyllabic and acoustically distinct animal words in both languages (e.g., “paard”–“horse”). Multivariate pattern analysis (MVPA) was applied to identify EEG response patterns that discriminate between individual words within one language (within-language discrimination) and generalize meaning across two languages (across-language generalization). Furthermore, employing two EEG feature selection approaches, we assessed the contribution of temporal and oscillatory EEG features to our classification results. MVPA revealed that within-language discrimination was possible in a broad time-window (~50–620 ms) after word onset probably reflecting acoustic-phonetic and semantic-conceptual differences between the words. Most interestingly, significant across-language generalization was possible around 550–600 ms, suggesting the activation of common semantic-conceptual representations from the Dutch and English nouns. Both types of classification, showed a strong contribution of oscillations below 12 Hz, indicating the importance of low frequency oscillations in the neural representation of individual words and concepts. This study demonstrates the feasibility of MVPA to decode individual spoken words from EEG responses and to assess the spectro-temporal dynamics of their language invariant semantic-conceptual representations. We discuss how this method and results could be relevant to track the neural mechanisms underlying conceptual encoding in comprehension and production.
Collapse
Affiliation(s)
- João M Correia
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Bernadette Jansma
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Lars Hausfeld
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Sanne Kikkert
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Milene Bonte
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| |
Collapse
|
15
|
Kühnis J, Elmer S, Jäncke L. Auditory evoked responses in musicians during passive vowel listening are modulated by functional connectivity between bilateral auditory-related brain regions. J Cogn Neurosci 2014; 26:2750-61. [PMID: 24893742 DOI: 10.1162/jocn_a_00674] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Currently, there is striking evidence showing that professional musical training can substantially alter the response properties of auditory-related cortical fields. Such plastic changes have previously been shown not only to abet the processing of musical sounds, but likewise spectral and temporal aspects of speech. Therefore, here we used the EEG technique and measured a sample of musicians and nonmusicians while the participants were passively exposed to artificial vowels in the context of an oddball paradigm. Thereby, we evaluated whether increased intracerebral functional connectivity between bilateral auditory-related brain regions may promote sensory specialization in musicians, as reflected by altered cortical N1 and P2 responses. This assumption builds on the reasoning that sensory specialization is dependent, at least in part, on the amount of synchronization between the two auditory-related cortices. Results clearly revealed that auditory-evoked N1 responses were shaped by musical expertise. In addition, in line with our reasoning musicians showed an overall increased intracerebral functional connectivity (as indexed by lagged phase synchronization) in theta, alpha, and beta bands. Finally, within-group correlative analyses indicated a relationship between intracerebral beta band connectivity and cortical N1 responses, however only within the musicians' group. Taken together, we provide first electrophysiological evidence for a relationship between musical expertise, auditory-evoked brain responses, and intracerebral functional connectivity among auditory-related brain regions.
Collapse
|
16
|
Ley A, Vroomen J, Formisano E. How learning to abstract shapes neural sound representations. Front Neurosci 2014; 8:132. [PMID: 24917783 PMCID: PMC4043152 DOI: 10.3389/fnins.2014.00132] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2014] [Accepted: 05/14/2014] [Indexed: 12/04/2022] Open
Abstract
The transformation of acoustic signals into abstract perceptual representations is the essence of the efficient and goal-directed neural processing of sounds in complex natural environments. While the human and animal auditory system is perfectly equipped to process the spectrotemporal sound features, adequate sound identification and categorization require neural sound representations that are invariant to irrelevant stimulus parameters. Crucially, what is relevant and irrelevant is not necessarily intrinsic to the physical stimulus structure but needs to be learned over time, often through integration of information from other senses. This review discusses the main principles underlying categorical sound perception with a special focus on the role of learning and neural plasticity. We examine the role of different neural structures along the auditory processing pathway in the formation of abstract sound representations with respect to hierarchical as well as dynamic and distributed processing models. Whereas most fMRI studies on categorical sound processing employed speech sounds, the emphasis of the current review lies on the contribution of empirical studies using natural or artificial sounds that enable separating acoustic and perceptual processing levels and avoid interference with existing category representations. Finally, we discuss the opportunities of modern analyses techniques such as multivariate pattern analysis (MVPA) in studying categorical sound representations. With their increased sensitivity to distributed activation changes—even in absence of changes in overall signal level—these analyses techniques provide a promising tool to reveal the neural underpinnings of perceptually invariant sound representations.
Collapse
Affiliation(s)
- Anke Ley
- Department of Medical Psychology and Neuropsychology, Tilburg School of Social and Behavioral Sciences, Tilburg University Tilburg, Netherlands ; Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University Maastricht, Netherlands
| | - Jean Vroomen
- Department of Medical Psychology and Neuropsychology, Tilburg School of Social and Behavioral Sciences, Tilburg University Tilburg, Netherlands
| | - Elia Formisano
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University Maastricht, Netherlands
| |
Collapse
|
17
|
van Atteveldt N, Murray MM, Thut G, Schroeder CE. Multisensory integration: flexible use of general operations. Neuron 2014; 81:1240-1253. [PMID: 24656248 DOI: 10.1016/j.neuron.2014.02.044] [Citation(s) in RCA: 176] [Impact Index Per Article: 17.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/27/2014] [Indexed: 11/25/2022]
Abstract
Research into the anatomical substrates and "principles" for integrating inputs from separate sensory surfaces has yielded divergent findings. This suggests that multisensory integration is flexible and context dependent and underlines the need for dynamically adaptive neuronal integration mechanisms. We propose that flexible multisensory integration can be explained by a combination of canonical, population-level integrative operations, such as oscillatory phase resetting and divisive normalization. These canonical operations subsume multisensory integration into a fundamental set of principles as to how the brain integrates all sorts of information, and they are being used proactively and adaptively. We illustrate this proposition by unifying recent findings from different research themes such as timing, behavioral goal, and experience-related differences in integration.
Collapse
Affiliation(s)
- Nienke van Atteveldt
- Neuroimaging & Neuromodeling group, Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts and Sciences, Meibergdreef 47, 1105 BA Amsterdam, The Netherlands; Department of Educational Neuroscience, Faculty of Psychology & Education and Institute LEARN!, VU University Amsterdam, van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands; Department of Cognitive Neuroscience, Faculty of Psychology & Neuroscience, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands.
| | - Micah M Murray
- The Laboratory for Investigative Neurophysiology (the LINE), Neuropsychology and Neurorehabilitation Service and Radiodiagnostic Service, University Hospital Center and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland; EEG Brain Mapping Core, Centre for Biomedical Imaging (CIBM), Rue du Bugnon 46, 1011 Lausanne, Switzerland
| | - Gregor Thut
- Institute of Neuroscience and Psychology, University of Glasgow, 58 Hillhead Street, Glasgow, G12 8QB, UK
| | - Charles E Schroeder
- Columbia University, Department Psychiatry, and the New York State Psychiatric Institute, 1051 Riverside Drive, New York, NY 10032, USA; Nathan S. Kline Institute, Cognitive Neuroscience & Schizophrenia Program, 140 Old Orangeburg Road, Orangeburg, NY 10962, USA.
| |
Collapse
|
18
|
Task-dependent decoding of speaker and vowel identity from auditory cortical response patterns. J Neurosci 2014; 34:4548-57. [PMID: 24672000 DOI: 10.1523/jneurosci.4339-13.2014] [Citation(s) in RCA: 65] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
Selective attention to relevant sound properties is essential for everyday listening situations. It enables the formation of different perceptual representations of the same acoustic input and is at the basis of flexible and goal-dependent behavior. Here, we investigated the role of the human auditory cortex in forming behavior-dependent representations of sounds. We used single-trial fMRI and analyzed cortical responses collected while subjects listened to the same speech sounds (vowels /a/, /i/, and /u/) spoken by different speakers (boy, girl, male) and performed a delayed-match-to-sample task on either speech sound or speaker identity. Univariate analyses showed a task-specific activation increase in the right superior temporal gyrus/sulcus (STG/STS) during speaker categorization and in the right posterior temporal cortex during vowel categorization. Beyond regional differences in activation levels, multivariate classification of single trial responses demonstrated that the success with which single speakers and vowels can be decoded from auditory cortical activation patterns depends on task demands and subject's behavioral performance. Speaker/vowel classification relied on distinct but overlapping regions across the (right) mid-anterior STG/STS (speakers) and bilateral mid-posterior STG/STS (vowels), as well as the superior temporal plane including Heschl's gyrus/sulcus. The task dependency of speaker/vowel classification demonstrates that the informative fMRI response patterns reflect the top-down enhancement of behaviorally relevant sound representations. Furthermore, our findings suggest that successful selection, processing, and retention of task-relevant sound properties relies on the joint encoding of information across early and higher-order regions of the auditory cortex.
Collapse
|
19
|
Kim J, Lee SK, Lee B. EEG classification in a single-trial basis for vowel speech perception using multivariate empirical mode decomposition. J Neural Eng 2014; 11:036010. [PMID: 24809722 DOI: 10.1088/1741-2560/11/3/036010] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVE The objective of this study is to find components that might be related to phoneme representation in the brain and to discriminate EEG responses for each speech sound on a trial basis. APPROACH We used multivariate empirical mode decomposition (MEMD) and common spatial pattern for feature extraction. We chose three vowel stimuli, /a/, /i/ and /u/, based on previous findings, such that the brain can detect change in formant frequency (F2) of vowels. EEG activity was recorded from seven native Korean speakers at Gwangju Institute of Science and Technology. We applied MEMD over EEG channels to extract speech-related brain signal sources, and looked for the intrinsic mode functions which were dominant in the alpha bands. After the MEMD procedure, we applied the common spatial pattern algorithm for enhancing the classification performance, and used linear discriminant analysis (LDA) as a classifier. MAIN RESULTS The brain responses to the three vowels could be classified as one of the learned phonemes on a single-trial basis with our approach. SIGNIFICANCE The results of our study show that brain responses to vowels can be classified for single trials using MEMD and LDA. This approach may not only become a useful tool for the brain-computer interface but it could also be used for discriminating the neural correlates of categorical speech perception.
Collapse
Affiliation(s)
- Jongin Kim
- Department of Medical System Engineering (DMSE), Gwangju Institute of Science and Technology (GIST), Gwangju, Republic of Korea
| | | | | |
Collapse
|
20
|
Varvatsoulias G. Voice-Sensitive Areas in the Brain: A Single Participant Study Coupled With Brief Evolutionary Psychological Considerations. PSYCHOLOGICAL THOUGHT 2014. [DOI: 10.5964/psyct.v7i1.98] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
|
21
|
Schall S, Kiebel SJ, Maess B, von Kriegstein K. Voice Identity Recognition: Functional Division of the Right STS and Its Behavioral Relevance. J Cogn Neurosci 2014; 27:280-91. [DOI: 10.1162/jocn_a_00707] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Abstract
The human voice is the primary carrier of speech but also a fingerprint for person identity. Previous neuroimaging studies have revealed that speech and identity recognition is accomplished by partially different neural pathways, despite the perceptual unity of the vocal sound. Importantly, the right STS has been implicated in voice processing, with different contributions of its posterior and anterior parts. However, the time point at which vocal and speech processing diverge is currently unknown. Also, the exact role of the right STS during voice processing is so far unclear because its behavioral relevance has not yet been established. Here, we used the high temporal resolution of magnetoencephalography and a speech task control to pinpoint transient behavioral correlates: we found, at 200 msec after stimulus onset, that activity in right anterior STS predicted behavioral voice recognition performance. At the same time point, the posterior right STS showed increased activity during voice identity recognition in contrast to speech recognition whereas the left mid STS showed the reverse pattern. In contrast to the highly speech-sensitive left STS, the current results highlight the right STS as a key area for voice identity recognition and show that its anatomical-functional division emerges around 200 msec after stimulus onset. We suggest that this time point marks the speech-independent processing of vocal sounds in the posterior STS and their successful mapping to vocal identities in the anterior STS.
Collapse
Affiliation(s)
- Sonja Schall
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig
| | - Stefan J. Kiebel
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig
- 2University Clinic Jena
- 3Technical University, Dresden
| | - Burkhard Maess
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig
| | - Katharina von Kriegstein
- 1Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig
- 4Humboldt University of Berlin
| |
Collapse
|
22
|
Talja S, Alho K, Rinne T. Source analysis of event-related potentials during pitch discrimination and pitch memory tasks. Brain Topogr 2013; 28:445-58. [PMID: 24043402 DOI: 10.1007/s10548-013-0307-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2013] [Accepted: 08/10/2013] [Indexed: 11/26/2022]
Abstract
Our previous studies using fMRI have demonstrated that activations in human auditory cortex (AC) are strongly dependent on the characteristics of the task. The present study tested whether source estimation of scalp-recorded event-related potentials (ERPs) can be used to investigate task-dependent AC activations. Subjects were presented with frequency-varying two-part tones during pitch discrimination, pitch n-back memory, and visual tasks identical to our previous fMRI study (Rinne et al., J Neurosci 29:13338-13343, 2009). ERPs and their minimum-norm source estimates in AC were strongly modulated by task at 200-700 ms from tone onset. As in the fMRI study, the pitch discrimination and pitch memory tasks were associated with distinct AC activation patterns. In the pitch discrimination task, increased activity in the anterior AC was detected relatively late at 300-700 ms from tone onset. Therefore, this activity was probably not associated with enhanced pitch processing but rather with the actual discrimination process (comparison between the two parts of tone). Increased activity in more posterior areas associated with the pitch memory task, in turn, occurred at 200-700 ms suggesting that this activity was related to operations on pitch categories after pitch analysis was completed. Finally, decreased activity associated with the pitch memory task occurred at 150-300 ms consistent with the notion that, in the demanding pitch memory task, spectrotemporal analysis is actively halted as soon as category information has been obtained. These results demonstrate that ERP source analysis can be used to complement fMRI to investigate task-dependent activations of human AC.
Collapse
Affiliation(s)
- Suvi Talja
- Institute of Behavioural Sciences, University of Helsinki, PO Box 9, 00014, Helsinki, Finland,
| | | | | |
Collapse
|
23
|
Zion Golumbic EM, Poeppel D, Schroeder CE. Temporal context in speech processing and attentional stream selection: a behavioral and neural perspective. BRAIN AND LANGUAGE 2012; 122:151-61. [PMID: 22285024 PMCID: PMC3340429 DOI: 10.1016/j.bandl.2011.12.010] [Citation(s) in RCA: 108] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/07/2011] [Revised: 12/14/2011] [Accepted: 12/16/2011] [Indexed: 05/04/2023]
Abstract
The human capacity for processing speech is remarkable, especially given that information in speech unfolds over multiple time scales concurrently. Similarly notable is our ability to filter out of extraneous sounds and focus our attention on one conversation, epitomized by the 'Cocktail Party' effect. Yet, the neural mechanisms underlying on-line speech decoding and attentional stream selection are not well understood. We review findings from behavioral and neurophysiological investigations that underscore the importance of the temporal structure of speech for achieving these perceptual feats. We discuss the hypothesis that entrainment of ambient neuronal oscillations to speech's temporal structure, across multiple time-scales, serves to facilitate its decoding and underlies the selection of an attended speech stream over other competing input. In this regard, speech decoding and attentional stream selection are examples of 'Active Sensing', emphasizing an interaction between proactive and predictive top-down modulation of neuronal dynamics and bottom-up sensory input.
Collapse
Affiliation(s)
- Elana M Zion Golumbic
- Department of Psychiatry, Columbia University Medical Center, 710 W 168th St., New York, NY 10032, USA.
| | | | | |
Collapse
|
24
|
Pattern analysis of EEG responses to speech and voice: Influence of feature grouping. Neuroimage 2012; 59:3641-51. [DOI: 10.1016/j.neuroimage.2011.11.056] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2011] [Revised: 10/24/2011] [Accepted: 11/16/2011] [Indexed: 11/21/2022] Open
|
25
|
Obleser J, Kotz SA. Multiple brain signatures of integration in the comprehension of degraded speech. Neuroimage 2011; 55:713-23. [PMID: 21172443 DOI: 10.1016/j.neuroimage.2010.12.020] [Citation(s) in RCA: 101] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2010] [Revised: 11/26/2010] [Accepted: 12/06/2010] [Indexed: 11/20/2022] Open
Affiliation(s)
- Jonas Obleser
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | | |
Collapse
|
26
|
Telenczuk B, Nikulin VV, Curio G. Role of neuronal synchrony in the generation of evoked EEG/MEG responses. J Neurophysiol 2010; 104:3557-67. [PMID: 20943941 DOI: 10.1152/jn.00138.2010] [Citation(s) in RCA: 27] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Evoked EEG/MEG responses are a primary real-time measure of perceptual and cognitive activity in the human brain, but their neuronal generator mechanisms are not yet fully understood. Arguments have been put forward in favor of either "phase-reset" of ongoing oscillations or "added-energy" models. Instead of advocating for one or the other model, here we show theoretically that the differentiation between these two generation mechanisms might not be possible if based solely on macroscopic EEG/MEG recordings. Using mathematical modeling, we show that a simultaneous phase reset of multiple oscillating neuronal (microscopic) sources contributing to EEG/MEG can produce evoked responses in agreement with both, the "added-energy" and the "phase-reset" model. We observe a smooth transition between the two models by just varying the strength of synchronization between the multiple microscopic sources. Consequently, because precise knowledge about the strength of microscopic ensemble synchronization is commonly not available in noninvasive EEG/MEG studies, they cannot, in principle, differentiate between the two mechanisms for macroscopic-evoked responses.
Collapse
Affiliation(s)
- Bartosz Telenczuk
- Dept. of Neurology, Charité-Universitätsmedizin, Hindenburgdamm 30, 12203 Berlin, Germany.
| | | | | |
Collapse
|
27
|
Rao A, Zhang Y, Miller S. Selective listening of concurrent auditory stimuli: An event-related potential study. Hear Res 2010; 268:123-32. [DOI: 10.1016/j.heares.2010.05.013] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/23/2009] [Revised: 04/28/2010] [Accepted: 05/21/2010] [Indexed: 10/19/2022]
|
28
|
Monahan PJ, Idsardi WJ. Auditory Sensitivity to Formant Ratios:Toward an Account of Vowel Normalization. LANGUAGE AND COGNITIVE PROCESSES 2010; 25:808-839. [PMID: 20606713 PMCID: PMC2893733 DOI: 10.1080/01690965.2010.490047] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
A long-standing question in speech perception research is how do listeners extract linguistic content from a highly variable acoustic input. In the domain of vowel perception, formant ratios, or the calculation of relative bark differences between vowel formants, have been a sporadically proposed solution. We propose a novel formant ratio algorithm in which the first (F1) and second (F2) formants are compared against the third formant (F3). Results from two magnetoencephelographic (MEG) experiments are presented that suggest auditory cortex is sensitive to formant ratios. Our findings also demonstrate that the perceptual system shows heightened sensitivity to formant ratios for tokens located in more crowded regions of the vowel space. Additionally, we present statistical evidence that this algorithm eliminates speaker-dependent variation based on age and gender from vowel productions. We conclude that these results present an impetus to reconsider formant ratios as a legitimate mechanistic component in the solution to the problem of speaker normalization.
Collapse
Affiliation(s)
- Philip J. Monahan
- Basque Center on Cognition, Brain and Language, Donostia-San Sebastián, Spain
| | - William J. Idsardi
- Department of Linguistics, University of Maryland, USA
- Neuroscience and Cognitive Science Program University of Maryland, USA
| |
Collapse
|
29
|
Abstract
Normal listeners possess the remarkable perceptual ability to select a single speech stream among many competing talkers. However, few studies of selective attention have addressed the unique nature of speech as a temporally extended and complex auditory object. We hypothesized that sustained selective attention to speech in a multitalker environment would act as gain control on the early auditory cortical representations of speech. Using high-density electroencephalography and a template-matching analysis method, we found selective gain to the continuous speech content of an attended talker, greatest at a frequency of 4-8 Hz, in auditory cortex. In addition, the difference in alpha power (8-12 Hz) at parietal sites across hemispheres indicated the direction of auditory attention to speech, as has been previously found in visual tasks. The strength of this hemispheric alpha lateralization, in turn, predicted an individual's attentional gain of the cortical speech signal. These results support a model of spatial speech stream segregation, mediated by a supramodal attention mechanism, enabling selection of the attended representation in auditory cortex.
Collapse
|
30
|
Riecke L, Esposito F, Bonte M, Formisano E. Hearing illusory sounds in noise: the timing of sensory-perceptual transformations in auditory cortex. Neuron 2009; 64:550-61. [PMID: 19945396 DOI: 10.1016/j.neuron.2009.10.016] [Citation(s) in RCA: 55] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/05/2009] [Indexed: 11/18/2022]
Abstract
Constructive mechanisms in the auditory system may restore a fragmented sound when a gap in this sound is rendered inaudible by noise to yield a continuity illusion. Using combined psychoacoustic and electroencephalography experiments in humans, we found that the sensory-perceptual mechanisms that enable restoration suppress auditory cortical encoding of gaps in interrupted sounds. When physically interrupted tones are perceptually restored, stimulus-evoked synchronization of cortical oscillations at approximately 4 Hz is suppressed as if physically uninterrupted sounds were encoded. The restoration-specific suppression is induced most strongly in primary-like regions in the right auditory cortex during illusorily filled gaps and also shortly before and after these gaps. Our results reveal that spontaneous modulations in slow evoked auditory cortical oscillations that are involved in encoding acoustic boundaries may determine the perceived continuity of sounds in noise. Such fluctuations could facilitate stable hearing of fragmented sounds in natural environments.
Collapse
Affiliation(s)
- Lars Riecke
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, 6200 MD Maastricht, The Netherlands.
| | | | | | | |
Collapse
|
31
|
Fukuda M, Rothermel R, Juhász C, Nishida M, Sood S, Asano E. Cortical gamma-oscillations modulated by listening and overt repetition of phonemes. Neuroimage 2009; 49:2735-45. [PMID: 19874898 DOI: 10.1016/j.neuroimage.2009.10.047] [Citation(s) in RCA: 31] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2009] [Revised: 10/12/2009] [Accepted: 10/14/2009] [Indexed: 10/20/2022] Open
Abstract
Both superior temporal gyrus and inferior Rolandic area have been reported to be involved in perception and production of speech in humans. Here, we determined how these cortical structures were activated by listening and subsequent overt articulation of syllables, by measuring event-related gamma-oscillations as quantitative measures of cortical activation. Fifteen subjects were presented an auditory syllable consisting of either "fee [fi:]," "faa [falpha:]," "hee [hi:]," or "haa [halpha:]," and were instructed to overtly repeat each given syllable. Gamma-oscillations in the superior temporal gyrus were highly augmented during syllable-presentation, least augmented at the onset of syllable-articulation, and again highly augmented following syllable-articulation. Gamma-oscillations were augmented in the inferior Rolandic area prior to and during syllable-articulation with the onset and peak occurring earlier in the left side. Subsets of the inferior Rolandic sites, more frequently on the left side, showed differential gamma-augmentation elicited by articulation of phoneme [f] more than [h] or phoneme [i:] more than [alpha:]. Our observations suggest that the superior temporal gyrus may be active when externally presented or articulated auditory stimuli are present, and may be minimally active when articulation is about to be initiated. Our novel observation of phoneme-specific differential gamma-augmentation in the inferior Rolandic area may be partially attributed to the mouth position during phoneme-articulation. Our observations support the hypothesis that positioning of the mouth to articulate phonemes is predominantly driven and/or monitored by the primary sensorimotor area on the left side.
Collapse
Affiliation(s)
- Miho Fukuda
- Department of Pediatrics, Children's Hospital of Michigan, Wayne State University, Detroit, MI 48201, USA
| | | | | | | | | | | |
Collapse
|