1
|
Manca AD, Di Russo F, Sigona F, Grimaldi M. Electrophysiological evidence of phonemotopic representations of vowels in the primary and secondary auditory cortex. Cortex 2019; 121:385-398. [PMID: 31678684 DOI: 10.1016/j.cortex.2019.09.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2018] [Revised: 05/18/2019] [Accepted: 09/20/2019] [Indexed: 11/25/2022]
Abstract
How the brain encodes the speech acoustic signal into phonological representations is a fundamental question for the neurobiology of language. Determining whether this process is characterized by tonotopic maps in primary or secondary auditory areas, with bilateral or leftward activity, remains a long-standing challenge. Magnetoencephalographic studies failed to show hierarchical and asymmetric hints for speech processing. We employed high-density electroencephalography to map the Salento Italian vowel system onto cortical sources using the N1 auditory evoked component. We found evidence that the N1 is characterized by hierarchical and asymmetrical indexes in primary and secondary auditory areas structuring vowel representations. Importantly, the N1 was characterized by early and late phases. The early N1 peaked at 125-135 msec and was localized in the primary auditory cortex; the late N1 peaked at 145-155 msec and was localized in the left superior temporal gyrus. We showed that early in the primary auditory cortex, the cortical spatial arrangements-along the lateral-medial and anterior-posterior gradients-are broadly warped by phonemotopic patterns according to the distinctive feature principle. These phonemotopic patterns are carefully refined in the superior temporal gyrus along the inferior-superior and anterior-posterior gradients. The dynamical and hierarchical interface between primary and secondary auditory areas and the interaction effects between Height and Place features generate the categorical representation of the Salento Italian vowels.
Collapse
Affiliation(s)
- Anna Dora Manca
- Centro di Ricerca Interdisciplinare sul Linguaggio (CRIL), University of Salento, Lecce, Italy; Laboratorio Diffuso di Ricerca interdisciplinare Applicata alla Medicina (DReAM), Lecce, Italy
| | - Francesco Di Russo
- Dipartimento di Scienze Motorie, Umane e della Salute, University of Rome "Foro Italico", Rome, Italy; IRCCS Fondazione Santa Lucia, Rome, Italy
| | - Francesco Sigona
- Centro di Ricerca Interdisciplinare sul Linguaggio (CRIL), University of Salento, Lecce, Italy; Laboratorio Diffuso di Ricerca interdisciplinare Applicata alla Medicina (DReAM), Lecce, Italy
| | - Mirko Grimaldi
- Centro di Ricerca Interdisciplinare sul Linguaggio (CRIL), University of Salento, Lecce, Italy; Laboratorio Diffuso di Ricerca interdisciplinare Applicata alla Medicina (DReAM), Lecce, Italy.
| |
Collapse
|
2
|
What's what in auditory cortices? Neuroimage 2018; 176:29-40. [DOI: 10.1016/j.neuroimage.2018.04.028] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2018] [Revised: 04/04/2018] [Accepted: 04/12/2018] [Indexed: 11/30/2022] Open
|
3
|
Silva DMR, Melges DB, Rothe-Neves R. N1 response attenuation and the mismatch negativity (MMN) to within- and across-category phonetic contrasts. Psychophysiology 2017; 54:591-600. [PMID: 28169421 DOI: 10.1111/psyp.12824] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2016] [Accepted: 12/07/2016] [Indexed: 11/29/2022]
Abstract
According to the neural adaptation model of the mismatch negativity (MMN), the sensitivity of this event-related response to both acoustic and categorical information in speech sounds can be accounted for by assuming that (a) the degree of overlapping between neural representations of two sounds depends on both the acoustic difference between them and whether or not they belong to distinct phonetic categories, and (b) a release from stimulus-specific adaptation causes an enhanced N1 obligatory response to infrequent deviant stimuli. On the basis of this view, we tested in Experiment 1 whether the N1 response to the second sound of a pair (S2 ) would be more attenuated in pairs of identical vowels compared with pairs of different vowels, and in pairs of exemplars of the same vowel category compared with pairs of exemplars of different categories. The psychoacoustic distance between S1 and S2 was the same for all within-category and across-category pairs. While N1 amplitudes decreased markedly from S1 to S2 , responses to S2 were quite similar across pair types, indicating that the attenuation effect in such conditions is not stimulus specific. In Experiment 2, a pronounced MMN was elicited by a deviant vowel sound in an across-category oddball sequence, but not when the exact same deviant vowel was presented in a within-category oddball sequence. This adds evidence that MMN reflects categorical phonetic processing. Taken together, the results suggest that different neural processes underlie the attenuation of the N1 response to S2 and the MMN to vowels.
Collapse
Affiliation(s)
- Daniel M R Silva
- Graduate Program in Neuroscience, Federal University of Minas Gerais, Belo Horizonte, Brazil
| | - Danilo B Melges
- Graduate Program in Electrical Engineering, Department of Electrical Engineering, Federal University of Minas Gerais, Belo Horizonte, Brazil
| | - Rui Rothe-Neves
- Phonetics Lab, Faculty of Letters, Federal University of Minas Gerais, Belo Horizonte, Brazil
| |
Collapse
|
4
|
Kawase T, Yahata I, Kanno A, Sakamoto S, Takanashi Y, Takata S, Nakasato N, Kawashima R, Katori Y. Impact of Audio-Visual Asynchrony on Lip-Reading Effects -Neuromagnetic and Psychophysical Study. PLoS One 2016; 11:e0168740. [PMID: 28030631 PMCID: PMC5193434 DOI: 10.1371/journal.pone.0168740] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2016] [Accepted: 12/05/2016] [Indexed: 11/18/2022] Open
Abstract
The effects of asynchrony between audio and visual (A/V) stimuli on the N100m responses of magnetoencephalography in the left hemisphere were compared with those on the psychophysical responses in 11 participants. The latency and amplitude of N100m were significantly shortened and reduced in the left hemisphere by the presentation of visual speech as long as the temporal asynchrony between A/V stimuli was within 100 ms, but were not significantly affected with audio lags of -500 and +500 ms. However, some small effects were still preserved on average with audio lags of 500 ms, suggesting similar asymmetry of the temporal window to that observed in psychophysical measurements, which tended to be more robust (wider) for audio lags; i.e., the pattern of visual-speech effects as a function of A/V lag observed in the N100m in the left hemisphere grossly resembled that in psychophysical measurements on average, although the individual responses were somewhat varied. The present results suggest that the basic configuration of the temporal window of visual effects on auditory-speech perception could be observed from the early auditory processing stage.
Collapse
Affiliation(s)
- Tetsuaki Kawase
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
- Laboratory of Rehabilitative Auditory Science, Tohoku University Graduate School of Biomedical Engineering, Sendai, Miyagi, Japan
- Department of Audiology, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
- * E-mail:
| | - Izumi Yahata
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Akitake Kanno
- Department of Functional Brain Imaging, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Miyagi, Japan
| | - Shuichi Sakamoto
- Research Institute of Electrical Communication, Tohoku University, Sendai, Miyagi, Japan
| | - Yoshitaka Takanashi
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Shiho Takata
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Nobukazu Nakasato
- Department of Epileptology, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Ryuta Kawashima
- Department of Functional Brain Imaging, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Miyagi, Japan
| | - Yukio Katori
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| |
Collapse
|
5
|
Manca AD, Grimaldi M. Vowels and Consonants in the Brain: Evidence from Magnetoencephalographic Studies on the N1m in Normal-Hearing Listeners. Front Psychol 2016; 7:1413. [PMID: 27713712 PMCID: PMC5031792 DOI: 10.3389/fpsyg.2016.01413] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2016] [Accepted: 09/05/2016] [Indexed: 01/07/2023] Open
Abstract
Speech sound perception is one of the most fascinating tasks performed by the human brain. It involves a mapping from continuous acoustic waveforms onto the discrete phonological units computed to store words in the mental lexicon. In this article, we review the magnetoencephalographic studies that have explored the timing and morphology of the N1m component to investigate how vowels and consonants are computed and represented within the auditory cortex. The neurons that are involved in the N1m act to construct a sensory memory of the stimulus due to spatially and temporally distributed activation patterns within the auditory cortex. Indeed, localization of auditory fields maps in animals and humans suggested two levels of sound coding, a tonotopy dimension for spectral properties and a tonochrony dimension for temporal properties of sounds. When the stimulus is a complex speech sound, tonotopy and tonochrony data may give important information to assess whether the speech sound parsing and decoding are generated by pure bottom-up reflection of acoustic differences or whether they are additionally affected by top-down processes related to phonological categories. Hints supporting pure bottom-up processing coexist with hints supporting top-down abstract phoneme representation. Actually, N1m data (amplitude, latency, source generators, and hemispheric distribution) are limited and do not help to disentangle the issue. The nature of these limitations is discussed. Moreover, neurophysiological studies on animals and neuroimaging studies on humans have been taken into consideration. We compare also the N1m findings with the investigation of the magnetic mismatch negativity (MMNm) component and with the analogous electrical components, the N1 and the MMN. We conclude that N1 seems more sensitive to capture lateralization and hierarchical processes than N1m, although the data are very preliminary. Finally, we suggest that MEG data should be integrated with EEG data in the light of the neural oscillations framework and we propose some concerns that should be addressed by future investigations if we want to closely line up language research with issues at the core of the functional brain mechanisms.
Collapse
Affiliation(s)
- Anna Dora Manca
- Dipartimento di Studi Umanistici, Centro di Ricerca Interdisciplinare sul Linguaggio, University of SalentoLecce, Italy; Laboratorio Diffuso di Ricerca Interdisciplinare Applicata alla MedicinaLecce, Italy
| | - Mirko Grimaldi
- Dipartimento di Studi Umanistici, Centro di Ricerca Interdisciplinare sul Linguaggio, University of SalentoLecce, Italy; Laboratorio Diffuso di Ricerca Interdisciplinare Applicata alla MedicinaLecce, Italy
| |
Collapse
|
6
|
Correia JM, Jansma B, Hausfeld L, Kikkert S, Bonte M. EEG decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations. Front Psychol 2015; 6:71. [PMID: 25705197 PMCID: PMC4319403 DOI: 10.3389/fpsyg.2015.00071] [Citation(s) in RCA: 83] [Impact Index Per Article: 9.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2014] [Accepted: 01/13/2015] [Indexed: 11/13/2022] Open
Abstract
Spoken word recognition and production require fast transformations between acoustic, phonological, and conceptual neural representations. Bilinguals perform these transformations in native and non-native languages, deriving unified semantic concepts from equivalent, but acoustically different words. Here we exploit this capacity of bilinguals to investigate input invariant semantic representations in the brain. We acquired EEG data while Dutch subjects, highly proficient in English listened to four monosyllabic and acoustically distinct animal words in both languages (e.g., “paard”–“horse”). Multivariate pattern analysis (MVPA) was applied to identify EEG response patterns that discriminate between individual words within one language (within-language discrimination) and generalize meaning across two languages (across-language generalization). Furthermore, employing two EEG feature selection approaches, we assessed the contribution of temporal and oscillatory EEG features to our classification results. MVPA revealed that within-language discrimination was possible in a broad time-window (~50–620 ms) after word onset probably reflecting acoustic-phonetic and semantic-conceptual differences between the words. Most interestingly, significant across-language generalization was possible around 550–600 ms, suggesting the activation of common semantic-conceptual representations from the Dutch and English nouns. Both types of classification, showed a strong contribution of oscillations below 12 Hz, indicating the importance of low frequency oscillations in the neural representation of individual words and concepts. This study demonstrates the feasibility of MVPA to decode individual spoken words from EEG responses and to assess the spectro-temporal dynamics of their language invariant semantic-conceptual representations. We discuss how this method and results could be relevant to track the neural mechanisms underlying conceptual encoding in comprehension and production.
Collapse
Affiliation(s)
- João M Correia
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Bernadette Jansma
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Lars Hausfeld
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Sanne Kikkert
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| | - Milene Bonte
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht Brain Imaging Center (M-BIC), Maastricht University Maastricht, Netherlands
| |
Collapse
|
7
|
Scharinger M, Monahan PJ, Idsardi WJ. Asymmetries in the processing of vowel height. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2012; 55:903-918. [PMID: 22232394 DOI: 10.1044/1092-4388(2011/11-0065)] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
PURPOSE Speech perception can be described as the transformation of continuous acoustic information into discrete memory representations. Therefore, research on neural representations of speech sounds is particularly important for a better understanding of this transformation. Speech perception models make specific assumptions regarding the representation of mid vowels (e.g., [ε]) that are articulated with a neutral position in regard to height. One hypothesis is that their representation is less specific than the representation of vowels with a more specific position (e.g., [æ]). METHOD In a magnetoencephalography study, we tested the underspecification of mid vowel in American English. Using a mismatch negativity (MMN) paradigm, mid and low lax vowels ([ε]/[æ]), and high and low lax vowels ([i]/[æ]), were opposed, and M100/N1 dipole source parameters as well as MMN latency and amplitude were examined. RESULTS Larger MMNs occurred when the mid vowel [ε] was a deviant to the standard [æ], a result consistent with less specific representations for mid vowels. MMNs of equal magnitude were elicited in the high-low comparison, consistent with more specific representations for both high and low vowels. M100 dipole locations support early vowel categorization on the basis of linguistically relevant acoustic-phonetic features. CONCLUSION We take our results to reflect an abstract long-term representation of vowels that do not include redundant specifications at very early stages of processing the speech signal. Moreover, the dipole locations indicate extraction of distinctive features and their mapping onto representationally faithful cortical locations (i.e., a feature map).
Collapse
|
8
|
Pattern analysis of EEG responses to speech and voice: Influence of feature grouping. Neuroimage 2012; 59:3641-51. [DOI: 10.1016/j.neuroimage.2011.11.056] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2011] [Revised: 10/24/2011] [Accepted: 11/16/2011] [Indexed: 11/21/2022] Open
|
9
|
Obleser J, Meyer L, Friederici AD. Dynamic assignment of neural resources in auditory comprehension of complex sentences. Neuroimage 2011; 56:2310-20. [DOI: 10.1016/j.neuroimage.2011.03.035] [Citation(s) in RCA: 53] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2010] [Revised: 03/08/2011] [Accepted: 03/11/2011] [Indexed: 11/26/2022] Open
|
10
|
Scharinger M, Idsardi WJ, Poe S. A comprehensive three-dimensional cortical map of vowel space. J Cogn Neurosci 2011; 23:3972-82. [PMID: 21568638 DOI: 10.1162/jocn_a_00056] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Mammalian cortex is known to contain various kinds of spatial encoding schemes for sensory information including retinotopic, somatosensory, and tonotopic maps. Tonotopic maps are especially interesting for human speech sound processing because they encode linguistically salient acoustic properties. In this study, we mapped the entire vowel space of a language (Turkish) onto cortical locations by using the magnetic N1 (M100), an auditory-evoked component that peaks approximately 100 msec after auditory stimulus onset. We found that dipole locations could be structured into two distinct maps, one for vowels produced with the tongue positioned toward the front of the mouth (front vowels) and one for vowels produced in the back of the mouth (back vowels). Furthermore, we found spatial gradients in lateral-medial, anterior-posterior, and inferior-superior dimensions that encoded the phonetic, categorical distinctions between all the vowels of Turkish. Statistical model comparisons of the dipole locations suggest that the spatial encoding scheme is not entirely based on acoustic bottom-up information but crucially involves featural-phonetic top-down modulation. Thus, multiple areas of excitation along the unidimensional basilar membrane are mapped into higher dimensional representations in auditory cortex.
Collapse
|
11
|
Scharinger M, Monahan PJ, Idsardi WJ. You had me at "Hello": Rapid extraction of dialect information from spoken words. Neuroimage 2011; 56:2329-38. [PMID: 21511041 DOI: 10.1016/j.neuroimage.2011.04.007] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2010] [Revised: 03/22/2011] [Accepted: 04/04/2011] [Indexed: 10/18/2022] Open
Abstract
Research on the neuronal underpinnings of speaker identity recognition has identified voice-selective areas in the human brain with evolutionary homologues in non-human primates who have comparable areas for processing species-specific calls. Most studies have focused on estimating the extent and location of these areas. In contrast, relatively few experiments have investigated the time-course of speaker identity, and in particular, dialect processing and identification by electro- or neuromagnetic means. We show here that dialect extraction occurs speaker-independently, pre-attentively and categorically. We used Standard American English and African-American English exemplars of 'Hello' in a magnetoencephalographic (MEG) Mismatch Negativity (MMN) experiment. The MMN as an automatic change detection response of the brain reflected dialect differences that were not entirely reducible to acoustic differences between the pronunciations of 'Hello'. Source analyses of the M100, an auditory evoked response to the vowels suggested additional processing in voice-selective areas whenever a dialect change was detected. These findings are not only relevant for the cognitive neuroscience of language, but also for the social sciences concerned with dialect and race perception.
Collapse
Affiliation(s)
- Mathias Scharinger
- Department of Linguistics, University of Maryland, College Park, MD, USA.
| | | | | |
Collapse
|
12
|
Obleser J, Kotz SA. Multiple brain signatures of integration in the comprehension of degraded speech. Neuroimage 2011; 55:713-23. [PMID: 21172443 DOI: 10.1016/j.neuroimage.2010.12.020] [Citation(s) in RCA: 101] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2010] [Revised: 11/26/2010] [Accepted: 12/06/2010] [Indexed: 11/20/2022] Open
Affiliation(s)
- Jonas Obleser
- Department of Neuropsychology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | | |
Collapse
|
13
|
Scharinger M, Merickel J, Riley J, Idsardi WJ. Neuromagnetic evidence for a featural distinction of English consonants: sensor- and source-space data. BRAIN AND LANGUAGE 2011; 116:71-82. [PMID: 21185073 PMCID: PMC3031676 DOI: 10.1016/j.bandl.2010.11.002] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2009] [Revised: 10/31/2010] [Accepted: 11/15/2010] [Indexed: 05/30/2023]
Abstract
Speech sounds can be classified on the basis of their underlying articulators or on the basis of the acoustic characteristics resulting from particular articulatory positions. Research in speech perception suggests that distinctive features are based on both articulatory and acoustic information. In recent years, neuroelectric and neuromagnetic investigations provided evidence for the brain's early sensitivity to distinctive features and their acoustic consequences, particularly for place of articulation distinctions. Here, we compare English consonants in a Mismatch Field design across two broad and distinct places of articulation - labial and coronal - and provide further evidence that early evoked auditory responses are sensitive to these features. We further add to the findings of asymmetric consonant processing, although we do not find support for coronal underspecification. Labial glides (Experiment 1) and fricatives (Experiment 2) elicited larger Mismatch responses than their coronal counterparts. Interestingly, their M100 dipoles differed along the anterior/posterior dimension in the auditory cortex that has previously been found to spatially reflect place of articulation differences. Our results are discussed with respect to acoustic and articulatory bases of featural speech sound classifications and with respect to a model that maps distinctive phonetic features onto long-term representations of speech sounds.
Collapse
Affiliation(s)
- Mathias Scharinger
- Department of Linguistics, University of Maryland, College Park, MD 20742-7505, USA.
| | | | | | | |
Collapse
|
14
|
Obleser J, Leaver AM, Vanmeter J, Rauschecker JP. Segregation of vowels and consonants in human auditory cortex: evidence for distributed hierarchical organization. Front Psychol 2010; 1:232. [PMID: 21738513 PMCID: PMC3125530 DOI: 10.3389/fpsyg.2010.00232] [Citation(s) in RCA: 40] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2010] [Accepted: 12/08/2010] [Indexed: 11/24/2022] Open
Abstract
The speech signal consists of a continuous stream of consonants and vowels, which must be de- and encoded in human auditory cortex to ensure the robust recognition and categorization of speech sounds. We used small-voxel functional magnetic resonance imaging to study information encoded in local brain activation patterns elicited by consonant-vowel syllables, and by a control set of noise bursts. First, activation of anterior–lateral superior temporal cortex was seen when controlling for unspecific acoustic processing (syllables versus band-passed noises, in a “classic” subtraction-based design). Second, a classifier algorithm, which was trained and tested iteratively on data from all subjects to discriminate local brain activation patterns, yielded separations of cortical patches discriminative of vowel category versus patches discriminative of stop-consonant category across the entire superior temporal cortex, yet with regional differences in average classification accuracy. Overlap (voxels correctly classifying both speech sound categories) was surprisingly sparse. Third, lending further plausibility to the results, classification of speech–noise differences was generally superior to speech–speech classifications, with the no\ exception of a left anterior region, where speech–speech classification accuracies were significantly better. These data demonstrate that acoustic–phonetic features are encoded in complex yet sparsely overlapping local patterns of neural activity distributed hierarchically across different regions of the auditory cortex. The redundancy apparent in these multiple patterns may partly explain the robustness of phonemic representations.
Collapse
Affiliation(s)
- Jonas Obleser
- Laboratory of Integrative Neuroscience and Cognition, Department of Physiology and Biophysics, Georgetown University Medical Center Washington, DC, USA
| | | | | | | |
Collapse
|
15
|
Sharp DJ, Awad M, Warren JE, Wise RJS, Vigliocco G, Scott SK. The neural response to changing semantic and perceptual complexity during language processing. Hum Brain Mapp 2010; 31:365-77. [PMID: 19777554 PMCID: PMC6870623 DOI: 10.1002/hbm.20871] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2009] [Revised: 06/21/2009] [Accepted: 07/05/2009] [Indexed: 11/07/2022] Open
Abstract
Speech comprehension involves processing at different levels of analysis, such as acoustic, phonetic, and lexical. We investigated neural responses to manipulating the difficulty of processing at two of these levels. Twelve subjects underwent positron emission tomographic scanning while making decisions based upon the semantic relatedness between heard nouns. We manipulated perceptual difficulty by presenting either clear or acoustically degraded speech, and semantic difficulty by varying the degree of semantic relatedness between words. Increasing perceptual difficulty was associated with greater activation of the left superior temporal gyrus, an auditory-perceptual region involved in speech processing. Increasing semantic difficulty was associated with reduced activity in both superior temporal gyri and increased activity within the left angular gyrus, a heteromodal region involved in accessing word meaning. Comparing across all the conditions, we also observed increased activation within the left inferior prefrontal cortex as the complexity of language processing increased. These results demonstrate a flexible system for language processing, where activity within distinct parts of the network is modulated as processing demands change.
Collapse
Affiliation(s)
- David J Sharp
- Imperial College London, Hammersmith Hospital Campus, United Kingdom.
| | | | | | | | | | | |
Collapse
|
16
|
Zhang Y, Kuhl PK, Imada T, Iverson P, Pruitt J, Stevens EB, Kawakatsu M, Tohkura Y, Nemoto I. Neural signatures of phonetic learning in adulthood: a magnetoencephalography study. Neuroimage 2009; 46:226-40. [PMID: 19457395 PMCID: PMC2811417 DOI: 10.1016/j.neuroimage.2009.01.028] [Citation(s) in RCA: 78] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2008] [Revised: 11/19/2008] [Accepted: 01/18/2009] [Indexed: 10/21/2022] Open
Abstract
The present study used magnetoencephalography (MEG) to examine perceptual learning of American English /r/ and /l/ categories by Japanese adults who had limited English exposure. A training software program was developed based on the principles of infant phonetic learning, featuring systematic acoustic exaggeration, multi-talker variability, visible articulation, and adaptive listening. The program was designed to help Japanese listeners utilize an acoustic dimension relevant for phonemic categorization of /r-l/ in English. Although training did not produce native-like phonetic boundary along the /r-l/ synthetic continuum in the second language learners, success was seen in highly significant identification improvement over twelve training sessions and transfer of learning to novel stimuli. Consistent with behavioral results, pre-post MEG measures showed not only enhanced neural sensitivity to the /r-l/ distinction in the left-hemisphere mismatch field (MMF) response but also bilateral decreases in equivalent current dipole (ECD) cluster and duration measures for stimulus coding in the inferior parietal region. The learning-induced increases in neural sensitivity and efficiency were also found in distributed source analysis using Minimum Current Estimates (MCE). Furthermore, the pre-post changes exhibited significant brain-behavior correlations between speech discrimination scores and MMF amplitudes as well as between the behavioral scores and ECD measures of neural efficiency. Together, the data provide corroborating evidence that substantial neural plasticity for second-language learning in adulthood can be induced with adaptive and enriched linguistic exposure. Like the MMF, the ECD cluster and duration measures are sensitive neural markers of phonetic learning.
Collapse
Affiliation(s)
- Yang Zhang
- Department of Speech-Language-Hearing Sciences and Center for Neurobehavioral Development, University of Minnesota, Minneapolis, MN 55455, USA.
| | | | | | | | | | | | | | | | | |
Collapse
|
17
|
Bonte M, Valente G, Formisano E. Dynamic and task-dependent encoding of speech and voice by phase reorganization of cortical oscillations. J Neurosci 2009; 29:1699-706. [PMID: 19211877 PMCID: PMC6666288 DOI: 10.1523/jneurosci.3694-08.2009] [Citation(s) in RCA: 36] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2008] [Revised: 01/05/2009] [Accepted: 01/07/2009] [Indexed: 11/21/2022] Open
Abstract
Speech and vocal sounds are at the core of human communication. Cortical processing of these sounds critically depends on behavioral demands. However, the neurocomputational mechanisms enabling this adaptive processing remain elusive. Here we examine the task-dependent reorganization of electroencephalographic responses to natural speech sounds (vowels /a/, /i/, /u/) spoken by three speakers (two female, one male) while listeners perform a one-back task on either vowel or speaker identity. We show that dynamic changes of sound-evoked responses and phase patterns of cortical oscillations in the alpha band (8-12 Hz) closely reflect the abstraction and analysis of the sounds along the task-relevant dimension. Vowel categorization leads to a significant temporal realignment of responses to the same vowel, e.g., /a/, independent of who pronounced this vowel, whereas speaker categorization leads to a significant temporal realignment of responses to the same speaker, e.g., speaker 1, independent of which vowel she/he pronounced. This transient and goal-dependent realignment of neuronal responses to physically different external events provides a robust cortical coding mechanism for forming and processing abstract representations of auditory (speech) input.
Collapse
Affiliation(s)
- Milene Bonte
- Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, 6200 MD Maastricht, The Netherlands.
| | | | | |
Collapse
|
18
|
Kuriki S, Ohta K, Koyama S. Persistent responsiveness of long-latency auditory cortical activities in response to repeated stimuli of musical timbre and vowel sounds. ACTA ACUST UNITED AC 2007; 17:2725-32. [PMID: 17289776 DOI: 10.1093/cercor/bhl182] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Long-latency auditory-evoked magnetic field and potential show strong attenuation of N1m/N1 responses when an identical stimulus is presented repeatedly due to adaptation of auditory cortical neurons. This adaptation is weak in subsequently occurring P2m/P2 responses, being weaker for piano chords than single piano notes. The adaptation of P2m is more suppressed in musicians having long-term musical training than in nonmusicians, whereas the amplitude of P2 is enhanced preferentially in musicians as the spectral complexity of musical tones increases. To address the key issues of whether such high responsiveness of P2m/P2 responses to complex sounds is intrinsic and common to nonmusical sounds, we conducted a magnetoencephalographic study on participants who had no experience of musical training, using consecutive trains of piano and vowel sounds. The dipole moment of the P2m sources located in the auditory cortex indicated significantly suppressed adaptation in the right hemisphere both to piano and vowel sounds. Thus, the persistent responsiveness of the P2m activity may be inherent, not induced by intensive training, and common to spectrally complex sounds. The right hemisphere dominance of the responsiveness to musical and speech sounds suggests analysis of acoustic features of object sounds to be a significant function of P2m activity.
Collapse
Affiliation(s)
- Shinya Kuriki
- Research Institute for Electronic Science, Hokkaido University, Sapporo, Japan
| | | | | |
Collapse
|
19
|
Salmelin R. Clinical neurophysiology of language: The MEG approach. Clin Neurophysiol 2007; 118:237-54. [PMID: 17008126 DOI: 10.1016/j.clinph.2006.07.316] [Citation(s) in RCA: 158] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2006] [Revised: 06/01/2006] [Accepted: 07/28/2006] [Indexed: 01/12/2023]
Abstract
Clinical evaluation of language function and basic neuroscience research into the neurophysiology of language are tied together. Whole-head MEG systems readily facilitate detailed spatiotemporal characterization of language processes. A fair amount of information is available about the cortical sequence of word perception and comprehension in the auditory and visual domain, which can be applied for clinical use. Language production remains, at present, somewhat less well charted. In clinical practice, the most obvious needs are noninvasive evaluation of the language-dominant hemisphere and mapping of areas involved in language performance to assist surgery. Multiple experimental designs and analysis approaches have been proposed for estimation of language lateralization. Some of them have been compared with the invasive Wada test and need to be tested further. Development of approaches for more comprehensive pre-surgical characterization of language cortex should build on basic neuroscience research, making use of parametric designs that allow functional mapping. Studies of the neural basis of developmental and acquired language disorders, such as dyslexia, stuttering, and aphasia can currently be regarded more as clinical or basic neuroscience research rather than as clinical routine. Such investigations may eventually provide tools for development of individually targeted training procedures and their objective evaluation.
Collapse
Affiliation(s)
- Riitta Salmelin
- Brain Research Unit, Low Temperature Laboratory, Helsinki University of Technology, Espoo, Finland.
| |
Collapse
|
20
|
Sörös P, Michael N, Tollkötter M, Pfleiderer B. The neurochemical basis of human cortical auditory processing: combining proton magnetic resonance spectroscopy and magnetoencephalography. BMC Biol 2006; 4:25. [PMID: 16884545 PMCID: PMC1553472 DOI: 10.1186/1741-7007-4-25] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2006] [Accepted: 08/03/2006] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND A combination of magnetoencephalography and proton magnetic resonance spectroscopy was used to correlate the electrophysiology of rapid auditory processing and the neurochemistry of the auditory cortex in 15 healthy adults. To assess rapid auditory processing in the left auditory cortex, the amplitude and decrement of the N1m peak, the major component of the late auditory evoked response, were measured during rapidly successive presentation of acoustic stimuli. We tested the hypothesis that: (i) the amplitude of the N1m response and (ii) its decrement during rapid stimulation are associated with the cortical neurochemistry as determined by proton magnetic resonance spectroscopy. RESULTS Our results demonstrated a significant association between the concentrations of N-acetylaspartate, a marker of neuronal integrity, and the amplitudes of individual N1m responses. In addition, the concentrations of choline-containing compounds, representing the functional integrity of membranes, were significantly associated with N1m amplitudes. No significant association was found between the concentrations of the glutamate/glutamine pool and the amplitudes of the first N1m. No significant associations were seen between the decrement of the N1m (the relative amplitude of the second N1m peak) and the concentrations of N-acetylaspartate, choline-containing compounds, or the glutamate/glutamine pool. However, there was a trend for higher glutamate/glutamine concentrations in individuals with higher relative N1m amplitude. CONCLUSION These results suggest that neuronal and membrane functions are important for rapid auditory processing. This investigation provides a first link between the electrophysiology, as recorded by magnetoencephalography, and the neurochemistry, as assessed by proton magnetic resonance spectroscopy, of the auditory cortex.
Collapse
Affiliation(s)
- Peter Sörös
- Department of Imaging Research, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Nikolaus Michael
- Department of Psychiatry, University Hospital Münster, Münster, Germany
| | - Melanie Tollkötter
- Department of Clinical Radiology, University Hospital Münster, Münster, Germany
| | - Bettina Pfleiderer
- Department of Clinical Radiology, University Hospital Münster, Münster, Germany
| |
Collapse
|
21
|
Tiitinen H, Mäkelä AM, Mäkinen V, May PJC, Alku P. Disentangling the effects of phonation and articulation: hemispheric asymmetries in the auditory N1m response of the human brain. BMC Neurosci 2005; 6:62. [PMID: 16225699 PMCID: PMC1280927 DOI: 10.1186/1471-2202-6-62] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2005] [Accepted: 10/15/2005] [Indexed: 11/16/2022] Open
Abstract
Background The cortical activity underlying the perception of vowel identity has typically been addressed by manipulating the first and second formant frequency (F1 & F2) of the speech stimuli. These two values, originating from articulation, are already sufficient for the phonetic characterization of vowel category. In the present study, we investigated how the spectral cues caused by articulation are reflected in cortical speech processing when combined with phonation, the other major part of speech production manifested as the fundamental frequency (F0) and its harmonic integer multiples. To study the combined effects of articulation and phonation we presented vowels with either high (/a/) or low (/u/) formant frequencies which were driven by three different types of excitation: a natural periodic pulseform reflecting the vibration of the vocal folds, an aperiodic noise excitation, or a tonal waveform. The auditory N1m response was recorded with whole-head magnetoencephalography (MEG) from ten human subjects in order to resolve whether brain events reflecting articulation and phonation are specific to the left or right hemisphere of the human brain. Results The N1m responses for the six stimulus types displayed a considerable dynamic range of 115–135 ms, and were elicited faster (~10 ms) by the high-formant /a/ than by the low-formant /u/, indicating an effect of articulation. While excitation type had no effect on the latency of the right-hemispheric N1m, the left-hemispheric N1m elicited by the tonally excited /a/ was some 10 ms earlier than that elicited by the periodic and the aperiodic excitation. The amplitude of the N1m in both hemispheres was systematically stronger to stimulation with natural periodic excitation. Also, stimulus type had a marked (up to 7 mm) effect on the source location of the N1m, with periodic excitation resulting in more anterior sources than aperiodic and tonal excitation. Conclusion The auditory brain areas of the two hemispheres exhibit differential tuning to natural speech signals, observable already in the passive recording condition. The variations in the latency and strength of the auditory N1m response can be traced back to the spectral structure of the stimuli. More specifically, the combined effects of the harmonic comb structure originating from the natural voice excitation caused by the fluctuating vocal folds and the location of the formant frequencies originating from the vocal tract leads to asymmetric behaviour of the left and right hemisphere.
Collapse
Affiliation(s)
- Hannu Tiitinen
- Apperception & Cortical Dynamics (ACD), Department of Psychology, P.O.B. 9, FIN-00014 University of Helsinki, Finland
- BioMag Laboratory, Engineering Centre, Helsinki University Central Hospital, Finland
| | - Anna Mari Mäkelä
- Apperception & Cortical Dynamics (ACD), Department of Psychology, P.O.B. 9, FIN-00014 University of Helsinki, Finland
- BioMag Laboratory, Engineering Centre, Helsinki University Central Hospital, Finland
| | - Ville Mäkinen
- Apperception & Cortical Dynamics (ACD), Department of Psychology, P.O.B. 9, FIN-00014 University of Helsinki, Finland
- BioMag Laboratory, Engineering Centre, Helsinki University Central Hospital, Finland
| | - Patrick JC May
- Apperception & Cortical Dynamics (ACD), Department of Psychology, P.O.B. 9, FIN-00014 University of Helsinki, Finland
- BioMag Laboratory, Engineering Centre, Helsinki University Central Hospital, Finland
| | - Paavo Alku
- Laboratory of Acoustics and Audio Signal Processing, Helsinki University of Technology, Espoo, Finland
| |
Collapse
|
22
|
Obleser J, Scott SK, Eulitz C. Now you hear it, now you don't: transient traces of consonants and their nonspeech analogues in the human brain. ACTA ACUST UNITED AC 2005; 16:1069-76. [PMID: 16207930 DOI: 10.1093/cercor/bhj047] [Citation(s) in RCA: 59] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
The apparently effortless identification of speech is one of the human auditory cortex' finest and least understood functions. This is partly due to difficulties to tease apart effects of acoustic and phonetic attributes of speech sounds. Here we present evidence from magnetic source imaging that the auditory cortex represents speech sounds (such as [g] and [t]) in a topographically orderly fashion that is based on phonetic features. Moreover, this mapping is dependent on intelligibility. Only when consonants are identifiable as members of a native speech sound category is topographical spreading out in the auditory cortex observed. Feature separation in the cortex also varies with a listener's ability to tell these easy-to-confuse consonants from one another. This is the first demonstration that speech-specific maps of features can be identified in human auditory cortex, and it will further help us to delineate speech processing pathways based on models from functional neuroimaging and non-human primates.
Collapse
Affiliation(s)
- Jonas Obleser
- Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London, WC1N 3AR, UK.
| | | | | |
Collapse
|
23
|
Coath M, Brader JM, Fusi S, Denham SL. Multiple views of the response of an ensemble of spectro-temporal features support concurrent classification of utterance, prosody, sex and speaker identity. NETWORK (BRISTOL, ENGLAND) 2005; 16:285-300. [PMID: 16411500 DOI: 10.1080/09548980500290120] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/06/2023]
Abstract
Models of auditory processing, particularly of speech, face many difficulties. These difficulties include variability among speakers, variability in speech rate and robustness to moderate distortions such as time compression. In contrast to the 'invariance of percept' (across different speakers, of different sexes, using different intonation, and so on) is the observation that we are sensitive to the identity, sex and intonation of the speaker. In previous work we have reported that a model based on ensembles of spectro-temporal feature detectors, derived from onset sensitive pre-processing of a limited class of stimuli, preserves significant information about the stimulus class. We have also shown that this is robust with respect to the exact choice of feature set, moderate time compression in the stimulus and speaker variation. Here we extend these results to show a) that by using a classifier based on a network of spiking neurons with spike-driven plasticity, the output of the ensemble constitutes an effective rate coding representation of complex sounds; and b) that the same set of spectro-temporal features concurrently preserve information about a range of qualitatively different classes into which the stimulus might fall. We show that it is possible for multiple views of the same pattern of responses to generate different percepts. This is consistent with suggestions that multiple parallel processes exist within the auditory 'what' pathway with attentional modulation enhancing the task-relevant classification type. We also show that the responses of the ensemble are sparse in the sense that a small number of features respond for each stimulus type. This has implications for the ensembles' ability to generalise, and to respond differentially to a wide variety of stimulus classes.
Collapse
Affiliation(s)
- M Coath
- Centre for Theoretical and Computational Neuroscience, University of Plymouth, Plymouth, UK.
| | | | | | | |
Collapse
|
24
|
Bonte M, Parviainen T, Hytönen K, Salmelin R. Time course of top-down and bottom-up influences on syllable processing in the auditory cortex. ACTA ACUST UNITED AC 2005; 16:115-23. [PMID: 15829731 DOI: 10.1093/cercor/bhi091] [Citation(s) in RCA: 56] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
In speech perception, extraction of meaning from complex streams of sounds is surprisingly fast and efficient. By tracking the neural time course of syllable processing with magnetoencephalography we show that this continuous construction of meaning-based representations is aided by both top-down (context-based) expectations and bottom-up (acoustic-phonetic) cues in the speech signal. Syllables elicited a sustained response at 200-600 ms (N400m) which became most similar to that evoked by words when the expectation for meaningful speech was increased by presenting the syllables among words and sentences or using sentence-initial syllables. This word-like cortical processing of meaningless syllables emerged at the build-up of the N400m response, 200-300 ms after speech onset, during the transition from perceptual to lexical-semantic analysis. These findings show that the efficiency of meaning-based analysis of speech is subserved by a cortical system finely tuned to lexically relevant acoustic-phonetic and contextual cues.
Collapse
Affiliation(s)
- Milene Bonte
- Department of Cognitive Neuroscience, Faculty of Psychology, University of Maastricht, PO Box 616, 6200 MD, Maastricht, The Netherlands.
| | | | | | | |
Collapse
|