1
|
Kolozsvári OB, Xu W, Leppänen PHT, Hämäläinen JA. Top-Down Predictions of Familiarity and Congruency in Audio-Visual Speech Perception at Neural Level. Front Hum Neurosci 2019; 13:243. [PMID: 31354459 PMCID: PMC6639789 DOI: 10.3389/fnhum.2019.00243] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2019] [Accepted: 06/28/2019] [Indexed: 11/13/2022] Open
Abstract
During speech perception, listeners rely on multimodal input and make use of both auditory and visual information. When presented with speech, for example syllables, the differences in brain responses to distinct stimuli are not, however, caused merely by the acoustic or visual features of the stimuli. The congruency of the auditory and visual information and the familiarity of a syllable, that is, whether it appears in the listener's native language or not, also modulates brain responses. We investigated how the congruency and familiarity of the presented stimuli affect brain responses to audio-visual (AV) speech in 12 adult Finnish native speakers and 12 adult Chinese native speakers. They watched videos of a Chinese speaker pronouncing syllables (/pa/, /pha/, /ta/, /tha/, /fa/) during a magnetoencephalography (MEG) measurement where only /pa/ and /ta/ were part of Finnish phonology while all the stimuli were part of Chinese phonology. The stimuli were presented in audio-visual (congruent or incongruent), audio only, or visual only conditions. The brain responses were examined in five time-windows: 75-125, 150-200, 200-300, 300-400, and 400-600 ms. We found significant differences for the congruency comparison in the fourth time-window (300-400 ms) in both sensor and source level analysis. Larger responses were observed for the incongruent stimuli than for the congruent stimuli. For the familiarity comparisons no significant differences were found. The results are in line with earlier studies reporting on the modulation of brain responses for audio-visual congruency around 250-500 ms. This suggests a much stronger process for the general detection of a mismatch between predictions based on lip movements and the auditory signal than for the top-down modulation of brain responses based on phonological information.
Collapse
Affiliation(s)
- Orsolya B Kolozsvári
- Department of Psychology, University of Jyväskylä, Jyväskylä, Finland.,Jyväskylä Centre for Interdisciplinary Brain Research (CIBR), University of Jyväskylä, Jyväskylä, Finland
| | - Weiyong Xu
- Department of Psychology, University of Jyväskylä, Jyväskylä, Finland.,Jyväskylä Centre for Interdisciplinary Brain Research (CIBR), University of Jyväskylä, Jyväskylä, Finland
| | - Paavo H T Leppänen
- Department of Psychology, University of Jyväskylä, Jyväskylä, Finland.,Jyväskylä Centre for Interdisciplinary Brain Research (CIBR), University of Jyväskylä, Jyväskylä, Finland
| | - Jarmo A Hämäläinen
- Department of Psychology, University of Jyväskylä, Jyväskylä, Finland.,Jyväskylä Centre for Interdisciplinary Brain Research (CIBR), University of Jyväskylä, Jyväskylä, Finland
| |
Collapse
|
2
|
Liebenthal E, Möttönen R. An interactive model of auditory-motor speech perception. BRAIN AND LANGUAGE 2018; 187:33-40. [PMID: 29268943 PMCID: PMC6005717 DOI: 10.1016/j.bandl.2017.12.004] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/05/2017] [Revised: 10/03/2017] [Accepted: 12/02/2017] [Indexed: 05/30/2023]
Abstract
Mounting evidence indicates a role in perceptual decoding of speech for the dorsal auditory stream connecting between temporal auditory and frontal-parietal articulatory areas. The activation time course in auditory, somatosensory and motor regions during speech processing is seldom taken into account in models of speech perception. We critically review the literature with a focus on temporal information, and contrast between three alternative models of auditory-motor speech processing: parallel, hierarchical, and interactive. We argue that electrophysiological and transcranial magnetic stimulation studies support the interactive model. The findings reveal that auditory and somatomotor areas are engaged almost simultaneously, before 100 ms. There is also evidence of early interactions between auditory and motor areas. We propose a new interactive model of auditory-motor speech perception in which auditory and articulatory somatomotor areas are connected from early stages of speech processing. We also discuss how attention and other factors can affect the timing and strength of auditory-motor interactions and propose directions for future research.
Collapse
Affiliation(s)
- Einat Liebenthal
- Department of Psychiatry, Brigham & Women's Hospital, Harvard Medical School, Boston, USA.
| | - Riikka Möttönen
- Department of Experimental Psychology, University of Oxford, Oxford, UK; School of Psychology, University of Nottingham, Nottingham, UK
| |
Collapse
|
3
|
Zhang X, Li X, Chen J, Gong Q. Background Suppression and its Relation to Foreground Processing of Speech Versus Non-speech Streams. Neuroscience 2018; 373:60-71. [PMID: 29337239 DOI: 10.1016/j.neuroscience.2018.01.009] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2017] [Revised: 01/02/2018] [Accepted: 01/03/2018] [Indexed: 10/18/2022]
Abstract
Since sound perception takes place against a background with a certain amount of noise, both speech and non-speech processing involve extraction of target signals and suppression of background noise. Previous works on early processing of speech phonemes largely neglected how background noise is encoded and suppressed. This study aimed to fill in this gap. We adopted an oddball paradigm where speech (vowels) or non-speech stimuli (complex tones) were presented with or without a background of amplitude-modulated noise and analyzed cortical responses related to foreground stimulus processing, including mismatch negativity (MMN), N2b, and P300, as well as neural representations of the background noise, that is, auditory steady-state response (ASSR). We found that speech deviants elicited later and weaker MMN, later N2b, and later P300 than non-speech ones, but N2b and P300 had similar strength, suggesting more complex processing of certain acoustic features in speech. Only for vowels, background noise enhanced N2b strength relative to silence, suggesting an attention-related speech-specific process to improve perception of foreground targets. In addition, noise suppression in speech contexts, quantified by ASSR amplitude reduction after stimulus onset, was lateralized towards the left hemisphere. The left-lateralized suppression following N2b was associated with the N2b enhancement in noise for speech, indicating that foreground processing may interact with background suppression, particularly during speech processing. Together, our findings indicate that the differences between perception of speech and non-speech sounds involve not only the processing of target information in the foreground but also the suppression of irrelevant aspects in the background.
Collapse
Affiliation(s)
- Xiaochen Zhang
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Xiaolin Li
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Jingjing Chen
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China
| | - Qin Gong
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing, China; Research Center of Biomedical Engineering, Graduate School at Shenzhen, Tsinghua University, Shenzhen, Guangdong Province, China.
| |
Collapse
|
4
|
Kawase T, Yahata I, Kanno A, Sakamoto S, Takanashi Y, Takata S, Nakasato N, Kawashima R, Katori Y. Impact of Audio-Visual Asynchrony on Lip-Reading Effects -Neuromagnetic and Psychophysical Study. PLoS One 2016; 11:e0168740. [PMID: 28030631 PMCID: PMC5193434 DOI: 10.1371/journal.pone.0168740] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2016] [Accepted: 12/05/2016] [Indexed: 11/18/2022] Open
Abstract
The effects of asynchrony between audio and visual (A/V) stimuli on the N100m responses of magnetoencephalography in the left hemisphere were compared with those on the psychophysical responses in 11 participants. The latency and amplitude of N100m were significantly shortened and reduced in the left hemisphere by the presentation of visual speech as long as the temporal asynchrony between A/V stimuli was within 100 ms, but were not significantly affected with audio lags of -500 and +500 ms. However, some small effects were still preserved on average with audio lags of 500 ms, suggesting similar asymmetry of the temporal window to that observed in psychophysical measurements, which tended to be more robust (wider) for audio lags; i.e., the pattern of visual-speech effects as a function of A/V lag observed in the N100m in the left hemisphere grossly resembled that in psychophysical measurements on average, although the individual responses were somewhat varied. The present results suggest that the basic configuration of the temporal window of visual effects on auditory-speech perception could be observed from the early auditory processing stage.
Collapse
Affiliation(s)
- Tetsuaki Kawase
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
- Laboratory of Rehabilitative Auditory Science, Tohoku University Graduate School of Biomedical Engineering, Sendai, Miyagi, Japan
- Department of Audiology, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
- * E-mail:
| | - Izumi Yahata
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Akitake Kanno
- Department of Functional Brain Imaging, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Miyagi, Japan
| | - Shuichi Sakamoto
- Research Institute of Electrical Communication, Tohoku University, Sendai, Miyagi, Japan
| | - Yoshitaka Takanashi
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Shiho Takata
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Nobukazu Nakasato
- Department of Epileptology, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| | - Ryuta Kawashima
- Department of Functional Brain Imaging, Institute of Development, Aging and Cancer, Tohoku University, Sendai, Miyagi, Japan
| | - Yukio Katori
- Department of Otolaryngology-Head and Neck Surgery, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan
| |
Collapse
|
5
|
Almeida D, Poeppel D, Corina D. The Processing of Biologically Plausible and Implausible forms in American Sign Language: Evidence for Perceptual Tuning. LANGUAGE, COGNITION AND NEUROSCIENCE 2015; 31:361-374. [PMID: 27135041 PMCID: PMC4849140 DOI: 10.1080/23273798.2015.1100315] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/18/2015] [Accepted: 09/16/2015] [Indexed: 05/29/2023]
Abstract
The human auditory system distinguishes speech-like information from general auditory signals in a remarkably fast and efficient way. Combining psychophysics and neurophysiology (MEG), we demonstrate a similar result for the processing of visual information used for language communication in users of sign languages. We demonstrate that the earliest visual cortical responses in deaf signers viewing American Sign Language (ASL) signs show specific modulations to violations of anatomic constraints that would make the sign either possible or impossible to articulate. These neural data are accompanied with a significantly increased perceptual sensitivity to the anatomical incongruity. The differential effects in the early visual evoked potentials arguably reflect an expectation-driven assessment of somatic representational integrity, suggesting that language experience and/or auditory deprivation may shape the neuronal mechanisms underlying the analysis of complex human form. The data demonstrate that the perceptual tuning that underlies the discrimination of language and non-language information is not limited to spoken languages but extends to languages expressed in the visual modality.
Collapse
Affiliation(s)
- Diogo Almeida
- Division of Sciences, Psychology program, New York University – Abu Dhabi, Abu Dhabi, UAE
| | - David Poeppel
- Department of Psychology, New York University, New York, NY, USA
- Department of Neuroscience, Max-Planck-Institute (MPIEA), Frankfurt, Germany
| | - David Corina
- Department of Linguistics and the Center for Mind and Brain, University of California, Davis, CA, USA
| |
Collapse
|
6
|
Abstract
The motor regions that control movements of the articulators activate during listening to speech and contribute to performance in demanding speech recognition and discrimination tasks. Whether the articulatory motor cortex modulates auditory processing of speech sounds is unknown. Here, we aimed to determine whether the articulatory motor cortex affects the auditory mechanisms underlying discrimination of speech sounds in the absence of demanding speech tasks. Using electroencephalography, we recorded responses to changes in sound sequences, while participants watched a silent video. We also disrupted the lip or the hand representation in left motor cortex using transcranial magnetic stimulation. Disruption of the lip representation suppressed responses to changes in speech sounds, but not piano tones. In contrast, disruption of the hand representation had no effect on responses to changes in speech sounds. These findings show that disruptions within, but not outside, the articulatory motor cortex impair automatic auditory discrimination of speech sounds. The findings provide evidence for the importance of auditory-motor processes in efficient neural analysis of speech sounds.
Collapse
Affiliation(s)
- Riikka Möttönen
- Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford OX1 3UD, UK.
| | | | | |
Collapse
|
7
|
Parviainen T, Helenius P, Poskiparta E, Niemi P, Salmelin R. Speech perception in the child brain: cortical timing and its relevance to literacy acquisition. Hum Brain Mapp 2011; 32:2193-206. [PMID: 21391257 DOI: 10.1002/hbm.21181] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2010] [Revised: 08/18/2010] [Accepted: 09/09/2010] [Indexed: 11/11/2022] Open
Abstract
Speech processing skills go through intensive development during mid-childhood, providing basis also for literacy acquisition. The sequence of auditory cortical processing of speech has been characterized in adults, but very little is known about the neural representation of speech sound perception in the developing brain. We used whole-head magnetoencephalography (MEG) to record neural responses to speech and nonspeech sounds in first-graders (7-8-year-old) and compared the activation sequence to that in adults. In children, the general location of neural activity in the superior temporal cortex was similar to that in adults, but in the time domain the sequence of activation was strikingly different. Cortical differentiation between sound types emerged in a prolonged response pattern at about 250 ms after sound onset, in both hemispheres, clearly later than the corresponding effect at about 100 ms in adults that was detected specifically in the left hemisphere. Better reading skills were linked with shorter-lasting neural activation, speaking for interdependence of the maturing neural processes of auditory perception and developing linguistic skills. This study uniquely utilized the potential of MEG in comparing both spatial and temporal characteristics of neural activation between adults and children. Besides depicting the group-typical features in cortical auditory processing, the results revealed marked interindividual variability in children.
Collapse
Affiliation(s)
- Tiina Parviainen
- Brain Research Unit, Low Temperature Laboratory, Aalto University School of Science and Technology, Espoo, Finland.
| | | | | | | | | |
Collapse
|
8
|
The analysis of simple and complex auditory signals in human auditory cortex: magnetoencephalographic evidence from M100 modulation. Ear Hear 2010; 31:515-26. [PMID: 20445455 DOI: 10.1097/aud.0b013e3181d99a75] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Ecologically valid signals (e.g., vowels) have multiple components of substantially different frequencies and amplitudes that may not be equally cortically represented. In this study, we investigate a relatively simple signal at an intermediate level of complexity, two-frequency composite tones, a stimulus lying between simple sinusoids and ecologically valid signals such as speech. We aim to characterize the cortical response properties to better understand how complex signals may be represented in auditory cortex. DESIGN Using magnetoencephalography, we assessed the sensitivity of the M100/N100m auditory-evoked component to manipulations of the power ratio of the individual frequency components of the two-frequency complexes. Fourteen right-handed subjects with normal hearing were scanned while passively listening to 10 complex and 12 simple signals. The complex signals were composed of one higher frequency and one lower frequency sinusoid; the lower frequency sinusoidal component was at one of the five loudness levels relative to the higher frequency one: -20, -10, 0, +10, +20 dB. The simple signals comprised all the complex signal components presented in isolation. RESULTS The data replicate and extend several previous findings: (1) the systematic dependence of the M100 latency on signal intensity and (2) the dependence of the M100 latency on signal frequency, with lower frequency signals ( approximately 100 Hz) exhibiting longer latencies than higher frequency signals ( approximately 1000 Hz) even at matched loudness levels. (3) Importantly, we observe that, relative to simple signals, complex signals show increased response amplitude-as one might predict-but decreased M100 latencies. CONCLUSION : The data suggest that by the time the M100 is generated in auditory cortex ( approximately 70 to 80 msecs after stimulus onset), integrative processing across frequency channels has taken place which is observable in the M100 modulation. In light of these data models that attribute more time and processing resources to a complex stimulus merit reevaluation, in that our data show that acoustically more complex signals are associated with robust temporal facilitation, across frequencies and signal amplitude level.
Collapse
|
9
|
Changing meaning causes coupling changes within higher levels of the cortical hierarchy. Proc Natl Acad Sci U S A 2009; 106:11765-70. [PMID: 19553207 DOI: 10.1073/pnas.0811402106] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
Processing of speech and nonspeech sounds occurs bilaterally within primary auditory cortex and surrounding regions of the superior temporal gyrus; however, the manner in which these regions interact during speech and nonspeech processing is not well understood. Here, we investigate the underlying neuronal architecture of the auditory system with magnetoencephalography and a mismatch paradigm. We used a spoken word as a repeating "standard" and periodically introduced 3 "oddball" stimuli that differed in the frequency spectrum of the word's vowel. The closest deviant was perceived as the same vowel as the standard, whereas the other 2 deviants were perceived as belonging to different vowel categories. The neuronal responses to these vowel stimuli were compared with responses elicited by perceptually matched tone stimuli under the same paradigm. For both speech and tones, deviant stimuli induced coupling changes within the same bilateral temporal lobe system. However, vowel oddball effects increased coupling within the left posterior superior temporal gyrus, whereas perceptually equivalent nonspeech oddball effects increased coupling within the right primary auditory cortex. Thus, we show a dissociation in neuronal interactions, occurring at both different hierarchal levels of the auditory system (superior temporal versus primary auditory cortex) and in different hemispheres (left versus right). This hierarchical specificity depends on whether auditory stimuli are embedded in a perceptual context (i.e., a word). Furthermore, our lateralization results suggest left hemisphere specificity for the processing of phonological stimuli, regardless of their elemental (i.e., spectrotemporal) characteristics.
Collapse
|
10
|
Roberts TPL, Schmidt GL, Egeth M, Blaskey L, Rey MM, Edgar JC, Levy SE. Electrophysiological signatures: magnetoencephalographic studies of the neural correlates of language impairment in autism spectrum disorders. Int J Psychophysiol 2008; 68:149-60. [PMID: 18336941 DOI: 10.1016/j.ijpsycho.2008.01.012] [Citation(s) in RCA: 40] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2007] [Revised: 10/22/2007] [Accepted: 11/06/2007] [Indexed: 10/22/2022]
Abstract
While magnetoencephalography (MEG) is of increasing utility in the assessment of pediatric patients with seizure disorders, this reflects only a part of the clinical potential of the technology. Beyond epilepsy, a broad range of developmental psychiatric disorders require the spatial and temporal resolution of brain activity offered by MEG. This article reviews the application of MEG in the study of auditory processing as an aspect of language impairment in children. Specifically, the potential application of MEG is elaborated in autism spectrum disorders (ASD), a devastating disorder with prevalence of 1 in 150. Results demonstrate the sensitivity of MEG for detection of abnormalities of auditory processing in ASD ('electrophysiological signatures') and their clinical correlates. These findings offer promise for the comprehensive assessment of developmental neuropsychiatric disorders.
Collapse
Affiliation(s)
- Timothy P L Roberts
- Department of Radiology, Children's Hospital of Philadelphia, Philadelphia, PA 19104, United States.
| | | | | | | | | | | | | |
Collapse
|
11
|
The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clin Neurophysiol 2007; 118:2544-90. [PMID: 17931964 DOI: 10.1016/j.clinph.2007.04.026] [Citation(s) in RCA: 1693] [Impact Index Per Article: 99.6] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2006] [Revised: 04/18/2007] [Accepted: 04/28/2007] [Indexed: 11/22/2022]
Abstract
In the present article, the basic research using the mismatch negativity (MMN) and analogous results obtained by using the magnetoencephalography (MEG) and other brain-imaging technologies is reviewed. This response is elicited by any discriminable change in auditory stimulation but recent studies extended the notion of the MMN even to higher-order cognitive processes such as those involving grammar and semantic meaning. Moreover, MMN data also show the presence of automatic intelligent processes such as stimulus anticipation at the level of auditory cortex. In addition, the MMN enables one to establish the brain processes underlying the initiation of attention switch to, conscious perception of, sound change in an unattended stimulus stream.
Collapse
|
12
|
Uusvuori J, Parviainen T, Inkinen M, Salmelin R. Spatiotemporal Interaction between Sound Form and Meaning during Spoken Word Perception. Cereb Cortex 2007; 18:456-66. [PMID: 17566066 DOI: 10.1093/cercor/bhm076] [Citation(s) in RCA: 36] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Cortical dynamics of spoken word perception is not well understood. The possible interplay between analysis of sound form and meaning, in particular, remains elusive. We used magnetoencephalography to study cortical manifestation of phonological and semantic priming. Ten subjects listened to lists of 4 words. The first 3 words set a semantic or phonological context, and the list-final word was congruent or incongruent with this context. Attenuation of activation by priming during the first 3 words and increase of activation to semantic or phonological mismatch in the list-final word provided converging evidence: The superior temporal cortex bilaterally was involved in both analysis of sound form and meaning but the role of each hemisphere varied over time. Sensitivity to sound form was observed at approximately 100 ms after word onset, followed by sensitivity to semantic aspects from approximately 250 ms onwards, in the left hemisphere. From approximately 450 ms onwards, the picture was changed, with semantic effects now present bilaterally, accompanied by a subtle late effect of sound form in the right hemisphere. Present MEG data provide a detailed spatiotemporal account of neural mechanisms during speech perception that may underlie characterizations obtained with other neuroimaging methods less sensitive in temporal or spatial domain.
Collapse
Affiliation(s)
- Johanna Uusvuori
- Brain Research Unit, Low Temperature Laboratory, Helsinki University of Technology, Espoo 02015 TKK, Finland.
| | | | | | | |
Collapse
|
13
|
Salmelin R. Clinical neurophysiology of language: The MEG approach. Clin Neurophysiol 2007; 118:237-54. [PMID: 17008126 DOI: 10.1016/j.clinph.2006.07.316] [Citation(s) in RCA: 158] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2006] [Revised: 06/01/2006] [Accepted: 07/28/2006] [Indexed: 01/12/2023]
Abstract
Clinical evaluation of language function and basic neuroscience research into the neurophysiology of language are tied together. Whole-head MEG systems readily facilitate detailed spatiotemporal characterization of language processes. A fair amount of information is available about the cortical sequence of word perception and comprehension in the auditory and visual domain, which can be applied for clinical use. Language production remains, at present, somewhat less well charted. In clinical practice, the most obvious needs are noninvasive evaluation of the language-dominant hemisphere and mapping of areas involved in language performance to assist surgery. Multiple experimental designs and analysis approaches have been proposed for estimation of language lateralization. Some of them have been compared with the invasive Wada test and need to be tested further. Development of approaches for more comprehensive pre-surgical characterization of language cortex should build on basic neuroscience research, making use of parametric designs that allow functional mapping. Studies of the neural basis of developmental and acquired language disorders, such as dyslexia, stuttering, and aphasia can currently be regarded more as clinical or basic neuroscience research rather than as clinical routine. Such investigations may eventually provide tools for development of individually targeted training procedures and their objective evaluation.
Collapse
Affiliation(s)
- Riitta Salmelin
- Brain Research Unit, Low Temperature Laboratory, Helsinki University of Technology, Espoo, Finland.
| |
Collapse
|
14
|
Oram Cardy JE, Flagg EJ, Roberts W, Roberts TPL. Delayed mismatch field for speech and non-speech sounds in children with autism. Neuroreport 2005; 16:521-5. [PMID: 15770164 DOI: 10.1097/00001756-200504040-00021] [Citation(s) in RCA: 79] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
This study investigated the magnetic mismatch field elicited by changes in streams of vowels or spectrally matched tones in children with autism spectrum disorder (ASD) relative to children with typical development to explore whether impaired sound discrimination may contribute to language impairments in autism spectrum disorder. Using magnetoencephalography, we recorded evoked neural activity to 300-Hz and 700-Hz tones (and /u/ and /a/ vowels) presented in an oddball paradigm with deviant stimuli (15%) occurring within a train of standards (85%). The magnetic mismatch field was robustly observed in both groups, but children with autism spectrum disorder demonstrated a significantly delayed magnetic mismatch field compared with typically developing peers. Difficulty parsing transient differences in sounds may lead to impaired acoustic or phonological representations and subsequent language impairment in autism spectrum disorder.
Collapse
Affiliation(s)
- Janis E Oram Cardy
- Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada.
| | | | | | | |
Collapse
|
15
|
Bonte M, Parviainen T, Hytönen K, Salmelin R. Time course of top-down and bottom-up influences on syllable processing in the auditory cortex. ACTA ACUST UNITED AC 2005; 16:115-23. [PMID: 15829731 DOI: 10.1093/cercor/bhi091] [Citation(s) in RCA: 56] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
In speech perception, extraction of meaning from complex streams of sounds is surprisingly fast and efficient. By tracking the neural time course of syllable processing with magnetoencephalography we show that this continuous construction of meaning-based representations is aided by both top-down (context-based) expectations and bottom-up (acoustic-phonetic) cues in the speech signal. Syllables elicited a sustained response at 200-600 ms (N400m) which became most similar to that evoked by words when the expectation for meaningful speech was increased by presenting the syllables among words and sentences or using sentence-initial syllables. This word-like cortical processing of meaningless syllables emerged at the build-up of the N400m response, 200-300 ms after speech onset, during the transition from perceptual to lexical-semantic analysis. These findings show that the efficiency of meaning-based analysis of speech is subserved by a cortical system finely tuned to lexically relevant acoustic-phonetic and contextual cues.
Collapse
Affiliation(s)
- Milene Bonte
- Department of Cognitive Neuroscience, Faculty of Psychology, University of Maastricht, PO Box 616, 6200 MD, Maastricht, The Netherlands.
| | | | | | | |
Collapse
|
16
|
Lin YY, Chen WT, Liao KK, Yeh TC, Wu ZA, Ho LT. Hemispheric balance in coding speech and non-speech sounds in Chinese participants. Neuroreport 2005; 16:469-73. [PMID: 15770153 DOI: 10.1097/00001756-200504040-00010] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
To study the role of neuromagnetic auditory approximately 100-ms responses (N100m) in phonetic processing, we recorded N100m in 24 right-handed Chinese participants using a whole-head neuromagnetometer. The stimuli included vowel /a/ and consonant-vowels /ba/ and /da/, spoken by one Chinese speaker, and a 1-kHz tone. N100m to tones was larger in the right hemisphere, whereas that to speech sounds was bilaterally similar. The amplitude ratio of speech to non-speech N100m was larger in the left hemisphere. N100m dipoles in the left hemisphere were approximately 2 mm more anterior for speech than for tone stimuli. The results suggest that N100m reflects both acoustics and phonetic processing. Moreover, the ratio of speech to non-speech activation in individual hemispheres may be useful for language lateralization.
Collapse
Affiliation(s)
- Yung-Yang Lin
- Department of Medical Research and Education, Taipei Veterans General Hospital, Taipei, Taiwan.
| | | | | | | | | | | |
Collapse
|
17
|
Parviainen T, Helenius P, Salmelin R. Cortical differentiation of speech and nonspeech sounds at 100 ms: implications for dyslexia. ACTA ACUST UNITED AC 2004; 15:1054-63. [PMID: 15563727 DOI: 10.1093/cercor/bhh206] [Citation(s) in RCA: 53] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Neurophysiological measures indicate cortical sensitivity to speech sounds by 150 ms after stimulus onset. In this time window dyslexic subjects start to show abnormal cortical processing. We investigated whether phonetic analysis is reflected in the robust auditory cortical activation at approximately 100 ms (N100m), and whether dyslexic subjects show abnormal N100m responses to speech or nonspeech sounds. We used magnetoencephalography to record auditory responses of 10 normally reading and 10 dyslexic adults. The speech stimuli were synthetic Finnish speech sounds (/a/, /u/, /pa/, /ka/). The nonspeech stimuli were complex nonspeech sounds and simple sine wave tones, composed of the F1+F2+F3 and F2 formant frequencies of the speech sounds, respectively. All sounds evoked a prominent N100m response in the bilateral auditory cortices. The N100m activation was stronger to speech than nonspeech sounds in the left but not in the right auditory cortex, in both subject groups. The leftward shift of hemispheric balance for speech sounds is likely to reflect analysis at the phonetic level. In dyslexic subjects the overall interhemispheric amplitude balance and timing were altered for all sound types alike. Dyslexic individuals thus seem to have an unusual cortical organization of general auditory processing in the time window of speech-sensitive analysis.
Collapse
Affiliation(s)
- Tiina Parviainen
- Brain Research Unit, Low Temperature Laboratory, Helsinki University of Technology, Espoo, Finland.
| | | | | |
Collapse
|
18
|
Eulitz C, Obleser J, Lahiri A. Intra-subject replication of brain magnetic activity during the processing of speech sounds. ACTA ACUST UNITED AC 2004; 19:82-91. [PMID: 14972361 DOI: 10.1016/j.cogbrainres.2003.11.004] [Citation(s) in RCA: 18] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/18/2003] [Indexed: 11/20/2022]
Abstract
The present study examined the cortical activity during processing of vocalic segments by means of whole-head magnetoencephalography (MEG) to see whether respective cortical maps are stable across repeated measurements. We investigated the spatial configuration and temporal characteristics of the N100m generators of the auditory-evoked field during the processing of the synthetic German vowels [a], [e] and [i] across 10 repeated measurements in a single subject. Between vowels, N100m latency as well as source location differences were found with the latency differences being in accordance with tonochronic principles. The spatial configuration of the different vowel sources was related to differences in acoustic/phonological features. Vowels differing maximally in those features, i.e., [a] and [i], showed larger Euclidean distances between N100m vowel sources than [e] and [i]. This pattern was repeatable across sessions and independent of the source modeling strategy for left-hemispheric data. Compared to a pure tone control condition, the N100m generators of vowels were localized in more anterior, superior and lateral parts of the temporal lobe and showed longer latencies. Being aware of the limited significance of conclusions drawn from a single case study, the study yielded a repeatable spatial and temporal pattern of vowel source activity in the auditory cortex which was determined by the distinctiveness of the formant frequencies corresponding to abstract phonological features.
Collapse
Affiliation(s)
- Carsten Eulitz
- Department of Clinical Psychology, University of Konstanz, Giessberg 10, P.O. Box D25, 78457 Konstanz, Germany.
| | | | | |
Collapse
|
19
|
Obleser J, Lahiri A, Eulitz C. Magnetic Brain Response Mirrors Extraction of Phonological Features from Spoken Vowels. J Cogn Neurosci 2004; 16:31-9. [PMID: 15006034 DOI: 10.1162/089892904322755539] [Citation(s) in RCA: 103] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
This study further elucidates determinants of vowel perception in the human auditory cortex. The vowel inventory of a given language can be classified on the basis of phonological features which are closely linked to acoustic properties. A cortical representation of speech sounds based on these phonological features might explain the surprisingly inverse correlation between immense variance in the acoustic signal and high accuracy of speech recognition. We investigated timing and mapping of the N100m elicited by 42 tokens of seven natural German vowels varying along the phonological features tongue height (corresponding to the frequency of the first formant) and place of articulation (corresponding to the frequency of the second and third formants). Auditoryevoked fields were recorded using a 148-channel whole-head magnetometer while subjects performed target vowel detection tasks. Source location differences appeared to be driven by place of articulation: Vowels with mutually exclusive place of articulation features, namely, coronal and dorsal elicited separate centers of activation along the posterior-anterior axis. Additionally, the time course of activation as reflected in the N100m peak latency distinguished between vowel categories especially when the spatial distinctiveness of cortical activation was low. In sum, results suggest that both N100m latency and source location as well as their interaction reflect properties of speech stimuli that correspond to abstract phonological features.
Collapse
Affiliation(s)
- Jonas Obleser
- University of Konstanz, Universitätstrasse 10, PO Box D25, 78457 Konstanz, Germany.
| | | | | |
Collapse
|
20
|
Lattner S, Maess B, Wang Y, Schauer M, Alter K, Friederici AD. Dissociation of human and computer voices in the brain: evidence for a preattentive gestalt-like perception. Hum Brain Mapp 2003; 20:13-21. [PMID: 12953302 PMCID: PMC6871997 DOI: 10.1002/hbm.10118] [Citation(s) in RCA: 19] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
We investigated the early ("preattentive") cortical processing of voice information, using the so-called "mismatch response". This brain potential allows inferences to be made about the sensory short-term store. Most importantly, the mismatch potential also provides information about the organization of long-term memory traces in the auditory system. Such traces have reliably been reported for phonemes. However, it is unclear whether they also exist for human voice information. To explore this issue, 10 healthy subjects were presented with a single word stimulus uttered by voices of different prototypicality (natural, manipulated, synthetic) in a mismatch experiment (stimulus duration 380 msec, onset-to-onset interval 900 msec). The event-related magnetic fields were recorded by a 148-channel whole-head magnetometer and a source current density modeling of the magnetic field data was performed using a minimum-norm estimate. Each deviating voice signal in a series of standard-voice stimuli evoked a mismatch response that was localized in temporal brain regions bilaterally. Increased mismatch related magnetic flux was observed in response to decreased prototypicality of a presented voice signal, but did not correspond to the acoustic similarity of standard voice and deviant voices. We, therefore, conclude that the mismatch activation predominantly reflects the ecological validity of the voice signals. We further demonstrate that the findings cannot be explained by mere acoustic feature processing, but rather point towards a holistic mapping of the incoming voice signal onto long-term representations in the auditory memory.
Collapse
Affiliation(s)
- Sonja Lattner
- Max‐Planck‐Institute of Cognitive Neuroscience, Leipzig, Germany
| | - Burkhard Maess
- Max‐Planck‐Institute of Cognitive Neuroscience, Leipzig, Germany
| | - Yunhua Wang
- Max‐Planck‐Institute of Cognitive Neuroscience, Leipzig, Germany
| | - Michael Schauer
- Max‐Planck‐Institute of Cognitive Neuroscience, Leipzig, Germany
| | - Kai Alter
- Max‐Planck‐Institute of Cognitive Neuroscience, Leipzig, Germany
| | | |
Collapse
|
21
|
Vihla M, Salmelin R. Hemispheric balance in processing attended and non-attended vowels and complex tones. BRAIN RESEARCH. COGNITIVE BRAIN RESEARCH 2003; 16:167-73. [PMID: 12668224 DOI: 10.1016/s0926-6410(02)00248-3] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
We compared cortical processing of attended and non-attended vowels and complex tones, using a whole-head neuromagnetometer, to test for possible hemispheric differences. Stimuli included vowels [a] and [i], spoken by two female Finnish speakers, and two complex tones, each with two pure tone components corresponding to the first and second formant frequencies (F1-F2) of the vowels spoken by speaker 1. Sequences including both vowels and complex tones were presented to eight Finnish males during passive and active (phoneme/speaker/complex tone identification) listening. Sequences including only vowels were presented to five of the subjects during passive listening and during a phoneme identification task. The vowel [i] spoken by speaker 1 and the corresponding complex tone were frequent, non-target stimuli. Responses evoked by these frequent stimuli were analyzed. Cortical activation at approximately 100 ms was stronger for the complex tone than the vowel in the right hemisphere (RH). Responses were similar during active and passive listening. Hemispheric balance remained the same when the vowel was presented in sequences including only vowels. The reduction of RH activation for vowels as compared with complex tones indicates a relative increase of left hemisphere involvement, possibly reflecting a shift towards more language-specific processing.
Collapse
Affiliation(s)
- Minna Vihla
- Brain Research Unit, Low Temperature Laboratory, Helsinki University of Technology, PO Box 2200, 02015 HUT Espoo, Finland.
| | | |
Collapse
|
22
|
Mäkelä AM, Alku P, Mäkinen V, Valtonen J, May P, Tiitinen H. Human cortical dynamics determined by speech fundamental frequency. Neuroimage 2002; 17:1300-5. [PMID: 12414269 DOI: 10.1006/nimg.2002.1279] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Evidence for speech-specific brain processes has been searched for through the manipulation of formant frequencies which mediate phonetic content and which are, in evolutionary terms, relatively "new" aspects of speech. Here we used whole-head magnetoencephalography and advanced stimulus reproduction methodology to examine the contribution of the fundamental frequency F0 and its harmonic integer multiples in cortical processing. The subjects were presented with a vowel, a frequency-matched counterpart of the vowel lacking in phonetic contents, and a pure tone. The F0 of the stimuli was set at that of a typical male (i.e., 100 Hz), female (200 Hz), or infant (270 Hz) speaker. We found that speech sounds, both with and without phonetic content, elicited the N1m response in human auditory cortex at a constant latency of 120 ms, whereas pure tones matching the speech sounds in frequency, intensity, and duration gave rise to N1m responses whose latency varied between 120 and 160 ms. Thus, it seems that the fundamental frequency F0 and its harmonics determine the temporal dynamics of speech processing in human auditory cortex and that speech specificity arises out of cortical sensitivity to the complex acoustic structure determined by the human sound production apparatus.
Collapse
Affiliation(s)
- Anna Mari Mäkelä
- Apperception & Cortical Dynamics, Department of Psychology, P. O. Box 13, FIN-00014, University of Helsinki, Finland
| | | | | | | | | | | |
Collapse
|
23
|
|
24
|
Helenius P, Salmelin R, Richardson U, Leinonen S, Lyytinen H. Abnormal auditory cortical activation in dyslexia 100 msec after speech onset. J Cogn Neurosci 2002; 14:603-17. [PMID: 12126501 DOI: 10.1162/08989290260045846] [Citation(s) in RCA: 48] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Reading difficulties are associated with problems in processing and manipulating speech sounds. Dyslexic individuals seem to have, for instance, difficulties in perceiving the length and identity of consonants. Using magnetoencephalography (MEG), we characterized the spatio-temporal pattern of auditory cortical activation in dyslexia evoked by three types of natural bisyllabic pseudowords (/ata/, /atta/, and /a a/), complex nonspeech sound pairs (corresponding to /atta/ and /a a/) and simple 1-kHz tones. The most robust difference between dyslexic and non-reading-impaired adults was seen in the left supratemporal auditory cortex 100 msec after the onset of the vowel /a/. This N100m response was abnormally strong in dyslexic individuals. For the complex nonspeech sounds and tone, the N100m response amplitudes were similar in dyslexic and nonimpaired individuals. The responses evoked by syllable /ta/ of the pseudoword /atta/ also showed modest latency differences between the two subject groups. The responses evoked by the corresponding nonspeech sounds did not differ between the two subject groups. Further, when the initial formant transition, that is, the consonant, was removed from the syllable /ta/, the N100m latency was normal in dyslexic individuals. Thus, it appears that dyslexia is reflected as abnormal activation of the auditory cortex already 100 msec after speech onset, manifested as abnormal response strengths for natural speech and as delays for speech sounds containing rapid frequency transition. These differences between the dyslexic and nonimpaired individuals also imply that the N100m response codes stimulus-specific features likely to be critical for speech perception. Which features of speech (or nonspeech stimuli) are critical in eliciting the abnormally strong N100m response in dyslexic individuals should be resolved in future studies.
Collapse
|
25
|
Abstract
We used magnetoencephalography to elucidate the cortical activation associated with the segmentation of spoken words in nonreading-impaired and dyslexic adults. The subjects listened to binaurally presented sentences where the sentence-ending words were either semantically appropriate or inappropriate to the preceding sentence context. Half of the inappropriate final words shared two or three initial phonemes with the highly expected semantically appropriate words. Two temporally and functionally distinct response patterns were detected in the superior temporal lobe. The first response peaked at approximately 100 msec in the supratemporal plane and showed no sensitivity to the semantic appropriateness of the final word. This presemantic N100m response was abnormally strong in the left hemisphere of dyslexic individuals. After the N100m response, the semantically inappropriate sentence-ending words evoked stronger activation than the expected endings in the superior temporal cortex in the vicinity of the auditory cortex. This N400m response was delayed for words starting with the same two or three first few phonemes as the expected words but only until the first evidence of acoustic-phonetic dissimilarity emerged. This subtle delay supports the notion of initial lexical access being based on phonemes or acoustic features. In dyslexic participants, this qualitative aspect of word processing appeared to be normal. However, for all words alike, the ascending slope of the semantic activation in the left hemisphere was delayed by approximately 50 msec as compared with control subjects. The delay in the auditory N400m response in dyslexic subjects is likely to result from presemantic-phonological deficits possibly reflected in the abnormal N100m response.
Collapse
|
26
|
Eggermont JJ, Ponton CW. The neurophysiology of auditory perception: from single units to evoked potentials. Audiol Neurootol 2002; 7:71-99. [PMID: 12006736 DOI: 10.1159/000057656] [Citation(s) in RCA: 135] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022] Open
Abstract
Evoked electric potential and magnetic field studies have the immense benefit that they can be conducted in awake, behaving humans and can be directly correlated with aspects of perception. As such, they are powerful objective indicators of perceptual properties. However, given a set of evoked potential and/or evoked field waveforms and their source locations, obtained for an exhaustive set of stimuli and stimulus contrasts, is it possible to determine blindly, i.e. predict, what the stimuli or stimulus contrasts were? If this can be done with some success, then a useful amount of information resides in scalp-recorded activity for, e.g., the study of auditory speech processing. In this review, we compare neural representations based on single-unit and evoked response activity for vowels and consonant-vowel phonemes with distinctions in formant glides and voice onset time. We conclude that temporal aspects of evoked responses can track some of the dominant response features present in single-unit activity. However, N1 morphology does not reliably predict phonetic identification of stimuli varying in voice onset time, and the reported appearance of a double-peak onset response in aggregate recordings from the auditory cortex does not indicate a cortical correlate of the perception of voicelessness. This suggests that temporal aspects of single-unit population activity are likely not inclusive enough for representation of categorical perception boundaries. In contrast to population activity based on single-unit recording, the ability to accurately localize the sources of scalp-evoked activity is one of the bottlenecks in obtaining an accessible neurophysiological substrate of perception. Attaining this is one of the requisites to arrive at the prospect of blind determination of stimuli on the basis of evoked responses. At the current sophistication level of recording and analysis, evoked responses remain in the realm of extremely sensitive objective indicators of stimulus change or stimulus differences. As such, they are signs of perceptual activity, but not comprehensive representations thereof.
Collapse
|
27
|
Kotchoubey B, Lang S, Baales R, Herb E, Maurer P, Mezger G, Schmalohr D, Bostanov V, Birbaumer N. Brain potentials in human patients with extremely severe diffuse brain damage. Neurosci Lett 2001; 301:37-40. [PMID: 11239711 DOI: 10.1016/s0304-3940(01)01600-7] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
To test higher cortical functions of neurological patients, oddball tasks are often used in which a frequent and a rare stimulus are randomly presented and a P3 brain wave is recorded to the rare stimulus. We examined 33 patients with extremely severe brain injury. Three oddball conditions were used: with two sine tones (ST), with two complex tones (CT) and with vowels 'o' and 'i'. Across all patients, CT elicited P3 more often than ST, and the occurrence of the P3 after vowels was intermediate. However, among patients who showed a distinct P3 wave, its amplitude in the subgroup with traumatic brain injury was larger to vowels than to CT. In patients with non-traumatic etiology, CT and vowels elicited a P3 of a nearly equal amplitude. Stimuli of sufficient complexity should be used when the P3 technique is applied for assessment of cortical functions in severely impaired patients.
Collapse
Affiliation(s)
- B Kotchoubey
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Gartenstrasse 29, 72074, Tübingen, Germany.
| | | | | | | | | | | | | | | | | |
Collapse
|