1
|
Shim YJ, Jung WH, Billig AJ, Sedley W, Song JJ. Hippocampal atrophy is associated with hearing loss in cognitively normal adults. Front Neurosci 2023; 17:1276883. [PMID: 37942139 PMCID: PMC10628109 DOI: 10.3389/fnins.2023.1276883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Accepted: 10/11/2023] [Indexed: 11/10/2023] Open
Abstract
Objectives A growing body of evidence suggests that age-related hearing loss (HL) is associated with morphological changes of the cerebral cortex, but the results have been drawn from a small amount of data in most studies. The aim of this study is to investigate the correlation between HL and gray matter volume (GMV) in a large number of subjects, strictly controlling for an extensive set of possible biases. Methods Medical records of 576 subjects who underwent pure tone audiometry, brain magnetic resonance imaging (MRI), and the Korean Mini-Mental State Exam (K-MMSE) were reviewed. Among them, subjects with normal cognitive function and free of central nervous system disorders or coronary artery disease were included. Outliers were excluded after a sample homogeneity check. In the end, 405 subjects were enrolled. Pure tone hearing thresholds were determined at 0.5, 1, 2, and 4 kHz in the better ear. Enrolled subjects were divided into 3 groups according to pure tone average: normal hearing (NH), mild HL (MHL), and moderate-to-severe HL (MSHL) groups. Using voxel-based morphometry, we evaluated GMV changes that may be associated with HL. Sex, age, total intracranial volume, type of MRI scanner, education level, K-MMSE score, smoking status, and presence of hypertension, diabetes mellitus and dyslipidemia were used as covariates. Results A statistically significant negative correlation between the hearing thresholds and GMV of the hippocampus was elucidated. Additionally, in group comparisons, the left hippocampal GMV of the MSHL group was significantly smaller than that of the NH and MHL groups. Conclusion Based on the negative correlation between hearing thresholds and hippocampal GMV in cognitively normal old adults, the current study indicates that peripheral deafferentation could be a potential contributing factor to hippocampal atrophy.
Collapse
Affiliation(s)
- Ye Ji Shim
- Department of Otorhinolaryngology-Head and Neck Surgery, Healthcare System Gangnam Center, Seoul National University Hospital, Seoul, Republic of Korea
- Sensory Organ Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| | - Wi Hoon Jung
- Department of Psychology, Gachon University, Seongnam, Republic of Korea
| | | | - William Sedley
- Translational and Clinical Research Institute, Newcastle University Medical School, Newcastle upon Tyne, United Kingdom
| | - Jae-Jin Song
- Sensory Organ Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
- Department of Otorhinolaryngology-Head and Neck Surgery, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
| |
Collapse
|
2
|
Zhang Y, Rennig J, Magnotti JF, Beauchamp MS. Multivariate fMRI responses in superior temporal cortex predict visual contributions to, and individual differences in, the intelligibility of noisy speech. Neuroimage 2023; 278:120271. [PMID: 37442310 PMCID: PMC10460966 DOI: 10.1016/j.neuroimage.2023.120271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2023] [Revised: 06/20/2023] [Accepted: 07/06/2023] [Indexed: 07/15/2023] Open
Abstract
Humans have the unique ability to decode the rapid stream of language elements that constitute speech, even when it is contaminated by noise. Two reliable observations about noisy speech perception are that seeing the face of the talker improves intelligibility and the existence of individual differences in the ability to perceive noisy speech. We introduce a multivariate BOLD fMRI measure that explains both observations. In two independent fMRI studies, clear and noisy speech was presented in visual, auditory and audiovisual formats to thirty-seven participants who rated intelligibility. An event-related design was used to sort noisy speech trials by their intelligibility. Individual-differences multidimensional scaling was applied to fMRI response patterns in superior temporal cortex and the dissimilarity between responses to clear speech and noisy (but intelligible) speech was measured. Neural dissimilarity was less for audiovisual speech than auditory-only speech, corresponding to the greater intelligibility of noisy audiovisual speech. Dissimilarity was less in participants with better noisy speech perception, corresponding to individual differences. These relationships held for both single word and entire sentence stimuli, suggesting that they were driven by intelligibility rather than the specific stimuli tested. A neural measure of perceptual intelligibility may aid in the development of strategies for helping those with impaired speech perception.
Collapse
Affiliation(s)
- Yue Zhang
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States; Department of Neurosurgery, Baylor College of Medicine, Houston, TX, United States
| | - Johannes Rennig
- Division of Neuropsychology, Center of Neurology, Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | - John F Magnotti
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Michael S Beauchamp
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States.
| |
Collapse
|
3
|
Li N, Ma W, Ren F, Li X, Li F, Zong W, Wu L, Dai Z, Hui SCN, Edden RAE, Li M, Gao F. Neurochemical and functional reorganization of the cognitive-ear link underlies cognitive impairment in presbycusis. Neuroimage 2023; 268:119861. [PMID: 36610677 PMCID: PMC10026366 DOI: 10.1016/j.neuroimage.2023.119861] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2022] [Revised: 12/11/2022] [Accepted: 01/03/2023] [Indexed: 01/06/2023] Open
Abstract
Recent studies suggest that the interaction between presbycusis and cognitive impairment may be partially explained by the cognitive-ear link. However, the underlying neurophysiological mechanisms remain largely unknown. In this study, we combined magnetic resonance spectroscopy (MRS) and resting-state functional magnetic resonance imaging (fMRI) to investigate auditory gamma-aminobutyric acid (GABA) and glutamate (Glu) levels, intra- and inter-network functional connectivity, and their relationships with auditory and cognitive function in 51 presbycusis patients and 51 well-matched healthy controls. Our results confirmed reorganization of the cognitive-ear link in presbycusis, including decreased auditory GABA and Glu levels and aberrant functional connectivity involving auditory networks (AN) and cognitive-related networks, which were associated with reduced speech perception or cognitive impairment. Moreover, mediation analyses revealed that decreased auditory GABA levels and dysconnectivity between the AN and default mode network (DMN) mediated the association between hearing loss and impaired information processing speed in presbycusis. These findings highlight the importance of AN-DMN dysconnectivity in cognitive-ear link reorganization leading to cognitive impairment, and hearing loss may drive reorganization via decreased auditory GABA levels. Modulation of GABA neurotransmission may lead to new treatment strategies for cognitive impairment in presbycusis patients.
Collapse
Affiliation(s)
- Ning Li
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China
| | - Wen Ma
- Department of Otolaryngology, the Central Hospital of Jinan City, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Fuxin Ren
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China; Department of Radiology, Shandong Provincial Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Xiao Li
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China; Department of Radiology, Shandong Provincial Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Fuyan Li
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China; Department of Radiology, Shandong Provincial Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Wei Zong
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China; Department of Radiology, Shandong Provincial Hospital, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Lili Wu
- CAS Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
| | - Zongrui Dai
- Westa College, Southwest University, Chongqing, China
| | - Steve C N Hui
- Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine, Baltimore, MD, USA; F. M. Kirby Research Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore, MD, USA
| | - Richard A E Edden
- Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine, Baltimore, MD, USA; F. M. Kirby Research Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore, MD, USA
| | - Muwei Li
- Vanderbilt University Institute of Imaging Science, Nashville, TN, USA
| | - Fei Gao
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China.
| |
Collapse
|
4
|
Tarawneh HY, Jayakody DM, Sohrabi HR, Martins RN, Mulders WH. Understanding the Relationship Between Age-Related Hearing Loss and Alzheimer’s Disease: A Narrative Review. J Alzheimers Dis Rep 2022; 6:539-556. [PMID: 36275417 PMCID: PMC9535607 DOI: 10.3233/adr-220035] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Accepted: 08/16/2022] [Indexed: 12/02/2022] Open
Abstract
Evidence suggests that hearing loss (HL), even at mild levels, increases the long-term risk of cognitive decline and incident dementia. Hearing loss is one of the modifiable risk factors for dementia, with approximately 4 million of the 50 million cases of dementia worldwide possibly attributed to untreated HL. This paper describes four possible mechanisms that have been suggested for the relationship between age-related hearing loss (ARHL) and Alzheimer’s disease (AD), which is the most common form of dementia. The first mechanism suggests mitochondrial dysfunction and altered signal pathways due to aging as a possible link between ARHL and AD. The second mechanism proposes that sensory degradation in hearing impaired people could explain the relationship between ARHL and AD. The occupation of cognitive resource (third) mechanism indicates that the association between ARHL and AD is a result of increased cognitive processing that is required to compensate for the degraded sensory input. The fourth mechanism is an expansion of the third mechanism, i.e., the function and structure interaction involves both cognitive resource occupation (neural activity) and AD pathology as the link between ARHL and AD. Exploring the specific mechanisms that provide the link between ARHL and AD has the potential to lead to innovative ideas for the diagnosis, prevention, and/or treatment of AD. This paper also provides insight into the current evidence for the use of hearing treatments as a possible treatment/prevention for AD, and if auditory assessments could provide an avenue for early detection of cognitive impairment associated with AD.
Collapse
Affiliation(s)
- Hadeel Y. Tarawneh
- School of Human Sciences, The University of Western Australia, Crawley, WA, Australia
- Ear Science Institute Australia, Subiaco, WA, Australia
| | - Dona M.P. Jayakody
- Ear Science Institute Australia, Subiaco, WA, Australia
- Centre of Ear Science, Medical School, The University of Western Australia, Crawley, WA, Australia
| | - Hamid R. Sohrabi
- Centre for Healthy Ageing, College of Science, Health, Engineering and Education, Murdoch University, WA, Australia
- School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
- Department of Biomedical Sciences, Faculty of Medicine and Health Sciences, Macquarie University, NSW, Australia
| | - Ralph N. Martins
- School of Medical and Health Sciences, Edith Cowan University, Joondalup, WA, Australia
- Department of Biomedical Sciences, Faculty of Medicine and Health Sciences, Macquarie University, NSW, Australia
| | | |
Collapse
|
5
|
Geva S, Schneider LM, Khan S, Lorca-Puls DL, Gajardo-Vidal A, Hope TMH, Green DW, Price CJ. Enhanced left superior parietal activation during successful speech production in patients with left dorsal striatal damage and error-prone neurotypical participants. Cereb Cortex 2022; 33:3437-3453. [PMID: 35965059 PMCID: PMC10068299 DOI: 10.1093/cercor/bhac282] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 06/24/2022] [Accepted: 06/25/2022] [Indexed: 11/15/2022] Open
Abstract
Abstract
Functional imaging studies of neurotypical adults report activation in the left putamen during speech production. The current study asked how stroke survivors with left putamen damage are able to produce correct spoken responses during a range of speech production tasks. Using functional magnetic resonance imaging, activation during correct speech production responses was assessed in 5 stroke patients with circumscribed left dorsal striatal lesions, 66 stroke patient controls who did not have focal left dorsal striatal lesions, and 54 neurotypical adults. As a group, patients with left dorsal striatal damage (our patients of interest) showed higher activation than neurotypical controls in the left superior parietal cortex during successful speech production. This effect was not specific to patients with left dorsal striatal lesions as we observed enhanced activation in the same region in some patient controls and also in more error-prone neurotypical participants. Our results strongly suggest that enhanced left superior parietal activation supports speech production in diverse challenging circumstances, including those caused by stroke damage. They add to a growing body of literature indicating how upregulation within undamaged parts of the neural systems already recruited by neurotypical adults contributes to recovery after stroke.
Collapse
Affiliation(s)
- Sharon Geva
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
| | - Letitia M Schneider
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- Department of Cognition , Emotion, and Methods in Psychology, Faculty of Psychology, , Universitätsring 1, 1010 Vienna , Austria
- University of Vienna , Emotion, and Methods in Psychology, Faculty of Psychology, , Universitätsring 1, 1010 Vienna , Austria
| | - Shamima Khan
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
| | - Diego L Lorca-Puls
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- Sección Neurología , Departamento de Especialidades, Facultad de Medicina, , Victor Lamas 1290, Concepción, 4030000 , Chile
- Universidad de Concepción , Departamento de Especialidades, Facultad de Medicina, , Victor Lamas 1290, Concepción, 4030000 , Chile
| | - Andrea Gajardo-Vidal
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- Faculty of Health Sciences, Universidad del Desarrollo , Ainavillo 456, Concepción, 4070001 , Chile
| | - Thomas M H Hope
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
| | - David W Green
- Department of Experimental Psychology , Faculty of Brain Sciences, , 26 Bedford Way, London, WC1H 0AP , United Kingdom
- University College London , Faculty of Brain Sciences, , 26 Bedford Way, London, WC1H 0AP , United Kingdom
| | - Cathy J Price
- Wellcome Centre for Human Neuroimaging , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
- University College London , Institute of Neurology, , 12 Queen Square, London WC1N 3AR , United Kingdom
| | | |
Collapse
|
6
|
Bsharat-Maalouf D, Karawani H. Bilinguals' speech perception in noise: Perceptual and neural associations. PLoS One 2022; 17:e0264282. [PMID: 35196339 PMCID: PMC8865662 DOI: 10.1371/journal.pone.0264282] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2021] [Accepted: 02/07/2022] [Indexed: 01/26/2023] Open
Abstract
The current study characterized subcortical speech sound processing among monolinguals and bilinguals in quiet and challenging listening conditions and examined the relation between subcortical neural processing and perceptual performance. A total of 59 normal-hearing adults, ages 19–35 years, participated in the study: 29 native Hebrew-speaking monolinguals and 30 Arabic-Hebrew-speaking bilinguals. Auditory brainstem responses to speech sounds were collected in a quiet condition and with background noise. The perception of words and sentences in quiet and background noise conditions was also examined to assess perceptual performance and to evaluate the perceptual-physiological relationship. Perceptual performance was tested among bilinguals in both languages (first language (L1-Arabic) and second language (L2-Hebrew)). The outcomes were similar between monolingual and bilingual groups in quiet. Noise, as expected, resulted in deterioration in perceptual and neural responses, which was reflected in lower accuracy in perceptual tasks compared to quiet, and in more prolonged latencies and diminished neural responses. However, a mixed picture was observed among bilinguals in perceptual and physiological outcomes in noise. In the perceptual measures, bilinguals were significantly less accurate than their monolingual counterparts. However, in neural responses, bilinguals demonstrated earlier peak latencies compared to monolinguals. Our results also showed that perceptual performance in noise was related to subcortical resilience to the disruption caused by background noise. Specifically, in noise, increased brainstem resistance (i.e., fewer changes in the fundamental frequency (F0) representations or fewer shifts in the neural timing) was related to better speech perception among bilinguals. Better perception in L1 in noise was correlated with fewer changes in F0 representations, and more accurate perception in L2 was related to minor shifts in auditory neural timing. This study delves into the importance of using neural brainstem responses to speech sounds to differentiate individuals with different language histories and to explain inter-subject variability in bilinguals’ perceptual abilities in daily life situations.
Collapse
Affiliation(s)
- Dana Bsharat-Maalouf
- Department of Communication Sciences and Disorders, University of Haifa, Haifa, Israel
| | - Hanin Karawani
- Department of Communication Sciences and Disorders, University of Haifa, Haifa, Israel
- * E-mail:
| |
Collapse
|
7
|
Marsicano G, Cerpelloni F, Melcher D, Ronconi L. Lower multisensory temporal acuity in individuals with high schizotypal traits: a web-based study. Sci Rep 2022; 12:2782. [PMID: 35177673 PMCID: PMC8854550 DOI: 10.1038/s41598-022-06503-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 01/25/2022] [Indexed: 12/02/2022] Open
Abstract
Natural events are often multisensory, requiring the brain to combine information from the same spatial location and timing, across different senses. The importance of temporal coincidence has led to the introduction of the temporal binding window (TBW) construct, defined as the time range within which multisensory inputs are highly likely to be perceptually bound into a single entity. Anomalies in TBWs have been linked to confused perceptual experiences and inaccurate filtering of sensory inputs coming from different environmental sources. Indeed, larger TBWs have been associated with disorders such as schizophrenia and autism and are also correlated to a higher level of subclinical traits of these conditions in the general population. Here, we tested the feasibility of using a web-based version of a classic audio-visual simultaneity judgment (SJ) task with simple flash-beep stimuli in order to measure multisensory temporal acuity and its relationship with schizotypal traits as measured in the general population. Results show that: (i) the response distribution obtained in the web-based SJ task was strongly similar to those reported by studies carried out in controlled laboratory settings, and (ii) lower multisensory temporal acuity was associated with higher schizotypal traits in the “cognitive-perceptual” domains. Our findings reveal the possibility of adequately using a web-based audio-visual SJ task outside a controlled laboratory setting, available to a more diverse and representative pool of participants. These results provide additional evidence for a close relationship between lower multisensory acuity and the expression of schizotypal traits in the general population.
Collapse
Affiliation(s)
- Gianluca Marsicano
- School of Psychology, Vita-Salute San Raffaele University, Milan, Italy.,Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| | - Filippo Cerpelloni
- Center for Mind/Brain Sciences and Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy.,Laboratory of Biological Psychology, Department of Brain and Cognition, Leuven Brain Institute, KU Leuve, Leuven, Belgium.,Institute of Research in Psychology (IPSY) & Institute of Neuroscience (IoNS)-University of Louvain (UCLouvain), Leuven, Belgium
| | - David Melcher
- Center for Mind/Brain Sciences and Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy. .,Psychology Program, Division of Science, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates.
| | - Luca Ronconi
- School of Psychology, Vita-Salute San Raffaele University, Milan, Italy.,Division of Neuroscience, IRCCS San Raffaele Scientific Institute, Milan, Italy
| |
Collapse
|
8
|
Rennig J, Beauchamp MS. Intelligibility of audiovisual sentences drives multivoxel response patterns in human superior temporal cortex. Neuroimage 2022; 247:118796. [PMID: 34906712 PMCID: PMC8819942 DOI: 10.1016/j.neuroimage.2021.118796] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 11/18/2021] [Accepted: 12/08/2021] [Indexed: 11/18/2022] Open
Abstract
Regions of the human posterior superior temporal gyrus and sulcus (pSTG/S) respond to the visual mouth movements that constitute visual speech and the auditory vocalizations that constitute auditory speech, and neural responses in pSTG/S may underlie the perceptual benefit of visual speech for the comprehension of noisy auditory speech. We examined this possibility through the lens of multivoxel pattern responses in pSTG/S. BOLD fMRI data was collected from 22 participants presented with speech consisting of English sentences presented in five different formats: visual-only; auditory with and without added auditory noise; and audiovisual with and without auditory noise. Participants reported the intelligibility of each sentence with a button press and trials were sorted post-hoc into those that were more or less intelligible. Response patterns were measured in regions of the pSTG/S identified with an independent localizer. Noisy audiovisual sentences with very similar physical properties evoked very different response patterns depending on their intelligibility. When a noisy audiovisual sentence was reported as intelligible, the pattern was nearly identical to that elicited by clear audiovisual sentences. In contrast, an unintelligible noisy audiovisual sentence evoked a pattern like that of visual-only sentences. This effect was less pronounced for noisy auditory-only sentences, which evoked similar response patterns regardless of intelligibility. The successful integration of visual and auditory speech produces a characteristic neural signature in pSTG/S, highlighting the importance of this region in generating the perceptual benefit of visual speech.
Collapse
Affiliation(s)
- Johannes Rennig
- Division of Neuropsychology, Center of Neurology, Hertie-Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany
| | - Michael S Beauchamp
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Richards Medical Research Building, A607, 3700 Hamilton Walk, Philadelphia, PA 19104-6016, United States.
| |
Collapse
|
9
|
Uchida Y, Nishita Y, Otsuka R, Sugiura S, Sone M, Yamasoba T, Kato T, Iwata K, Nakamura A. Aging Brain and Hearing: A Mini-Review. Front Aging Neurosci 2022; 13:791604. [PMID: 35095475 PMCID: PMC8792606 DOI: 10.3389/fnagi.2021.791604] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Accepted: 12/24/2021] [Indexed: 02/03/2023] Open
Abstract
Brain reserve is a topic of great interest to researchers in aging medicine field. Some individuals retain well-preserved cognitive function until they fulfill their lives despite significant brain pathology. One concept that explains this paradox is the reserve hypothesis, including brain reserve that assumes a virtual ability to mitigate the effects of neuropathological changes and reduce the effects on clinical symptoms flexibly and efficiently by making complete use of the cognitive and compensatory processes. One of the surrogate measures of reserve capacity is brain volume. Evidence that dementia and hearing loss are interrelated has been steadily accumulating, and age-related hearing loss is one of the most promising modifiable risk factors of dementia. Research focused on the imaging analysis of the aged brain relative to auditory function has been gradually increasing. Several morphological studies have been conducted to understand the relationship between hearing loss and brain volume. In this mini review, we provide a brief overview of the concept of brain reserve, followed by a small review of studies addressing brain morphology and hearing loss/hearing compensation, including the findings obtained from our previous study that hearing loss after middle age could affect hippocampal and primary auditory cortex atrophy.
Collapse
Affiliation(s)
- Yasue Uchida
- Department of Otolaryngology, Aichi Medical University, Nagakute, Japan
- Department of Otorhinolaryngology, National Center for Geriatrics and Gerontology, Obu, Japan
- *Correspondence: Yasue Uchida,
| | - Yukiko Nishita
- Department of Epidemiology of Aging, Center for Gerontology and Social Science, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Rei Otsuka
- Section of NILS-LSA, Center for Gerontology and Social Science, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Saiko Sugiura
- Department of Otorhinolaryngology, National Center for Geriatrics and Gerontology, Obu, Japan
- Toyota Josui Mental Clinic, Toyota, Japan
| | - Michihiko Sone
- Department of Otorhinolaryngology, Nagoya University Graduate School of Medicine, Nagoya, Japan
| | - Tatsuya Yamasoba
- Department of Otolaryngology-Head and Neck Surgery, Faculty of Medicine, Graduate School of Medicine, The University of Tokyo, Tokyo, Japan
| | - Takashi Kato
- Department of Clinical and Experimental Neuroimaging, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Kaori Iwata
- Department of Clinical and Experimental Neuroimaging, National Center for Geriatrics and Gerontology, Obu, Japan
| | - Akinori Nakamura
- Department of Clinical and Experimental Neuroimaging, National Center for Geriatrics and Gerontology, Obu, Japan
| |
Collapse
|
10
|
Butera IM, Larson ED, DeFreese AJ, Lee AKC, Gifford RH, Wallace MT. Functional localization of audiovisual speech using near infrared spectroscopy. Brain Topogr 2022; 35:416-430. [PMID: 35821542 PMCID: PMC9334437 DOI: 10.1007/s10548-022-00904-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2021] [Accepted: 05/19/2022] [Indexed: 11/21/2022]
Abstract
Visual cues are especially vital for hearing impaired individuals such as cochlear implant (CI) users to understand speech in noise. Functional Near Infrared Spectroscopy (fNIRS) is a light-based imaging technology that is ideally suited for measuring the brain activity of CI users due to its compatibility with both the ferromagnetic and electrical components of these implants. In a preliminary step toward better elucidating the behavioral and neural correlates of audiovisual (AV) speech integration in CI users, we designed a speech-in-noise task and measured the extent to which 24 normal hearing individuals could integrate the audio of spoken monosyllabic words with the corresponding visual signals of a female speaker. In our behavioral task, we found that audiovisual pairings provided average improvements of 103% and 197% over auditory-alone listening conditions in -6 and -9 dB signal-to-noise ratios consisting of multi-talker background noise. In an fNIRS task using similar stimuli, we measured activity during auditory-only listening, visual-only lipreading, and AV listening conditions. We identified cortical activity in all three conditions over regions of middle and superior temporal cortex typically associated with speech processing and audiovisual integration. In addition, three channels active during the lipreading condition showed uncorrected correlations associated with behavioral measures of audiovisual gain as well as with the McGurk effect. Further work focusing primarily on the regions of interest identified in this study could test how AV speech integration may differ for CI users who rely on this mechanism for daily communication.
Collapse
Affiliation(s)
- Iliza M. Butera
- grid.152326.10000 0001 2264 7217Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN USA
| | - Eric D. Larson
- grid.34477.330000000122986657Institute for Learning & Brain Sciences, University of Washington, Seattle Washington, USA
| | - Andrea J. DeFreese
- grid.152326.10000 0001 2264 7217Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN USA
| | - Adrian KC Lee
- grid.34477.330000000122986657Institute for Learning & Brain Sciences, University of Washington, Seattle Washington, USA ,grid.34477.330000000122986657Department of Speech and Hearing Sciences, University of Washington, Seattle, Washington USA
| | - René H. Gifford
- grid.152326.10000 0001 2264 7217Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN USA
| | - Mark T. Wallace
- grid.152326.10000 0001 2264 7217Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN USA ,grid.152326.10000 0001 2264 7217Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN USA ,grid.412807.80000 0004 1936 9916Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, TN USA
| |
Collapse
|
11
|
Vander Ghinst M, Bourguignon M, Wens V, Naeije G, Ducène C, Niesen M, Hassid S, Choufani G, Goldman S, De Tiège X. Inaccurate cortical tracking of speech in adults with impaired speech perception in noise. Brain Commun 2021; 3:fcab186. [PMID: 34541530 PMCID: PMC8445395 DOI: 10.1093/braincomms/fcab186] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2021] [Revised: 06/05/2021] [Accepted: 06/08/2021] [Indexed: 01/17/2023] Open
Abstract
Impaired speech perception in noise despite normal peripheral auditory function is a common problem in young adults. Despite a growing body of research, the pathophysiology of this impairment remains unknown. This magnetoencephalography study characterizes the cortical tracking of speech in a multi-talker background in a group of highly selected adult subjects with impaired speech perception in noise without peripheral auditory dysfunction. Magnetoencephalographic signals were recorded from 13 subjects with impaired speech perception in noise (six females, mean age: 30 years) and matched healthy subjects while they were listening to 5 different recordings of stories merged with a multi-talker background at different signal to noise ratios (No Noise, +10, +5, 0 and −5 dB). The cortical tracking of speech was quantified with coherence between magnetoencephalographic signals and the temporal envelope of (i) the global auditory scene (i.e. the attended speech stream and the multi-talker background noise), (ii) the attended speech stream only and (iii) the multi-talker background noise. Functional connectivity was then estimated between brain areas showing altered cortical tracking of speech in noise in subjects with impaired speech perception in noise and the rest of the brain. All participants demonstrated a selective cortical representation of the attended speech stream in noisy conditions, but subjects with impaired speech perception in noise displayed reduced cortical tracking of speech at the syllable rate (i.e. 4–8 Hz) in all noisy conditions. Increased functional connectivity was observed in subjects with impaired speech perception in noise in Noiseless and speech in noise conditions between supratemporal auditory cortices and left-dominant brain areas involved in semantic and attention processes. The difficulty to understand speech in a multi-talker background in subjects with impaired speech perception in noise appears to be related to an inaccurate auditory cortex tracking of speech at the syllable rate. The increased functional connectivity between supratemporal auditory cortices and language/attention-related neocortical areas probably aims at supporting speech perception and subsequent recognition in adverse auditory scenes. Overall, this study argues for a central origin of impaired speech perception in noise in the absence of any peripheral auditory dysfunction.
Collapse
Affiliation(s)
- Marc Vander Ghinst
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Service, d'ORL et de chirurgie cervico-faciale, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Mathieu Bourguignon
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Laboratory of Neurophysiology and Movement Biomechanics, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Basque Center on Cognition, Brain and Language (BCBL), Donostia/San Sebastian 20009, Spain
| | - Vincent Wens
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Clinics of Functional Neuroimaging, Service of Nuclear Medicine, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Gilles Naeije
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Service de Neurologie, ULB-Hôpital Erasme, Université libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Cecile Ducène
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Service, d'ORL et de chirurgie cervico-faciale, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Maxime Niesen
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Service, d'ORL et de chirurgie cervico-faciale, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Sergio Hassid
- Service, d'ORL et de chirurgie cervico-faciale, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Georges Choufani
- Service, d'ORL et de chirurgie cervico-faciale, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Serge Goldman
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Clinics of Functional Neuroimaging, Service of Nuclear Medicine, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| | - Xavier De Tiège
- Laboratoire de Cartographie fonctionnelle du Cerveau, UNI-ULB Neuroscience Institute, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium.,Clinics of Functional Neuroimaging, Service of Nuclear Medicine, CUB Hôpital Erasme, Université Libre de Bruxelles (ULB), Brussels 1070, Belgium
| |
Collapse
|
12
|
Mihai PG, Tschentscher N, von Kriegstein K. Modulation of the Primary Auditory Thalamus When Recognizing Speech with Background Noise. J Neurosci 2021; 41:7136-7147. [PMID: 34244362 PMCID: PMC8372015 DOI: 10.1523/jneurosci.2902-20.2021] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Revised: 05/18/2021] [Accepted: 05/20/2021] [Indexed: 11/21/2022] Open
Abstract
Recognizing speech in background noise is a strenuous daily activity, yet most humans can master it. An explanation of how the human brain deals with such sensory uncertainty during speech recognition is to-date missing. Previous work has shown that recognition of speech without background noise involves modulation of the auditory thalamus (medial geniculate body; MGB): there are higher responses in left MGB for speech recognition tasks that require tracking of fast-varying stimulus properties in contrast to relatively constant stimulus properties (e.g., speaker identity tasks) despite the same stimulus input. Here, we tested the hypotheses that (1) this task-dependent modulation for speech recognition increases in parallel with the sensory uncertainty in the speech signal, i.e., the amount of background noise; and that (2) this increase is present in the ventral MGB, which corresponds to the primary sensory part of the auditory thalamus. In accordance with our hypothesis, we show, by using ultra-high-resolution functional magnetic resonance imaging (fMRI) in male and female human participants, that the task-dependent modulation of the left ventral MGB (vMGB) for speech is particularly strong when recognizing speech in noisy listening conditions in contrast to situations where the speech signal is clear. The results imply that speech in noise recognition is supported by modifications at the level of the subcortical sensory pathway providing driving input to the auditory cortex.SIGNIFICANCE STATEMENT Speech recognition in noisy environments is a challenging everyday task. One reason why humans can master this task is the recruitment of additional cognitive resources as reflected in recruitment of non-language cerebral cortex areas. Here, we show that also modulation in the primary sensory pathway is specifically involved in speech in noise recognition. We found that the left primary sensory thalamus (ventral medial geniculate body; vMGB) is more involved when recognizing speech signals as opposed to a control task (speaker identity recognition) when heard in background noise versus when the noise was absent. This finding implies that the brain optimizes sensory processing in subcortical sensory pathway structures in a task-specific manner to deal with speech recognition in noisy environments.
Collapse
Affiliation(s)
- Paul Glad Mihai
- Chair of Cognitive and Clinical Neuroscience, Faculty of Psychology, Technische Universität Dresden, Dresden 01187, Germany
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig 04103, Germany
| | - Nadja Tschentscher
- Research Unit Biological Psychology, Department of Psychology, Ludwig-Maximilians-University Munich, Munich 80802, Germany
| | - Katharina von Kriegstein
- Chair of Cognitive and Clinical Neuroscience, Faculty of Psychology, Technische Universität Dresden, Dresden 01187, Germany
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig 04103, Germany
| |
Collapse
|
13
|
Griffiths TD, Lad M, Kumar S, Holmes E, McMurray B, Maguire EA, Billig AJ, Sedley W. How Can Hearing Loss Cause Dementia? Neuron 2020; 108:401-412. [PMID: 32871106 PMCID: PMC7664986 DOI: 10.1016/j.neuron.2020.08.003] [Citation(s) in RCA: 151] [Impact Index Per Article: 37.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2020] [Revised: 07/31/2020] [Accepted: 08/05/2020] [Indexed: 12/11/2022]
Abstract
Epidemiological studies identify midlife hearing loss as an independent risk factor for dementia, estimated to account for 9% of cases. We evaluate candidate brain bases for this relationship. These bases include a common pathology affecting the ascending auditory pathway and multimodal cortex, depletion of cognitive reserve due to an impoverished listening environment, and the occupation of cognitive resources when listening in difficult conditions. We also put forward an alternate mechanism, drawing on new insights into the role of the medial temporal lobe in auditory cognition. In particular, we consider how aberrant activity in the service of auditory pattern analysis, working memory, and object processing may interact with dementia pathology in people with hearing loss. We highlight how the effect of hearing interventions on dementia depends on the specific mechanism and suggest avenues for work at the molecular, neuronal, and systems levels to pin this down.
Collapse
Affiliation(s)
- Timothy D Griffiths
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne NE2 4HH, UK; Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK; Human Brain Research Laboratory, Department of Neurosurgery, University of Iowa Hospitals and Clinics, Iowa City, IA 52242, USA.
| | - Meher Lad
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne NE2 4HH, UK
| | - Sukhbinder Kumar
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne NE2 4HH, UK
| | - Emma Holmes
- Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | - Bob McMurray
- Departments of Psychological and Brain Sciences, Communication Sciences and Disorders, Otolaryngology, University of Iowa, Iowa City, IA 52242, USA
| | - Eleanor A Maguire
- Wellcome Centre for Human Neuroimaging, UCL Queen Square Institute of Neurology, University College London, London WC1N 3AR, UK
| | | | - William Sedley
- Biosciences Institute, Newcastle University Medical School, Newcastle upon Tyne NE2 4HH, UK
| |
Collapse
|
14
|
Keitel A, Gross J, Kayser C. Shared and modality-specific brain regions that mediate auditory and visual word comprehension. eLife 2020; 9:e56972. [PMID: 32831168 PMCID: PMC7470824 DOI: 10.7554/elife.56972] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2020] [Accepted: 08/18/2020] [Indexed: 12/22/2022] Open
Abstract
Visual speech carried by lip movements is an integral part of communication. Yet, it remains unclear in how far visual and acoustic speech comprehension are mediated by the same brain regions. Using multivariate classification of full-brain MEG data, we first probed where the brain represents acoustically and visually conveyed word identities. We then tested where these sensory-driven representations are predictive of participants' trial-wise comprehension. The comprehension-relevant representations of auditory and visual speech converged only in anterior angular and inferior frontal regions and were spatially dissociated from those representations that best reflected the sensory-driven word identity. These results provide a neural explanation for the behavioural dissociation of acoustic and visual speech comprehension and suggest that cerebral representations encoding word identities may be more modality-specific than often upheld.
Collapse
Affiliation(s)
- Anne Keitel
- Psychology, University of DundeeDundeeUnited Kingdom
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
| | - Joachim Gross
- Institute of Neuroscience and Psychology, University of GlasgowGlasgowUnited Kingdom
- Institute for Biomagnetism and Biosignalanalysis, University of MünsterMünsterGermany
| | - Christoph Kayser
- Department for Cognitive Neuroscience, Faculty of Biology, Bielefeld UniversityBielefeldGermany
| |
Collapse
|
15
|
Age-related hearing loss influences functional connectivity of auditory cortex for the McGurk illusion. Cortex 2020; 129:266-280. [PMID: 32535378 DOI: 10.1016/j.cortex.2020.04.022] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Revised: 03/30/2020] [Accepted: 04/09/2020] [Indexed: 01/23/2023]
Abstract
Age-related hearing loss affects hearing at high frequencies and is associated with difficulties in understanding speech. Increased audio-visual integration has recently been found in age-related hearing impairment, the brain mechanisms that contribute to this effect are however unclear. We used functional magnetic resonance imaging in elderly subjects with normal hearing and mild to moderate uncompensated hearing loss. Audio-visual integration was studied using the McGurk task. In this task, an illusionary fused percept can occur if incongruent auditory and visual syllables are presented. The paradigm included unisensory stimuli (auditory only, visual only), congruent audio-visual and incongruent (McGurk) audio-visual stimuli. An illusionary precept was reported in over 60% of incongruent trials. These McGurk illusion rates were equal in both groups of elderly subjects and correlated positively with speech-in-noise perception and daily listening effort. Normal-hearing participants showed an increased neural response in left pre- and postcentral gyri and right middle frontal gyrus for incongruent stimuli (McGurk) compared to congruent audio-visual stimuli. Activation patterns were however not different between groups. Task-modulated functional connectivity differed between groups showing increased connectivity from auditory cortex to visual, parietal and frontal areas in hard of hearing participants as compared to normal-hearing participants when comparing incongruent stimuli (McGurk) with congruent audio-visual stimuli. These results suggest that changes in functional connectivity of auditory cortex rather than activation strength during processing of audio-visual McGurk stimuli accompany age-related hearing loss.
Collapse
|
16
|
Leminen A, Verwoert M, Moisala M, Salmela V, Wikman P, Alho K. Modulation of Brain Activity by Selective Attention to Audiovisual Dialogues. Front Neurosci 2020; 14:436. [PMID: 32477054 PMCID: PMC7235384 DOI: 10.3389/fnins.2020.00436] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2019] [Accepted: 04/09/2020] [Indexed: 01/08/2023] Open
Abstract
In real-life noisy situations, we can selectively attend to conversations in the presence of irrelevant voices, but neurocognitive mechanisms in such natural listening situations remain largely unexplored. Previous research has shown distributed activity in the mid superior temporal gyrus (STG) and sulcus (STS) while listening to speech and human voices, in the posterior STS and fusiform gyrus when combining auditory, visual and linguistic information, as well as in left-hemisphere temporal and frontal cortical areas during comprehension. In the present functional magnetic resonance imaging (fMRI) study, we investigated how selective attention modulates neural responses to naturalistic audiovisual dialogues. Our healthy adult participants (N = 15) selectively attended to video-taped dialogues between a man and woman in the presence of irrelevant continuous speech in the background. We modulated the auditory quality of dialogues with noise vocoding and their visual quality by masking speech-related facial movements. Both increased auditory quality and increased visual quality were associated with bilateral activity enhancements in the STG/STS. In addition, decreased audiovisual stimulus quality elicited enhanced fronto-parietal activity, presumably reflecting increased attentional demands. Finally, attention to the dialogues, in relation to a control task where a fixation cross was attended and the dialogue ignored, yielded enhanced activity in the left planum polare, angular gyrus, the right temporal pole, as well as in the orbitofrontal/ventromedial prefrontal cortex and posterior cingulate gyrus. Our findings suggest that naturalistic conversations effectively engage participants and reveal brain networks related to social perception in addition to speech and semantic processing networks.
Collapse
Affiliation(s)
- Alina Leminen
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Cognitive Science, Department of Digital Humanities, Helsinki Centre for Digital Humanities (Heldig), University of Helsinki, Helsinki, Finland
- Cognitive Brain Research Unit, Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Center for Cognition and Decision Making, Institute of Cognitive Neuroscience, National Research University – Higher School of Economics, Moscow, Russia
| | - Maxime Verwoert
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Mona Moisala
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Viljami Salmela
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Patrik Wikman
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Kimmo Alho
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
- Advanced Magnetic Imaging Centre, Aalto NeuroImaging, Aalto University, Espoo, Finland
| |
Collapse
|
17
|
Zhou HY, Cheung EFC, Chan RCK. Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions. Neuropsychologia 2020; 140:107396. [PMID: 32087206 DOI: 10.1016/j.neuropsychologia.2020.107396] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2019] [Revised: 02/14/2020] [Accepted: 02/15/2020] [Indexed: 12/21/2022]
Abstract
To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.
Collapse
Affiliation(s)
- Han-Yu Zhou
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
| | | | - Raymond C K Chan
- Neuropsychology and Applied Cognitive Neuroscience Laboratory, CAS Key Laboratory of Mental Health, Institute of Psychology, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
18
|
Li Y, Wang F, Chen Y, Cichocki A, Sejnowski T. The Effects of Audiovisual Inputs on Solving the Cocktail Party Problem in the Human Brain: An fMRI Study. Cereb Cortex 2019; 28:3623-3637. [PMID: 29029039 DOI: 10.1093/cercor/bhx235] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2017] [Indexed: 11/13/2022] Open
Abstract
At cocktail parties, our brains often simultaneously receive visual and auditory information. Although the cocktail party problem has been widely investigated under auditory-only settings, the effects of audiovisual inputs have not. This study explored the effects of audiovisual inputs in a simulated cocktail party. In our fMRI experiment, each congruent audiovisual stimulus was a synthesis of 2 facial movie clips, each of which could be classified into 1 of 2 emotion categories (crying and laughing). Visual-only (faces) and auditory-only stimuli (voices) were created by extracting the visual and auditory contents from the synthesized audiovisual stimuli. Subjects were instructed to selectively attend to 1 of the 2 objects contained in each stimulus and to judge its emotion category in the visual-only, auditory-only, and audiovisual conditions. The neural representations of the emotion features were assessed by calculating decoding accuracy and brain pattern-related reproducibility index based on the fMRI data. We compared the audiovisual condition with the visual-only and auditory-only conditions and found that audiovisual inputs enhanced the neural representations of emotion features of the attended objects instead of the unattended objects. This enhancement might partially explain the benefits of audiovisual inputs for the brain to solve the cocktail party problem.
Collapse
Affiliation(s)
- Yuanqing Li
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Applications, Guangzhou, China
| | - Fangyi Wang
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Applications, Guangzhou, China
| | - Yongbin Chen
- Center for Brain Computer Interfaces and Brain Information Processing, South China University of Technology, Guangzhou, China.,Guangzhou Key Laboratory of Brain Computer Interaction and Applications, Guangzhou, China
| | - Andrzej Cichocki
- Riken Brain Science Institute, Wako shi, Japan.,Skolkovo Institute of Science and Technology (SKOTECH), Moscow, Russia
| | - Terrence Sejnowski
- Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, CA, USA
| |
Collapse
|
19
|
Sengupta P, Burgaleta M, Zamora-López G, Basora A, Sanjuán A, Deco G, Sebastian-Galles N. Traces of statistical learning in the brain's functional connectivity after artificial language exposure. Neuropsychologia 2019; 124:246-253. [PMID: 30521815 DOI: 10.1016/j.neuropsychologia.2018.12.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2018] [Revised: 11/27/2018] [Accepted: 12/01/2018] [Indexed: 10/27/2022]
Abstract
Our environment is full of statistical regularities, and we are attuned to learn about these regularities by employing Statistical Learning (SL), a domain-general ability that enables the implicit detection of probabilistic regularities in our surrounding environment. The role of brain connectivity on SL has been previously explored, highlighting the relevance of structural and functional connections between frontal, parietal, and temporal cortices. However, whether SL can induce changes in the functional connections of the resting state brain has yet to be investigated. To address this question, we applied a pre-post design where participants (n = 38) were submitted to resting-state fMRI acquisition before and after in-scanner exposure to either an artificial language stream (formed by 4 concatenated words) or a random audio stream. Our results showed that exposure to an artificial language stream significantly changed (corrected p < 0.05) the functional connectivity between Right Posterior Cingulum and Left Superior Parietal Lobule. This suggests that functional connectivity between brain networks supporting attentional and working memory processes may play an important role in statistical learning.
Collapse
Affiliation(s)
- Pallabi Sengupta
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Miguel Burgaleta
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Gorka Zamora-López
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Anna Basora
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Ana Sanjuán
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Gustavo Deco
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| | - Nuria Sebastian-Galles
- Center for Brain and Cognition, Department of Technology, Universitat Pompeu Fabra, 08018 Barcelona, Spain.
| |
Collapse
|
20
|
Cortical Tracking of Speech-in-Noise Develops from Childhood to Adulthood. J Neurosci 2019; 39:2938-2950. [PMID: 30745419 DOI: 10.1523/jneurosci.1732-18.2019] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2018] [Revised: 01/08/2019] [Accepted: 01/12/2019] [Indexed: 11/21/2022] Open
Abstract
In multitalker backgrounds, the auditory cortex of adult humans tracks the attended speech stream rather than the global auditory scene. Still, it is unknown whether such preferential tracking also occurs in children whose speech-in-noise (SiN) abilities are typically lower compared with adults. We used magnetoencephalography (MEG) to investigate the frequency-specific cortical tracking of different elements of a cocktail party auditory scene in 20 children (age range, 6-9 years; 8 females) and 20 adults (age range, 21-40 years; 10 females). During MEG recordings, subjects attended to four different 5 min stories, mixed with different levels of multitalker background at four signal-to-noise ratios (SNRs; noiseless, +5, 0, and -5 dB). Coherence analysis quantified the coupling between the time courses of the MEG activity and attended speech stream, multitalker background, or global auditory scene, respectively. In adults, statistically significant coherence was observed between MEG signals originating from the auditory system and the attended stream at <1, 1-4, and 4-8 Hz in all SNR conditions. Children displayed similar coupling at <1 and 1-4 Hz, but increasing noise impaired the coupling more strongly than in adults. Also, children displayed drastically lower coherence at 4-8 Hz in all SNR conditions. These results suggest that children's difficulties to understand speech in noisy conditions are related to an immature selective cortical tracking of the attended speech streams. Our results also provide unprecedented evidence for an acquired cortical tracking of speech at syllable rate and argue for a progressive development of SiN abilities in humans.SIGNIFICANCE STATEMENT Behaviorally, children are less proficient than adults at understanding speech-in-noise. Here, neuromagnetic signals were recorded while healthy adults and typically developing 6- to 9-year-old children attended to a speech stream embedded in a multitalker background noise with varying intensity. Results demonstrate that auditory cortices of both children and adults selectively track the attended speaker's voice rather than the global acoustic input at phrasal and word rates. However, increments of noise compromised the tracking significantly more in children than in adults. Unexpectedly, children displayed limited tracking of both the attended voice and the global acoustic input at the 4-8 Hz syllable rhythm. Thus, both speech-in-noise abilities and cortical tracking of speech syllable repetition rate seem to mature later in adolescence.
Collapse
|
21
|
Tremblay P, Perron M, Deschamps I, Kennedy‐Higgins D, Houde J, Dick AS, Descoteaux M. The role of the arcuate and middle longitudinal fasciculi in speech perception in noise in adulthood. Hum Brain Mapp 2019; 40:226-241. [PMID: 30277622 PMCID: PMC6865648 DOI: 10.1002/hbm.24367] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2018] [Revised: 08/07/2018] [Accepted: 08/08/2018] [Indexed: 12/13/2022] Open
Abstract
In this article, we used High Angular Resolution Diffusion Imaging (HARDI) with advanced anatomically constrained particle filtering tractography to investigate the role of the arcuate fasciculus (AF) and the middle longitudinal fasciculus (MdLF) in speech perception in noise in younger and older adults. Fourteen young and 15 elderly adults completed a syllable discrimination task in the presence of broadband masking noise. Mediation analyses revealed few effects of age on white matter (WM) in these fascicles but broad effects of WM on speech perception, independently of age, especially in terms of sensitivity and criterion (response bias), after controlling for individual differences in hearing sensitivity and head size. Indirect (mediated) effects of age on speech perception through WM microstructure were also found, after controlling for individual differences in hearing sensitivity and head size, with AF microstructure related to sensitivity, response bias and phonological priming, and MdLF microstructure more strongly related to response bias. These findings suggest that pathways of the perisylvian region contribute to speech processing abilities, with relatively distinct contributions for the AF (sensitivity) and MdLF (response bias), indicative of a complex contribution of both phonological and cognitive processes to age-related speech perception decline. These results provide new and important insights into the roles of these pathways as well as the factors that may contribute to elderly speech perception deficits. They also highlight the need for a greater focus to be placed on studying the role of WM microstructure to understand cognitive aging.
Collapse
Affiliation(s)
- Pascale Tremblay
- CERVO Brain Research CenterQuebec CityCanada
- Département de Réadaptation, Faculté de MédecineUniversité LavalQuebec CityCanada
| | | | - Isabelle Deschamps
- CERVO Brain Research CenterQuebec CityCanada
- Département de Réadaptation, Faculté de MédecineUniversité LavalQuebec CityCanada
| | - Dan Kennedy‐Higgins
- CERVO Brain Research CenterQuebec CityCanada
- Department of Speech, Hearing and Phonetic SciencesUniversity College LondonUnited Kingdom
| | - Jean‐Christophe Houde
- Département d'informatique, Faculté des Sciences, Sherbrooke Connectivity Imaging LabUniversité de SherbrookeSherbrookeCanada
| | | | - Maxime Descoteaux
- Département d'informatique, Faculté des Sciences, Sherbrooke Connectivity Imaging LabUniversité de SherbrookeSherbrookeCanada
| |
Collapse
|
22
|
Werchan DM, Baumgartner HA, Lewkowicz DJ, Amso D. The origins of cortical multisensory dynamics: Evidence from human infants. Dev Cogn Neurosci 2018; 34:75-81. [PMID: 30099263 PMCID: PMC6629259 DOI: 10.1016/j.dcn.2018.07.002] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2017] [Revised: 07/03/2018] [Accepted: 07/13/2018] [Indexed: 12/15/2022] Open
Abstract
Classic views of multisensory processing suggest that cortical sensory regions are specialized. More recent views argue that cortical sensory regions are inherently multisensory. To date, there are no published neuroimaging data that directly test these claims in infancy. Here we used fNIRS to show that temporal and occipital cortex are functionally coupled in 3.5-5-month-old infants (N = 65), and that the extent of this coupling during a synchronous, but not an asynchronous, audiovisual event predicted whether occipital cortex would subsequently respond to sound-only information. These data suggest that multisensory experience may shape cortical dynamics to adapt to the ubiquity of synchronous multisensory information in the environment, and invoke the possibility that adaptation to the environment can also reflect broadening of the computational range of sensory systems.
Collapse
Affiliation(s)
- Denise M Werchan
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, 190 Thayer St. Providence, RI, 02912, United States
| | - Heidi A Baumgartner
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, 190 Thayer St. Providence, RI, 02912, United States
| | - David J Lewkowicz
- Department of Communication Sciences and Disorders, Northeastern University, 360 Huntington Ave., Boston, MA, 02115, United States
| | - Dima Amso
- Department of Cognitive, Linguistic, and Psychological Sciences, Brown University, 190 Thayer St. Providence, RI, 02912, United States.
| |
Collapse
|
23
|
Bottenhorn KL, Flannery JS, Boeving ER, Riedel MC, Eickhoff SB, Sutherland MT, Laird AR. Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional MRI results. Netw Neurosci 2018; 3:27-48. [PMID: 30793072 PMCID: PMC6326731 DOI: 10.1162/netn_a_00050] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2017] [Accepted: 03/02/2018] [Indexed: 11/04/2022] Open
Abstract
Cognitive processes do not occur by pure insertion and instead depend on the full complement of co-occurring mental processes, including perceptual and motor functions. As such, there is limited ecological validity to human neuroimaging experiments that use highly controlled tasks to isolate mental processes of interest. However, a growing literature shows how dynamic, interactive tasks have allowed researchers to study cognition as it more naturally occurs. Collective analysis across such neuroimaging experiments may answer broader questions regarding how naturalistic cognition is biologically distributed throughout the brain. We applied an unbiased, data-driven, meta-analytic approach that uses k-means clustering to identify core brain networks engaged across the naturalistic functional neuroimaging literature. Functional decoding allowed us to, then, delineate how information is distributed between these networks throughout the execution of dynamical cognition in realistic settings. This analysis revealed six recurrent patterns of brain activation, representing sensory, domain-specific, and attentional neural networks that support the cognitive demands of naturalistic paradigms. Although gaps in the literature remain, these results suggest that naturalistic fMRI paradigms recruit a common set of networks that allow both separate processing of different streams of information and integration of relevant information to enable flexible cognition and complex behavior.
Collapse
Affiliation(s)
| | | | - Emily R. Boeving
- Department of Psychology, Florida International University, Miami, FL, USA
| | - Michael C. Riedel
- Department of Physics, Florida International University, Miami, FL, USA
| | - Simon B. Eickhoff
- Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich, Germany
| | | | - Angela R. Laird
- Department of Physics, Florida International University, Miami, FL, USA
| |
Collapse
|
24
|
Cross-decoding supramodal information in the human brain. Brain Struct Funct 2018; 223:4087-4098. [PMID: 30143866 DOI: 10.1007/s00429-018-1740-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2018] [Accepted: 08/21/2018] [Indexed: 10/28/2022]
Abstract
Perceptual decision making is the cognitive process wherein the brain classifies stimuli into abstract categories for more efficient downstream processing. A system that, during categorization, can process information regardless of the information's original sensory modality (i.e., a supramodal system) would have a substantial advantage over a system with dedicated processes for specific sensory modalities. While many studies have probed decision processes through the lens of one sensory modality, it remains unclear whether there are such supramodal brain areas that can flexibly process task-relevant information regardless of the original "format" of the information. To investigate supramodality, one must ensure that supramodal information exists somewhere within the functional architecture by rendering information from multiple sensory systems necessary but insufficient for categorization. To this aim, we tasked participants with categorizing auditory and tactile frequency-modulated sweeps according to learned, supramodal categories in a delayed match-to-category paradigm while we measured their blood-oxygen-level dependent signal with functional MRI. To detect supramodal information, we implemented a set of cross-modality pattern classification analyses, which demonstrated that the left caudate nucleus encodes category-level information but not stimulus-specific information (such as spatial directions and stimulus modalities), while the right inferior frontal gyrus, showing the opposite pattern, encodes stimulus-specific information but not category-level information. Given our paradigm, these results reveal abstract representations in the brain that are independent of motor, semantic, and sensory-specific processing, instead reflecting supramodal, categorical information, which points to the caudate nucleus as a locus of cognitive processes involved in complex behavior.
Collapse
|
25
|
Hausfeld L, Riecke L, Formisano E. Acoustic and higher-level representations of naturalistic auditory scenes in human auditory and frontal cortex. Neuroimage 2018. [DOI: 10.1016/j.neuroimage.2018.02.065] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/17/2022] Open
|
26
|
Stevenson RA, Sheffield SW, Butera IM, Gifford RH, Wallace MT. Multisensory Integration in Cochlear Implant Recipients. Ear Hear 2018; 38:521-538. [PMID: 28399064 DOI: 10.1097/aud.0000000000000435] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Speech perception is inherently a multisensory process involving integration of auditory and visual cues. Multisensory integration in cochlear implant (CI) recipients is a unique circumstance in that the integration occurs after auditory deprivation and the provision of hearing via the CI. Despite the clear importance of multisensory cues for perception, in general, and for speech intelligibility, specifically, the topic of multisensory perceptual benefits in CI users has only recently begun to emerge as an area of inquiry. We review the research that has been conducted on multisensory integration in CI users to date and suggest a number of areas needing further research. The overall pattern of results indicates that many CI recipients show at least some perceptual gain that can be attributable to multisensory integration. The extent of this gain, however, varies based on a number of factors, including age of implantation and specific task being assessed (e.g., stimulus detection, phoneme perception, word recognition). Although both children and adults with CIs obtain audiovisual benefits for phoneme, word, and sentence stimuli, neither group shows demonstrable gain for suprasegmental feature perception. Additionally, only early-implanted children and the highest performing adults obtain audiovisual integration benefits similar to individuals with normal hearing. Increasing age of implantation in children is associated with poorer gains resultant from audiovisual integration, suggesting a sensitive period in development for the brain networks that subserve these integrative functions, as well as length of auditory experience. This finding highlights the need for early detection of and intervention for hearing loss, not only in terms of auditory perception, but also in terms of the behavioral and perceptual benefits of audiovisual processing. Importantly, patterns of auditory, visual, and audiovisual responses suggest that underlying integrative processes may be fundamentally different between CI users and typical-hearing listeners. Future research, particularly in low-level processing tasks such as signal detection will help to further assess mechanisms of multisensory integration for individuals with hearing loss, both with and without CIs.
Collapse
Affiliation(s)
- Ryan A Stevenson
- 1Department of Psychology, University of Western Ontario, London, Ontario, Canada; 2Brain and Mind Institute, University of Western Ontario, London, Ontario, Canada; 3Walter Reed National Military Medical Center, Audiology and Speech Pathology Center, London, Ontario, Canada; 4Vanderbilt Brain Institute, Nashville, Tennesse; 5Vanderbilt Kennedy Center, Nashville, Tennesse; 6Department of Psychology, Vanderbilt University, Nashville, Tennesse; 7Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennesse; and 8Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennesse
| | | | | | | | | |
Collapse
|
27
|
Ozker M, Yoshor D, Beauchamp MS. Converging Evidence From Electrocorticography and BOLD fMRI for a Sharp Functional Boundary in Superior Temporal Gyrus Related to Multisensory Speech Processing. Front Hum Neurosci 2018; 12:141. [PMID: 29740294 PMCID: PMC5928751 DOI: 10.3389/fnhum.2018.00141] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2018] [Accepted: 03/28/2018] [Indexed: 01/15/2023] Open
Abstract
Although humans can understand speech using the auditory modality alone, in noisy environments visual speech information from the talker’s mouth can rescue otherwise unintelligible auditory speech. To investigate the neural substrates of multisensory speech perception, we compared neural activity from the human superior temporal gyrus (STG) in two datasets. One dataset consisted of direct neural recordings (electrocorticography, ECoG) from surface electrodes implanted in epilepsy patients (this dataset has been previously published). The second dataset consisted of indirect measures of neural activity using blood oxygen level dependent functional magnetic resonance imaging (BOLD fMRI). Both ECoG and fMRI participants viewed the same clear and noisy audiovisual speech stimuli and performed the same speech recognition task. Both techniques demonstrated a sharp functional boundary in the STG, spatially coincident with an anatomical boundary defined by the posterior edge of Heschl’s gyrus. Cortex on the anterior side of the boundary responded more strongly to clear audiovisual speech than to noisy audiovisual speech while cortex on the posterior side of the boundary did not. For both ECoG and fMRI measurements, the transition between the functionally distinct regions happened within 10 mm of anterior-to-posterior distance along the STG. We relate this boundary to the multisensory neural code underlying speech perception and propose that it represents an important functional division within the human speech perception network.
Collapse
Affiliation(s)
- Muge Ozker
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, United States
| | - Daniel Yoshor
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, United States.,Michael E. DeBakey Veterans Affairs Medical Center, Houston, TX, United States
| | - Michael S Beauchamp
- Department of Neurosurgery, Baylor College of Medicine, Houston, TX, United States
| |
Collapse
|
28
|
Rosemann S, Thiel CM. Audio-visual speech processing in age-related hearing loss: Stronger integration and increased frontal lobe recruitment. Neuroimage 2018; 175:425-437. [PMID: 29655940 DOI: 10.1016/j.neuroimage.2018.04.023] [Citation(s) in RCA: 41] [Impact Index Per Article: 6.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2017] [Revised: 03/09/2018] [Accepted: 04/09/2018] [Indexed: 11/19/2022] Open
Abstract
Hearing loss is associated with difficulties in understanding speech, especially under adverse listening conditions. In these situations, seeing the speaker improves speech intelligibility in hearing-impaired participants. On the neuronal level, previous research has shown cross-modal plastic reorganization in the auditory cortex following hearing loss leading to altered processing of auditory, visual and audio-visual information. However, how reduced auditory input effects audio-visual speech perception in hearing-impaired subjects is largely unknown. We here investigated the impact of mild to moderate age-related hearing loss on processing audio-visual speech using functional magnetic resonance imaging. Normal-hearing and hearing-impaired participants performed two audio-visual speech integration tasks: a sentence detection task inside the scanner and the McGurk illusion outside the scanner. Both tasks consisted of congruent and incongruent audio-visual conditions, as well as auditory-only and visual-only conditions. We found a significantly stronger McGurk illusion in the hearing-impaired participants, which indicates stronger audio-visual integration. Neurally, hearing loss was associated with an increased recruitment of frontal brain areas when processing incongruent audio-visual, auditory and also visual speech stimuli, which may reflect the increased effort to perform the task. Hearing loss modulated both the audio-visual integration strength measured with the McGurk illusion and brain activation in frontal areas in the sentence task, showing stronger integration and higher brain activation with increasing hearing loss. Incongruent compared to congruent audio-visual speech revealed an opposite brain activation pattern in left ventral postcentral gyrus in both groups, with higher activation in hearing-impaired participants in the incongruent condition. Our results indicate that already mild to moderate hearing loss impacts audio-visual speech processing accompanied by changes in brain activation particularly involving frontal areas. These changes are modulated by the extent of hearing loss.
Collapse
Affiliation(s)
- Stephanie Rosemann
- Biological Psychology, Department of Psychology, Department for Medicine and Health Sciences, Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany; Cluster of Excellence "Hearing4all", Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany.
| | - Christiane M Thiel
- Biological Psychology, Department of Psychology, Department for Medicine and Health Sciences, Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany; Cluster of Excellence "Hearing4all", Carl von Ossietzky Universität Oldenburg, Oldenburg, Germany
| |
Collapse
|
29
|
Does hearing aid use affect audiovisual integration in mild hearing impairment? Exp Brain Res 2018; 236:1161-1179. [PMID: 29453491 DOI: 10.1007/s00221-018-5206-6] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2017] [Accepted: 02/11/2018] [Indexed: 10/18/2022]
Abstract
There is converging evidence for altered audiovisual integration abilities in hearing-impaired individuals and those with profound hearing loss who are provided with cochlear implants, compared to normal-hearing adults. Still, little is known on the effects of hearing aid use on audiovisual integration in mild hearing loss, although this constitutes one of the most prevalent conditions in the elderly and, yet, often remains untreated in its early stages. This study investigated differences in the strength of audiovisual integration between elderly hearing aid users and those with the same degree of mild hearing loss who were not using hearing aids, the non-users, by measuring their susceptibility to the sound-induced flash illusion. We also explored the corresponding window of integration by varying the stimulus onset asynchronies. To examine general group differences that are not attributable to specific hearing aid settings but rather reflect overall changes associated with habitual hearing aid use, the group of hearing aid users was tested unaided while individually controlling for audibility. We found greater audiovisual integration together with a wider window of integration in hearing aid users compared to their age-matched untreated peers. Signal detection analyses indicate that a change in perceptual sensitivity as well as in bias may underlie the observed effects. Our results and comparisons with other studies in normal-hearing older adults suggest that both mild hearing impairment and hearing aid use seem to affect audiovisual integration, possibly in the sense that hearing aid use may reverse the effects of hearing loss on audiovisual integration. We suggest that these findings may be particularly important for auditory rehabilitation and call for a longitudinal study.
Collapse
|
30
|
Ross LA, Del Bene VA, Molholm S, Woo YJ, Andrade GN, Abrahams BS, Foxe JJ. Common variation in the autism risk gene CNTNAP2, brain structural connectivity and multisensory speech integration. BRAIN AND LANGUAGE 2017; 174:50-60. [PMID: 28738218 DOI: 10.1016/j.bandl.2017.07.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2017] [Revised: 04/07/2017] [Accepted: 07/11/2017] [Indexed: 06/07/2023]
Abstract
Three lines of evidence motivated this study. 1) CNTNAP2 variation is associated with autism risk and speech-language development. 2) CNTNAP2 variations are associated with differences in white matter (WM) tracts comprising the speech-language circuitry. 3) Children with autism show impairment in multisensory speech perception. Here, we asked whether an autism risk-associated CNTNAP2 single nucleotide polymorphism in neurotypical adults was associated with multisensory speech perception performance, and whether such a genotype-phenotype association was mediated through white matter tract integrity in speech-language circuitry. Risk genotype at rs7794745 was associated with decreased benefit from visual speech and lower fractional anisotropy (FA) in several WM tracts (right precentral gyrus, left anterior corona radiata, right retrolenticular internal capsule). These structural connectivity differences were found to mediate the effect of genotype on audiovisual speech perception, shedding light on possible pathogenic pathways in autism and biological sources of inter-individual variation in audiovisual speech processing in neurotypicals.
Collapse
Affiliation(s)
- Lars A Ross
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Children's Evaluation and Rehabilitation Center (CERC), Department of Pediatrics, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, NY 10461, USA.
| | - Victor A Del Bene
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Children's Evaluation and Rehabilitation Center (CERC), Department of Pediatrics, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, NY 10461, USA; Ferkauf Graduate School of Psychology Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Sophie Molholm
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Children's Evaluation and Rehabilitation Center (CERC), Department of Pediatrics, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, NY 10461, USA; Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Young Jae Woo
- Department of Genetics, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - Gizely N Andrade
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Children's Evaluation and Rehabilitation Center (CERC), Department of Pediatrics, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, NY 10461, USA
| | - Brett S Abrahams
- Department of Genetics, Albert Einstein College of Medicine, Bronx, NY 10461, USA; Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY 10461, USA
| | - John J Foxe
- The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Children's Evaluation and Rehabilitation Center (CERC), Department of Pediatrics, Albert Einstein College of Medicine & Montefiore Medical Center, Bronx, NY 10461, USA; Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY 10461, USA; Ernest J. Del Monte Institute for Neuroscience, Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY 14642, USA.
| |
Collapse
|
31
|
Stevenson YA, Baum SH, Segers M, Ferber S, Barense MD, Wallace MT. Multisensory speech perception in autism spectrum disorder: From phoneme to whole-word perception. Autism Res 2017; 10:1280-1290. [PMID: 28339177 PMCID: PMC5513806 DOI: 10.1002/aur.1776] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2016] [Revised: 01/19/2017] [Accepted: 02/06/2017] [Indexed: 11/11/2022]
Abstract
Speech perception in noisy environments is boosted when a listener can see the speaker's mouth and integrate the auditory and visual speech information. Autistic children have a diminished capacity to integrate sensory information across modalities, which contributes to core symptoms of autism, such as impairments in social communication. We investigated the abilities of autistic and typically-developing (TD) children to integrate auditory and visual speech stimuli in various signal-to-noise ratios (SNR). Measurements of both whole-word and phoneme recognition were recorded. At the level of whole-word recognition, autistic children exhibited reduced performance in both the auditory and audiovisual modalities. Importantly, autistic children showed reduced behavioral benefit from multisensory integration with whole-word recognition, specifically at low SNRs. At the level of phoneme recognition, autistic children exhibited reduced performance relative to their TD peers in auditory, visual, and audiovisual modalities. However, and in contrast to their performance at the level of whole-word recognition, both autistic and TD children showed benefits from multisensory integration for phoneme recognition. In accordance with the principle of inverse effectiveness, both groups exhibited greater benefit at low SNRs relative to high SNRs. Thus, while autistic children showed typical multisensory benefits during phoneme recognition, these benefits did not translate to typical multisensory benefit of whole-word recognition in noisy environments. We hypothesize that sensory impairments in autistic children raise the SNR threshold needed to extract meaningful information from a given sensory input, resulting in subsequent failure to exhibit behavioral benefits from additional sensory information at the level of whole-word recognition. Autism Res 2017. © 2017 International Society for Autism Research, Wiley Periodicals, Inc. Autism Res 2017, 10: 1280-1290. © 2017 International Society for Autism Research, Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- yan A. Stevenson
- Department of Psychology, Western University, London, ON, Canada
- Brain and Mind Institute, Western University, London, ON, Canada
| | - Sarah H. Baum
- Department of Psychology, University of Washington, Seattle, WA, USA
| | | | - Susanne Ferber
- Dept. of Psychology, University of Toronto, Toronto, ON, Canada
- Rotman Research Institute, Toronto, ON, Canada
| | - Morgan D. Barense
- Dept. of Psychology, University of Toronto, Toronto, ON, Canada
- Rotman Research Institute, Toronto, ON, Canada
| | - Mark T. Wallace
- Vanderbilt Brain Institute, Nashville, TN, USA
- Vanderbilt Kennedy Center, Nashville, TN, USA
- Vanderbilt University, Nashville, TN, USA
- Dept. of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
- Dept. of Psychology, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
32
|
Ozker M, Schepers IM, Magnotti JF, Yoshor D, Beauchamp MS. A Double Dissociation between Anterior and Posterior Superior Temporal Gyrus for Processing Audiovisual Speech Demonstrated by Electrocorticography. J Cogn Neurosci 2017; 29:1044-1060. [PMID: 28253074 DOI: 10.1162/jocn_a_01110] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Human speech can be comprehended using only auditory information from the talker's voice. However, comprehension is improved if the talker's face is visible, especially if the auditory information is degraded as occurs in noisy environments or with hearing loss. We explored the neural substrates of audiovisual speech perception using electrocorticography, direct recording of neural activity using electrodes implanted on the cortical surface. We observed a double dissociation in the responses to audiovisual speech with clear and noisy auditory component within the superior temporal gyrus (STG), a region long known to be important for speech perception. Anterior STG showed greater neural activity to audiovisual speech with clear auditory component, whereas posterior STG showed similar or greater neural activity to audiovisual speech in which the speech was replaced with speech-like noise. A distinct border between the two response patterns was observed, demarcated by a landmark corresponding to the posterior margin of Heschl's gyrus. To further investigate the computational roles of both regions, we considered Bayesian models of multisensory integration, which predict that combining the independent sources of information available from different modalities should reduce variability in the neural responses. We tested this prediction by measuring the variability of the neural responses to single audiovisual words. Posterior STG showed smaller variability than anterior STG during presentation of audiovisual speech with noisy auditory component. Taken together, these results suggest that posterior STG but not anterior STG is important for multisensory integration of noisy auditory and visual speech.
Collapse
Affiliation(s)
- Muge Ozker
- 1 University of Texas Graduate School of Biomedical Sciences at Houston.,2 Baylor College of Medicine
| | | | | | | | | |
Collapse
|
33
|
Haß K, Sinke C, Reese T, Roy M, Wiswede D, Dillo W, Oranje B, Szycik GR. Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion. Cogn Neuropsychiatry 2017; 22:145-158. [PMID: 28253091 DOI: 10.1080/13546805.2017.1287693] [Citation(s) in RCA: 30] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
INTRODUCTION In the present study we were interested in the processing of audio-visual integration in schizophrenia compared to healthy controls. The amount of sound-induced double-flash illusions served as an indicator of audio-visual integration. We expected an altered integration as well as a different window of temporal integration for patients. METHODS Fifteen schizophrenia patients and 15 healthy volunteers matched for age and gender were included in this study. We used stimuli with eight different temporal delays (stimulus onset asynchronys (SOAs) 25, 50, 75, 100, 125, 150, 200 and 300 ms) to induce a double-flash illusion. Group differences and the widths of temporal integration windows were calculated on percentages of reported double-flash illusions. RESULTS Patients showed significantly more illusions (ca. 36-44% vs. 9-16% in control subjects) for SOAs 150-300. The temporal integration window for control participants went from SOAs 25 to 200 whereas for patients integration was found across all included temporal delays. We found no significant relationship between the amount of illusions and either illness severity, chlorpromazine equivalent doses or duration of illness in patients. CONCLUSIONS Our results are interpreted in favour of an enlarged temporal integration window for audio-visual stimuli in schizophrenia patients, which is consistent with previous research.
Collapse
Affiliation(s)
- Katharina Haß
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Christopher Sinke
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Tanya Reese
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Mandy Roy
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Daniel Wiswede
- b Department of Neurology , University of Lübeck , Lübeck , Germany
| | - Wolfgang Dillo
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| | - Bob Oranje
- c Department of Psychiatry , University Medical Centre , Utrecht , The Netherlands
| | - Gregor R Szycik
- a Clinic for Psychiatry, Social Psychiatry and Psychotherapy , Hannover Medical School , Hanover , Germany
| |
Collapse
|
34
|
Speech-in-noise perception in musicians: A review. Hear Res 2017; 352:49-69. [PMID: 28213134 DOI: 10.1016/j.heares.2017.02.006] [Citation(s) in RCA: 83] [Impact Index Per Article: 11.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2016] [Revised: 02/01/2017] [Accepted: 02/05/2017] [Indexed: 11/23/2022]
Abstract
The ability to understand speech in the presence of competing sound sources is an important neuroscience question in terms of how the nervous system solves this computational problem. It is also a critical clinical problem that disproportionally affects the elderly, children with language-related learning disorders, and those with hearing loss. Recent evidence that musicians have an advantage on this multifaceted skill has led to the suggestion that musical training might be used to improve or delay the decline of speech-in-noise (SIN) function. However, enhancements have not been universally reported, nor have the relative contributions of different bottom-up versus top-down processes, and their relation to preexisting factors been disentangled. This information that would be helpful to establish whether there is a real effect of experience, what exactly is its nature, and how future training-based interventions might target the most relevant components of cognitive processes. These questions are complicated by important differences in study design and uneven coverage of neuroimaging modality. In this review, we aim to systematize recent results from studies that have specifically looked at musician-related differences in SIN by their study design properties, to summarize the findings, and to identify knowledge gaps for future work.
Collapse
|
35
|
Gao Y, Wang Q, Ding Y, Wang C, Li H, Wu X, Qu T, Li L. Selective Attention Enhances Beta-Band Cortical Oscillation to Speech under "Cocktail-Party" Listening Conditions. Front Hum Neurosci 2017; 11:34. [PMID: 28239344 PMCID: PMC5300994 DOI: 10.3389/fnhum.2017.00034] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2016] [Accepted: 01/16/2017] [Indexed: 11/16/2022] Open
Abstract
Human listeners are able to selectively attend to target speech in a noisy environment with multiple-people talking. Using recordings of scalp electroencephalogram (EEG), this study investigated how selective attention facilitates the cortical representation of target speech under a simulated “cocktail-party” listening condition with speech-on-speech masking. The result shows that the cortical representation of target-speech signals under the multiple-people talking condition was specifically improved by selective attention relative to the non-selective-attention listening condition, and the beta-band activity was most strongly modulated by selective attention. Moreover, measured with the Granger Causality value, selective attention to the single target speech in the mixed-speech complex enhanced the following four causal connectivities for the beta-band oscillation: the ones (1) from site FT7 to the right motor area, (2) from the left frontal area to the right motor area, (3) from the central frontal area to the right motor area, and (4) from the central frontal area to the right frontal area. However, the selective-attention-induced change in beta-band causal connectivity from the central frontal area to the right motor area, but not other beta-band causal connectivities, was significantly correlated with the selective-attention-induced change in the cortical beta-band representation of target speech. These findings suggest that under the “cocktail-party” listening condition, the beta-band oscillation in EEGs to target speech is specifically facilitated by selective attention to the target speech that is embedded in the mixed-speech complex. The selective attention-induced unmasking of target speech may be associated with the improved beta-band functional connectivity from the central frontal area to the right motor area, suggesting a top-down attentional modulation of the speech-motor process.
Collapse
Affiliation(s)
- Yayue Gao
- Beijing Key Laboratory of Behavior and Mental Health, School of Psychological and Cognitive Sciences, Peking University Beijing, China
| | - Qian Wang
- Beijing Key Laboratory of Behavior and Mental Health, School of Psychological and Cognitive Sciences, Peking University Beijing, China
| | - Yu Ding
- Beijing Key Laboratory of Behavior and Mental Health, School of Psychological and Cognitive Sciences, Peking University Beijing, China
| | - Changming Wang
- Beijing Anding Hospital, Capital Medical UniversityBeijing, China; Beijing Institute for Brain Disorders, Capital Medical UniversityBeijing, China
| | - Haifeng Li
- School of Computer Science and Technology, Harbin Institute of Technology Harbin, China
| | - Xihong Wu
- Department of Machine Intelligence, Peking UniversityBeijing, China; Key Laboratory on Machine Perception - Ministry of Education, Speech and Hearing Research Center, Peking UniversityBeijing, China
| | - Tianshu Qu
- Department of Machine Intelligence, Peking UniversityBeijing, China; Key Laboratory on Machine Perception - Ministry of Education, Speech and Hearing Research Center, Peking UniversityBeijing, China
| | - Liang Li
- Beijing Key Laboratory of Behavior and Mental Health, School of Psychological and Cognitive Sciences, Peking UniversityBeijing, China; Beijing Institute for Brain Disorders, Capital Medical UniversityBeijing, China; Department of Machine Intelligence, Peking UniversityBeijing, China
| |
Collapse
|
36
|
Left Superior Temporal Gyrus Is Coupled to Attended Speech in a Cocktail-Party Auditory Scene. J Neurosci 2016; 36:1596-606. [PMID: 26843641 DOI: 10.1523/jneurosci.1730-15.2016] [Citation(s) in RCA: 70] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
UNLABELLED Using a continuous listening task, we evaluated the coupling between the listener's cortical activity and the temporal envelopes of different sounds in a multitalker auditory scene using magnetoencephalography and corticovocal coherence analysis. Neuromagnetic signals were recorded from 20 right-handed healthy adult humans who listened to five different recorded stories (attended speech streams), one without any multitalker background (No noise) and four mixed with a "cocktail party" multitalker background noise at four signal-to-noise ratios (5, 0, -5, and -10 dB) to produce speech-in-noise mixtures, here referred to as Global scene. Coherence analysis revealed that the modulations of the attended speech stream, presented without multitalker background, were coupled at ∼0.5 Hz to the activity of both superior temporal gyri, whereas the modulations at 4-8 Hz were coupled to the activity of the right supratemporal auditory cortex. In cocktail party conditions, with the multitalker background noise, the coupling was at both frequencies stronger for the attended speech stream than for the unattended Multitalker background. The coupling strengths decreased as the Multitalker background increased. During the cocktail party conditions, the ∼0.5 Hz coupling became left-hemisphere dominant, compared with bilateral coupling without the multitalker background, whereas the 4-8 Hz coupling remained right-hemisphere lateralized in both conditions. The brain activity was not coupled to the multitalker background or to its individual talkers. The results highlight the key role of listener's left superior temporal gyri in extracting the slow ∼0.5 Hz modulations, likely reflecting the attended speech stream within a multitalker auditory scene. SIGNIFICANCE STATEMENT When people listen to one person in a "cocktail party," their auditory cortex mainly follows the attended speech stream rather than the entire auditory scene. However, how the brain extracts the attended speech stream from the whole auditory scene and how increasing background noise corrupts this process is still debated. In this magnetoencephalography study, subjects had to attend a speech stream with or without multitalker background noise. Results argue for frequency-dependent cortical tracking mechanisms for the attended speech stream. The left superior temporal gyrus tracked the ∼0.5 Hz modulations of the attended speech stream only when the speech was embedded in multitalker background, whereas the right supratemporal auditory cortex tracked 4-8 Hz modulations during both noiseless and cocktail-party conditions.
Collapse
|
37
|
Erickson LC, Rauschecker JP, Turkeltaub PE. Meta-analytic connectivity modeling of the human superior temporal sulcus. Brain Struct Funct 2016; 222:267-285. [PMID: 27003288 DOI: 10.1007/s00429-016-1215-z] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2015] [Accepted: 03/06/2016] [Indexed: 12/11/2022]
Abstract
The superior temporal sulcus (STS) is a critical region for multiple neural processes in the human brain Hein and Knight (J Cogn Neurosci 20(12): 2125-2136, 2008). To better understand the multiple functions of the STS it would be useful to know more about its consistent functional coactivations with other brain regions. We used the meta-analytic connectivity modeling technique to determine consistent functional coactivation patterns across experiments and behaviors associated with bilateral anterior, middle, and posterior anatomical STS subregions. Based on prevailing models for the cortical organization of audition and language, we broadly hypothesized that across various behaviors the posterior STS (pSTS) would coactivate with dorsal-stream regions, whereas the anterior STS (aSTS) would coactivate with ventral-stream regions. The results revealed distinct coactivation patterns for each STS subregion, with some overlap in the frontal and temporal areas, and generally similar coactivation patterns for the left and right STS. Quantitative comparison of STS subregion coactivation maps demonstrated that the pSTS coactivated more strongly than other STS subregions in the same hemisphere with dorsal-stream regions, such as the inferior parietal lobule (only left pSTS), homotopic pSTS, precentral gyrus and supplementary motor area. In contrast, the aSTS showed more coactivation with some ventral-stream regions, such as the homotopic anterior temporal cortex and left inferior frontal gyrus, pars orbitalis (only right aSTS). These findings demonstrate consistent coactivation maps across experiments and behaviors for different anatomical STS subregions, which may help future studies consider various STS functions in the broader context of generalized coactivations for individuals with and without neurological disorders.
Collapse
Affiliation(s)
- Laura C Erickson
- Neurology Department, Georgetown University Medical Center, 4000 Reservoir Road NW, Building D, Suite 165, Washington, DC, 20057, USA.,Neuroscience Department, Georgetown University Medical Center, 3900 Reservoir Road NW, New Research Building, Room WP19, Washington, DC, 20057, USA
| | - Josef P Rauschecker
- Neuroscience Department, Georgetown University Medical Center, 3900 Reservoir Road NW, New Research Building, Room WP19, Washington, DC, 20057, USA.,Institute for Advanced Study, Technische Universität München, Lichtenbergstraße 2, 85748, Garching bei München, Germany
| | - Peter E Turkeltaub
- Neurology Department, Georgetown University Medical Center, 4000 Reservoir Road NW, Building D, Suite 165, Washington, DC, 20057, USA. .,Research Division, MedStar National Rehabilitation Hospital, 102 Irving St NW, Washington, DC, 20010, USA.
| |
Collapse
|
38
|
Zmigrod L, Zmigrod S. On the Temporal Precision of Thought: Individual Differences in the Multisensory Temporal Binding Window Predict Performance on Verbal and Nonverbal Problem Solving Tasks. Multisens Res 2016. [DOI: 10.1163/22134808-00002532] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
Although psychology is greatly preoccupied by the tight link between the way that individuals perceive the world and their intelligent, creative behavior, there is little experimental work on the relationship between individual differences in perception and cognitive ability in healthy populations. Here, individual differences in problem solving ability were examined in relation to multisensory perception as measured by tolerance for temporal asynchrony between auditory and visual inputs, i.e., the multisensory temporal binding window. The results demonstrated that enhanced performance in both verbal and nonverbal problem solving tasks (the Remote Associates Test and Raven’s Advanced Progressive Matrices Task) is predicted by a narrower audio-visual temporal binding window, which reflects greater sensitivity to subtle discrepancies in sensory inputs. This suggests that the precision of individuals’ temporal window of multisensory integration might mirror their capacities for complex reasoning and thus the precision of their thoughts.
Collapse
Affiliation(s)
- Leor Zmigrod
- Department of Psychology, University of Cambridge, Cambridge, UK
| | - Sharon Zmigrod
- Institute for Psychological Research & Leiden Institute for Brain and Cognition, Leiden University, Leiden, The Netherlands
| |
Collapse
|
39
|
Stevenson RA, Segers M, Ferber S, Barense MD, Camarata S, Wallace MT. Keeping time in the brain: Autism spectrum disorder and audiovisual temporal processing. Autism Res 2015; 9:720-38. [PMID: 26402725 DOI: 10.1002/aur.1566] [Citation(s) in RCA: 57] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2015] [Revised: 08/22/2015] [Accepted: 08/29/2015] [Indexed: 12/21/2022]
Abstract
A growing area of interest and relevance in the study of autism spectrum disorder (ASD) focuses on the relationship between multisensory temporal function and the behavioral, perceptual, and cognitive impairments observed in ASD. Atypical sensory processing is becoming increasingly recognized as a core component of autism, with evidence of atypical processing across a number of sensory modalities. These deviations from typical processing underscore the value of interpreting ASD within a multisensory framework. Furthermore, converging evidence illustrates that these differences in audiovisual processing may be specifically related to temporal processing. This review seeks to bridge the connection between temporal processing and audiovisual perception, and to elaborate on emerging data showing differences in audiovisual temporal function in autism. We also discuss the consequence of such changes, the specific impact on the processing of different classes of audiovisual stimuli (e.g. speech vs. nonspeech, etc.), and the presumptive brain processes and networks underlying audiovisual temporal integration. Finally, possible downstream behavioral implications, and possible remediation strategies are outlined. Autism Res 2016, 9: 720-738. © 2015 International Society for Autism Research, Wiley Periodicals, Inc.
Collapse
Affiliation(s)
- Ryan A Stevenson
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada
| | - Magali Segers
- Department of Psychology, York University, Toronto, Ontario, Canada
| | - Susanne Ferber
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Toronto, Ontario, Canada
| | - Morgan D Barense
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada.,Rotman Research Institute, Toronto, Ontario, Canada
| | - Stephen Camarata
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, Tennessee
| | - Mark T Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Kennedy Center, Vanderbilt University Medical Center, Nashville, Tennessee.,Vanderbilt Brain Institute, Vanderbilt University Medical Center, Nashville, Tennessee.,Department of Psychology, Vanderbilt University, Nashville, Tennessee.,Department of Psychiatry, Vanderbilt University Medical Center, Nashville, Tennessee
| |
Collapse
|
40
|
Chen Z, Yuan W. Central plasticity and dysfunction elicited by aural deprivation in the critical period. Front Neural Circuits 2015; 9:26. [PMID: 26082685 PMCID: PMC4451366 DOI: 10.3389/fncir.2015.00026] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2014] [Accepted: 05/13/2015] [Indexed: 12/31/2022] Open
Abstract
The acoustic signal is crucial for animals to obtain information from the surrounding environment. Like other sensory modalities, the central auditory system undergoes adaptive changes (i.e., plasticity) during the developmental stage as well as other stages of life. Owing to its plasticity, auditory centers may be susceptible to various factors, such as medical intervention, variation in ambient acoustic signals and lesion of the peripheral hearing organ. There are critical periods during which auditory centers are vulnerable to abnormal experiences. Particularly in the early postnatal development period, aural inputs are essential for functional maturity of auditory centers. An aural deprivation model, which can be achieved by attenuating or blocking the peripheral acoustic afferent input to the auditory center, is ideal for investigating plastic changes of auditory centers. Generally, auditory plasticity includes structural and functional changes, some of which can be irreversible. Aural deprivation can distort tonotopic maps, disrupt the binaural integration, reorganize the neural network and change the synaptic transmission in the primary auditory cortex or at lower levels of the auditory system. The regulation of specific gene expression and the modified signal pathway may be the deep molecular mechanism of these plastic changes. By studying this model, researchers may explore the pathogenesis of hearing loss and reveal plastic changes of the auditory cortex, facilitating the therapeutic advancement in patients with severe hearing loss. After summarizing developmental features of auditory centers in auditory deprived animals and discussing changes of central auditory remodeling in hearing loss patients, we aim at stressing the significant of an early and well-designed auditory training program for the hearing rehabilitation.
Collapse
Affiliation(s)
- Zhiji Chen
- Department of Otorhinolaryngology Head and Neck Surgery, Southwest Hospital, Third Military Medical University Chongqing, China
| | - Wei Yuan
- Department of Otorhinolaryngology Head and Neck Surgery, Southwest Hospital, Third Military Medical University Chongqing, China
| |
Collapse
|
41
|
Baum SH, Stevenson RA, Wallace MT. Testing sensory and multisensory function in children with autism spectrum disorder. J Vis Exp 2015:e52677. [PMID: 25938209 PMCID: PMC4541595 DOI: 10.3791/52677] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/31/2022] Open
Abstract
In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are now recognized as a core symptom in autism spectrum disorder (ASD). Our ability to perceive and interact with the external world is rooted in sensory processing. For example, listening to a conversation entails processing the auditory cues coming from the speaker (speech content, prosody, syntax) as well as the associated visual information (facial expressions, gestures). Collectively, the "integration" of these multisensory (i.e., combined audiovisual) pieces of information results in better comprehension. Such multisensory integration has been shown to be strongly dependent upon the temporal relationship of the paired stimuli. Thus, stimuli that occur in close temporal proximity are highly likely to result in behavioral and perceptual benefits--gains believed to be reflective of the perceptual system's judgment of the likelihood that these two stimuli came from the same source. Changes in this temporal integration are expected to strongly alter perceptual processes, and are likely to diminish the ability to accurately perceive and interact with our world. Here, a battery of tasks designed to characterize various aspects of sensory and multisensory temporal processing in children with ASD is described. In addition to its utility in autism, this battery has great potential for characterizing changes in sensory function in other clinical populations, as well as being used to examine changes in these processes across the lifespan.
Collapse
Affiliation(s)
- Sarah H Baum
- Vanderbilt Brain Institute, Vanderbilt University Medical Center
| | | | - Mark T Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University;
| |
Collapse
|
42
|
Bhat J, Miller LM, Pitt MA, Shahin AJ. Putative mechanisms mediating tolerance for audiovisual stimulus onset asynchrony. J Neurophysiol 2015; 113:1437-50. [PMID: 25505102 DOI: 10.1152/jn.00200.2014] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Audiovisual (AV) speech perception is robust to temporal asynchronies between visual and auditory stimuli. We investigated the neural mechanisms that facilitate tolerance for audiovisual stimulus onset asynchrony (AVOA) with EEG. Individuals were presented with AV words that were asynchronous in onsets of voice and mouth movement and judged whether they were synchronous or not. Behaviorally, individuals tolerated (perceived as synchronous) longer AVOAs when mouth movement preceded the speech (V-A) stimuli than when the speech preceded mouth movement (A-V). Neurophysiologically, the P1-N1-P2 auditory evoked potentials (AEPs), time-locked to sound onsets and known to arise in and surrounding the primary auditory cortex (PAC), were smaller for the in-sync than the out-of-sync percepts. Spectral power of oscillatory activity in the beta band (14–30 Hz) following the AEPs was larger during the in-sync than out-of-sync perception for both A-V and V-A conditions. However, alpha power (8–14 Hz), also following AEPs, was larger for the in-sync than out-of-sync percepts only in the V-A condition. These results demonstrate that AVOA tolerance is enhanced by inhibiting low-level auditory activity (e.g., AEPs representing generators in and surrounding PAC) that code for acoustic onsets. By reducing sensitivity to acoustic onsets, visual-to-auditory onset mapping is weakened, allowing for greater AVOA tolerance. In contrast, beta and alpha results suggest the involvement of higher-level neural processes that may code for language cues (phonetic, lexical), selective attention, and binding of AV percepts, allowing for wider neural windows of temporal integration, i.e., greater AVOA tolerance.
Collapse
Affiliation(s)
- Jyoti Bhat
- Department of Otolaryngology—Head and Neck Surgery, The Ohio State University, College of Medicine, Columbus, Ohio
| | - Lee M. Miller
- Center for Mind and Brain, University of California, Davis, California
- Department of Neurobiology, Physiology, and Behavior, University of California, Davis, California; and
| | - Mark A. Pitt
- Department of Psychology, The Ohio State University, Columbus, Ohio
| | - Antoine J. Shahin
- Department of Otolaryngology—Head and Neck Surgery, The Ohio State University, College of Medicine, Columbus, Ohio
- Center for Mind and Brain, University of California, Davis, California
| |
Collapse
|
43
|
Lin Y, Liu B, Liu Z, Gao X. EEG gamma-band activity during audiovisual speech comprehension in different noise environments. Cogn Neurodyn 2015; 9:389-98. [PMID: 26157512 DOI: 10.1007/s11571-015-9333-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2014] [Revised: 01/15/2015] [Accepted: 01/23/2015] [Indexed: 10/24/2022] Open
Abstract
The presence of cross-modal stochastic resonance in different noise environments has been proved in previous behavioral and event-related potential studies, while it was still unclear whether the gamma-band oscillation study was another evidence of cross-modal stochastic resonance. The multisensory gain of gamma-band activity between the audiovisual (AV) and auditory-only conditions in different noise environments was analyzed. Videos of face motion articulating words concordant with different levels of pink noise were used as stimuli. Signal-to-noise ratios (SNRs) of 0, -4, -8, -12 and -16 dB were selected to measure the speech recognition accuracy and EEG activity for 20 healthy subjects. The power and phase of EEG gamma-band oscillations increased in a time window of 50-90 ms. The multisensory gains of evoked and total activity, as well as phase-locking factor, were greatest at the -12 dB SNR, which were consistent with the behavioral result. The multisensory gain of gamma-band activity showed an inverted U-shaped curve as a function of SNR. This finding confirmed the presence of cross-modal stochastic resonance. In addition, there was a significant correlation between evoked activity and phase-locking factor of gamma-band at five different SNRs. Gamma-band oscillation was believed to play a role in the rapid processing and information linkage strengthening of AV modalities in the early stage of cognitive processes.
Collapse
Affiliation(s)
- Yanfei Lin
- School of Medicine, Tsinghua University, Beijing, 100084 People's Republic of China ; School of Information and Electronics, Beijing Institute of Technology, Beijing, 100081 People's Republic of China
| | - Baolin Liu
- School of Computer Science and Technology, Tianjin University, Tianjin, 300072 People's Republic of China
| | - Zhiwen Liu
- School of Information and Electronics, Beijing Institute of Technology, Beijing, 100081 People's Republic of China
| | - Xiaorong Gao
- School of Medicine, Tsinghua University, Beijing, 100084 People's Republic of China
| |
Collapse
|
44
|
McNorgan C, Booth JR. Skill dependent audiovisual integration in the fusiform induces repetition suppression. BRAIN AND LANGUAGE 2015; 141:110-123. [PMID: 25585276 PMCID: PMC4303511 DOI: 10.1016/j.bandl.2014.12.002] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/03/2014] [Revised: 10/27/2014] [Accepted: 12/06/2014] [Indexed: 06/04/2023]
Abstract
Learning to read entails mapping existing phonological representations to novel orthographic representations and is thus an ideal context for investigating experience driven audiovisual integration. Because two dominant brain-based theories of reading development hinge on the sensitivity of the visual-object processing stream to phonological information, we were interested in how reading skill relates to audiovisual integration in this area. Thirty-two children between 8 and 13 years of age spanning a range of reading skill participated in a functional magnetic resonance imaging experiment. Participants completed a rhyme judgment task to word pairs presented unimodally (auditory- or visual-only) and cross-modally (auditory followed by visual). Skill-dependent sub-additive audiovisual modulation was found in left fusiform gyrus, extending into the putative visual word form area, and was correlated with behavioral orthographic priming. These results suggest learning to read promotes facilitatory audiovisual integration in the ventral visual-object processing stream and may optimize this region for orthographic processing.
Collapse
Affiliation(s)
| | - James R Booth
- Northwestern University, United States; The University of Texas at Austin, United States
| |
Collapse
|
45
|
Maidment DW, Kang HJ, Stewart HJ, Amitay S. Audiovisual integration in children listening to spectrally degraded speech. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2015; 58:61-68. [PMID: 25203539 DOI: 10.1044/2014_jslhr-s-14-0044] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/12/2014] [Accepted: 09/03/2014] [Indexed: 06/03/2023]
Abstract
PURPOSE The study explored whether visual information improves speech identification in typically developing children with normal hearing when the auditory signal is spectrally degraded. METHOD Children (n=69) and adults (n=15) were presented with noise-vocoded sentences from the Children's Co-ordinate Response Measure (Rosen, 2011) in auditory-only or audiovisual conditions. The number of bands was adaptively varied to modulate the degradation of the auditory signal, with the number of bands required for approximately 79% correct identification calculated as the threshold. RESULTS The youngest children (4- to 5-year-olds) did not benefit from accompanying visual information, in comparison to 6- to 11-year-old children and adults. Audiovisual gain also increased with age in the child sample. CONCLUSIONS The current data suggest that children younger than 6 years of age do not fully utilize visual speech cues to enhance speech perception when the auditory signal is degraded. This evidence not only has implications for understanding the development of speech perception skills in children with normal hearing but may also inform the development of new treatment and intervention strategies that aim to remediate speech perception difficulties in pediatric cochlear implant users.
Collapse
|
46
|
Stevenson RA, Nelms CE, Baum SH, Zurkovsky L, Barense MD, Newhouse PA, Wallace MT. Deficits in audiovisual speech perception in normal aging emerge at the level of whole-word recognition. Neurobiol Aging 2015; 36:283-91. [PMID: 25282337 PMCID: PMC4268368 DOI: 10.1016/j.neurobiolaging.2014.08.003] [Citation(s) in RCA: 36] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/20/2014] [Revised: 07/22/2014] [Accepted: 08/02/2014] [Indexed: 01/20/2023]
Abstract
Over the next 2 decades, a dramatic shift in the demographics of society will take place, with a rapid growth in the population of older adults. One of the most common complaints with healthy aging is a decreased ability to successfully perceive speech, particularly in noisy environments. In such noisy environments, the presence of visual speech cues (i.e., lip movements) provide striking benefits for speech perception and comprehension, but previous research suggests that older adults gain less from such audiovisual integration than their younger peers. To determine at what processing level these behavioral differences arise in healthy-aging populations, we administered a speech-in-noise task to younger and older adults. We compared the perceptual benefits of having speech information available in both the auditory and visual modalities and examined both phoneme and whole-word recognition across varying levels of signal-to-noise ratio. For whole-word recognition, older adults relative to younger adults showed greater multisensory gains at intermediate SNRs but reduced benefit at low SNRs. By contrast, at the phoneme level both younger and older adults showed approximately equivalent increases in multisensory gain as signal-to-noise ratio decreased. Collectively, the results provide important insights into both the similarities and differences in how older and younger adults integrate auditory and visual speech cues in noisy environments and help explain some of the conflicting findings in previous studies of multisensory speech perception in healthy aging. These novel findings suggest that audiovisual processing is intact at more elementary levels of speech perception in healthy-aging populations and that deficits begin to emerge only at the more complex word-recognition level of speech signals.
Collapse
Affiliation(s)
- Ryan A Stevenson
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada; Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA; Vanderbilt Brain Institute, Nashville, TN, USA; Vanderbilt Kennedy Center, Nashville, TN, USA.
| | - Caitlin E Nelms
- Department of Psychology, Austin Peay State University, Clarksville, TN, USA; Department of Communication Sciences and Disorders, University of Memphis, Memphis, TN, USA
| | - Sarah H Baum
- Vanderbilt Brain Institute, Nashville, TN, USA; Department of Neurobiology and Anatomy, University of Texas Medical School at Houston, TX, USA
| | - Lilia Zurkovsky
- Center for Cognitive Medicine, Department of Psychiatry, Vanderbilt University, Nashville, TN, USA
| | - Morgan D Barense
- Department of Psychology, University of Toronto, Toronto, Ontario, Canada; Rotman Research Institute, Toronto, Ontario, Canada
| | - Paul A Newhouse
- Center for Cognitive Medicine, Department of Psychiatry, Vanderbilt University, Nashville, TN, USA
| | - Mark T Wallace
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA; Vanderbilt Brain Institute, Nashville, TN, USA; Vanderbilt Kennedy Center, Nashville, TN, USA; Center for Cognitive Medicine, Department of Psychiatry, Vanderbilt University, Nashville, TN, USA; Department of Psychology, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
47
|
Wallace MT, Stevenson RA. The construct of the multisensory temporal binding window and its dysregulation in developmental disabilities. Neuropsychologia 2014; 64:105-23. [PMID: 25128432 PMCID: PMC4326640 DOI: 10.1016/j.neuropsychologia.2014.08.005] [Citation(s) in RCA: 195] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2014] [Revised: 08/04/2014] [Accepted: 08/05/2014] [Indexed: 01/18/2023]
Abstract
Behavior, perception and cognition are strongly shaped by the synthesis of information across the different sensory modalities. Such multisensory integration often results in performance and perceptual benefits that reflect the additional information conferred by having cues from multiple senses providing redundant or complementary information. The spatial and temporal relationships of these cues provide powerful statistical information about how these cues should be integrated or "bound" in order to create a unified perceptual representation. Much recent work has examined the temporal factors that are integral in multisensory processing, with many focused on the construct of the multisensory temporal binding window - the epoch of time within which stimuli from different modalities is likely to be integrated and perceptually bound. Emerging evidence suggests that this temporal window is altered in a series of neurodevelopmental disorders, including autism, dyslexia and schizophrenia. In addition to their role in sensory processing, these deficits in multisensory temporal function may play an important role in the perceptual and cognitive weaknesses that characterize these clinical disorders. Within this context, focus on improving the acuity of multisensory temporal function may have important implications for the amelioration of the "higher-order" deficits that serve as the defining features of these disorders.
Collapse
Affiliation(s)
- Mark T Wallace
- Vanderbilt Brain Institute, Vanderbilt University, 465 21st Avenue South, Nashville, TN 37232, USA; Department of Hearing & Speech Sciences, Vanderbilt University, Nashville, TN, USA; Department of Psychology, Vanderbilt University, Nashville, TN, USA; Department of Psychiatry, Vanderbilt University, Nashville, TN, USA.
| | - Ryan A Stevenson
- Department of Psychology, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
48
|
Kamali A, Sair HI, Radmanesh A, Hasan KM. Decoding the superior parietal lobule connections of the superior longitudinal fasciculus/arcuate fasciculus in the human brain. Neuroscience 2014; 277:577-83. [PMID: 25086308 DOI: 10.1016/j.neuroscience.2014.07.035] [Citation(s) in RCA: 67] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2014] [Revised: 07/21/2014] [Accepted: 07/21/2014] [Indexed: 11/26/2022]
Abstract
The temporo-parietal (TP) white matter connections between the inferior parietal lobule and superior temporal gyrus as part of the superior longitudinal fasciculus/arcuate fasciculus (SLF/AF) or middle longitudinal fasciculus (MdLF) have been studied in prior diffusion tensor tractography (DTT) studies. However, few studies have been focusing on the higher TP connections of the superior parietal lobule with the temporal lobe. These higher TP connections have been shown to have a role in core processes such as attention, memory, emotions, and language. Our most recent study, for the first time, hinted to the possibility of a long white matter connection interconnecting the superior parietal lobule (SPL) with the posterior temporal lobe in human brain which we call the SLF/AF TP-SPL and for a shorter abbreviation, the TP-SPL. We decided to further investigate this white matter connection using fiber assignment by continuous tracking deterministic tractography and high spatial resolution diffusion tensor imaging on 3T. Five healthy right-handed men (age range 24-37 years) were studied. We delineated the SPL connections of the SLF/AF TP bilaterally in five normal adult human brains. Using a high resolution DTT technique, we demonstrate for the first time, the trajectory of a long fiber bundle connectivity between the SPL and posterior temporal lobe, called the SLF/AF TP-SPL (or the TP-SPL), bilaterally in five healthy adult human brains. We also demonstrate the trajectory of the vertically oriented posterior TP connections, interconnecting the inferior parietal lobule (IPL) with the posterior temporal lobe (TP-IPL) in relation to the TP-SPL, arcuate fasciculus and other major language pathways. In the current study, for the first time, we categorized the TP connections into the anterior and posterior connectivity groups and subcategorized each one into the SPL or IPL connections.
Collapse
Affiliation(s)
- A Kamali
- Department of Diagnostic Radiology, Division of Neuroradiology, Johns Hopkins University, Baltimore, MD, USA.
| | - H I Sair
- Department of Diagnostic Radiology, Division of Neuroradiology, Johns Hopkins University, Baltimore, MD, USA
| | - A Radmanesh
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - K M Hasan
- Department of Diagnostic Radiology, University of Texas at Houston, Houston, TX, USA
| |
Collapse
|
49
|
Erickson LC, Heeg E, Rauschecker JP, Turkeltaub PE. An ALE meta-analysis on the audiovisual integration of speech signals. Hum Brain Mapp 2014; 35:5587-605. [PMID: 24996043 DOI: 10.1002/hbm.22572] [Citation(s) in RCA: 28] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2013] [Revised: 05/28/2014] [Accepted: 06/24/2014] [Indexed: 11/09/2022] Open
Abstract
The brain improves speech processing through the integration of audiovisual (AV) signals. Situations involving AV speech integration may be crudely dichotomized into those where auditory and visual inputs contain (1) equivalent, complementary signals (validating AV speech) or (2) inconsistent, different signals (conflicting AV speech). This simple framework may allow the systematic examination of broad commonalities and differences between AV neural processes engaged by various experimental paradigms frequently used to study AV speech integration. We conducted an activation likelihood estimation metaanalysis of 22 functional imaging studies comprising 33 experiments, 311 subjects, and 347 foci examining "conflicting" versus "validating" AV speech. Experimental paradigms included content congruency, timing synchrony, and perceptual measures, such as the McGurk effect or synchrony judgments, across AV speech stimulus types (sublexical to sentence). Colocalization of conflicting AV speech experiments revealed consistency across at least two contrast types (e.g., synchrony and congruency) in a network of dorsal stream regions in the frontal, parietal, and temporal lobes. There was consistency across all contrast types (synchrony, congruency, and percept) in the bilateral posterior superior/middle temporal cortex. Although fewer studies were available, validating AV speech experiments were localized to other regions, such as ventral stream visual areas in the occipital and inferior temporal cortex. These results suggest that while equivalent, complementary AV speech signals may evoke activity in regions related to the corroboration of sensory input, conflicting AV speech signals recruit widespread dorsal stream areas likely involved in the resolution of conflicting sensory signals.
Collapse
Affiliation(s)
- Laura C Erickson
- Department of Neurology, Georgetown University Medical Center, Washington, District of Columbia; Department of Neuroscience, Georgetown University Medical Center, Washington, District of Columbia
| | | | | | | |
Collapse
|
50
|
Bhat J, Pitt MA, Shahin AJ. Visual context due to speech-reading suppresses the auditory response to acoustic interruptions in speech. Front Neurosci 2014; 8:173. [PMID: 25053937 PMCID: PMC4078912 DOI: 10.3389/fnins.2014.00173] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2014] [Accepted: 06/05/2014] [Indexed: 11/23/2022] Open
Abstract
Speech reading enhances auditory perception in noise. One means by which this perceptual facilitation comes about is through information from visual networks reinforcing the encoding of the congruent speech signal by ignoring interfering acoustic signals. We tested this hypothesis neurophysiologically by acquiring EEG while individuals listened to words with a fixed portion of each word replaced by white noise. Congruent (meaningful) or incongruent (reversed frames) mouth movements accompanied the words. Individuals judged whether they heard the words as continuous (illusion) or interrupted (illusion failure) through the noise. We hypothesized that congruent, as opposed to incongruent, mouth movements should further enhance illusory perception by suppressing the auditory cortex's response to interruption onsets and offsets. Indeed, we found that the N1 auditory evoked potential (AEP) to noise onsets and offsets was reduced when individuals experienced the illusion during congruent, but not incongruent, audiovisual streams. This N1 inhibitory effect was most prominent at noise offsets, suggesting that visual influences on auditory perception are instigated to a greater extent during noisy periods. These findings suggest that visual context due to speech-reading disengages (inhibits) neural processes associated with interfering sounds (e.g., noisy interruptions) during speech perception.
Collapse
Affiliation(s)
- Jyoti Bhat
- Department of Otolaryngology-Head and Neck Surgery, College of Medicine, The Ohio State University Columbus, OH, USA
| | - Mark A Pitt
- Department of Psychology, The Ohio State University Columbus, OH, USA
| | - Antoine J Shahin
- Department of Otolaryngology-Head and Neck Surgery, College of Medicine, The Ohio State University Columbus, OH, USA
| |
Collapse
|