1
|
Ning M, Duwadi S, Yücel MA, von Lühmann A, Boas DA, Sen K. fNIRS dataset during complex scene analysis. Front Hum Neurosci 2024; 18:1329086. [PMID: 38576451 PMCID: PMC10991699 DOI: 10.3389/fnhum.2024.1329086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 03/06/2024] [Indexed: 04/06/2024] Open
Affiliation(s)
- Matthew Ning
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
| | - Sudan Duwadi
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
| | - Meryem A. Yücel
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
| | - Alexander von Lühmann
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
- BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
- Intelligent Biomedical Sensing (IBS) Lab, Technical University Berlin, Berlin, Germany
| | - David A. Boas
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
| | - Kamal Sen
- Department of Biomedical Engineering, Neurophotonics Center, Boston University, Boston, MA, United States
| |
Collapse
|
2
|
Ning M, Duwadi S, Yücel MA, Von Lühmann A, Boas DA, Sen K. fNIRS Dataset During Complex Scene Analysis. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.01.23.576715. [PMID: 38328139 PMCID: PMC10849700 DOI: 10.1101/2024.01.23.576715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/09/2024]
Abstract
When analyzing complex scenes, humans often focus their attention on an object at a particular spatial location. The ability to decode the attended spatial location would facilitate brain computer interfaces for complex scene analysis (CSA). Here, we investigated capability of functional near-infrared spectroscopy (fNIRS) to decode audio-visual spatial attention in the presence of competing stimuli from multiple locations. We targeted dorsal frontoparietal network including frontal eye field (FEF) and intra-parietal sulcus (IPS) as well as superior temporal gyrus/planum temporal (STG/PT). They all were shown in previous functional magnetic resonance imaging (fMRI) studies to be activated by auditory, visual, or audio-visual spatial tasks. To date, fNIRS has not been applied to decode auditory and visual-spatial attention during CSA, and thus, no such dataset exists yet. This report provides an open-access fNIRS dataset that can be used to develop, test, and compare machine learning algorithms for classifying attended locations based on the fNIRS signals on a single trial basis.
Collapse
Affiliation(s)
- Matthew Ning
- Berenson-Allen Center for Noninvasive Brain Stimulation, Beth Israel Deaconess Medical Center, Boston, MA, USA
| | - Sudan Duwadi
- Neurophotonics Center, Department of Biomedical Engineering, Boston University
| | - Meryem A. Yücel
- Neurophotonics Center, Department of Biomedical Engineering, Boston University
| | - Alexander Von Lühmann
- Neurophotonics Center, Department of Biomedical Engineering, Boston University
- BIFOLD – Berlin Institute for the Foundations of Learning and Data, 10587 Berlin, Germany
- Intelligent Biomedical Sensing (IBS) Lab, Technische Universität Berlin, 10587 Berlin, Germany
| | - David A. Boas
- Neurophotonics Center, Department of Biomedical Engineering, Boston University
| | - Kamal Sen
- Neurophotonics Center, Department of Biomedical Engineering, Boston University
| |
Collapse
|
3
|
Zhou X, Sobczak GS, McKay CM, Litovsky RY. Effects of degraded speech processing and binaural unmasking investigated using functional near-infrared spectroscopy (fNIRS). PLoS One 2022; 17:e0267588. [PMID: 35468160 PMCID: PMC9037936 DOI: 10.1371/journal.pone.0267588] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Accepted: 04/11/2022] [Indexed: 12/24/2022] Open
Abstract
The present study aimed to investigate the effects of degraded speech perception and binaural unmasking using functional near-infrared spectroscopy (fNIRS). Normal hearing listeners were tested when attending to unprocessed or vocoded speech, presented to the left ear at two speech-to-noise ratios (SNRs). Additionally, by comparing monaural versus diotic masker noise, we measured binaural unmasking. Our primary research question was whether the prefrontal cortex and temporal cortex responded differently to varying listening configurations. Our a priori regions of interest (ROIs) were located at the left dorsolateral prefrontal cortex (DLPFC) and auditory cortex (AC). The left DLPFC has been reported to be involved in attentional processes when listening to degraded speech and in spatial hearing processing, while the AC has been reported to be sensitive to speech intelligibility. Comparisons of cortical activity between these two ROIs revealed significantly different fNIRS response patterns. Further, we showed a significant and positive correlation between self-reported task difficulty levels and fNIRS responses in the DLPFC, with a negative but non-significant correlation for the left AC, suggesting that the two ROIs played different roles in effortful speech perception. Our secondary question was whether activity within three sub-regions of the lateral PFC (LPFC) including the DLPFC was differentially affected by varying speech-noise configurations. We found significant effects of spectral degradation and SNR, and significant differences in fNIRS response amplitudes between the three regions, but no significant interaction between ROI and speech type, or between ROI and SNR. When attending to speech with monaural and diotic noises, participants reported the latter conditions being easier; however, no significant main effect of masker condition on cortical activity was observed. For cortical responses in the LPFC, a significant interaction between SNR and masker condition was observed. These findings suggest that binaural unmasking affects cortical activity through improving speech reception threshold in noise, rather than by reducing effort exerted.
Collapse
Affiliation(s)
- Xin Zhou
- Waisman Center, University of Wisconsin-Madison, Madison, WI, United States of America
| | - Gabriel S. Sobczak
- School of Medicine and Public Health, University of Wisconsin-Madison, Madison, WI, United States of America
| | - Colette M. McKay
- The Bionics Institute of Australia, Melbourne, VIC, Australia
- Department of Medical Bionics, University of Melbourne, Melbourne, VIC, Australia
| | - Ruth Y. Litovsky
- Waisman Center, University of Wisconsin-Madison, Madison, WI, United States of America
- Department of Communication Science and Disorders, University of Wisconsin-Madison, Madison, WI, United States of America
- Division of Otolaryngology, Department of Surgery, University of Wisconsin-Madison, Madison, WI, United States of America
| |
Collapse
|
4
|
Cui J, Sawamura D, Sakuraba S, Saito R, Tanabe Y, Miura H, Sugi M, Yoshida K, Watanabe A, Tokikuni Y, Yoshida S, Sakai S. Effect of Audiovisual Cross-Modal Conflict during Working Memory Tasks: A Near-Infrared Spectroscopy Study. Brain Sci 2022; 12:brainsci12030349. [PMID: 35326305 PMCID: PMC8946709 DOI: 10.3390/brainsci12030349] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 03/01/2022] [Accepted: 03/01/2022] [Indexed: 12/04/2022] Open
Abstract
Cognitive conflict effects are well characterized within unimodality. However, little is known about cross-modal conflicts and their neural bases. This study characterizes the two types of visual and auditory cross-modal conflicts through working memory tasks and brain activities. The participants consisted of 31 healthy, right-handed, young male adults. The Paced Auditory Serial Addition Test (PASAT) and the Paced Visual Serial Addition Test (PVSAT) were performed under distractor and no distractor conditions. Distractor conditions comprised two conditions in which either the PASAT or PVSAT was the target task, and the other was used as a distractor stimulus. Additionally, oxygenated hemoglobin (Oxy-Hb) concentration changes in the frontoparietal regions were measured during tasks. The results showed significantly lower PASAT performance under distractor conditions than under no distractor conditions, but not in the PVSAT. Oxy-Hb changes in the bilateral ventrolateral prefrontal cortex (VLPFC) and inferior parietal cortex (IPC) significantly increased in the PASAT with distractor compared with no distractor conditions, but not in the PVSAT. Furthermore, there were significant positive correlations between Δtask performance accuracy and ΔOxy-Hb in the bilateral IPC only in the PASAT. Visual cross-modal conflict significantly impairs auditory task performance, and bilateral VLPFC and IPC are key regions in inhibiting visual cross-modal distractors.
Collapse
Affiliation(s)
- Jiahong Cui
- Graduate School of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (J.C.); (R.S.); (H.M.); (A.W.); (Y.T.)
| | - Daisuke Sawamura
- Department of Rehabilitation Science, Faculty of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (K.Y.); (S.S.)
- Correspondence:
| | - Satoshi Sakuraba
- Department of Rehabilitation Sciences, Health Sciences University of Hokkaido, Sapporo 061-0293, Japan; (S.S.); (S.Y.)
| | - Ryuji Saito
- Graduate School of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (J.C.); (R.S.); (H.M.); (A.W.); (Y.T.)
| | - Yoshinobu Tanabe
- Department of Rehabilitation, Shinsapporo Paulo Hospital, Sapporo 004-0002, Japan;
| | - Hiroshi Miura
- Graduate School of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (J.C.); (R.S.); (H.M.); (A.W.); (Y.T.)
| | - Masaaki Sugi
- Department of Rehabilitation, Tokeidai Memorial Hospital, Sapporo 060-0031, Japan;
| | - Kazuki Yoshida
- Department of Rehabilitation Science, Faculty of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (K.Y.); (S.S.)
| | - Akihiro Watanabe
- Graduate School of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (J.C.); (R.S.); (H.M.); (A.W.); (Y.T.)
| | - Yukina Tokikuni
- Graduate School of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (J.C.); (R.S.); (H.M.); (A.W.); (Y.T.)
| | - Susumu Yoshida
- Department of Rehabilitation Sciences, Health Sciences University of Hokkaido, Sapporo 061-0293, Japan; (S.S.); (S.Y.)
| | - Shinya Sakai
- Department of Rehabilitation Science, Faculty of Health Sciences, Hokkaido University, Sapporo 060-0812, Japan; (K.Y.); (S.S.)
| |
Collapse
|
5
|
Can visual capture of sound separate auditory streams? Exp Brain Res 2022; 240:813-824. [PMID: 35048159 DOI: 10.1007/s00221-021-06281-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 11/21/2021] [Indexed: 11/04/2022]
Abstract
In noisy contexts, sound discrimination improves when the auditory sources are separated in space. This phenomenon, named Spatial Release from Masking (SRM), arises from the interaction between the auditory information reaching the ear and spatial attention resources. To examine the relative contribution of these two factors, we exploited an audio-visual illusion in a hearing-in-noise task to create conditions in which the initial stimulation to the ears is held constant, while the perceived separation between speech and masker is changed illusorily (visual capture of sound). In two experiments, we asked participants to identify a string of five digits pronounced by a female voice, embedded in either energetic (Experiment 1) or informational (Experiment 2) noise, before reporting the perceived location of the heard digits. Critically, the distance between target digits and masking noise was manipulated both physically (from 22.5 to 75.0 degrees) and illusorily, by pairing target sounds with visual stimuli either at same (audio-visual congruent) or different positions (15 degrees offset, leftward or rightward: audio-visual incongruent). The proportion of correctly reported digits increased with the physical separation between the target and masker, as expected from SRM. However, despite effective visual capture of sounds, performance was not modulated by illusory changes of target sound position. Our results are compatible with a limited role of central factors in the SRM phenomenon, at least in our experimental setting. Moreover, they add to the controversial literature on the limited effects of audio-visual capture in auditory stream separation.
Collapse
|
6
|
Zhang M, Alamatsaz N, Ihlefeld A. Hemodynamic Responses Link Individual Differences in Informational Masking to the Vicinity of Superior Temporal Gyrus. Front Neurosci 2021; 15:675326. [PMID: 34366772 PMCID: PMC8339305 DOI: 10.3389/fnins.2021.675326] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2021] [Accepted: 05/13/2021] [Indexed: 01/20/2023] Open
Abstract
Suppressing unwanted background sound is crucial for aural communication. A particularly disruptive type of background sound, informational masking (IM), often interferes in social settings. However, IM mechanisms are incompletely understood. At present, IM is identified operationally: when a target should be audible, based on suprathreshold target/masker energy ratios, yet cannot be heard because target-like background sound interferes. We here confirm that speech identification thresholds differ dramatically between low- vs. high-IM background sound. However, speech detection thresholds are comparable across the two conditions. Moreover, functional near infrared spectroscopy recordings show that task-evoked blood oxygenation changes near the superior temporal gyrus (STG) covary with behavioral speech detection performance for high-IM but not low-IM background sound, suggesting that the STG is part of an IM-dependent network. Moreover, listeners who are more vulnerable to IM show increased hemodynamic recruitment near STG, an effect that cannot be explained based on differences in task difficulty across low- vs. high-IM. In contrast, task-evoked responses near another auditory region of cortex, the caudal inferior frontal sulcus (cIFS), do not predict behavioral sensitivity, suggesting that the cIFS belongs to an IM-independent network. Results are consistent with the idea that cortical gating shapes individual vulnerability to IM.
Collapse
Affiliation(s)
- Min Zhang
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, United States
- Rutgers Biomedical and Health Sciences, Rutgers University, Newark, NJ, United States
| | - Nima Alamatsaz
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, United States
- Rutgers Biomedical and Health Sciences, Rutgers University, Newark, NJ, United States
| | - Antje Ihlefeld
- Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, United States
| |
Collapse
|
7
|
Luke R, Larson E, Shader MJ, Innes-Brown H, Van Yper L, Lee AKC, Sowman PF, McAlpine D. Analysis methods for measuring passive auditory fNIRS responses generated by a block-design paradigm. NEUROPHOTONICS 2021; 8:025008. [PMID: 34036117 PMCID: PMC8140612 DOI: 10.1117/1.nph.8.2.025008] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 04/28/2021] [Indexed: 05/20/2023]
Abstract
Significance: Functional near-infrared spectroscopy (fNIRS) is an increasingly popular tool in auditory research, but the range of analysis procedures employed across studies may complicate the interpretation of data. Aim: We aim to assess the impact of different analysis procedures on the morphology, detection, and lateralization of auditory responses in fNIRS. Specifically, we determine whether averaging or generalized linear model (GLM)-based analysis generates different experimental conclusions when applied to a block-protocol design. The impact of parameter selection of GLMs on detecting auditory-evoked responses was also quantified. Approach: 17 listeners were exposed to three commonly employed auditory stimuli: noise, speech, and silence. A block design, comprising sounds of 5 s duration and 10 to 20 s silent intervals, was employed. Results: Both analysis procedures generated similar response morphologies and amplitude estimates, and both indicated that responses to speech were significantly greater than to noise or silence. Neither approach indicated a significant effect of brain hemisphere on responses to speech. Methods to correct for systemic hemodynamic responses using short channels improved detection at the individual level. Conclusions: Consistent with theoretical considerations, simulations, and other experimental domains, GLM and averaging analyses generate the same group-level experimental conclusions. We release this dataset publicly for use in future development and optimization of algorithms.
Collapse
Affiliation(s)
- Robert Luke
- Macquarie University, Macquarie University Hearing & Department of Linguistics, Australian Hearing Hub, Sydney, New South Wales, Australia
- The Bionics Institute, Melbourne, Victoria, Australia
| | - Eric Larson
- University of Washington, Institute for Learning & Brain Sciences, Seattle, Washington, United States
| | - Maureen J. Shader
- The Bionics Institute, Melbourne, Victoria, Australia
- The University of Melbourne, Department of Medical Bionics, Melbourne, Victoria, Australia
| | - Hamish Innes-Brown
- The University of Melbourne, Department of Medical Bionics, Melbourne, Victoria, Australia
- Eriksholm Research Centre, Oticon A/S, Snekkersten, Denmark
| | - Lindsey Van Yper
- Macquarie University, Macquarie University Hearing & Department of Linguistics, Australian Hearing Hub, Sydney, New South Wales, Australia
| | - Adrian K. C. Lee
- University of Washington, Institute for Learning & Brain Sciences, Seattle, Washington, United States
- University of Washington, Department of Speech & Hearing Sciences and Institute for Learning & Brain Sciences, Seattle, Washington, United States
| | - Paul F. Sowman
- Macquarie University, Department of Cognitive Science, Faculty of Medicine, Health and Human Sciences, Sydney, New South Wales, Australia
| | - David McAlpine
- Macquarie University, Macquarie University Hearing & Department of Linguistics, Australian Hearing Hub, Sydney, New South Wales, Australia
| |
Collapse
|
8
|
Bell L, Peng ZE, Pausch F, Reindl V, Neuschaefer-Rube C, Fels J, Konrad K. fNIRS Assessment of Speech Comprehension in Children with Normal Hearing and Children with Hearing Aids in Virtual Acoustic Environments: Pilot Data and Practical Recommendations. CHILDREN (BASEL, SWITZERLAND) 2020; 7:E219. [PMID: 33171753 PMCID: PMC7695031 DOI: 10.3390/children7110219] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Revised: 11/02/2020] [Accepted: 11/05/2020] [Indexed: 11/16/2022]
Abstract
The integration of virtual acoustic environments (VAEs) with functional near-infrared spectroscopy (fNIRS) offers novel avenues to investigate behavioral and neural processes of speech-in-noise (SIN) comprehension in complex auditory scenes. Particularly in children with hearing aids (HAs), the combined application might offer new insights into the neural mechanism of SIN perception in simulated real-life acoustic scenarios. Here, we present first pilot data from six children with normal hearing (NH) and three children with bilateral HAs to explore the potential applicability of this novel approach. Children with NH received a speech recognition benefit from low room reverberation and target-distractors' spatial separation, particularly when the pitch of the target and the distractors was similar. On the neural level, the left inferior frontal gyrus appeared to support SIN comprehension during effortful listening. Children with HAs showed decreased SIN perception across conditions. The VAE-fNIRS approach is critically compared to traditional SIN assessments. Although the current study shows that feasibility still needs to be improved, the combined application potentially offers a promising tool to investigate novel research questions in simulated real-life listening. Future modified VAE-fNIRS applications are warranted to replicate the current findings and to validate its application in research and clinical settings.
Collapse
Affiliation(s)
- Laura Bell
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany; (V.R.); (K.K.)
| | - Z. Ellen Peng
- Teaching and Research Area of Medical Acoustics, Institute of Technical Acoustics, RWTH Aachen University, 52074 Aachen, Germany; (F.P.); (J.F.)
- Waisman Center, University of Wisconsin-Madison, Madison, WI 53705, USA;
| | - Florian Pausch
- Teaching and Research Area of Medical Acoustics, Institute of Technical Acoustics, RWTH Aachen University, 52074 Aachen, Germany; (F.P.); (J.F.)
| | - Vanessa Reindl
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany; (V.R.); (K.K.)
- JARA-Brain Institute II, Molecular Neuroscience and Neuroimaging, RWTH Aachen & Research Centre Juelich, 52428 Juelich, Germany
| | - Christiane Neuschaefer-Rube
- Clinic of Phoniatrics, Pedaudiology, and Communication Disorders, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany;
| | - Janina Fels
- Teaching and Research Area of Medical Acoustics, Institute of Technical Acoustics, RWTH Aachen University, 52074 Aachen, Germany; (F.P.); (J.F.)
| | - Kerstin Konrad
- Child Neuropsychology Section, Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Medical Faculty, RWTH Aachen University, 52074 Aachen, Germany; (V.R.); (K.K.)
- JARA-Brain Institute II, Molecular Neuroscience and Neuroimaging, RWTH Aachen & Research Centre Juelich, 52428 Juelich, Germany
| |
Collapse
|