Temporal Cortex Activation to Audiovisual Speech in Normal-Hearing and Cochlear Implant Users Measured with Functional Near-Infrared Spectroscopy

Low frequency oscillations reflect neurovascular coupling and disappear after cerebral death

Spectrum power analysis in the low frequency oscillations (LFO) region of functional near infrared spectroscopy (fNIRS) is a promising method to deliver information about brain activation and therefore might be used for prognostication in patients with disorders of consciousness in the neurocritical care unit alongside with established methods. In this study, we measure the cortical hemodynamic response measured by fNIRS in the LFO region following auditory and somatosensory stimulation in healthy subjects. The significant hemodynamic reaction in the contralateral hemisphere correlation with the physiologic electric response suggests neurovascular coupling. In addition, we investigate power spectrum changes in steady state measurements of cerebral death patients and healthy subjects in the LFO region, the frequency of the heartbeat and respiration. The spectral power within the LFO region was lower in the patients with cerebral death compared to the healthy subjects, whereas there were no differences in spectral power for physiological activities such as heartbeat and respiration rate. This finding indicates the cerebral origin of our low frequency measurements. Therefore, LFO measurements are a potential method to detect brain activation in patients with disorders of consciousness and cerebral death. However, further studies in patients are needed to investigate its potential clinical use.

Audiovisual integration in children with cochlear implants revealed through EEG and fNIRS

Sensory deprivation can offset the balance of audio versus visual information in multimodal processing. Such a phenomenon could persist for children born deaf, even after they receive cochlear implants (CIs), and could potentially explain why one modality is given priority over the other. Here, we recorded cortical responses to a single speaker uttering two syllables, presented in audio-only (A), visual-only (V), and audio-visual (AV) modes. Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) were successively recorded in seventy-five school-aged children. Twenty-five were children with normal hearing (NH) and fifty wore CIs, among whom 26 had relatively high language abilities (HL) comparable to those of NH children, while 24 others had low language abilities (LL). In EEG data, visual-evoked potentials were captured in occipital regions, in response to V and AV stimuli, and they were accentuated in the HL group compared to the LL group (the NH group being intermediate). Close to the vertex, auditory-evoked potentials were captured in response to A and AV stimuli and reflected a differential treatment of the two syllables but only in the NH group. None of the EEG metrics revealed any interaction between group and modality. In fNIRS data, each modality induced a corresponding activity in visual or auditory regions, but no group difference was observed in A, V, or AV stimulation. The present study did not reveal any sign of abnormal AV integration in children with CI. An efficient multimodal integrative network (at least for rudimentary speech materials) is clearly not a sufficient condition to exhibit good language and literacy.

fNIRS Studies of Individuals with Speech and Language Impairment Underreport Sociodemographics: A Systematic Review

Functional near-infrared spectroscopy (fNIRS) is a promising tool for scientific discovery and clinical application. However, its utility depends upon replicable reporting. We evaluate reporting of sociodemographics in fNIRS studies of speech and language impairment and asked the following: (1) Do refereed fNIRS publications report participant sociodemographics? (2) For what reasons are participants excluded from analysis? This systematic review was preregistered with PROSPERO (CRD42022342959) and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses protocol. Searches in August 2022 included the terms: (a) fNIRS or functional near-infrared spectroscopy or NIRS or near-infrared spectroscopy, (b) speech or language, and (c) disorder or impairment or delay. Searches yielded 38 qualifying studies from 1997 to present. Eight studies (5%) reported at least partial information on race or ethnicity. Few studies reported SES (26%) or language background (47%). Most studies reported geographic location (100%) and gender/sex (89%). Underreporting of sociodemographics in fNIRS studies of speech and language impairment hinders the generalizability of findings. Replicable reporting is imperative for advancing the utility of fNIRS.

Dynamic networks differentiate the language ability of children with cochlear implants

Background

Cochlear implantation (CI) in prelingually deafened children has been shown to be an effective intervention for developing language and reading skill. However, there is a substantial proportion of the children receiving CI who struggle with language and reading. The current study-one of the first to implement electrical source imaging in CI population was designed to identify the neural underpinnings in two groups of CI children with good and poor language and reading skill.

Methods

Data using high density electroencephalography (EEG) under a resting state condition was obtained from 75 children, 50 with CIs having good (HL) or poor language skills (LL) and 25 normal hearing (NH) children. We identified coherent sources using dynamic imaging of coherent sources (DICS) and their effective connectivity computing time-frequency causality estimation based on temporal partial directed coherence (TPDC) in the two CI groups compared to a cohort of age and gender matched NH children.

Findings

Sources with higher coherence amplitude were observed in three frequency bands (alpha, beta and gamma) for the CI groups when compared to normal hearing children. The two groups of CI children with good (HL) and poor (LL) language ability exhibited not only different cortical and subcortical source profiles but also distinct effective connectivity between them. Additionally, a support vector machine (SVM) algorithm using these sources and their connectivity patterns for each CI group across the three frequency bands was able to predict the language and reading scores with high accuracy.

Interpretation

Increased coherence in the CI groups suggest overall that the oscillatory activity in some brain areas become more strongly coupled compared to the NH group. Moreover, the different sources and their connectivity patterns and their association to language and reading skill in both groups, suggest a compensatory adaptation that either facilitated or impeded language and reading development. The neural differences in the two groups of CI children may reflect potential biomarkers for predicting outcome success in CI children.

Cortical Activation of Swallowing Using fNIRS: A Proof of Concept Study with Healthy Adults

The purpose of this study was to determine whether functional near-infrared spectroscopy (fNIRS) could reliably identify cortical activation patterns as healthy adults engaged in single sip and continuous swallowing tasks. Thirty-three right-handed adults completed two functional swallowing tasks, one control jaw movement task, and one rest task while being imaged with fNIRS. Swallowing tasks included a single sip of 5 mL of water via syringe and continuous straw drinking. fNIRS patches for acquisition of neuroimaging data were placed parallel over left and right hemispheres. Stimuli presentation was controlled with set time intervals and audio instructions. Using a series of linear mixed effect models, results demonstrated clear cortical activation patterns during swallowing. The continuous swallowing task demonstrated significant differences in blood oxygenation and deoxygenation concentration values across nearly all regions examined, but most notably M1 in both hemispheres. Of note is that there were areas of greater activation, particularly on the right hemisphere, when comparing the single sip swallow to the jaw movement control and rest tasks. Results from the current study support the use of fNIRS during investigation of swallowing. The utilization of healthy adults as a method for acquiring normative data is vital for comparison purposes when investigating individuals with disorders, but also in the development of rehabilitation techniques. Identifying activation areas that pertain to swallowing will have important implications for individuals requiring dysphagia therapy.

A direct comparison of voice pitch processing in acoustic and electric hearing

In single-sided deafness patients fitted with a cochlear implant (CI) in the affected ear and preserved normal hearing in the other ear, acoustic and electric hearing can be directly compared without the need for an external control group. Although poor pitch perception is a crucial limitation when listening through CIs, it remains unclear how exactly the cortical processing of pitch information differs between acoustic and electric hearing. Hence, we separately presented both ears of 20 of these patients with vowel sequences in which the pitch contours were either repetitive or variable, while simultaneously recording functional near-infrared spectroscopy (fNIRS) and EEG data. Overall, the results showed smaller and delayed auditory cortex activity in electric hearing, particularly for the P2 event-related potential component, which appears to reflect the processing of voice pitch information. Both the fNIRS data and EEG source reconstructions furthermore showed that vowel sequences with variable pitch contours evoked additional activity in posterior right auditory cortex in electric but not acoustic hearing. This surprising discrepancy demonstrates, firstly, that the acoustic detail transmitted by CIs is sufficient to distinguish between speech sounds that only vary regarding their pitch information. Secondly, the absence of a condition difference when stimulating the normal-hearing ears suggests a saturation of cortical activity levels following unilateral deafness. Taken together, these results provide strong evidence in favour of using CIs in this patient group.

The Effects of Cortical Reorganization and Applications of Functional Near-Infrared Spectroscopy in Deaf People and Cochlear Implant Users

A cochlear implant (CI) is currently the only FDA-approved biomedical device that can restore hearing for the majority of patients with severe-to-profound sensorineural hearing loss (SNHL). While prelingually and postlingually deaf individuals benefit substantially from CI, the outcomes after implantation vary greatly. Numerous studies have attempted to study the variables that affect CI outcomes, including the personal characteristics of CI candidates, environmental variables, and device-related variables. Up to 80% of the results remained unexplainable because all these variables could only roughly predict auditory performance with a CI. Brain structure/function differences after hearing deprivation, that is, cortical reorganization, has gradually attracted the attention of neuroscientists. The cross-modal reorganization in the auditory cortex following deafness is thought to be a key factor in the success of CI. In recent years, the adaptive and maladaptive effects of this reorganization on CI rehabilitation have been argued because the neural mechanisms of how this reorganization impacts CI learning and rehabilitation have not been revealed. Due to the lack of brain processes describing how this plasticity affects CI learning and rehabilitation, the adaptive and deleterious consequences of this reorganization on CI outcomes have recently been the subject of debate. This review describes the evidence for different roles of cross-modal reorganization in CI performance and attempts to explore the possible reasons. Additionally, understanding the core influencing mechanism requires taking into account the cortical changes from deafness to hearing restoration. However, methodological issues have restricted longitudinal research on cortical function in CI. Functional near-infrared spectroscopy (fNIRS) has been increasingly used for the study of brain function and language assessment in CI because of its unique advantages, which are considered to have great potential. Here, we review studies on auditory cortex reorganization in deaf patients and CI recipients, and then we try to illustrate the feasibility of fNIRS as a neuroimaging tool in predicting and assessing speech performance in CI recipients. Here, we review research on the cross-modal reorganization of the auditory cortex in deaf patients and CI recipients and seek to demonstrate the viability of using fNIRS as a neuroimaging technique to predict and evaluate speech function in CI recipients.

Investigating effortful speech perception using fNIRS and pupillometry measures

The current study examined the neural mechanisms for mental effort and its correlation to speech perception using functional near-infrared spectroscopy (fNIRS) in listeners with normal hearing (NH). Data were collected while participants listened and responded to unprocessed and degraded sentences, where words were presented in grammatically correct or shuffled order. Effortful listening and task difficulty due to stimulus manipulations was confirmed using a subjective questionnaire and a well-established objective measure of mental effort - pupillometry. fNIRS measures focused on cortical responses in two a priori regions of interest, the left auditory cortex (AC) and lateral frontal cortex (LFC), which are closely related to auditory speech perception and listening effort, respectively. We examined the relations between the two objective measures and behavioral measures of speech perception (task performance) and task difficulty.

Results

demonstrated that changes in pupil dilation were positively correlated with the self-reported task difficulty levels and negatively correlated with the task performance scores. A significant and negative correlation between the two behavioral measures was also found. That is, as perceived task demands increased and task performance scores decreased, pupils dilated more. fNIRS measures (cerebral oxygenation) in the left AC and LFC were both negatively correlated with the self-reported task difficulty levels and positively correlated with task performance scores. These results suggest that pupillometry measures can indicate task demands and listening effort; whereas, fNIRS measures using a similar paradigm seem to reflect speech processing, but not effort.

Functional localization of audiovisual speech using near infrared spectroscopy

Visual cues are especially vital for hearing impaired individuals such as cochlear implant (CI) users to understand speech in noise. Functional Near Infrared Spectroscopy (fNIRS) is a light-based imaging technology that is ideally suited for measuring the brain activity of CI users due to its compatibility with both the ferromagnetic and electrical components of these implants. In a preliminary step toward better elucidating the behavioral and neural correlates of audiovisual (AV) speech integration in CI users, we designed a speech-in-noise task and measured the extent to which 24 normal hearing individuals could integrate the audio of spoken monosyllabic words with the corresponding visual signals of a female speaker. In our behavioral task, we found that audiovisual pairings provided average improvements of 103% and 197% over auditory-alone listening conditions in -6 and -9 dB signal-to-noise ratios consisting of multi-talker background noise. In an fNIRS task using similar stimuli, we measured activity during auditory-only listening, visual-only lipreading, and AV listening conditions. We identified cortical activity in all three conditions over regions of middle and superior temporal cortex typically associated with speech processing and audiovisual integration. In addition, three channels active during the lipreading condition showed uncorrected correlations associated with behavioral measures of audiovisual gain as well as with the McGurk effect. Further work focusing primarily on the regions of interest identified in this study could test how AV speech integration may differ for CI users who rely on this mechanism for daily communication.

Use of Functional Near-Infrared Spectroscopy to Predict and Measure Cochlear Implant Outcomes: A Scoping Review

Outcomes following cochlear implantation vary widely for both adults and children, and behavioral tests are currently relied upon to assess this. However, these behavioral tests rely on subjective judgements that can be unreliable, particularly for infants and young children. The addition of an objective test of outcome following cochlear implantation is therefore desirable. The aim of this scoping review was to comprehensively catalogue the evidence for the potential of functional near infrared spectroscopy (fNIRS) to be used as a tool to objectively predict and measure cochlear implant outcomes. A scoping review of the literature was conducted following the PRISMA extension for scoping review framework. Searches were conducted in the MEDLINE, EMBASE, PubMed, CINAHL, SCOPUS, and Web of Science electronic databases, with a hand search conducted in Google Scholar. Key terms relating to near infrared spectroscopy and cochlear implants were used to identify relevant publications. Eight records met the criteria for inclusion. Seven records reported on adult populations, with five records only including post-lingually deaf individuals and two including both pre- and post-lingually deaf individuals. Studies were either longitudinal or cross-sectional, and all studies compared fNIRS measurements with receptive speech outcomes. This review identified and collated key work in this field. The homogeneity of the populations studied so far identifies key gaps for future research, including the use of fNIRS in infants. By mapping the literature on this important topic, this review contributes knowledge towards the improvement of outcomes following cochlear implantation.

Multisensory Integration-Attention Trade-Off in Cochlear-Implanted Deaf Individuals

The cochlear implant (CI) allows profoundly deaf individuals to partially recover hearing. Still, due to the coarse acoustic information provided by the implant, CI users have considerable difficulties in recognizing speech, especially in noisy environments. CI users therefore rely heavily on visual cues to augment speech recognition, more so than normal-hearing individuals. However, it is unknown how attention to one (focused) or both (divided) modalities plays a role in multisensory speech recognition. Here we show that unisensory speech listening and reading were negatively impacted in divided-attention tasks for CI users—but not for normal-hearing individuals. Our psychophysical experiments revealed that, as expected, listening thresholds were consistently better for the normal-hearing, while lipreading thresholds were largely similar for the two groups. Moreover, audiovisual speech recognition for normal-hearing individuals could be described well by probabilistic summation of auditory and visual speech recognition, while CI users were better integrators than expected from statistical facilitation alone. Our results suggest that this benefit in integration comes at a cost. Unisensory speech recognition is degraded for CI users when attention needs to be divided across modalities. We conjecture that CI users exhibit an integration-attention trade-off. They focus solely on a single modality during focused-attention tasks, but need to divide their limited attentional resources in situations with uncertainty about the upcoming stimulus modality. We argue that in order to determine the benefit of a CI for speech recognition, situational factors need to be discounted by presenting speech in realistic or complex audiovisual environments.

Enhanced spatiotemporal resolution imaging of neuronal activity using joint electroencephalography and diffuse optical tomography

Significance: Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) are both commonly used methodologies for neuronal source reconstruction. While EEG has high temporal resolution (millisecond-scale), its spatial resolution is on the order of centimeters. On the other hand, in comparison to EEG, fNIRS, or diffuse optical tomography (DOT), when used for source reconstruction, can achieve relatively high spatial resolution (millimeter-scale), but its temporal resolution is poor because the hemodynamics that it measures evolve on the order of several seconds. This has important neuroscientific implications: e.g., if two spatially close neuronal sources are activated sequentially with only a small temporal separation, single-modal measurements using either EEG or DOT alone would fail to resolve them correctly. Aim: We attempt to address this issue by performing joint EEG and DOT neuronal source reconstruction. Approach: We propose an algorithm that utilizes DOT reconstruction as the spatial prior of EEG reconstruction, and demonstrate the improvements using simulations based on the ICBM152 brain atlas. Results: We show that neuronal sources can be reconstructed with higher spatiotemporal resolution using our algorithm than using either modality individually. Further, we study how the performance of the proposed algorithm can be affected by the locations of the neuronal sources, and how the performance can be enhanced by improving the placement of EEG electrodes and DOT optodes. Conclusions: We demonstrate using simulations that two sources separated by 2.3-3.3 cm and 50 ms can be recovered accurately using the proposed algorithm by suitably combining EEG and DOT, but not by either in isolation. We also show that the performance can be enhanced by optimizing the electrode and optode placement according to the locations of the neuronal sources.

fNIRS Assessment of Speech Comprehension in Children with Normal Hearing and Children with Hearing Aids in Virtual Acoustic Environments: Pilot Data and Practical Recommendations

The integration of virtual acoustic environments (VAEs) with functional near-infrared spectroscopy (fNIRS) offers novel avenues to investigate behavioral and neural processes of speech-in-noise (SIN) comprehension in complex auditory scenes. Particularly in children with hearing aids (HAs), the combined application might offer new insights into the neural mechanism of SIN perception in simulated real-life acoustic scenarios. Here, we present first pilot data from six children with normal hearing (NH) and three children with bilateral HAs to explore the potential applicability of this novel approach. Children with NH received a speech recognition benefit from low room reverberation and target-distractors' spatial separation, particularly when the pitch of the target and the distractors was similar. On the neural level, the left inferior frontal gyrus appeared to support SIN comprehension during effortful listening. Children with HAs showed decreased SIN perception across conditions. The VAE-fNIRS approach is critically compared to traditional SIN assessments. Although the current study shows that feasibility still needs to be improved, the combined application potentially offers a promising tool to investigate novel research questions in simulated real-life listening. Future modified VAE-fNIRS applications are warranted to replicate the current findings and to validate its application in research and clinical settings.

NIRS measures in pain and analgesia: Fundamentals, features, and function

Current pain assessment techniques based only on clinical evaluation and self-reports are not objective and may lead to inadequate treatment. Having a functional biomarker will add to the clinical fidelity, diagnosis, and perhaps improve treatment efficacy in patients. While many approaches have been deployed in pain biomarker discovery, functional near-infrared spectroscopy (fNIRS) is a technology that allows for non-invasive measurement of cortical hemodynamics. The utility of fNIRS is especially attractive given its ability to detect specific changes in the somatosensory and high-order cortices as well as its ability to measure (1) brain function similar to functional magnetic resonance imaging, (2) graded responses to noxious and innocuous stimuli, (3) analgesia, and (4) nociception under anesthesia. In this review, we evaluate the utility of fNIRS in nociception/pain with particular focus on its sensitivity and specificity, methodological advantages and limitations, and the current and potential applications in various pain conditions. Everything considered, fNIRS technology could enhance our ability to evaluate evoked and persistent pain across different age groups and clinical populations.

Functional Near-Infrared Spectroscopy in the Study of Speech and Language Impairment Across the Life Span: A Systematic Review

Purpose Functional brain imaging is playing an increasingly important role in the diagnosis and treatment of communication disorders, yet many populations and settings are incompatible with functional magnetic resonance imaging and other commonly used techniques. We conducted a systematic review of neuroimaging studies using functional near-infrared spectroscopy (fNIRS) with individuals with speech or language impairment across the life span. We aimed to answer the following question: To what extent has fNIRS been used to investigate the neural correlates of speech-language impairment? Method This systematic review was preregistered with PROSPERO, the international prospective register of systematic reviews (CRD42019136464). We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol for preferred reporting items for systematic reviews. The database searches were conducted between February and March of 2019 with the following search terms: (a) fNIRS or functional near-infrared spectroscopy or NIRS or near-infrared spectroscopy, (b) speech or language, and (c) disorder or impairment or delay. Results We found 34 fNIRS studies that involved individuals with speech or language impairment across nine categories: (a) autism spectrum disorders; (b) developmental speech and language disorders; (c) cochlear implantation and deafness; (d) dementia, dementia of the Alzheimer's type, and mild cognitive impairment; (e) locked-in syndrome; (f) neurologic speech disorders/dysarthria; (g) stroke/aphasia; (h) stuttering; and (i) traumatic brain injury. Conclusions Though it is not without inherent challenges, fNIRS may have advantages over other neuroimaging techniques in the areas of speech and language impairment. fNIRS has clinical applications that may lead to improved early and differential diagnosis, increase our understanding of response to treatment, improve neuroprosthetic functioning, and advance neurofeedback.

Advantages of double density alignment of fNIRS optodes to evaluate cortical activities related to phonological short-term memory using NIRS-SPM

Phonological short-term memory (STM) and working memory (WM) capacity, which is temporal storage of phonological information, facilitates language development. This capacity is often impaired in cochlear implant (CI) users who have congenital hearing loss. Functional near-infrared spectroscopy (fNIRS) is potentially useful to reveal underlying mechanisms of this impairment due to its tolerance to magnetic and electrical artifacts generated by CIs. The spatial resolution of the standard fNIRS, however, seems inadequate to evaluate cortical activity associated with the maintenance of phonological information in the STM/WM tasks. In the present study recruiting 14 normal hearing adults, we applied a double density alignment of fNIRS optodes to improve spatial resolution in the generalized linear model (GLM)-based statistical analysis using NIRS-SPM, in which cortical activities were estimated during each of three stages (encoding, maintenance, and retrieval) in pseudoword STM tasks with auditory or visual presentation. Since the double density alignment of fNIRS optodes contains two sets of standard density arrays, in the off-line analysis the measured cortical hemodynamic responses can be analyzed as data from two independent standard density arrays as well as those from one double density array (DD-array) which has two times higher density of channels. The two standard arrays demonstrated a similar pattern of cortical activation at each stage of the auditory and visual tasks, which proved the reliability of our fNIRS analysis, but failed to detect significant cortical activation in 2 of 12 conditions including the maintenance stage in the visual task. On the other hand, DD-array revealed significant cortical activation in all conditions. These differences were observed when estimated cortical activation was localized in small regions, which suggests higher spatial resolution in DD-array than the standard arrays. In our knowledge, this is the first clinical study supporting the previous experimental phantom study which demonstrated improvement of spatial resolution in the double density arrangement of fNIRS optodes. These findings imply that the double density alignment of fNIRS optodes improves reliability and spatial resolution in fNIRS-based estimation of cortical activity in the STM/WM studies, although further studies are required to determine usefulness of this method in the CI population.

Does task complexity influence motor facilitation and visuo-motor memory during mirror therapy in post-stroke patients?

Stroke is one of the most common causes of mortality and reduced disability-adjusted life years worldwide. Hemiparesis due to reduced skeletal-muscle power is an effect of brain lesions. Mirror therapy can significantly improve motor performance among post-stroke patients. To determine if altering the complexity of the mirror task in the mirror therapy paradigm would enhance top-down motor facilitation and visuo-motor memory demand, we conducted a pilot study on four post-stroke patients. Our preliminary results showed that performing complex finger tapping task resulted in enhanced activities in the primary motor cortex and precuneus, ipsilateral to the moving hand in the mirror therapy paradigm. We hypothesise the following: (a) complex finger tapping would result in stronger top-down motor facilitation and higher demand on visuo-motor memory than simple finger tapping in the mirror therapy paradigm, and (b) observing a blurred mirror image would result in increased top-down motor facilitation and higher demand on visuo-motor memory than a clear mirror image. To confirm these hypotheses, we propose a cross-sectional observational study on a large sample of post-stroke patients. This paper reports the findings of the pilot study, the rationale for testing the hypotheses, the experimental set-up, the task design and the assessment protocol for functional near-infrared spectroscopy.

English spoken word segmentation activates the prefrontal cortex and temporo-parietal junction in Chinese ESL learners: A functional near-infrared spectroscopy (fNIRS) study

A direct measure of spoken lexical processing based on neuroimaging technology would provide us useful information to understand the neural mechanisms underlying speech or auditory language processing. The neural mechanisms of spoken word segmentation for English as a second language (ESL) learners remain elusive. The present study, using functional near-infrared spectroscopy (fNIRS), addresses this issue by measuring hemodynamic responses in the temporo-parietal junction (TPJ) and the prefrontal cortex (PFC) in a word-spotting task, designed with two task conditions (easy vs. difficult). Thirty participants, divided into a high listening proficiency group (HLG) and a low listening proficiency group (LLG), were tested. Results revealed significantly less TPJ activation in the HLG than in the LLG. Further analyses supported this result by showing that activation in the TPJ was in a negative correlation with listening proficiency. This association appears to be related to the more efficient use of processing resources in a bottom-up fashion for accurate and efficient sensory representations in high proficient language learners. In contrast, cortical activation in the PFC increased with listening proficiency and was stronger in the difficult task condition than in the easy task condition, implying that recruitment of top-down cognitive control functions might play a role in word segmentation. Our results suggest that the combination of the functions mediated via bottom-up sensory input processing (demonstrated in the TPJ activation) and top-down cognitive processing (demonstrated in the PFC activation) are crucial for ESL listeners' spoken word segmentation.

The Principle of Inverse Effectiveness in Audiovisual Speech Perception

We assessed how synchronous speech listening and lipreading affects speech recognition in acoustic noise. In simple audiovisual perceptual tasks, inverse effectiveness is often observed, which holds that the weaker the unimodal stimuli, or the poorer their signal-to-noise ratio, the stronger the audiovisual benefit. So far, however, inverse effectiveness has not been demonstrated for complex audiovisual speech stimuli. Here we assess whether this multisensory integration effect can also be observed for the recognizability of spoken words. To that end, we presented audiovisual sentences to 18 native-Dutch normal-hearing participants, who had to identify the spoken words from a finite list. Speech-recognition performance was determined for auditory-only, visual-only (lipreading), and auditory-visual conditions. To modulate acoustic task difficulty, we systematically varied the auditory signal-to-noise ratio. In line with a commonly observed multisensory enhancement on speech recognition, audiovisual words were more easily recognized than auditory-only words (recognition thresholds of -15 and -12 dB, respectively). We here show that the difficulty of recognizing a particular word, either acoustically or visually, determines the occurrence of inverse effectiveness in audiovisual word integration. Thus, words that are better heard or recognized through lipreading, benefit less from bimodal presentation. Audiovisual performance at the lowest acoustic signal-to-noise ratios (45%) fell below the visual recognition rates (60%), reflecting an actual deterioration of lipreading in the presence of excessive acoustic noise. This suggests that the brain may adopt a strategy in which attention has to be divided between listening and lipreading.

Cortical Speech Processing in Postlingually Deaf Adult Cochlear Implant Users, as Revealed by Functional Near-Infrared Spectroscopy

An experiment was conducted to investigate the feasibility of using functional near-infrared spectroscopy (fNIRS) to image cortical activity in the language areas of cochlear implant (CI) users and to explore the association between the activity and their speech understanding ability. Using fNIRS, 15 experienced CI users and 14 normal-hearing participants were imaged while presented with either visual speech or auditory speech. Brain activation was measured from the prefrontal, temporal, and parietal lobe in both hemispheres, including the language-associated regions. In response to visual speech, the activation levels of CI users in an a priori region of interest (ROI)—the left superior temporal gyrus or sulcus—were negatively correlated with auditory speech understanding. This result suggests that increased cross-modal activity in the auditory cortex is predictive of poor auditory speech understanding. In another two ROIs, in which CI users showed significantly different mean activation levels in response to auditory speech compared with normal-hearing listeners, activation levels were significantly negatively correlated with CI users’ auditory speech understanding. These ROIs were located in the right anterior temporal lobe (including a portion of prefrontal lobe) and the left middle superior temporal lobe. In conclusion, fNIRS successfully revealed activation patterns in CI users associated with their auditory speech understanding.

Functional Near-Infrared Spectroscopy as a Measure of Listening Effort in Older Adults Who Use Hearing Aids

Listening effort may be reduced when hearing aids improve access to the acoustic signal. However, this possibility is difficult to evaluate because many neuroimaging methods used to measure listening effort are incompatible with hearing aid use. Functional near-infrared spectroscopy (fNIRS), which can be used to measure the concentration of oxygen in the prefrontal cortex (PFC), appears to be well-suited to this application. The first aim of this study was to establish whether fNIRS could measure cognitive effort during listening in older adults who use hearing aids. The second aim was to use fNIRS to determine if listening effort, a form of cognitive effort, differed depending on whether or not hearing aids were used when listening to sound presented at 35 dB SL (flat gain). Sixteen older adults who were experienced hearing aid users completed an auditory n-back task and a visual n-back task; both tasks were completed with and without hearing aids. We found that PFC oxygenation increased with n-back working memory demand in both modalities, supporting the use of fNIRS to measure cognitive effort during listening in this population. PFC oxygenation was weakly and nonsignificantly correlated with self-reported listening effort and reaction time, respectively, suggesting that PFC oxygenation assesses a dimension of listening effort that differs from these other measures. Furthermore, the extent to which hearing aids reduced PFC oxygenation in the left lateral PFC was positively correlated with age and pure-tone average thresholds. The implications of these findings as well as future directions are discussed.

Functional near-infrared spectroscopy as a tool for assessing speech and spoken language processing in pediatric and adult cochlear implant users

Much of what is known about the course of auditory learning in following cochlear implantation is based on behavioral indicators that users are able to perceive sound. Both prelingually deafened children and postlingually deafened adults who receive cochlear implants display highly variable speech and language processing outcomes, although the basis for this is poorly understood. To date, measuring neural activity within the auditory cortex of implant recipients of all ages has been challenging, primarily because the use of traditional neuroimaging techniques is limited by the implant itself. Functional near-infrared spectroscopy (fNIRS) is an imaging technology that works with implant users of all ages because it is non-invasive, compatible with implant devices, and not subject to electrical artifacts. Thus, fNIRS can provide insight into processing factors that contribute to variations in spoken language outcomes in implant users, both children and adults. There are important considerations to be made when using fNIRS, particularly with children, to maximize the signal-to-noise ratio and to best identify and interpret cortical responses. This review considers these issues, recent data, and future directions for using fNIRS as a tool to understand spoken language processing in children and adults who hear through a cochlear implant.

Human central auditory plasticity: A review of functional near-infrared spectroscopy (fNIRS) to measure cochlear implant performance and tinnitus perception

OBJECTIVE

Functional near-infrared spectroscopy (fNIRS) is an emerging noninvasive technology used to study cerebral cortex activity. Being virtually silent and compatible with cochlear implants has helped establish fNIRS as an important tool when investigating auditory cortex as well as cortices involved with hearing and language processing in adults and during child development. With respect to this review article, more recently, fNIRS has also been used to investigate central auditory plasticity following hearing loss and tinnitus or phantom sound perception.

METHODS

Here, we review the currently available literature reporting the use of fNIRS in human studies with cochlear implants and tinnitus to measure human central auditory cortical circuits. We also provide the reader with detailed reviews of the technology and traditional recording paradigms/methods used in these auditory-based studies.

RESULTS

The purpose of this review article is to summarize theoretical advancements in our understanding of the neurocognitive mechanisms underlying auditory processes and their plasticity through fNIRS research of human auditory performance with cochlear implantation and plasticity that may contribute to the central percepts of tinnitus.

CONCLUSION

fNIRS is an emerging noninvasive brain imaging technology that has wide reaching application that can be applied to human studies involving cochlear implants and tinnitus.

LEVEL OF EVIDENCE

N/A.

Measuring Cortical Activity During Auditory Processing with Functional Near-Infrared Spectroscopy

Functional near-infrared spectroscopy (fNIRS) is an optical, non-invasive neuroimaging technique that investigates human brain activity by calculating concentrations of oxy- and deoxyhemoglobin. The aim of this publication is to review the current state of the art as to how fNIRS has been used to study auditory function. We address temporal and spatial characteristics of the hemodynamic response to auditory stimulation as well as experimental factors that affect fNIRS data such as acoustic and stimulus-driven effects. The rising importance that fNIRS is generating in auditory neuroscience underlines the strong potential of the technology, and it seems likely that fNIRS will become a useful clinical tool.

Optical Mapping of Brain Activation and Connectivity in Occipitotemporal Cortex During Chinese Character Recognition

In this study, functional near-infrared spectroscopy (fNIRS) was used to examine the brain activation and connectivity in occipitotemporal cortex during Chinese character recognition (CCR). Eighteen healthy participants were recruited to perform a well-designed task with three categories of stimuli (real characters, pseudo characters, and checkerboards). By inspecting the brain activation difference and its relationship with behavioral data, the left laterality during CCR was clearly identified in the Brodmann area (BA) 18 and 19. In addition, our novel findings also demonstrated that the bilateral superior temporal gyrus (STG), bilateral BA 19, and left fusiform gyrus were also involved in high-level lexical information processing such as semantic and phonological ones. Meanwhile, by examining functional brain networks, we discovered that the right BA 19 exhibited enhanced brain connectivity. In particular, the connectivity in the right fusiform gyrus, right BA 19, and left STG showed significant correlation with the performance of CCR. Consequently, the combination of fNIRS technique with functional network analysis paves a new avenue for improved understanding of the cognitive mechanism underlying CCR.

Spatial Release From Informational Masking: Evidence From Functional Near Infrared Spectroscopy

Informational masking (IM) can greatly reduce speech intelligibility, but the neural mechanisms underlying IM are not understood. Binaural differences between target and masker can improve speech perception. In general, improvement in masked speech intelligibility due to provision of spatial cues is called spatial release from masking. Here, we focused on an aspect of spatial release from masking, specifically, the role of spatial attention. We hypothesized that in a situation with IM background sound (a) attention to speech recruits lateral frontal cortex (LFCx) and (b) LFCx activity varies with direction of spatial attention. Using functional near infrared spectroscopy, we assessed LFCx activity bilaterally in normal-hearing listeners. In Experiment 1, two talkers were simultaneously presented. Listeners either attended to the target talker (speech task) or they listened passively to an unintelligible, scrambled version of the acoustic mixture (control task). Target and masker differed in pitch and interaural time difference (ITD). Relative to the passive control, LFCx activity increased during attentive listening. Experiment 2 measured how LFCx activity varied with ITD, by testing listeners on the speech task in Experiment 1, except that talkers either were spatially separated by ITD or colocated. Results show that directing of auditory attention activates LFCx bilaterally. Moreover, right LFCx is recruited more strongly in the spatially separated as compared with colocated configurations. Findings hint that LFCx function contributes to spatial release from masking in situations with IM.