An online brain-computer interface based on shifting attention to concurrent streams of auditory stimuli

We report on the development and online testing of an electroencephalogram-based brain-computer interface (BCI) that aims to be usable by completely paralysed users-for whom visual or motor-system-based BCIs may not be suitable, and among whom reports of successful BCI use have so far been very rare. The current approach exploits covert shifts of attention to auditory stimuli in a dichotic-listening stimulus design. To compare the efficacy of event-related potentials (ERPs) and steady-state auditory evoked potentials (SSAEPs), the stimuli were designed such that they elicited both ERPs and SSAEPs simultaneously. Trial-by-trial feedback was provided online, based on subjects' modulation of N1 and P3 ERP components measured during single 5 s stimulation intervals. All 13 healthy subjects were able to use the BCI, with performance in a binary left/right choice task ranging from 75% to 96% correct across subjects (mean 85%). BCI classification was based on the contrast between stimuli in the attended stream and stimuli in the unattended stream, making use of every stimulus, rather than contrasting frequent standard and rare 'oddball' stimuli. SSAEPs were assessed offline: for all subjects, spectral components at the two exactly known modulation frequencies allowed discrimination of pre-stimulus from stimulus intervals, and of left-only stimuli from right-only stimuli when one side of the dichotic stimulus pair was muted. However, attention modulation of SSAEPs was not sufficient for single-trial BCI communication, even when the subject's attention was clearly focused well enough to allow classification of the same trials via ERPs. ERPs clearly provided a superior basis for BCI. The ERP results are a promising step towards the development of a simple-to-use, reliable yes/no communication system for users in the most severely paralysed states, as well as potential attention-monitoring and -training applications outside the context of assistive technology.

Attentional modulation of the inner ear: a combined otoacoustic emission and EEG study

Attending to a single stimulus in a complex multisensory environment requires the ability to select relevant information while ignoring distracting input. The underlying mechanism and involved neuronal levels of this attentional gain control are still a matter of debate. Here, we investigated the influence of intermodal attention on different levels of auditory processing in humans. It is known that the activity of the cochlear amplifier can be modulated by efferent neurons of the medial olivocochlear complex. We used distortion product otoacoustic emission (DPOAE) measurements to monitor cochlear activity during an intermodal cueing paradigm. Simultaneously, central auditory processing was assessed by electroencephalography (EEG) with a steady-state paradigm targeting early cortical responses and analysis of alpha oscillations reflecting higher cognitive control of attentional modulation. We found effects of selective attention at all measured levels of the auditory processing: DPOAE levels differed significantly between periods of visual and auditory attention, showing a reduction during visual attention, but no change during auditory attention. Primary auditory cortex activity, as measured by the auditory steady-state response (ASSR), differed between conditions, with higher ASSRs during auditory than visual attention. Furthermore, the analysis of cortical oscillatory activity revealed increased alpha power over occipitoparietal and frontal regions during auditory compared with visual attention, putatively reflecting suppression of visual processing. In conclusion, this study showed both enhanced processing of attended acoustic stimuli in early sensory cortex and reduced processing of distracting input, both at higher cortical levels and at the most peripheral level of the hearing system, the cochlea.

Brain state-dependent abnormal LFP activity in the auditory cortex of a schizophrenia mouse model

In schizophrenia, evoked 40-Hz auditory steady-state responses (ASSRs) are impaired, which reflects the sensory deficits in this disorder, and baseline spontaneous oscillatory activity also appears to be abnormal. It has been debated whether the evoked ASSR impairments are due to the possible increase in baseline power. GABAergic interneuron-specific NMDA receptor (NMDAR) hypofunction mutant mice mimic some behavioral and pathophysiological aspects of schizophrenia. To determine the presence and extent of sensory deficits in these mutant mice, we recorded spontaneous local field potential (LFP) activity and its click-train evoked ASSRs from primary auditory cortex of awake, head-restrained mice. Baseline spontaneous LFP power in the pre-stimulus period before application of the first click trains was augmented at a wide range of frequencies. However, when repetitive ASSR stimuli were presented every 20 s, averaged spontaneous LFP power amplitudes during the inter-ASSR stimulus intervals in the mutant mice became indistinguishable from the levels of control mice. Nonetheless, the evoked 40-Hz ASSR power and their phase locking to click trains were robustly impaired in the mutants, although the evoked 20-Hz ASSRs were also somewhat diminished. These results suggested that NMDAR hypofunction in cortical GABAergic neurons confers two brain state-dependent LFP abnormalities in the auditory cortex; (1) a broadband increase in spontaneous LFP power in the absence of external inputs, and (2) a robust deficit in the evoked ASSR power and its phase-locking despite of normal baseline LFP power magnitude during the repetitive auditory stimuli. The “paradoxically” high spontaneous LFP activity of the primary auditory cortex in the absence of external stimuli may possibly contribute to the emergence of schizophrenia-related aberrant auditory perception.

GABAergic modulation of the 40 Hz auditory steady-state response in a rat model of schizophrenia

Auditory steady-state auditory responses (ASSRs), in which the evoked potential entrains to stimulus frequency and phase, are reduced in magnitude in patients with schizophrenia, particularly at 40 Hz. While the neural mechanisms responsible for ASSR generation and its perturbation in schizophrenia are unknown, it has been hypothesized that the GABAA receptor subtype may have an important role. Using an established rat model of schizophrenia, the neonatal ventral hippocampal lesion (NVHL) model, 40-Hz ASSRs were elicited from NVHL and sham rats to determine if NVHL rats show deficits comparable to schizophrenia, and to examine the role of GABAA receptors in ASSR generation. ASSR parameters were found to be stable across time in both NVHL and sham rats. Manipulation of the GABAA receptor by muscimol, a GABAA agonist, yielded a strong lesion x drug interaction, with ASSR magnitude and synchronization decreased in NVHL and increased in sham rats. The lesion x muscimol interaction was blocked by a GABAA receptor antagonist when given prior to muscimol administration, confirming the observed interaction was GABAA mediated. Together, these data suggest an alteration involving GABAA receptor function, and hence inhibitory transmission, in the neuronal networks responsible for ASSR generation in NVHL rats. These findings are consistent with prior evidence for alterations in GABA neurotransmitter systems in the NVHL model and suggest the utility of this animal modelling approach for exploring neurobiological mechanisms that generate or modulate ASSRs.

Hemispheric asymmetry in auditory processing of speech envelope modulations in prereading children

The temporal envelope of speech is an important cue contributing to speech intelligibility. Theories about the neural foundations of speech perception postulate that the left and right auditory cortices are functionally specialized in analyzing speech envelope information at different time scales: the right hemisphere is thought to be specialized in processing syllable rate modulations, whereas a bilateral or left hemispheric specialization is assumed for phoneme rate modulations. Recently, it has been found that this functional hemispheric asymmetry is different in individuals with language-related disorders such as dyslexia. Most studies were, however, performed in adults and school-aged children, and only a little is known about how neural auditory processing at these specific rates manifests and develops in very young children before reading acquisition. Yet, studying hemispheric specialization for processing syllable and phoneme rate modulations in preliterate children may reveal early neural markers for dyslexia. In the present study, human cortical evoked potentials to syllable and phoneme rate modulations were measured in 5-year-old children at high and low hereditary risk for dyslexia. The results demonstrate a right hemispheric preference for processing syllable rate modulations and a symmetric pattern for phoneme rate modulations, regardless of hereditary risk for dyslexia. These results suggest that, while hemispheric specialization for processing syllable rate modulations seems to be mature in prereading children, hemispheric specialization for phoneme rate modulation processing may still be developing. These findings could have important implications for the development of phonological and reading skills.

White matter lateralization and interhemispheric coherence to auditory modulations in normal reading and dyslexic adults

Neural activation of slow acoustic variations that are important for syllable identification is more lateralized to the right hemisphere than activation of fast acoustic changes that are important for phoneme identification. It has been suggested that this complementary function at different hemispheres is rooted in a different degree of white matter myelination in the left versus right hemisphere. The present study will investigate this structure-function relationship with Diffusion Tensor Imaging (DTI) and Auditory Steady-State Responses (ASSR), respectively. With DTI we examined white matter lateralization in the cortical auditory and language regions (i.e. posterior region of the superior temporal gyrus and the arcuate fasciculus) and white matter integrity in the splenium of the corpus callosum. With ASSR we examined interhemispheric coherence to slow, syllabic-rate (i.e. 4 Hz) and fast, phonemic-rate (i.e. 20 Hz) modulations. These structural and functional techniques were applied in a group of normal reading adults and a group of dyslexic adults for whom previously reduced functional interhemispheric connectivity at 20 Hz has been reported (Poelmans et al. (2012). Ear and Hearing, 33, 134-143). This sample was chosen since it is hypothesized that in dyslexic readers insufficient hemispheric asymmetry in myelination might relate to their auditory and phonological problems. Results demonstrate reduced white matter lateralization in the posterior superior temporal gyrus and the arcuate fasciculus in the dyslexic readers. Additionally, white matter lateralization in the posterior superior temporal gyrus and white matter integrity in the splenium of the corpus callosum related to interhemispheric coherence to phonemic-rate modulations (i.e. 20 Hz). Interestingly, this correlation pattern was opposite in normal versus dyslexic readers. These results might imply that less pronounced left white matter dominance in dyslexic adults might relate to their problems to process phonemic-rate acoustic information and to integrate them into the phonological system.

Investigating the Effect of Cochlear Synaptopathy on Envelope Following Responses Using a Model of the Auditory Nerve

The healthy auditory system enables communication in challenging situations with high levels of background noise. Yet, despite normal sensitivity to pure tones, many listeners complain about having difficulties in such situations. Recent animal studies demonstrated that noise overexposure that produces temporary threshold shifts can cause the loss of auditory nerve (AN) fiber synapses (i.e., cochlear synaptopathy, CS), which appears to predominantly affect medium- and low-spontaneous rate (SR) fibers. In the present study, envelope following response (EFR) magnitude-level functions were recorded in normal hearing (NH) threshold and mildly hearing-impaired (HI) listeners with thresholds elevated above 2 kHz. EFRs were elicited by sinusoidally amplitude modulated (SAM) tones presented in quiet with a carrier frequency of 2 kHz, modulated at 93 Hz, and modulation depths of 0.85 (deep) and 0.25 (shallow). While EFR magnitude-level functions for deeply modulated tones were similar for all listeners, EFR magnitudes for shallowly modulated tones were reduced at medium stimulation levels in some NH threshold listeners and saturated in all HI listeners for the whole level range. A phenomenological model of the AN was used to investigate the extent to which hair-cell dysfunction and/or CS could explain the trends observed in the EFR data. Hair-cell dysfunction alone, including postulated elevated hearing thresholds at extended high frequencies (EHF) beyond 8 kHz, could not account for the recorded EFR data. Postulated CS led to simulations generally consistent with the recorded data, but a loss of all types of AN fibers was required within the model framework. The effects of off-frequency contributions (i.e., away from the characteristic place of the stimulus) and the differential loss of different AN fiber types on EFR magnitude-level functions were analyzed. When using SAM tones in quiet as the stimulus, model simulations suggested that (1) EFRs are dominated by the activity of high-SR fibers at all stimulus intensities, and (2) EFRs at medium-to-high stimulus levels are dominated by off-frequency contributions.

A Comparison of Two Objective Measures of Binaural Processing: The Interaural Phase Modulation Following Response and the Binaural Interaction Component

There has been continued interest in clinical objective measures of binaural processing. One commonly proposed measure is the binaural interaction component (BIC), which is obtained typically by recording auditory brainstem responses (ABRs)—the BIC reflects the difference between the binaural ABR and the sum of the monaural ABRs (i.e., binaural − (left + right)). We have recently developed an alternative, direct measure of sensitivity to interaural time differences, namely, a following response to modulations in interaural phase difference (the interaural phase modulation following response; IPM-FR). To obtain this measure, an ongoing diotically amplitude-modulated signal is presented, and the interaural phase difference of the carrier is switched periodically at minima in the modulation cycle. Such periodic modulations to interaural phase difference can evoke a steady state following response. BIC and IPM-FR measurements were compared from 10 normal-hearing subjects using a 16-channel electroencephalographic system. Both ABRs and IPM-FRs were observed most clearly from similar electrode locations—differential recordings taken from electrodes near the ear (e.g., mastoid) in reference to a vertex electrode (Cz). Although all subjects displayed clear ABRs, the BIC was not reliably observed. In contrast, the IPM-FR typically elicited a robust and significant response. In addition, the IPM-FR measure required a considerably shorter recording session. As the IPM-FR magnitude varied with interaural phase difference modulation depth, it could potentially serve as a correlate of perceptual salience. Overall, the IPM-FR appears a more suitable clinical measure than the BIC.

Representation of frequency-modulated sounds in the human brain

Frequency-modulation is a ubiquitous sound feature present in communicative sounds of various animal species and humans. Functional imaging of the human auditory system has seen remarkable advances in the last two decades and studies pertaining to frequency-modulation have centered around two major questions: a) are there dedicated feature-detectors encoding frequency-modulation in the brain and b) is there concurrent representation with amplitude-modulation, another temporal sound feature? In this review, we first describe how these two questions are motivated by psychophysical studies and neurophysiology in animal models. We then review how human non-invasive neuroimaging studies have furthered our understanding of the representation of frequency-modulated sounds in the brain. Finally, we conclude with some suggestions on how human neuroimaging could be used in future studies to address currently still open questions on this fundamental sound feature. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

Desynchronisation of auditory steady-state responses related to changes in interaural phase differences: an objective measure of binaural hearing

OBJECTIVE

Binaural processing can be measured objectively as a desynchronisation of phase-locked neural activity to changes in interaural phase differences (IPDs). This was reported in a magnetoencephalography study for 40 Hz amplitude modulated tones. The goal of this study was to measure this desynchronisation using electroencephalography and explore the outcomes for different modulation frequencies.

DESIGN

Auditory steady-state responses (ASSRs) were recorded to pure tones, amplitude modulated at 20, 40 or 80 Hz. IPDs switched between 0 and 180° at fixed time intervals.

STUDY SAMPLE

Sixteen young listeners with bilateral normal hearing thresholds (≤25 dB HL at 125-8000 Hz) participated in this study.

RESULTS

Significant ASSR phase desynchronisations to IPD changes were detected in 14 out of 16 participants for 40 Hz and in 8, respectively 9, out of 13 participants for 20 and 80 Hz modulators. Desynchronisation and restoration of ASSR phase took place significantly faster for 80 Hz than for 40 and 20 Hz.

CONCLUSIONS

ASSR desynchronisation to IPD changes was successfully recorded using electroencephalography. It was feasible for 20, 40 and 80 Hz modulators and could be an objective tool to assess processing of changes in binaural information.

Objective Binaural Loudness Balancing Based on 40-Hz Auditory Steady-State Responses. Part I: Normal Hearing

Psychophysical procedures are used to balance loudness across the ears. However, they can be difficult and require active cooperation. We investigated whether 40-Hz auditory steady-state response (ASSR) amplitudes can be used to objectively estimate the balanced loudness across the ears for a group of young, normal-hearing participants. The 40-Hz ASSRs were recorded using monaural stimuli with carrier frequencies of 500, 1000, or 2000 Hz over a range of levels between 40 and 80 dB SPL. Behavioral loudness balancing was performed for at least one reference level of the left ear. ASSR amplitude growth functions were listener dependent, but median across-ear ratios in ASSR amplitudes were close to 1. The differences between the ASSR-predicted balanced levels and the behaviorally found balanced levels were smaller than 5 dB in 59% of cases and smaller than 10 dB in 85% of cases. The differences between the ASSR-predicted balanced levels and the reference levels were smaller than 5 dB in 54% of cases and smaller than 10 dB in 87% of cases. No clear hemispheric lateralization was found for 40-Hz ASSRs, with the exception of responses evoked by stimulus levels of 40 to 60 dB SPL at 2000 Hz.

Atypical processing in neural source analysis of speech envelope modulations in adolescents with dyslexia

Different studies have suggested that language and developmental disorders such as dyslexia are associated with a disturbance of auditory entrainment and of the functional hemispheric asymmetries during speech processing. These disorders typically result from an issue in the phonological component of language that causes problems to represent and manipulate the phonological structure of words at the syllable and/or phoneme level. We used Auditory Steady-State Responses (ASSRs) in EEG recordings to investigate the brain activation and hemisphere asymmetry of theta, alpha, beta and low-gamma range oscillations in typical readers and readers with dyslexia. The aim was to analyse whether the group differences found in previous electrode level studies were caused by a different source activation pattern or conversely was an effect that could be found on the active brain sources. We could not find differences in the brain locations of the main active brain sources. However, we observed differences in the extracted waveforms. The group average of the first DSS component of all signal-to-noise ratios of ASSR at source level was higher than the group averages at the electrode level. These analyses included a lower alpha synchronisation in adolescents with dyslexia and the possibility of compensatory mechanisms in theta, beta and low-gamma frequency bands. The main brain auditory sources were located in cortical regions around the auditory cortex. Thus, the differences observed in auditory EEG experiments would, according to our findings, have their origin in the intrinsic oscillatory mechanisms of the brain cortical sources related to speech perception.

Diagnosing awareness in disorders of consciousness with gamma-band auditory responses

Introduction

The prolonged disorders of consciousness (pDOC) describe a group of neurological conditions characterized by severe impairment of consciousness resulting from the injury of the central nervous system. As the behavioral diagnosis of pDOC remains challenging, the methods based on observing brain activity appear as promising alternatives. One of these methods is electroencephalography, which allows for noninvasive assessment of brain function.

Methods

In this study, we evaluated evoked auditory responses to the chirp-modulated auditory stimulation as a potential biomarker of awareness in pDOC. Chirp-modulated stimulation is based on the repetitive presentation of auditory stimuli with a changing frequency over time. Two protocols were tested: amplitude-modulated narrow-band chirps (frequency range 25-55 Hz) and click-based wide-band chirps (30-100 Hz). The studied pDOC patient group included 62 patients (19 females and 43 males, mean age 40.72 years) diagnosed with Coma Recovery Scale-Revised. Envelope-following responses to stimulation were examined using the intertrial phase clustering coefficient.

Results

For both types of stimulation, the strength of the response in the low-gamma range (around 40 Hz) was related to the diagnosis of pDOC. Patients diagnosed with unresponsive wakefulness syndrome exhibited diminished responses, while more favorable diagnoses, suggesting awareness (minimally conscious state or emergence from the minimally conscious state), showed elevated responses. The variations in the integrity of the auditory pathway and the etiology of brain injury altered the observed response strength. Narrow-band stimulation yielded a more systematic relationship between low-gamma response and pDOC diagnosis.

Discussion

The results suggest the potential role of low gamma-band responses to chirp-modulated stimulation as the supportive diagnostic tool to detect awareness in the pDOC patient group.

[Speech perception and the state of sound perception thresholds in patients with cochlear implants]

OBJECTIVE

To assess the state of the thresholds of sound perception at speech frequencies in users of cochlear implantation (CI) systems with developed speech skills.

MATERIAL AND METHODS

The study involved 30 patients - users of CI systems, of whom 17 girls, 13 boys aged 6 to 14 years. All patients underwent speech audiometry in a free sound field twice (at the beginning of the study and after adjusting the settings of the speech processor). The ASSR (auditory steady-state responses) thresholds were recorded as the second stage to assess the perception of non-speech stimuli after adjusting the settings of the speech processor.

RESULTS

Correlation analysis was carried out between the percentage of speech intelligibility and the average ASSR thresholds. There was no statistically significant relationship between lowering the ASSR thresholds and speech intelligibility. The absence of a relationship between speech intelligibility and user satisfaction with the speech processor settings was revealed. Improving the intelligibility of the user's speech by the CI system did not affect the improvement on the response to the addressed speech.

[On possible causes of discrepancy between the results of auditory brainstem responses (ABR) and auditory steady-state responses (ASSR) recordings]

Auditory brainstem responses (ABR) and auditory steady-state responses (ASSR) are the most widely used tests for objective hearing assessment. These tests are included in clinical guidelines for sensorineural hearing loss and other protocols. The literature sources describe examples of inconsistency between the results of these two tests, which are suggested to be interpreted in favor of ABR as the test with the highest values of sensitivity and specificity. Also, in a number of cases, discrepancy between the results of ABR and ASSR recordings in patients with neurological pathology was revealed. Given the expanding indications for auditory evoked potentials recording in groups of patients with neurological or psychiatric pathologies (for example, ABR and ASSR are recommended for use in clinical guidelines for the management of patients with autism spectrum disorders in 2020), an attempt has been made to assess the diagnostic value of these research methods in relation to conditions unrelated to hearing impairment.

MATERIAL AND METHODS

A diagnostic search of domestic and foreign literature was conducted on the combined interpretation of the results of ABR and ASSR recording was carried out. Literature on the use of these tests in groups of patients without sensorineural hearing loss was also reviewed.

RESULTS

The review includes the results of a literature search that provides a comparative analysis of ABR and ASSR results, including in patients with neurologic/psychiatric pathology. The analysis showed that ASSR recording results may significantly differ from the ABR response, which should be taken into account in patients with comorbidities.