Predictors for estimating subcortical EEG responses to continuous speech

Perception of sounds and speech involves structures in the auditory brainstem that rapidly process ongoing auditory stimuli. The role of these structures in speech processing can be investigated by measuring their electrical activity using scalp-mounted electrodes. However, typical analysis methods involve averaging neural responses to many short repetitive stimuli that bear little relevance to daily listening environments. Recently, subcortical responses to more ecologically relevant continuous speech were detected using linear encoding models. These methods estimate the temporal response function (TRF), which is a regression model that minimises the error between the measured neural signal and a predictor derived from the stimulus. Using predictors that model the highly non-linear peripheral auditory system may improve linear TRF estimation accuracy and peak detection. Here, we compare predictors from both simple and complex peripheral auditory models for estimating brainstem TRFs on electroencephalography (EEG) data from 24 participants listening to continuous speech. We also investigate the data length required for estimating subcortical TRFs, and find that around 12 minutes of data is sufficient for clear wave V peaks (>3 dB SNR) to be seen in nearly all participants. Interestingly, predictors derived from simple filterbank-based models of the peripheral auditory system yield TRF wave V peak SNRs that are not significantly different from those estimated using a complex model of the auditory nerve, provided that the nonlinear effects of adaptation in the auditory system are appropriately modelled. Crucially, computing predictors from these simpler models is more than 50 times faster compared to the complex model. This work paves the way for efficient modelling and detection of subcortical processing of continuous speech, which may lead to improved diagnosis metrics for hearing impairment and assistive hearing technology.

Extending Subcortical EEG Responses to Continuous Speech to the Sound-Field

The auditory brainstem response (ABR) is a valuable clinical tool for objective hearing assessment, which is conventionally detected by averaging neural responses to thousands of short stimuli. Progressing beyond these unnatural stimuli, brainstem responses to continuous speech presented via earphones have been recently detected using linear temporal response functions (TRFs). Here, we extend earlier studies by measuring subcortical responses to continuous speech presented in the sound-field, and assess the amount of data needed to estimate brainstem TRFs. Electroencephalography (EEG) was recorded from 24 normal hearing participants while they listened to clicks and stories presented via earphones and loudspeakers. Subcortical TRFs were computed after accounting for non-linear processing in the auditory periphery by either stimulus rectification or an auditory nerve model. Our results demonstrated that subcortical responses to continuous speech could be reliably measured in the sound-field. TRFs estimated using auditory nerve models outperformed simple rectification, and 16 minutes of data was sufficient for the TRFs of all participants to show clear wave V peaks for both earphones and sound-field stimuli. Subcortical TRFs to continuous speech were highly consistent in both earphone and sound-field conditions, and with click ABRs. However, sound-field TRFs required slightly more data (16 minutes) to achieve clear wave V peaks compared to earphone TRFs (12 minutes), possibly due to effects of room acoustics. By investigating subcortical responses to sound-field speech stimuli, this study lays the groundwork for bringing objective hearing assessment closer to real-life conditions, which may lead to improved hearing evaluations and smart hearing technologies.

Improved speech intelligibility in the presence of congruent vibrotactile speech input

Vibrotactile stimulation is believed to enhance auditory speech perception, offering potential benefits for cochlear implant (CI) users who may utilize compensatory sensory strategies. Our study advances previous research by directly comparing tactile speech intelligibility enhancements in normal-hearing (NH) and CI participants, using the same paradigm. Moreover, we assessed tactile enhancement considering stimulus non-specific, excitatory effects through an incongruent audio-tactile control condition that did not contain any speech-relevant information. In addition to this incongruent audio-tactile condition, we presented sentences in an auditory only and a congruent audio-tactile condition, with the congruent tactile stimulus providing low-frequency envelope information via a vibrating probe on the index fingertip. The study involved 23 NH listeners and 14 CI users. In both groups, significant tactile enhancements were observed for congruent tactile stimuli (5.3% for NH and 5.4% for CI participants), but not for incongruent tactile stimulation. These findings replicate previously observed tactile enhancement effects. Juxtaposing our study with previous research, the informational content of the tactile stimulus emerges as a modulator of intelligibility: Generally, congruent stimuli enhanced, non-matching tactile stimuli reduced, and neutral stimuli did not change test outcomes. We conclude that the temporal cues provided by congruent vibrotactile stimuli may aid in parsing continuous speech signals into syllables and words, consequently leading to the observed improvements in intelligibility.

Combining Multiple Psychophysiological Measures of Listening Effort: Challenges and Recommendations

About one-third of all recently published studies on listening effort have used at least one physiological measure, providing evidence of the popularity of such measures in listening effort research. However, the specific measures employed, as well as the rationales used to justify their inclusion, vary greatly between studies, leading to a literature that is fragmented and difficult to integrate. A unified approach that assesses multiple psychophysiological measures justified by a single rationale would be preferable because it would advance our understanding of listening effort. However, such an approach comes with a number of challenges, including the need to develop a clear definition of listening effort that links to specific physiological measures, customized equipment that enables the simultaneous assessment of multiple measures, awareness of problems caused by the different timescales on which the measures operate, and statistical approaches that minimize the risk of type-I error inflation. This article discusses in detail the various obstacles for combining multiple physiological measures in listening effort research and provides recommendations on how to overcome them.

Effect of audio-tactile congruence on vibrotactile music enhancement

Music listening experiences can be enhanced with tactile vibrations. However, it is not known which parameters of the tactile vibration must be congruent with the music to enhance it. Devices that aim to enhance music with tactile vibrations often require coding an acoustic signal into a congruent vibrotactile signal. Therefore, understanding which of these audio-tactile congruences are important is crucial. Participants were presented with a simple sine wave melody through supra-aural headphones and a haptic actuator held between the thumb and forefinger. Incongruent versions of the stimuli were made by randomizing physical parameters of the tactile stimulus independently of the auditory stimulus. Participants were instructed to rate the stimuli against the incongruent stimuli based on preference. It was found making the intensity of the tactile stimulus incongruent with the intensity of the auditory stimulus, as well as misaligning the two modalities in time, had the biggest negative effect on ratings for the melody used. Future vibrotactile music enhancement devices can use time alignment and intensity congruence as a baseline coding strategy, which improved strategies can be tested against.

Human cortical processing of interaural coherence

Sounds reach the ears as a mixture of energy generated by different sources. Listeners extract cues that distinguish different sources from one another, including how similar sounds arrive at the two ears, the interaural coherence (IAC). Here, we find listeners cannot reliably distinguish two completely interaurally coherent sounds from a single sound with reduced IAC. Pairs of sounds heard toward the front were readily confused with single sounds with high IAC, whereas those heard to the sides were confused with single sounds with low IAC. Sounds that hold supra-ethological spatial cues are perceived as more diffuse than can be accounted for by their IAC, and this is accounted for by a computational model comprising a restricted, and sound-frequency dependent, distribution of auditory-spatial detectors. We observed elevated cortical hemodynamic responses for sounds with low IAC, suggesting that the ambiguity elicited by sounds with low interaural similarity imposes elevated cortical load.

•

Listeners report ambiguous spatial percepts for sounds with reduced interaural coherence

•

Supra-ethological spatial cues impose an additional diffuse quality to sounds

•

A frequency-dependent distribution of spatial detectors explains behavioral results

•

Sounds with low interaural coherence impose elevated cortical load

Development of an Australian behavioural method for assessing listening task difficulty at high speech intelligibility levels

OBJECTIVE

To develop and validate an Australian version of a behavioural test for assessing listening task difficulty at high speech intelligibility levels.

DESIGN

In the SWIR-Aus test, listeners perform two tasks: identify the last word of each of seven sentences in a list and recall the identified words after each list. First, the test material was developed by creating seven-sentence lists with similar final-word features. Then, for the validation, participant's performance on the SWIR-Aus test was compared when a binary mask noise reduction algorithm was on and off.

STUDY SAMPLE

All participants in this study had normal hearing thresholds. Nine participants (23.8-56.0 years) participated in the characterisation of the speech material. Another thirteen participants (18.4-59.1 years) participated in a pilot test to determine the SNR to use at the validation stage. Finally, twenty-four new participants (20.0-56.9 years) participated in the validation of the test.

RESULTS

The results of the validation of the test showed that recall and identification scores were significantly better when the binary mask noise reduction algorithm was on compared to off.

CONCLUSIONS

The SWIR-Aus test was developed using Australian speech material and can be used for assessing task difficulty at high speech intelligibility levels.

Spectral features of cortical auditory evoked potentials inform hearing threshold and intensity percepts in acoustic and electric hearing

Objective. Stimulus-elicited changes in electroencephalography (EEG) recordings can be represented using Fourier magnitude and phase features (Makeiget al(2004Trends Cogn. Sci.8204-10)). The present study aimed to quantify how much information about hearing responses are contained in the magnitude, quantified by event-related spectral perturbations (ERSPs); and the phase, quantified by inter-trial coherence (ITC). By testing if one feature contained more information and whether this information was mutually exclusive to the features, we aimed to relate specific EEG magnitude and phase features to hearing perception.Approach.EEG responses were recorded from 20 adults who were presented with acoustic stimuli, and 20 adult cochlear implant users with electrical stimuli. Both groups were presented with short, 50 ms stimuli at varying intensity levels relative to their hearing thresholds. Extracted ERSP and ITC features were inputs for a linear discriminant analysis classifier (Wonget al(2016J. Neural. Eng.13036003)). The classifier then predicted whether the EEG signal contained information about the sound stimuli based on the input features. Classifier decoding accuracy was quantified with the mutual information measure (Cottaris and Elfar (2009J. Neural. Eng.6026007), Hawelleket al(2016Proc. Natl Acad. Sci.11313492-7)), and compared across the two feature sets, and to when both feature sets were combined.Main results. We found that classifiers using either ITC or ERSP feature sets were both able to decode hearing perception, but ITC-feature classifiers were able to decode responses to a lower but still audible stimulation intensity, making ITC more useful than ERSP for hearing threshold estimation. We also found that combining the information from both feature sets did not improve decoding significantly, implying that ERSP brain dynamics has a limited contribution to the EEG response, possibly due to the stimuli used in this study.Significance.We successfully related hearing perception to an EEG measure, which does not require behavioral feedback from the listener; an objective measure is important in both neuroscience research and clinical audiology.

Analysis methods for measuring passive auditory fNIRS responses generated by a block-design paradigm

Significance: Functional near-infrared spectroscopy (fNIRS) is an increasingly popular tool in auditory research, but the range of analysis procedures employed across studies may complicate the interpretation of data. Aim: We aim to assess the impact of different analysis procedures on the morphology, detection, and lateralization of auditory responses in fNIRS. Specifically, we determine whether averaging or generalized linear model (GLM)-based analysis generates different experimental conclusions when applied to a block-protocol design. The impact of parameter selection of GLMs on detecting auditory-evoked responses was also quantified. Approach: 17 listeners were exposed to three commonly employed auditory stimuli: noise, speech, and silence. A block design, comprising sounds of 5 s duration and 10 to 20 s silent intervals, was employed. Results: Both analysis procedures generated similar response morphologies and amplitude estimates, and both indicated that responses to speech were significantly greater than to noise or silence. Neither approach indicated a significant effect of brain hemisphere on responses to speech. Methods to correct for systemic hemodynamic responses using short channels improved detection at the individual level. Conclusions: Consistent with theoretical considerations, simulations, and other experimental domains, GLM and averaging analyses generate the same group-level experimental conclusions. We release this dataset publicly for use in future development and optimization of algorithms.

Effects of Hearing Aid Noise Reduction on Early and Late Cortical Representations of Competing Talkers in Noise

OBJECTIVES

Previous research using non-invasive (magnetoencephalography, MEG) and invasive (electrocorticography, ECoG) neural recordings has demonstrated the progressive and hierarchical representation and processing of complex multi-talker auditory scenes in the auditory cortex. Early responses (<85 ms) in primary-like areas appear to represent the individual talkers with almost equal fidelity and are independent of attention in normal-hearing (NH) listeners. However, late responses (>85 ms) in higher-order non-primary areas selectively represent the attended talker with significantly higher fidelity than unattended talkers in NH and hearing-impaired (HI) listeners. Motivated by these findings, the objective of this study was to investigate the effect of a noise reduction scheme (NR) in a commercial hearing aid (HA) on the representation of complex multi-talker auditory scenes in distinct hierarchical stages of the auditory cortex by using high-density electroencephalography (EEG).

DESIGN

We addressed this issue by investigating early (<85 ms) and late (>85 ms) EEG responses recorded in 34 HI subjects fitted with HAs. The HA noise reduction (NR) was either on or off while the participants listened to a complex auditory scene. Participants were instructed to attend to one of two simultaneous talkers in the foreground while multi-talker babble noise played in the background (+3 dB SNR). After each trial, a two-choice question about the content of the attended speech was presented.

RESULTS

Using a stimulus reconstruction approach, our results suggest that the attention-related enhancement of neural representations of target and masker talkers located in the foreground, as well as suppression of the background noise in distinct hierarchical stages is significantly affected by the NR scheme. We found that the NR scheme contributed to the enhancement of the foreground and of the entire acoustic scene in the early responses, and that this enhancement was driven by better representation of the target speech. We found that the target talker in HI listeners was selectively represented in late responses. We found that use of the NR scheme resulted in enhanced representations of the target and masker speech in the foreground and a suppressed representation of the noise in the background in late responses. We found a significant effect of EEG time window on the strengths of the cortical representation of the target and masker.

CONCLUSION

Together, our analyses of the early and late responses obtained from HI listeners support the existing view of hierarchical processing in the auditory cortex. Our findings demonstrate the benefits of a NR scheme on the representation of complex multi-talker auditory scenes in different areas of the auditory cortex in HI listeners.

Cortical Pitch Response Components Correlate with the Pitch Salience of Resolved and Unresolved components of Mandarin Tones

Cortical pitch responses (CPRs) are generated at the initiation of pitch-bearing sounds. CPR components have been shown to reflect the pitch salience of iterated rippled noise with different temporal periodicity. However, it is unclear whether features of the CPR correlate with the pitch salience of resolved and unresolved harmonics of speech when the temporal periodicity is identical, and whether CPRs could be a neural index for auditory cortical pitch processing. In this study, CPRs were recorded to two speech sounds: a set including only resolved harmonics and a set including only unresolved harmonics. Speech-shaped noise preceding and following the speech was used to temporally discriminate the neural activity coding the onset of acoustic energy from the onset of time-varying pitch. Analysis of CPR peak latency and peak amplitude (Na) showed that the peak latency to speech sounds with only resolved harmonics was significantly shorter than for sounds with unresolved harmonics (p = 0.01), and that peak amplitude to sounds with only resolved harmonics was significantly higher than for sounds with unresolved harmonics (p <; 0.001). Further, the CPR peak phase locking value in response to sounds with only resolved harmonics was significantly higher than to sounds with only unresolved harmonics (p <; 0.001). Our findings suggest that the CPR changes with pitch salience and that CPR is a potentially useful indicator of auditory cortical pitch processing.

Audio-visual integration in cochlear implant listeners and the effect of age difference

This study aimed to investigate differences in audio-visual (AV) integration between cochlear implant (CI) listeners and normal-hearing (NH) adults. A secondary aim was to investigate the effect of age differences by examining AV integration in groups of older and younger NH adults. Seventeen CI listeners, 13 similarly aged NH adults, and 16 younger NH adults were recruited. Two speech identification experiments were conducted to evaluate AV integration of speech cues. In the first experiment, reaction times in audio-alone (A-alone), visual-alone (V-alone), and AV conditions were measured during a speeded task in which participants were asked to identify a target sound /aSa/ among 11 alternatives. A race model was applied to evaluate AV integration. In the second experiment, identification accuracies were measured using a closed set of consonants and an open set of consonant-nucleus-consonant words. The authors quantified AV integration using a combination of a probability model and a cue integration model (which model participants' AV accuracy by assuming no or optimal integration, respectively). The results found that experienced CI listeners showed no better AV integration than their similarly aged NH adults. Further, there was no significant difference in AV integration between the younger and older NH adults.

Cortical Speech Processing in Postlingually Deaf Adult Cochlear Implant Users, as Revealed by Functional Near-Infrared Spectroscopy

An experiment was conducted to investigate the feasibility of using functional near-infrared spectroscopy (fNIRS) to image cortical activity in the language areas of cochlear implant (CI) users and to explore the association between the activity and their speech understanding ability. Using fNIRS, 15 experienced CI users and 14 normal-hearing participants were imaged while presented with either visual speech or auditory speech. Brain activation was measured from the prefrontal, temporal, and parietal lobe in both hemispheres, including the language-associated regions. In response to visual speech, the activation levels of CI users in an a priori region of interest (ROI)—the left superior temporal gyrus or sulcus—were negatively correlated with auditory speech understanding. This result suggests that increased cross-modal activity in the auditory cortex is predictive of poor auditory speech understanding. In another two ROIs, in which CI users showed significantly different mean activation levels in response to auditory speech compared with normal-hearing listeners, activation levels were significantly negatively correlated with CI users’ auditory speech understanding. These ROIs were located in the right anterior temporal lobe (including a portion of prefrontal lobe) and the left middle superior temporal lobe. In conclusion, fNIRS successfully revealed activation patterns in CI users associated with their auditory speech understanding.

Assessing hearing by measuring heartbeat: The effect of sound level

Functional near-infrared spectroscopy (fNIRS) is a non-invasive brain imaging technique that measures changes in oxygenated and de-oxygenated hemoglobin concentration and can provide a measure of brain activity. In addition to neural activity, fNIRS signals contain components that can be used to extract physiological information such as cardiac measures. Previous studies have shown changes in cardiac activity in response to different sounds. This study investigated whether cardiac responses collected using fNIRS differ for different loudness of sounds. fNIRS data were collected from 28 normal hearing participants. Cardiac response measures evoked by broadband, amplitude-modulated sounds were extracted for four sound intensities ranging from near-threshold to comfortably loud levels (15, 40, 65 and 90 dB Sound Pressure Level (SPL)). Following onset of the noise stimulus, heart rate initially decreased for sounds of 15 and 40 dB SPL, reaching a significantly lower rate at 15 dB SPL. For sounds at 65 and 90 dB SPL, increases in heart rate were seen. To quantify the timing of significant changes, inter-beat intervals were assessed. For sounds at 40 dB SPL, an immediate significant change in the first two inter-beat intervals following sound onset was found. At other levels, the most significant change appeared later (beats 3 to 5 following sound onset). In conclusion, changes in heart rate were associated with the level of sound with a clear difference in response to near-threshold sounds compared to comfortably loud sounds. These findings may be used alone or in conjunction with other measures such as fNIRS brain activity for evaluation of hearing ability.

Auditory Brainstem Representation of the Voice Pitch Contours in the Resolved and Unresolved Components of Mandarin Tones

Accurate perception of voice pitch plays a vital role in speech understanding, especially for tonal languages such as Mandarin. Lexical tones are primarily distinguished by the fundamental frequency (F0) contour of the acoustic waveform. It has been shown that the auditory system could extract the F0 from the resolved and unresolved harmonics, and the tone identification performance of resolved harmonics was better than unresolved harmonics. To evaluate the neural response to the resolved and unresolved components of Mandarin tones in quiet and in speech-shaped noise, we recorded the frequency-following response. In this study, four types of stimuli were used: speech with either only-resolved harmonics or only-unresolved harmonics, both in quiet and in speech-shaped noise. Frequency-following responses (FFRs) were recorded to alternating-polarity stimuli and were added or subtracted to enhance the neural response to the envelope (FFR_ENV) or fine structure (FFR_TFS), respectively. The neural representation of the F0 strength reflected by the FFR_ENV was evaluated by the peak autocorrelation value in the temporal domain and the peak phase-locking value (PLV) at F0 in the spectral domain. Both evaluation methods showed that the FFR_ENV F0 strength in quiet was significantly stronger than in noise for speech including unresolved harmonics, but not for speech including resolved harmonics. The neural representation of the temporal fine structure reflected by the FFR_TFS was assessed by the PLV at the harmonic near to F1 (4th of F0). The PLV at harmonic near to F1 (4th of F0) of FFR_TFS to resolved harmonics was significantly larger than to unresolved harmonics. Spearman's correlation showed that the FFR_ENV F0 strength to unresolved harmonics was correlated with tone identification performance in noise (0 dB SNR). These results showed that the FFR_ENV F0 strength to speech sounds with resolved harmonics was not affected by noise. In contrast, the response to speech sounds with unresolved harmonics, which were significantly smaller in noise compared to quiet. Our results suggest that coding resolved harmonics was more important than coding envelope for tone identification performance in noise.

Hearing Aid Use in Older Adults With Postlingual Sensorineural Hearing Loss: Protocol for a Prospective Cohort Study

Background

Older adults with postlingual sensorineural hearing loss (SNHL) exhibit a poor prognosis that not only includes impaired auditory function but also rapid cognitive decline, especially speech-related cognition, in addition to psychosocial dysfunction and an increased risk of dementia. Consistent with this prognosis, individuals with SNHL exhibit global atrophic brain alteration as well as altered neural function and regional brain organization within the cortical substrates that underlie auditory and speech processing. Recent evidence suggests that the use of hearing aids might ameliorate this prognosis.

Objective

The objective was to study the effects of a hearing aid use intervention on neurocognitive and psychosocial functioning in individuals with SNHL aged ≥55 years.

Methods

All aspects of this study will be conducted at Swinburne University of Technology (Hawthorn, Victoria, Australia). We will recruit 2 groups (n=30 per group) of individuals with mild to moderate SNHL from both the community and audiology health clinics (Alison Hennessy Audiology, Chelsea Hearing Pty Ltd). These groups will include individuals who have worn a hearing aid for, at least, 12 months or never worn a hearing aid. All participants would be asked to complete, at 2 time points (t) including baseline (t=0) and follow-up (t=6 months), tests of hearing and psychosocial and cognitive function and attend a magnetic resonance imaging (MRI) session. The MRI session will include both structural and functional MRI (sMRI and fMRI) scans, the latter involving the performance of a novel speech processing task.

Results

This research is funded by the Barbara Dicker Brain Sciences Foundation Grants, the Australian Research Council, Alison Hennessy Audiology, and Chelsea Hearing Pty Ltd under the Industry Transformation Training Centre Scheme (ARC Project #IC140100023). We obtained the ethics approval on November 18, 2017 (Swinburne University Human Research Ethics Committee protocol number SHR Project 2017/266). The recruitment began in December 2017 and will be completed by December 2020.

Conclusions

This is the first study to assess the effect hearing aid use has on neural, cognitive, and psychosocial factors in individuals with SNHL who have never used hearing aids. Furthermore, this study is expected to clarify the relationships among altered brain structure and function, psychosocial factors, and cognition in response to the hearing aid use.

Trial Registration

Australian New Zealand Clinical Trials Registry: ACTRN12617001616369; https://anzctr.org.au/Trial/Registration/TrialReview.aspx?ACTRN=12617001616369 (Accessed by WebCite at http://www.webcitation.org/70yatZ9ze)

International Registered Report Identifier (IRRID)

RR1-10.2196/9916

Auditory Stream Segregation and Selective Attention for Cochlear Implant Listeners: Evidence From Behavioral Measures and Event-Related Potentials

The role of the spatial separation between the stimulating electrodes (electrode separation) in sequential stream segregation was explored in cochlear implant (CI) listeners using a deviant detection task. Twelve CI listeners were instructed to attend to a series of target sounds in the presence of interleaved distractor sounds. A deviant was randomly introduced in the target stream either at the beginning, middle or end of each trial. The listeners were asked to detect sequences that contained a deviant and to report its location within the trial. The perceptual segregation of the streams should, therefore, improve deviant detection performance. The electrode range for the distractor sounds was varied, resulting in different amounts of overlap between the target and the distractor streams. For the largest electrode separation condition, event-related potentials (ERPs) were recorded under active and passive listening conditions. The listeners were asked to perform the behavioral task for the active listening condition and encouraged to watch a muted movie for the passive listening condition. Deviant detection performance improved with increasing electrode separation between the streams, suggesting that larger electrode differences facilitate the segregation of the streams. Deviant detection performance was best for deviants happening late in the sequence, indicating that a segregated percept builds up over time. The analysis of the ERP waveforms revealed that auditory selective attention modulates the ERP responses in CI listeners. Specifically, the responses to the target stream were, overall, larger in the active relative to the passive listening condition. Conversely, the ERP responses to the distractor stream were not affected by selective attention. However, no significant correlation was observed between the behavioral performance and the amount of attentional modulation. Overall, the findings from the present study suggest that CI listeners can use electrode separation to perceptually group sequential sounds. Moreover, selective attention can be deployed on the resulting auditory objects, as reflected by the attentional modulation of the ERPs at the group level.

Temporal Coding of Voice Pitch Contours in Mandarin Tones

Accurate perception of time-variant pitch is important for speech recognition, particularly for tonal languages with different lexical tones such as Mandarin, in which different tones convey different semantic information. Previous studies reported that the auditory nerve and cochlear nucleus can encode different pitches through phase-locked neural activities. However, little is known about how the inferior colliculus (IC) encodes the time-variant periodicity pitch of natural speech. In this study, the Mandarin syllable /ba/ pronounced with four lexical tones (flat, rising, falling then rising and falling) were used as stimuli. Local field potentials (LFPs) and single neuron activity were simultaneously recorded from 90 sites within contralateral IC of six urethane-anesthetized and decerebrate guinea pigs in response to the four stimuli. Analysis of the temporal information of LFPs showed that 93% of the LFPs exhibited robust encoding of periodicity pitch. Pitch strength of LFPs derived from the autocorrelogram was significantly (p < 0.001) stronger for rising tones than flat and falling tones. Pitch strength are also significantly increased (p < 0.05) with the characteristic frequency (CF). On the other hand, only 47% (42 or 90) of single neuron activities were significantly synchronized to the fundamental frequency of the stimulus suggesting that the temporal spiking pattern of single IC neuron could encode the time variant periodicity pitch of speech robustly. The difference between the number of LFPs and single neurons that encode the time-variant F0 voice pitch supports the notion of a transition at the level of IC from direct temporal coding in the spike trains of individual neurons to other form of neural representation.

Cortical Processing Related to Intensity of a Modulated Noise Stimulus-a Functional Near-Infrared Study

Sound intensity is a key feature of auditory signals. A profound understanding of cortical processing of this feature is therefore highly desirable. This study investigates whether cortical functional near-infrared spectroscopy (fNIRS) signals reflect sound intensity changes and where on the brain cortex maximal intensity-dependent activations are located. The fNIRS technique is particularly suitable for this kind of hearing study, as it runs silently. Twenty-three normal hearing subjects were included and actively participated in a counterbalanced block design task. Four intensity levels of a modulated noise stimulus with long-term spectrum and modulation characteristics similar to speech were applied, evenly spaced from 15 to 90 dB SPL. Signals from auditory processing cortical fields were derived from a montage of 16 optodes on each side of the head. Results showed that fNIRS responses originating from auditory processing areas are highly dependent on sound intensity level: higher stimulation levels led to higher concentration changes. Caudal and rostral channels showed different waveform morphologies, reflecting specific cortical signal processing of the stimulus. Channels overlying the supramarginal and caudal superior temporal gyrus evoked a phasic response, whereas channels over Broca's area showed a broad tonic pattern. This data set can serve as a foundation for future auditory fNIRS research to develop the technique as a hearing assessment tool in the normal hearing and hearing-impaired populations.

Dichotic Listening Can Improve Perceived Clarity of Music in Cochlear Implant Users

Musical enjoyment for cochlear implant (CI) recipients is often reported to be unsatisfactory. Our goal was to determine whether the musical experience of postlingually deafened adult CI recipients could be enriched by presenting the bass and treble clef parts of short polyphonic piano pieces separately to each ear (dichotic). Dichotic presentation should artificially enhance the lateralization cues of each part and help the listeners to better segregate them and thus provide greater clarity. We also hypothesized that perception of the intended emotion of the pieces and their overall enjoyment would be enhanced in the dichotic mode compared with the monophonic (both parts in the same ear) and the diotic mode (both parts in both ears). Twenty-eight piano pieces specifically composed to induce sad or happy emotions were selected. The tempo of the pieces, which ranged from lento to presto covaried with the intended emotion (from sad to happy). Thirty participants (11 normal-hearing listeners, 11 bimodal CI and hearing-aid users, and 8 bilaterally implanted CI users) participated in this study. Participants were asked to rate the perceived clarity, the intended emotion, and their preference of each piece in different listening modes. Results indicated that dichotic presentation produced small significant improvements in subjective ratings based on perceived clarity. We also found that preference and clarity ratings were significantly higher for pieces with fast tempi compared with slow tempi. However, no significant differences between diotic and dichotic presentation were found for the participants’ preference ratings, or their judgments of intended emotion.

Tone, rhythm, and timbre perception in school-age children using cochlear implants and hearing aids

BACKGROUND

Children with hearing impairments, especially those using hearing devices such as the cochlear implant (CI) or hearing aid (HA), are sometimes not encouraged to attend music classes, as they or their parents and teachers may be unsure whether the child can perform basic musical tasks.

PURPOSE

The objective of the current study was to provide a baseline for the performance of children using CIs and HAs on standardized tests of rhythm and pitch perception as well as an instrument timbre identification task. An additional aim was to determine the effect of structured music training on these measures during the course of a school year.

RESEARCH DESIGN

The Intermediate Measures of Music Audiation (IMMA) Tonal and Rhythmic subtests were administered four times, with 6 wk between tests. All children in the study were also enrolled in "Music Club" teaching sessions. Measures were compared between groups and across the four testing sessions.

STUDY SAMPLE

Twenty children from a single school in Melbourne, Australia, were recruited. Eleven (four girls) had impaired hearing, including six with a unilateral CI or CI and HA together (two girls) and five with bilateral HAs (two girls). Nine were normally hearing, selected to match the age and gender of the hearing-impaired children. Ages ranged from 9-13 yr.

INTERVENTION

All children participated in a weekly Music Club--a 45 min session of musical activities based around vocal play and the integration of aural, visual, and kinesthetic modes of learning.

DATA COLLECTION AND ANALYSIS

Audiological data were collected from clinical files. IMMA scores were converted to percentile ranks using published norms. Between-group differences were tested using repeated-measures analysis of variance, and between-session differences were tested using a linear mixed model. Linear regression was used to model the effect of hearing loss on the test scores.

RESULTS

In the first session, normally hearing children had a mean percentile rank of ∼50 in both the Tonal and Rhythmic subtests of the IMMA. Children using CIs showed trends toward lower scores in the Tonal, but not the Rhythmic, subtests. No significant improvements were found between sessions. In the timbre test, children generally made fewer errors within the set of percussive compared to nonpercussive instruments. The hearing loss level partially predicted performance in the Tonal, but not the Rhythmic, task, and predictions were more significant for nonpercussive compared to percussive instruments.

CONCLUSIONS

The findings highlight the importance of temporal cues in the perception of music, and indicate that temporal cues may be used by children with CIs and HAs in the perception of not only rhythm, but also of some aspects of timbre. We were not able to link participation in the Music Club with increased scores on the Tonal, Rhythmic, and Timbre tests. However, anecdotal evidence from the children and their teachers suggested a wide range of benefits from participation in the Music Club that extended from increased engagement and interest in music classes into the children's social situations.

Loudness of time-varying stimuli with electric stimulation

McKay, Henshall, Farrell, and McDermott [J. Acoust. Soc. Am. 113, 2054-2063 (2003)] developed a practical method to estimate the loudness of periodic electrical signals presented through a cochlear implant. In the present work, this method was extended to time-varying sounds based on two models of time-varying loudness for normal listeners. To fit the model parameters, loudness balancing data was collected with six cochlear implant listeners. The pulse rate of a modulated pulse train was adjusted to equalize its loudness to a reference stimulus. The stimuli were single-electrode time-limited pulse bursts, repeated at a rate of 50 Hz, with on-times varying between 2 and 20 ms. The parameters of two different models of time-varying loudness were fitted to the results. For each model, parameters defining the time windows over which the electrical pulses contribute to instantaneous loudness were optimized. In each case, a good fit was obtained with the loudness balancing results. Therefore, the practical method was successfully extended to time-varying sounds by combining it with existing models of time-varying loudness for acoustic stimulation.

Neurophysiological correlates of configural face processing in schizotypy

BACKGROUND

Face processing impairment in schizophrenia appears to be underpinned by poor configural (as opposed to feature-based) processing; however, few studies have sought to characterize this impairment electrophysiologically. Given the sensitivity of event-related potentials to antipsychotic medications, and the potential for neurophysiological abnormalities to serve as vulnerability markers for schizophrenia, a handful of studies have investigated early visual P100 and face-selective N170 in "at risk" populations. However, this is the first known neurophysiological investigation of configural face processing in a non-clinical schizotypal sample.

METHODS

Using stimuli designed to engage configural processing in face perception (upright and inverted Mooney and photographic faces), P100 and N170 components were recorded in healthy individuals characterized by high (N = 14) and low (N = 14) schizotypal traits according to the Oxford-Liverpool Inventory of Feelings and Experiences.

RESULTS

High schizotypes showed significantly reduced N170 amplitudes to inverted photographic faces. Typical N170 latency and amplitude inversion effects (delayed and enhanced N170 to inverted relative to upright photographic faces, and enhanced amplitude to upright versus inverted Mooney faces), were demonstrated by low, but not high, schizotypes. No group differences were shown for P100 analyses.

CONCLUSIONS

The findings suggest that neurophysiological deficits in processing facial configurations (N170) are apparent in schizotypy, while the early sensory processing (P100) of faces appears intact. This work adds to the mounting evidence for analogous neural processing anomalies at the healthy end of the psychosis continuum.

The acoustic and perceptual cues affecting melody segregation for listeners with a cochlear implant

Our ability to listen selectively to single sound sources in complex auditory environments is termed "auditory stream segregation."This ability is affected by peripheral disorders such as hearing loss, as well as plasticity in central processing such as occurs with musical training. Brain plasticity induced by musical training can enhance the ability to segregate sound, leading to improvements in a variety of auditory abilities. The melody segregation ability of 12 cochlear-implant recipients was tested using a new method to determine the perceptual distance needed to segregate a simple 4-note melody from a background of interleaved random-pitch distractor notes. In experiment 1, participants rated the difficulty of segregating the melody from distracter notes. Four physical properties of the distracter notes were changed. In experiment 2, listeners were asked to rate the dissimilarity between melody patterns whose notes differed on the four physical properties simultaneously. Multidimensional scaling analysis transformed the dissimilarity ratings into perceptual distances. Regression between physical and perceptual cues then derived the minimal perceptual distance needed to segregate the melody. The most efficient streaming cue for CI users was loudness. For the normal hearing listeners without musical backgrounds, a greater difference on the perceptual dimension correlated to the temporal envelope is needed for stream segregation in CI users. No differences in streaming efficiency were found between the perceptual dimensions linked to the F0 and the spectral envelope. Combined with our previous results in normally-hearing musicians and non-musicians, the results show that differences in training as well as differences in peripheral auditory processing (hearing impairment and the use of a hearing device) influences the way that listeners use different acoustic cues for segregating interleaved musical streams.

Adaptation of the communicative brain to post-lingual deafness. Evidence from functional imaging

Not having access to one sense profoundly modifies our interactions with the environment, in turn producing changes in brain organization. Deafness and its rehabilitation by cochlear implantation offer a unique model of brain adaptation during sensory deprivation and recovery. Functional imaging allows the study of brain plasticity as a function of the times of deafness and implantation. Even long after the end of the sensitive period for auditory brain physiological maturation, some plasticity may be observed. In this way the mature brain that becomes deaf after language acquisition can adapt to its modified sensory inputs. Oral communication difficulties induced by post-lingual deafness shape cortical reorganization of brain networks already specialized for processing oral language. Left hemisphere language specialization tends to be more preserved than functions of the right hemisphere. We hypothesize that the right hemisphere offers cognitive resources re-purposed to palliate difficulties in left hemisphere speech processing due to sensory and auditory memory degradation. If cochlear implantation is considered, this reorganization during deafness may influence speech understanding outcomes positively or negatively. Understanding brain plasticity during post-lingual deafness should thus inform the development of cognitive rehabilitation, which promotes positive reorganization of the brain networks that process oral language before surgery. This article is part of a Special Issue entitled Human Auditory Neuroimaging.

Evidence for enhanced multisensory facilitation with stimulus relevance: an electrophysiological investigation

Currently debate exists relating to the interplay between multisensory processes and bottom-up and top-down influences. However, few studies have looked at neural responses to newly paired audiovisual stimuli that differ in their prescribed relevance. For such newly associated audiovisual stimuli, optimal facilitation of motor actions was observed only when both components of the audiovisual stimuli were targets. Relevant auditory stimuli were found to significantly increase the amplitudes of the event-related potentials at the occipital pole during the first 100 ms post-stimulus onset, though this early integration was not predictive of multisensory facilitation. Activity related to multisensory behavioral facilitation was observed approximately 166 ms post-stimulus, at left central and occipital sites. Furthermore, optimal multisensory facilitation was found to be associated with a latency shift of induced oscillations in the beta range (14–30 Hz) at right hemisphere parietal scalp regions. These findings demonstrate the importance of stimulus relevance to multisensory processing by providing the first evidence that the neural processes underlying multisensory integration are modulated by the relevance of the stimuli being combined. We also provide evidence that such facilitation may be mediated by changes in neural synchronization in occipital and centro-parietal neural populations at early and late stages of neural processing that coincided with stimulus selection, and the preparation and initiation of motor action.

Interhemispheric transfer time in patients with auditory hallucinations: an auditory event-related potential study

Central auditory processing in schizophrenia patients with a history of auditory hallucinations has been reported to be impaired, and abnormalities of interhemispheric transfer have been implicated in these patients. This study examined interhemispheric functional connectivity between auditory cortical regions, using temporal information obtained from latency measures of the auditory N1 evoked potential. Interhemispheric Transfer Times (IHTTs) were compared across 3 subject groups: schizophrenia patients who had experienced auditory hallucinations, schizophrenia patients without a history of auditory hallucinations, and normal controls. Pure tones and single-syllable words were presented monaurally to each ear, while EEG was recorded continuously. IHTT was calculated for each stimulus type by comparing the latencies of the auditory N1 evoked potential recorded contralaterally and ipsilaterally to the ear of stimulation. The IHTTs for pure tones did not differ between groups. For word stimuli, the IHTT was significantly different across the 3 groups: the IHTT was close to zero in normal controls, was highest in the AH group, and was negative (shorter latencies ipsilaterally) in the nonAH group. Differences in IHTTs may be attributed to transcallosal dysfunction in the AH group, but altered or reversed cerebral lateralization in nonAH participants is also possible.

The effect of visual cues on difficulty ratings for segregation of musical streams in listeners with impaired hearing

Background

Enjoyment of music is an important part of life that may be degraded for people with hearing impairments, especially those using cochlear implants. The ability to follow separate lines of melody is an important factor in music appreciation. This ability relies on effective auditory streaming, which is much reduced in people with hearing impairment, contributing to difficulties in music appreciation. The aim of this study was to assess whether visual cues could reduce the subjective difficulty of segregating a melody from interleaved background notes in normally hearing listeners, those using hearing aids, and those using cochlear implants.

Methodology/Principal Findings

Normally hearing listeners (N = 20), hearing aid users (N = 10), and cochlear implant users (N = 11) were asked to rate the difficulty of segregating a repeating four-note melody from random interleaved distracter notes. The pitch of the background notes was gradually increased or decreased throughout blocks, providing a range of difficulty from easy (with a large pitch separation between melody and distracter) to impossible (with the melody and distracter completely overlapping). Visual cues were provided on half the blocks, and difficulty ratings for blocks with and without visual cues were compared between groups. Visual cues reduced the subjective difficulty of extracting the melody from the distracter notes for normally hearing listeners and cochlear implant users, but not hearing aid users.

Conclusion/Significance

Simple visual cues may improve the ability of cochlear implant users to segregate lines of music, thus potentially increasing their enjoyment of music. More research is needed to determine what type of acoustic cues to encode visually in order to optimise the benefits they may provide.

The relationship between multisensory integration and IQ in children

It is well accepted that multisensory integration has a facilitative effect on perceptual and motor processes, evolutionarily enhancing the chance of survival of many species, including humans. Yet, there is limited understanding of the relationship between multisensory processes, environmental noise, and children's cognitive abilities. Thus, this study investigated the relationship between multisensory integration, auditory background noise, and the general intellectual abilities of school-age children (N = 88, mean age = 9 years, 7 months) using a simple audiovisual detection paradigm. We provide evidence that children with enhanced multisensory integration in quiet and noisy conditions are likely to score above average on the Full-Scale IQ of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV). Conversely, approximately 45% of tested children, with relatively low verbal and nonverbal intellectual abilities, showed reduced multisensory integration in either quiet or noise. Interestingly, approximately 20% of children showed improved multisensory integration abilities in the presence of auditory background noise. The findings of the present study suggest that stable and consistent multisensory integration in quiet and noisy environments is associated with the development of optimal general intellectual abilities. Further theoretical implications are discussed.

Reduced connectivity of the auditory cortex in patients with auditory hallucinations: a resting state functional magnetic resonance imaging study

BACKGROUND

Previous research has reported auditory processing deficits that are specific to schizophrenia patients with a history of auditory hallucinations (AH). One explanation for these findings is that there are abnormalities in the interhemispheric connectivity of auditory cortex pathways in AH patients; as yet this explanation has not been experimentally investigated. We assessed the interhemispheric connectivity of both primary (A1) and secondary (A2) auditory cortices in n=13 AH patients, n=13 schizophrenia patients without auditory hallucinations (non-AH) and n=16 healthy controls using functional connectivity measures from functional magnetic resonance imaging (fMRI) data.

METHOD

Functional connectivity was estimated from resting state fMRI data using regions of interest defined for each participant based on functional activation maps in response to passive listening to words. Additionally, stimulus-induced responses were regressed out of the stimulus data and the functional connectivity was estimated for the same regions to investigate the reliability of the estimates.

RESULTS

AH patients had significantly reduced interhemispheric connectivity in both A1 and A2 when compared with non-AH patients and healthy controls. The latter two groups did not show any differences in functional connectivity. Further, this pattern of findings was similar across the two datasets, indicating the reliability of our estimates.

CONCLUSIONS

These data have identified a trait deficit specific to AH patients. Since this deficit was characterized within both A1 and A2 it is expected to result in the disruption of multiple auditory functions, for example, the integration of basic auditory information between hemispheres (via A1) and higher-order language processing abilities (via A2).

A multivariate electrophysiological endophenotype, from a unitary cohort, shows greater research utility than any single feature in the Western Australian family study of schizophrenia

BACKGROUND

Previous studies have found several electrophysiological endophenotypes that each co-varies individually with schizophrenia. This study extends these investigations to compare and contrast four electrophysiological endophenotype, mismatch negativity, P50, P300, and antisaccades, and analyze their covariance on the basis of a single cohort tested with all paradigms. We report a multivariate endophenotype that is maximally associated with diagnosis and evaluate this new endophenotype with respect to its application to genetic analysis.

METHODS

Group differences and covariance were analyzed for probands (n = 60), family members (n = 53), and control subjects (n = 44). Associations between individual endophenotypes and diagnostic groups, as well as between the multivariate endophenotype and diagnostic groups, were investigated with logistic regression.

RESULTS

Results from all four individual endophenotypes replicated previous findings of deficits in the proband group. The P50 and P300 endophenotypes similarly replicated significant deficits in the family member group, whereas mismatch negativity and antisaccade measures showed a trend. There was minimal correlation between the different endophenotypes. A logistic regression model based on all four features significantly represented the diagnostic grouping (chi(2) = 32.7; p < .001), with 80% accuracy in predicting group membership.

CONCLUSIONS

A multivariate endophenotype, based on a weighted combination of electrophysiological features, provides greater diagnostic classification power than any single endophenotype.

Duration mismatch negativity in biological relatives of patients with schizophrenia spectrum disorders

BACKGROUND

One of the most consistent findings in schizophrenia research over the past decade is a reduction in the amplitude of an auditory event-related brain potential known as mismatch negativity (MMN), which is generated whenever a deviant sound occurs in a background of repetitive auditory stimulation. The reduced amplitude of MMN in schizophrenia was first observed for deviant sounds that differ in duration relative to background standard sounds, and similar findings have been observed for sounds that are deviant in frequency. The aim of this study was to determine whether first-degree relatives of schizophrenia patients show a similar reduction in MMN amplitude to duration deviants.

METHODS

We measured MMN to duration increments (deviants 100 msec vs. standards 50 msec) in 22 medicated patients with a diagnosis in the schizophrenia spectrum, 17 individuals who were first-degree unaffected relatives of patients, and 21 healthy control subjects.

RESULTS

Mismatch negativity amplitude was reduced in patients and relatives compared with control subjects. There were no significant differences between patients and relatives. In contrast, the subsequent positive component, P3a, was larger in relatives compared with patients.

CONCLUSIONS

These findings suggest that a reduced MMN amplitude may be an endophenotype marker of the predisposition to schizophrenia.