Spatial release from masking with noise-vocoded speech

Predicting speech-in-speech recognition: Short-term audibility and spatial separation

Quantifying the factors that predict variability in speech-in-speech recognition represents a fundamental challenge in auditory science. Stimulus factors associated with energetic and informational masking (IM) modulate variability in speech-in-speech recognition, but energetic effects can be difficult to estimate in spectro-temporally dynamic speech maskers. The current experiment characterized the effects of short-term audibility and differences in target and masker location (or perceived location) on the horizontal plane for sentence recognition in two-talker speech. Thirty young adults with normal hearing (NH) participated. Speech reception thresholds and keyword recognition at a fixed signal-to-noise ratio (SNR) were measured in each spatial condition. Short-term audibility for each keyword was quantified using a glimpsing model. Results revealed that speech-in-speech recognition depended on the proportion of audible glimpses available in the target + masker keyword stimulus in each spatial condition, even across stimuli presented at a fixed global SNR. Short-term audibility requirements were greater for colocated than spatially separated speech-in-speech recognition, and keyword recognition improved more rapidly as a function of increases in target audibility with spatial separation. Results indicate that spatial cues enhance glimpsing efficiency in competing speech for young adults with NH and provide a quantitative framework for estimating IM for speech-in-speech recognition in different spatial configurations.

Strength of target source segregation cues affects the outcome of speech-on-speech masking experiments

In speech-on-speech listening experiments, some means for designating which talker is the "target" must be provided for the listener to perform better than chance. However, the relative strength of the segregation variables designating the target could affect the results of the experiment. Here, we examine the interaction of two source segregation variables-spatial separation and talker gender differences-and demonstrate that the relative strengths of these cues may affect the interpretation of the results. Participants listened to sentence pairs spoken by different-gender target and masker talkers, presented naturally or vocoded (degrading gender cues), either colocated or spatially separated. Target and masker words were temporally interleaved to eliminate energetic masking in either an every-other-word or randomized order of presentation. Results showed that the order of interleaving had no effect on recall performance. For natural speech with strong talker gender cues, spatial separation of sources yielded no improvement in performance. For vocoded speech with degraded talker gender cues, performance improved significantly with spatial separation of sources. These findings reveal that listeners may shift among target source segregation cues contingent on cue viability. Finally, performance was poor when the target was designated after stimulus presentation, indicating strong reliance on the cues.

Maturation of Speech-in-Speech Recognition for Whispered and Voiced Speech

PURPOSE

Some speech recognition data suggest that children rely less on voice pitch and harmonicity to support auditory scene analysis than adults. Two experiments evaluated development of speech-in-speech recognition using voiced speech and whispered speech, which lacks the harmonic structure of voiced speech.

METHOD

Listeners were 5- to 7-year-olds and adults with normal hearing. Targets were monosyllabic words organized into three-word sets that differ in vowel content. Maskers were two-talker or one-talker streams of speech. Targets and maskers were recorded by different female talkers in both voiced and whispered speaking styles. For each masker, speech reception thresholds (SRTs) were measured in all four combinations of target and masker speech, including matched and mismatched speaking styles for the target and masker.

RESULTS

Children performed more poorly than adults overall. For the two-talker masker, this age effect was smaller for the whispered target and masker than for the other three conditions. Children's SRTs in this condition were predominantly positive, suggesting that they may have relied on a wholistic listening strategy rather than segregating the target from the masker. For the one-talker masker, age effects were consistent across the four conditions. Reduced informational masking for the one-talker masker could be responsible for differences in age effects for the two maskers. A benefit of mismatching the target and masker speaking style was observed for both target styles in the two-talker masker and for the voiced targets in the one-talker masker.

CONCLUSIONS

These results provide no compelling evidence that young school-age children and adults are differentially sensitive to the cues present in voiced and whispered speech. Both groups benefit from mismatches in speaking style under some conditions. These benefits could be due to a combination of reduced perceptual similarity, harmonic cancelation, and differences in energetic masking.

The time-course of linguistic interference during native and non-native speech-in-speech listening

Recognizing speech in a noisy background is harder when the background is time-forward than for time-reversed speech, a masker direction effect, and harder when the masker is in a known rather than an unknown language, indicating linguistic interference. We examined the masker direction effect when the masker was a known vs unknown language and calculated performance over 50 trials to assess differential masker adaptation. In experiment 1, native English listeners transcribing English sentences showed a larger masker direction effect with English than Mandarin maskers. In experiment 2, Mandarin non-native speakers of English transcribing Mandarin sentences showed a mirror pattern. Both experiments thus support the target-masker linguistic similarity hypothesis, where interference is maximal when target and masker languages are the same. In experiment 3, Mandarin non-native speakers of English transcribing English sentences showed comparable results for English and Mandarin maskers. Non-native listening is therefore consistent with the known-language interference hypothesis, where interference is maximal when the masker language is known to the listener, whether or not it matches the target language. A trial-by-trial analysis showed that the masker direction effect increased over time during native listening but not during non-native listening. The results indicate different target-to-masker streaming strategies during native and non-native speech-in-speech listening.

Effects of better-ear glimpsing, binaural unmasking, and spectral resolution on spatial release from masking in cochlear-implant users

Bilateral cochlear-implant (BICI) listeners obtain less spatial release from masking (SRM; speech-recognition improvement for spatially separated vs co-located conditions) than normal-hearing (NH) listeners, especially for symmetrically placed maskers that produce similar long-term target-to-masker ratios at the two ears. Two experiments examined possible causes of this deficit, including limited better-ear glimpsing (using speech information from the more advantageous ear in each time-frequency unit), limited binaural unmasking (using interaural differences to improve signal-in-noise detection), or limited spectral resolution. Listeners had NH (presented with unprocessed or vocoded stimuli) or BICIs. Experiment 1 compared natural symmetric maskers, idealized monaural better-ear masker (IMBM) stimuli that automatically performed better-ear glimpsing, and hybrid stimuli that added worse-ear information, potentially restoring binaural cues. BICI and NH-vocoded SRM was comparable to NH-unprocessed SRM for idealized stimuli but was 14%-22% lower for symmetric stimuli, suggesting limited better-ear glimpsing ability. Hybrid stimuli improved SRM for NH-unprocessed listeners but degraded SRM for BICI and NH-vocoded listeners, suggesting they experienced across-ear interference instead of binaural unmasking. In experiment 2, increasing the number of vocoder channels did not change NH-vocoded SRM. BICI SRM deficits likely reflect a combination of across-ear interference, limited better-ear glimpsing, and poorer binaural unmasking that stems from cochlear-implant-processing limitations other than reduced spectral resolution.

Spatial release from informational masking enhances the early cortical representation of speech sounds

INTRODUCTION

Spatial separation between competing speech streams reduces their confusion (informational masking), improving speech processing under challenging listening conditions. The precise stages of auditory processing involved in this benefit are not fully understood. This study used event-related potentials to examine the processing of target speech under conditions of informational masking and its spatial release.

METHODS

Participants detected noise-vocoded target speech presented with two-talker noise-vocoded masking speech. In separate conditions, the same set of targets were spatially co-located with maskers to produce informational masking and spatially separated from maskers using a perceptual manipulation to release the informational masking.

RESULTS

An increase in N1 and P2 amplitude, consistent with cortical auditory evoked potentials, and a later sustained positivity (P300) were observed in response to target onsets only under conditions supporting release from informational masking. At target intensities above masking threshold in both spatial conditions, N1 and P2 latencies were shorter when targets and maskers were perceptually separated.

DISCUSSION

These results indicate that spatial release from informational masking benefits speech representation beginning in the early stages of auditory perception. Additionally, these results suggest that the auditory evoked potential itself may be heavily dependent upon how information is perceptually organized rather than physically organized.

Sex-mismatch benefit for speech-in-speech recognition by pediatric and adult cochlear implant users

This project investigated whether pediatric (5-14 years) and adult (30-60 years) cochlear implant (CI) users benefit from a target/masker sex-mismatch for speech-in-speech recognition. Speech recognition thresholds were estimated in a two-male-talker or a two-female-talker masker. Target and masker speech were either sex-matched or sex-mismatched. For both age groups, performance for sex-matched talkers was worse for male than female speech. Sex-mismatch benefit was observed for the two-male-talker masker, indicating CI users can benefit from a target/masker sex mismatch. No benefit was observed for the two-female-talker masker, suggesting this effect may depend on the relative contributions of energetic and informational masking.

Head Shadow, Summation, and Squelch in Bilateral Cochlear-Implant Users With Linked Automatic Gain Controls

Speech understanding in noise is poorer in bilateral cochlear-implant (BICI) users compared to normal-hearing counterparts. Independent automatic gain controls (AGCs) may contribute to this because adjusting processor gain independently can reduce interaural level differences that BICI listeners rely on for bilateral benefits. Bilaterally linked AGCs may improve bilateral benefits by increasing the magnitude of interaural level differences. The effects of linked AGCs on bilateral benefits (summation, head shadow, and squelch) were measured in nine BICI users. Speech understanding for a target talker at 0° masked by a single talker at 0°, 90°, or −90° azimuth was assessed under headphones with sentences at five target-to-masker ratios. Research processors were used to manipulate AGC type (independent or linked) and test ear (left, right, or both). Sentence recall was measured in quiet to quantify individual interaural asymmetry in functional performance. The results showed that AGC type did not significantly change performance or bilateral benefits. Interaural functional asymmetries, however, interacted with ear such that greater summation and squelch benefit occurred when there was larger functional asymmetry, and interacted with interferer location such that smaller head shadow benefit occurred when there was larger functional asymmetry. The larger benefits for those with larger asymmetry were driven by improvements from adding a better-performing ear, rather than a true binaural-hearing benefit. In summary, linked AGCs did not significantly change bilateral benefits in cases of speech-on-speech masking with a single-talker masker, but there was also no strong detriment across a range of target-to-masker ratios, within a small and diverse BICI listener population.

Spoken narrative comprehension for young adult listeners: effects of competing voices and noise

OBJECTIVE

To examine the influence of competing voices or noise on the comprehension of spoken narratives for young adults.

DESIGN

First, an intelligibility assessment of the target narratives was conducted to establish a signal-to-noise ratio ensuring accurate initial speech recognition. Then, narrative comprehension for two target types (fixed and varied target talker) was measured in four listening conditions (quiet, one-talker speech, speech babble, speech-shaped noise). After hearing target narratives in each listening condition, participants completed a visual recognition memory task that assessed the comprehension of the narrative materials at three levels of representation (surface form, propositional, event model).

STUDY SAMPLE

Seventy adults (18-32 years of age).

RESULTS

Narrative comprehension results revealed a main effect of listening condition at the event model level, indicating poorer narrative memory of described situations for all noise conditions compared to quiet. Increased positive responses to thematically consistent but situationally "wrong" memory probes drove this effect. No other significant effects were observed.

CONCLUSION

Despite near-perfect speech recognition, background noise negatively influenced aspects of spoken narrative comprehension and memory. Specifically, noise did not disrupt memory for what was said (surface form and propositional memory), but only memory for what was talked about (event model memory).

Spatial release from informational masking declines with age: Evidence from a detection task in a virtual separation paradigm

Declines in spatial release from informational masking may contribute to the speech-processing difficulties that older adults often experience within complex listening environments. The present study sought to answer two fundamental questions: (1) Does spatial release from informational masking decline with age and, if so, (2) does age predict this decline independently of age-typical hearing loss? Younger (18-34 years) and older (60-80 years) adults with age-typical hearing completed a yes/no target-detection task with low-pass filtered noise-vocoded speech designed to reduce non-spatial segregation cues and control for hearing loss. Participants detected a target voice among two-talker masking babble while a virtual spatial separation paradigm [Freyman, Helfer, McCall, and Clifton, J. Acoust. Soc. Am. 106(6), 3578-3588 (1999)] was used to isolate informational masking release. The younger and older adults both exhibited spatial release from informational masking, but masking release was reduced among the older adults. Furthermore, age predicted this decline controlling for hearing loss, while there was no indication that hearing loss played a role. These findings provide evidence that declines specific to aging limit spatial release from informational masking under challenging listening conditions.

The Effect of Nonlinear Amplitude Growth on the Speech Perception Benefits Provided by a Single-Sided Vocoder

Purpose For listeners with single-sided deafness, a cochlear implant (CI) can improve speech understanding by giving the listener access to the ear with the better target-to-masker ratio (TMR; head shadow) or by providing interaural difference cues to facilitate the perceptual separation of concurrent talkers (squelch). CI simulations presented to listeners with normal hearing examined how these benefits could be affected by interaural differences in loudness growth in a speech-on-speech masking task. Method Experiment 1 examined a target-masker spatial configuration where the vocoded ear had a poorer TMR than the nonvocoded ear. Experiment 2 examined the reverse configuration. Generic head-related transfer functions simulated free-field listening. Compression or expansion was applied independently to each vocoder channel (power-law exponents: 0.25, 0.5, 1, 1.5, or 2). Results Compression reduced the benefit provided by the vocoder ear in both experiments. There was some evidence that expansion increased squelch in Experiment 1 but reduced the benefit in Experiment 2 where the vocoder ear provided a combination of head-shadow and squelch benefits. Conclusions The effects of compression and expansion are interpreted in terms of envelope distortion and changes in the vocoded-ear TMR (for head shadow) or changes in perceived target-masker spatial separation (for squelch). The compression parameter is a candidate for clinical optimization to improve single-sided deafness CI outcomes.

The Effect of Simulated Interaural Frequency Mismatch on Speech Understanding and Spatial Release From Masking

OBJECTIVE

The binaural-hearing system interaurally compares inputs, which underlies the ability to localize sound sources and to better understand speech in complex acoustic environments. Cochlear implants (CIs) are provided in both ears to increase binaural-hearing benefits; however, bilateral CI users continue to struggle with understanding speech in the presence of interfering sounds and do not achieve the same level of spatial release from masking (SRM) as normal-hearing listeners. One reason for diminished SRM in CI users could be that the electrode arrays are inserted at different depths in each ear, which would cause an interaural frequency mismatch. Because interaural frequency mismatch diminishes the salience of interaural differences for relatively simple stimuli, it may also diminish binaural benefits for spectral-temporally complex stimuli like speech. This study evaluated the effect of simulated frequency-to-place mismatch on speech understanding and SRM.

DESIGN

Eleven normal-hearing listeners were tested on a speech understanding task. There was a female target talker who spoke five-word sentences from a closed set of words. There were two interfering male talkers who spoke unrelated sentences. Nonindividualized head-related transfer functions were used to simulate a virtual auditory space. The target was presented from the front (0°), and the interfering speech was either presented from the front (colocated) or from 90° to the right (spatially separated). Stimuli were then processed by an eight-channel vocoder with tonal carriers to simulate aspects of listening through a CI. Frequency-to-place mismatch ("shift") was introduced by increasing the center frequency of the synthesis filters compared with the corresponding analysis filters. Speech understanding was measured for different shifts (0, 3, 4.5, and 6 mm) and target-to-masker ratios (TMRs: +10 to -10 dB). SRM was calculated as the difference in the percentage of correct words for the colocated and separated conditions. Two types of shifts were tested: (1) bilateral shifts that had the same frequency-to-place mismatch in both ears, but no interaural frequency mismatch, and (2) unilateral shifts that produced an interaural frequency mismatch.

RESULTS

For the bilateral shift conditions, speech understanding decreased with increasing shift and with decreasing TMR, for both colocated and separate conditions. There was, however, no interaction between shift and spatial configuration; in other words, SRM was not affected by shift. For the unilateral shift conditions, speech understanding decreased with increasing interaural mismatch and with decreasing TMR for both the colocated and spatially separated conditions. Critically, there was a significant interaction between the amount of shift and spatial configuration; in other words, SRM decreased for increasing interaural mismatch.

CONCLUSIONS

A frequency-to-place mismatch in one or both ears resulted in decreased speech understanding. SRM, however, was only affected in conditions with unilateral shifts and interaural frequency mismatch. Therefore, matching frequency information between the ears provides listeners with larger binaural-hearing benefits, for example, improved speech understanding in the presence of interfering talkers. A clinical procedure to reduce interaural frequency mismatch when programming bilateral CIs may improve benefits in speech segregation that are due to binaural-hearing abilities.

Segregation of voices with single or double fundamental frequencies

In cocktail-party situations, listeners can use the fundamental frequency (F0) of a voice to segregate it from competitors, but other cues in speech could help, such as co-modulation of envelopes across frequency or more complex cues related to the semantic/syntactic content of the utterances. For simplicity, this (non-pitch) form of grouping is referred to as "articulatory." By creating a new type of speech with two steady F0s, it was examined how these two forms of segregation compete: articulatory grouping would bind the partials of a double-F0 source together, whereas harmonic segregation would tend to split them in two subsets. In experiment 1, maskers were two same-male sentences. Speech reception thresholds were high in this task (vicinity of 0 dB), and harmonic segregation behaved as though double-F0 stimuli were two independent sources. This was not the case in experiment 2, where maskers were speech-shaped complexes (buzzes). First, double-F0 targets were immune to the masking of a single-F0 buzz matching one of the two target F0s. Second, double-F0 buzzes were particularly effective at masking a single-F0 target matching one of the two buzz F0s. As a conclusion, the strength of F0-segregation appears to depend on whether the masker is speech or not.

The Effects of Musical Training on Speech Detection in the Presence of Informational and Energetic Masking

Recent research has suggested that musicians have an advantage in some speech-in-noise paradigms, but not all. Whether musicians outperform nonmusicians on a given speech-in-noise task may well depend on the type of noise involved. To date, few groups have specifically studied the role that informational masking plays in the observation of a musician advantage. The current study investigated the effect of musicianship on listeners’ ability to overcome informational versus energetic masking of speech. Monosyllabic words were presented in four conditions that created similar energetic masking but either high or low informational masking. Two of these conditions used noise-vocoded target and masking stimuli to determine whether the absence of natural fine structure and spectral variations influenced any musician advantage. Forty young normal-hearing listeners (20 musicians and 20 nonmusicians) completed the study. There was a significant overall effect of participant group collapsing across the four conditions; however, planned comparisons showed musicians’ thresholds were only significantly better in the high informational masking natural speech condition, where the musician advantage was approximately 3 dB. These results add to the mounting evidence that informational masking plays a role in the presence and amount of musician benefit.

Factors Affecting Speech Reception in Background Noise with a Vocoder Implementation of the FAST Algorithm

Speech segregation in background noise remains a difficult task for individuals with hearing loss. Several signal processing strategies have been developed to improve the efficacy of hearing assistive technologies in complex listening environments. The present study measured speech reception thresholds in normal-hearing listeners attending to a vocoder based on the Fundamental Asynchronous Stimulus Timing algorithm (FAST: Smith et al. 2014), which triggers pulses based on the amplitudes of channel magnitudes in order to preserve envelope timing cues, with two different reconstruction bandwidths (narrowband and broadband) to control the degree of spectrotemporal resolution. Five types of background noise were used including same male talker, female talker, time-reversed male talker, time-reversed female talker, and speech-shaped noise to probe the contributions of different types of speech segregation cues and to elucidate how degradation affects speech reception across these conditions. Maskers were spatialized using head-related transfer functions in order to create co-located and spatially separated conditions. Results indicate that benefits arising from voicing and spatial cues can be preserved using the FAST algorithm but are reduced with a reduction in spectral resolution.

Spatial Release From Masking in Adults With Bilateral Cochlear Implants: Effects of Distracter Azimuth and Microphone Location

PURPOSE

The primary purpose of this study was to derive spatial release from masking (SRM) performance-azimuth functions for bilateral cochlear implant (CI) users to provide a thorough description of SRM as a function of target/distracter spatial configuration. The secondary purpose of this study was to investigate the effect of the microphone location for SRM in a within-subject study design.

METHOD

Speech recognition was measured in 12 adults with bilateral CIs for 11 spatial separations ranging from -90° to +90° in 20° steps using an adaptive block design. Five of the 12 participants were tested with both the behind-the-ear microphones and a T-mic configuration to further investigate the effect of mic location on SRM.

RESULTS

SRM can be significantly affected by the hemifield origin of the distracter stimulus-particularly for listeners with interaural asymmetry in speech understanding. The greatest SRM was observed with a distracter positioned 50° away from the target. There was no effect of mic location on SRM for the current experimental design.

CONCLUSION

Our results demonstrate that the traditional assessment of SRM with a distracter positioned at 90° azimuth may underestimate maximum performance for individuals with bilateral CIs.

The Effect of Interaural Mismatches on Contralateral Unmasking With Single-Sided Vocoders

OBJECTIVES

Cochlear-implant (CI) users with single-sided deafness (SSD)-that is, one normal-hearing (NH) ear and one CI ear-can obtain some unmasking benefits when a mixture of target and masking voices is presented to the NH ear and a copy of just the masking voices is presented to the CI ear. NH listeners show similar benefits in a simulation of SSD-CI listening, whereby a mixture of target and masking voices is presented to one ear and a vocoded copy of the masking voices is presented to the opposite ear. However, the magnitude of the benefit for SSD-CI listeners is highly variable across individuals and is on average less than for NH listeners presented with vocoded stimuli. One possible explanation for the limited benefit observed for some SSD-CI users is that temporal and spectral discrepancies between the acoustic and electric ears might interfere with contralateral unmasking. The present study presented vocoder simulations to NH participants to examine the effects of interaural temporal and spectral mismatches on contralateral unmasking.

DESIGN

Speech-reception performance was measured in a competing-talker paradigm for NH listeners presented with vocoder simulations of SSD-CI listening. In the monaural condition, listeners identified target speech masked by two same-gender interferers, presented to the left ear. In the bilateral condition, the same stimuli were presented to the left ear, but the right ear was presented with a noise-vocoded copy of the interfering voices. This paradigm tested whether listeners could integrate the interfering voices across the ears to better hear the monaural target. Three common distortions inherent in CI processing were introduced to the vocoder processing: spectral shifts, temporal delays, and reduced frequency selectivity.

RESULTS

In experiment 1, contralateral unmasking (i.e., the benefit from adding the vocoded maskers to the second ear) was impaired by spectral mismatches of four equivalent rectangular bandwidths or greater. This is equivalent to roughly a 3.6-mm mismatch between the cochlear places stimulated in the electric and acoustic ears, which is on the low end of the average expected mismatch for SSD-CI listeners. In experiment 2, performance was negatively affected by a temporal mismatch of 24 ms or greater, but not for mismatches in the 0 to 12 ms range expected for SSD-CI listeners. Experiment 3 showed an interaction between spectral shift and spectral resolution, with less effect of interaural spectral mismatches when the number of vocoder channels was reduced. Experiment 4 applied interaural spectral and temporal mismatches in combination. Performance was best when both frequency and timing were aligned, but in cases where a mismatch was present in one dimension (either frequency or latency), the addition of mismatch in the second dimension did not further disrupt performance.

CONCLUSIONS

These results emphasize the need for interaural alignment-in timing and especially in frequency-to maximize contralateral unmasking for NH listeners presented with vocoder simulations of SSD-CI listening. Improved processing strategies that reduce mismatch between the electric and acoustic ears of SSD-CI listeners might improve their ability to obtain binaural benefits in multitalker environments.

Head Shadow and Binaural Squelch for Unilaterally Deaf Cochlear Implantees

BACKGROUND

Cochlear implants (CIs) can improve speech-in-noise performance for listeners with unilateral sensorineural deafness. But these benefits are modest and in most cases are limited to head-shadow advantages, with little evidence of binaural squelch.

HYPOTHESIS

The goal of the investigation was to determine whether CI listeners with normal hearing or moderate hearing loss in the contralateral ear would receive a larger head-shadow benefit for target speech and noise originating from opposite sides of the head, and whether listeners would experience binaural squelch in the free field in a test involving interfering talkers.

METHODS

Eleven CI listeners performed a speech-identification task in the presence of interfering noise or speech. Six listeners had single-sided deafness (normal or near-normal audiometric thresholds in the acoustic ear) and five had asymmetric hearing loss (hearing loss in the acoustic ear treated with a hearing aid). Listeners were tested with the acoustic ear only and bilaterally with the CI turned on. One set of conditions examined head-shadow effects with target speech and masking noise presented from azimuths of 0 or ±108 degrees. A second set of conditions examined binaural squelch, with target speech presented from the front and interfering talkers symmetrically placed on both sides.

RESULTS

On average, the largest head-shadow benefit (5 dB) occurred when the target and masking noise were presented on opposite sides of the head. Listeners also showed an average of 2 dB of squelch, but only when the target speech was masked by interfering talkers of the same sex as the target.

CONCLUSIONS

CIs provide listeners with unilateral deafness important benefits for speech perception in complex spatial environments, including a larger head-shadow benefit when speech and noise originate on opposite sides of the head, and an improved ability to perceptually organize an auditory scene with multiple competing voices.The views expressed in this abstract are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or US Government.

Listeners Experience Linguistic Masking Release in Noise-Vocoded Speech-in-Speech Recognition

PURPOSE

The purpose of this study was to evaluate whether listeners with normal hearing perceiving noise-vocoded speech-in-speech demonstrate better intelligibility of target speech when the background speech was mismatched in language (linguistic release from masking [LRM]) and/or location (spatial release from masking [SRM]) relative to the target. We also assessed whether the spectral resolution of the noise-vocoded stimuli affected the presence of LRM and SRM under these conditions.

METHOD

In Experiment 1, a mixed factorial design was used to simultaneously manipulate the masker language (within-subject, English vs. Dutch), the simulated masker location (within-subject, right, center, left), and the spectral resolution (between-subjects, 6 vs. 12 channels) of noise-vocoded target-masker combinations presented at +25 dB signal-to-noise ratio (SNR). In Experiment 2, the study was repeated using a spectral resolution of 12 channels at +15 dB SNR.

RESULTS

In both experiments, listeners' intelligibility of noise-vocoded targets was better when the background masker was Dutch, demonstrating reliable LRM in all conditions. The pattern of results in Experiment 1 was not reliably different across the 6- and 12-channel noise-vocoded speech. Finally, a reliable spatial benefit (SRM) was detected only in the more challenging SNR condition (Experiment 2).

CONCLUSION

The current study is the first to report a clear LRM benefit in noise-vocoded speech-in-speech recognition. Our results indicate that this benefit is available even under spectrally degraded conditions and that it may augment the benefit due to spatial separation of target speech and competing backgrounds.

Having Two Ears Facilitates the Perceptual Separation of Concurrent Talkers for Bilateral and Single-Sided Deaf Cochlear Implantees

OBJECTIVES

Listening to speech with multiple competing talkers requires the perceptual separation of the target voice from the interfering background. Normal-hearing listeners are able to take advantage of perceived differences in the spatial locations of competing sound sources to facilitate this process. Previous research suggests that bilateral (BI) cochlear-implant (CI) listeners cannot do so, and it is unknown whether single-sided deaf (SSD) CI users (one acoustic and one CI ear) have this ability. This study investigated whether providing a second ear via cochlear implantation can facilitate the perceptual separation of targets and interferers in a listening situation involving multiple competing talkers.

DESIGN

BI-CI and SSD-CI listeners were required to identify speech from a target talker mixed with one or two interfering talkers. In the baseline monaural condition, the target speech and the interferers were presented to one of the CIs (for the BI-CI listeners) or to the acoustic ear (for the SSD-CI listeners). In the bilateral condition, the target was still presented to the first ear but the interferers were presented to both the target ear and the listener's second ear (always a CI), thereby testing whether CI listeners could use information about the interferer obtained from a second ear to facilitate perceptual separation of the target and interferer.

RESULTS

Presenting a copy of the interfering signals to the second ear improved performance, up to 4 to 5 dB (12 to 18 percentage points), but the amount of improvement depended on the type of interferer. For BI-CI listeners, the improvement occurred mainly in conditions involving one interfering talker, regardless of gender. For SSD-CI listeners, the improvement occurred in conditions involving one or two interfering talkers of the same gender as the target. This interaction is consistent with the idea that the SSD-CI listeners had access to pitch cues in their normal-hearing ear to separate the opposite-gender target and interferers, while the BI-CI listeners did not.

CONCLUSIONS

These results suggest that a second auditory input via a CI can facilitate the perceptual separation of competing talkers in situations where monaural cues are insufficient to do so, thus partially restoring a key advantage of having two ears that was previously thought to be inaccessible to CI users.

Role of Binaural Temporal Fine Structure and Envelope Cues in Cocktail-Party Listening

While conversing in a crowded social setting, a listener is often required to follow a target speech signal amid multiple competing speech signals (the so-called "cocktail party" problem). In such situations, separation of the target speech signal in azimuth from the interfering masker signals can lead to an improvement in target intelligibility, an effect known as spatial release from masking (SRM). This study assessed the contributions of two stimulus properties that vary with separation of sound sources, binaural envelope (ENV) and temporal fine structure (TFS), to SRM in normal-hearing (NH) human listeners. Target speech was presented from the front and speech maskers were either colocated with or symmetrically separated from the target in azimuth. The target and maskers were presented either as natural speech or as "noise-vocoded" speech in which the intelligibility was conveyed only by the speech ENVs from several frequency bands; the speech TFS within each band was replaced with noise carriers. The experiments were designed to preserve the spatial cues in the speech ENVs while retaining/eliminating them from the TFS. This was achieved by using the same/different noise carriers in the two ears. A phenomenological auditory-nerve model was used to verify that the interaural correlations in TFS differed across conditions, whereas the ENVs retained a high degree of correlation, as intended. Overall, the results from this study revealed that binaural TFS cues, especially for frequency regions below 1500 Hz, are critical for achieving SRM in NH listeners. Potential implications for studying SRM in hearing-impaired listeners are discussed.

SIGNIFICANCE STATEMENT

Acoustic signals received by the auditory system pass first through an array of physiologically based band-pass filters. Conceptually, at the output of each filter, there are two principal forms of temporal information: slowly varying fluctuations in the envelope (ENV) and rapidly varying fluctuations in the temporal fine structure (TFS). The importance of these two types of information in everyday listening (e.g., conversing in a noisy social situation; the "cocktail-party" problem) has not been established. This study assessed the contributions of binaural ENV and TFS cues for understanding speech in multiple-talker situations. Results suggest that, whereas the ENV cues are important for speech intelligibility, binaural TFS cues are critical for perceptually segregating the different talkers and thus for solving the cocktail party problem.

Effects of linear and nonlinear speech rate changes on speech intelligibility in stationary and fluctuating maskers

Algorithmic modifications to the durational structure of speech designed to avoid intervals of intense masking lead to increases in intelligibility, but the basis for such gains is not clear. The current study addressed the possibility that the reduced information load produced by speech rate slowing might explain some or all of the benefits of durational modifications. The study also investigated the influence of masker stationarity on the effectiveness of durational changes. Listeners identified keywords in sentences that had undergone linear and nonlinear speech rate changes resulting in overall temporal lengthening in the presence of stationary and fluctuating maskers. Relative to unmodified speech, a slower speech rate produced no intelligibility gains for the stationary masker, suggesting that a reduction in information rate does not underlie intelligibility benefits of durationally modified speech. However, both linear and nonlinear modifications led to substantial intelligibility increases in fluctuating noise. One possibility is that overall increases in speech duration provide no new phonetic information in stationary masking conditions, but that temporal fluctuations in the background increase the likelihood of glimpsing additional salient speech cues. Alternatively, listeners may have benefitted from an increase in the difference in speech rates between the target and background.

Reverberation limits the release from informational masking obtained in the harmonic and binaural domains

A difference in fundamental frequency (ΔF0) and a difference in spatial location (ΔSL) are two cues known to provide masking releases when multiple speakers talk at once in a room. We examined situations in which reverberation should have no effect on the mechanisms underlying the releases from energetic masking produced by these two cues. Speech reception thresholds using both unpredictable target sentences and the coordinate response measure followed a similar pattern. Both ΔF0s and ΔSLs provided masking releases in the presence of nonspeech maskers (matched in excitation pattern and temporal envelope to the speech maskers) that, as intended, were robust to reverberation. Larger masking releases were obtained for speech maskers, but critically, they were affected by reverberation. These results suggest that reverberation either limits the amount of informational masking that is present to begin with or affects its release by ΔF0s or ΔSLs.

A Within-Subject Comparison of Bimodal Hearing, Bilateral Cochlear Implantation, and Bilateral Cochlear Implantation With Bilateral Hearing Preservation: High-Performing Patients

OBJECTIVE

To compare speech understanding with bimodal hearing and bilateral cochlear implants (CIs).

STUDY DESIGN

Within-subjects, repeated-measures.

METHODS

Speech understanding was assessed in the following conditions: unilateral hearing aid (HA) in the non-implanted ear, unilateral CI, bimodal (CI + HA), and bilateral CI. In addition, three participants had bilateral hearing preservation and were also tested with bilateral CIs and bilateral HAs (BiBi).

SETTING

Tertiary academic CI center.

PATIENTS

Eight adult sequential bilateral recipients who, despite achieving incredibly high performance with the first CI, self-selected for bilateral cochlear implantation.

INTERVENTION(S)

Bilateral cochlear implantation.

MAIN OUTCOME MEASURE(S)

Speech understanding for the adult minimum speech test battery as well as sentences in semidiffuse noise using the R-SPACE system.

RESULTS

Bilateral CIs afforded significant individual improvement in a complex listening environment even for individuals demonstrating near perfect sentence scores with both the first CI alone as well as the bimodal condition. The 3 BiBi participants demonstrated additional significant benefit over the bilateral CI condition-presumably because of the availability of interaural time difference cues.

CONCLUSIONS

These data suggest that, for noisy environments, adding a second implant can significantly improve speech understanding-even for high-performing unilateral CI with bimodal hearing. In diffuse noise conditions, bilateral acoustic hearing can yield even greater benefits beyond that offered by bilateral implantation.

Decreased ability in the segregation of dynamically changing vowel-analog streams: a factor in the age-related cocktail-party deficit?

Pairs of harmonic complexes with different fundamental frequencies f0 (105 and 189 Hz or 105 and 136 Hz) but identical bandwidth (0.25–3 kHz) were band-pass filtered using a filter having an identical center frequency of 1 kHz. The filter's center frequency was modulated using a triangular wave having a 5-Hz modulation frequency f_mod to obtain a pair of vowel-analog waveforms with dynamically varying single-formant transitions. The target signal S contained a single modulation cycle starting either at a phase of −π/2 (up-down) or π/2 (down-up), whereas the longer distracter N contained several cycles of the modulating triangular wave starting at a random phase. The level at which the target formant's modulating phase could be correctly identified was adaptively determined for several distracter levels and several extents of frequency swing (10–55%) in a group of experienced normal-hearing young and a group of experienced elderly individuals with hearing loss not exceeding one considered moderate. The most important result was that, for the two f0 differences, all distracter levels, and all frequency swing extents tested, elderly listeners needed about 20 dB larger S/N ratios than the young. Results also indicate that identification thresholds of both the elderly and the young listeners are between 4 and 12 dB higher than similarly determined detection thresholds and that, contrary to detection, identification is not a linear function of distracter level. Since formant transitions represent potent cues for speech intelligibility, the large S/N ratios required by the elderly for correct discrimination of single-formant transition dynamics may at least partially explain the well-documented intelligibility loss of speech in babble noise by the elderly.

Development of speech glimpsing in synchronously and asynchronously modulated noise

This study investigated development of the ability to integrate glimpses of speech in modulated noise. Noise was modulated synchronously across frequency or asynchronously such that when noise below 1300 Hz was "off," noise above 1300 Hz was "on," and vice versa. Asynchronous masking was used to examine the ability of listeners to integrate speech glimpses separated across time and frequency. The study used the Word Intelligibility by Picture Identification (WIPI) test and included adults, older children (age 8-10 yr) and younger children (5-7 yr). Results showed poorer masking release for the children than the adults for synchronous modulation but not for asynchronous modulation. It is possible that children can integrate cues relatively well when all intervals provide at least partial speech information (asynchronous modulation) but less well when some intervals provide little or no information (synchronous modulation). Control conditions indicated that children appeared to derive less benefit than adults from speech cues below 1300 Hz. This frequency effect was supported by supplementary conditions where the noise was unmodulated and the speech was low- or high-pass filtered. Possible sources of the developmental frequency effect include differences in frequency weighting, effective speech bandwidth, and the signal-to-noise ratio in the unmodulated noise condition.

Intelligibility of whispered speech in stationary and modulated noise maskers

This study investigated the role of natural periodic temporal fine structure in helping listeners take advantage of temporal valleys in amplitude-modulated masking noise when listening to speech. Young normal-hearing participants listened to natural, whispered, and/or vocoded nonsense sentences in a variety of masking conditions. Whispering alters normal waveform temporal fine structure dramatically but, unlike vocoding, does not degrade spectral details created by vocal tract resonances. The improvement in intelligibility, or masking release, due to introducing 16-Hz square-wave amplitude modulations in an otherwise steady speech-spectrum noise was reduced substantially with vocoded sentences relative to natural speech, but was not reduced for whispered sentences. In contrast to natural speech, masking release for whispered sentences was observed even at positive signal-to-noise ratios. Whispered speech has a different short-term amplitude distribution relative to natural speech, and this appeared to explain the robust masking release for whispered speech at high signal-to-noise ratios. Recognition of whispered speech was not disproportionately affected by unpredictable modulations created by a speech-envelope modulated noise masker. Overall, the presence or absence of periodic temporal fine structure did not have a major influence on the degree of benefit obtained from imposing temporal fluctuations on a noise masker.

Effects of age and hearing impairment on the ability to benefit from temporal and spectral modulation

OBJECTIVES

At poor signal-to-noise ratios, speech understanding may depend on the ability to combine speech fragments that are distributed across time and frequency. The goal of this study was to determine the effects of development and hearing impairment on this ability.

DESIGN

Listeners in the present study included adults and children with normal hearing and with hearing impairment. The children with normal hearing included a younger group (aged 4.6-6.9 yr, N = 10) and an older group (aged 7.3-11.1 yr, N = 11). The adults with normal hearing were aged 19 to 27 yr (N = 10). Adults (aged 19-54 yr, N = 9) and children (aged 7.2-10.7 yr, N = 8) with hearing impairment were also tested. The two groups with hearing impairment had comparable mild/moderate bilateral sensorineural hearing impairment. Masked speech reception thresholds for sentences were determined in a baseline condition of steady speech-shaped noise and in noise that was temporally modulated, spectrally modulated, or both temporally and spectrally modulated.

RESULTS

The results of normal-hearing listeners indicated higher masked speech reception thresholds for children than adults in steady noise. Adults and children showed the same magnitude of masking release for spectral modulation. Adults showed more masking release than the younger children for temporal modulation and showed more masking release than both the younger and older children for combined temporal/spectral modulation. Comparing normal-hearing and hearing-impaired listeners, the hearing-impaired listeners had higher masked speech reception thresholds in the steady noise condition and reduced masking release in the modulated noise conditions. Neither the two-way interaction between age and hearing impairment nor the three-way interaction between age, hearing impairment, and masking configuration was significant.

CONCLUSIONS

Although the reduced masking release for temporal modulation shown by the younger children with normal hearing could be a result of poor temporal resolution, it more likely reflects inefficient use of speech cues in temporal gaps or factors stemming from higher signal-to-noise ratios required by children in the baseline condition. The reduced masking release for combined temporal/spectral modulation demonstrated by both the younger and older children with normal hearing may indicate that children in the age range tested here have some difficulty in combining speech information that is distributed across temporal and spectral gaps. Hearing impairment was associated with higher thresholds and reduced masking release in all modulation conditions. Children with hearing impairment showed the poorest performance of any group, consistent with additive effects of hearing loss and development.

The influence of non-spatial factors on measures of spatial release from masking

This study tested the hypothesis that the reduction in spatial release from masking (SRM) resulting from sensorineural hearing loss in competing speech mixtures is influenced by the characteristics of the interfering speech. A frontal speech target was presented simultaneously with two intelligible or two time-reversed (unintelligible) speech maskers that were either colocated with the target or were symmetrically separated from the target in the horizontal plane. The difference in SRM between listeners with hearing impairment and listeners with normal hearing was substantially larger for the forward maskers (deficit of 5.8 dB) than for the reversed maskers (deficit of 1.6 dB). This was driven by the fact that all listeners, regardless of hearing abilities, performed similarly (and poorly) in the colocated condition with intelligible maskers. The same conditions were then tested in listeners with normal hearing using headphone stimuli that were degraded by noise vocoding. Reducing the number of available spectral channels systematically reduced the measured SRM, and again, more so for forward (reduction of 3.8 dB) than for reversed speech maskers (reduction of 1.8 dB). The results suggest that non-spatial factors can strongly influence both the magnitude of SRM and the apparent deficit in SRM for listeners with impaired hearing.

Effects of spectral smearing and temporal fine-structure distortion on the fluctuating-masker benefit for speech at a fixed signal-to-noise ratio

Normal-hearing listeners receive less benefit from momentary dips in the level of a fluctuating masker for speech processed to degrade spectral detail or temporal fine structure (TFS) than for unprocessed speech. This has been interpreted as evidence that the magnitude of the fluctuating-masker benefit (FMB) reflects the ability to resolve spectral detail and TFS. However, the FMB for degraded speech is typically measured at a higher signal-to-noise ratio (SNR) to yield performance similar to normal speech for the baseline (stationary-noise) condition. Because the FMB decreases with increasing SNR, this SNR difference might account for the reduction in FMB for degraded speech. In this study, the FMB for unprocessed and processed (TFS-removed or spectrally smeared) speech was measured in a paradigm that adjusts word-set size, rather than SNR, to equate stationary-noise performance across processing conditions. Compared at the same SNR and percent-correct level (but with different set sizes), processed and unprocessed stimuli yielded a similar FMB for four different fluctuating maskers (speech-modulated noise, one opposite-gender interfering talker, two same-gender interfering talkers, and 16-Hz interrupted noise). These results suggest that, for these maskers, spectral or TFS distortions do not directly impair the ability to benefit from momentary dips in masker level.

Simulations of cochlear-implant speech perception in modulated and unmodulated noise

Experiment 1 replicated the finding that normal-hearing listeners identify speech better in modulated than in unmodulated noise. This modulated-unmodulated difference ("MUD") has been previously shown to be reduced or absent for cochlear-implant listeners and for normal-hearing listeners presented with noise-vocoded speech. Experiments 2-3 presented normal-hearing listeners with noise-vocoded speech in unmodulated or 16-Hz-square-wave modulated noise, and investigated whether the introduction of simple binaural differences between target and masker could restore the masking release. Stimuli were presented over headphones. When the target and masker were presented to one ear, adding a copy of the masker to the other ear ("diotic configuration") aided performance but did so to a similar degree for modulated and unmodulated maskers, thereby failing to improve the modulation masking release. Presenting an uncorrelated noise to the opposite ear ("dichotic configuration") had no effect, either for modulated or unmodulated maskers, consistent with the improved performance in the diotic configuration being due to interaural decorrelation processing. For noise-vocoded speech, the provision of simple spatial differences did not allow listeners to take greater advantage of the dips present in a modulated masker.

Role of binaural hearing in speech intelligibility and spatial release from masking using vocoded speech

A cochlear implant vocoder was used to evaluate relative contributions of spectral and binaural temporal fine-structure cues to speech intelligibility. In Study I, stimuli were vocoded, and then convolved through head related transfer functions (HRTFs) to remove speech temporal fine structure but preserve the binaural temporal fine-structure cues. In Study II, the order of processing was reversed to remove both speech and binaural temporal fine-structure cues. Speech reception thresholds (SRTs) were measured adaptively in quiet, and with interfering speech, for unprocessed and vocoded speech (16, 8, and 4 frequency bands), under binaural or monaural (right-ear) conditions. Under binaural conditions, as the number of bands decreased, SRTs increased. With decreasing number of frequency bands, greater benefit from spatial separation of target and interferer was observed, especially in the 8-band condition. The present results demonstrate a strong role of the binaural cues in spectrally degraded speech, when the target and interfering speech are more likely to be confused. The nearly normal binaural benefits under present simulation conditions and the lack of order of processing effect further suggest that preservation of binaural cues is likely to improve performance in bilaterally implanted recipients.

Auditory and auditory-visual intelligibility of speech in fluctuating maskers for normal-hearing and hearing-impaired listeners

Speech intelligibility for audio-alone and audiovisual (AV) sentences was estimated as a function of signal-to-noise ratio (SNR) for a female target talker presented in a stationary noise, an interfering male talker, or a speech-modulated noise background, for eight hearing-impaired (HI) and five normal-hearing (NH) listeners. At the 50% keywords-correct performance level, HI listeners showed 7-12 dB less fluctuating-masker benefit (FMB) than NH listeners, consistent with previous results. Both groups showed significantly more FMB under AV than audio-alone conditions. When compared at the same stationary-noise SNR, FMB differences between listener groups and modalities were substantially smaller, suggesting that most of the FMB differences at the 50% performance level may reflect a SNR dependence of the FMB. Still, 1-5 dB of the FMB difference between listener groups remained, indicating a possible role for reduced audibility, limited spectral or temporal resolution, or an inability to use auditory source-segregation cues, in directly limiting the ability to listen in the dips of a fluctuating masker. A modified version of the extended speech-intelligibility index that predicts a larger FMB at less favorable SNRs accounted for most of the FMB differences between listener groups and modalities. Overall, these data suggest that HI listeners retain more of an ability to listen in the dips of a fluctuating masker than previously thought. Instead, the fluctuating-masker difficulties exhibited by HI listeners may derive from the reduced FMB associated with the more favorable SNRs they require to identify a reasonable proportion of the target speech.

Lexical and indexical cues in masking by competing speech

Three experiments were conducted using the TVM sentences, a new set of stimuli for competing speech research. These open-set sentences incorporate a cue name that allows the experimenter to direct the listener's attention to a target sentence. The first experiment compared the relative efficacy of directing the listener's attention to the cue name versus instructing the subject to listen for a particular talker's voice. Results demonstrated that listeners could use either cue about equally well to find the target sentence. Experiment 2 was designed to determine whether differences in intelligibility among talkers' voices that were noted when three utterances were presented together persisted when each talker's sentences were presented in steady-state noise. Results of experiment 2 showed only minor intelligibility differences between talkers' utterances presented in noise. The final experiment considered how providing accurate and inaccurate information about the target talker's voice influenced speech recognition performance. This voice cue was found to have minimal effect on listeners' ability to understand the target utterance or ignore a masking voice.

Masking release for low- and high-pass-filtered speech in the presence of noise and single-talker interference

Speech intelligibility was measured for sentences presented in spectrally matched steady noise, single-talker interference, or speech-modulated noise. The stimuli were unfiltered or were low-pass (LP) (1200 Hz cutoff) or high-pass (HP) (1500 Hz cutoff) filtered. The cutoff frequencies were selected to produce equal performance in both LP and HP conditions in steady noise and to limit access to the temporal fine structure of resolved harmonics in the HP conditions. Masking release, or the improvement in performance between the steady noise and single-talker interference, was substantial with no filtering. Under LP and HP filtering, masking release was roughly equal but was much less than in unfiltered conditions. When the average F0 of the interferer was shifted lower than that of the target, similar increases in masking release were observed under LP and HP filtering. Similar LP and HP results were also obtained for the speech-modulated-noise masker. The findings are not consistent with the idea that pitch conveyed by the temporal fine structure of low-order harmonics plays a crucial role in masking release. Instead, any reduction in speech redundancy, or manipulation that increases the target-to-masker ratio necessary for intelligibility to beyond around 0 dB, may result in reduced masking release.