Prediction of binaural speech intelligibility with frequency-dependent interaural phase differences

Spatial speech-in-noise performance in simulated single-sided deaf and bimodal cochlear implant users in comparison with real patients

OBJECTIVE

Speech reception thresholds (SRTs) in spatial scenarios were measured in simulated cochlear implant (CI) listeners with either contralateral normal hearing, or aided hearing impairment (bimodal), and compared to SRTs of real patients, who were measured using the exact same paradigm, to assess goodness of simulation.

DESIGN

CI listening was simulated using a vocoder incorporating actual CI signal processing and physiologic details of electric stimulation on one side. Unprocessed signals or simulation of aided moderate or profound hearing impairment was used contralaterally. Three spatial speech-in-noise scenarios were tested using virtual acoustics to assess spatial release from masking (SRM) and combined benefit.

STUDY SAMPLE

Eleven normal-hearing listeners participated in the experiment.

RESULTS

For contralateral normal and aided moderately impaired hearing, bilaterally assessed SRTs were not statistically different from unilateral SRTs of the better ear, indicating "better-ear-listening". Combined benefit was only found for contralateral profound impaired hearing. As in patients, SRM was highest for contralateral normal hearing and decreased systematically with more severe simulated impairment. Comparison to actual patients showed good reproduction of SRTs, SRM, and better-ear-listening.

CONCLUSIONS

The simulations reproduced better-ear-listening as in patients and suggest that combined benefit in spatial scenes predominantly occurs when both ears show poor speech-in-noise performance.

Modelling speech reception thresholds and their improvements due to spatial noise reduction algorithms in bimodal cochlear implant users

•

This paper compares two modelling approaches to predict the speech recognition ability of bimodal CI users and the benefit of using beamformers.

•

The modelling approaches vary in computational complexity and fitting requirements.

•

A complex cafeteria spatial scenario with three localized single noise source scenario and a diffuse multi-talker babble noise is used.

•

The automatic speech recognizer is more accurate across the different spatial scenarios and noise types and requires less fitting compared to the statistical modelling approach.

Spatial noise reduction algorithms (“beamformers”) can considerably improve speech reception thresholds (SRTs) for bimodal cochlear implant (CI) users. The goal of this study was to model SRTs and SRT-benefit due to beamformers for bimodal CI users. Two existing model approaches varying in computational complexity and binaural processing assumption were compared: (i) the framework of auditory discrimination experiments (FADE) and (ii) the binaural speech intelligibility model (BSIM), both with CI and aided hearing-impaired front-ends. The exact same acoustic scenarios, and open-access beamformers as in the comparison clinical study Zedan et al. (2021) were used to quantify goodness of prediction. FADE was capable of modeling SRTs ab-initio, i.e., no calibration of the model was necessary to achieve high correlations and low root-mean square errors (RMSE) to both, measured SRTs (r = 0.85, RMSE = 2.8 dB) and to measured SRT-benefits (r = 0.96). BSIM achieved somewhat poorer predictions to both, measured SRTs (r = 0.78, RMSE = 6.7 dB) and to measured SRT-benefits (r = 0.91) and needs to be calibrated for matching average SRTs in one condition. Greatest deviations in predictions of BSIM were observed in diffuse multi-talker babble noise, which were not found with FADE. SRT-benefit predictions of both models were similar to instrumental signal-to-noise ratio (iSNR) improvements due to the beamformers. This indicates that FADE is preferrable for modeling absolute SRTs. However, for prediction of SRT-benefit due to spatial noise reduction algorithms in bimodal CI users, the average iSNR is a much simpler approach with similar performance.

Modeling Binaural Unmasking of Speech Using a Blind Binaural Processing Stage

The equalization cancellation model is often used to predict the binaural masking level difference. Previously its application to speech in noise has required separate knowledge about the speech and noise signals to maximize the signal-to-noise ratio (SNR). Here, a novel, blind equalization cancellation model is introduced that can use the mixed signals. This approach does not require any assumptions about particular sound source directions. It uses different strategies for positive and negative SNRs, with the switching between the two steered by a blind decision stage utilizing modulation cues. The output of the model is a single-channel signal with enhanced SNR, which we analyzed using the speech intelligibility index to compare speech intelligibility predictions. In a first experiment, the model was tested on experimental data obtained in a scenario with spatially separated target and masker signals. Predicted speech recognition thresholds were in good agreement with measured speech recognition thresholds with a root mean square error less than 1 dB. A second experiment investigated signals at positive SNRs, which was achieved using time compressed and low-pass filtered speech. The results demonstrated that binaural unmasking of speech occurs at positive SNRs and that the modulation-based switching strategy can predict the experimental results.

Further validation of a binaural model predicting speech intelligibility against envelope-modulated noises

Collin and Lavandier [J. Acoust. Soc. Am. 134, 1146-1159 (2013)] proposed a binaural model predicting speech intelligibility against envelope-modulated noises, evaluated in 24 acoustic conditions, involving similar masker types. The aim of the present study was to test the model robustness modeling 80 additional conditions, and evaluate the influence of its parameters using an approach inspired by a variance-based sensitivity analysis. First, the data from four experiments from the literature and one specifically designed for the present study were used to evaluate the prediction performance of the model, investigate potential interactions between its parameters, and define their values leading to the best predictions. A revision of the model allowed to account for binaural sluggishness. Finally, the optimized model was tested on an additional dataset not used to define its parameters. Overall, one hundred conditions split into six experiments were modeled. Correlation between data and predictions ranged from 0.85 to 0.96 across experiments, and mean absolute prediction errors were between 0.5 and 1.4 dB.

Measurement and Prediction of Binaural-Temporal Integration of Speech Reflections

For speech intelligibility in rooms, the temporal integration of speech reflections is typically modeled by separating the room impulse response (RIR) into an early (assumed beneficial for speech intelligibility) and a late part (assumed detrimental). This concept was challenged in this study by employing binaural RIRs with systematically varied interaural phase differences (IPDs) and amplitude of the direct sound and a variable number of reflections delayed by up to 200 ms. Speech recognition thresholds in stationary noise were measured in normal-hearing listeners for 86 conditions. The data showed that direct sound and one or several early speech reflections could be perfectly integrated when they had the same IPD. Early reflections with the same IPD as the noise (but not as the direct sound) could not be perfectly integrated with the direct sound. All conditions in which the dominant speech information was within the early RIR components could be well predicted by a binaural speech intelligibility model using classic early/late separation. In contrast, when amplitude or IPD favored late RIR components, listeners appeared to be capable of focusing on these components rather than on the precedent direct sound. This could not be modeled by an early/late separation window but required a temporal integration window that can be flexibly shifted along the RIR.

Modeling Sluggishness in Binaural Unmasking of Speech for Maskers With Time-Varying Interaural Phase Differences

In studies investigating binaural processing in human listeners, relatively long and task-dependent time constants of a binaural window ranging from 10 ms to 250 ms have been observed. Such time constants are often thought to reflect “binaural sluggishness.” In this study, the effect of binaural sluggishness on binaural unmasking of speech in stationary speech-shaped noise is investigated in 10 listeners with normal hearing. In order to design a masking signal with temporally varying binaural cues, the interaural phase difference of the noise was modulated sinusoidally with frequencies ranging from 0.25 Hz to 64 Hz. The lowest, that is the best, speech reception thresholds (SRTs) were observed for the lowest modulation frequency. SRTs increased with increasing modulation frequency up to 4 Hz. For higher modulation frequencies, SRTs remained constant in the range of 1 dB to 1.5 dB below the SRT determined in the diotic situation. The outcome of the experiment was simulated using a short-term binaural speech intelligibility model, which combines an equalization–cancellation (EC) model with the speech intelligibility index. This model segments the incoming signal into 23.2-ms time frames in order to predict release from masking in modulated noises. In order to predict the results from this study, the model required a further time constant applied to the EC mechanism representing binaural sluggishness. The best agreement with perceptual data was achieved using a temporal window of 200 ms in the EC mechanism.

The Extended Speech Transmission Index: Predicting speech intelligibility in fluctuating noise and reverberant rooms

The Speech Transmission Index (STI) is used to predict speech intelligibility in noise and reverberant environments. However, measurements and predictions in fluctuating noises lead to inaccuracies. In the current paper, the Extended Speech Transmission Index (ESTI) is presented in order to deal with these shortcomings. Speech intelligibility in normally hearing subjects was measured using stationary and fluctuating maskers. These results served to optimize model parameters. Data from the literature were then used to verify the ESTI-model. Model outcomes were accurate for stationary maskers, maskers with artificial fluctuations, and maskers with real life non-speech modulations. Maskers with speech-like characteristics introduced systematic errors in the model outcomes, probably due to a combination of modulation masking, context effects, and informational masking.

A method for degrading sound localization while preserving binaural advantages for speech reception in noise

This study developed and tested a real-time processing algorithm designed to degrade sound localization (LocDeg algorithm) without affecting binaural benefits for speech reception in noise. Input signals were divided into eight frequency channels. The odd-numbered channels were mixed between the ears to confuse the direction of interaural cues while preserving interaural cues in the even-numbered channels. The LocDeg algorithm was evaluated for normal-hearing listeners performing sound localization and speech-reception tasks. Results showed that the LocDeg algorithm successfully degraded sound-localization performance without affecting speech-reception performance or spatial release from masking for speech in noise. The LocDeg algorithm did, however, degrade speech-reception performance in a task involving spatially separated talkers in a multi-talker environment, which is thought to depend on differences in perceived spatial location of concurrent talkers. This LocDeg algorithm could be a valuable tool for isolating the importance of sound-localization ability from other binaural benefits in real-world environments.

Binaural frequency selectivity in humans

Several behavioural studies in humans have shown that listening to sounds with two ears that is binaural hearing, provides the human auditory system with extra information on the sound source that is not available when sounds are only perceived through one ear that is monaurally. Binaural processing involves the analysis of phase and level differences between the two ear signals. As monaural cochlea processing (in each ear) precedes the neural stages responsible for binaural processing properties it is reasonable to assume that properties of the cochlea may also be observed in binaural processing. A main characteristic of cochlea processing is its frequency selectivity. In psychoacoustics, there is an ongoing discussion on the frequency selectivity of the binaural auditory system. While some psychoacoustic experiments seem to indicate poorer frequency selectivity of the binaural system than that of the monaural processing others seem to indicate the same frequency selectivity for monaural and binaural processing. This study provides an overview of these seemingly controversial results and the different explanations that were provided to account for the different results.

Application of a short-time version of the Equalization-Cancellation model to speech intelligibility experiments with speech maskers

A short-time-processing version of the Equalization-Cancellation (EC) model of binaural processing is described and applied to speech intelligibility tasks in the presence of multiple maskers, including multiple speech maskers. This short-time EC model, called the STEC model, extends the model described by Wan et al. [J. Acoust. Soc. Am. 128, 3678-3690 (2010)] to allow the EC model's equalization parameters τ and α to be adjusted as a function of time, resulting in improved masker cancellation when the dominant masker location varies in time. Using the Speech Intelligibility Index, the STEC model is applied to speech intelligibility with maskers that vary in number, type, and spatial arrangements. Most notably, when maskers are located on opposite sides of the target, this STEC model predicts improved thresholds when the maskers are modulated independently with speech-envelope modulators; this includes the most relevant case of independent speech maskers. The STEC model describes the spatial dependence of the speech reception threshold with speech maskers better than the steady-state model. Predictions are also improved for independently speech-modulated noise maskers but are poorer for reversed-speech maskers. In general, short-term processing is useful, but much remains to be done in the complex task of understanding speech in speech maskers.

A model that predicts the binaural advantage to speech intelligibility from the mixed target and interferer signals

A model is presented that predicts the binaural advantage to speech intelligibility by analyzing the right and left recordings at the two ears containing mixed target and interferer signals. This auditory-inspired model implements an equalization-cancellation stage to predict the binaural unmasking (BU) component, in conjunction with a modulation-frequency estimation block to estimate the "better ear" effect (BE) component of the binaural advantage. The model's performance was compared to experimental data obtained under anechoic and reverberant conditions using a single speech-shaped noise interferer paradigm. The internal BU and BE components were compared to those of the speech intelligibility model recently proposed by Lavandier et al. [J. Acoust. Soc. Am. 131, 218-231 (2012)], which requires separate inputs for target and interferer. The data indicate that the proposed model provides comparably good predictions from a mixed-signals input under both anechoic and reverberant conditions.

Informational masking and spatial hearing in listeners with and without unilateral hearing loss

PURPOSE

This study assessed selective listening for speech in individuals with and without unilateral hearing loss (UHL) and the potential relationship between spatial release from informational masking and localization ability in listeners with UHL.

METHOD

Twelve adults with UHL and 12 normal-hearing controls completed a series of monaural and binaural speech tasks that were designed to measure informational masking. They also completed a horizontal localization task.

RESULTS

Monaural performance by participants with UHL was comparable to that of normal-hearing participants. Unlike the normal-hearing participants, the participants with UHL did not exhibit a true spatial release from informational masking. Rather, their performance could be predicted by head shadow effects. Performance among participants with UHL in the localization task was quite variable, with some showing near-normal abilities and others demonstrating no localization ability.

CONCLUSION

Individuals with UHL did not show deficits in all listening situations but were at a significant disadvantage when listening to speech in environments where normal-hearing listeners benefit from spatial separation between target and masker. This inability to capitalize on spatial cues for selective listening does not appear to be related to localization ability.

Revision, extension, and evaluation of a binaural speech intelligibility model

This study presents revision, extension, and evaluation of a binaural speech intelligibility model (Beutelmann, R., and Brand, T. (2006). J. Acoust. Soc. Am. 120, 331-342) that yields accurate predictions of speech reception thresholds (SRTs) in the presence of a stationary noise source at arbitrary azimuths and in different rooms. The modified model is based on an analytical expression of binaural unmasking for arbitrary input signals and is computationally more efficient, while maintaining the prediction quality of the original model. An extension for nonstationary interferers was realized by applying the model to short time frames of the input signals and averaging over the predicted SRT results. Binaural SRTs from 8 normal-hearing and 12 hearing-impaired subjects, incorporating all combinations of four rooms, three source setups, and three noise types were measured and compared to the model's predictions. Depending on the noise type, the parametric correlation coefficients between observed and predicted SRTs were 0.80-0.93 for normal-hearing subjects and 0.59-0.80 for hearing-impaired subjects. The mean absolute prediction error was 3 dB for the mean normal-hearing data and 4 dB for the individual hearing-impaired data. 70% of the variance of the SRTs of hearing-impaired subjects could be explained by the model, which is based only on the audiogram.