Cochlear implant speech intelligibility outcomes with structured and unstructured binary mask errors

Objective intelligibility measurement of reverberant vocoded speech for normal-hearing listeners: Towards facilitating the development of speech enhancement algorithms for cochlear implants

Cochlear implant (CI) recipients often struggle to understand speech in reverberant environments. Speech enhancement algorithms could restore speech perception for CI listeners by removing reverberant artifacts from the CI stimulation pattern. Listening studies, either with cochlear-implant recipients or normal-hearing (NH) listeners using a CI acoustic model, provide a benchmark for speech intelligibility improvements conferred by the enhancement algorithm but are costly and time consuming. To reduce the associated costs during algorithm development, speech intelligibility could be estimated offline using objective intelligibility measures. Previous evaluations of objective measures that considered CIs primarily assessed the combined impact of noise and reverberation and employed highly accurate enhancement algorithms. To facilitate the development of enhancement algorithms, we evaluate twelve objective measures in reverberant-only conditions characterized by a gradual reduction of reverberant artifacts, simulating the performance of an enhancement algorithm during development. Measures are validated against the performance of NH listeners using a CI acoustic model. To enhance compatibility with reverberant CI-processed signals, measure performance was assessed after modifying the reference signal and spectral filterbank. Measures leveraging the speech-to-reverberant ratio, cepstral distance and, after modifying the reference or filterbank, envelope correlation are strong predictors of intelligibility for reverberant CI-processed speech.

Prediction of Individual Cochlear Implant Recipient Speech Perception With the Output Signal to Noise Ratio Metric

OBJECTIVES

A cochlear implant (CI) implements a variety of sound processing algorithms that seek to improve speech intelligibility. Typically, only a small number of parameter combinations are evaluated with recipients but the optimal configuration may differ for individuals. The present study evaluates a novel methodology which uses the output signal to noise ratio (OSNR) to predict complete psychometric functions that relate speech recognition to signal to noise ratio for individual CI recipients.

DESIGN

Speech scores from sentence-in-noise tests in a "reference" condition were mapped to OSNR and a psychometric function was fitted. The reference variability was defined as the root mean square error between the reference scores and the fitted curve. To predict individual scores in a different condition, OSNRs in that condition were calculated and the corresponding scores were read from the reference psychometric function. In a retrospective experiment, scores were predicted for each condition and subject in three existing data sets of sentence scores. The prediction error was defined as the root mean square error between observed and predicted scores. In data set 1, sentences were mixed with 20 talker babble or speech weighted noise and presented at 65 dB sound pressure level (SPL). An adaptive test procedure was used. Sound processing was advanced combinatorial encoding (ACE, Cochlear Limited) and ACE with ideal binary mask processing, with five different threshold settings. In data set 2, sentences were mixed with speech weighted noise, street-side city noise or cocktail party noise and presented at 65 dB SPL. An adaptive test procedure was used. Sound processing was ACE and ACE with two different noise reduction schemes. In data set 3, sentences were mixed with four-talker babble at two input SNRs and presented at levels of 55-89 dB SPL. Sound processing utilised three different automatic gain control configurations.

RESULTS

For data set 1, the median of individual prediction errors across all subjects, noise types and conditions, was 12% points, slightly better than the reference variability. The OSNR prediction method was inaccurate for the specific condition with a gain threshold of +10 dB. For data set 2, the median of individual prediction errors was 17% points and the reference variability was 11% points. For data set 3, the median prediction error was 9% points and the reference variability was 7% points. A Monte Carlo simulation found that the OSNR prediction method, which used reference scores and OSNR to predict individual scores in other conditions, was significantly more accurate (p < 0.01) than simply using reference scores as predictors.

CONCLUSIONS

The results supported the hypothesis that the OSNR prediction method could accurately predict individual recipient scores for a range of algorithms and noise types, for all but one condition. The medians of the individual prediction errors for each data set were accurate within 6% points of the reference variability and compared favourably with prediction methodologies in other recent studies. Overall, the novel OSNR-based prediction method shows promise as a tool to assist researchers and clinicians in the development or fitting of CI sound processors.

Constraints on ideal binary masking for the perception of spectrally-reduced speech

This study investigated recognition of sentences processed using ideal binary masking (IBM) with limited spectral resolution. Local thresholds (LCs) of -12, 0, and 5 dB were applied which altered the target and masker power following IBM. Recognition was reduced due to persistence of the masker and limited target recovery, thus preventing IBM from ideal target-masker segregation. Linear regression and principal component analyses showed that, regardless of masker type and number of spectral channels, higher LCs were associated with poorer recognition. In addition, limitations on target recovery led to more detrimental effects on speech recognition compared to persistence of the masker.

The impact of exploiting spectro-temporal context in computational speech segregation

Computational speech segregation aims to automatically segregate speech from interfering noise, often by employing ideal binary mask estimation. Several studies have tried to exploit contextual information in speech to improve mask estimation accuracy by using two frequently-used strategies that (1) incorporate delta features and (2) employ support vector machine (SVM) based integration. In this study, two experiments were conducted. In Experiment I, the impact of exploiting spectro-temporal context using these strategies was investigated in stationary and six-talker noise. In Experiment II, the delta features were explored in detail and tested in a setup that considered novel noise segments of the six-talker noise. Computing delta features led to higher intelligibility than employing SVM based integration and intelligibility increased with the amount of spectral information exploited via the delta features. The system did not, however, generalize well to novel segments of this noise type. Measured intelligibility was subsequently compared to extended short-term objective intelligibility, hit-false alarm rate, and the amount of mask clustering. None of these objective measures alone could account for measured intelligibility. The findings may have implications for the design of speech segregation systems, and for the selection of a cost function that correlates with intelligibility.

Representing the intelligibility advantage of ideal binary masking with the most energetic channels

This study investigates how the intelligibility advantage of ideal binary mask (IBM) processing in synthesizing speech is affected by the use of a small number of the most energetic channels. In experiment 1, IBM-processed Mandarin speech that had been corrupted by speech spectrum-shaped noise or two-talker babble was synthesized by using as few as four of the most energetic target-dominated channels at each frame. This approach provided intelligibility comparable to that of speech synthesized with all of the target-dominated channels. Experiments 2, 3, and 4 examined how the intelligibility advantage of IBM processing from experiment 1 was affected by the local SNR threshold, low-frequency region (LFR) cut-off frequency, and vowel-based segmentation, respectively. Experiments 2 and 3 showed that a threshold of 0 dB for local SNR and a cutoff of 3000 Hz for LFR were optimal choices for improving the intelligibility of IBM processing based on the most energetic channels. Experiment 4 found that the intelligibility advantage of IBM processing with the most energetic channels was preserved at the segmental level of vowel-only IBM-processed speech. Taken together, the results suggest that compared to IBM-processed speech synthesized with all of the target-dominated channels, Mandarin speech synthesized by selecting a small number of the most energetic target-dominated channels can achieve similar levels of intelligibility.

Outcome measures based on classification performance fail to predict the intelligibility of binary-masked speech

To date, the most commonly used outcome measure for assessing ideal binary mask estimation algorithms is based on the difference between the hit rate and the false alarm rate (H-FA). Recently, the error distribution has been shown to substantially affect intelligibility. However, H-FA treats each mask unit independently and does not take into account how errors are distributed. Alternatively, algorithms can be evaluated with the short-time objective intelligibility (STOI) metric using the reconstructed speech. This study investigates the ability of H-FA and STOI to predict intelligibility for binary-masked speech using masks with different error distributions. The results demonstrate the inability of H-FA to predict the behavioral intelligibility and also illustrate the limitations of STOI. Since every estimation algorithm will make errors that are distributed in different ways, performance evaluations should not be made solely on the basis of these metrics.