A neural-based vocoder implementation for evaluating cochlear implant coding strategies

CompHEAR: A Customizable and Scalable Web-Enabled Auditory Performance Evaluation Platform for Cochlear Implant Sound Processing Research

Objective

Cochlear implants (CIs) are auditory prostheses for individuals with severe to profound hearing loss, offering substantial but incomplete restoration of hearing function by stimulating the auditory nerve using electrodes. However, progress in CI performance and innovation has been constrained by the inability to rapidly test multiple sound processing strategies. Current research interfaces provided by major CI manufacturers have limitations in supporting a wide range of auditory experiments due to portability, programming difficulties, and the lack of direct comparison between sound processing algorithms. To address these limitations, we present the CompHEAR research platform, designed specifically for the Cochlear Implant Hackathon, enabling researchers to conduct diverse auditory experiments on a large scale.

Study Design

Quasi-experimental.

Setting

Virtual.

Methods

CompHEAR is an open-source, user-friendly platform which offers flexibility and ease of customization, allowing researchers to set up a broad set of auditory experiments. CompHEAR employs a vocoder to simulate novel sound coding strategies for CIs. It facilitates even distribution of listening tasks among participants and delivers real-time metrics for evaluation. The software architecture underlies the platform's flexibility in experimental design and its wide range of applications in sound processing research.

Results

Performance testing of the CompHEAR platform ensured that it could support at least 10,000 concurrent users. The CompHEAR platform was successfully implemented during the COVID-19 pandemic and enabled global collaboration for the CI Hackathon (www.cihackathon.com).

Conclusion

The CompHEAR platform is a useful research tool that permits comparing diverse signal processing strategies across a variety of auditory tasks with crowdsourced judging. Its versatility, scalability, and ease of use can enable further research with the goal of promoting advancements in cochlear implant performance and improved patient outcomes.

The effects of estimation accuracy, estimation approach, and number of selected channels using formant-priority channel selection for an "n-of-m" sound processing strategy for cochlear implants

Previously, selection of l channels was prioritized according to formant frequency locations in an l-of-n-of-m-based signal processing strategy to provide important voicing information independent of listening environments for cochlear implant (CI) users. In this study, ideal, or ground truth, formants were incorporated into the selection stage to determine the effect of accuracy on (1) subjective speech intelligibility, (2) objective channel selection patterns, and (3) objective stimulation patterns (current). An average +11% improvement (p < 0.05) was observed across six CI users in quiet, but not for noise or reverberation conditions. Analogous increases in channel selection and current for the upper range of F1 and a decrease across mid-frequencies with higher corresponding current, were both observed at the expense of noise-dominant channels. Objective channel selection patterns were analyzed a second time to determine the effects of estimation approach and number of selected channels (n). A significant effect of estimation approach was only observed in the noise and reverberation condition with minor differences in channel selection and significantly decreased stimulated current. Results suggest that estimation method, accuracy, and number of channels in the proposed strategy using ideal formants may improve intelligibility when corresponding stimulated current of formant channels are not masked by noise-dominant channels.

A computational model to simulate spectral modulation and speech perception experiments of cochlear implant users

Speech understanding in cochlear implant (CI) users presents large intersubject variability that may be related to different aspects of the peripheral auditory system, such as the electrode–nerve interface and neural health conditions. This variability makes it more challenging to proof differences in performance between different CI sound coding strategies in regular clinical studies, nevertheless, computational models can be helpful to assess the speech performance of CI users in an environment where all these physiological aspects can be controlled. In this study, differences in performance between three variants of the HiRes Fidelity 120 (F120) sound coding strategy are studied with a computational model. The computational model consists of (i) a processing stage with the sound coding strategy, (ii) a three-dimensional electrode-nerve interface that accounts for auditory nerve fiber (ANF) degeneration, (iii) a population of phenomenological ANF models, and (iv) a feature extractor algorithm to obtain the internal representation (IR) of the neural activity. As the back-end, the simulation framework for auditory discrimination experiments (FADE) was chosen. Two experiments relevant to speech understanding were performed: one related to spectral modulation threshold (SMT), and the other one related to speech reception threshold (SRT). These experiments included three different neural health conditions (healthy ANFs, and moderate and severe ANF degeneration). The F120 was configured to use sequential stimulation (F120-S), and simultaneous stimulation with two (F120-P) and three (F120-T) simultaneously active channels. Simultaneous stimulation causes electric interaction that smears the spectrotemporal information transmitted to the ANFs, and it has been hypothesized to lead to even worse information transmission in poor neural health conditions. In general, worse neural health conditions led to worse predicted performance; nevertheless, the detriment was small compared to clinical data. Results in SRT experiments indicated that performance with simultaneous stimulation, especially F120-T, were more affected by neural degeneration than with sequential stimulation. Results in SMT experiments showed no significant difference in performance. Although the proposed model in its current state is able to perform SMT and SRT experiments, it is not reliable to predict real CI users' performance yet. Nevertheless, improvements related to the ANF model, feature extraction, and predictor algorithm are discussed.

Temporal envelope cues and simulations of cochlear implant signal processing

Conventional signal processing implemented on clinical cochlear implant (CI) sound processors is based on envelope signals extracted from overlapping frequency regions. Conventional strategies do not encode temporal envelope or temporal fine-structure cues with high fidelity. In contrast, several research strategies have been developed recently to enhance the encoding of temporal envelope and fine-structure cues. The present study examines the salience of temporal envelope cues when encoded into vocoder representations of CI signal processing. Normal-hearing listeners were evaluated on measures of speech reception, speech quality ratings, and spatial hearing when listening to vocoder representations of CI signal processing. Conventional vocoder techniques using envelope signals with noise- or tone-excited reconstruction were evaluated in comparison to a novel approach based on impulse-response reconstruction. A variation of this impulse-response approach was based on a research strategy, the Fundamentally Asynchronous Stimulus Timing (FAST) algorithm, designed to improve temporal precision of envelope cues. The results indicate that the introduced impulse-response approach, combined with the FAST algorithm, produces similar results on speech reception measures as the conventional vocoder approaches, while providing significantly better sound quality and spatial hearing outcomes. This novel approach for stimulating how temporal envelope cues are encoded into CI stimulation has potential for examining diverse aspects of hearing, particularly in aspects of musical pitch perception and spatial hearing.

Investigating the use of a Gammatone filterbank for a cochlear implant coding strategy

•

A Gammatone filterbank has the potential to better resolve the harmonics of complex tones.

•

A total delay of a Gammatone filterbank can be made smaller compared to an FFT filterbank with the same frequency resolution at low frequencies.

•

Melody contour identification improved with longer frame size or higher frequency resolution.

Background

Contemporary speech processing strategies in cochlear implants (CIs) such as the Advanced Combination Encoder (ACE) use a standard Fast Fourier Transform (FFT) filterbank to extract envelopes. The assignment of the FFT bins to approximate the frequency resolution of the basilar membrane is only partly based on physiology, especially since the bins are distributed linearly below 1000 Hz and logarithmically above 1000 Hz.

New method

A Gammatone filterbank which provides a closer approximation to the bandwidths of filters in the human auditory system could replace the standard FFT filterbank in the ACE strategy. An infinite impulse response (IIR) all-pole design of the Gammatone filterbank was compared to the FFT filterbank with 128, 256 and 512 points resolutions and the effect of the frequency boundaries of the filters was also investigated.

Results

Melodic contour identification (MCI) and just noticeable difference (JND) experiments, both involving synthetic clarinet notes in octaves 3 and 4, were conducted with 6 normal hearing (NH) participants using noise vocoded stimuli; and 10 CI recipients just performed the MCI experiment. The MCI results for both NH and CI subjects, showed a significant effect of the filterbank on the percentage correct responses of the participants.

Comparison with existing methods

The Gammatone filterbank can better resolve the harmonics of tested synthetic clarinet notes which led to better performances in the MCI experiment.

Conclusions

The total delay of the Gammatone filterbank can be made smaller than the delay of the FFT filterbank with the same frequency resolution at low frequencies.

The history and future of neural modeling for cochlear implants

This special issue of Network: Computation in Neural Systems on the topic of "Computational models of the electrically stimulated auditory system" incorporates review articles spanning a wide range of approaches to modeling cochlear implant stimulation of the auditory system. The purpose of this overview paper is to provide a historical context for the different modeling endeavors and to point toward how computational modeling could play a key role in the understanding, evaluation, and improvement of cochlear implants in the future.