Further examination of complex pitch perception in the absence of a place-rate match

Characterizing the relationship between modulation sensitivity and pitch resolution in cochlear implant users

Cochlear implants are medical devices that have restored hearing to approximately one million people around the world. Outcomes are impressive and most recipients attain excellent speech comprehension in quiet without relying on lip-reading cues, but pitch resolution is poor compared to normal hearing. Amplitude modulation of electrical stimulation is a primary cue for pitch perception in cochlear implant users. The experiments described in this article focus on the relationship between sensitivity to amplitude modulations and pitch resolution based on changes in the frequency of amplitude modulations. In the first experiment, modulation sensitivity and pitch resolution were measured in adults with no known hearing loss and in cochlear implant users with sounds presented to and processed by their clinical devices. Stimuli were amplitude-modulated sinusoids and amplitude-modulated narrow-band noises. Modulation detection and modulation frequency discrimination were measured for modulation frequencies centered on 110, 220, and 440 Hz. Pitch resolution based on changes in modulation frequency was measured for modulation depths of 25 %, 50 %, 100 %, and for a half-waved rectified modulator. Results revealed a strong linear relationship between modulation sensitivity and pitch resolution for cochlear implant users and peers with no known hearing loss. In the second experiment, cochlear implant users took part in analogous procedures of modulation sensitivity and pitch resolution but bypassing clinical sound processing using single-electrode stimulation. Results indicated that modulation sensitivity and pitch resolution was better conveyed by single-electrode stimulation than by clinical processors. Results at 440 Hz were worse, but also not well conveyed by clinical sound processing, so it remains unclear whether the 300 Hz perceptual limit described in the literature is a technological or biological limitation. These results highlight modulation depth and sensitivity as critical factors for pitch resolution in cochlear implant users and characterize the relationship that should inform the design of modulation enhancement algorithms for cochlear implants.

Electrophysiological and Psychophysical Measures of Temporal Pitch Sensitivity in Normal-hearing Listeners

To obtain combined behavioural and electrophysiological measures of pitch perception, we presented harmonic complexes, bandpass filtered to contain only high-numbered harmonics, to normal-hearing listeners. These stimuli resemble bandlimited pulse trains and convey pitch using a purely temporal code. A core set of conditions consisted of six stimuli with baseline pulse rates of 94, 188 and 280 pps, filtered into a HIGH (3365-4755 Hz) or VHIGH (7800-10,800 Hz) region, alternating with a 36% higher pulse rate. Brainstem and cortical processing were measured using the frequency following response (FFR) and auditory change complex (ACC), respectively. Behavioural rate change difference limens (DLs) were measured by requiring participants to discriminate between a stimulus that changed rate twice (up-down or down-up) during its 750-ms presentation from a constant-rate pulse train. FFRs revealed robust brainstem phase locking whose amplitude decreased with increasing rate. Moderate-sized but reliable ACCs were obtained in response to changes in purely temporal pitch and, like the psychophysical DLs, did not depend consistently on the direction of rate change or on the pulse rate for baseline rates between 94 and 280 pps. ACCs were larger and DLs lower for stimuli in the HIGH than in the VHGH region. We argue that the ACC may be a useful surrogate for behavioural measures of rate discrimination, both for normal-hearing listeners and for cochlear-implant users. We also showed that rate DLs increased markedly when the baseline rate was reduced to 48 pps, and compared the behavioural and electrophysiological findings to recent cat data obtained with similar stimuli and methods.

Temporal Pitch Sensitivity in an Animal Model: Psychophysics and Scalp Recordings : Temporal Pitch Sensitivity in Cat

Cochlear implant (CI) users show limited sensitivity to the temporal pitch conveyed by electric stimulation, contributing to impaired perception of music and of speech in noise. Neurophysiological studies in cats suggest that this limitation is due, in part, to poor transmission of the temporal fine structure (TFS) by the brainstem pathways that are activated by electrical cochlear stimulation. It remains unknown, however, how that neural limit might influence perception in the same animal model. For that reason, we developed non-invasive psychophysical and electrophysiological measures of temporal (i.e., non-spectral) pitch processing in the cat. Normal-hearing (NH) cats were presented with acoustic pulse trains consisting of band-limited harmonic complexes that simulated CI stimulation of the basal cochlea while removing cochlear place-of-excitation cues. In the psychophysical procedure, trained cats detected changes from a base pulse rate to a higher pulse rate. In the scalp-recording procedure, the cortical-evoked acoustic change complex (ACC) and brainstem-generated frequency following response (FFR) were recorded simultaneously in sedated cats for pulse trains that alternated between the base and higher rates. The range of perceptual sensitivity to temporal pitch broadly resembled that of humans but was shifted to somewhat higher rates. The ACC largely paralleled these perceptual patterns, validating its use as an objective measure of temporal pitch sensitivity. The phase-locked FFR, in contrast, showed strong brainstem encoding for all tested pulse rates. These measures demonstrate the cat's perceptual sensitivity to pitch in the absence of cochlear-place cues and may be valuable for evaluating neural mechanisms of temporal pitch perception in the feline animal model of stimulation by a CI or novel auditory prostheses.

Advantages of Pulse Rate Compared to Modulation Frequency for Temporal Pitch Perception in Cochlear Implant Users

Most cochlear implants encode the fundamental frequency of periodic sounds by amplitude modulation of constant-rate pulsatile stimulation. Pitch perception provided by such stimulation strategies is markedly poor. Two experiments are reported here that consider potential advantages of pulse rate compared to modulation frequency for providing stimulation timing cues for pitch. The first experiment examines beat frequency distortion that occurs when modulating constant-rate pulsatile stimulation. This distortion has been reported on previously, but the results presented here indicate that distortion occurs for higher stimulation rates than previously reported. The second experiment examines pitch resolution as provided by pulse rate compared to modulation frequency. The results indicate that pitch discrimination is better with pulse rate than with modulation frequency. The advantage was large for rates near what has been suggested as the upper limit of temporal pitch perception conveyed by cochlear implants. The results are relevant to sound processing design for cochlear implants particularly for algorithms that encode fundamental frequency into deep envelope modulations or into precisely timed pulsatile stimulation.

Pitch perception is more robust to interference and better resolved when provided by pulse rate than by modulation frequency of cochlear implant stimulation

Cochlear implants are medical devices that have been used to restore hearing to more than half a million people worldwide. Most recipients achieve high levels of speech comprehension through these devices, but speech comprehension in background noise and music appreciation in general are markedly poor compared to normal hearing. A key aspect of hearing that is notably diminished in cochlear implant outcomes is the sense of pitch provided by these devices. Pitch perception is an important factor affecting speech comprehension in background noise and is critical for music perception. The present article summarizes two experiments that examine the robustness and resolution of pitch perception as provided by cochlear implant stimulation timing. The driving hypothesis is that pitch conveyed by stimulation timing cues is more robust and better resolved when provided by variable pulse rates than by modulation frequency of constant-rate stimulation. Experiment 1 examines the robustness for hearing a large, one-octave, pitch difference in the presence of interfering electrical stimulation. With robustness to interference characterized for an otherwise easily discernible pitch difference, Experiment 2 examines the resolution of discrimination thresholds in the presence of interference as conveyed by modulation frequency or by pulse rate. These experiments test for an advantage of stimulation with precise temporal cues. The results indicate that pitch provided by pulse rate is both more robust to interference and is better resolved compared to when provided by modulation frequency. These results should inform the development of new sound processing strategies for cochlear implants designed to encode fundamental frequency of sounds into precise temporal stimulation.

Pitch Matching in Cochlear Implant Users With Single-Sided Deafness: Effects of Electrode Position and Acoustic Stimulus Type

Previous studies in patients with single-sided deafness (SSD) have reported results of pitch comparisons between electric stimulation of their cochlear implant (CI) and acoustic stimulation presented to their near-normal hearing contralateral ear. These comparisons typically used sinusoids, although the percept elicited by electric stimulation may be closer to a wideband stimulus. Furthermore, it has been shown that pitch comparisons between sounds with different timbres is a difficult task and subjected to various types of range biases. The present study aims to introduce a method to minimize non-sensory biases, and to investigate the effect of different acoustic stimulus types on the frequency and variability of the electric-acoustic pitch matches. Pitch matches were collected from 13 CI users with SSD using the binary search procedure. Electric stimulation was presented at either an apical or a middle electrode position, at a rate of 800 pps. Acoustic stimulus types were sinusoids (SINE), 1/3-octave wide narrow bands of Gaussian noises (NBN), or 1/3-octave wide pulse spreading harmonic complexes (PSHC). On the one hand, NBN and PSHC are presumed to better mimic the spread of excitation produced by a single-electrode stimulation than SINE. On the other hand, SINE and PSHC contain less inherent fluctuations than NBN and may therefore provide a temporal pattern closer to that produced by a constant-amplitude electric pulse train. Analysis of mean pitch match variance showed no differences between stimulus types. However, mean pitch matches showed effects of electrode position and stimulus type, with the middle electrode always matched to a higher frequency than the apical one (p < 0.001), and significantly higher across-subject pitch matches for PSHC compared with SINE (p = 0.017). Mean pitch matches for all stimulus types were better predicted by place-dependent characteristic frequencies (CFs) based on an organ of Corti map compared with a spiral ganglion map. CF predictions were closest to pitch matches with SINE for the apical electrode position, and conversely with NBN or PSHC for the middle electrode position. These results provide evidence that the choice of acoustic stimulus type can have a significant effect on electric-acoustic pitch matching.

Low-Frequency Pitch Perception in Cochlear Implant Recipients With Normal Hearing in the Contralateral Ear

Purpose Three experiments were carried out to evaluate the low-frequency pitch perception of adults with unilateral hearing loss who received a cochlear implant (CI). Method Participants were recruited from a cohort of CI users with unilateral hearing loss and normal hearing in the contralateral ear. First, low-frequency pitch perception was assessed for the 5 most apical electrodes at 1, 3, 6, and 12 months after CI activation using an adaptive pitch-matching task. Participants listened with a coding strategy that presents low-frequency temporal fine structure (TFS) and compared the pitch to that of an acoustic target presented to the normal hearing ear. Next, participants listened with an envelope-only, continuous interleaved sampling strategy. Pitch perception was compared between coding strategies to assess the influence of TFS cues on low-frequency pitch perception. Finally, participants completed a vocal pitch-matching task to corroborate the results obtained with the adaptive pitch-matching task. Results Pitch matches roughly corresponded to electrode center frequencies (CFs) in the CI map. Adaptive pitch matches exceeded the CF for the most apical electrode, an effect that was larger for continuous interleaved sampling than TFS. Vocal pitch matches were variable but correlated with the CF of the 3 most apical electrodes. There was no evidence that pitch matches changed between the 1- and 12-month intervals. Conclusions Relatively accurate and asymptotic pitch perception was observed at the 1-month interval, indicating either very rapid acclimatization or the provision of familiar place and rate cues. Early availability of appropriate pitch cues could have played a role in the early improvements in localization and masked speech recognition previously observed in this cohort. Supplemental Material https://doi.org/10.23641/asha.8862389.

Fundamental-frequency discrimination based on temporal-envelope cues: Effects of bandwidth and interference

Both music and speech perception rely on hearing out one pitch in the presence of others. Pitch discrimination of narrowband sounds based only on temporal-envelope cues is rendered nearly impossible by introducing interferers in both normal-hearing listeners and cochlear-implant (CI) users. This study tested whether performance improves in normal-hearing listeners if the target is presented over a broad spectral region. The results indicate that performance is still strongly affected by spectrally remote interferers, despite increases in bandwidth, suggesting that envelope-based pitch is unlikely to allow CI users to perceive pitch when multiple harmonic sounds are presented at once.

How We Hear: The Perception and Neural Coding of Sound

Auditory perception is our main gateway to communication with others via speech and music, and it also plays an important role in alerting and orienting us to new events. This review provides an overview of selected topics pertaining to the perception and neural coding of sound, starting with the first stage of filtering in the cochlea and its profound impact on perception. The next topic, pitch, has been debated for millennia, but recent technical and theoretical developments continue to provide us with new insights. Cochlear filtering and pitch both play key roles in our ability to parse the auditory scene, enabling us to attend to one auditory object or stream while ignoring others. An improved understanding of the basic mechanisms of auditory perception will aid us in the quest to tackle the increasingly important problem of hearing loss in our aging population.

Rate discrimination at low pulse rates in normal-hearing and cochlear implant listeners: Influence of intracochlear stimulation site

Temporal pitch perception in cochlear implantees remains weaker than in normal hearing listeners and is usually limited to rates below about 300 pulses per second (pps). Recent studies have suggested that stimulating the apical part of the cochlea may improve the temporal coding of pitch by cochlear implants (CIs), compared to stimulating other sites. The present study focuses on rate discrimination at low pulse rates (ranging from 20 to 104 pps). Two experiments measured and compared pulse rate difference limens (DLs) at four fundamental frequencies (ranging from 20 to 104 Hz) in both CI and normal-hearing (NH) listeners. Experiment 1 measured DLs in users of the (Med-El CI, Innsbruck, Austria) device for two electrodes (one apical and one basal). In experiment 2, DLs for NH listeners were compared for unresolved harmonic complex tones filtered in two frequency regions (lower cut-off frequencies of 1200 and 3600 Hz, respectively) and for different bandwidths. Pulse rate discrimination performance was significantly better when stimulation was provided by the apical electrode in CI users and by the lower-frequency tone complexes in NH listeners. This set of data appears consistent with better temporal coding when stimulation originates from apical regions of the cochlea.

Training improves cochlear implant rate discrimination on a psychophysical task

The purpose of this study was to determine the extent to which cochlear implant (CI) rate discrimination can be improved through training. Six adult CI users took part in a study that included 32 h of training and assessment on rate discrimination measures. Rate difference limens (DLs) were measured from 110 to 3520 Hz in octave steps using 500 ms biphasic pulse trains; the target and standard stimuli were loudness-balanced with the target always at an adaptively lower rate. DLs were measured at four electrode positions corresponding to basal, mid-basal, mid-apical, and apical locations. Procedural variations were implemented to determine if rate discrimination was impacted by random variations in stimulus amplitude or by amplitude modulation. DLs improved by more than a factor of 2 across subjects, electrodes, and standard rates. Factor analysis indicated that the effect of training was comparable for all electrodes and standard rates tested. Neither level roving nor amplitude modulation had a significant effect on rate DLs. In conclusion, the results demonstrate that training can significantly improve CI rate discrimination on a psychophysical task.

Revisiting place and temporal theories of pitch

The nature of pitch and its neural coding have been studied for over a century. A popular debate has revolved around the question of whether pitch is coded via "place" cues in the cochlea, or via timing cues in the auditory nerve. In the most recent incarnation of this debate, the role of temporal fine structure has been emphasized in conveying important pitch and speech information, particularly because the lack of temporal fine structure coding in cochlear implants might explain some of the difficulties faced by cochlear implant users in perceiving music and pitch contours in speech. In addition, some studies have postulated that hearing-impaired listeners may have a specific deficit related to processing temporal fine structure. This article reviews some of the recent literature surrounding the debate, and argues that much of the recent evidence suggesting the importance of temporal fine structure processing can also be accounted for using spectral (place) or temporal-envelope cues.