On the relative influence of individual harmonics on pitch judgment

Pitch discrimination is better for synthetic timbre than natural musical instrument timbres despite familiarity

Pitch discrimination is better for complex tones than pure tones, but how pitch discrimination differs between natural and artificial sounds is not fully understood. This study compared pitch discrimination thresholds for flat-spectrum harmonic complex tones with those for natural sounds played by musical instruments of three different timbres (violin, trumpet, and flute). To investigate whether natural familiarity with sounds of particular timbres affects pitch discrimination thresholds, this study recruited non-musicians and musicians who were trained on one of the three instruments. We found that flautists and trumpeters could discriminate smaller differences in pitch for artificial flat-spectrum tones, despite their unfamiliar timbre, than for sounds played by musical instruments, which are regularly heard in everyday life (particularly by musicians who play those instruments). Furthermore, thresholds were no better for the instrument a musician was trained to play than for other instruments, suggesting that even extensive experience listening to and producing sounds of particular timbres does not reliably improve pitch discrimination thresholds for those timbres. The results show that timbre familiarity provides minimal improvements to auditory acuity, and physical acoustics (e.g., the presence of equal-amplitude harmonics) determine pitch discrimination thresholds more than does experience with natural sounds and timbre-specific training.

Spectral weighting functions for lateralization and localization of complex sound

Perceptual weighting of sound localization cues across spectral components was measured over headphones [experiment (expt.) 1] and in the free field (expt. 2) and quantified in the form of spectral weighting functions (SWFs). Stimuli consisted of five complex sounds (conditions), each containing seven frequency components. Participants judged the spatial position of the stimuli with spatial cues varying across frequency components. In separate experiments, free-field stimuli were presented anechoically (expt. 2), in the presence of simulated reverberation (expt. 3), or with stimuli varying in level either corrected for equal loudness (expt. 4.1) or sloped by ±6 dB per component (expt. 4.2). Overall results revealed greatest weight in the vicinity of 800 Hz, for both localization and interaural time difference (ITD)-based lateralization, although specific features of the SWFs did vary across stimulus conditions. The shape of the SWF follows the pattern of ITD sensitivity across frequency and is consistent with previous descriptions of an ITD "dominance region" peaking around 600-800 Hz. The close similarity of free field and ITD-based SWFs is further consistent with the hypothesized dominant role of low-frequency ITD in localization of broadband sounds. Other conditions revealed relatively modest effects of reverberation and component level.

Cross-frequency weights in normal and impaired hearing: Stimulus factors, stimulus dimensions, and associations with speech recognition

Previous studies of level discrimination reported that listeners with high-frequency sensorineural hearing loss (SNHL) place greater weight on high frequencies than normal-hearing (NH) listeners. It is not clear whether these results are influenced by stimulus factors (e.g., group differences in presentation levels, cross-frequency discriminability of level differences used to measure weights) and whether such weights generalize to other tasks. Here, NH and SNHL weights were measured for level, duration, and frequency discrimination of two-tone complexes after measuring discriminability just-noticeable differences for each frequency and stimulus dimension. Stimuli were presented at equal sensation level (SL) or equal sound pressure level (SPL). Results showed that weights could change depending on which frequency contained the more discriminable level difference with uncontrolled cross-frequency discriminability. When cross-frequency discriminability was controlled, weights were consistent for level and duration discrimination, but not for frequency discrimination. Comparing equal SL and equal SPL weights indicated greater weight on the higher-level tone for level and duration discrimination. Weights were unrelated to improvements in recognition of low-pass-filtered speech with increasing cutoff frequency. These results suggest that cross-frequency weights and NH and SNHL weighting differences are influenced by stimulus factors and may not generalize to the use of speech cues in specific frequency regions.

Effect of lowest harmonic rank on fundamental-frequency difference limens varies with fundamental frequency

This study investigated the relationship between fundamental frequency difference limens (F0DLs) and the lowest harmonic number present over a wide range of F0s (30-2000 Hz) for 12-component harmonic complex tones that were presented in either sine or random phase. For fundamental frequencies (F0s) between 100 and 400 Hz, a transition from low (∼1%) to high (∼5%) F0DLs occurred as the lowest harmonic number increased from about seven to ten, in line with earlier studies. At lower and higher F0s, the transition between low and high F0DLs occurred at lower harmonic numbers. The worsening performance at low F0s was reasonably well predicted by the expected decrease in spectral resolution below about 500 Hz. At higher F0s, the degradation in performance at lower harmonic numbers could not be predicted by changes in spectral resolution but remained relatively good (<2%-3%) in some conditions, even when all harmonics were above 8 kHz, confirming that F0 can be extracted from harmonics even when temporal envelope or fine-structure cues are weak or absent.

On averaging multiple estimates of decision weights within or across listeners

As a practical guide, a formula is provided for averaging multiple estimates of decision weights, for the purpose of improving the accuracy and reliability of the final estimate of decision weights. The averaging over multiple weight estimates can take place either within or across listeners.

Harmonic template neurons in primate auditory cortex underlying complex sound processing

Harmonicity is a fundamental element of music, speech, and animal vocalizations. How the auditory system extracts harmonic structures embedded in complex sounds and uses them to form a coherent unitary entity is not fully understood. Despite the prevalence of sounds rich in harmonic structures in our everyday hearing environment, it has remained largely unknown what neural mechanisms are used by the primate auditory cortex to extract these biologically important acoustic structures. In this study, we discovered a unique class of harmonic template neurons in the core region of auditory cortex of a highly vocal New World primate, the common marmoset (Callithrix jacchus), across the entire hearing frequency range. Marmosets have a rich vocal repertoire and a similar hearing range to that of humans. Responses of these neurons show nonlinear facilitation to harmonic complex sounds over inharmonic sounds, selectivity for particular harmonic structures beyond two-tone combinations, and sensitivity to harmonic number and spectral regularity. Our findings suggest that the harmonic template neurons in auditory cortex may play an important role in processing sounds with harmonic structures, such as animal vocalizations, human speech, and music.

Low-frequency fine-structure cues allow for the online use of lexical stress during spoken-word recognition in spectrally degraded speech

English listeners use suprasegmental cues to lexical stress during spoken-word recognition. Prosodic cues are, however, less salient in spectrally degraded speech, as provided by cochlear implants. The present study examined how spectral degradation with and without low-frequency fine-structure information affects normal-hearing listeners' ability to benefit from suprasegmental cues to lexical stress in online spoken-word recognition. To simulate electric hearing, an eight-channel vocoder spectrally degraded the stimuli while preserving temporal envelope information. Additional lowpass-filtered speech was presented to the opposite ear to simulate bimodal hearing. Using a visual world paradigm, listeners' eye fixations to four printed words (target, competitor, two distractors) were tracked, while hearing a word. The target and competitor overlapped segmentally in their first two syllables but mismatched suprasegmentally in their first syllables, as the initial syllable received primary stress in one word and secondary stress in the other (e.g., "'admiral," "'admi'ration"). In the vocoder-only condition, listeners were unable to use lexical stress to recognize targets before segmental information disambiguated them from competitors. With additional lowpass-filtered speech, however, listeners efficiently processed prosodic information to speed up online word recognition. Low-frequency fine-structure cues in simulated bimodal hearing allowed listeners to benefit from suprasegmental cues to lexical stress during word recognition.

Informational Masking in Normal-Hearing and Hearing-Impaired Listeners Measured in a Nonspeech Pattern Identification Task

Individuals with sensorineural hearing loss (SNHL) often experience more difficulty with listening in multisource environments than do normal-hearing (NH) listeners. While the peripheral effects of sensorineural hearing loss certainly contribute to this difficulty, differences in central processing of auditory information may also contribute. To explore this issue, it is important to account for peripheral differences between NH and these hearing-impaired (HI) listeners so that central effects in multisource listening can be examined. In the present study, NH and HI listeners performed a tonal pattern identification task at two distant center frequencies (CFs), 850 and 3500 Hz. In an attempt to control for differences in the peripheral representations of the stimuli, the patterns were presented at the same sensation level (15 dB SL), and the frequency deviation of the tones comprising the patterns was adjusted to obtain equal quiet pattern identification performance across all listeners at both CFs. Tonal sequences were then presented at both CFs simultaneously (informational masking conditions), and listeners were asked either to selectively attend to a source (CF) or to divide attention between CFs and identify the pattern at a CF designated after each trial. There were large differences between groups in the frequency deviations necessary to perform the pattern identification task. After compensating for these differences, there were small differences between NH and HI listeners in the informational masking conditions. HI listeners showed slightly greater performance asymmetry between the low and high CFs than did NH listeners, possibly due to central differences in frequency weighting between groups.

Harmonic Frequency Lowering: Effects on the Perception of Music Detail and Sound Quality

A novel algorithm for frequency lowering in music was developed and experimentally tested in hearing-impaired listeners. Harmonic frequency lowering (HFL) combines frequency transposition and frequency compression to preserve the harmonic content of music stimuli. Listeners were asked to make judgments regarding detail and sound quality in music stimuli. Stimuli were presented under different signal processing conditions: original, low-pass filtered, HFL, and nonlinear frequency compressed. Results showed that participants reported perceiving the most detail in the HFL condition. In addition, there was no difference in sound quality across conditions.

Dynamic representation of spectral edges in guinea pig primary auditory cortex

The central representation of a given acoustic motif is thought to be strongly context dependent, i.e., to rely on the spectrotemporal past and present of the acoustic mixture in which it is embedded. The present study investigated the cortical representation of spectral edges (i.e., where stimulus energy changes abruptly over frequency) and its dependence on stimulus duration and depth of the spectral contrast in guinea pig. We devised a stimulus ensemble composed of random tone pips with or without an attenuated frequency band (AFB) of variable depth. Additionally, the multitone ensemble with AFB was interleaved with periods of silence or with multitone ensembles without AFB. We have shown that the representation of the frequencies near but outside the AFB is greatly enhanced, whereas the representation of frequencies near and inside the AFB is strongly suppressed. These cortical changes depend on the depth of the AFB: although they are maximal for the largest depth of the AFB, they are also statistically significant for depths as small as 10 dB. Finally, the cortical changes are quick, occurring within a few seconds of stimulus ensemble presentation with AFB, and are very labile, disappearing within a few seconds after the presentation without AFB. Overall, this study demonstrates that the representation of spectral edges is dynamically enhanced in the auditory centers. These central changes may have important functional implications, particularly in noisy environments where they could contribute to preserving the central representation of spectral edges.

Congenital amusia: a cognitive disorder limited to resolved harmonics and with no peripheral basis

Pitch plays a fundamental role in audition, from speech and music perception to auditory scene analysis. Congenital amusia is a neurogenetic disorder that appears to affect primarily pitch and melody perception. Pitch is normally conveyed by the spectro-temporal fine structure of low harmonics, but some pitch information is available in the temporal envelope produced by the interactions of higher harmonics. Using 10 amusic subjects and 10 matched controls, we tested the hypothesis that amusics suffer exclusively from impaired processing of spectro-temporal fine structure. We also tested whether the inability of amusics to process acoustic temporal fine structure extends beyond pitch by measuring sensitivity to interaural time differences, which also rely on temporal fine structure. Further tests were carried out on basic intensity and spectral resolution. As expected, pitch perception based on spectro-temporal fine structure was impaired in amusics; however, no significant deficits were observed in amusics' ability to perceive the pitch conveyed via temporal-envelope cues. Sensitivity to interaural time differences was also not significantly different between the amusic and control groups, ruling out deficits in the peripheral coding of temporal fine structure. Finally, no significant differences in intensity or spectral resolution were found between the amusic and control groups. The results demonstrate a pitch-specific deficit in fine spectro-temporal information processing in amusia that seems unrelated to temporal or spectral coding in the auditory periphery. These results are consistent with the view that there are distinct mechanisms dedicated to processing resolved and unresolved harmonics in the general population, the former being altered in congenital amusia while the latter is spared.

The dominant region for the pitch of complex tones with low fundamental frequencies

The dominant region for pitch for complex tones with low fundamental frequency (F0) was investigated. Thresholds for detection of a change in F0 (F0DLs) were measured for a group of harmonics (group B) embedded in a group of fixed non-overlapping harmonics (group A) with the same mean F0. It was assumed that F0DLs would be smallest when the harmonics in group B fell in the dominant region. The rank of the lowest harmonic in group B, N, was varied from 1 to 15. When all components had the same level, F0DLs increased with increasing N, but the increase started at a lower value of N for F0 = 200 Hz than for F0 = 50 or 100 Hz, the opposite of what would be expected if the dominant region corresponds to resolved harmonics. When the component levels followed an equal-loudness contour, F0DLs for F0 = 50 Hz were lowest for N = 1, but overall performance was much worse than for equal-level components, suggesting that the lowest harmonics were masking information from the higher harmonics.

Enhanced representation of spectral contrasts in the primary auditory cortex

The role of early auditory processing may be to extract some elementary features from an acoustic mixture in order to organize the auditory scene. To accomplish this task, the central auditory system may rely on the fact that sensory objects are often composed of spectral edges, i.e., regions where the stimulus energy changes abruptly over frequency. The processing of acoustic stimuli may benefit from a mechanism enhancing the internal representation of spectral edges. While the visual system is thought to rely heavily on this mechanism (enhancing spatial edges), it is still unclear whether a related process plays a significant role in audition. We investigated the cortical representation of spectral edges, using acoustic stimuli composed of multi-tone pips whose time-averaged spectral envelope contained suppressed or enhanced regions. Importantly, the stimuli were designed such that neural responses properties could be assessed as a function of stimulus frequency during stimulus presentation. Our results suggest that the representation of acoustic spectral edges is enhanced in the auditory cortex, and that this enhancement is sensitive to the characteristics of the spectral contrast profile, such as depth, sharpness and width. Spectral edges are maximally enhanced for sharp contrast and large depth. Cortical activity was also suppressed at frequencies within the suppressed region. To note, the suppression of firing was larger at frequencies nearby the lower edge of the suppressed region than at the upper edge. Overall, the present study gives critical insights into the processing of spectral contrasts in the auditory system.

Enhancement of temporal cues to pitch in cochlear implants: effects on pitch ranking

The abilities to hear changes in pitch for sung vowels and understand speech using an experimental sound coding strategy (eTone) that enhanced coding of temporal fundamental frequency (F0) information were tested in six cochlear implant users, and compared with performance using their clinical (ACE) strategy. In addition, rate- and modulation rate-pitch difference limens (DLs) were measured using synthetic stimuli with F0s below 300 Hz to determine psychophysical abilities of each subject and to provide experience in attending to rate cues for the judgment of pitch. Sung-vowel pitch ranking tests for stimuli separated by three semitones presented across an F0 range of one octave (139-277 Hz) showed a significant benefit for the experimental strategy compared to ACE. Average d-prime (d') values for eTone (d' = 1.05) were approximately three time larger than for ACE (d' = 0.35). Similar scores for both strategies in the speech recognition tests showed that coding of segmental speech information by the experimental strategy was not degraded. Average F0 DLs were consistent with results from previous studies and for all subjects were less than or equal to approximately three semitones for F0s of 125 and 200 Hz.

Effects of fundamental frequency and vocal-tract length cues on sentence segregation by listeners with hearing loss

The purpose was to determine the effect of hearing loss on the ability to separate competing talkers using talker differences in fundamental frequency (F0) and apparent vocal-tract length (VTL). Performance of 13 adults with hearing loss and 6 adults with normal hearing was measured using the Coordinate Response Measure. For listeners with hearing loss, the speech was amplified and filtered according to the NAL-RP hearing aid prescription. Target-to-competition ratios varied from 0 to 9 dB. The target sentence was randomly assigned to the higher or lower values of F0 or VTL on each trial. Performance improved for F0 differences up to 9 and 6 semitones for people with normal hearing and hearing loss, respectively, but only when the target talker had the higher F0. Recognition for the lower F0 target improved when trial-to-trial uncertainty was removed (9-semitone condition). Scores improved with increasing differences in VTL for the normal-hearing group. On average, hearing-impaired listeners did not benefit from VTL cues, but substantial inter-subject variability was observed. The amount of benefit from VTL cues was related to the average hearing loss in the 1-3-kHz region when the target talker had the shorter VTL.

Psychometric functions for pure-tone frequency discrimination

The form of the psychometric function (PF) for auditory frequency discrimination is of theoretical interest and practical importance. In this study, PFs for pure-tone frequency discrimination were measured for several standard frequencies (200-8000 Hz) and levels [35-85 dB sound pressure level (SPL)] in normal-hearing listeners. The proportion-correct data were fitted using a cumulative-Gaussian function of the sensitivity index, d', computed as a power transformation of the frequency difference, Δf. The exponent of the power function corresponded to the slope of the PF on log(d')-log(Δf) coordinates. The influence of attentional lapses on PF-slope estimates was investigated. When attentional lapses were not taken into account, the estimated PF slopes on log(d')-log(Δf) coordinates were found to be significantly lower than 1, suggesting a nonlinear relationship between d' and Δf. However, when lapse rate was included as a free parameter in the fits, PF slopes were found not to differ significantly from 1, consistent with a linear relationship between d' and Δf. This was the case across the wide ranges of frequencies and levels tested in this study. Therefore, spectral and temporal models of frequency discrimination must account for a linear relationship between d' and Δf across a wide range of frequencies and levels.

Development of a temporal fundamental frequency coding strategy for cochlear implants

A sound-coding strategy for users of cochlear implants, named enhanced-envelope-encoded tone (eTone), was developed to improve coding of fundamental frequency (F0) in the temporal envelopes of the electrical stimulus signals. It is based on the advanced combinational encoder (ACE) strategy and includes additional processing that explicitly applies F0 modulation to channel envelope signals that contain harmonics of prominent complex tones. Channels that contain only inharmonic signals retain envelopes normally produced by ACE. The strategy incorporates an F0 estimator to determine the frequency of modulation and a harmonic probability estimator to control the amount of modulation enhancement applied to each channel. The F0 estimator was designed to provide an accurate estimate of F0 with minimal processing lag and robustness to the effects of competing noise. Error rates for the F0 estimator and accuracy of the harmonic probability estimator were compared with previous approaches and outcomes demonstrated that the strategy operates effectively across a range of signals and conditions that are relevant to cochlear implant users.

Harmonic pitch: dependence on resolved partials, spectral edges, and combination tones

Perceptual weights were estimated in a pitch-comparison experiment to assess the relative influences of individual partial tones on listeners' pitch judgments. The stimuli were harmonic sounds (F0=200 Hz) with partials up to the 12th. Low-numbered partials were removed step-by-step, so that the remaining higher-numbered partials would have a better chance of showing any effect. The individual frequencies of the partials were perturbed randomly on each stimulus presentation, and weights were estimated as the correlation coefficients between the frequency perturbations and the listeners' responses. When the harmonic sounds contained all twelve partials, the listeners depended mostly on the low-numbered, resolved partials within the well-established dominance region. As the low-numbered partials were taken out of the dominance region, the listeners mostly listened to the lowest and highest partials at the spectral edges. For one listener, such an edge-listening strategy took the form of relying on nonlinear combination tones. Overall, there was no indication of any influence on pitch from unresolved partials, thus no evidence of contribution to pitch from temporal cues carried by this group of partials. The estimated patterns of weights were well described by the predictions of Goldstein's optimal-processor model. The predicted weights were inversely proportional to the amount of error for estimating the individual frequencies of the partials. The agreement between the predicted and measured weights suggests that, for harmonic sounds, partials whose frequencies are perceived with the best precision will likely have the greatest influence on perceived pitch.

Cochlear implant melody recognition as a function of melody frequency range, harmonicity, and number of electrodes

OBJECTIVE

The primary goal of the present study was to determine how cochlear implant melody recognition was affected by the frequency range of the melodies, the harmonicity of these melodies, and the number of activated electrodes. The secondary goal was to investigate whether melody recognition and speech recognition were differentially affected by the limitations imposed by cochlear implant processing.

DESIGN

Four experiments were conducted. In the first experiment, 11 cochlear implant users used their clinical processors to recognize melodies of complex harmonic tones with their fundamental frequencies being in the low (104-262 Hz), middle (207-523 Hz), and high (414-1046 Hz) ranges. In the second experiment, melody recognition with pure tones was compared to melody recognition with complex harmonic tones in four subjects. In the third experiment, melody recognition was measured as a function of the number of electrodes in five subjects. In the fourth experiment, vowel and consonant recognition were measured as a function of the number of electrodes in the same five subjects who participated in the third experiment.

RESULTS

Frequency range significantly affected cochlear implant melody recognition, with higher frequency ranges producing better performance. Pure tones produced significantly better performance than complex harmonic tones. Increasing the number of activated electrodes did not affect performance with low- and middle-frequency melodies but produced better performance with high-frequency melodies. Large individual variability was observed for melody recognition, but its source seemed to be different from the source of the large variability observed in speech recognition.

CONCLUSION

Contemporary cochlear implants do not adequately encode either temporal pitch or place pitch cues. Melody recognition and speech recognition require different signal processing strategies in future cochlear implants.

Psychophysical reverse correlation with multiple response alternatives

Psychophysical reverse-correlation methods such as the "classification image" technique provide a unique tool to uncover the internal representations and decision strategies of individual participants in perceptual tasks. Over the past 30 years, these techniques have gained increasing popularity among both visual and auditory psychophysicists. However, thus far, principled applications of the psychophysical reverse-correlation approach have been almost exclusively limited to two-alternative decision (detection or discrimination) tasks. Whether and how reverse-correlation methods can be applied to uncover perceptual templates and decision strategies in situations involving more than just two response alternatives remain largely unclear. Here, the authors consider the problem of estimating perceptual templates and decision strategies in stimulus identification tasks with multiple response alternatives. They describe a modified correlational approach, which can be used to solve this problem. The approach is evaluated under a variety of simulated conditions, including different ratios of internal-to-external noise, different degrees of correlations between the sensory observations, and various statistical distributions of stimulus perturbations. The results indicate that the proposed approach is reasonably robust, suggesting that it could be used in future empirical studies.

Combining information across frequency regions in fundamental frequency discrimination

Sensitivity to fundamental frequency (F0) differences was measured for two complex tones, A and B, which had the same F0 but were filtered into two different frequency regions. Tones were presented either alone or together. A signal-detection analysis was used to predict effects of combining F0 information across frequency regions. For 400-ms tones containing only unresolved harmonics, the first experiment showed that performance (in terms of d(')) for the combined presentation was better than for the isolated tones but was not optimal (assuming independent channels and noises) and was independent of the relative timing of pulses in the envelopes of tones A and B (varied by changing the starting phase of components of tone B relative to those of tone A). The nonoptimal performance was shown not to be due to peripheral masking (experiment II), or to listeners paying attention mainly to one frequency region (experiment III), nor was it specific to conditions where all harmonics were unresolved (experiment IV). In contrast, optimal performance in F0 discrimination for combined presentation was observed for 50-ms tones (experiment V). The results may reflect the limited ability of the human auditory system to integrate information simultaneously in the time and the frequency domains.

Ambiguous pitch and the temporal representation of inharmonic iterated rippled noise in the ventral cochlear nucleus

Neural coding of the pitch of complex sounds is vital for animals' ability to communicate and to perceptually organize natural acoustic scenes. Harmonic complex sounds typically have a well defined pitch corresponding to their fundamental frequency, whereas inharmonic sounds can exhibit pitch ambiguity: their pitch can have more than one value. Iterated rippled noise (IRN), a common "pitch stimulus," is generated from broadband noise by a cascade of delay-and-add steps, with the delayed noise phase-shifted by varphi degrees. By varying varphi, the (in)harmonicity, and therefore the pitch ambiguity, of IRN can be manipulated. Recordings were made from single-units in the ventral cochlear nucleus of anesthetized guinea pigs in response to IRN and complex tones, systematically varying the inharmonicity. In their all-order interspike interval distributions, primary-like and chopper units tuned within the phase-locking range of best frequencies represent the waveform temporal fine structure (which varies with varphi). In contrast, those units tuned to higher frequencies represent the temporal-envelope modulation (independent of varphi). We show a temporal representation of ambiguous pitch for IRN and complex tones based on responses to the stimulus fine structure. Within the dominance region for pitch this representation follows the predictions of classic human behavioral experiments and provides a unifying contribution to possible neuro-temporal explanations for the pitch shift and pitch ambiguity associated with many inharmonic sounds.

Reduced contribution of a nonsimultaneous mistuned harmonic to residue pitch: the role of harmonic number

Listeners adjusted a matching sound so that its pitch equaled that of a subsequent 90-ms complex tone (12 harmonics of a 155-Hz F0), whose mistuned harmonic (+/-3%) was presented either simultaneously with or after the other harmonics. The third or the fourth harmonic was mistuned. For both harmonics, pitch shifts were significantly smaller when the mistuned harmonic was presented after rather than simultaneously with the remainder of the target complex and when the component corresponding to the mistuned harmonic was absent (as opposed to present) from the matching sound. Presenting the mistuned component after the remainder of the complex generally reduced the pitch shifts more for the third than for the fourth harmonic; the nonsimultaneous presentation reduced shifts by a factor of 2.8 (third) and 1.7 (fourth) in the presence, and 6.4 (third) and 1.9 (fourth) in the absence of the nominally mistuned harmonic from the matching sound. The results are discussed in terms of an analytical listening and matching strategy that results in larger pitch shifts for a nonsimultaneous mistuned harmonic that has an octave or double-octave relationship to the F0 than for one that does not, without it necessarily being temporally integrated into the residue pitch of the complex.

Pitch perception and auditory stream segregation: implications for hearing loss and cochlear implants

Pitch is important for speech and music perception, and may also play a crucial role in our ability to segregate sounds that arrive from different sources. This article reviews some basic aspects of pitch coding in the normal auditory system and explores the implications for pitch perception in people with hearing impairments and cochlear implants. Data from normal-hearing listeners suggest that the low-frequency, low-numbered harmonics within complex tones are of prime importance in pitch perception and in the perceptual segregation of competing sounds. The poorer frequency selectivity experienced by many hearing-impaired listeners leads to less access to individual harmonics, and the coding schemes currently employed in cochlear implants provide little or no representation of individual harmonics. These deficits in the coding of harmonic sounds may underlie some of the difficulties experienced by people with hearing loss and cochlear implants, and may point to future areas where sound representation in auditory prostheses could be improved.

Reverberation challenges the temporal representation of the pitch of complex sounds

Accurate neural coding of the pitch of complex sounds is an essential part of auditory scene analysis; differences in pitch help segregate concurrent sounds, while similarities in pitch can help group sounds from a common source. In quiet, nonreverberant backgrounds, pitch can be derived from timing information in broadband high-frequency auditory channels and/or from frequency and timing information carried in narrowband low-frequency auditory channels. Recording from single neurons in the cochlear nucleus of anesthetized guinea pigs, we show that the neural representation of pitch based on timing information is severely degraded in the presence of reverberation. This degradation increases with both increasing reverberation strength and channel bandwidth. In a parallel human psychophysical pitch-discrimination task, reverberation impaired the ability to distinguish a high-pass harmonic sound from noise. Together, these findings explain the origin of perceptual difficulties experienced by both normal-hearing and hearing-impaired listeners in reverberant spaces.

The temporal representation of the delay of dynamic iterated rippled noise with positive and negative gain by single units in the ventral cochlear nucleus

Spike trains were recorded from single units in the ventral cochlear nucleus of the anaesthetised guinea-pig in response to dynamic iterated rippled noise with positive and negative gain. The short-term running waveform autocorrelation functions of these stimuli show peaks at integer multiples of the time-varying delay when the gain is +1, and troughs at odd-integer multiples and peaks at even-integer multiples of the time-varying delay when the gain is -1. In contrast, the short-term autocorrelation of the Hilbert envelope shows peaks at integer multiples of the time-varying delay for both positive and negative gain stimuli. A running short-term all-order interspike interval analysis demonstrates the ability of single units to represent the modulated pitch contour in their short-term interval statistics. For units with low best frequency (approximate < or = 1.1 kHz) the temporal discharge pattern reflected the waveform fine structure regardless of unit classification (Primary-like, Chopper). For higher best frequency units the pattern of response varied according to unit type. Chopper units with best frequency approximate > or = 1.1 kHz responded to envelope modulation; showing no difference between their response to stimuli with positive and negative gain. Primary-like units with best frequencies in the range 1-3 kHz were still able to represent the difference in the temporal fine structure between dynamic rippled noise with positive and negative gain. No unit with a best frequency above 3 kHz showed a response to the temporal fine structure. Chopper units in this high frequency group showed significantly greater representation of envelope modulation relative to primary-like units with the same range of best frequencies. These results show that at the level of the cochlear nucleus there exists sufficient information in the time domain to represent the time-varying pitch associated with dynamic iterated rippled noise.

Across-frequency pitch discrimination interference between complex tones containing resolved harmonics

Pitch discrimination interference (PDI) refers to an impairment in the ability to discriminate changes in the fundamental frequency (F0) of a target harmonic complex, caused by another harmonic complex (the interferer) presented simultaneously in a remote spectral region. So far, PDI has been demonstrated for target complexes filtered into a higher spectral region than the interferer and containing no peripherally resolved harmonics in their passband. Here, it is shown that PDI also occurs when the target harmonic complex contains resolved harmonics in its passband (experiment 1). PDI was also observed when the target was filtered into a lower spectral region than that of the interferer (experiment 2), revealing that differences in relative harmonic dominance and pitch salience between the simultaneous target and the interferer, as confirmed using pitch matches (experiment 3), do not entirely explain PDI. When the target was in the higher spectral region, and the F0 separation between the target and the interferer was around 7% or 10%, dramatic PDI effects were observed despite the relatively large FO separation between the two sequential targets (14%-20%). Overall, the results suggest that PDI is more general than previously thought, and is not limited to targets consisting only of unresolved harmonics.

Effect of duration on the frequency discrimination of individual partials in a complex tone and on the discrimination of fundamental frequency

Thresholds for the discrimination of fundamental frequency (FODLs) and frequency difference limens (FDLs) for individual partials within a complex tone (F0=250 Hz, harmonics 1-7) were measured for stimulus durations of 200, 50, and 16 ms. The FDLs increased with decreasing duration. Although the results differed across subjects, the effect of duration generally decreased as the harmonic number increased from 1 to 4, then increased as the harmonic number increased to 6, and finally decreased for the seventh harmonic. For each duration, FODLs were smaller than the smallest FDL for any individual harmonic, indicating that information is combined across harmonics in the discrimination of FO. FODLs predicted from the FDLs corresponded well with observed FODLs for the 200- and 16-ms durations but were significantly larger than observed FODLs for the 50-ms duration. A supplementary pitch-matching experiment using two subjects indicated that the contribution of the seventh harmonic to the pitch of the 16-ms complex tone was smaller than would be predicted from the FDL for that harmonic. The results are consistent with the idea that the dominant region shifts upward with decreasing duration, but that the weight assigned to individual harmonics is not always adjusted in an optimal way.

Effects of introducing unprocessed low-frequency information on the reception of envelope-vocoder processed speech

This study investigated the benefits of adding unprocessed low-frequency information to acoustic simulations of cochlear-implant processing in normal-hearing listeners. Implant processing was simulated using an eight-channel noise-excited envelope vocoder, and low-frequency information was added by replacing the lower frequency channels of the processor with a low-pass-filtered version of the original stimulus. Experiment 1 measured sentence-level speech reception as a function of target-to-masker ratio, with either steady-state speech-shaped noise or single-talker maskers. Experiment 2 measured listeners' ability to identify two vowels presented simultaneously, as a function of the F0 difference between the two vowels. In both experiments low-frequency information was added below either 300 or 600 Hz. The introduction of the additional low-frequency information led to substantial and significant improvements in performance in both experiments, with a greater improvement observed for the higher (600 Hz) than for the lower (300 Hz) cutoff frequency. However, performance never equaled performance in the unprocessed conditions. The results confirm other recent demonstrations that added low-frequency information can provide significant benefits in intelligibility, which may at least in part be attributed to improvements in F0 representation. The findings provide further support for efforts to make use of residual acoustic hearing in cochlear-implant users.

Pitch is determined by naturally occurring periodic sounds

The phenomenology of pitch has been difficult to rationalize and remains the subject of much debate. Here we test the hypothesis that audition generates pitch percepts by relating inherently ambiguous sound stimuli to their probable sources in the human auditory environment. A database of speech sounds, the principal source of periodic sound energy for human listeners, was compiled and the dominant periodicity of each speech sound determined. A set of synthetic test stimuli were used to assess whether the major pitch phenomena described in the literature could be explained by the probabilistic relationship between the stimuli and their probable sources (i.e., speech sounds). The phenomena tested included the perception of the missing fundamental, the pitch-shift of the residue, spectral dominance and the perception of pitch strength. In each case, the conditional probability distribution of speech sound periodicities accurately predicted the pitches normally heard in response to the test stimuli. We conclude from these findings that pitch entails an auditory process that relates inevitably ambiguous sound stimuli to their probable natural sources.

Dominance region for pitch: effects of duration and dichotic presentation

The dominance region (DR) for pitch was determined for 16- and 200-ms complex tones containing the first seven harmonics of a fundamental frequency (F0) of 250 Hz. A tone was presented with one of the harmonics mistuned upwards or downwards by 3%, followed 500 ms later by a perfectly harmonic tone of the same duration. Listeners adjusted the F0 of the harmonic tone so that its pitch matched that of the mistuned complex. In experiment 1, stimuli were presented monaurally. The DR was significantly higher in harmonic number for the short than for the long duration. The overall sum of the pitch shifts produced by all harmonics was significantly larger for the short than for the long duration, presumably due to stronger perceptual fusion for the former. In experiment 2, the mistuned harmonic was presented only contralaterally to the remainder of the complex. A similar shift in the DR with duration was observed, although the pitch shifts were smaller than for monaural presentation. There was no significant effect of duration on the overall pitch shifts. The results are discussed in terms of pattern recognition and autocorrelation models of pitch perception, and a role of attention in pitch matching is suggested.

Across-frequency interference effects in fundamental frequency discrimination: questioning evidence for two pitch mechanisms

Carlyon and Shackleton [J. Acoust. Soc. Am. 95, 3541-3554 (1994)] presented an influential study supporting the existence of two pitch mechanisms, one for complex tones containing resolved and one for complex tones containing only unresolved components. The current experiments provide an alternative explanation for their finding, namely the existence of across-frequency interference in fundamental frequency (F0) discrimination. Sensitivity (d') was measured for F0 discrimination between two sequentially presented 400 ms complex (target) tones containing only unresolved components. In experiment 1, the target was filtered between 1375 and 15,000 Hz, had a nominal F0 of 88 Hz, and was presented either alone or with an additional complex tone ("interferer"). The interferer was filtered between 125-625 Hz, and its F0 varied between 88 and 114.4 Hz across blocks. Sensitivity was significantly reduced in the presence of the interferer, and this effect decreased as its F0 was moved progressively further from that of the target. Experiment 2 showed that increasing the level of a synchronously gated lowpass noise that spectrally overlapped with the interferer reduced this "pitch discrimination interference (PDI)". In experiment 3A, the target was filtered between 3900 and 5400 Hz and had an F0 of either 88 or 250 Hz. It was presented either alone or with an interferer, filtered between 1375 and 1875 Hz with an F0 corresponding to the nominal target F0. PDI was larger in the presence of the resolved (250 Hz F0) than in the presence of the unresolved (88 Hz F0) interferer, presumably because the pitch of the former was more salient than that of the latter. Experiments 4A and 4B showed that PDI was reduced but not eliminated when the interferer was gated on 200 ms before and off 200 ms after the target, and that some PDI was observed with a continuous interferer. The current findings provide an alternative interpretation of a study supposedly providing strong evidence for the existence of two pitch mechanisms.

Better place-coding of the fundamental frequency in cochlear implants

In current cochlear implant systems, the fundamental frequency F0 of a complex sound is encoded by temporal fluctuations in the envelope of the electrical signals presented on the electrodes. In normal hearing, the lower harmonics of a complex sound are resolved, in contrast with a cochlear implant system. In the present study, it is investigated whether "place-coding" of the first harmonic improves the ability of an implantee to discriminate complex sounds with different fundamental frequencies. Therefore, a new filter bank was constructed, for which the first harmonic is always resolved in two adjacent filters, and the balance between both filter outputs is directly related to the frequency of the first harmonic. The new filter bank was compared with a filter bank that is typically used in clinical processors, both with and without the presence of temporal cues in the stimuli. Four users of the LAURA cochlear implant participated in a pitch discrimination task to determine detection thresholds for F0 differences. The results show that these thresholds decrease noticeably for the new filter bank, if no temporal cues are present in the stimuli. If temporal cues are included, the differences between the results for both filter banks become smaller, but a clear advantage is still observed for the new filter bank. This demonstrates the feasibility of using place-coding for the fundamental frequency.

The perception of Cantonese lexical tones by early-deafened cochlear implantees

This study investigated whether cochlear implant users can identify Cantonese lexical tones, which differ primarily in their F0 pattern. Seventeen early-deafened deaf children (age= 4 years, 6 months to 8 years, 11 months; postoperative period= 11-41 months) took part in the study. Sixteen children were fitted with the Nucleus 24 cochlear implant system; one child was fitted with a Nucleus 22 implant. Participants completed a 2AFC picture identification task in which they identified one of the six contrastive Cantonese tones produced on the monosyllabic target word /ji/. Each target stimulus represented a concrete object and was presented within a carrier phrase in sentence-medial position. Group performance was significantly above chance for three contrasts. However, the cochlear implant listeners performed much worse than a 6 1/2-year-old, moderately hearing impaired control listener who was tested on the same task. These findings suggest that this group of cochlear implant users had great difficulty in extracting the pitch information needed to accurately identify Cantonese lexical tones.