Harmonic segregation through mistuning can improve fundamental frequency discrimination

Effect of diotic versus dichotic presentation on the pitch perception of tone complexes at medium and very high frequencies

Difference limens for fundamental frequency (F0), F0DLs, are usually small for complex tones containing low harmonics that are resolved in the auditory periphery, but worsen when the rank of the lowest harmonic increases above about 6-8 and harmonics become less resolved. The traditional explanation for this, in terms of resolvability, has been challenged and an alternative explanation in terms of harmonic rank was suggested. Here, to disentangle the effects of resolvability and harmonic rank the complex tones were presented either diotically (all harmonics to both ears) or dichotically (even and odd harmonics to opposite ears); the latter increases resolvability but does not affect harmonic rank. F0DLs were measured for 14 listeners for complex tones containing harmonics 6-10 with F0s of 280 and 1400 Hz, presented diotically or dichotically. For the low F0, F0DLs were significantly lower for the dichotic than for the diotic condition. This is consistent with a benefit of increased resolvability of harmonics for F0 discrimination and extends previous results to harmonics as low as the sixth. In contrast, for the high F0, F0DLs were similar for the two presentation modes, adding to evidence for differences in pitch perception between tones with low-to-medium and very-high frequency content.

On mistuning detection and beat perception for harmonic complex tones at low and very high frequencies

This study assessed the detection of mistuning of a single harmonic in complex tones (CTs) containing either low-frequency harmonics or very high-frequency harmonics, for which phase locking to the temporal fine structure is weak or absent. CTs had F0s of either 280 or 1400 Hz and contained harmonics 6-10, the 8th of which could be mistuned. Harmonics were presented either diotically or dichotically (odd and even harmonics to different ears). In the diotic condition, mistuning-detection thresholds were very low for both F0s and consistent with detection of temporal interactions (beats) produced by peripheral interactions of components. In the dichotic condition, for which the components in each ear were more widely spaced and beats were not reported, the mistuned component was perceptually segregated from the complex for the low F0, but subjects reported no "popping out" for the high F0 and performance was close to chance. This is consistent with the idea that phase locking is required for perceptual segregation to occur. For diotic presentation, the perceived beat rate corresponded to the amount of mistuning (in Hz). It is argued that the beat percept cannot be explained solely by interactions between the mistuned component and its two closest harmonic neighbours.

Harmonic Cancellation-A Fundamental of Auditory Scene Analysis

This paper reviews the hypothesis of harmonic cancellation according to which an interfering sound is suppressed or canceled on the basis of its harmonicity (or periodicity in the time domain) for the purpose of Auditory Scene Analysis. It defines the concept, discusses theoretical arguments in its favor, and reviews experimental results that support it, or not. If correct, the hypothesis may draw on time-domain processing of temporally accurate neural representations within the brainstem, as required also by the classic equalization-cancellation model of binaural unmasking. The hypothesis predicts that a target sound corrupted by interference will be easier to hear if the interference is harmonic than inharmonic, all else being equal. This prediction is borne out in a number of behavioral studies, but not all. The paper reviews those results, with the aim to understand the inconsistencies and come up with a reliable conclusion for, or against, the hypothesis of harmonic cancellation within the auditory system.

Effect of lowest harmonic rank on fundamental-frequency difference limens varies with fundamental frequency

This study investigated the relationship between fundamental frequency difference limens (F0DLs) and the lowest harmonic number present over a wide range of F0s (30-2000 Hz) for 12-component harmonic complex tones that were presented in either sine or random phase. For fundamental frequencies (F0s) between 100 and 400 Hz, a transition from low (∼1%) to high (∼5%) F0DLs occurred as the lowest harmonic number increased from about seven to ten, in line with earlier studies. At lower and higher F0s, the transition between low and high F0DLs occurred at lower harmonic numbers. The worsening performance at low F0s was reasonably well predicted by the expected decrease in spectral resolution below about 500 Hz. At higher F0s, the degradation in performance at lower harmonic numbers could not be predicted by changes in spectral resolution but remained relatively good (<2%-3%) in some conditions, even when all harmonics were above 8 kHz, confirming that F0 can be extracted from harmonics even when temporal envelope or fine-structure cues are weak or absent.

Pitch discrimination with mixtures of three concurrent harmonic complexes

In natural listening contexts, especially in music, it is common to hear three or more simultaneous pitches, but few empirical or theoretical studies have addressed how this is achieved. Place and pattern-recognition theories of pitch require at least some harmonics to be spectrally resolved for pitch to be extracted, but it is unclear how often such conditions exist when multiple complex tones are presented together. In three behavioral experiments, mixtures of three concurrent complexes were filtered into a single bandpass spectral region, and the relationship between the fundamental frequencies and spectral region was varied in order to manipulate the extent to which harmonics were resolved either before or after mixing. In experiment 1, listeners discriminated major from minor triads (a difference of 1 semitone in one note of the triad). In experiments 2 and 3, listeners compared the pitch of a probe tone with that of a subsequent target, embedded within two other tones. All three experiments demonstrated above-chance performance, even in conditions where the combinations of harmonic components were unlikely to be resolved after mixing, suggesting that fully resolved harmonics may not be necessary to extract the pitch from multiple simultaneous complexes.

Inharmonic speech reveals the role of harmonicity in the cocktail party problem

The "cocktail party problem" requires us to discern individual sound sources from mixtures of sources. The brain must use knowledge of natural sound regularities for this purpose. One much-discussed regularity is the tendency for frequencies to be harmonically related (integer multiples of a fundamental frequency). To test the role of harmonicity in real-world sound segregation, we developed speech analysis/synthesis tools to perturb the carrier frequencies of speech, disrupting harmonic frequency relations while maintaining the spectrotemporal envelope that determines phonemic content. We find that violations of harmonicity cause individual frequencies of speech to segregate from each other, impair the intelligibility of concurrent utterances despite leaving intelligibility of single utterances intact, and cause listeners to lose track of target talkers. However, additional segregation deficits result from replacing harmonic frequencies with noise (simulating whispering), suggesting additional grouping cues enabled by voiced speech excitation. Our results demonstrate acoustic grouping cues in real-world sound segregation.

Complex-Tone Pitch Discrimination in Listeners With Sensorineural Hearing Loss

Physiological studies have shown that noise-induced sensorineural hearing loss (SNHL) enhances the amplitude of envelope coding in auditory-nerve fibers. As pitch coding of unresolved complex tones is assumed to rely on temporal envelope coding mechanisms, this study investigated pitch-discrimination performance in listeners with SNHL. Pitch-discrimination thresholds were obtained for 14 normal-hearing (NH) and 10 hearing-impaired (HI) listeners for sine-phase (SP) and random-phase (RP) complex tones. When all harmonics were unresolved, the HI listeners performed, on average, worse than NH listeners in the RP condition but similarly to NH listeners in the SP condition. The increase in pitch-discrimination performance for the SP relative to the RP condition (F₀DL ratio) was significantly larger in the HI as compared with the NH listeners. Cochlear compression and auditory-filter bandwidths were estimated in the same listeners. The estimated reduction of cochlear compression was significantly correlated with the increase in the F₀DL ratio, while no correlation was found with filter bandwidth. The effects of degraded frequency selectivity and loss of compression were considered in a simplified peripheral model as potential factors in envelope enhancement. The model revealed that reducing cochlear compression significantly enhanced the envelope of an unresolved SP complex tone, while not affecting the envelope of a RP complex tone. This envelope enhancement in the SP condition was significantly correlated with the increased pitch-discrimination performance for the SP relative to the RP condition in the HI listeners.

A function for binaural integration in auditory grouping and segregation in the inferior colliculus

Responses of neurons to binaural, harmonic complex stimuli in urethane-anesthetized guinea pig inferior colliculus (IC) are reported. To assess the binaural integration of harmonicity cues for sound segregation and grouping, responses were measured to harmonic complexes with different fundamental frequencies presented to each ear. Simultaneously gated harmonic stimuli with fundamental frequencies of 125 Hz and 145 Hz were presented to the left and right ears, respectively, and recordings made from 96 neurons with characteristic frequencies >2 kHz in the central nucleus of the IC. Of these units, 70 responded continuously throughout the stimulus and were excited by the stimulus at the contralateral ear. The stimulus at the ipsilateral ear excited (EE: 14%; 10/70), inhibited (EI: 33%; 23/70), or had no significant effect (EO: 53%; 37/70), defined by the effect on firing rate. The neurons phase locked to the temporal envelope at each ear to varying degrees depending on signal level. Many of the cells (predominantly EO) were dominated by the response to the contralateral stimulus. Another group (predominantly EI) synchronized to the contralateral stimulus and were suppressed by the ipsilateral stimulus in a phasic manner. A third group synchronized to the stimuli at both ears (predominantly EE). Finally, a group only responded when the waveform peaks from each ear coincided. We conclude that these groups of neurons represent different "streams" of information but exhibit modifications of the response rather than encoding a feature of the stimulus, like pitch.

Pitch perception of concurrent harmonic tones with overlapping spectra

Fundamental frequency difference limens (F0DLs) were measured for a target harmonic complex tone with nominal fundamental frequency (F0) of 200 Hz, in the presence and absence of a harmonic masker with overlapping spectrum. The F0 of the masker was 0, ± 3, or ± 6 semitones relative to 200 Hz. The stimuli were bandpass filtered into three regions: 0-1000 Hz (low, L), 1600-2400 Hz (medium, M), and 2800-3600 Hz (high, H), and a background noise was used to mask combination tones and to limit the audibility of components falling on the filter skirts. The components of the target or masker started either in cosine or random phase. Generally, the effect of F0 difference between target and masker was small. For the target alone, F0DLs were larger for random than cosine phase for region H. For the target plus masker, F0DLs were larger when the target had random phase than cosine phase for regions M and H. F0DLs increased with increasing center frequency of the bandpass filter. Modeling using excitation patterns and "summary autocorrelation" and "stabilized auditory image" models suggested that use of temporal fine structure information can account for the small F0DLs obtained when harmonics are barely, if at all, resolved.

Effects of pulsing of a target tone on the ability to hear it out in different types of complex sounds

Judgments of whether a sinusoidal probe is higher or lower in frequency than the closest partial ("target") in a multi-partial complex are improved when the target is pulsed on and off. These experiments explored the contribution of reduction in perceptual confusion and recovery from adaptation to this effect. In experiment 1, all partials except the target were replaced by noise to reduce perceptual confusion. Performance was much better than when the background was composed of multiple partials. When the level of the target was reduced to avoid ceiling effects, no effect of pulsing the target occurred. In experiment 2, the target and background partials were irregularly and independently amplitude modulated. This gave a large effect of pulsing the target, suggesting that if recovery from adaptation contributes to the effect, amplitude fluctuations do not prevent this. In experiment 3, the background was composed of multiple steady partials, but the target was irregularly amplitude modulated. This gave better performance than when the target was unmodulated and a moderate effect of pulsing the target. It is argued that when the target and background are steady tones, pulsing the target may result both in reduction of perceptual confusion and recovery from adaptation.

Spectral processing of two concurrent harmonic complexes

In a concurrent profile analysis task, each of the two observation intervals was the sum of two harmonic complexes. In the first interval one of the harmonic complexes had a flat spectrum and the other had a broad spectral peak at 1 kHz. In the second interval, the association between the spectral profiles and the complexes was either consistent with the first interval, or inconsistent so that profile changes (flat versus peaked) could be created in both of the complexes. In two experiments, thresholds and psychometric functions for detecting the profile change were measured in terms of the spectral peak's magnitude as functions of three types of segregation cues: Difference in fundamental frequency, onset asynchrony, and difference in interaural time difference between the two complexes. Decreasing the magnitude of each cue led to higher thresholds, and shallower psychometric functions whose upper asymptotes often failed to reach 100% correct. The patterns of the threshold and psychometric functions varied across cue types and across individual listeners. The results suggest that informational masking is present in the concurrent profile analysis task. Segregation cues appear to contribute to the release from informational masking, but the process depends on listening strategies adopted by individual listeners.

Resolvability of components in complex tones and implications for theories of pitch perception

This paper reviews methods that have been used to estimate the resolvability of individual partials in harmonic and inharmonic complex tones and considers the implications of the results for theories of pitch perception. The methods include: requiring comparisons of the pitch of an isolated pure tone and a partial within a complex tone as a measure of the ability to "hear out" that partial; considering the magnitude of ripples in the calculated excitation pattern of a complex tone; using a complex tone as a forward masker and using ripples in the masking pattern to estimate resolvability; measuring sensitivity to the relative phase of the components within complex tones. The measures are broadly consistent in indicating that harmonics with numbers up to about five are well resolved, but that resolution decreases for higher harmonics. Most measures suggest that harmonics with numbers above eight are poorly, if at all, resolved. However, there are uncertainties associated with each method that make the exact upper limit of resolvability uncertain. Evidence is presented suggesting a partial dissociation between resolution in the excitation pattern and the ability to hear out a partial. It is proposed that the latter requires information from temporal fine structure (phase locking).

Auditory stream segregation and the perception of across-frequency synchrony

This study explored the extent to which sequential auditory grouping affects the perception of temporal synchrony. In Experiment 1, listeners discriminated between 2 pairs of asynchronous "target" tones at different frequencies, A and B, in which the B tone either led or lagged. Thresholds were markedly higher when the target tones were temporally surrounded by "captor tones" at the A frequency than when the captor tones were absent or at a remote frequency. Experiment 2 extended these findings to asynchrony detection, revealing that the perception of synchrony, one of the most potent cues for simultaneous auditory grouping, is not immune to competing effects of sequential grouping. Experiment 3 examined the influence of ear separation on the interactions between sequential and simultaneous grouping cues. The results showed that, although ear separation could facilitate perceptual segregation and impair asynchrony detection, it did not prevent the perceptual integration of simultaneous sounds.

The role of peripheral resolvability in pitch-sequence processing

The authors previously reported that same/different judgments on pitch sequences were more accurate for tones with resolved (low-rank) harmonics compared to unresolved (high-rank) harmonics, even when discriminability between tones was equated [Cousineau et al. (2009). J. Acoust. Soc. Am. 126, 3179-3187]. Here, peripheral resolvability, defined by the number of harmonics per cochlear filter, was contrasted with harmonic number. Tones were presented either diotically or dichotically. In the latter case, even and odd harmonics were presented to different ears, thus halving the number of harmonics per cochlear filter. Performance was better for dichotic than for diotic presentations. This indicates that peripheral resolvability is necessary and sufficient for efficient pitch-sequence processing.

Musical intervals and relative pitch: frequency resolution, not interval resolution, is special

Pitch intervals are central to most musical systems, which utilize pitch at the expense of other acoustic dimensions. It seemed plausible that pitch might uniquely permit precise perception of the interval separating two sounds, as this could help explain its importance in music. To explore this notion, a simple discrimination task was used to measure the precision of interval perception for the auditory dimensions of pitch, brightness, and loudness. Interval thresholds were then expressed in units of just-noticeable differences for each dimension, to enable comparison across dimensions. Contrary to expectation, when expressed in these common units, interval acuity was actually worse for pitch than for loudness or brightness. This likely indicates that the perceptual dimension of pitch is unusual not for interval perception per se, but rather for the basic frequency resolution it supports. The ubiquity of pitch in music may be due in part to this fine-grained basic resolution.

Does fundamental-frequency discrimination measure virtual pitch discrimination?

Studies of pitch perception often involve measuring difference limens for complex tones (DLCs) that differ in fundamental frequency (F0). These measures are thought to reflect F0 discrimination and to provide an indirect measure of subjective pitch strength. However, in many situations discrimination may be based on cues other than the pitch or the F0, such as differences in the frequencies of individual components or timbre (brightness). Here, DLCs were measured for harmonic and inharmonic tones under various conditions, including a randomized or fixed lowest harmonic number, with and without feedback. The inharmonic tones were produced by shifting the frequencies of all harmonics upwards by 6.25%, 12.5%, or 25% of F0. It was hypothesized that, if DLCs reflect residue-pitch discrimination, these frequency-shifted tones, which produced a weaker and more ambiguous pitch than would yield larger DLCs than the harmonic tones. However, if DLCs reflect comparisons of component pitches, or timbre, they should not be systematically influenced by frequency shifting. The results showed larger DLCs and more scattered pitch matches for inharmonic than for harmonic complexes, confirming that the inharmonic tones produced a less consistent pitch than the harmonic tones, and consistent with the idea that DLCs reflect F0 pitch discrimination.

Pitch perception for mixtures of spectrally overlapping harmonic complex tones

This study measured difference limens for fundamental frequency (DLF0s) for a target harmonic complex in the presence of a simultaneous spectrally overlapping harmonic masker. The resolvability of the target harmonics was manipulated by bandpass filtering the stimuli into a low (800-2400 Hz) or high (1600-3200 Hz) spectral region, using different nominal F0s for the targets (100, 200, and 400 Hz), and different masker F0s (0, +9, or -9 semitones) relative to the target. Three different modes of masker presentation, relative to the target, were tested: ipsilateral, contralateral, and dichotic, with a higher masker level in the contralateral ear. Ipsilateral and dichotic maskers generally caused marked elevations in DLF0s compared to both the unmasked and contralateral masker conditions. Analyses based on excitation patterns revealed that ipsilaterally masked F0 difference limens were small (<2%) only when the excitation patterns evoked by the target-plus-masker mixture contained several salient (>1 dB) peaks at or close to target harmonic frequencies, even though these peaks were rarely produced by the target alone. The findings are discussed in terms of place- or place-time mechanisms of pitch perception.

The role of temporal fine structure in harmonic segregation through mistuning

Bernstein and Oxenham [(2008). J. Acoust. Soc. Am. 124, 1653-1667] measured thresholds for discriminating the fundamental frequency, F0, of a complex tone that was passed through a fixed bandpass filter. They found that performance worsened when the F0 was decreased so that only harmonics above the tenth were audible. However, performance in this case was improved by mistuning the odd harmonics by 3%. Bernstein and Oxenham considered whether the results could be explained in terms of temporal fine structure information available at the output of a single auditory filter and concluded that their results did not appear to be consistent with such an explanation. Here, it is argued that such cues could have led to the improvement in performance produced by mistuning the odd harmonics.

Pitch, harmonicity and concurrent sound segregation: psychoacoustical and neurophysiological findings

Harmonic complex tones are a particularly important class of sounds found in both speech and music. Although these sounds contain multiple frequency components, they are usually perceived as a coherent whole, with a pitch corresponding to the fundamental frequency (F0). However, when two or more harmonic sounds occur concurrently, e.g., at a cocktail party or in a symphony, the auditory system must separate harmonics and assign them to their respective F0s so that a coherent and veridical representation of the different sounds sources is formed. Here we review both psychophysical and neurophysiological (single-unit and evoked-potential) findings, which provide some insight into how, and how well, the auditory system accomplishes this task. A survey of computational models designed to estimate multiple F0s and segregate concurrent sources is followed by a review of the empirical literature on the perception and neural coding of concurrent harmonic sounds, including vowels, as well as findings obtained using single complex tones with mistuned harmonics.

Effects of pulsing of the target tone on the audibility of partials in inharmonic complex tones

The audibility of partials was measured for complex tones with partials uniformly spaced on an ERB(N)-number scale. On each trial, subjects heard a sinusoidal "probe" followed by a complex tone. The probe was mistuned downwards or upwards (at random) by 3% or 4.5% from the frequency of one randomly selected partial in the complex (the "target"). The subject indicated whether the target was higher or lower in frequency than the probe. The probe and the target were pulsed on and off and the ramp times and inter-pulse intervals were systematically varied. Performance was better for longer ramp times and longer inter-pulse intervals. In a second experiment, the ability to detect which of two complex tones contained a pulsed partial was measured. The pattern of results was similar to that for experiment 1. A model of auditory processing including an adaptation stage was able to account for the general pattern of the results of experiment 2. The results suggest that the improvement in ability to hear out a partial in a complex tone produced by pulsing that partial is partly mediated by a release from adaptation produced by the pulsing, and does not result solely from reduction of perceptual confusion.

Can temporal fine structure represent the fundamental frequency of unresolved harmonics?

At least two modes of pitch perception exist: in one, the fundamental frequency (F0) of harmonic complex tones is estimated using the temporal fine structure (TFS) of individual low-order resolved harmonics; in the other, F0 is derived from the temporal envelope of high-order unresolved harmonics that interact in the auditory periphery. Pitch is typically more accurate in the former than in the latter mode. Another possibility is that pitch can sometimes be coded via the TFS from unresolved harmonics. A recent study supporting this third possibility [Moore et al. (2006a). J. Acoust. Soc. Am. 119, 480-490] based its conclusion on a condition where phase interaction effects (implying unresolved harmonics) accompanied accurate F0 discrimination (implying TFS processing). The present study tests whether these results were influenced by audible distortion products. Experiment 1 replicated the original results, obtained using a low-level background noise. However, experiments 2-4 found no evidence for the use of TFS cues with unresolved harmonics when the background noise level was raised, or the stimulus level was lowered, to render distortion inaudible. Experiment 5 measured the presence and phase dependence of audible distortion products. The results provide no evidence that TFS cues are used to code the F0 of unresolved harmonics.