Effects of spectral modulation filtering on vowel identification

Feasibility exploration of cortical auditory evoked potentials evoked by spectro-temporal modulation signals

BACKGROUND

An objective measurement of speech perception would be valuable in hearing-impaired patients who are unable to perform auditory tasks reliably.

OBJECTIVES

To investigate the feasibility of cortical auditory evoked potentials (CAEPs) evoked by different spectro-temporal modulation (STM) signals and provide reference for the further exploration of acoustic change complex (ACC) in hearing-impaired patients.

METHOD

29 normal hearing (NH) adults were recruited and stimulated randomly by STM signals at 6 spectral modulation rates: 0, 1, 2, 4, 8, 16 cycles/octave, at each of 4 temporal modulation rates: 0, 2, 4, 8 Hz, to elicit ACC response.

RESULTS

The change of spectral modulation rates ≤ 16 cycles/octave at 0 Hz temporal modulation rate had different impacts on latency. 0, 1 cycles/octave, and 4, 8 Hz made a difference to the P2-N1 amplitude easily. 8, 16 cycles/octave were more unstable in rejection rate. No significant difference existed in waveform repetition rate among different STM rates. No significant interaction effect existed in spectral and temporal modulation.

CONCLUSIONS

STM as a stimulus signal to induce ACC response had its feasibility. 2 cycles/octave at spectral modulation rate and 2 Hz at temporal modulation rate had better stability and waveform repeatability in NH adults.

Correlates of vowel clarity in the spectrotemporal modulation domain: Application to speech impairment evaluation

This article reports on vowel clarity metrics based on spectrotemporal modulations of speech signals. Motivated by previous findings on the relevance of modulation-based metrics for speech intelligibility assessment and pathology classification, the current study used factor analysis to identify regions within a bi-dimensional modulation space, the magnitude power spectrum, as in Elliott and Theunissen [(2009). PLoS Comput. Biol. 5(3), e1000302] by relating them to a set of conventional acoustic metrics of vowel space area and vowel distinctiveness. Two indices based on the energy ratio between high and low modulation rates across temporal and spectral dimensions of the modulation space emerged from the analyses. These indices served as input for measurements of central tendency and classification analyses that aimed to identify vowel-related speech impairments in French native speakers with head and neck cancer (HNC) and Parkinson dysarthria (PD). Following the analysis, vowel-related speech impairment was identified in HNC speakers, but not in PD. These results were consistent with findings based on subjective evaluations of speech intelligibility. The findings reported are consistent with previous studies indicating that impaired speech is associated with attenuation in energy in higher spectrotemporal modulation bands.

Distortion of Spectral Ripples Through Cochlear Implants Has Major Implications for Interpreting Performance Scores

The spectral ripple discrimination task is a psychophysical measure that has been found to correlate with speech recognition in listeners with cochlear implants (CIs). However, at ripple densities above a critical value (around 2 RPO, but device-specific), the sparse spectral sampling of CI processors results in stimulus distortions resulting in aliasing and unintended changes in modulation depth. As a result, spectral ripple thresholds above a certain number are not ordered monotonically along the RPO dimension and thus cannot be considered better or worse spectral resolution than each other, thus undermining correlation measurements. These stimulus distortions are not remediated by changing stimulus phase, indicating these issues cannot be solved by spectrotemporally modulated stimuli. Speech generally has very low-density spectral modulations, leading to questions about the mechanism of correlation between high ripple thresholds and speech recognition. Existing data showing correlations between ripple discrimination and speech recognition include many observations above the aliasing limit. These scores should be treated with caution, and experimenters could benefit by prospectively considering the limitations of the spectral ripple test.

Vision perceptually restores auditory spectral dynamics in speech

Visual speech facilitates auditory speech perception, but the visual cues responsible for these benefits and the information they provide remain unclear. Low-level models emphasize basic temporal cues provided by mouth movements, but these impoverished signals may not fully account for the richness of auditory information provided by visual speech. High-level models posit interactions among abstract categorical (i.e., phonemes/visemes) or amodal (e.g., articulatory) speech representations, but require lossy remapping of speech signals onto abstracted representations. Because visible articulators shape the spectral content of speech, we hypothesized that the perceptual system might exploit natural correlations between midlevel visual (oral deformations) and auditory speech features (frequency modulations) to extract detailed spectrotemporal information from visual speech without employing high-level abstractions. Consistent with this hypothesis, we found that the time-frequency dynamics of oral resonances (formants) could be predicted with unexpectedly high precision from the changing shape of the mouth during speech. When isolated from other speech cues, speech-based shape deformations improved perceptual sensitivity for corresponding frequency modulations, suggesting that listeners could exploit this cross-modal correspondence to facilitate perception. To test whether this type of correspondence could improve speech comprehension, we selectively degraded the spectral or temporal dimensions of auditory sentence spectrograms to assess how well visual speech facilitated comprehension under each degradation condition. Visual speech produced drastically larger enhancements during spectral degradation, suggesting a condition-specific facilitation effect driven by cross-modal recovery of auditory speech spectra. The perceptual system may therefore use audiovisual correlations rooted in oral acoustics to extract detailed spectrotemporal information from visual speech.

Psychometric Functions of Vowel Detection and Identification in Long-Term Speech-Shaped Noise

Purpose The goal of this study was to investigate vowel detection and identification in noise and provide baseline data regarding how vowel perception changed with signal-to-noise ratios. Psychometric functions of vowel detection and identification for 12 American English isolated vowels in long-term speech-shaped noise were examined for young listeners with normal hearing in this study. Method Vowel detection was measured at sensation levels from -10 to +5 dB (re: thresholds of vowel detection from the study of Liu and Eddins, 2008a ) with a 4-interval forced-choice procedure. Thresholds of vowel detection were computed for each listener as the speech level at which 70.7% correct performance was reached. Vowel identification was then examined at sensation levels from 0 to 12 dB relative to detection thresholds for each listener. Thresholds of vowel identification were calculated as the speech level with vowel identifiability ( d') equals to 1. Results Thresholds of vowel detection and identification were significantly affected by vowel category. Slopes of psychometric functions of vowel identification were significantly dependent on vowel category, whereas slopes of psychometric functions of vowel detection were not. Conclusions These results suggest that, given the same sensation levels, especially at low sensation levels, vowel sounds are not equally perceivable in terms of identifiability.

How Do Age and Hearing Loss Impact Spectral Envelope Perception?

Purpose

The goal was to evaluate the potential effects of increasing hearing loss and advancing age on spectral envelope perception.

Method

Spectral modulation detection was measured as a function of spectral modulation frequency from 0.5 to 8.0 cycles/octave. The spectral modulation task involved discrimination of a noise carrier (3 octaves wide from 400 to 3200 Hz) with a flat spectral envelope from a noise having a sinusoidal spectral envelope across a logarithmic audio frequency scale. Spectral modulation transfer functions (SMTFs; modulation threshold vs. modulation frequency) were computed and compared 4 listener groups: young normal hearing, older normal hearing, older with mild hearing loss, and older with moderate hearing loss. Estimates of the internal spectral contrast were obtained by computing excitation patterns.

Results

SMTFs for young listeners with normal hearing were bandpass with a minimum modulation detection threshold at 2 cycles/octave, and older listeners with normal hearing were remarkably similar to those of the young listeners. SMTFs for older listeners with mild and moderate hearing loss had a low-pass rather than a bandpass shape. Excitation patterns revealed that limited spectral resolution dictated modulation detection thresholds at high but not low spectral modulation frequencies. Even when factoring out (presumed) differences in frequency resolution among groups, the spectral envelope perception was worse for the group with moderate hearing loss than the other 3 groups.

Conclusions

The spectral envelope perception as measured by spectral modulation detection thresholds is compromised by hearing loss at higher spectral modulation frequencies, consistent with predictions of reduced spectral resolution known to accompany sensorineural hearing loss. Spectral envelope perception is not negatively impacted by advancing age at any spectral modulation frequency between 0.5 and 8.0 cycles/octave.

Auditory sensitivity to spectral modulation phase reversal as a function of modulation depth

The present study evaluated auditory sensitivity to spectral modulation by determining the modulation depth required to detect modulation phase reversal. This approach may be preferable to spectral modulation detection with a spectrally flat standard, since listeners appear unable to perform the task based on the detection of temporal modulation. While phase reversal thresholds are often evaluated by holding modulation depth constant and adjusting modulation rate, holding rate constant and adjusting modulation depth supports rate-specific assessment of modulation processing. Stimuli were pink noise samples, filtered into seven octave-wide bands (0.125–8 kHz) and spectrally modulated in dB. Experiment 1 measured performance as a function of modulation depth to determine appropriate units for adaptive threshold estimation. Experiment 2 compared thresholds in dB for modulation detection with a flat standard and modulation phase reversal; results supported the idea that temporal cues were available at high rates for the former but not the latter. Experiment 3 evaluated spectral modulation phase reversal thresholds for modulation that was restricted to either one or two neighboring bands. Flanking bands of unmodulated noise had a larger detrimental effect on one-band than two-band targets. Thresholds for high-rate modulation improved with increasing carrier frequency up to 2 kHz, whereas low-rate modulation appeared more consistent across frequency, particularly in the two-band condition. Experiment 4 measured spectral weights for spectral modulation phase reversal detection and found higher weights for bands in the spectral center of the stimulus than for the lowest (0.125 kHz) or highest (8 kHz) band. Experiment 5 compared performance for highly practiced and relatively naïve listeners, and found weak evidence of a larger practice effect at high than low spectral modulation rates. These results provide preliminary data for a task that may provide a better estimate of sensitivity to spectral modulation than spectral modulation detection with a flat standard.

Distribution of spectral modulation transfer functions in a young, normal-hearing population

Spectral modulation transfer functions (SMTFs) were measured in 49 young (18-35 years of age) normal-hearing listeners. Noise carriers spanned six octaves from 200 to 12 800 Hz. Sinusoidal (on a log-amplitude scale) spectral modulation with random starting phase was superimposed on the carrier at spectral modulation frequencies of 0.25, 0.5, 1.0, 2.0, 4.0, and 8.0 cycles/octave. Modulation detection thresholds (in dB) yielded SMTFs that were bandpass in nature, consistent with previous investigations reporting data for only a few subjects. Thresholds were notably consistent across subjects despite minimal practice. Population statistics are reported that may serve as reference data for future studies.

Spectro-temporal cues enhance modulation sensitivity in cochlear implant users

Although speech understanding is highly variable amongst cochlear implants (CIs) subjects, the remarkably high speech recognition performance of many CI users is unexpected and not well understood. Numerous factors, including neural health and degradation of the spectral information in the speech signal of CIs, likely contribute to speech understanding. We studied the ability to use spectro-temporal modulations, which may be critical for speech understanding and discrimination, and hypothesize that CI users adopt a different perceptual strategy than normal-hearing (NH) individuals, whereby they rely more heavily on joint spectro-temporal cues to enhance detection of auditory cues. Modulation detection sensitivity was studied in CI users and NH subjects using broadband "ripple" stimuli that were modulated spectrally, temporally, or jointly, i.e., spectro-temporally. The spectro-temporal modulation transfer functions of CI users and NH subjects was decomposed into spectral and temporal dimensions and compared to those subjects' spectral-only and temporal-only modulation transfer functions. In CI users, the joint spectro-temporal sensitivity was better than that predicted by spectral-only and temporal-only sensitivity, indicating a heightened spectro-temporal sensitivity. Such an enhancement through the combined integration of spectral and temporal cues was not observed in NH subjects. The unique use of spectro-temporal cues by CI patients can yield benefits for use of cues that are important for speech understanding. This finding has implications for developing sound processing strategies that may rely on joint spectro-temporal modulations to improve speech comprehension of CI users, and the findings of this study may be valuable for developing clinical assessment tools to optimize CI processor performance.

Spectrotemporal modulation sensitivity as a predictor of speech intelligibility for hearing-impaired listeners

BACKGROUND

A model that can accurately predict speech intelligibility for a given hearing-impaired (HI) listener would be an important tool for hearing-aid fitting or hearing-aid algorithm development. Existing speech-intelligibility models do not incorporate variability in suprathreshold deficits that are not well predicted by classical audiometric measures. One possible approach to the incorporation of such deficits is to base intelligibility predictions on sensitivity to simultaneously spectrally and temporally modulated signals.

PURPOSE

The likelihood of success of this approach was evaluated by comparing estimates of spectrotemporal modulation (STM) sensitivity to speech intelligibility and to psychoacoustic estimates of frequency selectivity and temporal fine-structure (TFS) sensitivity across a group of HI listeners.

RESEARCH DESIGN

The minimum modulation depth required to detect STM applied to an 86 dB SPL four-octave noise carrier was measured for combinations of temporal modulation rate (4, 12, or 32 Hz) and spectral modulation density (0.5, 1, 2, or 4 cycles/octave). STM sensitivity estimates for individual HI listeners were compared to estimates of frequency selectivity (measured using the notched-noise method at 500, 1000, 2000, and 4000 Hz), TFS processing ability (2 Hz frequency-modulation detection thresholds for 500, 1000, 2000, and 4000 Hz carriers) and sentence intelligibility in noise (at a 0 dB signal-to-noise ratio) that were measured for the same listeners in a separate study.

STUDY SAMPLE

Eight normal-hearing (NH) listeners and 12 listeners with a diagnosis of bilateral sensorineural hearing loss participated.

DATA COLLECTION AND ANALYSIS

STM sensitivity was compared between NH and HI listener groups using a repeated-measures analysis of variance. A stepwise regression analysis compared STM sensitivity for individual HI listeners to audiometric thresholds, age, and measures of frequency selectivity and TFS processing ability. A second stepwise regression analysis compared speech intelligibility to STM sensitivity and the audiogram-based Speech Intelligibility Index.

RESULTS

STM detection thresholds were elevated for the HI listeners, but only for low rates and high densities. STM sensitivity for individual HI listeners was well predicted by a combination of estimates of frequency selectivity at 4000 Hz and TFS sensitivity at 500 Hz but was unrelated to audiometric thresholds. STM sensitivity accounted for an additional 40% of the variance in speech intelligibility beyond the 40% accounted for by the audibility-based Speech Intelligibility Index.

CONCLUSIONS

Impaired STM sensitivity likely results from a combination of a reduced ability to resolve spectral peaks and a reduced ability to use TFS information to follow spectral-peak movements. Combining STM sensitivity estimates with audiometric threshold measures for individual HI listeners provided a more accurate prediction of speech intelligibility than audiometric measures alone. These results suggest a significant likelihood of success for an STM-based model of speech intelligibility for HI listeners.

Effects of Signal Level and Spectral Contrast on Vowel Formant Discrimination for Normal-Hearing and Hearing-Impaired Listeners

Purpose

The aim of this study was to determine whether increasing the overall speech level or the individual spectral contrasts of vowel sounds can improve vowel formant discrimination for listeners both with and without normal hearing.

Method

Thresholds of vowel formant discrimination were examined for the F2 frequencies of 3 American English vowels for listeners with and without normal hearing. Spectral contrasts of the F2 were enhanced by 3, 6, and 9 dB. Vowel stimuli were presented at 70 and 90 dB SPL.

Results

The thresholds of listeners with hearing impairment were reduced significantly after spectral enhancement was implemented, especially at 90 dB SPL, whereas normal-hearing listeners did not benefit from spectral enhancement.

Conclusion

These results indicate that a combination of spectral enhancement of F2 and high speech level is most beneficial to improve vowel formant discrimination for listeners with hearing impairment.

Assessing the role of spectral and intensity cues in spectral ripple detection and discrimination in cochlear-implant users

Measures of spectral ripple resolution have become widely used psychophysical tools for assessing spectral resolution in cochlear-implant (CI) listeners. The objective of this study was to compare spectral ripple discrimination and detection in the same group of CI listeners. Ripple detection thresholds were measured over a range of ripple frequencies and were compared to spectral ripple discrimination thresholds previously obtained from the same CI listeners. The data showed that performance on the two measures was correlated, but that individual subjects' thresholds (at a constant spectral modulation depth) for the two tasks were not equivalent. In addition, spectral ripple detection was often found to be possible at higher rates than expected based on the available spectral cues, making it likely that temporal-envelope cues played a role at higher ripple rates. Finally, spectral ripple detection thresholds were compared to previously obtained speech-perception measures. Results confirmed earlier reports of a robust relationship between detection of widely spaced ripples and measures of speech recognition. In contrast, intensity difference limens for broadband noise did not correlate with spectral ripple detection measures, suggesting a dissociation between the ability to detect small changes in intensity across frequency and across time.

Effects of roving level and spectral range on vowel formant discrimination

Thresholds of vowel formant discrimination for F1 and F2 of isolated vowels with full and partial vowel spectra were measured for normal-hearing listeners at fixed and roving speech levels. Performance of formant discrimination was significantly better for fixed levels than for roving levels with both full and partial spectra. The effect of vowel spectral range was present only for roving levels, but not for fixed levels. These results, consistent with studies of profile analysis, indicated different perceptual mechanisms for listeners to discriminate vowel formant frequency at fixed and roving levels.