Speech perception with music maskers by cochlear implant users and normal-hearing listeners

Familiarity of Background Music Modulates the Cortical Tracking of Target Speech at the "Cocktail Party"

The "cocktail party" problem-how a listener perceives speech in noisy environments-is typically studied using speech (multi-talker babble) or noise maskers. However, realistic cocktail party scenarios often include background music (e.g., coffee shops, concerts). Studies investigating music's effects on concurrent speech perception have predominantly used highly controlled synthetic music or shaped noise, which do not reflect naturalistic listening environments. Behaviorally, familiar background music and songs with vocals/lyrics inhibit concurrent speech recognition. Here, we investigated the neural bases of these effects. While recording multichannel EEG, participants listened to an audiobook while popular songs (or silence) played in the background at a 0 dB signal-to-noise ratio. Songs were either familiar or unfamiliar to listeners and featured either vocals or isolated instrumentals from the original audio recordings. Comprehension questions probed task engagement. We used temporal response functions (TRFs) to isolate cortical tracking to the target speech envelope and analyzed neural responses around 100 ms (i.e., auditory N1 wave). We found that speech comprehension was, expectedly, impaired during background music compared to silence. Target speech tracking was further hindered by the presence of vocals. When masked by familiar music, response latencies to speech were less susceptible to informational masking, suggesting concurrent neural tracking of speech was easier during music known to the listener. These differential effects of music familiarity were further exacerbated in listeners with less musical ability. Our neuroimaging results and their dependence on listening skills are consistent with early attentional-gain mechanisms where familiar music is easier to tune out (listeners already know the song's expectancies) and thus can allocate fewer attentional resources to the background music to better monitor concurrent speech material.

Song Properties and Familiarity Affect Speech Recognition in Musical Noise

"Cocktail party" speech perception is largely studied using either linguistic or nonspeech noise maskers. Few studies have addressed how listeners understand speech during concurrent music. We used popular songs to probe the effects of familiarity and different inherent properties of background music (i.e., isolated vocals, isolated instruments, or unprocessed song) on speech recognition. Participants performed an open-set sentence recognition task in the presence of familiar and unfamiliar music maskers (-5 dB signal-to-noise ratio [SNR]) composed of the full unprocessed song, only the instrumentals, or only the vocals. We found that full songs negatively affected recognition performance more so than isolated vocals and instrumentals. Surprisingly, there was also an interaction with music familiarity; well-known music impaired performance in the homologous full song and instrumental conditions. Our results show strong effects of song component and familiarity on speech recognition ability, highlighting interactions between both physical and psychological characteristics of musical noise on task performance. Familiarity impairs speech perception when background music features the instrumentals with or without the vocals. Our findings have implications for understanding the possible facilitation (or interference) of background music during concurrent linguistic tasks including academic study in attempts to promote learning.

Music perception and speech intelligibility in noise performance by Italian-speaking cochlear implant users

Objective

The goal of this study was to investigate the performance correlations between music perception and speech intelligibility in noise by Italian-speaking cochlear implant (CI) users.

Materials and methods

Twenty postlingually deafened adults with unilateral CIs (mean age 65 years, range 46–92 years) were tested with a music quality questionnaire using three passages of music from Classical Music, Jazz, and Soul. Speech recognition in noise was assessed using two newly developed adaptive tests in Italian: The Sentence Test with Adaptive Randomized Roving levels (STARR) and Matrix tests.

Results

Median quality ratings for Classical, Jazz and Soul music were 63%, 58% and 58%, respectively. Median SRTs for the STARR and Matrix tests were 14.3 dB and 7.6 dB, respectively. STARR performance was significantly correlated with Classical music ratings (r_s = − 0.49, p = 0.029), whereas Matrix performance was significantly correlated with both Classical (r_s = − 0.48, p = 0.031) and Jazz music ratings (r_s = − 0.56, p = 0.011).

Conclusion

Speech with competitive noise and music are naturally present in everyday listening environments. Recent speech perception tests based on an adaptive paradigm and sentence materials in relation with music quality measures might be representative of everyday performance in CI users. The present data contribute to cross-language studies and suggest that improving music perception in CI users may yield everyday benefit in speech perception in noise and may hence enhance the quality of listening for CI users.

Self-reported music perception is related to quality of life and self-reported hearing abilities in cochlear implant users

OBJECTIVES

To investigate the relationship between self-reported music perception and appreciation and (1) quality of life (QoL), and (2) self-assessed hearing ability in 98 post-lingually deafened cochlear implant (CI) users with a wide age range.

METHODS

Participants filled three questionnaires: (1) the Dutch Musical Background Questionnaire (DMBQ), which measures the music listening habits, the quality of the sound of music and the self-assessed perception of elements of music; (2) the Nijmegen Cochlear Implant Questionnaire (NCIQ), which measures health-related QoL; (3) the Speech, Spatial and Qualities (SSQ) of hearing scale, which measures self-assessed hearing ability. Additionally, speech perception was behaviorally measured with a phoneme-in-word identification.

RESULTS

A decline in music listening habits and a low rating of the quality of music after implantation are reported in DMBQ. A significant relationship is found between the music measures and the NCIQ and SSQ; no significant relationships are observed between the DMBQ and speech perception scores.

CONCLUSIONS

The findings suggest some relationship between CI users' self-reported music perception ability and QoL and self-reported hearing ability. While the causal relationship is not currently evaluated, the findings may imply that music training programs and/or device improvements that improve music perception may improve QoL and hearing ability.

Comparing Methods for Pairing Electrodes Across Ears With Cochlear Implants

OBJECTIVES

Currently, bilateral cochlear implants (CIs) are independently programmed in clinics using frequency allocations based on the relative location of a given electrode from the end of each electrode array. By pairing electrodes based on this method, bilateral CI recipients may have decreased sensitivity to interaural time differences (ITD) and/or interaural level differences (ILD), two cues critical for binaural tasks. There are multiple different binaural measures that can potentially be used to determine the optimal way to pair electrodes across the ears. Previous studies suggest that the optimal electrode pairing between the left and right ears may vary depending on the binaural task used. These studies, however, have only used one reference location or a single bilateral CI user. In both instances, it is difficult to determine if the results that were obtained reflect a measurement error or a systematic difference across binaural tasks. It is also difficult to determine from these studies if the differences between the three cues vary across electrode regions, which could result from differences in the availability of binaural cues across frequency regions. The purpose of this study was to determine if, after experience-dependent adaptation, there are systematic differences in the optimal pairing of electrodes at different points along the array for the optimal perception of ITD, ILD, and pitch.

DESIGN

Data from seven bilateral Nucleus users was collected and analyzed. Participants were tested with ITD, ILD, and pitch-matching tasks using five different reference electrodes in one ear, spaced across the array. Comparisons were conducted to determine if the optimal bilateral electrode pairs systematically differed in different regions depending on whether they were measured based on ITD sensitivity, ILD sensitivity, or pitch matching, and how those pairs differed from the pairing in the participants' clinical programs.

RESULTS

Results indicate that there was a significant difference in the optimal pairing depending on the cue measured, but only at the basal end of the array.

CONCLUSION

The results suggest that optimal electrode pairings differ depending on the cue measured to determine optimal pairing, at least for the basal end of the array. This also suggests that the improvements seen when using optimally paired electrodes may be tied to the particular percept being measured both to determine electrode pairing and to assess performance, at least for the basal end of the array.

Light-Based Neuronal Activation: The Future of Cranial Nerve Stimulation

Despite advances in implant hardware, neuroprosthetic devices in otolaryngology have sustained evolutionary rather than revolutionary changes over the past half century. Although electrical stimulation has the capacity for facile activation of neurons and high temporal resolution, it has limited spatial selectivity. Alternative strategies for neuronal stimulation are being investigated to improve spatial resolution. In particular, light-based neuronal stimulation is a viable alternative and complement to electrical stimulation. This article provides a broad overview of light-based neuronal stimulation technologies. Specific examples of active research on light-based prostheses, including cochlear implants, auditory brainstem implants, retinal implants, and facial nerve implants, are reviewed.

Factors Affecting Bimodal Benefit in Pediatric Mandarin-Speaking Chinese Cochlear Implant Users

OBJECTIVES

While fundamental frequency (F0) cues are important to both lexical tone perception and multitalker segregation, F0 cues are poorly perceived by cochlear implant (CI) users. Adding low-frequency acoustic hearing via a hearing aid in the contralateral ear may improve CI users' F0 perception. For English-speaking CI users, contralateral acoustic hearing has been shown to improve perception of target speech in noise and in competing talkers. For tonal languages such as Mandarin Chinese, F0 information is lexically meaningful. Given competing F0 information from multiple talkers and lexical tones, contralateral acoustic hearing may be especially beneficial for Mandarin-speaking CI users' perception of competing speech.

DESIGN

Bimodal benefit (CI+hearing aid - CI-only) was evaluated in 11 pediatric Mandarin-speaking Chinese CI users. In experiment 1, speech recognition thresholds (SRTs) were adaptively measured using a modified coordinated response measure test; subjects were required to correctly identify 2 keywords from among 10 choices in each category. SRTs were measured with CI-only or bimodal listening in the presence of steady state noise (SSN) or competing speech with the same (M+M) or different voice gender (M+F). Unaided thresholds in the non-CI ear and demographic factors were compared with speech performance. In experiment 2, SRTs were adaptively measured in SSN for recognition of 5 keywords, a more difficult listening task than the 2-keyword recognition task in experiment 1.

RESULTS

In experiment 1, SRTs were significantly lower for SSN than for competing speech in both the CI-only and bimodal listening conditions. There was no significant difference between CI-only and bimodal listening for SSN and M+F (p > 0.05); SRTs were significantly lower for CI-only than for bimodal listening for M+M (p < 0.05), suggesting bimodal interference. Subjects were able to make use of voice gender differences for bimodal listening (p < 0.05) but not for CI-only listening (p > 0.05). Unaided thresholds in the non-CI ear were positively correlated with bimodal SRTs for M+M (p < 0.006) but not for SSN or M+F. No significant correlations were observed between any demographic variables and SRTs (p > 0.05 in all cases). In experiment 2, SRTs were significantly lower with two than with five keywords (p < 0.05). A significant bimodal benefit was observed only for the 5-keyword condition (p < 0.05).

CONCLUSIONS

With the CI alone, subjects experienced greater interference with competing speech than with SSN and were unable to use voice gender difference to segregate talkers. For the coordinated response measure task, subjects experienced no bimodal benefit and even bimodal interference when competing talkers were the same voice gender. A bimodal benefit in SSN was observed for the five-keyword condition but not for the two-keyword condition, suggesting that bimodal listening may be more beneficial as the difficulty of the listening task increased. The present data suggest that bimodal benefit may depend on the type of masker and/or the difficulty of the listening task.

Adult Cochlear Implant Recipients' Perspectives on Experiences With Music in Everyday Life: A Multifaceted and Dynamic Phenomenon

BACKGROUND

Cochlear implants (CIs), which have been designed primarily to support spoken communication of persons with severe to profound hearing loss, are highly effective in supporting speech perception in quiet listening conditions. CI users as a group achieve significantly poorer perception and appraisal of music, and speech perception is compromised when background music is present, though outcomes vary considerably across recipients. A number of factors have been identified that contribute to variable music listening experiences, but many questions remain, particularly regarding experiences in everyday life from the perspective of CI users.

PURPOSE

The purpose of this study was twofold: The first aim was to explore the perspectives of adult CI recipients regarding two experiences with music in everyday life: purposeful music listening and background music that competes with spoken conversation. The second aim was to develop a framework of everyday music experiences based upon CI perspectives that could inform future rehabilitative practices and research initiatives.

METHODS

Qualitative and patient-engaged research methodologies were used to emphasize the perspectives of the CI users. Participants included 40 experienced adult CI users ranging in age from 19 to 81 enrolled in 13 CI centers. Participants completed on-line semi-structured open-ended questionnaires regarding purposeful music listening and background music in conjunction with spoken communication. Responses were analyzed using an iterative inductive coding process consistent with grounded theory methodology. The interrelated themes that emerged from the data were then organized into a model synthesizing components from models on music response and self-management for persons with chronic health conditions.

OUTCOMES

Data analyses informed the development of a Dynamic Problem Solving Model for Management of Music Listening Environments adapted from Hill-Briggs (2003) Problem Solving Model of Chronic Illness Self-Management. Key findings were: (1) Music listening is a dynamic, multifaceted experience; satisfactory listening depended upon optimal combinations of factors; (2) Music listening is effortful, but the extent of satisfaction is influenced by expectations and self-management of the situation; (3) CI users have limited access to resources for optimizing music experiences. Many CI users would consider rehabilitation, but level of commitment and priorities differ across CI users.

Gated Word Recognition by Postlingually Deafened Adults With Cochlear Implants: Influence of Semantic Context

PURPOSE

The main goal of this study was to investigate the minimum amount of sensory information required to recognize spoken words (isolation points [IPs]) in listeners with cochlear implants (CIs) and investigate facilitative effects of semantic contexts on the IPs.

METHOD

Listeners with CIs as well as those with normal hearing (NH) participated in the study. In Experiment 1, the CI users listened to unprocessed (full-spectrum) stimuli and individuals with NH listened to full-spectrum or vocoder processed speech. IPs were determined for both groups who listened to gated consonant-nucleus-consonant words that were selected based on lexical properties. In Experiment 2, the role of semantic context on IPs was evaluated. Target stimuli were chosen from the Revised Speech Perception in Noise corpus based on the lexical properties of the final words.

RESULTS

The results indicated that spectrotemporal degradations impacted IPs for gated words adversely, and CI users as well as participants with NH listening to vocoded speech had longer IPs than participants with NH who listened to full-spectrum speech. In addition, there was a clear disadvantage due to lack of semantic context in all groups regardless of the spectral composition of the target speech (full spectrum or vocoded). Finally, we showed that CI users (and users with NH with vocoded speech) can overcome such word processing difficulties with the help of semantic context and perform as well as listeners with NH.

CONCLUSION

Word recognition occurs even before the entire word is heard because listeners with NH associate an acoustic input with its mental representation to understand speech. The results of this study provide insight into the role of spectral degradation on the processing of spoken words in isolation and the potential benefits of semantic context. These results may also explain why CI users rely substantially on semantic context.

Effect of spatial separation and noise type on sentence recognition by Mandarin-speaking cochlear implant users

OBJECTIVES

To investigate the effects of spatial separation and noise type on sentence recognition by unilateral Mandarin-speaking cochlear implant (CI) users and normal-hearing (NH) listeners.

METHOD

Twenty-two unilateral Mandarin-speaking CI users and six NH listeners participated in this study. Speech reception thresholds were measured for three noise types (steady state noise, speech babble, and music). Sentences from the Mandarin Speech Perception test were presented directly in front of the listener (0°). Noise was presented from one of the five speaker locations: -90°, -45°, 0°, +45°, and +90°.

RESULTS

Overall, CI performance was significantly poorer than NH performance for all spatial separation and noise type conditions. NH listeners performed best with music and poorest with steady noise. CI users performed best with steady noise, and poorest with babble. Performance was significantly affected by noise location and noise type. There was no significant difference in head shadow effects among the different noise types for CI users.

CONCLUSIONS

Performance was much poorer in CI than in NH listeners for all noise types and spatial separations. Noise type differently affected unilateral CI users and NH listeners. The limited spectral resolution in CI users did not appear to affect head shadow.

Cognitive Compensation of Speech Perception With Hearing Impairment, Cochlear Implants, and Aging

External degradations in incoming speech reduce understanding, and hearing impairment further compounds the problem. While cognitive mechanisms alleviate some of the difficulties, their effectiveness may change with age. In our research, reviewed here, we investigated cognitive compensation with hearing impairment, cochlear implants, and aging, via (a) phonemic restoration as a measure of top-down filling of missing speech, (b) listening effort and response times as a measure of increased cognitive processing, and (c) visual world paradigm and eye gazing as a measure of the use of context and its time course. Our results indicate that between speech degradations and their cognitive compensation, there is a fine balance that seems to vary greatly across individuals. Hearing impairment or inadequate hearing device settings may limit compensation benefits. Cochlear implants seem to allow the effective use of sentential context, but likely at the cost of delayed processing. Linguistic and lexical knowledge, which play an important role in compensation, may be successfully employed in advanced age, as some compensatory mechanisms seem to be preserved. These findings indicate that cognitive compensation in hearing impairment can be highly complicated—not always absent, but also not easily predicted by speech intelligibility tests only.

Measurement of Acceptable Noise Level with Background Music

Background and Objectives

Acceptable noise level (ANL) is a measure of the maximum background noise level (BNL) that a person is willing to tolerate while following a target story. Although researchers have used various sources of target sound in ANL measures, a limited type of background noise has been used. Extending the previous study of Gordon-Hickey & Moore (2007), the current study determined the effect of music genre and tempo on ANLs as possible factors affecting ANLs. We also investigated the relationships between individual ANLs and the familiarity of music samples and between music ANLs and subjective preference.

Subjects and Methods

Forty-one participants were seperated into two groups according to their ANLs, 29 low-ANL listeners and 12 high-ANL listeners. Using Korean ANL material, the individual ANLs were measured based on the listeners' most comfortable listening level and BNL. The ANLs were measured in six conditions, with different music tempo (fast, slow) and genre (K-pop, pop, classical) in a counterbalanced order.

Results

Overall, ANLs did not differ by the tempo of background music, but music genre significantly affected individual ANLs. We observed relatively higher ANLs with K-pop music and relatively lower ANLs with classical music. This tendency was similar in both low-ANL and high-ANL groups. However, the subjective ratings of music familiarity and preference affected ANLs differently for low-ANL and high-ANL groups. In contrast to the low-ANL listeners, the ANLs of the high-ANL listeners were significantly affected by music familiarity and preference.

Conclusions

The genre of background music affected ANLs obtained using background music. The degree of music familiarity and preference appears to be associated with individual susceptibility to background music only for listeners who are greatly annoyed by background noise (high-ANL listeners).

Susceptibility to interference by music and speech maskers in middle-aged adults

Older listeners commonly complain about difficulty in understanding speech in noise. Previous studies have shown an age effect for both speech and steady noise maskers, and it is largest for speech maskers. In the present study, speech reception thresholds (SRTs) measured with competing speech, music, and steady noise maskers significantly differed between young (19 to 26 years) and middle-aged (51 to 63 years) adults. SRT differences ranged from 2.1 dB for competing speech, 0.4-1.6 dB for music maskers, and 0.8 dB for steady noise. The data suggest that aging effects are already evident in middle-aged adults without significant hearing impairment.

Speech perception in noise with a harmonic complex excited vocoder

A cochlear implant (CI) presents band-pass-filtered acoustic envelope information by modulating current pulse train levels. Similarly, a vocoder presents envelope information by modulating an acoustic carrier. By studying how normal hearing (NH) listeners are able to understand degraded speech signals with a vocoder, the parameters that best simulate electric hearing and factors that might contribute to the NH-CI performance difference may be better understood. A vocoder with harmonic complex carriers (fundamental frequency, f0 = 100 Hz) was used to study the effect of carrier phase dispersion on speech envelopes and intelligibility. The starting phases of the harmonic components were randomly dispersed to varying degrees prior to carrier filtering and modulation. NH listeners were tested on recognition of a closed set of vocoded words in background noise. Two sets of synthesis filters simulated different amounts of current spread in CIs. Results showed that the speech vocoded with carriers whose starting phases were maximally dispersed was the most intelligible. Superior speech understanding may have been a result of the flattening of the dispersed-phase carrier's intrinsic temporal envelopes produced by the large number of interacting components in the high-frequency channels. Cross-correlogram analyses of auditory nerve model simulations confirmed that randomly dispersing the carrier's component starting phases resulted in better neural envelope representation. However, neural metrics extracted from these analyses were not found to accurately predict speech recognition scores for all vocoded speech conditions. It is possible that central speech understanding mechanisms are insensitive to the envelope-fine structure dichotomy exploited by vocoders.