Reduction of the Harmonic Series Influences Musical Enjoyment With Cochlear Implants

A Concert-Based Study on Melodic Contour Identification among Varied Hearing Profiles-A Preliminary Report

Background: This study investigated how different hearing profiles influenced melodic contour identification (MCI) in a real-world concert setting with a live band including drums, bass, and a lead instrument. We aimed to determine the impact of various auditory assistive technologies on music perception in an ecologically valid environment. Methods: The study involved 43 participants with varying hearing capabilities: normal hearing, bilateral hearing aids, bimodal hearing, single-sided cochlear implants, and bilateral cochlear implants. Participants were exposed to melodies played on a piano or accordion, with and without an electric bass as a masker, accompanied by a basic drum rhythm. Bayesian logistic mixed-effects models were utilized to analyze the data. Results: The introduction of an electric bass as a masker did not significantly affect MCI performance for any hearing group when melodies were played on the piano, contrary to its effect on accordion melodies and previous studies. Greater challenges were observed with accordion melodies, especially when accompanied by an electric bass. Conclusions: MCI performance among hearing aid users was comparable to other hearing-impaired profiles, challenging the hypothesis that they would outperform cochlear implant users. A cohort of short melodies inspired by Western music styles was developed for future contour identification tasks.

[Influence of overtones and undertones on melody recognition with a cochlear implant with SSD]

Many cochlear implant (CI) users have difficulties recognising pitches and melodies because pitch transmission is blurred and shifted. This study investigates whether postlingually deafened adult CI users recognize melodies better when overtones are removed or undertones are added.Fifteen unilaterally postlingually deafened CI users (single sided deafness = SSD) were included aged 22 to 73 years (MW 52, SD 11.6) with CI hearing experience between 3 and 75 months (MW 33, SD 21.0) with varying MED-EL devices. Three short piano melodies were presented to them firstly to the normal-hearing ear and then in modified overtone or undertone variants and the original variant to the CI ear. These variants should be identified as one of the three original melodies. In addition, musical experience and ability were assessed by the Munich Music Questionnaire and the MiniPROMS music tests.The CI users showed the best melody recognition in the fundamental frequency variant. The overtone variant with the third overtone was as good as the original variant with all overtones with regard to melody recognition (p=1). However, the undertone variant with the first undertone was recognised significantly worse than the fundamental version (p=0.032). Furthermore, there was no correlation between musical experience or musical ability and the number of melodies recognised (p>0.1).Since a reduction of overtones did not worsen the melody recognition, overtone reduction should be considered in future music processing programs for the CI. This could reduce the energy consumption of the CI.

The Association of Hearing Loss With Active Music Enjoyment in Hearing Aid Users

OBJECTIVE

Hearing aids (HAs) are designed for speech rather than music listening. The impact of HAs on music enjoyment is poorly studied. We examine the effect of HAs on active music enjoyment in individuals with varying levels of hearing loss (HL).

STUDY DESIGN

Cross-sectional study.

SETTING

Tertiary medical center and community.

METHODS

Adult (≥18 years) bilateral HA users and normal hearing (NH) controls actively listened to musical stimuli and rated their enjoyment across 3 measures (pleasantness, musicality, naturalness) with and without HAs using a visual analog scale. Multivariable linear regression was used to assess the association between HL (measured by a pure-tone average [PTA] and word recognition score [WRS] of the better ear) and music enjoyment with and without HAs, adjusting for covariates. Music enjoyment was compared between HA users and NH controls, and HA users with and without their HAs.

RESULTS

One hundred bilateral HA users (mean age 66.0 years, 52% female, better ear mean [SD] PTA 50.2 [13.5] dBHL, mean WRS 84.5 [16.5]%) completed the study. Increasing severity of HL (PTA) was independently associated with decreased music enjoyment (pleasantness, musicality, naturalness) with and without HAs (p < .05). HA usage increased music enjoyment (musicality) in those with moderate to moderately severe HL. Music enjoyment in NH controls (n = 20) was significantly greater across all measures compared to HA users.

CONCLUSION

Increased severity of HL is associated with decreased music enjoyment that can be enhanced with HA usage. Thus, HA usage can positively enhance both speech and music appreciation.

A subjective evaluation of different music preprocessing approaches in cochlear implant listeners

Cochlear implants (CIs) can partially restore speech perception to relatively high levels in listeners with moderate to profound hearing loss. However, for most CI listeners, the perception and enjoyment of music remains notably poor. Since a number of technical and physiological restrictions of current implant designs cannot be easily overcome, a number of preprocessing methods for music signals have been proposed recently. They aim to emphasize the leading voice and rhythmic elements and to reduce their spectral complexity. In this study, CI listeners evaluated five remixing approaches in comparison to unprocessed signals. To identify potential explaining factors of CI preference ratings, different signal quality criteria of the processed signals were additionally assessed by normal-hearing listeners. Additional factors were investigated based on instrumental signal-level features. For three preprocessing methods, a significant improvement over the unprocessed reference was found. Especially, two deep neural network-based remix strategies proved to enhance music perception in CI listeners. These strategies provide remixes of the respective harmonic and percussive signal components of the four source stems "vocals," "bass," "drums," and "other accompaniment." Moreover, the results demonstrate that CI listeners prefer an attenuation of sustained components of drum source signals.

Perception of timbre in adult Cochlear implant users: Comparison of Iranian and Western musical instruments

OBJECTIVES

Cochlear implants (CI) have dramatically improved speech perception for patients with sensorineural hearing impairment. However, listening to music is a great challenge for them. This study examined the perception and appraisal of Iranian musical instruments comparing with similar Western instruments.

METHODS

Eleven adult CI users and 25 normal hearing (NH) individuals participated in this study. Musical stimuli of three commonly heard instrument pairs were prepared. Participants were asked to identify the instruments and rate their appraisal on a ten-point Likert scale (0 = dislike very much, 10 = like very much).

RESULTS

The instrument recognition rate was 40.6% among the CI users, and the mean appraisal score was 5.2 ± 2.7. NH listeners had none significant higher scores on both tasks with a recognition rate of 50.0% and the mean appraisal score of 6.9 ± 1.5. Iranian instruments were more recognized in both groups. Regarding their appraisal, the mean score for both types was almost equal in the NH group, while CI users more appraised Iranian instruments.

CONCLUSION

In addition to better recognition of Iranian instruments, they were particularly better appraised in the CI group. Iranian instruments provide suitable musical pieces for CI recipients that can be considered in rehabilitation programs.

Hearing Aids Enhance Music Enjoyment in Individuals With Hearing Loss

OBJECTIVE

To investigate music appreciation in hearing aid (HA) users with varying levels of hearing loss (HL).

STUDY DESIGN

Cross-sectional, within-subjects design.

SETTING

Tertiary medical center, community.

PATIENTS

Adults (≥18 yr) bilateral HA users.

INTERVENTIONS

HA usage.

MAIN OUTCOME MEASURES

Outcome variables included self-reported music enjoyment measures (pleasantness, musicality, naturalness) with and without HAs assessed with visual analogue scales (10 indicates highest level of enjoyment, 0 the least). Exposure variables include HL (better ear pure-tone average) and speech discrimination (word recognition scores [WRS]). Demographic information was collected.

RESULTS

One hundred nine bilateral HA users completed the study. Mean (standard deviation) age was 66.6 years (16.8 yr); 52.3% were female patients. Mean (standard deviation) better ear pure-tone average was 51.1 dB (16.3 dB) HL. Increased severity of HL and worse WRS were associated with decreased music enjoyment ( p < 0.05) across all measures without HAs, adjusting for sex, age, education, race, HA type, age of HL diagnosis, duration of HL, duration of HA use, musical preference, and musical experience. However, these associations were attenuated or no longer significant with HA usage. Moreover, among all subjects, HAs (vs. no HAs) provided increased music enjoyment in pleasantness (HA, 6.94, no HA, 5.74; p < 0.01), musicality (HA, 7.35; no HA, 6.13, p < 0.01), and naturalness (HA, 6.75; no HA, 6.02; p = 0.02).

CONCLUSION

HA users report increased music enjoyment with HAs compared with without HAs. Increased severity of HL and worse WRS were independently associated with decreased unaided music enjoyment. HA usage seems to mitigate this effect, particularly for those with worse HL.

Effect of Sound Coding Strategies on Music Perception with a Cochlear Implant

The goal of this study was to evaluate the music perception of cochlear implantees with two different sound processing strategies. Methods: Twenty-one patients with unilateral or bilateral cochlear implants (Oticon Medical®) were included. A music trial evaluated emotions (sad versus happy based on tempo and/or minor versus major modes) with three tests of increasing difficulty. This was followed by a test evaluating the perception of musical dissonances (marked out of 10). A novel sound processing strategy reducing spectral distortions (CrystalisXDP, Oticon Medical) was compared to the standard strategy (main peak interleaved sampling). Each strategy was used one week before the music trial. Results: Total music score was higher with CrystalisXDP than with the standard strategy. Nine patients (21%) categorized music above the random level (>5) on test 3 only based on mode with either of the strategies. In this group, CrystalisXDP improved the performances. For dissonance detection, 17 patients (40%) scored above random level with either of the strategies. In this group, CrystalisXDP did not improve the performances. Conclusions: CrystalisXDP, which enhances spectral cues, seemed to improve the categorization of happy versus sad music. Spectral cues could participate in musical emotions in cochlear implantees and improve the quality of musical perception.

A versatile deep-neural-network-based music preprocessing and remixing scheme for cochlear implant listeners

While cochlear implants (CIs) have proven to restore speech perception to a remarkable extent, access to music remains difficult for most CI users. In this work, a methodology for the design of deep learning-based signal preprocessing strategies that simplify music signals and emphasize rhythmic information is proposed. It combines harmonic/percussive source separation and deep neural network (DNN) based source separation in a versatile source mixture model. Two different neural network architectures were assessed with regard to their applicability for this task. The method was evaluated with instrumental measures and in two listening experiments for both network architectures and six mixing presets. Normal-hearing subjects rated the signal quality of the processed signals compared to the original both with and without a vocoder which provides an approximation of the auditory perception in CI listeners. Four combinations of remix models and DNNs have been selected for an evaluation with vocoded signals and were all rated significantly better in comparison to the unprocessed signal. In particular, the two best-performing remix networks are promising candidates for further evaluation in CI listeners.

Music perception and speech intelligibility in noise performance by Italian-speaking cochlear implant users

Objective

The goal of this study was to investigate the performance correlations between music perception and speech intelligibility in noise by Italian-speaking cochlear implant (CI) users.

Materials and methods

Twenty postlingually deafened adults with unilateral CIs (mean age 65 years, range 46–92 years) were tested with a music quality questionnaire using three passages of music from Classical Music, Jazz, and Soul. Speech recognition in noise was assessed using two newly developed adaptive tests in Italian: The Sentence Test with Adaptive Randomized Roving levels (STARR) and Matrix tests.

Results

Median quality ratings for Classical, Jazz and Soul music were 63%, 58% and 58%, respectively. Median SRTs for the STARR and Matrix tests were 14.3 dB and 7.6 dB, respectively. STARR performance was significantly correlated with Classical music ratings (r_s = − 0.49, p = 0.029), whereas Matrix performance was significantly correlated with both Classical (r_s = − 0.48, p = 0.031) and Jazz music ratings (r_s = − 0.56, p = 0.011).

Conclusion

Speech with competitive noise and music are naturally present in everyday listening environments. Recent speech perception tests based on an adaptive paradigm and sentence materials in relation with music quality measures might be representative of everyday performance in CI users. The present data contribute to cross-language studies and suggest that improving music perception in CI users may yield everyday benefit in speech perception in noise and may hence enhance the quality of listening for CI users.

Novel Web-Based Music Re-Engineering Software for Enhancement of Music Enjoyment Among Cochlear Implantees

OBJECTIVES

Cochlear implant (CI) listeners experience diminished music perception and enjoyment from a variety of patient-related and implant-related factors. We investigate the hypothesis that patient-directed music re-engineering may enhance music enjoyment with CI.

STUDY DESIGN

Prospective cohort study.

SETTING

Academic cochlear implant center.

SUBJECTS AND METHODS

A multidisciplinary team of neurotologists, audiologists, and a sound/audio engineer collaborated with a web developer to create a music re-engineering application. Experienced adult CI listeners rated original excerpts from five major genres of music on enjoyment using a visual analog scale (VAS). Subjects were then allowed to re-engineer the original by adjusting treble frequencies, bass frequencies, percussion emphasis, and reverberation and again rated on enjoyment.

RESULTS

Total of 46 subjects, with a mean age of 57.6 years (SD = 16; range, 18-90) participated in the study. User-mixed audio was rated higher across all measures of enjoyment than original recordings (mean difference +0.92; p < 0.05, CI [0.22, 1.62]), an effect that was seen across all genres except for country music. Subjects preferred louder bass frequencies (mean difference +7.1 dB; p < 0.01, CI [2.15, 24.3]) and more reverberation (mean difference +6.6 ms; p < 0.01, CI [2.85, 10.7]). Re-engineered music increased enjoyment in 57%, and 79% reported an interest in being able to mix music of their own choosing.

CONCLUSION

User-directed music re-engineering increases music enjoyment for CI listeners. The cochlear implantee preferred heightened bass, reverberation, and treble across musical genres. These findings support the implementation of patient-directed music re-engineering to enhance music enjoyment with technology that is readily available today.

Participation of Acoustic and Electric Hearing in Perceiving Musical Sounds

Introduction: The objective of our study was to evaluate musical perception and its relation to the quality of life in patients with bimodal binaural auditory stimulation.

Materials and Methods: Nineteen adult patients with a cochlear implant (CI) for minimum 6 months, and moderate to severe contralateral hearing loss with a hearing aid (HA), and 21 normal hearing adults were included in this prospective, cross-sectional study. Pure-tone and speech audiometry, musical test evaluating sound perception characteristics and musical listening abilities, Munich questionnaire for musical habits, and the APHAB questionnaire were recoded. Performance in musical perception test with HA, CI, and HA + CI, and potential correlations between music test, audiometry and questionnaires were investigated.

Results: Bimodal stimulation improved musical perception in several features (sound brightness, roughness, and clarity) in comparison to unimodal hearing, but CI did not add to HA performances in texture, polyphony or musical emotion and even appeared to interfere negatively in pitch perception with HA. Musical perception performances (sound clarity, instrument recognition) appeared to be correlated to hearing-related quality of life (APHAB RV and EC subdomains) but not with speech performances suggesting that the exploration of musical perception complements speech understanding evaluation to better describe every-day life hearing handicap.

Conclusion: Testing musical sound perception provides important information on hearing performances as a complement to speech audiometry and appears to be related to hearing-related quality of life.

The Perception of Multiple Simultaneous Pitches as a Function of Number of Spectral Channels and Spectral Spread in a Noise-Excited Envelope Vocoder

Cochlear implant (CI) listeners typically perform poorly on tasks involving the pitch of complex tones. This limitation in performance is thought to be mainly due to the restricted number of active channels and the broad current spread that leads to channel interactions and subsequent loss of precise spectral information, with temporal information limited primarily to temporal-envelope cues. Little is known about the degree of spectral resolution required to perceive combinations of multiple pitches, or a single pitch in the presence of other interfering tones in the same spectral region. This study used noise-excited envelope vocoders that simulate the limited resolution of CIs to explore the perception of multiple pitches presented simultaneously. The results show that the resolution required for perceiving multiple complex pitches is comparable to that found in a previous study using single complex tones. Although relatively high performance can be achieved with 48 channels, performance remained near chance when even limited spectral spread (with filter slopes as steep as 144 dB/octave) was introduced to the simulations. Overall, these tight constraints suggest that current CI technology will not be able to convey the pitches of combinations of spectrally overlapping complex tones.

Interactive Evaluation of a Music Preprocessing Scheme for Cochlear Implants Based on Spectral Complexity Reduction

Music is difficult to access for the majority of CI users as the reduced dynamic range and poor spectral resolution in cochlear implants (CI), amongst others constraints, severely impair their auditory perception. The reduction of spectral complexity is therefore a promising means to facilitate music enjoyment for CI listeners. We evaluate a spectral complexity reduction method for music signals based on principal component analysis that enforces spectral sparsity, emphasizes the melody contour and attenuates interfering accompanying voices. To cover a wide range of spectral complexity reduction levels a new experimental design for listening experiments was introduced. It allows CI users to select the preferred level of spectral complexity reduction interactively and in real-time. Ten adult CI recipients with post-lingual bilateral profound sensorineural hearing loss and CI experience of at least 6 months were enrolled in the study. In eight consecutive sessions over a period of 4 weeks they were asked to choose their preferred version out of 10 different complexity settings for a total number of 16 recordings of classical western chamber music. As the experiments were performed in consecutive sessions we also studied a potential long term effect. Therefore, we investigated the hypothesis that repeated engagement with music signals of reduced spectral complexity leads to a habituation effect which allows CI users to deal with music signals of increasing complexity. Questionnaires and tests about music listening habits and musical abilities complemented these experiments. The participants significantly preferred signals with high spectral complexity reduction levels over the unprocessed versions. While the results of earlier studies comprising only two preselected complexity levels were generally confirmed, this study revealed a tendency toward a selection of even higher spectral complexity reduction levels. Therefore, spectral complexity reduction for music signals is a useful strategy to enhance music enjoyment for CI users. Although there is evidence for a habituation effect in some subjects, such an effect has not been significant in general.

Music Appreciation after Cochlear Implantation in Adult Patients: A Systematic Review

Objective

The cochlear implant (CI) improves quality of life for people who are severely and profoundly deafened, allowing implantees to perceive speech at levels similar to those of individuals with normal hearing. However, patients with CIs generally report a reduced appreciation of music after implantation. We aimed to systematically review the English-language literature for studies evaluating music enjoyment and perception among adult patients with CIs.

Data Sources

A systematic review of PubMed/MEDLINE, Scopus, Embase, and the Cochrane Library.

Review Methods

The PRISMA statement was utilized to identify English-language studies reporting music appreciation among adults with CIs. Two independent reviewers performed searches through May 2017. Included studies investigated parameters related to music enjoyment and music perception, including (1) pitch and timbre perception, (2) noise-canceling algorithms, and (3) the presence of dissonant chords, lyrics, or visual cues.

Results

A total of 508 articles were screened for relevance. Forty-one full-text articles were evaluated, and 18 met final inclusion criteria. Studies used heterogeneous methods of outcome measurement for identifying music appreciation. The outcome measures suggest that rhythm and lyrics are important components of enjoyment. Patients with CIs had difficulty with pitch and timbre perception.

Conclusion

The heterogeneous outcome measures identified in this systematic review suggest that rhythm and lyrics are important components of enjoyment, while patients with CIs had difficulty with pitch and timbre perception. Because there is no standardized reporting metric for music appreciation among adult patients with CIs, a standardized validated outcome-measuring tool is warranted.

Spectral complexity reduction of music signals based on frequency-domain reduced-rank approximations: An evaluation with cochlear implant listeners

Methods for spectral complexity reduction of music signals were evaluated in a listening test with cochlear implant (CI) listeners. To this end, reduced-rank approximations were computed in the constant-Q spectral domain using blind and score-informed dimensionality reduction techniques, which were compared to a procedure using a supervised source separation and remixing scheme. Previous works have shown that timbre and pitch cues are transmitted inaccurately through CIs and thus cause perceptual distortions in CI listeners. Hence, the scope of this evaluation was narrowed down to classical chamber music, which is mainly characterized by timbre and pitch and less by rhythmic cues. Suitable music pieces were selected in accordance to a statistical experimental design, which took musically relevant influential factors into account. In a blind two-alternative forced choice task, 14 CI listeners were asked to indicate a preference either for the original signals or a specific processed variant. The results exhibit a statistically significant preference rate of up to 74% for the reduced-rank approximations, whereas the source separation and remixing scheme did not provide any improvement.