1
|
Zheng C, Xu C, Wang M, Li X, Moore BCJ. Evaluation of deep marginal feedback cancellation for hearing aids using speech and music. Trends Hear 2023; 27:23312165231192290. [PMID: 37551089 PMCID: PMC10408330 DOI: 10.1177/23312165231192290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2023] [Accepted: 05/22/2023] [Indexed: 08/09/2023] Open
Abstract
Speech and music both play fundamental roles in daily life. Speech is important for communication while music is important for relaxation and social interaction. Both speech and music have a large dynamic range. This does not pose problems for listeners with normal hearing. However, for hearing-impaired listeners, elevated hearing thresholds may result in low-level portions of sound being inaudible. Hearing aids with frequency-dependent amplification and amplitude compression can partly compensate for this problem. However, the gain required for low-level portions of sound to compensate for the hearing loss can be larger than the maximum stable gain of a hearing aid, leading to acoustic feedback. Feedback control is used to avoid such instability, but this can lead to artifacts, especially when the gain is only just below the maximum stable gain. We previously proposed a deep-learning method called DeepMFC for controlling feedback and reducing artifacts and showed that when the sound source was speech DeepMFC performed much better than traditional approaches. However, its performance using music as the sound source was not assessed and the way in which it led to improved performance for speech was not determined. The present paper reveals how DeepMFC addresses feedback problems and evaluates DeepMFC using speech and music as sound sources with both objective and subjective measures. DeepMFC achieved good performance for both speech and music when it was trained with matched training materials. When combined with an adaptive feedback canceller it provided over 13 dB of additional stable gain for hearing-impaired listeners.
Collapse
Affiliation(s)
- Chengshi Zheng
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Chenyang Xu
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Meihuang Wang
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Xiaodong Li
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Brian C. J. Moore
- Cambridge Hearing Group, Department of Psychology, University of Cambridge, Cambridge, UK
| |
Collapse
|
2
|
Zheng C, Wang M, Li X, Moore BCJ. A deep learning solution to the marginal stability problems of acoustic feedback systems for hearing aids. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 152:3616. [PMID: 36586835 DOI: 10.1121/10.0016589] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 12/01/2022] [Indexed: 06/17/2023]
Abstract
For hearing aids, it is critical to reduce the acoustic coupling between the receiver and microphone to ensure that prescribed gains are below the maximum stable gain, thus preventing acoustic feedback. Methods for doing this include fixed and adaptive feedback cancellation, phase modulation, and gain reduction. However, the behavior of hearing aids in situations where the prescribed gain is only just below the maximum stable gain, called here "marginally stable gain," is not well understood. This paper analyzed marginally stable systems and identified three problems, including increased gain at frequencies with the smallest gain margin, short whistles caused by the limited rate of decay of the output when the input drops, and coloration effects. A deep learning framework, called deep marginal feedback cancellation (DeepMFC), was developed to suppress short whistles, and reduce coloration effects, as well as to limit excess amplification at certain frequencies. To implement DeepMFC, many receiver signals in closed-loop systems and corresponding open-loop systems were simulated, and the receiver signals of the closed-loop and open-loop systems were paired together to obtain parallel signals for training. DeepMFC achieved much better performance than existing feedback control methods using objective and subjective measures.
Collapse
Affiliation(s)
- Chengshi Zheng
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, 100190, Beijing, China
| | - Meihuang Wang
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, 100190, Beijing, China
| | - Xiaodong Li
- Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy of Sciences, 100190, Beijing, China
| | - Brian C J Moore
- Cambridge Hearing Group, Department of Psychology, University of Cambridge, Downing Street, Cambridge CB2 3EB, United Kingdom
| |
Collapse
|
3
|
Peixoto MC, Pratas R, Miranda C, Bento M, Oliveira S, Correia da Silva V. Active Middle Ear Implant Patient Outcomes With Adaptive Feedback Canceller. Otol Neurotol 2021; 42:372-376. [PMID: 33332858 DOI: 10.1097/mao.0000000000002960] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE Evaluation of the audiological patient performance with an upgrade of the firmware from the fixed feedback canceller (FFC) to the adaptive feedback canceller (AFC) on an active middle ear implant. STUDY DESIGN Retrospective observational nonrandomized group study. SETTING Private hospital. PATIENTS/INTERVENTIONS From March 2018 to September 2019, 15 patients implanted with an active middle ear implant, with 6 or more months of experience with a FFC system, were upgraded to an AFC algorithm. MAIN OUTCOME MEASURES Functional gain, speech perception in silence and in noise, and sound localization capacities were examined. Feedback reduction was also analyzed. RESULTS Thirteen patients were analyzed. Pure tone audiometric evaluation with FFC (mean value of 48.02 dB) compared with AFC at 1 (mean value of 49.12 dB) and 6 months (mean value of 42.75 dB) revealed no statistically significant differences (p = 0.889 and p = 0.358 respectively).In speech discrimination in silence, clinically relevant improvements were observed with AFC at 1 and 6 months, with a mean value of 41.5 and 38.3 dB, respectively (p = 0.03 and p = 0.021 correspondingly). In speech discrimination in noisy environments, we observed an improvement of the different conditions tested. No differences were found in localization capacities between FFC and AFC at the two different moments of evaluation. CONCLUSIONS AFC is more effective than FFC in cancelling feedback and improving sound quality, allowing for better speech understanding in silence and in noise.
Collapse
Affiliation(s)
- Maria Conceição Peixoto
- Auditory Implant Unit, Department of Otorhinolaryngology, Hospital CUF Porto, Porto, Portugal
| | | | | | | | | | | |
Collapse
|
4
|
Pisha L, Warchall J, Zubatiy T, Hamilton S, Lee CH, Chockalingam G, Mercier PP, Gupta R, Rao BD, Garudadri H. A Wearable, Extensible, Open-Source Platform for Hearing Healthcare Research. IEEE ACCESS : PRACTICAL INNOVATIONS, OPEN SOLUTIONS 2019; 7:162083-162101. [PMID: 32547893 PMCID: PMC7297218 DOI: 10.1109/access.2019.2951145] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Hearing loss is one of the most common conditions affecting older adults worldwide. Frequent complaints from the users of modern hearing aids include poor speech intelligibility in noisy environments and high cost, among other issues. However, the signal processing and audiological research needed to address these problems has long been hampered by proprietary development systems, underpowered embedded processors, and the difficulty of performing tests in real-world acoustical environments. To facilitate existing research in hearing healthcare and enable new investigations beyond what is currently possible, we have developed a modern, open-source hearing research platform, Open Speech Platform (OSP). This paper presents the system design of the complete OSP wearable platform, from hardware through firmware and software to user applications. The platform provides a complete suite of basic and advanced hearing aid features which can be adapted by researchers. It serves web apps directly from a hotspot on the wearable hardware, enabling users and researchers to control the system in real time. In addition, it can simultaneously acquire high-quality electroencephalography (EEG) or other electrophysiological signals closely synchronized to the audio. All of these features are provided in a wearable form factor with enough battery life for hours of operation in the field.
Collapse
Affiliation(s)
- Louis Pisha
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Julian Warchall
- Department of Electrical and Computer Engineering, University of California, San Diego
| | | | - Sean Hamilton
- Department of Computer Science and Engineering, University of California, San Diego
| | - Ching-Hua Lee
- Department of Electrical and Computer Engineering, University of California, San Diego
| | | | - Patrick P Mercier
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Rajesh Gupta
- Department of Computer Science and Engineering, University of California, San Diego. Halıcıoğlu Data Science Institute
| | - Bhaskar D Rao
- Department of Electrical and Computer Engineering, University of California, San Diego
| | | |
Collapse
|
5
|
Lee CH, Rao BD, Garudadri H. Proportionate Adaptive Filters Based on Minimizing Diversity Measures for Promoting Sparsity. CONFERENCE RECORD. ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS 2019; 2019:769-773. [PMID: 33223796 PMCID: PMC7676632 DOI: 10.1109/ieeeconf44664.2019.9048716] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In this paper, a novel way of deriving proportionate adaptive filters is proposed based on diversity measure minimization using the iterative reweighting techniques well-known in the sparse signal recovery (SSR) area. The resulting least mean square (LMS)-type and normalized LMS (NLMS)-type sparse adaptive filtering algorithms can incorporate various diversity measures that have proved effective in SSR. Furthermore, by setting the regularization coefficient of the diversity measure term to zero in the resulting algorithms, Sparsity promoting LMS (SLMS) and Sparsity promoting NLMS (SNLMS) are introduced, which exploit but do not strictly enforce the sparsity of the system response if it already exists. Moreover, unlike most existing proportionate algorithms that design the step-size control factors based on heuristics, our SSR-based framework leads to designing the factors in a more systematic way. Simulation results are presented to demonstrate the convergence behavior of the derived algorithms for systems with different sparsity levels.
Collapse
Affiliation(s)
- Ching-Hua Lee
- Department of Electrical and Computer Engineering University of California, San Diego
| | - Bhaskar D Rao
- Department of Electrical and Computer Engineering University of California, San Diego
| | - Harinath Garudadri
- Department of Electrical and Computer Engineering University of California, San Diego
| |
Collapse
|
6
|
Lee CH, Chen KL, Harris F, Rao BD, Garudadri H. On Mitigating Acoustic Feedback in Hearing Aids with Frequency Warping by All-Pass Networks. INTERSPEECH 2019; 2019:4245-4249. [PMID: 33163529 PMCID: PMC7643774 DOI: 10.21437/interspeech.2019-3195] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Acoustic feedback control continues to be a challenging problem due to the emerging form factors in advanced hearing aids (HAs) and hearables. In this paper, we present a novel use of well-known all-pass filters in a network to perform frequency warping that we call "freping." Freping helps in breaking the Nyquist stability criterion and improves adaptive feedback cancellation (AFC). Based on informal subjective assessments, distortions due to freping are fairly benign. While common objective metrics like the perceptual evaluation of speech quality (PESQ) and the hearing-aid speech quality index (HASQI) may not adequately capture distortions due to freping and acoustic feedback artifacts from a perceptual perspective, they are still instructive in assessing the proposed method. We demonstrate quality improvements with freping for a basic AFC (PESQ: 2.56 to 3.52 and HASQI: 0.65 to 0.78) at a gain setting of 20; and an advanced AFC (PESQ: 2.75 to 3.17 and HASQI: 0.66 to 0.73) for a gain of 30. From our investigations, freping provides larger improvement for basic AFC, but still improves overall system performance for many AFC approaches.
Collapse
Affiliation(s)
- Ching-Hua Lee
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Kuan-Lin Chen
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Fred Harris
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Bhaskar D Rao
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Harinath Garudadri
- Department of Electrical and Computer Engineering, University of California, San Diego
| |
Collapse
|
7
|
Marcrum SC, Picou EM, Bohr C, Steffens T. Feedback reduction system influence on additional gain before feedback and maximum stable gain in open-fitted hearing aids. Int J Audiol 2018; 57:737-745. [DOI: 10.1080/14992027.2018.1487088] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Affiliation(s)
- Steven C. Marcrum
- Department of Otolaryngology, University Hospital Regensburg, Regensburg, Germany
| | - Erin M. Picou
- Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Christopher Bohr
- Department of Otolaryngology, University Hospital Regensburg, Regensburg, Germany
| | - Thomas Steffens
- Department of Otolaryngology, University Hospital Regensburg, Regensburg, Germany
| |
Collapse
|
8
|
Pisha L, Hamilton S, Sengupta D, Lee CH, Vastare KC, Zubatiy T, Luna S, Yalcin C, Grant A, Gupta R, Chockalingam G, Rao BD, Garudadri H. A Wearable Platform for Research in Augmented Hearing. CONFERENCE RECORD. ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS 2018; 2018:223-227. [PMID: 31379421 PMCID: PMC6677400 DOI: 10.1109/acssc.2018.8645557] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
We have previously reported a realtime, open-source speech-processing platform (OSP) for hearing aids (HAs) research. In this contribution, we describe a wearable version of this platform to facilitate audiological studies in the lab and in the field. The system is based on smartphone chipsets to leverage power efficiency in terms of FLOPS/watt and economies of scale. We present the system architecture and discuss salient design elements in support of HA research. The ear-level assemblies support up to 4 microphones on each ear, with 96 kHz, 24 bit codecs. The wearable unit runs OSP Release 2018c on top of 64-bit Debian Linux for binaural HA with an overall latency of 5.6 ms. The wearable unit also hosts an embedded web server (EWS) to monitor and control the HA state in realtime. We describe three example web apps in support of typical audiological studies they enable. Finally, we describe a baseline speech enhancement module included with Release 2018c, and describe extensions to the algorithms as future work.
Collapse
Affiliation(s)
- Louis Pisha
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Sean Hamilton
- Department of Computer Science and Engineering, University of California, San Diego
| | - Dhiman Sengupta
- Department of Computer Science and Engineering, University of California, San Diego
| | - Ching-Hua Lee
- Department of Electrical and Computer Engineering, University of California, San Diego
| | | | - Tamara Zubatiy
- California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego
| | - Sergio Luna
- Department of Mathematics, University of California, San Diego
| | - Cagri Yalcin
- California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego
| | - Alex Grant
- California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego
| | - Rajesh Gupta
- Department of Computer Science and Engineering, University of California, San Diego
| | - Ganz Chockalingam
- California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego
| | - Bhaskar D Rao
- Department of Electrical and Computer Engineering, University of California, San Diego
| | - Harinath Garudadri
- Department of Electrical and Computer Engineering, University of California, San Diego
- California Institute for Telecommunications and Information Technology (Calit2), University of California, San Diego
| |
Collapse
|