1
|
Karbasi M, Zeiler S, Kolossa D. Microscopic and Blind Prediction of Speech Intelligibility: Theory and Practice. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2022; 30:2141-2155. [PMID: 37007458 PMCID: PMC10065470 DOI: 10.1109/taslp.2022.3184888] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Being able to estimate speech intelligibility without the need for listening tests would confer great benefits for a wide range of speech processing applications. Many attempts have therefore been made to introduce an objective, and ideally referencefree measure for this purpose. Most works analyze speech intelligibility prediction (SIP) methods from a macroscopic point of view, averaging over longer time spans. This paper, in contrast, presents a theoretical framework for the microscopic evaluation of SIP methods. Within our framework, a Statistically estimated Accuracy based on Theory (StAT) is derived, which numerically quantifies the statistical limitations inherent in microscopic SIP. A state-of-the-art approach to microscopic SIP, namely, the use of automatic speech recognition (ASR) to directly predict listening test results, is evaluated within this framework. The practical results are in good agreement with the theory. As the final contribution, a fully blind DIscriminative Speech intelligibility Predictor (DISP) is introduced and is also evaluated within the StAT framework. It is shown that this novel, blind estimator can predict intelligibility as well as-and often even with better accuracy than-the non-blind ASR-based approach, and that its results are again in good agreement with its theoretically derived performance potential.
Collapse
Affiliation(s)
- Mahdie Karbasi
- Cognitive signal processing group, Electrical engineering department, Ruhr-Universität Bochum, Universitätsstraße 150, 44801 Bochum, NRW, Germany
| | - Steffen Zeiler
- Cognitive signal processing group, Electrical engineering department, Ruhr-Universität Bochum, Universitätsstraße 150, 44801 Bochum, NRW, Germany
| | - Dorothea Kolossa
- Cognitive signal processing group, Electrical engineering department, Ruhr-Universität Bochum, Universitätsstraße 150, 44801 Bochum, NRW, Germany
| |
Collapse
|
2
|
Fontan L, Gonçalves Braz L, Pinquier J, Stone MA, Füllgrabe C. Using Automatic Speech Recognition to Optimize Hearing-Aid Time Constants. Front Neurosci 2022; 16:779062. [PMID: 35368250 PMCID: PMC8969748 DOI: 10.3389/fnins.2022.779062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Accepted: 02/14/2022] [Indexed: 12/03/2022] Open
Abstract
Automatic speech recognition (ASR), when combined with hearing-aid (HA) and hearing-loss (HL) simulations, can predict aided speech-identification performances of persons with age-related hearing loss. ASR can thus be used to evaluate different HA configurations, such as combinations of insertion-gain functions and compression thresholds, in order to optimize HA fitting for a given person. The present study investigated whether, after fixing compression thresholds and insertion gains, a random-search algorithm could be used to optimize time constants (i.e., attack and release times) for 12 audiometric profiles. The insertion gains were either those recommended by the CAM2 prescription rule or those optimized using ASR, while compression thresholds were always optimized using ASR. For each audiometric profile, the random-search algorithm was used to vary time constants with the aim to maximize ASR performance. A HA simulator and a HL simulator simulator were used, respectively, to amplify and to degrade speech stimuli according to the input audiogram. The resulting speech signals were fed to an ASR system for recognition. For each audiogram, 1,000 iterations of the random-search algorithm were used to find the time-constant configuration yielding the highest ASR score. To assess the reproducibility of the results, the random search algorithm was run twice. Optimizing the time constants significantly improved the ASR scores when CAM2 insertion gains were used, but not when using ASR-based gains. Repeating the random search yielded similar ASR scores, but different time-constant configurations.
Collapse
Affiliation(s)
- Lionel Fontan
- Archean LABS, Montauban, France
- *Correspondence: Lionel Fontan,
| | | | | | - Michael A. Stone
- Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Christian Füllgrabe
- School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom
| |
Collapse
|
3
|
Roßbach J, Kollmeier B, Meyer BT. A model of speech recognition for hearing-impaired listeners based on deep learning. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 151:1417. [PMID: 35364918 DOI: 10.1121/10.0009411] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Accepted: 01/18/2022] [Indexed: 06/14/2023]
Abstract
Automatic speech recognition (ASR) has made major progress based on deep machine learning, which motivated the use of deep neural networks (DNNs) as perception models and specifically to predict human speech recognition (HSR). This study investigates if a modeling approach based on a DNN that serves as phoneme classifier [Spille, Ewert, Kollmeier, and Meyer (2018). Comput. Speech Lang. 48, 51-66] can predict HSR for subjects with different degrees of hearing loss when listening to speech embedded in different complex noises. The eight noise signals range from simple stationary noise to a single competing talker and are added to matrix sentences, which are presented to 20 hearing-impaired (HI) listeners (categorized into three groups with different types of age-related hearing loss) to measure their speech recognition threshold (SRT), i.e., the signal-to-noise ratio with 50% word recognition rate. These are compared to responses obtained from the ASR-based model using degraded feature representations that take into account the individual hearing loss of the participants captured by a pure-tone audiogram. Additionally, SRTs obtained from eight normal-hearing (NH) listeners are analyzed. For NH subjects and three groups of HI listeners, the average SRT prediction error is below 2 dB, which is lower than the errors of the baseline models.
Collapse
Affiliation(s)
- Jana Roßbach
- Communication Acoustics and Cluster of Excellence Hearing4all, Carl von Ossietzky University, D-26111 Oldenburg, Germany
| | - Birger Kollmeier
- Medical Physics and Cluster of Excellence Hearing4all, Carl von Ossietzky University, D-26111 Oldenburg, Germany
| | - Bernd T Meyer
- Communication Acoustics and Cluster of Excellence Hearing4all, Carl von Ossietzky University, D-26111 Oldenburg, Germany
| |
Collapse
|
4
|
Gonçalves Braz L, Fontan L, Pinquier J, Stone MA, Füllgrabe C. OPRA-RS: A Hearing-Aid Fitting Method Based on Automatic Speech Recognition and Random Search. Front Neurosci 2022; 16:779048. [PMID: 35264922 PMCID: PMC8899657 DOI: 10.3389/fnins.2022.779048] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Accepted: 01/13/2022] [Indexed: 12/27/2022] Open
Abstract
Hearing-aid (HA) prescription rules (such as NAL-NL2, DSL-v5, and CAM2) are used by HA audiologists to define initial HA settings (e.g., insertion gains, IGs) for patients. This initial fitting is later individually adjusted for each patient to improve clinical outcomes in terms of speech intelligibility and listening comfort. During this fine-tuning stage, speech-intelligibility tests are often carried out with the patient to assess the benefits associated with different HA settings. As these tests tend to be time-consuming and performance on them depends on the patient's level of fatigue and familiarity with the test material, only a limited number of HA settings can be explored. Consequently, it is likely that a suboptimal fitting is used for the patient. Recent studies have shown that automatic speech recognition (ASR) can be used to predict the effects of IGs on speech intelligibility for patients with age-related hearing loss (ARHL). The aim of the present study was to extend this approach by optimizing, in addition to IGs, compression thresholds (CTs). However, increasing the number of parameters to be fitted increases exponentially the number of configurations to be assessed. To limit the number of HA settings to be tested, three random-search (RS) genetic algorithms were used. The resulting new HA fitting method, combining ASR and RS, is referred to as "objective prescription rule based on ASR and random search" (OPRA-RS). Optimal HA settings were computed for 12 audiograms, representing average and individual audiometric profiles typical for various levels of ARHL severity, and associated ASR performances were compared to those obtained with the settings recommended by CAM2. Each RS algorithm was run twice to assess its reliability. For all RS algorithms, ASR scores obtained with OPRA-RS were significantly higher than those associated with CAM2. Each RS algorithm converged on similar optimal HA settings across repetitions. However, significant differences were observed between RS algorithms in terms of maximum ASR performance and processing costs. These promising results open the way to the use of ASR and RS algorithms for the fine-tuning of HAs with potential speech-intelligibility benefits for the patient.
Collapse
Affiliation(s)
- Libio Gonçalves Braz
- IRIT, CNRS, Université Paul Sabatier, Toulouse, France,*Correspondence: Libio Gonçalves Braz
| | | | | | - Michael A. Stone
- Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Christian Füllgrabe
- School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, United Kingdom
| |
Collapse
|
5
|
Hülsmeier D, Buhl M, Wardenga N, Warzybok A, Schädler MR, Kollmeier B. Inference of the distortion component of hearing impairment from speech recognition by predicting the effect of the attenuation component. Int J Audiol 2021; 61:205-219. [PMID: 34081564 DOI: 10.1080/14992027.2021.1929515] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
OBJECTIVE A model-based determination of the average supra-threshold ("distortion") component of hearing impairment which limits the benefit of hearing aid amplification. DESIGN Published speech recognition thresholds (SRTs) were predicted with the framework for auditory discrimination experiments (FADE), which simulates recognition processes, the speech intelligibility index (SII), which exploits frequency-dependent signal-to-noise ratios (SNR), and a modified SII with a hearing-loss-dependent band importance function (PAV). Their attenuation-component-based prediction errors were interpreted as estimates of the distortion component. STUDY SAMPLE Unaided SRTs of 315 hearing-impaired ears measured with the German matrix sentence test in stationary noise. RESULTS Overall, the models showed root-mean-square errors (RMSEs) of 7 dB, but for steeply sloping hearing loss FADE and PAV were more accurate (RMSE = 9 dB) than the SII (RMSE = 23 dB). Prediction errors of FADE and PAV increased linearly with the average hearing loss. The consideration of the distortion component estimate significantly improved the accuracy of FADE's and PAV's predictions. CONCLUSIONS The supra-threshold distortion component-estimated by prediction errors of FADE and PAV-seems to increase with the average hearing loss. Accounting for a distortion component improves the model predictions and implies a need for effective compensation strategies for supra-threshold processing deficits with increasing audibility loss.
Collapse
Affiliation(s)
- David Hülsmeier
- Medical Physics, CvO University Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, Oldenburg, Germany
| | - Mareike Buhl
- Medical Physics, CvO University Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, Oldenburg, Germany
| | - Nina Wardenga
- Cluster of Excellence Hearing4all, Oldenburg, Germany.,Department of Otolaryngology, Hannover Medical School, Hannover, Germany
| | - Anna Warzybok
- Medical Physics, CvO University Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, Oldenburg, Germany
| | - Marc René Schädler
- Medical Physics, CvO University Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, Oldenburg, Germany
| | - Birger Kollmeier
- Medical Physics, CvO University Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, Oldenburg, Germany
| |
Collapse
|
6
|
Fontan L, Le Coz M, Azzopardi C, Stone MA, Füllgrabe C. Improving hearing-aid gains based on automatic speech recognition. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2020; 148:EL227. [PMID: 33003882 DOI: 10.1121/10.0001866] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/14/2020] [Accepted: 08/13/2020] [Indexed: 06/11/2023]
Abstract
This study provides proof of concept that automatic speech recognition (ASR) can be used to improve hearing aid (HA) fitting. A signal-processing chain consisting of a HA simulator, a hearing-loss simulator, and an ASR system normalizing the intensity of input signals was used to find HA-gain functions yielding the highest ASR intelligibility scores for individual audiometric profiles of 24 listeners with age-related hearing loss. Significantly higher aided speech intelligibility scores and subjective ratings of speech pleasantness were observed when the participants were fitted with ASR-established gains than when fitted with the gains recommended by the CAM2 fitting rule.
Collapse
Affiliation(s)
- Lionel Fontan
- Archean LABS, 20 place Prax-Paris, 82000 Montauban, France
| | - Maxime Le Coz
- Archean LABS, 20 place Prax-Paris, 82000 Montauban, France
| | - Charlotte Azzopardi
- Ecole d'Audioprothèse de Cahors, Université Toulouse III Paul Sabatier, 31062 Toulouse, France
| | - Michael A Stone
- Manchester Centre for Audiology and Deafness, School of Health Sciences, University of Manchester, Oxford Road, Manchester, M139PL, United Kingdom
| | - Christian Füllgrabe
- School of Sport, Exercise and Health Sciences, Ashby Road, Loughborough University, Loughborough LE11 3TU, United , , , ,
| |
Collapse
|