1
|
Zhong D, Hu Y, Zhao K, Deng W, Hou P, Zhang J. Accurate separation of mixed high-dimension optical-chaotic signals using optical reservoir computing based on optically pumped VCSELs. OPTICS EXPRESS 2022; 30:39561-39581. [PMID: 36298905 DOI: 10.1364/oe.470857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Accepted: 09/28/2022] [Indexed: 06/16/2023]
Abstract
In this work, with the mixing fractions being known in advance or unknown, the schemes and theories for the separations of two groups of the mixed optical chaotic signals are proposed in detail, using the VCSEL-based reservoir computing (RC) systems. Here, two groups of the mixed optical chaotic signals are linearly combined with many beams of the chaotic x-polarization components (X-PCs) and Y-PCs emitted by the optically pumped spin-VCSELs operation alone. Two parallel reservoirs are performed by using the chaotic X-PC and Y-PC output by the optically pumped spin-VCSEL with both optical feedback and optical injection. Moreover, we further demonstrate the separation performances of the mixed chaotic signal linearly combined with no more than three beams of the chaotic X-PC or Y-PC. We find that two groups of the mixed optical chaos signals can be effectively separated by using two reservoirs in single RC system based on optically pumped Spin-VCSEL and their corresponding separated errors characterized by the training errors are no more than 0.093, when the mixing fractions are known as a certain value in advance. If the mixing fractions are unknown, we utilize two cascaded RC systems based on optically pumped Spin-VCSELs to separate each group of the mixed optical signals. The mixing fractions can be accurate predicted by using two parallel reservoirs in the first RC system. Based on the values of the predictive mixing fractions, two groups of the mixed optical chaos signals can be effectively separated by utilizing two parallel reservoirs in the second RC system, and their separated errors also are no more than 0.093. In the same way, the mixed optical chaos signal linearly superimposed with more than three beams of optical chaotic signals can be effectively separated. The method and idea for separation of complex optical chaos signals proposed by this paper may provide an impact to development of novel principles of multiple access and demultiplexing in multi-channel chaotic cryptography communication.
Collapse
|
2
|
Geravanchizadeh M, Zakeri S. Ear-EEG-based binaural speech enhancement (ee-BSE) using auditory attention detection and audiometric characteristics of hearing-impaired subjects. J Neural Eng 2021; 18. [PMID: 34289464 DOI: 10.1088/1741-2552/ac16b4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2020] [Accepted: 07/21/2021] [Indexed: 11/11/2022]
Abstract
Objective. Speech perception in cocktail party scenarios has been the concern of a group of researchers who are involved with the design of hearing-aid devices.Approach. In this paper, a new unified ear-EEG-based binaural speech enhancement system is introduced for hearing-impaired (HI) listeners. The proposed model, which is based on auditory attention detection (AAD) and individual hearing threshold (HT) characteristics, has four main processing stages. In the binaural processing stage, a system based on the deep neural network is trained to estimate auditory ratio masks for each of the speakers in the mixture signal. In the EEG processing stage, AAD is employed to select one ratio mask corresponding to the attended speech. Here, the same EEG data is also used to predict the HTs of listeners who participated in the EEG recordings. The third stage, called insertion gain computation, concerns the calculation of a special amplification gain based on individual HTs. Finally, in the selection-resynthesis-amplification stage, the attended speech signals of the target are resynthesized based on the selected auditory mask and then are amplified using the computed insertion gain.Main results. The detection of the attended speech and the HTs are achieved by classifiers that are trained with features extracted from the scalp EEG or the ear EEG signals. The results of evaluating AAD and HT detection show high detection accuracies. The systematic evaluations of the proposed system yield substantial intelligibility and quality improvements for the HI and normal-hearingaudiograms.Significance. The AAD method determines the direction of attention from single-trial EEG signals without access to audio signals of the speakers. The amplification procedure could be adjusted for each subject based on the individual HTs. The present model has the potential to be considered as an important processing tool to personalize the neuro-steered hearing aids.
Collapse
Affiliation(s)
- Masoud Geravanchizadeh
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666-15813, Iran
| | - Sahar Zakeri
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666-15813, Iran
| |
Collapse
|
3
|
Speech Enhancement for Hearing Aids with Deep Learning on Environmental Noises. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10176077] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Hearing aids are small electronic devices designed to improve hearing for persons with impaired hearing, using sophisticated audio signal processing algorithms and technologies. In general, the speech enhancement algorithms in hearing aids remove the environmental noise and enhance speech while still giving consideration to hearing characteristics and the environmental surroundings. In this study, a speech enhancement algorithm was proposed to improve speech quality in a hearing aid environment by applying noise reduction algorithms with deep neural network learning based on noise classification. In order to evaluate the speech enhancement in an actual hearing aid environment, ten types of noise were self-recorded and classified using convolutional neural networks. In addition, noise reduction for speech enhancement in the hearing aid were applied by deep neural networks based on the noise classification. As a result, the speech quality based on the speech enhancements removed using the deep neural networks—and associated environmental noise classification—exhibited a significant improvement over that of the conventional hearing aid algorithm. The improved speech quality was also evaluated by objective measure through the perceptual evaluation of speech quality score, the short-time objective intelligibility score, the overall quality composite measure, and the log likelihood ratio score.
Collapse
|
4
|
Krishnagopal S, Girvan M, Ott E, Hunt BR. Separation of chaotic signals by reservoir computing. CHAOS (WOODBURY, N.Y.) 2020; 30:023123. [PMID: 32113243 DOI: 10.1063/1.5132766] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2019] [Accepted: 01/20/2020] [Indexed: 06/10/2023]
Abstract
We demonstrate the utility of machine learning in the separation of superimposed chaotic signals using a technique called reservoir computing. We assume no knowledge of the dynamical equations that produce the signals and require only training data consisting of finite-time samples of the component signals. We test our method on signals that are formed as linear combinations of signals from two Lorenz systems with different parameters. Comparing our nonlinear method with the optimal linear solution to the separation problem, the Wiener filter, we find that our method significantly outperforms the Wiener filter in all the scenarios we study. Furthermore, this difference is particularly striking when the component signals have similar frequency spectra. Indeed, our method works well when the component frequency spectra are indistinguishable-a case where a Wiener filter performs essentially no separation.
Collapse
Affiliation(s)
| | | | - Edward Ott
- University of Maryland, College Park, Maryland 20742, USA
| | - Brian R Hunt
- University of Maryland, College Park, Maryland 20742, USA
| |
Collapse
|
5
|
Lu Z, Kim JZ, Bassett DS. Supervised chaotic source separation by a tank of water. CHAOS (WOODBURY, N.Y.) 2020; 30:021101. [PMID: 32113226 PMCID: PMC7007304 DOI: 10.1063/1.5142462] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/13/2019] [Accepted: 12/27/2019] [Indexed: 06/10/2023]
Abstract
Whether listening to overlapping conversations in a crowded room or recording the simultaneous electrical activity of millions of neurons, the natural world abounds with sparse measurements of complex overlapping signals that arise from dynamical processes. While tools that separate mixed signals into linear sources have proven necessary and useful, the underlying equational forms of most natural signals are unknown and nonlinear. Hence, there is a need for a framework that is general enough to extract sources without knowledge of their generating equations and flexible enough to accommodate nonlinear, even chaotic, sources. Here, we provide such a framework, where the sources are chaotic trajectories from independently evolving dynamical systems. We consider the mixture signal as the sum of two chaotic trajectories and propose a supervised learning scheme that extracts the chaotic trajectories from their mixture. Specifically, we recruit a complex dynamical system as an intermediate processor that is constantly driven by the mixture. We then obtain the separated chaotic trajectories based on this intermediate system by training the proper output functions. To demonstrate the generalizability of this framework in silico, we employ a tank of water as the intermediate system and show its success in separating two-part mixtures of various chaotic trajectories. Finally, we relate the underlying mechanism of this method to the state-observer problem. This relation provides a quantitative theory that explains the performance of our method, and why separation is difficult when two source signals are trajectories from the same chaotic system.
Collapse
Affiliation(s)
- Zhixin Lu
- Department of Bioengineering, School of Engineering and Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Jason Z Kim
- Department of Bioengineering, School of Engineering and Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| | - Danielle S Bassett
- Department of Bioengineering, School of Engineering and Applied Sciences, University of Pennsylvania, Philadelphia, Pennsylvania 19104, USA
| |
Collapse
|
6
|
Bramsløw L, Vatti M, Rossing R, Naithani G, Henrik Pontoppidan N. A Competing Voices Test for Hearing-Impaired Listeners Applied to Spatial Separation and Ideal Time-Frequency Masks. Trends Hear 2019; 23:2331216519848288. [PMID: 31104580 PMCID: PMC6610337 DOI: 10.1177/2331216519848288] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
People with hearing impairment find competing voices scenarios to be challenging,
both with respect to switching attention from one talker to the other, as well
as maintaining attention. With the Danish competing voices test (CVT) presented
here, the dual-attention skills can be assessed. The CVT provides sentences
spoken by three male and three female talkers, played in sentence pairs. The
task of the listener is to repeat the target sentence from the sentence pair
based on cueing either before or after playback. One potential way of assisting
segregation of two talkers is to take advantage of spatial unmasking by
presenting one talker per ear after application of time-frequency masks for
separating the mixture. Using the CVT, this study evaluated four spatial
conditions in 14 moderate-to-severely hearing-impaired listeners to establish
benchmark results for this type of algorithm applied to hearing-impaired
listeners. The four spatial conditions were as follows: summed (diotic),
separate, the ideal ratio mask, and the ideal binary mask. The results show that
the test is sensitive to the change in spatial condition. The temporal position
of the cue has a large impact, as cueing the target talker before playback
focuses the attention toward the target, whereas cueing after playback requires
equal attention to the two talkers, which is more difficult. Furthermore, both
applied ideal masks show test scores very close to the ideal separate spatial
condition, suggesting that this technique is useful for future separation
algorithms using estimated rather than ideal masks.
Collapse
Affiliation(s)
| | | | | | - Gaurav Naithani
- 2 Laboratory of Signal Processing, Tampere University, Finland
| | | |
Collapse
|
7
|
Copy-move detection of digital audio based on multi-feature decision. JOURNAL OF INFORMATION SECURITY AND APPLICATIONS 2018. [DOI: 10.1016/j.jisa.2018.10.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
|
8
|
Wang D, Chen J. Supervised Speech Separation Based on Deep Learning: An Overview. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2018; 26:1702-1726. [PMID: 31223631 PMCID: PMC6586438 DOI: 10.1109/taslp.2018.2842159] [Citation(s) in RCA: 119] [Impact Index Per Article: 19.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Speech separation is the task of separating target speech from background interference. Traditionally, speech separation is studied as a signal processing problem. A more recent approach formulates speech separation as a supervised learning problem, where the discriminative patterns of speech, speakers, and background noise are learned from training data. Over the past decade, many supervised separation algorithms have been put forward. In particular, the recent introduction of deep learning to supervised speech separation has dramatically accelerated progress and boosted separation performance. This paper provides a comprehensive overview of the research on deep learning based supervised speech separation in the last several years. We first introduce the background of speech separation and the formulation of supervised separation. Then, we discuss three main components of supervised separation: learning machines, training targets, and acoustic features. Much of the overview is on separation algorithms where we review monaural methods, including speech enhancement (speech-nonspeech separation), speaker separation (multitalker separation), and speech dereverberation, as well as multimicrophone techniques. The important issue of generalization, unique to supervised learning, is discussed. This overview provides a historical perspective on how advances are made. In addition, we discuss a number of conceptual issues, including what constitutes the target source.
Collapse
Affiliation(s)
- DeLiang Wang
- Department of Computer Science and Engineering and the Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, OH 43210 USA, and also with the Center of Intelligent Acoustics and Immersive Communications, Northwestern Polytechnical University, Xi'an 710072, China
| | - Jitong Chen
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210 USA. He is now with Silicon Valley AI Lab, Baidu Research, Sunnyvale, CA 94089 USA
| |
Collapse
|
9
|
Bramsløw L, Naithani G, Hafez A, Barker T, Pontoppidan NH, Virtanen T. Improving competing voices segregation for hearing impaired listeners using a low-latency deep neural network algorithm. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2018; 144:172. [PMID: 30075667 DOI: 10.1121/1.5045322] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Hearing aid users are challenged in listening situations with noise and especially speech-on-speech situations with two or more competing voices. Specifically, the task of attending to and segregating two competing voices is particularly hard, unlike for normal-hearing listeners, as shown in a small sub-experiment. In the main experiment, the competing voices benefit of a deep neural network (DNN) based stream segregation enhancement algorithm was tested on hearing-impaired listeners. A mixture of two voices was separated using a DNN and presented to the two ears as individual streams and tested for word score. Compared to the unseparated mixture, there was a 13%-point benefit from the separation, while attending to both voices. If only one output was selected as in a traditional target-masker scenario, a larger benefit of 37%-points was found. The results agreed well with objective metrics and show that for hearing-impaired listeners, DNNs have a large potential for improving stream segregation and speech intelligibility in difficult scenarios with two equally important targets without any prior selection of a primary target stream. An even higher benefit can be obtained if the user can select the preferred target via remote control.
Collapse
Affiliation(s)
- Lars Bramsløw
- Eriksholm Research Centre, Oticon A/S, Rørtangvej 20, DK-3070 Snekkersten, Denmark
| | - Gaurav Naithani
- Tampere University of Technology, Laboratory of Signal Processing, Tampere, P.O. Box 553, FI-33101 Tampere, Finland
| | - Atefeh Hafez
- Eriksholm Research Centre, Oticon A/S, Rørtangvej 20, DK-3070 Snekkersten, Denmark
| | - Tom Barker
- Tampere University of Technology, Laboratory of Signal Processing, Tampere, P.O. Box 553, FI-33101 Tampere, Finland
| | | | - Tuomas Virtanen
- Tampere University of Technology, Laboratory of Signal Processing, Tampere, P.O. Box 553, FI-33101 Tampere, Finland
| |
Collapse
|
10
|
Bentsen T, May T, Kressner AA, Dau T. The benefit of combining a deep neural network architecture with ideal ratio mask estimation in computational speech segregation to improve speech intelligibility. PLoS One 2018; 13:e0196924. [PMID: 29763459 PMCID: PMC5953465 DOI: 10.1371/journal.pone.0196924] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2018] [Accepted: 04/23/2018] [Indexed: 11/19/2022] Open
Abstract
Computational speech segregation attempts to automatically separate speech from noise. This is challenging in conditions with interfering talkers and low signal-to-noise ratios. Recent approaches have adopted deep neural networks and successfully demonstrated speech intelligibility improvements. A selection of components may be responsible for the success with these state-of-the-art approaches: the system architecture, a time frame concatenation technique and the learning objective. The aim of this study was to explore the roles and the relative contributions of these components by measuring speech intelligibility in normal-hearing listeners. A substantial improvement of 25.4 percentage points in speech intelligibility scores was found going from a subband-based architecture, in which a Gaussian Mixture Model-based classifier predicts the distributions of speech and noise for each frequency channel, to a state-of-the-art deep neural network-based architecture. Another improvement of 13.9 percentage points was obtained by changing the learning objective from the ideal binary mask, in which individual time-frequency units are labeled as either speech- or noise-dominated, to the ideal ratio mask, where the units are assigned a continuous value between zero and one. Therefore, both components play significant roles and by combining them, speech intelligibility improvements were obtained in a six-talker condition at a low signal-to-noise ratio.
Collapse
Affiliation(s)
- Thomas Bentsen
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
- * E-mail:
| | - Tobias May
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
| | - Abigail A. Kressner
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
| | - Torsten Dau
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, Kgs. Lyngby, Denmark
| |
Collapse
|
11
|
Bentsen T, Kressner AA, Dau T, May T. The impact of exploiting spectro-temporal context in computational speech segregation. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2018; 143:248. [PMID: 29390791 DOI: 10.1121/1.5020273] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Computational speech segregation aims to automatically segregate speech from interfering noise, often by employing ideal binary mask estimation. Several studies have tried to exploit contextual information in speech to improve mask estimation accuracy by using two frequently-used strategies that (1) incorporate delta features and (2) employ support vector machine (SVM) based integration. In this study, two experiments were conducted. In Experiment I, the impact of exploiting spectro-temporal context using these strategies was investigated in stationary and six-talker noise. In Experiment II, the delta features were explored in detail and tested in a setup that considered novel noise segments of the six-talker noise. Computing delta features led to higher intelligibility than employing SVM based integration and intelligibility increased with the amount of spectral information exploited via the delta features. The system did not, however, generalize well to novel segments of this noise type. Measured intelligibility was subsequently compared to extended short-term objective intelligibility, hit-false alarm rate, and the amount of mask clustering. None of these objective measures alone could account for measured intelligibility. The findings may have implications for the design of speech segregation systems, and for the selection of a cost function that correlates with intelligibility.
Collapse
Affiliation(s)
- Thomas Bentsen
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, 2800 Kongens Lyngby, Denmark
| | - Abigail A Kressner
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, 2800 Kongens Lyngby, Denmark
| | - Torsten Dau
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, 2800 Kongens Lyngby, Denmark
| | - Tobias May
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, 2800 Kongens Lyngby, Denmark
| |
Collapse
|
12
|
Kressner AA, May T, Rozell CJ. Outcome measures based on classification performance fail to predict the intelligibility of binary-masked speech. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2016; 139:3033. [PMID: 27369123 DOI: 10.1121/1.4952439] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
To date, the most commonly used outcome measure for assessing ideal binary mask estimation algorithms is based on the difference between the hit rate and the false alarm rate (H-FA). Recently, the error distribution has been shown to substantially affect intelligibility. However, H-FA treats each mask unit independently and does not take into account how errors are distributed. Alternatively, algorithms can be evaluated with the short-time objective intelligibility (STOI) metric using the reconstructed speech. This study investigates the ability of H-FA and STOI to predict intelligibility for binary-masked speech using masks with different error distributions. The results demonstrate the inability of H-FA to predict the behavioral intelligibility and also illustrate the limitations of STOI. Since every estimation algorithm will make errors that are distributed in different ways, performance evaluations should not be made solely on the basis of these metrics.
Collapse
Affiliation(s)
- Abigail Anne Kressner
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark
| | - Tobias May
- Hearing Systems Group, Department of Electrical Engineering, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark
| | - Christopher J Rozell
- School of Electrical and Computer Engineering, 777 Atlantic Drive NW, Georgia Institute of Technology, Atlanta, Georgia 30332, USA
| |
Collapse
|
13
|
Zhang XL, Wang D. A Deep Ensemble Learning Method for Monaural Speech Separation. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2016; 24:967-977. [PMID: 27917394 PMCID: PMC5131883 DOI: 10.1109/taslp.2016.2536478] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/14/2023]
Abstract
Monaural speech separation is a fundamental problem in robust speech processing. Recently, deep neural network (DNN)-based speech separation methods, which predict either clean speech or an ideal time-frequency mask, have demonstrated remarkable performance improvement. However, a single DNN with a given window length does not leverage contextual information sufficiently, and the differences between the two optimization objectives are not well understood. In this paper, we propose a deep ensemble method, named multicontext networks, to address monaural speech separation. The first multicontext network averages the outputs of multiple DNNs whose inputs employ different window lengths. The second multicontext network is a stack of multiple DNNs. Each DNN in a module of the stack takes the concatenation of original acoustic features and expansion of the soft output of the lower module as its input, and predicts the ratio mask of the target speaker; the DNNs in the same module employ different contexts. We have conducted extensive experiments with three speech corpora. The results demonstrate the effectiveness of the proposed method. We have also compared the two optimization objectives systematically and found that predicting the ideal time-frequency mask is more efficient in utilizing clean training speech, while predicting clean speech is less sensitive to SNR variations.
Collapse
Affiliation(s)
- Xiao-Lei Zhang
- Department of Computer Science and Engineering and the Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, OH 43210 USA, and also with the Center of Intelligent Acoustics and Immersive Communications, Northwestern Polytechnical University, Xi'an 710072, China
| | - DeLiang Wang
- Department of Computer Science and Engineering and the Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, OH 43210 USA, and also with the Center of Intelligent Acoustics and Immersive Communications, Northwestern Polytechnical University, Xi'an 710072, China
| |
Collapse
|
14
|
Bhargava S, Blättler F, Kollmorgen S, Liu SC, Hahnloser RHR. Linear Methods for Efficient and Fast Separation of Two Sources Recorded with a Single Microphone. Neural Comput 2015; 27:2231-59. [PMID: 26313599 DOI: 10.1162/neco_a_00776] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
This letter addresses the problem of separating two speakers from a single microphone recording. Three linear methods are tested for source separation, all of which operate directly on sound spectrograms: (1) eigenmode analysis of covariance difference to identify spectro-temporal features associated with large variance for one source and small variance for the other source; (2) maximum likelihood demixing in which the mixture is modeled as the sum of two gaussian signals and maximum likelihood is used to identify the most likely sources; and (3) suppression-regression, in which autoregressive models are trained to reproduce one source and suppress the other. These linear approaches are tested on the problem of separating a known male from a known female speaker. The performance of these algorithms is assessed in terms of the residual error of estimated source spectrograms, waveform signal-to-noise ratio, and perceptual evaluation of speech quality scores. This work shows that the algorithms compare favorably to nonlinear approaches such as nonnegative sparse coding in terms of simplicity, performance, and suitability for real-time implementations, and they provide benchmark solutions for monaural source separation tasks.
Collapse
Affiliation(s)
- Saurabh Bhargava
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
| | - Florian Blättler
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
| | - Sepp Kollmorgen
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
| | - Shih-Chii Liu
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
| | - Richard H R Hahnloser
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, 8057, Switzerland
| |
Collapse
|
15
|
Kressner AA, Rozell CJ. Structure in time-frequency binary masking errors and its impact on speech intelligibility. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2015; 137:2025-2035. [PMID: 25920853 DOI: 10.1121/1.4916271] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
Although requiring prior knowledge makes the ideal binary mask an impractical algorithm, substantial increases in measured intelligibility make it a desirable benchmark. While this benchmark has been studied extensively, many questions remain about the factors that influence the intelligibility of binary-masked speech with non-ideal masks. To date, researchers have used primarily uniformly random, uncorrelated mask errors and independently presented error types (i.e., false positives and negatives) to characterize the influence of estimation errors on intelligibility. However, practical estimation algorithms produce masks that contain errors of both types and with non-trivial amounts of structure. This paper introduces an investigation framework for binary masks and presents listener studies that use this framework to illustrate how interactions between error types and structure affect intelligibility. First, this study demonstrates that clustering (i.e., a form of structure) of mask errors reduces intelligibility. Furthermore, while previous research has suggested that false positives are more detrimental to intelligibility than false negatives, this study indicates that false negatives can be equally detrimental to intelligibility when they contain structure or when both error types are present. Finally, this study shows that listeners tolerate fewer mask errors when both types of errors are present, especially when the errors contain structure.
Collapse
Affiliation(s)
- Abigail A Kressner
- School of Electrical and Computer Engineering, 777 Atlantic Drive Northwest, Georgia Institute of Technology, Atlanta, Georgia 30332
| | - Christopher J Rozell
- School of Electrical and Computer Engineering, 777 Atlantic Drive Northwest, Georgia Institute of Technology, Atlanta, Georgia 30332
| |
Collapse
|
16
|
May T, Dau T. Computational speech segregation based on an auditory-inspired modulation analysis. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 136:3350. [PMID: 25480079 DOI: 10.1121/1.4901711] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/04/2023]
Abstract
A monaural speech segregation system is presented that estimates the ideal binary mask from noisy speech based on the supervised learning of amplitude modulation spectrogram (AMS) features. Instead of using linearly scaled modulation filters with constant absolute bandwidth, an auditory-inspired modulation filterbank with logarithmically scaled filters is employed. To reduce the dependency of the AMS features on the overall background noise level, a feature normalization stage is applied. In addition, a spectro-temporal integration stage is incorporated in order to exploit the context information about speech activity present in neighboring time-frequency units. In order to evaluate the generalization performance of the system to unseen acoustic conditions, the speech segregation system is trained with a limited set of low signal-to-noise ratio (SNR) conditions, but tested over a wide range of SNRs up to 20 dB. A systematic evaluation of the system demonstrates that auditory-inspired modulation processing can substantially improve the mask estimation accuracy in the presence of stationary and fluctuating interferers.
Collapse
Affiliation(s)
- Tobias May
- Centre for Applied Hearing Research, Department of Electrical Engineering, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark
| | - Torsten Dau
- Centre for Applied Hearing Research, Department of Electrical Engineering, Technical University of Denmark, DK-2800 Kongens Lyngby, Denmark
| |
Collapse
|
17
|
Wang Y, Narayanan A, Wang D. On Training Targets for Supervised Speech Separation. IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING 2014; 22:1849-1858. [PMID: 25599083 PMCID: PMC4293540 DOI: 10.1109/taslp.2014.2352935] [Citation(s) in RCA: 113] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/09/2023]
Abstract
Formulation of speech separation as a supervised learning problem has shown considerable promise. In its simplest form, a supervised learning algorithm, typically a deep neural network, is trained to learn a mapping from noisy features to a time-frequency representation of the target of interest. Traditionally, the ideal binary mask (IBM) is used as the target because of its simplicity and large speech intelligibility gains. The supervised learning framework, however, is not restricted to the use of binary targets. In this study, we evaluate and compare separation results by using different training targets, including the IBM, the target binary mask, the ideal ratio mask (IRM), the short-time Fourier transform spectral magnitude and its corresponding mask (FFT-MASK), and the Gammatone frequency power spectrum. Our results in various test conditions reveal that the two ratio mask targets, the IRM and the FFT-MASK, outperform the other targets in terms of objective intelligibility and quality metrics. In addition, we find that masking based targets, in general, are significantly better than spectral envelope based targets. We also present comparisons with recent methods in non-negative matrix factorization and speech enhancement, which show clear performance advantages of supervised speech separation.
Collapse
Affiliation(s)
- Yuxuan Wang
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210 USA
| | - Arun Narayanan
- Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43210 USA
| | - DeLiang Wang
- Department of Computer Science and Engineering and the Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, OH 43210 USA
| |
Collapse
|
18
|
Gao B, Woo WL, Khor LC. Cochleagram-based audio pattern separation using two-dimensional non-negative matrix factorization with automatic sparsity adaptation. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 135:1171-1185. [PMID: 24606260 DOI: 10.1121/1.4864294] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
An unsupervised single channel audio separation method from pattern recognition viewpoint is presented. The proposed method does not require training knowledge and the separation system is based on non-uniform time-frequency (TF) analysis and feature extraction. Unlike conventional research that concentrates on the use of spectrogram or its variants, the proposed separation algorithm uses an alternative TF representation based on the gammatone filterbank. In particular, the monaural mixed audio signal is shown to be considerably more separable in this non-uniform TF domain. The analysis of signal separability to verify this finding is provided. In addition, a variational Bayesian approach is derived to learn the sparsity parameters for optimizing the matrix factorization. Experimental tests have been conducted, which show that the extraction of the spectral dictionary and temporal codes is more efficient using sparsity learning and subsequently leads to better separation performance.
Collapse
Affiliation(s)
- Bin Gao
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, People's Republic of China
| | - W L Woo
- School of Electrical and Electronic Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, United Kingdom
| | - L C Khor
- School of Electrical and Electronic Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, United Kingdom
| |
Collapse
|
19
|
Hebb AO, Zhang JJ, Mahoor MH, Tsiokos C, Matlack C, Chizeck HJ, Pouratian N. Creating the feedback loop: closed-loop neurostimulation. Neurosurg Clin N Am 2013; 25:187-204. [PMID: 24262909 DOI: 10.1016/j.nec.2013.08.006] [Citation(s) in RCA: 47] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
Abstract
Current DBS therapy delivers a train of electrical pulses at set stimulation parameters. This open-loop design is effective for movement disorders, but therapy may be further optimized by a closed loop design. The technology to record biosignals has outpaced our understanding of their relationship to the clinical state of the whole person. Neuronal oscillations may represent or facilitate the cooperative functioning of brain ensembles, and may provide critical information to customize neuromodulation therapy. This review addresses advances to date, not of the technology per se, but of the strategies to apply neuronal signals to trigger or modulate stimulation systems.
Collapse
Affiliation(s)
- Adam O Hebb
- Colorado Neurological Institute, Department of Electrical and Computer Engineering, University of Denver, 499 E Hampden Ave Ste, 220 Englewood, CO 80113, USA.
| | | | | | | | | | | | | |
Collapse
|
20
|
Healy EW, Yoho SE, Wang Y, Wang D. An algorithm to improve speech recognition in noise for hearing-impaired listeners. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2013; 134:3029-38. [PMID: 24116438 PMCID: PMC3799726 DOI: 10.1121/1.4820893] [Citation(s) in RCA: 34] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/20/2013] [Revised: 08/22/2013] [Accepted: 08/26/2013] [Indexed: 05/09/2023]
Abstract
Despite considerable effort, monaural (single-microphone) algorithms capable of increasing the intelligibility of speech in noise have remained elusive. Successful development of such an algorithm is especially important for hearing-impaired (HI) listeners, given their particular difficulty in noisy backgrounds. In the current study, an algorithm based on binary masking was developed to separate speech from noise. Unlike the ideal binary mask, which requires prior knowledge of the premixed signals, the masks used to segregate speech from noise in the current study were estimated by training the algorithm on speech not used during testing. Sentences were mixed with speech-shaped noise and with babble at various signal-to-noise ratios (SNRs). Testing using normal-hearing and HI listeners indicated that intelligibility increased following processing in all conditions. These increases were larger for HI listeners, for the modulated background, and for the least-favorable SNRs. They were also often substantial, allowing several HI listeners to improve intelligibility from scores near zero to values above 70%.
Collapse
Affiliation(s)
- Eric W Healy
- Department of Speech and Hearing Science, and Center for Cognitive and Brain Sciences, The Ohio State University, Columbus, Ohio 43210
| | | | | | | |
Collapse
|
21
|
|