1
|
Pan X, Feng T, Zhang N. PVGAN: A Pathological Voice Generation Model Incorporating a Progressive Nesting Strategy. J Voice 2023:S0892-1997(23)00315-6. [PMID: 37940422 DOI: 10.1016/j.jvoice.2023.10.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 10/04/2023] [Accepted: 10/04/2023] [Indexed: 11/10/2023]
Abstract
The voice generation task is to solve the problem of limited samples in the voice dataset using computer technology. By increasing the number of samples, the accuracy of voice disorder diagnosis can be improved, which has a wide range of application value in medical diagnosis and other fields. At present, there are insufficient models for detailed features such as pitch, timbre, and different frequency components in pathological voice data. Therefore, this paper proposes a PVGAN network for learning different frequency information of audio to generate pathological voice data. The proposed network captures the multi-scale features and different periodic patterns of audio signals by designing multiscale perceptual residual blocks and periodic discriminators. At the same time, a progressive nesting strategy was proposed to combine the generator and the discriminator to improve the learning ability of different resolution information. In addition, a latent mapping network is designed to fuse the latent vector with the condition information to generate sound features related to specific diseases or pathological states. The loss function is optimized to further improve the model performance. On the Saarbruecken Voice Database(SVD), the average values of each index of the data generated after training with different pathological types as conditional information are similar to the original data. Finally, the generated data were used to expand the SVD dataset, and the accuracy of the two classification experiments was improved to a certain extent.
Collapse
Affiliation(s)
- Xiaoying Pan
- Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China; School of Computer Science & Technology, Xi'an University of Posts and Telecommunications, Xi'an 710121, China.
| | - Tong Feng
- Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China; School of Computer Science & Technology, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
| | - Nijuan Zhang
- Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China; School of Computer Science & Technology, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
| |
Collapse
|
2
|
Zakariah M, B R, Ajmi Alotaibi Y, Guo Y, Tran-Trung K, Elahi MM. An Analytical Study of Speech Pathology Detection Based on MFCC and Deep Neural Networks. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:7814952. [PMID: 35529259 PMCID: PMC9071878 DOI: 10.1155/2022/7814952] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 02/17/2022] [Accepted: 03/07/2022] [Indexed: 11/17/2022]
Abstract
Diseases of internal organs other than the vocal folds can also affect a person's voice. As a result, voice problems are on the rise, even though they are frequently overlooked. According to a recent study, voice pathology detection systems can successfully help the assessment of voice abnormalities and enable the early diagnosis of voice pathology. For instance, in the early identification and diagnosis of voice problems, the automatic system for distinguishing healthy and diseased voices has gotten much attention. As a result, artificial intelligence-assisted voice analysis brings up new possibilities in healthcare. The work was aimed at assessing the utility of several automatic speech signal analysis methods for diagnosing voice disorders and suggesting a strategy for classifying healthy and diseased voices. The proposed framework integrates the efficacy of three voice characteristics: chroma, mel spectrogram, and mel frequency cepstral coefficient (MFCC). We also designed a deep neural network (DNN) capable of learning from the retrieved data and producing a highly accurate voice-based disease prediction model. The study describes a series of studies using the Saarbruecken Voice Database (SVD) to detect abnormal voices. The model was developed and tested using the vowels /a/, /i/, and /u/ pronounced in high, low, and average pitches. We also maintained the "continuous sentence" audio files collected from SVD to select how well the developed model generalizes to completely new data. The highest accuracy achieved was 77.49%, superior to prior attempts in the same domain. Additionally, the model attains an accuracy of 88.01% by integrating speaker gender information. The designed model trained on selected diseases can also obtain a maximum accuracy of 96.77% (cordectomy × healthy). As a result, the suggested framework is the best fit for the healthcare industry.
Collapse
Affiliation(s)
- Mohammed Zakariah
- Department of Computer Science, College of Computer and Information Sciences, King Saud University, P.O. Box 57168, Riyadh 21574, Saudi Arabia
| | - Reshma B
- Division of Electronics Engineering, School of Engineering, Cochin University of Science and Technology, India
| | - Yousef Ajmi Alotaibi
- Department of Computer Engineering, College of Computer and Information Sciences, King Saud University, P.O. Box 57168, Riyadh 21574, Saudi Arabia
| | | | - Kiet Tran-Trung
- Faculty of Computer Science, Ho Chi Minh City Open University, 97 Vo Van Tan, Ward Vo Thi Sau, District 3, Ho Chi Minh City Code postal: 70000, Vietnam
| | - Mohammad Mamun Elahi
- Department of Computer Science and Engineering, United International University, Dhaka, Bangladesh
| |
Collapse
|
3
|
Albert G, Arnocky S, Puts DA, Hodges-Simeon CR. Can listeners assess men's self-reported health from their voice? EVOL HUM BEHAV 2021. [DOI: 10.1016/j.evolhumbehav.2020.08.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
|
4
|
Robust Detection of COVID-19 in Cough Sounds: Using Recurrence Dynamics and Variable Markov Model. SN COMPUTER SCIENCE 2021; 2:34. [PMID: 33458700 PMCID: PMC7802616 DOI: 10.1007/s42979-020-00422-6] [Citation(s) in RCA: 44] [Impact Index Per Article: 14.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/03/2020] [Accepted: 12/08/2020] [Indexed: 01/31/2023]
Abstract
COVID-19, otherwise known as the coronavirus, has precipitated the world into a pandemic that has infected, as of the time of writing, more than 10 million persons worldwide and caused the death of more than 500,000 persons. Early symptoms of the virus include trouble breathing, fever and fatigue and over 60% of people experience a dry cough. Due to the devastating impact of COVID-19 and the tragic loss of lives, it is of the utmost urgency to develop methods for the early detection of the disease that may help limit its spread as well as aid in the development of targeted solutions. Coughs and other vocal sounds contain pulmonary health information that can be used for diagnostic purposes, and recent studies in chaotic dynamics have shown that nonlinear phenomena exist in vocal signals. The present work investigates the use of symbolic recurrence quantification measures with MFCC features for the automatic detection of COVID-19 in cough sounds of healthy and sick individuals. Our performance evaluation reveals that our symbolic dynamics measures capture the complex dynamics in the vocal sounds and are highly effective at discriminating sick and healthy coughs. We apply our method to sustained vowel 'ah' recordings, and show that our model is robust for the detection of the disease in sustained vowel utterances as well. Furthermore, we introduce a robust novel method of informative undersampling using information rate to deal with the imbalance in our dataset, due to the unavailability of an equal number of sick and healthy recordings. The proposed model achieves a mean classification performance of 97% and 99%, and a mean F 1 -score of 91% and 89% after optimization, for coughs and sustained vowels, respectively.
Collapse
|
5
|
On the design of automatic voice condition analysis systems. Part I: Review of concepts and an insight to the state of the art. Biomed Signal Process Control 2019. [DOI: 10.1016/j.bspc.2018.12.024] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
|
6
|
Sodre B, Rosa M, Dassie-Leite A. Evaluating the use of neural networks and acoustic measurements to identify laryngeal pathologies. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:4590-4593. [PMID: 29060919 DOI: 10.1109/embc.2017.8037878] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Nineteen acoustical measurements were related to 23 larynx conditions by artificial neural networks (ANNs) and principal component analysis. An exhaustive analysis (combining all possible sets of acoustical measurements as ANN inputs) showed a performance of 99.4% for accuracy and 90.3% for sensitivity and specificity in classifying voice signals into normal and non-normal larynx conditions. In the case of individual larynx condition identification, the general sensitivity drop significantly (6.4%), although some conditions were better identified (including "vocal nodules" and "cysts") then others, reaching 99.8% of sensitivity. We also identified the acoustical measurements that produced the best classification results.
Collapse
|
7
|
Voice data mining for laryngeal pathology assessment. Comput Biol Med 2016; 69:270-6. [DOI: 10.1016/j.compbiomed.2015.07.026] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2014] [Revised: 07/24/2015] [Accepted: 07/29/2015] [Indexed: 11/18/2022]
|
8
|
Lachhab O, Di Martino J, Elhaj EI, Hammouch A. A preliminary study on improving the recognition of esophageal speech using a hybrid system based on statistical voice conversion. SPRINGERPLUS 2015; 4:644. [PMID: 26543778 PMCID: PMC4627987 DOI: 10.1186/s40064-015-1428-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/26/2015] [Accepted: 10/12/2015] [Indexed: 11/23/2022]
Abstract
In this paper, we propose a hybrid system based on a modified statistical GMM voice conversion algorithm for improving the recognition of esophageal speech. This hybrid system aims to compensate for the distorted information present in the esophageal acoustic features by using a voice conversion method. The esophageal speech is converted into a “target” laryngeal speech using an iterative statistical estimation of a transformation function. We did not apply a speech synthesizer for reconstructing the converted speech signal, given that the converted Mel cepstral vectors are used directly as input of our speech recognition system. Furthermore the feature vectors are linearly transformed by the HLDA (heteroscedastic linear discriminant analysis) method to reduce their size in a smaller space having good discriminative properties. The experimental results demonstrate that our proposed system provides an improvement of the phone recognition accuracy with an absolute increase of 3.40 % when compared with the phone recognition accuracy obtained with neither HLDA nor voice conversion.
Collapse
Affiliation(s)
- Othman Lachhab
- LRGE Laboratory, ENSET, Mohammed 5 University, Madinat Al Irfane, Rabat, Morocco
| | | | | | - Ahmed Hammouch
- LRGE Laboratory, ENSET, Mohammed 5 University, Madinat Al Irfane, Rabat, Morocco
| |
Collapse
|
9
|
Kim J, Kumar N, Tsiartas A, Li M, Narayanan SS. Automatic intelligibility classification of sentence-level pathological speech. COMPUT SPEECH LANG 2015; 29:132-144. [PMID: 25414544 DOI: 10.1016/j.csl.2014.02.001] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Pathological speech usually refers to the condition of speech distortion resulting from atypicalities in voice and/or in the articulatory mechanisms owing to disease, illness or other physical or biological insult to the production system. Although automatic evaluation of speech intelligibility and quality could come in handy in these scenarios to assist experts in diagnosis and treatment design, the many sources and types of variability often make it a very challenging computational processing problem. In this work we propose novel sentence-level features to capture abnormal variation in the prosodic, voice quality and pronunciation aspects in pathological speech. In addition, we propose a post-classification posterior smoothing scheme which refines the posterior of a test sample based on the posteriors of other test samples. Finally, we perform feature-level fusions and subsystem decision fusion for arriving at a final intelligibility decision. The performances are tested on two pathological speech datasets, the NKI CCRT Speech Corpus (advanced head and neck cancer) and the TORGO database (cerebral palsy or amyotrophic lateral sclerosis), by evaluating classification accuracy without overlapping subjects' data among training and test partitions. Results show that the feature sets of each of the voice quality subsystem, prosodic subsystem, and pronunciation subsystem, offer significant discriminating power for binary intelligibility classification. We observe that the proposed posterior smoothing in the acoustic space can further reduce classification errors. The smoothed posterior score fusion of subsystems shows the best classification performance (73.5% for unweighted, and 72.8% for weighted, average recalls of the binary classes).
Collapse
Affiliation(s)
- Jangwon Kim
- Signal Analysis and Interpretation Laboratory (SAIL) , University of Southern California, 3710 McClintock Ave., Los Angeles, CA 90089, USA
| | - Naveen Kumar
- Signal Analysis and Interpretation Laboratory (SAIL) , University of Southern California, 3710 McClintock Ave., Los Angeles, CA 90089, USA
| | - Andreas Tsiartas
- Signal Analysis and Interpretation Laboratory (SAIL) , University of Southern California, 3710 McClintock Ave., Los Angeles, CA 90089, USA
| | - Ming Li
- Signal Analysis and Interpretation Laboratory (SAIL) , University of Southern California, 3710 McClintock Ave., Los Angeles, CA 90089, USA
| | - Shrikanth S Narayanan
- Signal Analysis and Interpretation Laboratory (SAIL) , University of Southern California, 3710 McClintock Ave., Los Angeles, CA 90089, USA ; Department of Electrical Engineering, Computer Science, Linguistics and Psychology, University of Southern California (USC), 3620 McClintock Ave., Los Angeles, CA 90089, USA
| |
Collapse
|
10
|
Markaki M, Stylianou Y. Voice Pathology Detection and Discrimination Based on Modulation Spectral Features. ACTA ACUST UNITED AC 2011. [DOI: 10.1109/tasl.2010.2104141] [Citation(s) in RCA: 87] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
11
|
Muhammad G, Mesallam TA, Malki KH, Farahat M, Alsulaiman M, Bukhari M. Formant analysis in dysphonic patients and automatic Arabic digit speech recognition. Biomed Eng Online 2011; 10:41. [PMID: 21624137 PMCID: PMC3120728 DOI: 10.1186/1475-925x-10-41] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2010] [Accepted: 05/30/2011] [Indexed: 11/25/2022] Open
Abstract
Background and objective There has been a growing interest in objective assessment of speech in dysphonic patients for the classification of the type and severity of voice pathologies using automatic speech recognition (ASR). The aim of this work was to study the accuracy of the conventional ASR system (with Mel frequency cepstral coefficients (MFCCs) based front end and hidden Markov model (HMM) based back end) in recognizing the speech characteristics of people with pathological voice. Materials and methods The speech samples of 62 dysphonic patients with six different types of voice disorders and 50 normal subjects were analyzed. The Arabic spoken digits were taken as an input. The distribution of the first four formants of the vowel /a/ was extracted to examine deviation of the formants from normal. Results There was 100% recognition accuracy obtained for Arabic digits spoken by normal speakers. However, there was a significant loss of accuracy in the classifications while spoken by voice disordered subjects. Moreover, no significant improvement in ASR performance was achieved after assessing a subset of the individuals with disordered voices who underwent treatment. Conclusion The results of this study revealed that the current ASR technique is not a reliable tool in recognizing the speech of dysphonic patients.
Collapse
Affiliation(s)
- Ghulam Muhammad
- Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia.
| | | | | | | | | | | |
Collapse
|
12
|
Dibazar AA, Park HO, Berger TW. Nonlinear dynamic modeling of impaired voice. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2010; 2010:2770-2773. [PMID: 21095964 DOI: 10.1109/iembs.2010.5626361] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
This paper presents a nonlinear dynamic model for the purpose of modeling vowels uttered by patients who have problem in the control of voice box muscles. The proposed model will be utilized in the detection of speech pathologies and also automatic speech recognition systems to enhance patients' communication capabilities. The model of this study utilizes feedback, and also a sigmoid nonlinear function which is not included in the linear speech production models. The nonlinear function allows for the higher order dynamics of the signal to be captured and feedback increases dynamicity of the model. The model of the current research was applied to discriminate between few voice pathologies and normal cases. The statistical analysis of the parameters of the trained model showed that these parameters can provide independent and distinct features with which pathological classes can be discriminated.
Collapse
Affiliation(s)
- Alireza A Dibazar
- Department of Biomedical Engineering, University of Southern California, 1042 Downey Way, DRB-140, Los Angeles, CA 90089, USA.
| | | | | |
Collapse
|