Surface Electromyography-Based Recognition, Synthesis, and Perception of Prosodic Subvocal Speech

Identification of the Biomechanical Response of the Muscles That Contract the Most during Disfluencies in Stuttered Speech

Stuttering, affecting approximately 1% of the global population, is a complex speech disorder significantly impacting individuals' quality of life. Prior studies using electromyography (EMG) to examine orofacial muscle activity in stuttering have presented mixed results, highlighting the variability in neuromuscular responses during stuttering episodes. Fifty-five participants with stuttering and 30 individuals without stuttering, aged between 18 and 40, participated in the study. EMG signals from five facial and cervical muscles were recorded during speech tasks and analyzed for mean amplitude and frequency activity in the 5-15 Hz range to identify significant differences. Upon analysis of the 5-15 Hz frequency range, a higher average amplitude was observed in the zygomaticus major muscle for participants while stuttering (p < 0.05). Additionally, when assessing the overall EMG signal amplitude, a higher average amplitude was observed in samples obtained from disfluencies in participants who did not stutter, particularly in the depressor anguli oris muscle (p < 0.05). Significant differences in muscle activity were observed between the two groups, particularly in the depressor anguli oris and zygomaticus major muscles. These results suggest that the underlying neuromuscular mechanisms of stuttering might involve subtle aspects of timing and coordination in muscle activation. Therefore, these findings may contribute to the field of biosensors by providing valuable perspectives on neuromuscular mechanisms and the relevance of electromyography in stuttering research. Further research in this area has the potential to advance the development of biosensor technology for language-related applications and therapeutic interventions in stuttering.

Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

Silent speech interfaces have been pursued to restore spoken communication for individuals with voice disorders and to facilitate intuitive communications when acoustic-based speech communication is unreliable, inappropriate, or undesired. However, the current methodology for silent speech faces several challenges, including bulkiness, obtrusiveness, low accuracy, limited portability, and susceptibility to interferences. In this work, we present a wireless, unobtrusive, and robust silent speech interface for tracking and decoding speech-relevant movements of the temporomandibular joint. Our solution employs a single soft magnetic skin placed behind the ear for wireless and socially acceptable silent speech recognition. The developed system alleviates several concerns associated with existing interfaces based on face-worn sensors, including a large number of sensors, highly visible interfaces on the face, and obtrusive interconnections between sensors and data acquisition components. With machine learning-based signal processing techniques, good speech recognition accuracy is achieved (93.2% accuracy for phonemes, and 87.3% for a list of words from the same viseme groups). Moreover, the reported silent speech interface demonstrates robustness against noises from both ambient environments and users' daily motions. Finally, its potential in assistive technology and human-machine interactions is illustrated through two demonstrations - silent speech enabled smartphone assistants and silent speech enabled drone control.

Electromyogram-Based Lip-Reading via Unobtrusive Dry Electrodes and Machine Learning Methods

Lip-reading provides an effective speech communication interface for people with voice disorders and for intuitive human-machine interactions. Existing systems are generally challenged by bulkiness, obtrusiveness, and poor robustness against environmental interferences. The lack of a truly natural and unobtrusive system for converting lip movements to speech precludes the continuous use and wide-scale deployment of such devices. Here, the design of a hardware-software architecture to capture, analyze, and interpret lip movements associated with either normal or silent speech is presented. The system can recognize different and similar visemes. It is robust in a noisy or dark environment. Self-adhesive, skin-conformable, and semi-transparent dry electrodes are developed to track high-fidelity speech-relevant electromyogram signals without impeding daily activities. The resulting skin-like sensors can form seamless contact with the curvilinear and dynamic surfaces of the skin, which is crucial for a high signal-to-noise ratio and minimal interference. Machine learning algorithms are employed to decode electromyogram signals and convert them to spoken words. Finally, the applications of the developed lip-reading system in augmented reality and medical service are demonstrated, which illustrate the great potential in immersive interaction and healthcare applications.

Deep Learning Methods for Arabic Autoencoder Speech Recognition System for Electro-Larynx Device

Recent advances in speech recognition have achieved remarkable performance comparable with human transcribers’ abilities. But this significant performance is not the same for all the spoken languages. The Arabic language is one of them. Arabic speech recognition is bounded to the lack of suitable datasets. Artificial intelligence algorithms have shown promising capabilities for Arabic speech recognition. Arabic is the official language of 22 countries, and it has been estimated that 400 million people speak the Arabic language worldwide. Speech disabilities have been one of the expanding problems in the last decades, even in kids. Some devices can be used to generate speech for those people. One of these devices is the Servox Digital Electro-Larynx (EL). In this research, we developed an autoencoder with a combination of long short-term memory (LSTM) and gated recurrent units (GRU) models to recognize recorded signals from Servox Digital EL Electro-Larynx. The proposed framework consisted of three steps: denoising, feature extraction, and Arabic speech recognition. The experimental results show 95.31% accuracy for Arabic speech recognition with the proposed model. In this research, we evaluated different combinations of LSTM and GRU for constructing the best autoencoder. A rigorous evaluation process indicates better performance with the use of GRU in both encoder and decoder structures. The proposed model achieved a 4.69% word error rate (WER). Experimental results confirm that the proposed model can be used for developing a real-time app to recognize common Arabic spoken words.

Prediction of Voice Fundamental Frequency and Intensity from Surface Electromyographic Signals of the Face and Neck

Silent speech interfaces (SSIs) enable speech recognition and synthesis in the absence of an acoustic signal. Yet, the archetypal SSI fails to convey the expressive attributes of prosody such as pitch and loudness, leading to lexical ambiguities. The aim of this study was to determine the efficacy of using surface electromyography (sEMG) as an approach for predicting continuous acoustic estimates of prosody. Ten participants performed a series of vocal tasks including sustained vowels, phrases, and monologues while acoustic data was recorded simultaneously with sEMG activity from muscles of the face and neck. A battery of time-, frequency-, and cepstral-domain features extracted from the sEMG signals were used to train deep regression neural networks to predict fundamental frequency and intensity contours from the acoustic signals. We achieved an average accuracy of 0.01 ST and precision of 0.56 ST for the estimation of fundamental frequency, and an average accuracy of 0.21 dB SPL and precision of 3.25 dB SPL for the estimation of intensity. This work highlights the importance of using sEMG as an alternative means of detecting prosody and shows promise for improving SSIs in future development.