Development of a Low-Cost, Noninvasive, Portable Visual Speech Recognition Program

Machine learning in the evaluation of voice and swallowing in the head and neck cancer patient

PURPOSE OF REVIEW

The purpose of this review is to present recent advances and limitations in machine learning applied to the evaluation of speech, voice, and swallowing in head and neck cancer.

RECENT FINDINGS

Novel machine learning models incorporating diverse data modalities with improved discriminatory capabilities have been developed for predicting toxicities following head and neck cancer therapy, including dysphagia, dysphonia, xerostomia, and weight loss as well as guiding treatment planning. Machine learning has been applied to the care of posttreatment voice and swallowing dysfunction by offering objective and standardized assessments and aiding innovative technologies for functional restoration. Voice and speech are also being utilized in machine learning algorithms to screen laryngeal cancer.

SUMMARY

Machine learning has the potential to help optimize, assess, predict, and rehabilitate voice and swallowing function in head and neck cancer patients as well as aid in cancer screening. However, existing studies are limited by the lack of sufficient external validation and generalizability, insufficient transparency and reproducibility, and no clear superior predictive modeling strategies. Algorithms and applications will need to be trained on large multiinstitutional data sets, incorporate sociodemographic data to reduce bias, and achieve validation through clinical trials for optimal performance and utility.

Visual Speech Recognition: Improving Speech Perception in Noise through Artificial Intelligence

OBJECTIVES

To compare speech perception (SP) in noise for normal-hearing (NH) individuals and individuals with hearing loss (IWHL) and to demonstrate improvements in SP with use of a visual speech recognition program (VSRP).

STUDY DESIGN

Single-institution prospective study.

SETTING

Tertiary referral center.

SUBJECTS AND METHODS

Eleven NH and 9 IWHL participants in a sound-isolated booth facing a speaker through a window. In non-VSRP conditions, SP was evaluated on 40 Bamford-Kowal-Bench speech-in-noise test (BKB-SIN) sentences presented by the speaker at 50 A-weighted decibels (dBA) with multiperson babble noise presented from 50 to 75 dBA. SP was defined as the percentage of words correctly identified. In VSRP conditions, an infrared camera was used to track 35 points around the speaker's lips during speech in real time. Lip movement data were translated into speech-text via an in-house developed neural network-based VSRP. SP was evaluated similarly in the non-VSRP condition on 42 BKB-SIN sentences, with the addition of the VSRP output presented on a screen to the listener.

RESULTS

In high-noise conditions (70-75 dBA) without VSRP, NH listeners achieved significantly higher speech perception than IWHL listeners (38.7% vs 25.0%, P = .02). NH listeners were significantly more accurate with VSRP than without VSRP (75.5% vs 38.7%, P < .0001), as were IWHL listeners (70.4% vs 25.0% P < .0001). With VSRP, no significant difference in SP was observed between NH and IWHL listeners (75.5% vs 70.4%, P = .15).

CONCLUSIONS

The VSRP significantly increased speech perception in high-noise conditions for NH and IWHL participants and eliminated the difference in SP accuracy between NH and IWHL listeners.

Pilot study for a novel and personalized voice restoration device for patients with laryngectomy

BACKGROUND

The main modalities for voice restoration after laryngectomy are the electrolarynx, and the tracheoesophageal puncture [Correction added on 30 January 2020 after first online publication: The preceding sentence has been revised. It originally read "The main modalities for voice restoration after laryngectomy are the electrolarynx and the tracheoesophageal puncture."]. All have limitations and new technologies may offer innovative alternatives via silent speech.

OBJECTIVE

To describe a novel and personalized method of voice restoration using machine learning applied to electromyographic signal from articulatory muscles for the recognition of silent speech in a patient with total laryngectomy.

METHODS

Surface electromyographic (sEMG) signals of articulatory muscles were recorded from the face and neck of a patient with total laryngectomy who was articulating words silently. These sEMG signals were then used for automatic speech recognition via machine learning. Sensor placement was tailored to the patient's unique anatomy, following radiation and surgery. A personalized wearable mask covering the sensors was designed using 3D scanning and 3D printing.

RESULTS

Using seven sEMG sensors on the patient's face and neck and two grounding electrodes, we recorded EMG data while he was mouthing "Tedd" and "Ed." With data from 75 utterances for each of these words, we discriminated the sEMG signal with 86.4% accuracy using an XGBoost machine-learning model.

CONCLUSIONS

This pilot study demonstrates the feasibility of sEMG-based alaryngeal speech recognition, using tailored sensor placement and a personalized wearable device. Further refinement of this approach could allow translation of silently articulated speech into a synthesized voiced speech via portable devices.

The electrolarynx: voice restoration after total laryngectomy

The ability to speak and communicate with one’s voice is a unique human characteristic and is fundamental to many activities of daily living, such as talking on the phone and speaking to loved ones. When the larynx is removed during a total laryngectomy (TL), loss of voice can lead to a devastating decrease in a patient’s quality of life, and precipitate significant frustration over their inability to communicate with others effectively. Over the past 50 years there have been many advances in techniques of voice restoration after TL. Currently, there are three main methods of voice restoration: the electrolarynx, esophageal speech, and tracheoesophageal speech through a tracheoesophageal puncture (TEP) with voice prosthesis. Although TEP voice is the current gold standard for vocal rehabilitation, a significant minority of patients cannot use or obtain TEP speech for various reasons. As such, the electrolarynx is a viable and useful alternative for these patients. This article will focus on voice restoration using an electrolarynx with the following objectives: 1) To provide an understanding of the importance of voice restoration after total laryngectomy. 2) To discuss how the electrolarynx may be used to restore voice following total laryngectomy. 3) To outline some of the current electrolarynx devices available, including their mechanism of action and limitations. 4) To compare pros and cons of electrolaryngeal speech to TEP and esophageal speech.