Electromyogram-Based Lip-Reading via Unobtrusive Dry Electrodes and Machine Learning Methods

Bio-Inspired Untethered Robot-Sensor Platform for Minimally Invasive Biomedical Sensing

Conventional catheter- or probe-based in vivo biomedical sensing is uncomfortable, inconvenient, and sometimes infeasible for long-term monitoring. Existing implantable sensors often require an invasive procedure for sensor placement. Untethered soft robots with the capability to deliver the sensor to the desired monitoring point hold great promise for minimally invasive biomedical sensing. Inspired by the locomotion modes of snakes, we present here a soft kirigami robot for sensor deployment and real-time wireless sensing. The locomotion mechanism of the soft robot is achieved by kirigami patterns that offer asymmetric tribological properties that mimic the skin of the snake. The robot exhibits good deployability, excellent load capacity (up to 150 times its own weight), high-speed locomotion (0.25 body length per step), and wide environmental adaptability with multimodal movements (obstacle crossing, locomotion in wet and dry conditions, climbing, and inverted crawling). When integrated with passive sensors, the versatile soft robot can locomote inside the human body, deliver the passive sensor to the desired location, and hold the sensor in place for real-time monitoring in a minimally invasive manner. The proof-of-concept prototype demonstrates that the platform can perform real-time impedance monitoring for the diagnosis of gastroesophageal reflux disease.

Decoding silent speech commands from articulatory movements through soft magnetic skin and machine learning

Silent speech interfaces have been pursued to restore spoken communication for individuals with voice disorders and to facilitate intuitive communications when acoustic-based speech communication is unreliable, inappropriate, or undesired. However, the current methodology for silent speech faces several challenges, including bulkiness, obtrusiveness, low accuracy, limited portability, and susceptibility to interferences. In this work, we present a wireless, unobtrusive, and robust silent speech interface for tracking and decoding speech-relevant movements of the temporomandibular joint. Our solution employs a single soft magnetic skin placed behind the ear for wireless and socially acceptable silent speech recognition. The developed system alleviates several concerns associated with existing interfaces based on face-worn sensors, including a large number of sensors, highly visible interfaces on the face, and obtrusive interconnections between sensors and data acquisition components. With machine learning-based signal processing techniques, good speech recognition accuracy is achieved (93.2% accuracy for phonemes, and 87.3% for a list of words from the same viseme groups). Moreover, the reported silent speech interface demonstrates robustness against noises from both ambient environments and users' daily motions. Finally, its potential in assistive technology and human-machine interactions is illustrated through two demonstrations - silent speech enabled smartphone assistants and silent speech enabled drone control.