Towards a magnetic localization system for 3-D tracking of tongue movements in speech-language therapy

Pilot Study: Magnetic Motion Analysis for Swallowing Detection Using MEMS Cantilever Actuators

The swallowing process involves complex muscle coordination mechanisms. When alterations in such mechanisms are produced by neurological conditions or diseases, a swallowing disorder known as dysphagia occurs. The instrumental evaluation of dysphagia is currently performed by invasive and experience-dependent techniques. Otherwise, non-invasive magnetic methods have proven to be suitable for various biomedical applications and might also be applicable for an objective swallowing assessment. In this pilot study, we performed a novel approach for deglutition evaluation based on active magnetic motion sensing with permanent magnet cantilever actuators. During the intake of liquids with different consistency, we recorded magnetic signals of relative movements between a stationary sensor and a body-worn actuator on the cricoid cartilage. Our results indicate the detection capability of swallowing-related movements in terms of a characteristic pattern. Consequently, the proposed technique offers the potential for dysphagia screening and biofeedback-based therapies.

Automatic Assessment of Aphasic Speech Sensed by Audio Sensors for Classification into Aphasia Severity Levels to Recommend Speech Therapies

Aphasia is a type of speech disorder that can cause speech defects in a person. Identifying the severity level of the aphasia patient is critical for the rehabilitation process. In this research, we identify ten aphasia severity levels motivated by specific speech therapies based on the presence or absence of identified characteristics in aphasic speech in order to give more specific treatment to the patient. In the aphasia severity level classification process, we experiment on different speech feature extraction techniques, lengths of input audio samples, and machine learning classifiers toward classification performance. Aphasic speech is required to be sensed by an audio sensor and then recorded and divided into audio frames and passed through an audio feature extractor before feeding into the machine learning classifier. According to the results, the mel frequency cepstral coefficient (MFCC) is the most suitable audio feature extraction method for the aphasic speech level classification process, as it outperformed the classification performance of all mel-spectrogram, chroma, and zero crossing rates by a large margin. Furthermore, the classification performance is higher when 20 s audio samples are used compared with 10 s chunks, even though the performance gap is narrow. Finally, the deep neural network approach resulted in the best classification performance, which was slightly better than both K-nearest neighbor (KNN) and random forest classifiers, and it was significantly better than decision tree algorithms. Therefore, the study shows that aphasia level classification can be completed with accuracy, precision, recall, and F1-score values of 0.99 using MFCC for 20 s audio samples using the deep neural network approach in order to recommend corresponding speech therapy for the identified level. A web application was developed for English-speaking aphasia patients to self-diagnose the severity level and engage in speech therapies.

Reconstruction of stomach geometry using magnetic source localization

Routine diagnosis of gastric motility disorders represents a significant problem to current clinical practice. The non-invasive electrogastrogram (EGG) and magnetogastrogram (MGG) enable the assessment of gastric slow wave (SW) dysrhythmias that are associated with motility disorders. However, both modalities lack standardized methods for reliably detecting patterns of SW activity. Subject-specific anatomical information relating to the geometry of the stomach and its position within the torso have the potential to aid the development of relations between SWs and far-fields. In this study, we demonstrated the feasibility of using magnetic source localization to reconstruct the geometry of an anatomically realistic 3D stomach model. The magnetic fields produced by a small (6.35 × 6.35 mm) N35 neodymium magnet sequentially positioned at 64 positions were recorded by an array of 27 magnetometers. Finally, the magnetic dipole approximation and a particle swarm optimizer were used to estimate the position and orientation of the permanent magnet. Median position and orientation errors of 3.8 mm and 7.3° were achieved. The estimated positions were used to construct a surface mesh, and the Hausdorff Distance and Average Hausdorff Distance dissimilarity metrics for the reconstructed and ground-truth models were 11.6 mm and 2.4 mm, respectively. The results indicate that source localization using the magnetic dipole model can successfully reconstruct the geometry of the stomach.

Multimodal Speech Capture System for Speech Rehabilitation and Learning

Speech-language pathologists (SLPs) are trained to correct articulation of people diagnosed with motor speech disorders by analyzing articulators' motion and assessing speech outcome while patients speak. To assist SLPs in this task, we are presenting the multimodal speech capture system (MSCS) that records and displays kinematics of key speech articulators, the tongue and lips, along with voice, using unobtrusive methods. Collected speech modalities, tongue motion, lips gestures, and voice are visualized not only in real-time to provide patients with instant feedback but also offline to allow SLPs to perform post-analysis of articulators' motion, particularly the tongue, with its prominent but hardly visible role in articulation. We describe the MSCS hardware and software components, and demonstrate its basic visualization capabilities by a healthy individual repeating the words "Hello World." A proof-of-concept prototype has been successfully developed for this purpose, and will be used in future clinical studies to evaluate its potential impact on accelerating speech rehabilitation by enabling patients to speak naturally. Pattern matching algorithms to be applied to the collected data can provide patients with quantitative and objective feedback on their speech performance, unlike current methods that are mostly subjective, and may vary from one SLP to another.

Joint Magnetic Calibration and Localization Based on Expectation Maximization for Tongue Tracking

BACKGROUND

Tongue tracking, which helps researchers gain valuable insights into speech mechanism, has many applications in speech therapy and language learning. The wireless localization technique, which involves tracking a small magnetic tracer within the 3-D oral space, provides a low cost and convenient approach to capture tongue kinematics. In practice, this technique requires accurate calibration of three-axial magnetic sensors used in the tracking system. The data-driven calibration depends on the trajectories of magnetic tracer and the ambient noise, which may change across time and space.

METHODS

In this paper, we model the kinematics of tracer movement and the noisy magnetic measurements in a Bayesian framework, then present a joint calibration and localization (JCL) algorithm based on expectation maximization (EM), where the unscented Rauch-Tung-Striebel smoother is employed for tracer localization and the curvilinear search algorithm is applied for sensor calibration.

RESULTS

Based on measurements conducted on our tongue tracking system with a small magnetic tracer (diameter: 6.05 mm, thickness: 1.25 mm, residual induction: 14 800 G), the JCL algorithm achieves averaged root mean square error of 0.45 mm for tracer position estimation and for tracer orientation estimation, which are significantly lower than those of the separate calibration and localization algorithms.

CONCLUSION

These results show that JCL can help improve the localization accuracy of this system.

SIGNIFICANCE

A potentially high precision tongue tracking method is demonstrated.

A comprehensive method for magnetic sensor calibration: a precise system for 3-D tracking of the tongue movements

Magnetic localization has been used in a variety of applications, including the medical field. Small magnetic tracers are often modeled as dipoles and localization has been achieved by solving well-defined dipole equations. However, in practice, the precise calculation of the tracer location not only depends on solving the highly nonlinear dipole equations through numerical algorithms but also on the precision of the magnetic sensor, accuracy of the tracer magnetization, and the earth magnetic field (EMF) measurements. We have developed and implemented a comprehensive calibration method that addresses all of the aforementioned factors. We evaluated this method in a bench-top setting by moving the tracer along controlled trajectories. We also conducted several experiments to track the tongue movement in a human subject.

Geometrical Design of a Scalable Overlapping Planar Spiral Coil Array to Generate a Homogeneous Magnetic Field

We present a design methodology for an overlapping hexagonal planar spiral coil (hex-PSC) array, optimized for creation of a homogenous magnetic field for wireless power transmission to randomly moving objects. The modular hex-PSC array has been implemented in the form of three parallel conductive layers, for which an iterative optimization procedure defines the PSC geometries. Since the overlapping hex-PSCs in different layers have different characteristics, the worst case coil-coupling condition should be designed to provide the maximum power transfer efficiency (PTE) in order to minimize the spatial received power fluctuations. In the worst case, the transmitter (Tx) hex-PSC is overlapped by six PSCs and surrounded by six other adjacent PSCs. Using a receiver (Rx) coil, 20 mm in radius, at the coupling distance of 78 mm and maximum lateral misalignment of 49.1 mm (1/√3 of the PSC radius) we can receive power at a PTE of 19.6% from the worst case PSC. Furthermore, we have studied the effects of Rx coil tilting and concluded that the PTE degrades significantly when θ > 60°. Solutions are: 1) activating two adjacent overlapping hex-PSCs simultaneously with out-of-phase excitations to create horizontal magnetic flux and 2) inclusion of a small energy storage element in the Rx module to maintain power in the worst case scenarios. In order to verify the proposed design methodology, we have developed the EnerCage system, which aims to power up biological instruments attached to or implanted in freely behaving small animal subjects' bodies in long-term electrophysiology experiments within large experimental arenas.

Towards a smart experimental arena for long-term electrophysiology experiments

Wireless power and data transmission have created promising prospects in biomedical research by enabling perpetual data acquisition and stimulation systems. We present a work in progress towards such a system, called the EnerCage, equipped with scalable arrays of overlapping planar spiral coils (PSC) and 3-axis magnetic sensors for focused wireless power transmission to randomly moving targets, such as small freely behaving animal subjects. The EnerCage system includes a stationary unit for 3D non-line-of-sight localization and inductive power transmission through a geometrically optimized PSC array. The localization algorithm compares the magnetic sensor outputs with a threshold to activate a PSC. All PSCs are optimized based on the worst-case misalignment, considering parasitics from the overlapping and adjacent PSCs. EnerCage also has a mobile unit attached to or implanted in the subject's body, which includes a permanent magnetic tracer for localization and back telemetry circuit for efficient closed-loop inductive power regulation. The EnerCage system is designed to enable long-term electrophysiology experiments on freely behaving small animal subjects in large experimental arenas without requiring them to carry bulky batteries. A prototype of the EnerCage system with five PSCs and five magnetic sensors achieved power transfer efficiency (PTE) of 19.6% at the worst-case horizontal misalignment of 49.1 mm (√1/3 of the PSC radius) and coupling distance of 78 mm with a mobile unit coil, 20 mm in radius. The closed-loop power management mechanism maintains the mobile unit received power at 20 mW despite misalignments, tilting, and distance variations up to a maximum operating height of 120 mm (PTE = 5%).