A comprehensive method for magnetic sensor calibration: a precise system for 3-D tracking of the tongue movements

Multimodal Speech Capture System for Speech Rehabilitation and Learning

Speech-language pathologists (SLPs) are trained to correct articulation of people diagnosed with motor speech disorders by analyzing articulators' motion and assessing speech outcome while patients speak. To assist SLPs in this task, we are presenting the multimodal speech capture system (MSCS) that records and displays kinematics of key speech articulators, the tongue and lips, along with voice, using unobtrusive methods. Collected speech modalities, tongue motion, lips gestures, and voice are visualized not only in real-time to provide patients with instant feedback but also offline to allow SLPs to perform post-analysis of articulators' motion, particularly the tongue, with its prominent but hardly visible role in articulation. We describe the MSCS hardware and software components, and demonstrate its basic visualization capabilities by a healthy individual repeating the words "Hello World." A proof-of-concept prototype has been successfully developed for this purpose, and will be used in future clinical studies to evaluate its potential impact on accelerating speech rehabilitation by enabling patients to speak naturally. Pattern matching algorithms to be applied to the collected data can provide patients with quantitative and objective feedback on their speech performance, unlike current methods that are mostly subjective, and may vary from one SLP to another.

Joint Magnetic Calibration and Localization Based on Expectation Maximization for Tongue Tracking

BACKGROUND

Tongue tracking, which helps researchers gain valuable insights into speech mechanism, has many applications in speech therapy and language learning. The wireless localization technique, which involves tracking a small magnetic tracer within the 3-D oral space, provides a low cost and convenient approach to capture tongue kinematics. In practice, this technique requires accurate calibration of three-axial magnetic sensors used in the tracking system. The data-driven calibration depends on the trajectories of magnetic tracer and the ambient noise, which may change across time and space.

METHODS

In this paper, we model the kinematics of tracer movement and the noisy magnetic measurements in a Bayesian framework, then present a joint calibration and localization (JCL) algorithm based on expectation maximization (EM), where the unscented Rauch-Tung-Striebel smoother is employed for tracer localization and the curvilinear search algorithm is applied for sensor calibration.

RESULTS

Based on measurements conducted on our tongue tracking system with a small magnetic tracer (diameter: 6.05 mm, thickness: 1.25 mm, residual induction: 14 800 G), the JCL algorithm achieves averaged root mean square error of 0.45 mm for tracer position estimation and for tracer orientation estimation, which are significantly lower than those of the separate calibration and localization algorithms.

CONCLUSION

These results show that JCL can help improve the localization accuracy of this system.

SIGNIFICANCE

A potentially high precision tongue tracking method is demonstrated.