Magnetometer-free Realtime Inertial Motion Tracking by Exploitation of Kinematic Constraints in 2-DoF Joints

Self-Calibrating Magnetometer-Free Inertial Motion Tracking of 2-DoF Joints

Human motion analysis using inertial measurement units (IMUs) has recently been shown to provide accuracy similar to the gold standard, optical motion capture, but at lower costs and while being less restrictive and time-consuming. However, IMU-based motion analysis requires precise knowledge of the orientations in which the sensors are attached to the body segments. This knowledge is commonly obtained via time-consuming and error-prone anatomical calibration based on precisely defined poses or motions. In the present work, we propose a self-calibrating approach for magnetometer-free joint angle tracking that is suitable for joints with two degrees of freedom (DoF), such as the elbow, ankle, and metacarpophalangeal finger joints. The proposed methods exploit kinematic constraints in the angular rates and the relative orientations to simultaneously identify the joint axes and the heading offset. The experimental evaluation shows that the proposed methods are able to estimate plausible and consistent joint axes from just ten seconds of arbitrary elbow joint motion. Comparison with optical motion capture shows that the proposed methods yield joint angles with similar accuracy as a conventional IMU-based method while being much less restrictive. Therefore, the proposed methods improve the practical usability of IMU-based motion tracking in many clinical and biomedical applications.

Body-Worn IMU-Based Human Hip and Knee Kinematics Estimation during Treadmill Walking

Traditionally, inertial measurement unit (IMU)-based human joint angle estimation techniques are evaluated for general human motion where human joints explore all of their degrees of freedom. Pure human walking, in contrast, limits the motion of human joints and may lead to unobservability conditions that confound magnetometer-free IMU-based methods. This work explores the unobservability conditions emergent during human walking and expands upon a previous IMU-based method for the human knee to also estimate human hip angles relative to an assumed vertical datum. The proposed method is evaluated (N=12) in a human subject study and compared against an optical motion capture system. Accuracy of human knee flexion/extension angle (7.87∘ absolute root mean square error (RMSE)), hip flexion/extension angle (3.70∘ relative RMSE), and hip abduction/adduction angle (4.56∘ relative RMSE) during walking are similar to current state-of-the-art self-calibrating IMU methods that use magnetometers. Larger errors of hip internal/external rotation angle (6.27∘ relative RMSE) are driven by IMU heading drift characteristic of magnetometer-free approaches and non-hinge kinematics of the hip during gait, amongst other error sources. One of these sources of error, soft tissue perturbations during gait, is explored further in the context of knee angle estimation and it was observed that the IMU method may overestimate the angle during stance and underestimate the angle during swing. The presented method and results provide a novel combination of observability considerations, heuristic correction methods, and validation techniques to magnetic-blind, kinematic-only IMU-based skeletal pose estimation during human tasks with degenerate kinematics (e.g., straight line walking).

Reference in-vitro dataset for inertial-sensor-to-bone alignment applied to the tibiofemoral joint

Skin-attached inertial sensors are increasingly used for kinematic analysis. However, their ability to measure outside-lab can only be exploited after correctly aligning the sensor axes with the underlying anatomical axes. Emerging model-based inertial-sensor-to-bone alignment methods relate inertial measurements with a model of the joint to overcome calibration movements and sensor placement assumptions. It is unclear how good such alignment methods can identify the anatomical axes. Any misalignment results in kinematic cross-talk errors, which makes model validation and the interpretation of the resulting kinematics measurements challenging. This study provides an anatomically correct ground-truth reference dataset from dynamic motions on a cadaver. In contrast with existing references, this enables a true model evaluation that overcomes influences from soft-tissue artifacts, orientation and manual palpation errors. This dataset comprises extensive dynamic movements that are recorded with multimodal measurements including trajectories of optical and virtual (via computed tomography) anatomical markers, reference kinematics, inertial measurements, transformation matrices and visualization tools. The dataset can be used either as a ground-truth reference or to advance research in inertial-sensor-to-bone-alignment.

Inertial-Robotic Motion Tracking in End-Effector-Based Rehabilitation Robots

End-effector-based robotic systems provide easy-to-set-up motion support in rehabilitation of stroke and spinal-cord-injured patients. However, measurement information is obtained only about the motion of the limb segments to which the systems are attached and not about the adjacent limb segments. We demonstrate in one particular experimental setup that this limitation can be overcome by augmenting an end-effector-based robot with a wearable inertial sensor. Most existing inertial motion tracking approaches rely on a homogeneous magnetic field and thus fail in indoor environments and near ferromagnetic materials and electronic devices. In contrast, we propose a magnetometer-free sensor fusion method. It uses a quaternion-based algorithm to track the heading of a limb segment in real time by combining the gyroscope and accelerometer readings with position measurements of one point along that segment. We apply this method to an upper-limb rehabilitation robotics use case in which the orientation and position of the forearm and elbow are known, and the orientation and position of the upper arm and shoulder are estimated by the proposed method using an inertial sensor worn on the upper arm. Experimental data from five healthy subjects who performed 282 proper executions of a typical rehabilitation motion and 163 executions with compensation motion are evaluated. Using a camera-based system as a ground truth, we demonstrate that the shoulder position and the elbow angle are tracked with median errors around 4 cm and 4°, respectively; and that undesirable compensatory shoulder movements, which were defined as shoulder displacements greater ±10 cm for more than 20% of a motion cycle, are detected and classified 100% correctly across all 445 performed motions. The results indicate that wearable inertial sensors and end-effector-based robots can be combined to provide means for effective rehabilitation therapy with likewise detailed and accurate motion tracking for performance assessment, real-time biofeedback and feedback control of robotic and neuroprosthetic motion support.

Body-Worn IMU Human Skeletal Pose Estimation Using a Factor Graph-Based Optimization Framework

Traditionally, inertial measurement units- (IMU) based human joint angle estimation requires a priori knowledge about sensor alignment or specific calibration motions. Furthermore, magnetometer measurements can become unreliable indoors. Without magnetometers, however, IMUs lack a heading reference, which leads to unobservability issues. This paper proposes a magnetometer-free estimation method, which provides desirable observability qualities under joint kinematics that sufficiently excite the lower body degrees of freedom. The proposed lower body model expands on the current self-calibrating human-IMU estimation literature and demonstrates a novel knee hinge model, the inclusion of segment length anthropometry, segment cross-leg length discrepancy, and the relationship between the knee axis and femur/tibia segment. The maximum a posteriori problem is formulated as a factor graph and inference is performed via post-hoc, on-manifold global optimization. The method is evaluated (N = 12) for a prescribed human motion profile task. Accuracy of derived knee flexion/extension angle (4.34∘ root mean square error (RMSE)) without magnetometers is similar to current state-of-the-art with magnetometer use. The developed framework can be expanded for modeling additional joints and constraints.

Inertial Sensors-Applications and Challenges in a Nutshell

This editorial provides a concise introduction to the methods and applications of inertial sensors. We briefly describe the main characteristics of inertial sensors and highlight the broad range of applications as well as the methodological challenges. Finally, for the reader’s guidance, we give a succinct overview of the papers included in this special issue.

Artificial Neural Networks in Motion Analysis-Applications of Unsupervised and Heuristic Feature Selection Techniques

The use of machine learning to estimate joint angles from inertial sensors is a promising approach to in-field motion analysis. In this context, the simplification of the measurements by using a small number of sensors is of great interest. Neural networks have the opportunity to estimate joint angles from a sparse dataset, which enables the reduction of sensors necessary for the determination of all three-dimensional lower limb joint angles. Additionally, the dimensions of the problem can be simplified using principal component analysis. Training a long short-term memory neural network on the prediction of 3D lower limb joint angles based on inertial data showed that three sensors placed on the pelvis and both shanks are sufficient. The application of principal component analysis to the data of five sensors did not reveal improved results. The use of longer motion sequences compared to time-normalised gait cycles seems to be advantageous for the prediction accuracy, which bridges the gap to real-time applications of long short-term memory neural networks in the future.

Gait Analysis in a Box: A System Based on Magnetometer-Free IMUs or Clusters of Optical Markers with Automatic Event Detection

Gait analysis based on full-body motion capture technology (MoCap) can be used in rehabilitation to aid in decision making during treatments or therapies. In order to promote the use of MoCap gait analysis based on inertial measurement units (IMUs) or optical technology, it is necessary to overcome certain limitations, such as the need for magnetically controlled environments, which affect IMU systems, or the need for additional instrumentation to detect gait events, which affects IMUs and optical systems. We present a MoCap gait analysis system called Move Human Sensors (MH), which incorporates proposals to overcome both limitations and can be configured via magnetometer-free IMUs (MH-IMU) or clusters of optical markers (MH-OPT). Using a test-retest reliability experiment with thirty-three healthy subjects (20 men and 13 women, 21.7 ± 2.9 years), we determined the reproducibility of both configurations. The assessment confirmed that the proposals performed adequately and allowed us to establish usage considerations. This study aims to enhance gait analysis in daily clinical practice.

Inertial Sensor Based Solution for Finger Motion Tracking

Hand motion tracking plays an important role in virtual reality systems for immersion and interaction purposes. This paper discusses the problem of finger tracking and proposes the application of the extension of the Madgwick filter and a simple switching (motion recognition) algorithm as a comparison. The proposed algorithms utilize the three-link finger model and provide complete information about the position and orientation of the metacarpus. The numerical experiment shows that this approach is feasible and overcomes some of the major limitations of inertial motion tracking. The paper’s proposed solution was created in order to track a user’s pointing and grasping movements during the interaction with the virtual reconstruction of the cultural heritage of historical cities.

Supporting front crawl swimming in paraplegics using electrical stimulation: a feasibility study

Background

Participation in physical and therapeutic activities is usually severely restricted after a spinal cord injury (SCI). Reasons for this are the associated loss of voluntary motor function, inefficient temperature regulation of the affected extremities, and early muscle fatigue. Hydrotherapy or swim training offer an inherent weight relief, reduce spasticity and improve coordination, muscle strength and fitness.

Methods

We present a new hybrid exercise modality that combines functional electrical stimulation (FES) of the knee extensors and transcutaneous spinal cord stimulation (tSCS) with paraplegic front crawl swimming. tSCS is used to stimulate the afferent fibers of the L2–S2 posterior roots for spasticity reduction. By activating the tSCS, the trunk musculature is recruited at a motor level. This shall improve trunk stability and straighten the upper body. Within this feasibility study, two complete SCI subjects (both ASIA scale A, lesion level Th5/6), who have been proficient front crawl swimmers, conducted a 10-week swim training with stimulation support. In an additional assessment swim session nine months after the training, the knee extension, hip extension, and trunk roll angles where measured using waterproof inertial measurement units (IMUs) and compared for different swimming conditions (no stimulation, tSCS, FES, FES plus tSCS).

Results

For both subjects, a training effect over the 10-week swim training was observed in terms of measured lap times (16 m pool) for all swimming conditions. Swimming supported by FES reduced lap times by 15.4% and 8.7% on average for Subject A and Subject B, respectively. Adding tSCS support yielded even greater mean decreases of 19.3% and 20.9% for Subjects A and B, respectively. Additionally, both subjects individually reported that swimming with tSCS for 30–45 minutes eliminated spasticity in the lower extremities for up to 4 hours beyond the duration of the session. Comparing the median as well as the interquartile range of all different settings, the IMU-based motion analysis revealed that FES as well as FES+tSCS improve knee extension in both subjects, while hip extension was only increased in one subject. Trunk roll angles were similar for all swimming conditions. tSCS had no influence on the knee and hip joint angles. Both subjects reported that stimulation-assisted swimming is comfortable, enjoyable, and they would like to use such a device for recreational training and rehabilitation in the future.

Conclusions

Stimulation-assisted swimming seems to be a promising new form of hybrid exercise for SCI people. It is safe to use with reusable silicone electrodes and can be performed independently by experienced paraplegic swimmers except for transfer to water. The study results indicate that swimming speed can be increased by the proposed methods and spasticity can be reduced by prolonged swim sessions with tSCS and FES. The combination of stimulation with hydrotherapy might be a promising therapy for neurologic rehabilitation in incomplete SCI, stroke or multiples sclerosis patients. Therefore, further studies shall incorporate other neurologic disorders and investigate the potential benefits of FES and tSCS therapy in the water for gait and balance.

Estimation of Gait Mechanics Based on Simulated and Measured IMU Data Using an Artificial Neural Network

Enhancement of activity is one major topic related to the aging society. Therefore, it is necessary to understand people's motion and identify possible risk factors during activity. Technology can be used to monitor motion patterns during daily life. Especially the use of artificial intelligence combined with wearable sensors can simplify measurement systems and might at some point replace the standard motion capturing using optical measurement technologies. Therefore, this study aims to analyze the estimation of 3D joint angles and joint moments of the lower limbs based on IMU data using a feedforward neural network. The dataset summarizes optical motion capture data of former studies and additional newly collected IMU data. Based on the optical data, the acceleration and angular rate of inertial sensors was simulated. The data was augmented by simulating different sensor positions and orientations. In this study, gait analysis was undertaken with 30 participants using a conventional motion capture set-up based on an optoelectronic system and force plates in parallel with a custom IMU system consisting of five sensors. A mean correlation coefficient of 0.85 for the joint angles and 0.95 for the joint moments was achieved. The RMSE for the joint angle prediction was smaller than 4.8° and the nRMSE for the joint moment prediction was below 13.0%. Especially in the sagittal motion plane good results could be achieved. As the measured dataset is rather small, data was synthesized to complement the measured data. The enlargement of the dataset improved the prediction of the joint angles. While size did not affect the joint moment prediction, the addition of noise to the dataset resulted in an improved prediction accuracy. This indicates that research on appropriate augmentation techniques for biomechanical data is useful to further improve machine learning applications.