Body-Worn IMU Human Skeletal Pose Estimation Using a Factor Graph-Based Optimization Framework

A Study on Graph Optimization Method for GNSS/IMU Integrated Navigation System Based on Virtual Constraints

In GNSS/IMU integrated navigation systems, factors like satellite occlusion and non-line-of-sight can degrade satellite positioning accuracy, thereby impacting overall navigation system results. To tackle this challenge and leverage historical pseudorange information effectively, this paper proposes a graph optimization-based GNSS/IMU model with virtual constraints. These virtual constraints in the graph model are derived from the satellite's position from the previous time step, the rate of change of pseudoranges, and ephemeris data. This virtual constraint serves as an alternative solution for individual satellites in cases of signal anomalies, thereby ensuring the integrity and continuity of the graph optimization model. Additionally, this paper conducts an analysis of the graph optimization model based on these virtual constraints, comparing it with traditional graph models of GNSS/IMU and SLAM. The marginalization of the graph model involving virtual constraints is analyzed next. The experiment was conducted on a set of real-world data, and the results of the proposed method were compared with tightly coupled Kalman filtering and the original graph optimization method. In instantaneous performance testing, the method maintains an RMSE error within 5% compared with real pseudorange measurement, while in a continuous performance testing scenario with no available GNSS signal, the method shows approximately a 30% improvement in horizontal RMSE accuracy over the traditional graph optimization method during a 10-second period. This demonstrates the method's potential for practical applications.

Flexible circuit-based spatially aware modular optical brain imaging system for high-density measurements in natural settings

Significance

Functional near-infrared spectroscopy (fNIRS) presents an opportunity to study human brains in everyday activities and environments. However, achieving robust measurements under such dynamic conditions remains a significant challenge.

Aim

The modular optical brain imaging (MOBI) system is designed to enhance optode-to-scalp coupling and provide a real-time probe three-dimensional (3D) shape estimation to improve the use of fNIRS in everyday conditions.

Approach

The MOBI system utilizes a bendable and lightweight modular circuit-board design to enhance probe conformity to head surfaces and comfort for long-term wearability. Combined with automatic module connection recognition, the built-in orientation sensors on each module can be used to estimate optode 3D positions in real time to enable advanced tomographic data analysis and motion tracking.

Results

Optical characterization of the MOBI detector reports a noise equivalence power of 8.9 and 7.3    pW / Hz at 735 and 850 nm, respectively, with a dynamic range of 88 dB. The 3D optode shape acquisition yields an average error of 4.2 mm across 25 optodes in a phantom test compared with positions acquired from a digitizer. Results for initial in vivo validations, including a cuff occlusion and a finger-tapping test, are also provided.

Conclusions

To the best of our knowledge, the MOBI system is the first modular fNIRS system featuring fully flexible circuit boards. The self-organizing module sensor network and automatic 3D optode position acquisition, combined with lightweight modules ( 18    g / module ) and ergonomic designs, would greatly aid emerging explorations of brain function in naturalistic settings.

JointTracker: Real-time inertial kinematic chain tracking with joint position estimation

In-field human motion capture (HMC) is drawing increasing attention due to the multitude of application areas. Plenty of research is currently invested in camera-based (markerless) HMC, with the advantage of no infrastructure being required on the body, and additional context information being available from the surroundings. However, the inherent drawbacks of camera-based approaches are the limited field of view and occlusions. In contrast, inertial HMC (IHMC) does not suffer from occlusions, thus being a promising approach for capturing human motion outside the laboratory. However, one major challenge of such methods is the necessity of spatial registration. Typically, during a predefined calibration sequence, the orientation and location of each inertial sensor are registered with respect to the underlying skeleton model. This work contributes to calibration-free IHMC, as it proposes a recursive estimator for the simultaneous online estimation of all sensor poses and joint positions of a kinematic chain model like the human skeleton. The full derivation from an optimization objective is provided. The approach can directly be applied to a synchronized data stream from a body-mounted inertial sensor network. Successful evaluations are demonstrated on noisy simulated data from a three-link chain, real lower-body walking data from 25 young, healthy persons, and walking data captured from a humanoid robot. The estimated and derived quantities, global and relative sensor orientations, joint positions, and segment lengths can be exploited for human motion analysis and anthropometric measurements, as well as in the context of hybrid markerless visual-inertial HMC.

Drift-Free Joint Angle Calculation Using Inertial Measurement Units without Magnetometers: An Exploration of Sensor Fusion Methods for the Elbow and Wrist

Joint angles of the lower extremities have been calculated using gyroscope and accelerometer measurements from inertial measurement units (IMUs) without sensor drift by leveraging kinematic constraints. However, it is unknown whether these methods are generalizable to the upper extremity due to differences in motion dynamics. Furthermore, the extent that post-processed sensor fusion algorithms can improve measurement accuracy relative to more commonly used Kalman filter-based methods remains unknown. This study calculated the elbow and wrist joint angles of 13 participants performing a simple ≥30 min material transfer task at three rates (slow, medium, fast) using IMUs and kinematic constraints. The best-performing sensor fusion algorithm produced total root mean square errors (i.e., encompassing all three motion planes) of 6.6°, 3.6°, and 2.0° for the slow, medium, and fast transfer rates for the elbow and 2.2°, 1.7°, and 1.5° for the wrist, respectively.

Self-Calibrating Magnetometer-Free Inertial Motion Tracking of 2-DoF Joints

Human motion analysis using inertial measurement units (IMUs) has recently been shown to provide accuracy similar to the gold standard, optical motion capture, but at lower costs and while being less restrictive and time-consuming. However, IMU-based motion analysis requires precise knowledge of the orientations in which the sensors are attached to the body segments. This knowledge is commonly obtained via time-consuming and error-prone anatomical calibration based on precisely defined poses or motions. In the present work, we propose a self-calibrating approach for magnetometer-free joint angle tracking that is suitable for joints with two degrees of freedom (DoF), such as the elbow, ankle, and metacarpophalangeal finger joints. The proposed methods exploit kinematic constraints in the angular rates and the relative orientations to simultaneously identify the joint axes and the heading offset. The experimental evaluation shows that the proposed methods are able to estimate plausible and consistent joint axes from just ten seconds of arbitrary elbow joint motion. Comparison with optical motion capture shows that the proposed methods yield joint angles with similar accuracy as a conventional IMU-based method while being much less restrictive. Therefore, the proposed methods improve the practical usability of IMU-based motion tracking in many clinical and biomedical applications.

Evaluation of Error-State Kalman Filter Method for Estimating Human Lower-Limb Kinematics during Various Walking Gaits

Inertial measurement units (IMUs) offer an attractive way to study human lower-limb kinematics without traditional laboratory constraints. We present an error-state Kalman filter method to estimate 3D joint angles, joint angle ranges of motion, stride length, and step width using data from an array of seven body-worn IMUs. Importantly, this paper contributes a novel joint axis measurement correction that reduces joint angle drift errors without assumptions of strict hinge-like joint behaviors of the hip and knee. We evaluate the method compared to two optical motion capture methods on twenty human subjects performing six different types of walking gait consisting of forward walking (at three speeds), backward walking, and lateral walking (left and right). For all gaits, RMS differences in joint angle estimates generally remain below 5 degrees for all three ankle joint angles and for flexion/extension and abduction/adduction of the hips and knees when compared to estimates from reflective markers on the IMUs. Additionally, mean RMS differences in estimated stride length and step width remain below 0.13 m for all gait types, except stride length during slow walking. This study confirms the method's potential for non-laboratory based gait analysis, motivating further evaluation with IMU-only measurements and pathological gaits.

A Kinematic Information Acquisition Model That Uses Digital Signals from an Inertial and Magnetic Motion Capture System

This paper presents a model that enables the transformation of digital signals generated by an inertial and magnetic motion capture system into kinematic information. First, the operation and data generated by the used inertial and magnetic system are described. Subsequently, the five stages of the proposed model are described, concluding with its implementation in a virtual environment to display the kinematic information. Finally, the applied tests are presented to evaluate the performance of the model through the execution of four exercises on the upper limb: flexion and extension of the elbow, and pronation and supination of the forearm. The results show a mean squared error of 3.82° in elbow flexion-extension movements and 3.46° in forearm pronation-supination movements. The results were obtained by comparing the inertial and magnetic system versus an optical motion capture system, allowing for the identification of the usability and functionality of the proposed model.

The Implication of Pathway Turn and Task Condition on Gait Quantified Using SmartWalk: Changes With Age and Parkinson's Disease With Relevance to Postural Strategy and Risk of Fall

One's gait can be affected by aging, pathway with turns, task demands, etc., causing changes in gait-related indices and knee flexion (influencing posture). Walking on pathways with turns threatens stability, affecting one's gait-related indices and posture. The ability to overcome such deficits is compromised with age and neurological disorders, e.g., Parkinson's Disease (PD) leading to falls. Also, task demands imposed by single and dual-task (e.g., counting backward while walking) conditions affect the gait of individuals using different postural strategies varying with age and neurological disorder. Existing research has investigated either the effect of the pathway with turn or task condition on one's gait. However, none (to our knowledge) have explored the differentiated implications of the pathway with turn and task conditions on one's gait-related indices and knee flexion while walking. Our study had two phases with 30 participants. Phase 1 had healthy adults (young and old) and Phase 2 had age and gender-matched healthy elderly and individuals with Parkinson's disease (PD) who walked on pathways having turns under single and dual-task conditions. We analysed gait in terms of (i) gait-related indices (Phases 1 and 2) and (ii) knee flexion (Phase 2). Also, we analysed one's counting performance during dual task. One's gait-related indices and knee flexion were measured using a portable gait quantifier. The aim was to (i)understand whether both pathways with turn and task conditions are equally effective in affecting the gait of (a)individuals of varying ages and (b) gender-matched healthy older adults and individuals with PD, (ii)study variations of knee joint angles while walking on pathways having turns (under different task conditions) in terms of its clinical relevance, and (iii) explore the implication of pathway with turn on counting performance (with relevance to postural strategy) with varying age and PD. Results indicated that for the younger group, the task condition caused statistical variations in gait-related indices. For the older group, both pathways with turn and task conditions had statistical implications on gait-related indices. Additionally, individuals with PD demonstrated a higher variation in knee flexion than their healthy counterparts. Again, pathways with varying turns elicited variations in counting performance indicating different postural strategies being employed by the three groups.

Body-Worn IMU-Based Human Hip and Knee Kinematics Estimation during Treadmill Walking

Traditionally, inertial measurement unit (IMU)-based human joint angle estimation techniques are evaluated for general human motion where human joints explore all of their degrees of freedom. Pure human walking, in contrast, limits the motion of human joints and may lead to unobservability conditions that confound magnetometer-free IMU-based methods. This work explores the unobservability conditions emergent during human walking and expands upon a previous IMU-based method for the human knee to also estimate human hip angles relative to an assumed vertical datum. The proposed method is evaluated (N=12) in a human subject study and compared against an optical motion capture system. Accuracy of human knee flexion/extension angle (7.87∘ absolute root mean square error (RMSE)), hip flexion/extension angle (3.70∘ relative RMSE), and hip abduction/adduction angle (4.56∘ relative RMSE) during walking are similar to current state-of-the-art self-calibrating IMU methods that use magnetometers. Larger errors of hip internal/external rotation angle (6.27∘ relative RMSE) are driven by IMU heading drift characteristic of magnetometer-free approaches and non-hinge kinematics of the hip during gait, amongst other error sources. One of these sources of error, soft tissue perturbations during gait, is explored further in the context of knee angle estimation and it was observed that the IMU method may overestimate the angle during stance and underestimate the angle during swing. The presented method and results provide a novel combination of observability considerations, heuristic correction methods, and validation techniques to magnetic-blind, kinematic-only IMU-based skeletal pose estimation during human tasks with degenerate kinematics (e.g., straight line walking).

Wearable Technology for Evaluation of Risk of Falls

One's risk of fall can be quantified in terms of variability in one's gait, reflecting a loss of automatic rhythm of one's gait. In gait analysis, variability is commonly understood in terms of the fluctuation in the kinematic, kinetic, spatio-temporal, or physiological information. Here, we have focused on the estimation of knee joint angle (kinematic variable) synchronized with some of the kinetic and spatio-temporal gait parameters while an individual walked overground. Our system consisted of a pair of shoes with instrumented insoles and knee flexion/extension recorder unit having bend sensors. In addition, we have used the Coefficient of Variation for estimating the variability in the knee flexion/extension angle while walking overground as an indicator of the risk of fall. A study with healthy individuals (young and old) walking overground on pathways having 0⁰ and 180⁰ turning angles indicated the feasibility of our wearable system to compute the variability in knee flexion/extension angle as an indicator of the risk of fall.

In-vitro validation of inertial-sensor-to-bone alignment

A major shortcoming in kinematic estimation using skin-attached inertial sensors is the alignment of sensor-embedded and segment-embedded coordinate systems. Only a correct alignment results in clinically relevant kinematics. Model-based inertial-sensor-to-bone alignment methods relate inertial sensor measurements with a model of the joint. Therefore, they do not rely on properly executed calibration movements or a correct sensor placement. However, it is unknown how accurate such model-based methods align the sensor axes and the underlying segment-embedded axes, as defined by clinical definitions. Also, validation of the alignment models is challenging, since an optical motion capture ground truth can be prone to disturbances from soft tissue movement, orientation estimation and manual palpation errors. We present an anatomical tibiofemoral ground truth on an unloaded cadaveric measurement set-up that intrinsically overcomes these disturbances. Additionally, we validate existing model-based alignment strategies. Modeling the degrees of freedom leads to the identification of rotation axes. However, there is no reason why these axes would align with the segment-embedded axes. Relative inertial-sensor orientation information and rich arbitrary movements showed to aid in identifying the underlying joint axes. The first dominant sagittal rotation axis aligned sufficiently well with the underlying segment-embedded reference. The estimated axes that relate to secondary kinematics tend to deviate from the underlying segment-embedded axes as much as their expected range of motion around the axes. In order to interpret the secondary kinematics, the alignment model should more closely match the biomechanics of the joint.

Reference in-vitro dataset for inertial-sensor-to-bone alignment applied to the tibiofemoral joint

Skin-attached inertial sensors are increasingly used for kinematic analysis. However, their ability to measure outside-lab can only be exploited after correctly aligning the sensor axes with the underlying anatomical axes. Emerging model-based inertial-sensor-to-bone alignment methods relate inertial measurements with a model of the joint to overcome calibration movements and sensor placement assumptions. It is unclear how good such alignment methods can identify the anatomical axes. Any misalignment results in kinematic cross-talk errors, which makes model validation and the interpretation of the resulting kinematics measurements challenging. This study provides an anatomically correct ground-truth reference dataset from dynamic motions on a cadaver. In contrast with existing references, this enables a true model evaluation that overcomes influences from soft-tissue artifacts, orientation and manual palpation errors. This dataset comprises extensive dynamic movements that are recorded with multimodal measurements including trajectories of optical and virtual (via computed tomography) anatomical markers, reference kinematics, inertial measurements, transformation matrices and visualization tools. The dataset can be used either as a ground-truth reference or to advance research in inertial-sensor-to-bone-alignment.