Estimation of Lower Limb Joint Angles and Joint Moments during Different Locomotive Activities Using the Inertial Measurement Units and a Hybrid Deep Learning Model

Using inertial measurement units (IMUs) to estimate lower limb joint kinematics and kinetics can provide valuable information for disease diagnosis and rehabilitation assessment. To estimate gait parameters using IMUs, model-based filtering approaches have been proposed, such as the Kalman filter and complementary filter. However, these methods require special calibration and alignment of IMUs. The development of deep learning algorithms has facilitated the application of IMUs in biomechanics as it does not require particular calibration and alignment procedures of IMUs in use. To estimate hip/knee/ankle joint angles and moments in the sagittal plane, a subject-independent temporal convolutional neural network-bidirectional long short-term memory network (TCN-BiLSTM) model was proposed using three IMUs. A public benchmark dataset containing the most representative locomotive activities in daily life was used to train and evaluate the TCN-BiLSTM model. The mean Pearson correlation coefficient of joint angles and moments estimated by the proposed model reached 0.92 and 0.87, respectively. This indicates that the TCN-BiLSTM model can effectively estimate joint angles and moments in multiple scenarios, demonstrating its potential for application in clinical and daily life scenarios.

Magnetometer-free Kalman filter for motor-based assessment of prodromal Parkinson's disease

Observing the kinematics of specific motor tasks, such as finger tapping (FT), provides an objective and consistent quantification of the severity of neurodegenerative diseases. However, the current clinical practice mostly relies on visual observations performed by the clinician. Thus, the assessment is subjective. In this paper, we propose a magnetometer-free Kalman filter (KF) to assess FT features using wearable, inertial sensors. The KF was used to assess features during two different FT tasks, namely forefinger tapping (FTAP) and thumb-forefinger tapping (THFF). The proposed KF was validated against a camera-based reference and compared with a strap-down integration-based method. Comparison between KF method and camera reference showed low discrepancies in terms of root mean square error (RMSE) for considered features: namely number of repetitions (RMSE < 0.7), tapping frequency (RMSE < 0.1 Hz), and amplitude (RMSE < 2.6 deg). An high correlation coefficient between amplitudes was also obtained. The proposed KF performed better than the strap-down integration method on both FT tasks, showing lower RMSE on every feature as well as a higher correlation coefficient. Clinical Relevance- The wearable setup, as well as the proposed magnetometer-free KF, may provide a low-cost, easyto- use, non-invasive motion tracking system for protocols aimed to assess motor performances in neurodegenerative disorders.

Reconstruction of Dexterous 3D Motion Data From a Flexible Magnetic Sensor With Deep Learning and Structure-Aware Filtering

We propose IM3D+, a novel approach to reconstructing 3D motion data from a flexible magnetic flux sensor array using deep learning and a structure-aware temporal bilateral filter. Computing the 3D configuration of markers (inductor-capacitor (LC) coils) from flux sensor data is difficult because the existing numerical approaches suffer from system noise, dead angles, the need for initialization, and limitations in the sensor array's layout. We solve these issues with deep neural networks to learn the regression from the simulation flux values to the LC coils' 3D configuration, which can be applied to the actual LC coils at any location and orientation within the capture volume. To cope with the influence of system noise and the dead-angle limitation caused by the characteristics of the hardware and sensing principle, we propose a structure-aware temporal bilateral filter for reconstructing motion sequences. Our method can track various movements, including fingers that manipulate objects, beetles that move inside a vivarium with leaves and soil, and the flow of opaque fluid. Since no power supply is needed for the lightweight wireless markers, our method can robustly track movements for a very long time, making it suitable for various types of observations whose tracking is difficult with existing motion-tracking systems. Furthermore, the flexibility of the flux sensor layout allows users to reconfigure it based on their own applications, thus making our approach suitable for a variety of virtual reality applications.

A Wearable System Based on Multiple Magnetic and Inertial Measurement Units for Spine Mobility Assessment: A Reliability Study for the Evaluation of Ankylosing Spondylitis

Spinal mobility assessment is essential for the diagnostic of patients with ankylosing spondylitis. BASMI is a routine clinical evaluation of the spine; its measurements are made with goniometers and tape measures, implying systematic errors, subjectivity, and low sensitivity. Therefore, it is crucial to develop better mobility assessment methods. The design, implementation, and evaluation of a novel system for assessing the entire spine’s motion are presented. It consists of 16 magnetic and inertial measurement units (MIMUs) communicated wirelessly with a computer. The system evaluates the patient’s movements by implementing a sensor fusion of the triaxial gyroscope, accelerometer, and magnetometer signals using a Kalman filter. Fifteen healthy participants were assessed with the system through six movements involving the entire spine to calculate continuous kinematics and maximum range of motion (RoM). The intrarater reliability was computed over the observed RoM, showing excellent reliability levels (intraclass correlation >0.9) in five of the six movements. The results demonstrate the feasibility of the system for further clinical studies with patients. The system has the potential to improve the BASMI method. To the best of our knowledge, our system involves the highest number of sensors, thus providing more objective information than current similar systems.

Preliminary tests of an Inertial Measurement Units based System for Spine mobility assessment in patients with Ankylosing Spondylitis

This paper presents the preliminary tests of a novel system prototype for the physical assessment of mobility in patients with Ankylosing Spondylitis (AS). The system combines multi-inertial sensors arrays with Kalman Filters-based pose estimation for monitoring spine mobility in patients with AS. This system allows detecting movements with more reliable information than the manual clinical evaluation.

Magnetic-free Extended Kalman Filter for upper limb kinematic assessment in Yoga

Human motion analysis is gaining increased importance in several fields, from movement assessment in rehabilitation to recreational applications such as virtual coaching. Among all the technologies involved in motion capture, Magneto-Inertial Measurements Units (MIMUs) is one of the most promising due to their small dimensions and low costs. Nevertheless, their usage is strongly limited by different error sources, among which magnetic disturbances, which are particularly problematic in indoor environments. Inertial Measurement Units (IMUs) could, thus, be considered as alternative solution. Indeed, relying exclusively on accelerometers and gyroscopes, they are insensitive to magnetic disturbances. Even if the literature has started to propose few algorithms that do not take into account magnetometer input, their application is limited to robotics and aviation. The aim of the present work is to introduce a magnetic-free quaternion based Extended Kalman filter for upper limb kinematic assessment in human motion (i.e., yoga). The algorithm was tested on five expert yoga trainers during the execution of the sun salutation sequence. Joint angle estimations were compared with the ones obtained from an optoelectronic reference system by evaluating the Mean Absolute Errors (MAEs) and Pearson's correlation coefficients. The achieved worst-case was 6.17°, while the best one was 2.65° for MAEs mean values. The accuracy of the algorithm was further confirmed by the high values of the Pearson's correlation coefficients (lowest mean value of 0.86).Clinical Relevance- The proposed work validated a magnetic free algorithm for kinematic reconstruction with inertial units. It could be used as a wearable solution to track human movements in indoor environments being insensitive to magnetic disturbances, and thus could be potentially used also for rehabilitation purposes.

Extension of the Rigid-Constraint Method for the Heuristic Suboptimal Parameter Tuning to Ten Sensor Fusion Algorithms Using Inertial and Magnetic Sensing

The orientation of a magneto-inertial measurement unit can be estimated using a sensor fusion algorithm (SFA). However, orientation accuracy is greatly affected by the choice of the SFA parameter values which represents one of the most critical steps. A commonly adopted approach is to fine-tune parameter values to minimize the difference between estimated and true orientation. However, this can only be implemented within the laboratory setting by requiring the use of a concurrent gold-standard technology. To overcome this limitation, a Rigid-Constraint Method (RCM) was proposed to estimate suboptimal parameter values without relying on any orientation reference. The RCM method effectiveness was successfully tested on a single-parameter SFA, with an average error increase with respect to the optimal of 1.5 deg. In this work, the applicability of the RCM was evaluated on 10 popular SFAs with multiple parameters under different experimental scenarios. The average residual between the optimal and suboptimal errors amounted to 0.6 deg with a maximum of 3.7 deg. These encouraging results suggest the possibility to properly tune a generic SFA on different scenarios without using any reference. The synchronized dataset also including the optical data and the SFA codes are available online.

A Full-State Robust Extended Kalman Filter for Orientation Tracking During Long-Duration Dynamic Tasks Using Magnetic and Inertial Measurement Units

Accurate and robust orientation estimation using magnetic and inertial measurement units (MIMUs) has been a challenge for many years in long-duration measurements of joint angles and pedestrian dead-reckoning systems and has limited several real-world applications of MIMUs. Thus, this research aimed at developing a full-state Robust Extended Kalman Filter (REKF) for accurate and robust orientation tracking with MIMUs, particularly during long-duration dynamic tasks. First, we structured a novel EKF by including the orientation quaternion, non-gravitational acceleration, gyroscope bias, and magnetic disturbance in the state vector. Next, the a posteriori error covariance matrix equation was modified to build a REKF. We compared the accuracy and robustness of our proposed REKF with four filters from the literature using optimal filter gains. We measured the thigh, shank, and foot orientation of nine participants while performing short- and long-duration tasks using MIMUs and a camera motion-capture system. REKF outperformed the filters from literature significantly (p < 0.05) in terms of accuracy and robustness for long-duration tasks. For example, for foot MIMU, the median RMSE of (roll, pitch, yaw) were (6.5, 5.5, 7.8) and (22.8, 23.9, 25) deg for REKF and the best filter from the literature, respectively. For short-duration trials, REKF achieved significantly (p < 0.05) better or similar performance compared to the literature. We concluded that including non-gravitational acceleration, gyroscope bias, and magnetic disturbance in the state vector, as well as using a robust filter structure, is required for accurate and robust orientation tracking, at least in long-duration tasks.

BROAD—A Benchmark for Robust Inertial Orientation Estimation

Inertial measurement units (IMUs) enable orientation, velocity, and position estimation in several application domains ranging from robotics and autonomous vehicles to human motion capture and rehabilitation engineering. Errors in orientation estimation greatly affect any of those motion parameters. The present work explains the main challenges in inertial orientation estimation (IOE) and presents an extensive benchmark dataset that includes 3D inertial and magnetic data with synchronized optical marker-based ground truth measurements, the Berlin Robust Orientation Estimation Assessment Dataset (BROAD). The BROAD dataset consists of 39 trials that are conducted at different speeds and include various types of movement. Thereof, 23 trials are performed in an undisturbed indoor environment, and 16 trials are recorded with deliberate magnetometer and accelerometer disturbances. We furthermore propose error metrics that allow for IOE accuracy evaluation while separating the heading and inclination portions of the error and introduce well-defined benchmark metrics. Based on the proposed benchmark, we perform an exemplary case study on two widely used openly available IOE algorithms. Due to the broad range of motion and disturbance scenarios, the proposed benchmark is expected to provide valuable insight and useful tools for the assessment, selection, and further development of inertial sensor fusion methods and IMU-based application systems.

Real-time conversion of inertial measurement unit data to ankle joint angles using deep neural networks

Joint angle quantification from inertial measurement units (IMUs) is commonly performed using kinematic modelling, which depends on anatomical sensor placement and/or functional joint calibration; however, accurate three-dimensional joint motion measurement remains challenging to achieve. The aims of this study were firstly to employ deep neural networks to convert IMU data to ankle joint angles that are indistinguishable from those derived from motion capture-based inverse kinematics (IK) - the reference standard; and secondly, to validate the robustness of this approach across contrasting walking speeds in healthy individuals. Kinematics data were simultaneously calculated using IMUs and IK from 9 subjects during walking on a treadmill at 0.5 m/s, 1.0 m/s and 1.5 m/s. A generative adversarial network was trained using gait data at two of the walking speeds to predict ankle kinematics from IMU data alone for the third walking speed. There were significant differences between IK and IMU joint angle predictions for ankle eversion and internal rotation during walking at 0.5 m/s and 1.0 m/s (p < 0.001); however, no significant differences in joint angles were observed between the generative adversarial network prediction and IK at any speed or plane of joint motion (p < 0.05). The RMS difference in ankle joint kinematics between the generative adversarial network and IK for walking at 1.0 m/s was 3.8°, 2.1° and 3.5° for dorsiflexion, inversion and axial rotation, respectively. The modeling approach presented for real-time IMU to ankle joint angle conversion, which can be readily expanded to other joints, may provide enhanced IMU capability in applications such as telemedicine, remote monitoring and rehabilitation.

Analysis of the Accuracy of Ten Algorithms for Orientation Estimation Using Inertial and Magnetic Sensing under Optimal Conditions: One Size Does Not Fit All

The orientation of a magneto and inertial measurement unit (MIMU) is estimated by means of sensor fusion algorithms (SFAs) thus enabling human motion tracking. However, despite several SFAs implementations proposed over the last decades, there is still a lack of consensus about the best performing SFAs and their accuracy. As suggested by recent literature, the filter parameters play a central role in determining the orientation errors. The aim of this work is to analyze the accuracy of ten SFAs while running under the best possible conditions (i.e., their parameter values are set using the orientation reference) in nine experimental scenarios including three rotation rates and three commercial products. The main finding is that parameter values must be specific for each SFA according to the experimental scenario to avoid errors comparable to those obtained when the default parameter values are used. Overall, when optimally tuned, no statistically significant differences are observed among the different SFAs in all tested experimental scenarios and the absolute errors are included between 3.8 deg and 7.1 deg. Increasing the rotation rate generally leads to a significant performance worsening. Errors are also influenced by the MIMU commercial model. SFA MATLAB implementations have been made available online.

Reference Frame Unification of IMU-Based Joint Angle Estimation: The Experimental Investigation and a Novel Method

Inertial measurement unit (IMU)-based joint angle estimation is an increasingly mature technique that has a broad range of applications in clinics, biomechanics and robotics. However, the deviations of different IMUs’ reference frames, referring to IMUs’ individual orientations estimating errors, is still a challenge for improving the angle estimation accuracy due to conceptual confusion, relatively simple metrics and the lack of systematical investigation. In this paper, we clarify the determination of reference frame unification, experimentally study the time-varying characteristics of reference frames’ deviations and accordingly propose a novel method with a comprehensive metric to unify reference frames. To be specific, we firstly define the reference frame unification (RFU) and distinguish it with drift correction that has always been confused with the term RFU. Secondly, we design a mechanical gimbal-based experiment to study the deviations, where sensor-to-body alignment and rotation-caused differences of orientations are excluded. Thirdly, based on the findings of the experiment, we propose a novel method to utilize the consistency of the joint axis under the hinge-joint constraint, gravity acceleration and local magnetic field to comprehensively unify reference frames, which meets the nonlinear time-varying characteristics of the deviations. The results on ten human subjects reveal the feasibility of our proposed method and the improvement from previous methods. This work contributes to a relatively new perspective of considering and improving the accuracy of IMU-based joint angle estimation.

Towards User-Friendly Wearable Platforms for Monitoring Unconstrained Indoor and Outdoor Activities

Developing wearable platforms for unconstrained monitoring of limb movements has been an active recent topic of research due to potential applications such as clinical and athletic performance evaluation. However, practicality of these platforms might be affected by the dynamic and complexity of movements as well as characteristics of the surrounding environment. This paper addresses such issues by proposing a novel method for obtaining kinematic information of joints using a custom-designed wearable platform. The proposed method uses data from two gyroscopes and an array of textile stretch sensors to accurately track three-dimensional movements, including extension, flexion, and rotation, of a joint. More specifically, gyroscopes provide angular velocity data of two sides of a joint, while their relative orientation is estimated by a machine learning algorithm. An Unscented Kalman Filter (UKF) algorithm is applied to directly fuse angular velocity/relative orientation data and estimate the kinematic orientation of the joint. Experimental evaluations were carried out using data from 10 volunteers performing a series of predefined as well as unconstrained random three-dimensional trunk movements. Results show that the proposed sensor setup and the UKF-based data fusion algorithm can accurately estimate the orientation of the trunk relative to pelvis with an average error of less than 1.72 degrees in predefined movements and a comparable accuracy of 3.00 degrees in random movements. Moreover, the proposed platform is easy to setup, does not restrict body motion, and is not affected by environmental disturbances. This study is a further step towards developing user-friendly wearable sensor systems than can be readily used in indoor and outdoor settings without requiring bulky equipment or a tedious calibration phase.

A wearable motion capture device able to detect dynamic motion of human limbs

Limb motion capture is essential in human motion-recognition, motor-function assessment and dexterous human-robot interaction for assistive robots. Due to highly dynamic nature of limb activities, conventional inertial methods of limb motion capture suffer from serious drift and instability problems. Here, a motion capture method with integral-free velocity detection is proposed and a wearable device is developed by incorporating micro tri-axis flow sensors with micro tri-axis inertial sensors. The device allows accurate measurement of three-dimensional motion velocity, acceleration, and attitude angle of human limbs in daily activities, strenuous, and prolonged exercises. Additionally, we verify an intra-limb coordination relationship exists between thigh and shank in human walking and running, and establish a neural network model for it. Using the intra-limb coordination model, dynamic motion capture of human lower limbs including thigh and shank is tactfully implemented by a single shank-worn device, which simplifies the capture device and reduces cost. Experiments in strenuous activities and long-time running validate excellent performance and robustness of the wearable device in dynamic motion recognition and reconstruction of human limbs.

Current wearable motion capture technologies are unable to accurately detect dynamic motion of human limbs due to drift and instability problems. Here, the authors report a wearable motion capture device combining tri-axis velocity sensor and inertial sensors for accurate 3D limb motion capture.

Measuring upper arm elevation using an inertial measurement unit: An exploration of sensor fusion algorithms and gyroscope models

Many sensor fusion algorithms for analyzing human motion information collected with inertial measurement units have been reported in the scientific literature. Selecting which algorithm to use can be a challenge for ergonomists that may be unfamiliar with the strengths and limitations of the various options. In this paper, we describe fundamental differences among several algorithms, including differences in sensor fusion approach (e.g., complementary filter vs. Kalman Filter) and gyroscope error modeling (i.e., inclusion or exclusion of gyroscope bias). We then compare different sensor fusion algorithms considering the fundamentals discussed using laboratory-based measurements of upper arm elevation collected under three motion speeds. Results indicate peak displacement errors of <4.5° with a computationally efficient, non-proprietary complementary filter that did not account for gyroscope bias during each of the one-minute trials. Controlling for gyroscope bias reduced peak displacement errors to <3.0°. The complementary filters were comparable (<1° peak displacement difference) to the more complex Kalman filters.

External Disturbances Rejection for Vector Field Sensors in Attitude and Heading Reference Systems

The attitude and heading reference system (AHRS), which consists of tri-axial magnetometer, accelerometer, and gyroscope, has been widely adopted for three-dimensional attitude determination in recent years. It provides an economical means of passive navigation that only relies on gravity and geomagnetic fields. However, despite the advantages of small size, low cost, and low power, the magnetometer and accelerometer are susceptible to external disturbances, such as the magnetic interference from nearby ferromagnetic objects and current-carrying conductors, as well as the motional acceleration of the carrier. To eliminate such disturbances, a vector-based parallel structure is introduced for the attitude filter design, which can avoid the mutual interference between gravity and geomagnetic vectors. Meanwhile, an approach to estimate and compensate the external disturbances in real time for magnetometer and accelerometer is also presented. Compared with existing designs, the proposed filter architecture and external disturbance rejection algorithm can feasibly and effectively cooperate with mainstream data fusion techniques, including complementary filter and Kalman filter. According to experiment results, in the case that large and persistent external disturbances exist, the proposed method can improve the accuracy and robustness of attitude estimation, and it outperforms the existing methods such as switching filter and adaptive filter. Furthermore, through the experiments, the critical role of fading factor in handling the external disturbance is revealed.

An Open-Source 7-DOF Wireless Human Arm Motion-Tracking System for Use in Robotics Research

To extend the choice of inertial motion-tracking systems freely available to researchers and educators, this paper presents an alternative open-source design of a wearable 7-DOF wireless human arm motion-tracking system. Unlike traditional inertial motion-capture systems, the presented system employs a hybrid combination of two inertial measurement units and one potentiometer for tracking a single arm. The sequence of three design phases described in the paper demonstrates how the general concept of a portable human arm motion-tracking system was transformed into an actual prototype, by employing a modular approach with independent wireless data transmission to a control PC for signal processing and visualization. Experimental results, together with an application case study on real-time robot-manipulator teleoperation, confirm the applicability of the developed arm motion-tracking system for facilitating robotics research. The presented arm-tracking system also has potential to be employed in mechatronic system design education and related research activities. The system CAD design models and program codes are publicly available online and can be used by robotics researchers and educators as a design platform to build their own arm-tracking solutions for research and educational purposes.

Estimation and Observability Analysis of Human Motion on Lie Groups

This article proposes a framework for human-pose estimation from the wearable sensors that rely on a Lie group representation to model the geometry of the human movement. Human body joints are modeled by matrix Lie groups, using special orthogonal groups SO(2) and SO(3) for joint pose and special Euclidean group SE(3) for base-link pose representation. To estimate the human joint pose, velocity, and acceleration, we develop the equations for employing the extended Kalman filter on Lie groups (LG-EKF) to explicitly account for the non-Euclidean geometry of the state space. We present the observability analysis of an arbitrarily long kinematic chain of SO(3) elements based on a differential geometric approach, representing a generalization of kinematic chains of a human body. The observability is investigated for the system using marker position measurements. The proposed algorithm is compared with two competing approaches: 1) the extended Kalman filter (EKF) and 2) unscented KF (UKF) based on the Euler angle parametrization, in both simulations and extensive real-world experiments. The results show that the proposed approach achieves significant improvements over the Euler angle-based filters. It provides more accurate pose estimates, is not sensitive to gimbal lock, and more consistently estimates the covariances.

A Wearable Flow-MIMU Device for Monitoring Human Dynamic Motion

For personalized rehabilitation or sports training, it is necessary to monitor dynamic motions and track postures of human body and limbs. Traditional methods using micro inertial sensors usually suffer from drift problems in tracking dynamic motions. In this paper, a wearable flow-MIMU human motion capture device is proposed by incorporating micro flow sensor with micro inertial measurement unit (MIMU). Motion velocity is detected by a micro flow sensor and utilized to figure out the motion acceleration. The gravity accelerations are extracted by eliminating the motion accelerations from the accelerometer outputs. Finally, posture estimation is implemented by using a tailor-designed Kalman-based data fusion of the gyroscope outputs and the extracted gravity accelerations. The flow-MIMU device with wireless communication is designed like a watch to be wearable. Experimental results validate that the motion velocity, acceleration and posture of human limb are determined accurately and free of accumulative error in monitoring of dynamic motions using the proposed method. The wearable flow-MIMU device provides an advantageous monitoring approach for applications of personalized rehabilitation, sports training, intelligent prosthetics, etc.

A Novel Fuzzy-Adaptive Extended Kalman Filter for Real-Time Attitude Estimation of Mobile Robots

This paper proposes a novel fuzzy-adaptive extended Kalman filter (FAEKF) for the real-time attitude estimation of agile mobile platforms equipped with magnetic, angular rate, and gravity (MARG) sensor arrays. The filter structure employs both a quaternion-based EKF and an adaptive extension, in which novel measurement methods are used to calculate the magnitudes of system vibrations, external accelerations, and magnetic distortions. These magnitudes, as external disturbances, are incorporated into a sophisticated fuzzy inference machine, which executes fuzzy IF-THEN rules-based adaption laws to consistently modify the noise covariance matrices of the filter, thereby providing accurate and robust attitude results. A six-degrees of freedom (6 DOF) test bench is designed for filter performance evaluation, which executes various dynamic behaviors and enables measurement of the true attitude angles (ground truth) along with the raw MARG sensor data. The tuning of filter parameters is performed with numerical optimization based on the collected measurements from the test environment. A comprehensive analysis highlights that the proposed adaptive strategy significantly improves the attitude estimation quality. Moreover, the filter structure successfully rejects the effects of both slow and fast external perturbations. The FAEKF can be applied to any mobile system in which attitude estimation is necessary for localization and external disturbances greatly influence the filter accuracy.

Magnetic Condition-Independent 3D Joint Angle Estimation Using Inertial Sensors and Kinematic Constraints

In biomechanics, joint angle estimation using wearable inertial measurement units (IMUs) has been getting great popularity. However, magnetic disturbance issue is considered problematic as the disturbance can seriously degrade the accuracy of the estimated joint angles. This study proposes a magnetic condition-independent three-dimensional (3D) joint angle estimation method based on IMU signals. The proposed method is implemented in a sequential direction cosine matrix-based orientation Kalman filter (KF), which is composed of an attitude estimation KF followed by a heading estimation KF. In the heading estimation KF, an acceleration-level kinematic constraint from a spherical joint replaces the magnetometer signals for the correction procedure. Because the proposed method does not rely on the magnetometer, it is completely magnetic condition-independent and is not affected by the magnetic disturbance. For the averaged root mean squared errors of the three tests performed using a rigid two-link system, the proposed method produced 1.58°, while the conventional method with the magnetic disturbance compensation mechanism produced 5.38°, showing a higher accuracy of the proposed method in the magnetically disturbed conditions. Due to the independence of the proposed method from the magnetic condition, the proposed approach could be reliably applied in various fields that require robust 3D joint angle estimation through IMU signals in an unspecified arbitrary magnetic environment.

Testing of a MEMS Dynamic Inclinometer Using the Stewart Platform

The micro-electro-mechanical system (MEMS) dynamic inclinometer integrates a tri-axis gyroscope and a tri-axis accelerometer for real-time tilt measurement. The Stewart platform has the ability to generate six degrees of freedom of spatial orbits. The method of applying spatial orbits to the testing of MEMS inclinometers is investigated. Inverse and forward kinematics are analyzed for controlling and measuring the position and orientation of the Stewart platform. The Stewart platform is controlled to generate a conical motion, based on which the sensitivities of the gyroscope, accelerometer, and tilt sensing are determined. Spatial positional orbits are also generated in order to obtain the tilt angles caused by the cross-coupling influence. The experiment is conducted to show that the tested amplitude frequency deviations of the gyroscope and tilt sensing sensitivities between the Stewart platform and the traditional rotator are less than 0.2 dB and 0.1 dB, respectively.

UKF Magnetometer-Free Sensor Fusion for Pelvis Pose Estimation During Treadmill Walking

Inertial measurement units are an efficient tool to estimate the orientation of a rigid body with respect to a global or navigation frame. Thanks to their relatively small scale, these devices are often employed in clinical environments in form of wearable devices. A direct consequence of this large use of inertial sensors has been the development of many sensor fusion techniques for pose estimation in many practical applications. In this paper we study the feasibility of a nonlinear "Unscented" variant of the well-known Kalman Filter for gyroscope/accelerometer sensor fusion in pelvis pose estimation during treadmill walking. In addition, orientation estimation has been obtained without IMU magnetometer data, in order to propose a method suitable also for environments where magnetic disturbances could arise. Pelvis heading (yaw), bank (roll) and attitude (pitch) angles have been evaluated both using the proposed filter and a gold standard optometric system. The root mean square errors obtained using the proposed sensor fusion with respect to the gold standard are below 1 degree for each axis, showing also a significant high correlation (> 0.90). Findings of this study highlight the suitability of a magnetometer-free UKF approach for pose estimation of pelvis during human walking on treadmill, providing information useful also for further estimation of center of mass displacement in the same experimental conditions.

Measurement Noise Recommendation for Efficient Kalman Filtering over a Large Amount of Sensor Data

To effectively maintain and analyze a large amount of real-time sensor data, one often uses a filtering technique that reflects characteristics of original data well. This paper proposes a novel method for recommending the measurement noise for Kalman filtering, which is one of the most representative filtering techniques. Kalman filtering corrects inaccurate values of input sensor data, and its filtering performance varies depending on the input noise parameters. In particular, if the noise parameters determined based on the user’s experience are incorrect, the accuracy of Kalman filtering may be reduced significantly. Based on this observation, this paper addresses how to determine the measurement noise variance, a major input parameter of Kalman filtering, by analyzing past sensor data and how to use the estimated noise to improve the filtering accuracy. More specifically, to estimate the measurement noise variance, two analytical methods are proposed: one a transform-based method using a wavelet transform and the other a learning-based method using a denoising autoencoder. Experimental results show that the proposed methods estimated the measurement noise variance accurately and were superior to the experience-based method in the filtering accuracy.

Robust Inertial Measurement Unit-Based Attitude Determination Kalman Filter for Kinematically Constrained Links

The external acceleration of a fast-moving body induces uncertainty in attitude determination based on inertial measurement unit (IMU) signals and thus, frequently degrades the determination accuracy. Although previous works adopt acceleration-compensating mechanisms to deal with this problem, they cannot completely eliminate the uncertainty as they are, inherently, approaches to an underdetermined problem. This paper presents a novel constraint-augmented Kalman filter (KF) that eliminates the acceleration-induced uncertainty for a robust IMU-based attitude determination when IMU is attached to a constrained link. Particularly, this research deals with an acceleration-level kinematic constraint derived on the basis of a ball joint. Experimental results demonstrate the superiority of the proposed constrained KF over the conventional unconstrained KF: The average accuracy improved by 1.88° with a maximum of 4.18°. More importantly, whereas the accuracy of conventional KF is dependent to some extent on test acceleration conditions, that of the proposed KF is independent of these conditions. Due to the robustness of the proposed KF, it may be applied when accurate attitude estimation is needed regardless of dynamic conditions.

Wearables, Biomechanical Feedback, and Human Motor-Skills’ Learning & Optimization

Biomechanical feedback is a relevant key to improving sports and arts performance. Yet, the bibliometric keyword analysis on Web of Science publications reveals that, when comparing to other biofeedback applications, the real-time biomechanical feedback application lags far behind in sports and arts practice. While real-time physiological and biochemical biofeedback have seen routine applications, the use of real-time biomechanical feedback in motor learning and training is still rare. On that account, the paper aims to extract the specific research areas, such as three-dimensional (3D) motion capture, anthropometry, biomechanical modeling, sensing technology, and artificial intelligent (AI)/deep learning, which could contribute to the development of the real-time biomechanical feedback system. The review summarizes the past and current state of biomechanical feedback studies in sports and arts performance; and, by integrating the results of the studies with the contemporary wearable technology, proposes a two-chain body model monitoring using six IMUs (inertial measurement unit) with deep learning technology. The framework can serve as a basis for a breakthrough in the development. The review indicates that the vital step in the development is to establish a massive data, which could be obtained by using the synchronized measurement of 3D motion capture and IMUs, and that should cover diverse sports and arts skills. As such, wearables powered by deep learning models trained by the massive and diverse datasets can supply a feasible, reliable, and practical biomechanical feedback for athletic and artistic training.

HUSP: A Smart Haptic Probe for Reliable Training in Musculoskeletal Evaluation Using Motion Sensors

As a consequence of the huge development of IMU (Inertial Measurement Unit) sensors based on MEMS (Micro-Electromechanical Systems), innovative applications related to the analysis of human motion are now possible. In this paper, we present one of these applications: a portable platform for training in Ultrasound Imaging-based musculoskeletal (MSK) exploration in rehabilitation settings. Ultrasound Imaging (USI) in the diagnostic and treatment of MSK pathologies offers various advantages, but it is a strongly operator-dependent technique, so training and experience become of fundamental relevance for rehabilitation specialists. The key element of our platform is a replica of a real transducer (HUSP—Haptic US Probe), equipped with MEMS based IMU sensors, an embedded computing board to calculate its 3D orientation and a mouse board to obtain its relative position in the 2D plane. The sensor fusion algorithm used to resolve in real-time the 3D orientation (roll, pitch and yaw angles) of the probe from accelerometer, gyroscope and magnetometer data will be presented. Thanks to the results obtained, the integration of the probe into the learning platform allows a haptic sensation to be recreated in the rehabilitation trainee, with an attractive performance/cost ratio.

Estimating Three-Dimensional Body Orientation Based on an Improved Complementary Filter for Human Motion Tracking

Rigid body orientation determined by IMU (Inertial Measurement Unit) is widely applied in robotics, navigation, rehabilitation, and human-computer interaction. In this paper, aiming at dynamically fusing quaternions computed from angular rate integration and FQA algorithm, a quaternion-based complementary filter algorithm is proposed to support a computationally efficient, wearable motion-tracking system. Firstly, a gradient descent method is used to determine a function from several sample points. Secondly, this function is used to dynamically estimate the fusion coefficient based on the deviation between measured magnetic field, gravity vectors and their references in Earth-fixed frame. Thirdly, a test machine is designed to evaluate the performance of designed filter. Experimental results validate the filter design and show its potential of real-time human motion tracking.

Robust and Accurate Capture of Human Joint Pose Using an Inertial Sensor

Wearable inertial measurement units (IMU) measuring acceleration, earth magnetic field, and gyroscopic measurements can be considered for capturing human skeletal postures in real time. Number of movement disorders require accurate and robust estimation of the human joint pose. Though these movements are inherently slow, the accuracy of estimation is vital as many subtle moment patterns, such as tremor are useful to capture under many assessments scenarios. Also, as the end user is a patient with movement disabilities, the practical wearability aspects impose stringent requirements such as the use of minimal number of sensors as well as positioning them in conformable areas of the human body; particularly for longer term monitoring. Estimating skeletal and limb orientations to describe human posture dynamically via model-based approaches poses numerous challenges. In this paper, we convey that the use of measurement conversion ideas-a representation signifying a linear characterization of an inherently non linear estimation problem, pragmatically improves the overall estimation of the limb orientation. A quaternion, as opposed to the Euler angle-based approach is adopted to avoid Gimbal lock scenarios. We also lay a systematic basis for quaternion normalization, typically performed in the pre-filtering stage, by introducing an optimization-based mathematical justification. A robust version of the extended Kalman filter is configured to amalgamate the underlying ideas in enhancing the overall system performance while providing a structured and a comprehensive approach to IMU-based real time human pose estimation problem, particularly in a movement disability capture context.

Predictive trajectory estimation during rehabilitative tasks in augmented reality using inertial sensors

This paper presents a wireless kinematic tracking framework used for biomechanical analysis during rehabilitative tasks in augmented and virtual reality. The framework uses low-cost inertial measurement units and exploits the rigid connections of the human skeletal system to provide egocentric position estimates of joints to centimeter accuracy. On-board sensor fusion combines information from three-axis accelerometers, gyroscopes, and magnetometers to provide robust estimates in real-time. Sensor precision and accuracy were validated using the root mean square error of estimated joint angles against ground truth goniometer measurements. The sensor network produced a mean estimate accuracy of 2.81° with 1.06° precision, resulting in a maximum hand tracking error of 7.06 cm. As an application, the network is used to collect kinematic information from an unconstrained object manipulation task in augmented reality, from which dynamic movement primitives are extracted to characterize natural task completion in N = 3 able-bodied human subjects. These primitives are then leveraged for trajectory estimation in both a generalized and a subject-specific scheme resulting in 0.187 cm and 0.161 cm regression accuracy, respectively. Our proposed kinematic tracking network is wireless, accurate, and especially useful for predicting voluntary actuation in virtual and augmented reality applications.

Accurate Device-Free Tracking Using Inexpensive RFIDs

Without requiring targets to carry any device, device-free-based tracking is playing an important role in many emerging applications such as smart homes, fitness tracking, intruder detection, etc. While promising, current device-free tracking systems based on inexpensive commercial devices perform well in the training environment, but poorly in other environments because of different multipath reflections. This paper introduces RDTrack, a system that leverages changes in Doppler shifts, which are not sensitive to multipath, to accurately track the target. Moreover, RDTrack identifies particular patterns for fine-grained motions such as turning, walking straightly, etc., which can achieve accurate tracking. For the purpose of achieving a fine-grained device-free tracking system, this paper builds a trajectory estimating model using HMM (Hidden Markov Model) to improve the matching accuracy and reduce the time complexity. We address several challenges including estimating the tag influenced time period, identifying moving path and reducing false positives due to multipath. We implement RDTrack with inexpensive commercial off-the-shelf RFID (Radio Frequency IDentification) hardware and extensively evaluate RDTrack in a lobby, staircase and library. Our results show that RDTrack is effective in tracking the moving target, with a low tracking error of 32 cm. This accuracy is robust for different environments, highlighting RDTrack’s ability to enable future essential device-free moving-based interaction with RFID devices.

Effect of Strapdown Integration Order and Sampling Rate on IMU-Based Attitude Estimation Accuracy

This paper deals with the strapdown integration of attitude estimation Kalman filter (KF) based on inertial measurement unit (IMU) signals. In many low-cost wearable IMU applications, a first-order is selected for strapdown integration, which may degrade attitude estimation performance in high-speed angular motions. The purpose of this research is to provide insights into the effect of the strapdown integration order and sampling rate on the attitude estimation accuracy for low-cost IMU applications. Experimental results showed that the effect of integration order was small when the angular velocity was low and the sampling rate was large. However, as the angular velocity increased and the sampling rate decreased, the effect of integration order increased, i.e., obviously, the third-order KF resulted in better estimations than the first-order KF. When comparing the case where both transient matrix and process noise covariance matrix are applied to the corresponding order and the case where only the transient matrix is applied to the corresponding order but the process noise covariance matrix for the first-order is still used, both cases had almost equivalent estimation accuracy. However, in terms of the calculation cost, the latter case was more economical than the former, particularly for the third-order KF (i.e., the ratio of the former to the latter is 1.22 to 1).

Design of Ensemble Stacked Auto-Encoder for Classification of Horse Gaits with MEMS Inertial Sensor Technology

This paper discusses the classification of horse gaits for self-coaching using an ensemble stacked auto-encoder (ESAE) based on wavelet packets from the motion data of the horse rider. For this purpose, we built an ESAE and used probability values at the end of the softmax classifier. First, we initialized variables such as hidden nodes, weight, and max epoch using the options of the auto-encoder (AE). Second, the ESAE model is trained by feedforward, back propagation, and gradient calculation. Next, the parameters are updated by a gradient descent mechanism as new parameters. Finally, once the error value is satisfied, the algorithm terminates. The experiments were performed to classify horse gaits for self-coaching. We constructed the motion data of a horse rider. For the experiment, an expert horse rider of the national team wore a suit containing 16 inertial sensors based on a wireless network. To improve and quantify the performance of the classification, we used three methods (wavelet packet, statistical value, and ensemble model), as well as cross entropy with mean squared error. The experimental results revealed that the proposed method showed good performance when compared with conventional algorithms such as the support vector machine (SVM).

Accuracy of angular displacements and velocities from inertial-based inclinometers

The objective of this study was to evaluate the accuracy of various sensor fusion algorithms for measuring upper arm elevation relative to gravity (i.e., angular displacement and velocity summary measures) across different motion speeds. Thirteen participants completed a cyclic, short duration, arm-intensive work task that involved transfering wooden dowels at three work rates (slow, medium, fast). Angular displacement and velocity measurements of upper arm elevation were simultaneously measured using an inertial measurement unit (IMU) and an optical motion capture (OMC) system. Results indicated that IMU-based inclinometer solutions can reduce root-mean-square errors in comparison to accelerometer-based inclination estimates by as much as 87%, depending on the work rate and sensor fusion approach applied. The findings suggest that IMU-based inclinometers can substantially improve inclinometer accuracy in comparison to traditional accelerometer-based inclinometers. Ergonomists may use the non-proprietary sensor fusion algorithms provided here to more accurately estimate upper arm elevation.

Characterizing exposure to physical risk factors among reforestation hand planters in the Southeastern United States

Low back and neck/shoulder pain are commonly reported among reforestation hand planters. While some studies have documented the intensive cardiovascular demands of hand planting, limited information is available regarding exposures to physical risk factors associated with the development of musculoskeletal disorders (MSDs) among hand planters. This study used surface electromyography (EMG) and inertial measurement units (IMUs) to characterize the muscle activation patterns, upper arm and trunk postures, movement velocities, and physical activity (PA) of fourteen Southeastern reforestation hand planters over one work shift. Results indicated that hand planters are exposed to physical risk factors such as extreme trunk postures (32.5% of time spent in ≥45° trunk flexion) and high effort muscle exertions (e.g., mean root-mean-square right upper trapezius amplitude of 54.1% reference voluntary exertion) that may place them at increased risk for developing MSDs. The findings indicate a need for continued field-based research among hand planters to identify and/or develop maximally effective interventions.

How Magnetic Disturbance Influences the Attitude and Heading in Magnetic and Inertial Sensor-Based Orientation Estimation

With the advancements in micro-electromechanical systems (MEMS) technologies, magnetic and inertial sensors are becoming more and more accurate, lightweight, smaller in size as well as low-cost, which in turn boosts their applications in human movement analysis. However, challenges still exist in the field of sensor orientation estimation, where magnetic disturbance represents one of the obstacles limiting their practical application. The objective of this paper is to systematically analyze exactly how magnetic disturbances affects the attitude and heading estimation for a magnetic and inertial sensor. First, we reviewed four major components dealing with magnetic disturbance, namely decoupling attitude estimation from magnetic reading, gyro bias estimation, adaptive strategies of compensating magnetic disturbance and sensor fusion algorithms. We review and analyze the features of existing methods of each component. Second, to understand each component in magnetic disturbance rejection, four representative sensor fusion methods were implemented, including gradient descent algorithms, improved explicit complementary filter, dual-linear Kalman filter and extended Kalman filter. Finally, a new standardized testing procedure has been developed to objectively assess the performance of each method against magnetic disturbance. Based upon the testing results, the strength and weakness of the existing sensor fusion methods were easily examined, and suggestions were presented for selecting a proper sensor fusion algorithm or developing new sensor fusion method.

An Unobtrusive Fall Detection and Alerting System Based on Kalman Filter and Bayes Network Classifier

Falls are one of the main health risks among the elderly. A fall detection system based on inertial sensors can automatically detect fall event and alert a caregiver for immediate assistance, so as to reduce injuries causing by falls. Nevertheless, most inertial sensor-based fall detection technologies have focused on the accuracy of detection while neglecting quantization noise caused by inertial sensor. In this paper, an activity model based on tri-axial acceleration and gyroscope is proposed, and the difference between activities of daily living (ADLs) and falls is analyzed. Meanwhile, a Kalman filter is proposed to preprocess the raw data so as to reduce noise. A sliding window and Bayes network classifier are introduced to develop a wearable fall detection system, which is composed of a wearable motion sensor and a smart phone. The experiment shows that the proposed system distinguishes simulated falls from ADLs with a high accuracy of 95.67%, while sensitivity and specificity are 99.0% and 95.0%, respectively. Furthermore, the smart phone can issue an alarm to caregivers so as to provide timely and accurate help for the elderly, as soon as the system detects a fall.

An Adaptive Orientation Estimation Method for Magnetic and Inertial Sensors in the Presence of Magnetic Disturbances

Magnetic and inertial sensors have been widely used to estimate the orientation of human segments due to their low cost, compact size and light weight. However, the accuracy of the estimated orientation is easily affected by external factors, especially when the sensor is used in an environment with magnetic disturbances. In this paper, we propose an adaptive method to improve the accuracy of orientation estimations in the presence of magnetic disturbances. The method is based on existing gradient descent algorithms, and it is performed prior to sensor fusion algorithms. The proposed method includes stationary state detection and magnetic disturbance severity determination. The stationary state detection makes this method immune to magnetic disturbances in stationary state, while the magnetic disturbance severity determination helps to determine the credibility of magnetometer data under dynamic conditions, so as to mitigate the negative effect of the magnetic disturbances. The proposed method was validated through experiments performed on a customized three-axis instrumented gimbal with known orientations. The error of the proposed method and the original gradient descent algorithms were calculated and compared. Experimental results demonstrate that in stationary state, the proposed method is completely immune to magnetic disturbances, and in dynamic conditions, the error caused by magnetic disturbance is reduced by 51.2% compared with original MIMU gradient descent algorithm.

Global Kalman filter approaches to estimate absolute angles of lower limb segments

Background

In this paper we propose the use of global Kalman filters (KFs) to estimate absolute angles of lower limb segments. Standard approaches adopt KFs to improve the performance of inertial sensors based on individual link configurations. In consequence, for a multi-body system like a lower limb exoskeleton, the inertial measurements of one link (e.g., the shank) are not taken into account in other link angle estimations (e.g., foot). Global KF approaches, on the other hand, correlate the collective contribution of all signals from lower limb segments observed in the state-space model through the filtering process. We present a novel global KF (matricial global KF) relying only on inertial sensor data, and validate both this KF and a previously presented global KF (Markov Jump Linear Systems, MJLS-based KF), which fuses data from inertial sensors and encoders from an exoskeleton. We furthermore compare both methods to the commonly used local KF.

Results

The results indicate that the global KFs performed significantly better than the local KF, with an average root mean square error (RMSE) of respectively 0.942° for the MJLS-based KF, 1.167° for the matrical global KF, and 1.202° for the local KFs. Including the data from the exoskeleton encoders also resulted in a significant increase in performance.

Conclusion

The results indicate that the current practice of using KFs based on local models is suboptimal. Both the presented KF based on inertial sensor data, as well our previously presented global approach fusing inertial sensor data with data from exoskeleton encoders, were superior to local KFs. We therefore recommend to use global KFs for gait analysis and exoskeleton control.

Electronic supplementary material

The online version of this article (doi:10.1186/s12938-017-0346-7) contains supplementary material, which is available to authorized users.

Body position classification for cardiorespiratory measurement

Heart activity, or at least heart rate variability, is associated with body position. Our previous studies have confirmed that impedance pneumography may be used to record respiratory function, but the calibration coefficients for this method depend on position. Data were collected from 24 students (12 male, 12 female), who alternated positions between lying (on front, back, and right side), sitting and standing. Signals from an attached iPhone's internal sensors (accelerometer, gyroscope, magnetometer) were recorded and attitude relative to gravity was calculated. The signals were subsequently segmented and marked. Five algorithms were trained and cross-validated for different sensor combinations. Without differentiation of sitting and standing, 100% accuracy was achieved using all algorithms. The classifier best differentiating these two states was based on random forests, with overall accuracy of 90%. Simple methods based on a proposed hybrid classifier were tested for online measurement without the need for signal segmentation, with 99% accuracy. The prospect of the algorithms' use in long-term studies (particularly cardiorespiratory monitoring) was assessed.

Effects of Movement Speed and Magnetic Disturbance on the Accuracy of Inertial Measurement Units

This study evaluated the effects of motion speed and magnetic disturbance on the spatial orientation accuracy of an inertial measurement unit (IMU) on the hand. Thirteen participants performed six trials of a repetitive material transfer task. Movement speed (15, 30, 45 transfers/minute) and magnetic disturbance (absent, present) were the independent variables. Optical motion capture was the reference. Root-mean-square differences (RMSD) exceeded 20° when inclination measurements (pitch and roll) were calculated using the IMU accelerometer. A linear Kalman filter and a proprietary, embedded Kalman filter reduced inclination RMSD to <3° across all movement speeds. The RMSD in the heading direction (i.e., about gravity) increased (from <5° to 17°) under magnetic disturbance. The linear Kalman filter and the embedded Kalman filter reduced heading RMSD to <12° and <7°, respectively. Use of IMUs and Kalman filters can improve inclinometer measurement accuracy. However, magnetic disturbances continue to limit the accuracy of three-dimensional IMU motion capture.

Regression Model-Based Walking Speed Estimation Using Wrist-Worn Inertial Sensor

Walking speed is widely used to study human health status. Wearable inertial measurement units (IMU) are promising tools for the ambulatory measurement of walking speed. Among wearable inertial sensors, the ones worn on the wrist, such as a watch or band, have relatively higher potential to be easily incorporated into daily lifestyle. Using the arm swing motion in walking, this paper proposes a regression model-based method for longitudinal walking speed estimation using a wrist-worn IMU. A novel kinematic variable is proposed, which finds the wrist acceleration in the principal axis (i.e. the direction of the arm swing). This variable (called pca-acc) is obtained by applying sensor fusion on IMU data to find the orientation followed by the use of principal component analysis. An experimental evaluation was performed on 15 healthy young subjects during free walking trials. The experimental results show that the use of the proposed pca-acc variable can significantly improve the walking speed estimation accuracy when compared to the use of raw acceleration information (p<0.01). When Gaussian process regression is used, the resulting walking speed estimation accuracy and precision is about 5.9% and 4.7%, respectively.