Rhythmic Extended Kalman Filter for Gait Rehabilitation Motion Estimation and Segmentation

Bridging the sim2real gap. Investigating deviations between experimental motion measurements and musculoskeletal simulation results-a systematic review

Musculoskeletal simulations can be used to estimate biomechanical variables like muscle forces and joint torques from non-invasive experimental data using inverse and forward methods. Inverse kinematics followed by inverse dynamics (ID) uses body motion and external force measurements to compute joint movements and the corresponding joint loads, respectively. ID leads to residual forces and torques (residuals) that are not physically realistic, because of measurement noise and modeling assumptions. Forward dynamic simulations (FD) are found by tracking experimental data. They do not generate residuals but will move away from experimental data to achieve this. Therefore, there is a gap between reality (the experimental measurements) and simulations in both approaches, the sim2real gap. To answer (patho-) physiological research questions, simulation results have to be accurate and reliable; the sim2real gap needs to be handled. Therefore, we reviewed methods to handle the sim2real gap in such musculoskeletal simulations. The review identifies, classifies and analyses existing methods that bridge the sim2real gap, including their strengths and limitations. Using a systematic approach, we conducted an electronic search in the databases Scopus, PubMed and Web of Science. We selected and included 85 relevant papers that were sorted into eight different solution clusters based on three aspects: how the sim2real gap is handled, the mathematical method used, and the parameters/variables of the simulations which were adjusted. Each cluster has a distinctive way of handling the sim2real gap with accompanying strengths and limitations. Ultimately, the method choice largely depends on various factors: available model, input parameters/variables, investigated movement and of course the underlying research aim. Researchers should be aware that the sim2real gap remains for both ID and FD approaches. However, we conclude that multimodal approaches tracking kinematic and dynamic measurements may be one possible solution to handle the sim2real gap as methods tracking multimodal measurements (some combination of sensor position/orientation or EMG measurements), consistently lead to better tracking performances. Initial analyses show that motion analysis performance can be enhanced by using multimodal measurements as different sensor technologies can compensate each other's weaknesses.

SensAA-Design and Verification of a Cloud-Based Wearable Biomechanical Data Acquisition System

Exoskeletons designed to assist patients with activities of daily living are becoming increasingly popular, but still are subject to research. In order to gather requirements for the design of such systems, long-term gait observation of the patients over the course of multiple days in an environment of daily living are required. In this paper a wearable all-in-one data acquisition system for collecting and storing biomechanical data in everyday life is proposed. The system is designed to be cost efficient and easy to use, using off-the-shelf components and a cloud server system for centralized data storage. The measurement accuracy of the system was verified, by measuring the angle of the human knee joint at walking speeds between 3 and 12 km/h in reference to an optical motion analysis system. The acquired data were uploaded to a cloud database via a smartphone application. Verification results showed that the proposed toolchain works as desired. The system reached an RMSE from 2.9° to 8°, which is below that of most comparable systems. The system provides a powerful, scalable platform for collecting and processing biomechanical data, which can help to automize the generation of an extensive database for human kinematics.

Human Gait Activity Recognition Using Multimodal Sensors

Human activity recognition is an application of machine learning with the aim of identifying activities from the gathered activity raw data acquired by different sensors. In medicine, human gait is commonly analyzed by doctors to detect abnormalities and determine possible treatments for the patient. Monitoring the patient's activity is paramount in evaluating the treatment's evolution. This type of classification is still not enough precise, which may lead to unfavorable reactions and responses. A novel methodology that reduces the complexity of extracting features from multimodal sensors is proposed to improve human activity classification based on accelerometer data. A sliding window technique is used to demarcate the first dominant spectral amplitude, decreasing dimensionality and improving feature extraction. In this work, we compared several state-of-art machine learning classifiers evaluated on the HuGaDB dataset and validated on our dataset. Several configurations to reduce features and training time were analyzed using multimodal sensors: all-axis spectrum, single-axis spectrum, and sensor reduction.

Lower-limb kinematic reconstruction during pedaling tasks from EEG signals using Unscented Kalman filter

Kinematic reconstruction of lower-limb movements using electroencephalography (EEG) has been used in several rehabilitation systems. However, the nonlinear relationship between neural activity and limb movement may challenge decoders in real-time Brain-Computer Interface (BCI) applications. This paper proposes a nonlinear neural decoder using an Unscented Kalman Filter (UKF) to infer lower-limb kinematics from EEG signals during pedaling. The results demonstrated maximum decoding accuracy using slow cortical potentials in the delta band (0.1-4 Hz) of 0.33 for Pearson's r-value and 8 for the signal-to-noise ratio (SNR). This leaves an open door to the development of closed-loop EEG-based BCI systems for kinematic monitoring during pedaling rehabilitation tasks.

A scoping review on recent trends in wearable sensors to analyze gait in people with stroke: From sensor placement to validation against gold-standard equipment

The purpose of the review is to evaluate wearable sensor placement, their impact and validation of wearable sensors on analyzing gait, primarily the postural instability in people with stroke. Databases, namely PubMed, Cochrane, SpringerLink, and IEEE Xplore were searched to identify related articles published since January 2005. The authors have selected the articles by considering patient characteristics, intervention details, and outcome measurements by following the priorly set inclusion and exclusion criteria. From a total of 1077 articles, 142 were included in this study and classified into functional fields, namely postural stability (PS) assessments, physical activity monitoring (PA), gait pattern classification (GPC), and foot drop correction (FDC). The review covers the types of wearable sensors, their placement, and their performance in terms of reliability and validity. When employing a single wearable sensor, the pelvis and foot were the most used locations for detecting gait asymmetry and kinetic parameters, respectively. Multiple Inertial Measurement Units placed at different body parts were effectively used to estimate postural stability and gait pattern. This review article has compared results of placement of sensors at different locations helping researchers and clinicians to identify the best possible placement for sensors to measure specific kinematic and kinetic parameters in persons with stroke.

Sparse Visual-Inertial Measurement Units Placement for Gait Kinematics Assessment

This study investigates the possibility of estimating lower-limb joint kinematics and meaningful performance indexes for physiotherapists, during gait on a treadmill based on data collected from a sparse placement of new Visual Inertial Measurement Units (VIMU) and the use of an Extended Kalman Filter (EKF). The proposed EKF takes advantage of the biomechanics of the human body and of the investigated task to reduce sensor inaccuracies. Two state-vector formulations, one based on the use of constant acceleration model and one based on Fourier series, and the tuning of their corresponding parameters were analyzed. The constant acceleration model, due to its inherent inconsistency for human motion, required a cumbersome optimisation process and needed the a-priori knowledge of reference joint trajectories for EKF parameters tuning. On the other hand, the Fourier series formulation could be used without a specific parameters tuning process. In both cases, the average root mean square difference and correlation coefficient between the estimated joint angles and those reconstructed with a reference stereophotogrammetric system was 3.5deg and 0.70, respectively. Moreover, the stride lengths were estimated with a normalized root mean square difference inferior to 2% when using the forward kinematics model receiving as input the estimated joint angles. The popular gait deviation index was also estimated and showed similar results very close to 100, using both the proposed method and the reference stereophotogrammetric system. Such consistency was obtained using only three wireless and affordable VIMU located at the pelvis and both heels and tracked using two affordable RGB cameras. Being further easy-to-use and suitable for applications taking place outside of the laboratory, the proposed method thus represents a good compromise between accurate reference stereophotogrammetric systems and markerless ones for which accuracy is still under debate.

Lower body kinematics estimation from wearable sensors for walking and running: A deep learning approach

BACKGROUND

Inertial measurement units (IMUs) are promising tools for collecting human movement data. Model-based filtering approaches (e.g. Extended Kalman Filter) have been proposed to estimate joint angles from IMUs data but little is known about the potential of data-driven approaches.

RESEARCH QUESTION

Can deep learning models accurately predict lower limb joint angles from IMU data during gait?

METHODS

Lower-limb kinematic data were simultaneously measured with a marker-based motion capture system and running leggings with 5 integrated IMUs measuring acceleration and angular velocity at the pelvis, thighs and tibias. Data acquisition was performed on 27 participants (26.5 (3.9) years, 1.75 (0.07) m, 68.3 (10.0) kg) while walking at 4 and 6 km/h and running at 8, 10, 12 and 14 km/h on a treadmill. The model input consists of raw IMU data, while the output estimates the joint angles of the lower body. The model was trained with a nested k-fold cross-validation and tested considering a user-independent approach. Mean error (ME), mean absolute error (MAE) and Pearson correlation coefficient (r) were computed between the ground truth and predicted joint angles.

RESULTS

MAE for the DOFs ranged from 2.2(0.9) to 5.1(2.7)° with an average of 3.6(2.1)°. r ranged from 0.67(0.23) to 0.99(0.01) with moderate correlation (0.4≤r<0.7) was found for the hip right rotation and lumbar extension, strong correlation (0.7≤r<0.9) was found for the hip left rotation and ankle right/left inversion while all other DOFs showed very strong correlation (r≥0.9).

SIGNIFICANCE

The proposed model can reliably predict joint kinematics for walking, running and gait transitions without specific knowledge about the body characteristics of the wearer, or the position and orientation of the IMU relative to the attached segment. These results have been validated with treadmill gait, and have not yet been confirmed for gait in other settings.

Estimation and Observability Analysis of Human Motion on Lie Groups

This article proposes a framework for human-pose estimation from the wearable sensors that rely on a Lie group representation to model the geometry of the human movement. Human body joints are modeled by matrix Lie groups, using special orthogonal groups SO(2) and SO(3) for joint pose and special Euclidean group SE(3) for base-link pose representation. To estimate the human joint pose, velocity, and acceleration, we develop the equations for employing the extended Kalman filter on Lie groups (LG-EKF) to explicitly account for the non-Euclidean geometry of the state space. We present the observability analysis of an arbitrarily long kinematic chain of SO(3) elements based on a differential geometric approach, representing a generalization of kinematic chains of a human body. The observability is investigated for the system using marker position measurements. The proposed algorithm is compared with two competing approaches: 1) the extended Kalman filter (EKF) and 2) unscented KF (UKF) based on the Euler angle parametrization, in both simulations and extensive real-world experiments. The results show that the proposed approach achieves significant improvements over the Euler angle-based filters. It provides more accurate pose estimates, is not sensitive to gimbal lock, and more consistently estimates the covariances.

Inertial Sensor-Based Lower Limb Joint Kinematics: A Methodological Systematic Review

The use of inertial measurement units (IMUs) has gained popularity for the estimation of lower limb kinematics. However, implementations in clinical practice are still lacking. The aim of this review is twofold—to evaluate the methodological requirements for IMU-based joint kinematic estimation to be applicable in a clinical setting, and to suggest future research directions. Studies within the PubMed, Web Of Science and EMBASE databases were screened for eligibility, based on the following inclusion criteria: (1) studies must include a methodological description of how kinematic variables were obtained for the lower limb, (2) kinematic data must have been acquired by means of IMUs, (3) studies must have validated the implemented method against a golden standard reference system. Information on study characteristics, signal processing characteristics and study results was assessed and discussed. This review shows that methods for lower limb joint kinematics are inherently application dependent. Sensor restrictions are generally compensated with biomechanically inspired assumptions and prior information. Awareness of the possible adaptations in the IMU-based kinematic estimates by incorporating such prior information and assumptions is necessary, before drawing clinical decisions. Future research should focus on alternative validation methods, subject-specific IMU-based biomechanical joint models and disturbed movement patterns in real-world settings.

Mobile Applications in Evaluations of Knee Joint Kinematics: A Pilot Study

Because medical professionals lack the means to monitor exercises performed by patients in their home environment directly, there is a strong case for introducing technological solutions into this domain. They include methods that use wireless inertial sensors, which emit signals recorded and processed by special applications that work with mobile devices. This paper's aim is (a) to evaluate whether such sensors are suitable for qualitative and quantitative motion analysis, and (b) to determine the repeatability of results over a few recordings. Knee joint activity was analysed using a system of inertial sensors connected through a Wi-Fi network to mobile devices. The tested individuals did eight different activities, all of which engaged the knee joint. Each excercise was repeated three times. Study results did not reveal any statistically significant differences between the three measurements for most of the studied parameters. Furthermore, in almost every case, there were no statistically significant differences between the results of the right and left lower limb (p > 0.05). This study shows that easy use and repeatability of results combined with the feature of quantitative and qualitative analysis make the examined method useful for functional evaluations of the knee joint.

SoC Design Based on a FPGA for a Configurable Neural Network Trained by Means of an EKF

This work presents a configurable architecture for an artificial neural network implemented with a Field Programmable Gate Array (FPGA) in a System on Chip (SoC) environment. This architecture can reproduce the transfer function of different Multilayer Feedforward Neural Network (MFNN) configurations. The functionality of this configurable architecture relies on a single perceptron, multiplexers, and memory blocks that allow routing, storing, and processing information. The extended Kalman filter is the training algorithm that obtains the optimal weight values for the MFNN. The presented architecture was developed using Verilog Hardware Description Language, which permits designing hardware with a fair number of logical resources, and facilitates the portability to different FPGAs models without compatibility problems. A SoC that mainly incorporates a microprocessor and a FPGA is proposed, where the microprocessor is used for configuring the the MFNN and to enable and disable some functional blocks in the FPGA. The hardware was tested with measurements from a GaN class F power amplifier, using a 2.1 GHz Long Term Evolution signal with 5 MHz of bandwidth. In particular, a special case of an MFNN with two layers, i.e., a real-valued nonlinear autoregressive with an exogenous input neural network, was considered. The results reveal that a normalized mean square error value of −32.82 dB in steady-state was achievable, with a 71.36% generalization using unknown samples.

Motion Symmetry Evaluation Using Accelerometers and Energy Distribution

Analysis of motion symmetry constitutes an important area with many applications in engineering, robotics, neurology and biomedicine. This paper presents the use of microelectromechanical sensors (MEMS), including accelerometers and gyrometers, to acquire data via mobile devices so as to monitor physical activities and their irregularities. Special attention is devoted to the analysis of the symmetry of the motion of the body when the same exercises are performed by the right and the left limb. The analyzed data include the motion of the legs on a home exercise bike under different levels of load. The method is based on signal analysis using the discrete wavelet transform and the evaluation of signal segment features such as the relative energy at selected decomposition levels. The subsequent classification of the evaluated features is performed by k-nearest neighbours, a Bayesian approach, a support vector machine, and neural networks. The highest average classification accuracy attained is 91.0% and the lowest mean cross-validation error is 0.091, resulting from the use of a neural network. This paper presents the advantages of the use of simple sensors, their combination and intelligent data processing for the numerical evaluation of motion features in the rehabilitation and monitoring of physical activities.

Online Learning of Gait Models From Older Adult Data

This paper proposes a novel approach for online, individualized gait analysis, based on an adaptive periodic model of any gait signal. The proposed method learns a model of the gait cycle during online measurement, using a continuous representation that can adapt to inter- and intra-personal variability by creating an individualized model. Once the algorithm has converged to the input signal, key gait events can be identified based on the estimated gait phase and amplitude. The approach is implemented and tested on retirement home resident 6 min walk (6MW) data using wearable accelerometers at the ankle. The proposed approach converges within approximately four gait cycles and achieves 3% error in detecting initial swing events.11 An early version of this work was presented in [1]. A more extensive description of related work and an extended method, including optimization of learning rates, were added to this paper. Further, this paper applies and evaluates the method to a new and much larger gait dataset taken from older adults who each have a variety of medical conditions. Therefore, the experimental protocol was also updated and the results are entirely novel.

Smart Annotation of Cyclic Data Using Hierarchical Hidden Markov Models

Cyclic signals are an intrinsic part of daily life, such as human motion and heart activity. The detailed analysis of them is important for clinical applications such as pathological gait analysis and for sports applications such as performance analysis. Labeled training data for algorithms that analyze these cyclic data come at a high annotation cost due to only limited annotations available under laboratory conditions or requiring manual segmentation of the data under less restricted conditions. This paper presents a smart annotation method that reduces this cost of labeling for sensor-based data, which is applicable to data collected outside of strict laboratory conditions. The method uses semi-supervised learning of sections of cyclic data with a known cycle number. A hierarchical hidden Markov model (hHMM) is used, achieving a mean absolute error of 0.041 ± 0.020 s relative to a manually-annotated reference. The resulting model was also used to simultaneously segment and classify continuous, ‘in the wild’ data, demonstrating the applicability of using hHMM, trained on limited data sections, to label a complete dataset. This technique achieved comparable results to its fully-supervised equivalent. Our semi-supervised method has the significant advantage of reduced annotation cost. Furthermore, it reduces the opportunity for human error in the labeling process normally required for training of segmentation algorithms. It also lowers the annotation cost of training a model capable of continuous monitoring of cycle characteristics such as those employed to analyze the progress of movement disorders or analysis of running technique.