Machine learning approach to predict center of pressure trajectories in a complete gait cycle: a feedforward neural network vs. LSTM network

Prediction of medial knee contact force using multisource fusion recurrent neural network and transfer learning

Estimation of knee contact force (KCF) during gait provides essential information to evaluate knee joint function. Machine learning has been employed to estimate KCF because of the advantages of low computational cost and real-time. However, the existing machine learning models do not adequately consider gait-related data's temporal-dependent, multidimensional, and highly heterogeneous nature. This study is aimed at developing a multisource fusion recurrent neural network to predict the medial condyle KCF. First, a multisource fusion long short-term memory (MF-LSTM) model was established. Then, we developed a transfer learning strategy based on the MF-LSTM model for subject-specific medial KCF prediction. Four subjects with instrumented tibial prostheses were obtained from the literature. The results showed that the MF-LSTM model could predict medial KCF to a certain high level of accuracy (the mean of ρ = 0.970). The transfer learning model improved the prediction accuracy (the mean of ρ = 0.987). This study shows that the MF-LSTM model is a powerful and accurate computational tool for medial KCF prediction. Introducing transfer learning techniques could further improve the prediction performance for the target subject. This coupling strategy can help clinicians accurately estimate and track joint contact forces in real time.

Predicting vertical and shear ground reaction forces during walking and jogging using wearable plantar pressure insoles

BACKGROUND

The development of plantar pressure insoles has made them a potential replacement for force plates. These wearable devices can measure multiple steps and might be used outside of the lab environment for rehabilitation and evaluation of sport performance. However, they can only measure the vertical force which does not completely represent the vertical ground reaction force. In addition, they are not able to measure shear forces which play an import role in the dynamic performance of individuals. Indirect approaches might be implemented to improve the accuracy of the force estimated by plantar pressure systems.

RESEARCH QUESTION

The aim of this study was to predict the vertical and shear components of ground reaction force from plantar pressure data using recurrent neural networks.

METHODS

Ground reaction force and plantar pressure data were collected from 16 healthy individuals during 10 trials of walking and five trials of jogging using Bertec force plates at 1000 Hz and FScan plantar pressure insoles at 100 Hz. A long short-term memory neural network was built to consider the time dependency of pressure and force data in predictions. The data were split into three subsets of train, to train the model, evaluate, to optimize the model hyperparameters, and test sets, to assess the accuracy of the model predictions.

RESULTS

The results of this study showed that our long short-term memory model could accurately predict the shear and vertical force components during walking and jogging. The predictions were more accurate during walking compared to jogging. In addition, the predictions of mediolateral force had higher error and lower correlation compared to vertical and anteroposterior components.

SIGNIFICANCE

The long short-term memory model developed in this study may be an acceptable option for accurate estimation of ground reaction force during outdoor activities which can have significant impacts in rehabilitation, sport performance, and gaming.

Vibration motor stimulation device in smart leggings that promotes motor performance in older people

Globally, accelerated aging is taking place alongside increased life expectancy of the population. This poses a challenge to maintaining autonomy and independence as people age but preventing falls and disabilities. Currently, there are few specific technologies on the market that are focused on the rehabilitation and promotion of autonomy in older adults. This study presents the development of a prototype (Myoviber®) of a low-cost, wearable everyday garment, designed to stimulate the lower limbs by the application of focal muscle vibration and incorporating technical textile qualities. The presented approach is proactive and preventive, maintaining functionality for the elderly while integrating electronic technology into an everyday garment. For this, a comprehensive study was carried out that included the design of the leggings through anthropometric analyses, the development of vibration devices at a stable frequency located in the knee extensor muscle and a smart belt with wireless connection, and the optimization of the battery autonomy. The development of the prototype was carried out through the construction of a vibratory device, which was validated with biomechanical evaluations. The results show an increase in the functional capacity of the lower limbs, in relation to motor tasks such as postural balance and gait in older people.

Deep Neural Network for the Detections of Fall and Physical Activities Using Foot Pressures and Inertial Sensing

Fall detection and physical activity (PA) classification are important health maintenance issues for the elderly and people with mobility dysfunctions. The literature review showed that most studies concerning fall detection and PA classification addressed these issues individually, and many were based on inertial sensing from the trunk and upper extremities. While shoes are common footwear in daily off-bed activities, most of the aforementioned studies did not focus much on shoe-based measurements. In this paper, we propose a novel footwear approach to detect falls and classify various types of PAs based on a convolutional neural network and recurrent neural network hybrid. The footwear-based detections using deep-learning technology were demonstrated to be efficient based on the data collected from 32 participants, each performing simulated falls and various types of PAs: fall detection with inertial measures had a higher F1-score than detection using foot pressures; the detections of dynamic PAs (jump, jog, walks) had higher F1-scores while using inertial measures, whereas the detections of static PAs (sit, stand) had higher F1-scores while using foot pressures; the combination of foot pressures and inertial measures was most efficient in detecting fall, static, and dynamic PAs.

Predicting vertical ground reaction forces from 3D accelerometry using reservoir computers leads to accurate gait event detection

Accelerometers are low-cost measurement devices that can readily be used outside the lab. However, determining isolated gait events from accelerometer signals, especially foot-off events during running, is an open problem. We outline a two-step approach where machine learning serves to predict vertical ground reaction forces from accelerometer signals, followed by force-based event detection. We collected shank accelerometer signals and ground reaction forces from 21 adults during comfortable walking and running on an instrumented treadmill. We trained one common reservoir computer using segmented data using both walking and running data. Despite being trained on just a small number of strides, this reservoir computer predicted vertical ground reaction forces in continuous gait with high quality. The subsequent foot contact and foot off event detection proved highly accurate when compared to the gold standard based on co-registered ground reaction forces. Our proof-of-concept illustrates the capacity of combining accelerometry with machine learning for detecting isolated gait events irrespective of mode of locomotion.

Deep Learning-Based Near-Fall Detection Algorithm for Fall Risk Monitoring System Using a Single Inertial Measurement Unit

Proactively detecting falls and preventing injuries are among the primary keys to a healthy life for the elderly. Near-fall remote monitoring in daily life could provide key information to prevent future falls and obtain quantitative rehabilitation status for patients with weak balance ability. In this study, we developed a deep learning-based novel classification algorithm to precisely categorize three classes (falls, near-falls, and activities of daily living (ADLs)) using a single inertial measurement unit (IMU) device attached to the waist. A total of 34 young participants were included in this study. An IMU containing accelerometer and gyroscope sensors was fabricated to acquire acceleration and angular velocity signals. A comprehensive experiment including thirty-six types of activities (10 types of falls, 10 types of near-falls, and 16 types of ADLs) was designed based on previous studies. A modified directed acyclic graph-convolution neural network (DAG-CNN) architecture with hyperparameter optimization was proposed to predict fall, near-fall, and ADLs. Prediction results of the modified DAG-CNN structure were found to be approximately 7% more accurate than the traditional CNN structure. For the case of near-falls, the modified DAG-CNN demonstrated excellent prediction performance with accuracy of over 98% by combining gyroscope and accelerometer features. Additionally, by combining acceleration and angular velocity the trained model showed better performance than each model of acceleration and angular velocity. It is believed that information to preemptively handle the risk of falls and quantitatively evaluate the rehabilitation status of the elderly with weak balance will be provided by monitoring near-falls.

A novel optimization-based convolution neural network to estimate the contribution of sensory inputs to postural stability during quiet standing

Adequate postural control is maintained by integrating signals from the visual, somatosensory, and vestibular systems. The purpose of this study is to propose a novel convolutional neural network (CNN)-based protocol that can evaluate the contributions of each sensory input for postural stability (calculated a sensory analysis index) using center of pressure (COP) signals in a quiet standing posture. Raw COP signals in the anterior/posterior and medial/lateral directions were extracted from 330 patients in a quiet standing with their eyes open for 20 seconds. The COP signals augmented using jittering and pooling techniques were transformed into the frequency domain. The sensory analysis indices were used as the output information from the deep learning models. A ResNet-50 CNN was combined with the k-nearest neighbor, random forest, and support vector machine classifiers for the training model. Additionally, a novel optimization process was proposed to include an encoding design variable that can group outputs into sub-classes along with hyperparameters. The results of optimization considering only hyperparameters showed low performance, with an accuracy of 55% or less and F-1 scores of 54% or less in all models. However, when optimization was performed using the encoding design variable, the performance was markedly increased in the CNN-classifier combined models (r = 0.975). These results suggest it is possible to evaluate the contribution of sensory inputs for postural stability using COP signals during a quiet standing. This study will facilitate the expanded dissemination of a system that can quantitatively evaluate the balance ability and rehabilitation progress of patients with dizziness.

Deep learning approach to estimate foot pressure distribution in walking with application for a cost-effective insole system

Background

Foot pressure distribution can be used as a quantitative parameter for evaluating anatomical deformity of the foot and for diagnosing and treating pathological gait, falling, and pressure sores in diabetes. The objective of this study was to propose a deep learning model that could predict pressure distribution of the whole foot based on information obtained from a small number of pressure sensors in an insole.

Methods

Twenty young and twenty older adults walked a straight pathway at a preferred speed with a Pedar-X system in anti-skid socks. A long short-term memory (LSTM) model was used to predict foot pressure distribution. Pressure values of nine major sensors and the remaining 90 sensors in a Pedar-X system were used as input and output for the model, respectively. The performance of the proposed LSTM structure was compared with that of a traditionally used adaptive neuro-fuzzy interference system (ANFIS). A low-cost insole system consisting of a small number of pressure sensors was fabricated. A gait experiment was additionally performed with five young and five older adults, excluding subjects who were used to construct models. The Pedar-X system placed parallelly on top of the insole prototype developed in this study was in anti-skid socks. Sensor values from a low-cost insole prototype were used as input of the LSTM model. The accuracy of the model was evaluated by applying a leave-one-out cross-validation.

Results

Correlation coefficient and relative root mean square error (RMSE) of the LSTM model were 0.98 (0.92 ~ 0.99) and 7.9 ± 2.3%, respectively, higher than those of the ANFIS model. Additionally, the usefulness of the proposed LSTM model for fabricating a low-cost insole prototype with a small number of sensors was confirmed, showing a correlation coefficient of 0.63 to 0.97 and a relative RMSE of 12.7 ± 7.4%.

Conclusions

This model can be used as an algorithm to develop a low-cost portable smart insole system to monitor age-related physiological and anatomical alterations in foot. This model has the potential to evaluate clinical rehabilitation status of patients with pathological gait, falling, and various foot pathologies when more data of patients with various diseases are accumulated for training.

A Study of Athlete Pose Estimation Techniques in Sports Game Videos Combining Multiresidual Module Convolutional Neural Networks

In this paper, we propose a multiresidual module convolutional neural network-based method for athlete pose estimation in sports game videos. The network firstly designs an improved residual module based on the traditional residual module. Firstly, a large perceptual field residual module is designed to learn the correlation between the athlete components in the sports game video within a large perceptual field. A multiscale residual module is designed in the paper to better solve the inaccuracy of the pose estimation due to the problem of scale change of the athlete components in the sports game video. Secondly, these three residual modules are used as the building blocks of the convolutional neural network. When the resolution is high, the large perceptual field residual module and the multiscale residual module are used to capture information in a larger range as well as at each scale, and when the resolution is low, only the improved residual module is used. Finally, four multiresidual module convolutional neural networks are used to form the final multiresidual module stacked convolutional neural network. The neural network model proposed in this paper achieves high accuracy of 89.5% and 88.2% on the upper arm and lower arm, respectively, so the method in this paper reduces the influence of occlusion on the athlete's posture estimation to a certain extent. Through the experiments, it can be seen that the proposed multiresidual module stacked convolutional neural network-based method for athlete pose estimation in sports game videos further improves the accuracy of athlete pose estimation in sports game videos.

Machine Learning Model to Estimate Net Joint Moments during Lifting Task Using Wearable Sensors: A Preliminary Study for Design of Exoskeleton Control System

Accurately measuring the lower extremities and L5/S1 moments is important since L5/S1 moments are the principal parameters that measure the risk of musculoskeletal diseases during lifting. In this study, protocol that predicts lower extremities and L5/S1 moments with an insole sensor was proposed to replace the prior methods that have spatial constraints. The protocol is hierarchically composed of a classification model and a regression model to predict joint moments. Additionally, a single LSTM model was developed to compare with proposed protocol. To optimize hyperparameters of the machine learning model and input feature, Bayesian optimization method was adopted. As a result, the proposed protocol showed a relative root mean square error (rRMSE) of 8.06~13.88% while the single LSTM showed 9.30~18.66% rRMSE. This protocol in this research is expected to be a starting point for developing a system for estimating the lower extremity and L5/S1 moment during lifting that can replace the complex prior method and adopted to workplace environments. This novel study has the potential to precisely design a feedback iterative control system of an exoskeleton for the appropriate generation of an actuator torque.

Multiple Inertial Measurement Unit Combination and Location for Center of Pressure Prediction in Gait

Center of pressure (COP) during a gait cycle indicates crucial information with regard to fall risk such as balance capacity. The drawbacks of conventional research instruments include inconvenient use during activities of daily living and expensive costs. The present study illustrates the promising fall-relevant information predicted by acceleration and angular velocity data from different placement sensors with machine learning techniques. This approach is inspired by the emerging machine learning technique, specifically the long short-term memory (LSTM), which is often used in time series data and aims to decrease the burden of the user while using the novel wearable technology. The Jaccard similarity coefficient, which implies the consistency of profile alignment between prediction and real situation, achieved 94% accuracy in the walking direction. Furthermore, the number of sensors used and the placement influenced the feasibility of an application. The outcome revealed that the accuracy could exceed 90% with only one sensor placed on the foot in the walking direction, and the toe would be the best location for sensor placement. To examine the performance of machine learning, the current study employed two parameters from different perspectives. One is a commonly used parameter, which represented the error, and the other investigated the similarity between the prediction and ground truth. From a similarity perspective, the parameter can be used as a metric to assess the consistency of profile alignment.

Acceleration Magnitude at Impact Following Loss of Balance Can Be Estimated Using Deep Learning Model

Pre-impact fall detection can detect a fall before a body segment hits the ground. When it is integrated with a protective system, it can directly prevent an injury due to hitting the ground. An impact acceleration peak magnitude is one of key measurement factors that can affect the severity of an injury. It can be used as a design parameter for wearable protective devices to prevent injuries. In our study, a novel method is proposed to predict an impact acceleration magnitude after loss of balance using a single inertial measurement unit (IMU) sensor and a sequential-based deep learning model. Twenty-four healthy participants participated in this study for fall experiments. Each participant worn a single IMU sensor on the waist to collect tri-axial accelerometer and angular velocity data. A deep learning method, bi-directional long short-term memory (LSTM) regression, is applied to predict a fall's impact acceleration magnitude prior to fall impact (a fall in five directions). To improve prediction performance, a data augmentation technique with increment of dataset is applied. Our proposed model showed a mean absolute percentage error (MAPE) of 6.69 ± 0.33% with r value of 0.93 when all three different types of data augmentation techniques are applied. Additionally, there was a significant reduction of MAPE by 45.2% when the number of training datasets was increased by 4-fold. These results show that impact acceleration magnitude can be used as an activation parameter for fall prevention such as in a wearable airbag system by optimizing deployment process to minimize fall injury in real time.

Artificial Neural Networks in Motion Analysis-Applications of Unsupervised and Heuristic Feature Selection Techniques

The use of machine learning to estimate joint angles from inertial sensors is a promising approach to in-field motion analysis. In this context, the simplification of the measurements by using a small number of sensors is of great interest. Neural networks have the opportunity to estimate joint angles from a sparse dataset, which enables the reduction of sensors necessary for the determination of all three-dimensional lower limb joint angles. Additionally, the dimensions of the problem can be simplified using principal component analysis. Training a long short-term memory neural network on the prediction of 3D lower limb joint angles based on inertial data showed that three sensors placed on the pelvis and both shanks are sufficient. The application of principal component analysis to the data of five sensors did not reveal improved results. The use of longer motion sequences compared to time-normalised gait cycles seems to be advantageous for the prediction accuracy, which bridges the gap to real-time applications of long short-term memory neural networks in the future.