1
|
Loi I, Zacharaki EI, Moustakas K. Machine Learning Approaches for 3D Motion Synthesis and Musculoskeletal Dynamics Estimation: A Survey. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2024; 30:5810-5829. [PMID: 37624722 DOI: 10.1109/tvcg.2023.3308753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/27/2023]
Abstract
The inference of 3D motion and dynamics of the human musculoskeletal system has traditionally been solved using physics-based methods that exploit physical parameters to provide realistic simulations. Yet, such methods suffer from computational complexity and reduced stability, hindering their use in computer graphics applications that require real-time performance. With the recent explosion of data capture (mocap, video) machine learning (ML) has started to become popular as it is able to create surrogate models harnessing the huge amount of data stemming from various sources, minimizing computational time (instead of resource usage), and most importantly, approximate real-time solutions. The main purpose of this paper is to provide a review and classification of the most recent works regarding motion prediction, motion synthesis as well as musculoskeletal dynamics estimation problems using ML techniques, in order to offer sufficient insight into the state-of-the-art and draw new research directions. While the study of motion may appear distinct to musculoskeletal dynamics, these application domains provide jointly the link for more natural computer graphics character animation, since ML-based musculoskeletal dynamics estimation enables modeling of more long-term, temporally evolving, ergonomic effects, while offering automated and fast solutions. Overall, our review offers an in-depth presentation and classification of ML applications in human motion analysis, unlike previous survey articles focusing on specific aspects of motion prediction.
Collapse
|
2
|
Li Z, Gao L, Zhang G, Lu W, Wang D, Zhang J, Cao H. MMG-Based Knee Dynamic Extension Force Estimation Using Cross-Talk and IGWO-LSTM. Bioengineering (Basel) 2024; 11:470. [PMID: 38790337 PMCID: PMC11117547 DOI: 10.3390/bioengineering11050470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Revised: 04/29/2024] [Accepted: 05/06/2024] [Indexed: 05/26/2024] Open
Abstract
Mechanomyography (MMG) is an important muscle physiological activity signal that can reflect the amount of motor units recruited as well as the contraction frequency. As a result, MMG can be utilized to estimate the force produced by skeletal muscle. However, cross-talk and time-series correlation severely affect MMG signal recognition in the real world. These restrict the accuracy of dynamic muscle force estimation and their interaction ability in wearable devices. To address these issues, a hypothesis that the accuracy of knee dynamic extension force estimation can be improved by using MMG signals from a single muscle with less cross-talk is first proposed. The hypothesis is then confirmed using the estimation results from different muscle signal feature combinations. Finally, a novel model (improved grey wolf optimizer optimized long short-term memory networks, i.e., IGWO-LSTM) is proposed for further improving the performance of knee dynamic extension force estimation. The experimental results demonstrate that MMG signals from a single muscle with less cross-talk have a superior ability to estimate dynamic knee extension force. In addition, the proposed IGWO-LSTM provides the best performance metrics in comparison to other state-of-the-art models. Our research is expected to not only improve the understanding of the mechanisms of quadriceps contraction but also enhance the flexibility and interaction capabilities of future rehabilitation and assistive devices.
Collapse
Affiliation(s)
- Zebin Li
- Anhui Undergrowth Crop Intelligent Equipment Engineering Research Center, West Anhui University, Lu’an 237012, China;
- Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China; (L.G.); (D.W.); (H.C.)
| | - Lifu Gao
- Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China; (L.G.); (D.W.); (H.C.)
- Department of Science Island, University of Science and Technology of China, Hefei 230031, China
| | - Gang Zhang
- Anhui Undergrowth Crop Intelligent Equipment Engineering Research Center, West Anhui University, Lu’an 237012, China;
| | - Wei Lu
- School of Management, Fujian University of Technology, Fuzhou 350118, China;
| | - Daqing Wang
- Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China; (L.G.); (D.W.); (H.C.)
| | - Jinzhong Zhang
- Anhui Undergrowth Crop Intelligent Equipment Engineering Research Center, West Anhui University, Lu’an 237012, China;
| | - Huibin Cao
- Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China; (L.G.); (D.W.); (H.C.)
| |
Collapse
|
3
|
Zou J, Zhang X, Zhang Y, Jin Z. Prediction of medial knee contact force using multisource fusion recurrent neural network and transfer learning. Med Biol Eng Comput 2024; 62:1333-1346. [PMID: 38182944 DOI: 10.1007/s11517-023-03011-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 12/27/2023] [Indexed: 01/07/2024]
Abstract
Estimation of knee contact force (KCF) during gait provides essential information to evaluate knee joint function. Machine learning has been employed to estimate KCF because of the advantages of low computational cost and real-time. However, the existing machine learning models do not adequately consider gait-related data's temporal-dependent, multidimensional, and highly heterogeneous nature. This study is aimed at developing a multisource fusion recurrent neural network to predict the medial condyle KCF. First, a multisource fusion long short-term memory (MF-LSTM) model was established. Then, we developed a transfer learning strategy based on the MF-LSTM model for subject-specific medial KCF prediction. Four subjects with instrumented tibial prostheses were obtained from the literature. The results showed that the MF-LSTM model could predict medial KCF to a certain high level of accuracy (the mean of ρ = 0.970). The transfer learning model improved the prediction accuracy (the mean of ρ = 0.987). This study shows that the MF-LSTM model is a powerful and accurate computational tool for medial KCF prediction. Introducing transfer learning techniques could further improve the prediction performance for the target subject. This coupling strategy can help clinicians accurately estimate and track joint contact forces in real time.
Collapse
Affiliation(s)
- Jianjun Zou
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
| | - Xiaogang Zhang
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China.
| | - Yali Zhang
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
| | - Zhongmin Jin
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, 710049, China
- School of Mechanical Engineering, University of Leeds, Leeds, LS2 9JT, UK
| |
Collapse
|
4
|
Liu K, Liu Y, Ji S, Gao C, Fu J. Estimation of Muscle Forces of Lower Limbs Based on CNN-LSTM Neural Network and Wearable Sensor System. SENSORS (BASEL, SWITZERLAND) 2024; 24:1032. [PMID: 38339749 PMCID: PMC10857390 DOI: 10.3390/s24031032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 01/26/2024] [Accepted: 01/26/2024] [Indexed: 02/12/2024]
Abstract
Estimation of vivo muscle forces during human motion is important for understanding human motion control mechanisms and joint mechanics. This paper combined the advantages of the convolutional neural network (CNN) and long-short-term memory (LSTM) and proposed a novel muscle force estimation method based on CNN-LSTM. A wearable sensor system was also developed to collect the angles and angular velocities of the hip, knee, and ankle joints in the sagittal plane during walking, and the collected kinematic data were used as the input for the neural network model. In this paper, the muscle forces calculated using OpenSim based on the Static Optimization (SO) method were used as the standard value to train the neural network model. Four lower limb muscles of the left leg, including gluteus maximus (GM), rectus femoris (RF), gastrocnemius (GAST), and soleus (SOL), were selected as the studying objects in this paper. The experiment results showed that compared to the standard CNN and the standard LSTM, the CNN-LSTM performed better in muscle forces estimation under slow (1.2 m/s), medium (1.5 m/s), and fast walking speeds (1.8 m/s). The average correlation coefficients between true and estimated values of four muscle forces under slow, medium, and fast walking speeds were 0.9801, 0.9829, and 0.9809, respectively. The average correlation coefficients had smaller fluctuations under different walking speeds, which indicated that the model had good robustness. The external testing experiment showed that the CNN-LSTM also had good generalization. The model performed well when the estimated object was not included in the training sample. This article proposed a convenient method for estimating muscle forces, which could provide theoretical assistance for the quantitative analysis of human motion and muscle injury. The method has established the relationship between joint kinematic signals and muscle forces during walking based on a neural network model; compared to the SO method to calculate muscle forces in OpenSim, it is more convenient and efficient in clinical analysis or engineering applications.
Collapse
Affiliation(s)
- Kun Liu
- School of Mechanical and Aerospace Engineering, Jilin University, Changchun 130000, China; (Y.L.); (S.J.); (C.G.); (J.F.)
| | | | | | | | | |
Collapse
|
5
|
Lin N, Wu S, Wu Z, Ji S. Efficient Generation of Pretraining Samples for Developing a Deep Learning Brain Injury Model via Transfer Learning. Ann Biomed Eng 2023:10.1007/s10439-023-03354-3. [PMID: 37642795 DOI: 10.1007/s10439-023-03354-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Accepted: 08/18/2023] [Indexed: 08/31/2023]
Abstract
The large amount of training samples required to develop a deep learning brain injury model demands enormous computational resources. Here, we study how a transformer neural network (TNN) of high accuracy can be used to efficiently generate pretraining samples for a convolutional neural network (CNN) brain injury model to reduce computational cost. The samples use synthetic impacts emulating real-world events or augmented impacts generated from limited measured impacts. First, we verify that the TNN remains highly accurate for the two impact types (N = 100 each; [Formula: see text] of 0.948-0.967 with root mean squared error, RMSE, ~ 0.01, for voxelized peak strains). The TNN-estimated samples (1000-5000 for each data type) are then used to pretrain a CNN, which is further finetuned using directly simulated training samples (250-5000). An independent measured impact dataset considered of complete capture of impact event is used to assess estimation accuracy (N = 191). We find that pretraining can significantly improve CNN accuracy via transfer learning compared to a baseline CNN without pretraining. It is most effective when the finetuning dataset is relatively small (e.g., 2000-4000 pretraining synthetic or augmented samples improves success rate from 0.72 to 0.81 with 500 finetuning samples). When finetuning samples reach 3000 or more, no obvious improvement occurs from pretraining. These results support using the TNN to rapidly generate pretraining samples to facilitate a more efficient training strategy for future deep learning brain models, by limiting the number of costly direct simulations from an alternative baseline model. This study could contribute to a wider adoption of deep learning brain injury models for large-scale predictive modeling and ultimately, enhancing safety protocols and protective equipment.
Collapse
Affiliation(s)
- Nan Lin
- Department of Biomedical Engineering, Worcester Polytechnic Institute, 60 Prescott Street, Worcester, MA, 01605, USA
| | - Shaoju Wu
- Department of Biomedical Engineering, Worcester Polytechnic Institute, 60 Prescott Street, Worcester, MA, 01605, USA
| | - Zheyang Wu
- Mathematical Sciences, Worcester Polytechnic Institute, Worcester, MA, 01609, USA
| | - Songbai Ji
- Department of Biomedical Engineering, Worcester Polytechnic Institute, 60 Prescott Street, Worcester, MA, 01605, USA.
- Department of Mechanical Engineering, Worcester Polytechnic Institute, 100 Institute Road, Worcester, MA, 01609, USA.
| |
Collapse
|
6
|
Serbest K, Ozkan MT, Cilli M. Estimation of joint torques using an artificial neural network model based on kinematic and anthropometric data. Neural Comput Appl 2023. [DOI: 10.1007/s00521-023-08379-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/06/2023]
|
7
|
Mubarrat ST, Chowdhury S. Convolutional LSTM: a deep learning approach to predict shoulder joint reaction forces. Comput Methods Biomech Biomed Engin 2023; 26:65-77. [PMID: 35234548 DOI: 10.1080/10255842.2022.2045974] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
We developed a Convolutional LSTM (ConvLSTM) network to predict shoulder joint reaction forces using 3D shoulder kinematics data containing 30 different shoulder activities from eight human subjects. We considered simulation outcomes from the AnyBody musculoskeletal model as the baseline force dataset to validate ConvLSTM model predictions. Results showed a good correlation (>80% accuracy, r ≥ 0.82) between ConvLSTM predicted and AnyBody estimated force values, the generalization of the developed model for novel task type (p-value = 0.07 ∼ 0.33), and a better prediction accuracy for the ConvLSTM model than conventional CNN and LSTM models.
Collapse
Affiliation(s)
- S T Mubarrat
- Department of Industrial, Manufacturing, and Systems Engineering, Texas Tech University, Lubbock, TX, USA
| | - S Chowdhury
- Department of Industrial, Manufacturing, and Systems Engineering, Texas Tech University, Lubbock, TX, USA
| |
Collapse
|
8
|
Zou J, Zhang X, Zhang Y, Li J, Jin Z. Prediction on the medial knee contact force in patients with knee valgus using transfer learning approaches: Application to rehabilitation gaits. Comput Biol Med 2022; 150:106099. [PMID: 36150250 DOI: 10.1016/j.compbiomed.2022.106099] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 08/21/2022] [Accepted: 09/10/2022] [Indexed: 11/03/2022]
Abstract
The Knee contact force (KCF) is a key factor in evaluating knee joint function of patients with knee osteoarthritis. In vivo measurement of KCF based on the instrumented implants is limited due to the ethical issues and technical complexities. Machine learning can be used to predict tibiofemoral compartment contact forces. However, anthropometric differences between individuals make the accurate predictions challenging. The purpose of this study was to develop transfer learning models to predict the medial KCF of patients with knee valgus in rehabilitation gaits. Four subjects with instrumented tibial prostheses were considered, including one with knee valgus and three with normal knee joint alignment. Two transfer learning models were proposed: a fine-tuning model and an adaptive model. In particular, a synchronization method for extracting experimental data in a complete gait cycle was developed, since different types of experimental data have different sampling frequencies. The transfer learning models were pre-trained by the experiment data of patients with normal knee joint alignment, and re-trained by the data of the patient with knee valgus. Predictions of the transfer learning models and traditional machine learning model were validated against the in vivo measurements. The proposed transfer learning models were tested within two levels: the single subject (Level 1) and multiple subjects (Level 2). The results show that the two transfer learning models could more accurately predict the medial KCF of patients with knee valgus than the traditional machine learning model. The performance of the fine-tuning model is better than that of the adaptive model. Compared with the traditional machine learning and inverse dynamics analysis, transfer learning represents a much easier and more accurate method. It can be introduced to help clinicians validate and adjust the rehabilitation gait for specific patients.
Collapse
Affiliation(s)
- Jianjun Zou
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
| | - Xiaogang Zhang
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China.
| | - Yali Zhang
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
| | - Junyan Li
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China
| | - Zhongmin Jin
- School of Mechanical Engineering, Southwest Jiaotong University, Chengdu, 610031, China; School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, 710049, China; School of Mechanical Engineering, University of Leeds, LS2 9JT, UK
| |
Collapse
|
9
|
A Deep Learning Approach for Predicting Subject-Specific Human Skull Shape from Head Toward a Decision Support System for Home-Based Facial Rehabilitation. Ing Rech Biomed 2022. [DOI: 10.1016/j.irbm.2022.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
10
|
Deep reinforcement learning coupled with musculoskeletal modelling for a better understanding of elderly falls. Med Biol Eng Comput 2022; 60:1745-1761. [DOI: 10.1007/s11517-022-02567-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2021] [Accepted: 04/06/2022] [Indexed: 10/18/2022]
|
11
|
Recurrent neural network to predict hyperelastic constitutive behaviors of the skeletal muscle. Med Biol Eng Comput 2022; 60:1177-1185. [PMID: 35244859 DOI: 10.1007/s11517-022-02541-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Accepted: 02/23/2022] [Indexed: 10/18/2022]
Abstract
Hyperelastic constitutive laws have been commonly used to model the passive behavior of the human skeletal muscle. Despite many efforts, the use of accurate finite element formulations of hyperelastic constitutive laws is still time-consuming for a real-time medical simulation system. The objective of the present study was to develop a deep learning model to predict the hyperelastic constitutive behaviors of the skeletal muscle toward a fast estimation of the muscle tissue stress.A finite element (FE) model of the right psoas muscle was developed. Neo-Hookean and Mooney-Rivlin laws were used. A tensile test was performed with an applied body force. A learning database was built from this model using an automatic probabilistic generation process. A long-short term memory (LSTM) neural network was implemented to predict the stress evolution of the skeletal muscle tissue. A hyperparameter tuning process was conducted. Root mean square error (RMSE) and associated relative error was quantified to evaluate the precision of the predictive capacity of the developed deep learning model. Pearson correlation coefficients (R) was also computed.The nodal displacements and the maximal stresses range from 70 to 227 mm and from 2.79 to 5.61 MPa for Neo-Hookean and Monney-Rivlin laws, respectively. Regarding the LSTM predictions, the RMSE ranges from 224.3 ± 3.9 Pa (8%) to 227.5 [Formula: see text] 5.7 Pa (4%) for Neo-Hookean and Monney-Rivlin laws, respectively. Pearson correlation coefficients (R) of 0.78 [Formula: see text] 0.02 and 0.77 [Formula: see text] 0.02 were obtained for Neo-Hookean and Monney-Rivlin laws, respectively.The present study showed that, for the first time, the use of a deep learning model can reproduce the time-series behaviors of the complex FE formulations for skeletal muscle modeling. In particular, the use of a LSTM neural network leads to a fast and accurate surrogate model for the in silico prediction of the hyperelastic constitutive behaviors of the skeletal muscle. As perspectives, the developed deep learning model will be integrated into a real-time medical simulation of the skeletal muscle for prosthetic socket design and childbirth simulator.
Collapse
|
12
|
Zhang L, Soselia D, Wang R, Gutierrez-Farewik EM. Lower-limb Joint Torque Prediction using LSTM Neural Networks and Transfer Learning. IEEE Trans Neural Syst Rehabil Eng 2022; 30:600-609. [PMID: 35239487 DOI: 10.1109/tnsre.2022.3156786] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Estimation of joint torque during movement provides important information in several settings, such as effect of athletes' training or of a medical intervention, or analysis of the remaining muscle strength in a wearer of an assistive device. The ability to estimate joint torque during daily activities using wearable sensors is increasingly relevant in such settings. In this study, lower limb joint torques during ten daily activities were predicted by long short-term memory (LSTM) neural networks and transfer learning. LSTM models were trained with muscle electromyography signals and lower limb joint angles. Hip flexion/extension, hip abduction/adduction, knee flexion/extension and ankle dorsiflexion/plantarflexion torques were predicted. The LSTM models' performance in predicting torque was investigated in both intra-subject and inter-subject scenarios. Each scenario was further divided into intra-task and inter-task tests. We observed that LSTM models could predict lower limb joint torques during various activities accurately with relatively low error (root mean square error ≤ 0.14 Nm/kg, normalized root mean square error ≤8.7%) either through a uniform model or through ten separate models in intra-subject tests. Furthermore, a transfer learning technique was adopted in the inter-task and inter-subject tests to further improve the generalizability of LSTM models by pre-training a model on multiple subjects and/or tasks and transferring the learned knowledge to a target task/subject. Particularly in the inter-subject tests, we could predict joint torques accurately in several movements after training from only a few movements from new subjects.
Collapse
|
13
|
Aihara S, Shibata R, Mizukami R, Sakai T, Shionoya A. Deep Learning-Based Myoelectric Potential Estimation Method for Wheelchair Operation. SENSORS 2022; 22:s22041615. [PMID: 35214514 PMCID: PMC8875647 DOI: 10.3390/s22041615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Revised: 02/12/2022] [Accepted: 02/15/2022] [Indexed: 02/01/2023]
Abstract
Wheelchair sports are recognized as an international sport, and research and support are being promoted to increase the competitiveness of wheelchair sports. For example, an electromyogram can observe muscle activity. However, it is generally used under controlled conditions due to the complexity of preparing the measurement equipment and the movement restrictions imposed by cables and measurement equipment. It is difficult to perform measurements in actual competition environments. Therefore, in this study, we developed a method to estimate myoelectric potential that can be used in competitive environments and does not limit physical movement. We developed a deep learning model that outputs surface myoelectric potentials by inputting camera images of wheelchair movements and the measured values of inertial sensors installed on wheelchairs. For seven subjects, we estimated the myoelectric potential during chair work, which is important in wheelchair sports. As a result of creating an in-subject model and comparing the estimated myoelectric potential with the myoelectric potential measured by an electromyogram, we confirmed a correlation (correlation coefficient 0.5 or greater at a significance level of 0.1%). Since this method can estimate the myoelectric potential without limiting the movement of the body, it is considered that it can be applied to the performance evaluation of wheelchair sports.
Collapse
Affiliation(s)
- Shimpei Aihara
- Department of Sport Science, Japan Institute of Sports Sciences, 3-15-1 Nishigaoka, Kita-ku, Tokyo 115-0056, Japan
- School of Creative Science and Engineering, Waseda University, Wasedamachi-27, Shinjuku-ku, Tokyo 169-8050, Japan
- Correspondence: (S.A.); (R.S.)
| | - Ryusei Shibata
- Graduate School of Information and Management Systems Engineering, Nagaoka University of Technology, 1603-1, Kamitomioka, Nagaoka, Niigata 940-2188, Japan; (R.M.); (T.S.); (A.S.)
- Correspondence: (S.A.); (R.S.)
| | - Ryosuke Mizukami
- Graduate School of Information and Management Systems Engineering, Nagaoka University of Technology, 1603-1, Kamitomioka, Nagaoka, Niigata 940-2188, Japan; (R.M.); (T.S.); (A.S.)
| | - Takara Sakai
- Graduate School of Information and Management Systems Engineering, Nagaoka University of Technology, 1603-1, Kamitomioka, Nagaoka, Niigata 940-2188, Japan; (R.M.); (T.S.); (A.S.)
| | - Akira Shionoya
- Graduate School of Information and Management Systems Engineering, Nagaoka University of Technology, 1603-1, Kamitomioka, Nagaoka, Niigata 940-2188, Japan; (R.M.); (T.S.); (A.S.)
| |
Collapse
|
14
|
Zhang H, Duong TTH, Rao AK, Mazzoni P, Agrawal SK, Guo Y, Zanotto D. Transductive Learning Models for Accurate Ambulatory Gait Analysis in Elderly Residents of Assisted Living Facilities. IEEE Trans Neural Syst Rehabil Eng 2022; 30:124-134. [PMID: 35025747 DOI: 10.1109/tnsre.2022.3143094] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Instrumented footwear represents a promising technology for spatiotemporal gait analysis in out-of-the-lab conditions. However, moderate accuracy impacts this technology's ability to capture subtle, but clinically meaningful, changes in gait patterns that may indicate adverse outcomes or underlying neurological conditions. This limitation hampers the use of instrumented footwear to aid functional assessments and clinical decision making. This paper introduces new transductive-learning inference models that substantially reduce measurement errors relative to conventional data processing techniques, without requiring subject-specific labelled data. The proposed models use subject-optimized input features and hyperparameters to adjust the spatiotemporal gait metrics (i.e., stride time, length, and velocity, swing time, and double support time) obtained with conventional techniques, resulting in computationally simpler models compared to end-to-end machine learning approaches. Model validity and reliability were evaluated against a gold-standard electronic walkway during a clinical gait performance test (6-minute walk test) administered to N=95 senior residents of assisted living facilities with diverse levels of gait and balance impairments. Average reductions in absolute errors relative to conventional techniques were -42.0% and -33.5% for spatial and gait-phase parameters, respectively, indicating the potential of transductive learning models for improving the accuracy of instrumented footwear for ambulatory gait analysis.
Collapse
|
15
|
Nasr A, Inkol KA, Bell S, McPhee J. InverseMuscleNET: Alternative Machine Learning Solution to Static Optimization and Inverse Muscle Modeling. Front Comput Neurosci 2022; 15:759489. [PMID: 35002663 PMCID: PMC8735851 DOI: 10.3389/fncom.2021.759489] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 11/29/2021] [Indexed: 11/13/2022] Open
Abstract
InverseMuscleNET, a machine learning model, is proposed as an alternative to static optimization for resolving the redundancy issue in inverse muscle models. A recurrent neural network (RNN) was optimally configured, trained, and tested to estimate the pattern of muscle activation signals. Five biomechanical variables (joint angle, joint velocity, joint acceleration, joint torque, and activation torque) were used as inputs to the RNN. A set of surface electromyography (EMG) signals, experimentally measured around the shoulder joint for flexion/extension, were used to train and validate the RNN model. The obtained machine learning model yields a normalized regression in the range of 88-91% between experimental data and estimated muscle activation. A sequential backward selection algorithm was used as a sensitivity analysis to discover the less dominant inputs. The order of most essential signals to least dominant ones was as follows: joint angle, activation torque, joint torque, joint velocity, and joint acceleration. The RNN model required 0.06 s of the previous biomechanical input signals and 0.01 s of the predicted feedback EMG signals, demonstrating the dynamic temporal relationships of the muscle activation profiles. The proposed approach permits a fast and direct estimation ability instead of iterative solutions for the inverse muscle model. It raises the possibility of integrating such a model in a real-time device for functional rehabilitation and sports evaluation devices with real-time estimation and tracking. This method provides clinicians with a means of estimating EMG activity without an invasive electrode setup.
Collapse
Affiliation(s)
- Ali Nasr
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - Keaton A Inkol
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - Sydney Bell
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| | - John McPhee
- Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada
| |
Collapse
|
16
|
Sloboda J, Stegall P, McKindles RJ, Stirling L, Siu HC. Utility of Inter-subject Transfer Learning for Wearable-Sensor-Based Joint Torque Prediction Models. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4901-4907. [PMID: 34892307 DOI: 10.1109/embc46164.2021.9630652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
Generalizability between individuals and groups is often a significant hurdle in model development for human subjects research. In the domain of wearable-sensor-controlled exoskeleton devices, the ability to generalize models across subjects or fine-tune more general models to individual subjects is key to enabling widespread adoption of these technologies. Transfer learning techniques applied to machine learning models afford the ability to apply and investigate the viability and utility such knowledge-transfer scenarios. This paper investigates the utility of single- and multi-subject based parameter transfer on LSTM models trained for "sensor-to-joint torque" prediction tasks, with regards to task performance and computational resources required for network training. We find that parameter transfer between both single- and multi-subject models provide useful knowledge transfer, with varying results across specific "source" and "target" subject pairings. This could be leveraged to lower model training time or computational cost in compute-constrained environments or, with further study to understand causal factors of the observed variance in performance across source and target pairings, to minimize data collection and model retraining requirements to select and personalize a generic model for personalized wearable-sensor-based joint torque prediction technologies.
Collapse
|
17
|
Wu S, Zhao W, Barbat S, Ruan J, Ji S. Instantaneous Brain Strain Estimation for Automotive Head Impacts via Deep Learning. STAPP CAR CRASH JOURNAL 2021; 65:139-162. [PMID: 35512787 DOI: 10.4271/2021-22-0006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Efficient brain strain estimation is critical for routine application of a head injury model. Lately, a convolutional neural network (CNN) has been successfully developed to estimate spatially detailed brain strains instantly and accurately in contact sports. Here, we extend its application to automotive head impacts, where impact profiles are typically more complex with longer durations. Head impact kinematics (N=458) from two public databases were used to generate augmented impacts (N=2694). They were simulated using the anisotropic Worcester Head Injury Model (WHIM) V1.0, which provided baseline elementwise peak maximum principal strain (MPS). For each augmented impact, rotational velocity (vrot) and the corresponding rotational acceleration (arot) profiles were concatenated as static images to serve as CNN input. Three training strategies were evaluated: 1) "baseline", using random initial weights; 2) "transfer learning", using weight transfer from a previous CNN model trained on head impacts drawn from contact sports; and 3) "combined training", combining previous training data from contact sports (N=5661) for training. The combined training achieved the best performances. For peak MPS, the CNN achieved a coefficient of determination (R2) of 0.932 and root mean squared error (RMSE) of 0.031 for the real-world testing dataset. It also achieved a success rate of 60.5% and 94.8% for elementwise MPS, where the linear regression slope, k, and correlation coefficient, r, between estimated and simulated MPS did not deviate from 1.0 (when identical) by more than 0.1 and 0.2, respectively. Cumulative strain damage measure (CSDM) from the CNN estimation was also highly accurate compared to those from direct simulation across a range of thresholds (R2 of 0.899-0.943 with RMSE of 0.054-0.069). Finally, the CNN achieved an average k and r of 0.98±0.12 and 0.90±0.07, respectively, for six reconstructed car crash impacts drawn from two other sources independent of the training dataset. Importantly, the CNN is able to efficiently estimate elementwise MPS with sufficient accuracy while conventional kinematic injury metrics cannot. Therefore, the CNN has the potential to supersede current kinematic injury metrics that can only approximate a global peak MPS or CSDM. The CNN technique developed here may offer enhanced utility in the design and development of head protective countermeasures, including in the automotive industry. This is the first study aimed at instantly estimating spatially detailed brain strains for automotive head impacts, which employs >8.8 thousand impact simulations generated from ~1.5 years of nonstop computations on a high-performance computing platform.
Collapse
Affiliation(s)
- Shaoju Wu
- Department of Biomedical Engineering, Worcester Polytechnic Institute, Worcester, MA 01605, USA
| | - Wei Zhao
- Department of Biomedical Engineering, Worcester Polytechnic Institute, Worcester, MA 01605, USA
| | | | - Jesse Ruan
- Tianjin University of Science and Technology, Tianjin, 300222, China
| | - Songbai Ji
- Department of Biomedical Engineering, Worcester Polytechnic Institute, Worcester, MA 01605, USA
| |
Collapse
|
18
|
Aldini S, Lai Y, Carmichael MG, Paul G, Liu D. Real-time Estimation of the Strength Capacity of the Upper Limb for Physical Human-Robot Collaboration. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:4533-4536. [PMID: 34892225 DOI: 10.1109/embc46164.2021.9630230] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In physical Human-Robot Collaboration (pHRC), having an estimate of the operator's strength capacity can help implement control strategies. Currently, the trend is to integrate devices that can measure physiological signals. This is not always a viable option, especially for highly dynamic tasks. For pHRC tasks, the physical interaction point usually occurs at the operator's hand. Therefore, a musculo-skeletal model was used to have a real-time estimation of the strength capacity of the operator's upper limb. First, the model has been simplified to reduce the complexity of the problem. The model was used to obtain offline estimations of the strength capacity, that were then curve-fitted to enable real-time estimation. An experiment was carried out to compare the results of the approximated model with human data. Results suggest that this method for estimating the strength capacity of the upper limb is a viable solution for real-time applications.
Collapse
|
19
|
Enhanced facial expression recognition using 3D point sets and geometric deep learning. Med Biol Eng Comput 2021; 59:1235-1244. [PMID: 34028664 DOI: 10.1007/s11517-021-02383-1] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 05/08/2021] [Indexed: 10/21/2022]
Abstract
Facial expression recognition plays an essential role in human conversation and human-computer interaction. Previous research studies have recognized facial expressions mainly based on 2D image processing requiring sensitive feature engineering and conventional machine learning approaches. The purpose of the present study was to recognize facial expressions by applying a new class of deep learning called geometric deep learning directly on 3D point cloud data. Two databases (Bosphorus and SIAT-3DFE) were used. The Bosphorus database includes sixty-five subjects with seven basic expressions (i.e., anger, disgust, fearness, happiness, sadness, surprise, and neutral). The SIAT-3DFE database has 150 subjects and 4 basic facial expressions (neutral, happiness, sadness, and surprise). First, preprocessing procedures such as face center cropping, data augmentation, and point cloud denoising were applied on 3D face scans. Then, a geometric deep learning model called PointNet++ was applied. A hyperparameter tuning process was performed to find the optimal model parameters. Finally, the developed model was evaluated using the recognition rate and confusion matrix. The facial expression recognition accuracy on the Bosphorus database was 69.01% for 7 expressions and could reach 85.85% when recognizing five specific expressions (anger, disgust, happiness, surprise, and neutral). The recognition rate was 78.70% with the SIAT-3DFE database. The present study suggested that 3D point cloud could be directly processed for facial expression recognition by using geometric deep learning approach. In perspectives, the developed model will be applied for facial palsy patients to guide and optimize the functional rehabilitation program.
Collapse
|
20
|
Burton WS, Myers CA, Rullkoetter PJ. Machine learning for rapid estimation of lower extremity muscle and joint loading during activities of daily living. J Biomech 2021; 123:110439. [PMID: 34004394 DOI: 10.1016/j.jbiomech.2021.110439] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2020] [Revised: 03/25/2021] [Accepted: 04/09/2021] [Indexed: 01/09/2023]
Abstract
Joint contact and muscle forces estimated with musculoskeletal modeling techniques offer useful metrics describing movement quality that benefit multiple research and clinical applications. The expensive processing of laboratory data associated with generating these outputs presents challenges to researchers and clinicians, including significant time and expertise requirements that limit the number of subjects typically evaluated. The objective of the current study was to develop and compare machine learning techniques for rapid, data-driven estimation of musculoskeletal metrics from derived gait lab data. OpenSim estimates of patient joint and muscle forces during activities of daily living were simulated using laboratory data from 70 total knee replacement patients and used to develop 4 different machine learning algorithms. Trained machine learning models predicted both trend and magnitude of estimated joint contact (mean correlation coefficients ranging from 0.93 to 0.94 during gait) and muscle forces (mean correlation coefficients ranging from 0.83 to 0.91 during gait) based on anthropometrics, ground reaction forces, and joint angle data. Patient mechanics were accurately predicted by recurrent neural networks, even after removing dependence on key subsets of predictor features. The ability to quickly estimate patient mechanics from derived measurements of movement has the potential to broaden the impact of musculoskeletal modeling by enabling faster assessment in both clinical and research settings.
Collapse
Affiliation(s)
- William S Burton
- Center for Orthopaedic Biomechanics, University of Denver, Denver, CO, USA.
| | - Casey A Myers
- Center for Orthopaedic Biomechanics, University of Denver, Denver, CO, USA.
| | - Paul J Rullkoetter
- Center for Orthopaedic Biomechanics, University of Denver, Denver, CO, USA.
| |
Collapse
|
21
|
sEMG-Based Neural Network Prediction Model Selection of Gesture Fatigue and Dataset Optimization. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2020; 2020:8853314. [PMID: 33224188 PMCID: PMC7673936 DOI: 10.1155/2020/8853314] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/20/2020] [Revised: 10/12/2020] [Accepted: 10/18/2020] [Indexed: 11/17/2022]
Abstract
The fatigue energy consumption of independent gestures can be obtained by calculating the power spectrum of surface electromyography (sEMG) signals. The existing research studies focus on the fatigue of independent gestures, while the research studies on integrated gestures are few. However, the actual gesture operation mode is usually integrated by multiple independent gestures, so the fatigue degree of integrated gestures can be predicted by training neural network of independent gestures. Three natural gestures including browsing information, playing games, and typing are divided into nine independent gestures in this paper, and the predicted model is established and trained by calculating the energy consumption of independent gestures. The artificial neural networks (ANNs) including backpropagation (BP) neural network, recurrent neural network (RNN), and long short-term memory (LSTM) are used to predict the fatigue of gesture. The support vector machine (SVM) is used to assist verification. Mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) are utilized to evaluate the optimal prediction model. Furthermore, the different datasets of the processed sEMG signal and its decomposed wavelet coefficients are trained, respectively, and the changes of error functions of them are compared. The experimental results show that LSTM model is more suitable for gesture fatigue prediction. The processed sEMG signals are appropriate for using as the training set the fatigue degree of one-handed gesture. It is better to use wavelet decomposition coefficients as datasets to predict the high-dimensional sEMG signals of two-handed gestures. The experimental results can be applied to predict the fatigue degree of complex human-machine interactive gestures, help to avoid unreasonable gestures, and improve the user's interactive experience.
Collapse
|
22
|
Chen C, Huang K, Li D, Zhao Z, Hong J. Multi-Segmentation Parallel CNN Model for Estimating Assembly Torque Using Surface Electromyography Signals. SENSORS 2020; 20:s20154213. [PMID: 32751213 PMCID: PMC7435780 DOI: 10.3390/s20154213] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/13/2020] [Revised: 07/14/2020] [Accepted: 07/20/2020] [Indexed: 11/16/2022]
Abstract
The precise application of tightening torque is one of the important measures to ensure accurate bolt connection and improvement in product assembly quality. Currently, due to the limited assembly space and efficiency, a wrench without the function of torque measurement is still an extensively used assembly tool. Therefore, wrench torque monitoring is one of the urgent problems that needs to be solved. This study proposes a multi-segmentation parallel convolution neural network (MSP-CNN) model for estimating assembly torque using surface electromyography (sEMG) signals, which is a method of torque monitoring through classification methods. The MSP-CNN model contains two independent CNN models with different or offset torque granularities, and their outputs are fused to obtain a finer classification granularity, thus improving the accuracy of torque estimation. First, a bolt tightening test bench is established to collect sEMG signals and tightening torque signals generated when the operator tightens various bolts using a wrench. Second, the sEMG and torque signals are preprocessed to generate the sEMG signal graphs. The range of the torque transducer is divided into several equal subdivision ranges according to different or offset granularities, and each subdivision range is used as a torque label for each torque signal. Then, the training set, verification set, and test set are established for torque monitoring to train the MSP-CNN model. The effects of different signal preprocessing methods, torque subdivision granularities, and pooling methods on the recognition accuracy and torque monitoring accuracy of a single CNN network are compared experimentally. The results show that compared to maximum pooling, average pooling can improve the accuracy of CNN torque classification and recognition. Moreover, the MSP-CNN model can improve the accuracy of torque monitoring as well as solve the problems of non-convergence and slow convergence of independent CNN network models.
Collapse
Affiliation(s)
- Chengjun Chen
- School of Mechanical and Automotive Engineering, Qingdao University of Technology, Qingdao 266520, China; (K.H.); (D.L.); (Z.Z.)
- Key Lab of Industrial Fluid Energy Conservation and Pollution Control, Ministry of Education, Qingdao University of Technology, Qingdao 266520, China
- Correspondence: ; Tel.: +86-532-6805-2755
| | - Kai Huang
- School of Mechanical and Automotive Engineering, Qingdao University of Technology, Qingdao 266520, China; (K.H.); (D.L.); (Z.Z.)
- Key Lab of Industrial Fluid Energy Conservation and Pollution Control, Ministry of Education, Qingdao University of Technology, Qingdao 266520, China
| | - Dongnian Li
- School of Mechanical and Automotive Engineering, Qingdao University of Technology, Qingdao 266520, China; (K.H.); (D.L.); (Z.Z.)
- Key Lab of Industrial Fluid Energy Conservation and Pollution Control, Ministry of Education, Qingdao University of Technology, Qingdao 266520, China
| | - Zhengxu Zhao
- School of Mechanical and Automotive Engineering, Qingdao University of Technology, Qingdao 266520, China; (K.H.); (D.L.); (Z.Z.)
- Key Lab of Industrial Fluid Energy Conservation and Pollution Control, Ministry of Education, Qingdao University of Technology, Qingdao 266520, China
| | - Jun Hong
- School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an 710049, China;
| |
Collapse
|
23
|
Machine learning methods to support personalized neuromusculoskeletal modelling. Biomech Model Mechanobiol 2020; 19:1169-1185. [DOI: 10.1007/s10237-020-01367-8] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2020] [Accepted: 07/08/2020] [Indexed: 12/19/2022]
|
24
|
Hashemi S, Veisi H, Jafarzadehpur E, Rahmani R, Heshmati Z. Multi-view deep learning for rigid gas permeable lens base curve fitting based on Pentacam images. Med Biol Eng Comput 2020; 58:1467-1482. [PMID: 32363555 DOI: 10.1007/s11517-020-02154-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2019] [Accepted: 03/05/2020] [Indexed: 02/06/2023]
Abstract
Many studies in the rigid gas permeable (RGP) lens fitting field have focused on providing the best fit for patients with irregular astigmatism, a challenging issue. Despite the ease and accuracy of fitting in the current fitting methods, no studies have provided a high-pace solution with the final best fit to assist experts. This work presents a deep learning solution for identifying features in Pentacam four refractive maps and RGP base curve identification. An authentic dataset of 247 samples of Pentacam four refractive maps was gathered, providing a multi-view image of the corneal structure. Scratch-based convolutional neural network (CNN) architectures and well-known CNN architectures such as AlexNet, GoogLeNet, and ResNet have been used to extract features and transfer learning. Features are aggregated through a fusion technique. Based on a comparison of means square error (MSE) of normalized labels, the multi-view scratch-based CNN provided R-squared of 0.849, 0.846, 0.835, and 0.834 followed by GoogLeNet, comparable with current methods. Transfer learning outperforms various scratch-based CNN models, through which proper specifications some scratch-based models were able to increase coefficient of determinations. CNNs on multi-view Pentacam images have enabled fast detection of the RGP lens base curve, higher patient satisfaction, and reduced chair time. Graphical abstract The Pentacam four refractive maps is learned by the proposed scratch-based and transfer learning-based CNN methodology. The deep network-based solutions enable identification of rigid gas permeable lens for patients with irregular astigmatism.
Collapse
Affiliation(s)
- Sara Hashemi
- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran
| | - Hadi Veisi
- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran.
| | - Ebrahim Jafarzadehpur
- Department of Optometry, School of Rehabilitation Science, Iran University of Medical Sciences, Tehran, Iran
| | | | | |
Collapse
|
25
|
Transfer learning for informative-frame selection in laryngoscopic videos through learned features. Med Biol Eng Comput 2020; 58:1225-1238. [DOI: 10.1007/s11517-020-02127-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2019] [Accepted: 01/07/2020] [Indexed: 02/06/2023]
|
26
|
Zhu Y, Xu W, Luo G, Wang H, Yang J, Lu W. Random Forest enhancement using improved Artificial Fish Swarm for the medial knee contact force prediction. Artif Intell Med 2020; 103:101811. [PMID: 32143807 DOI: 10.1016/j.artmed.2020.101811] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2019] [Revised: 01/06/2020] [Accepted: 01/28/2020] [Indexed: 10/25/2022]
Abstract
Knee contact force (KCF) is an important factor to evaluate the knee joint function for the patients with knee joint impairment. However, the KCF measurement based on the instrumented prosthetic implants or inverse dynamics analysis is limited due to the invasive, expensive price and time consumption. In this work, we propose a KCF prediction method by integrating the Artificial Fish Swarm and the Random Forest algorithm. First, we train a Random Forest to learn the nonlinear relation between gait parameters (input) and contact pressures (output) based on a dataset of three patients instrumented with knee replacement. Then, we use the improved artificial fish group algorithm to optimize the main parameters of the Random Forest based KCF prediction model. The extensive experiments verify that our method can predict the medial knee contact force both before and after the intervention of gait patterns, and the performance outperforms the classical multi-body dynamics analysis and artificial neural network model.
Collapse
Affiliation(s)
- Yean Zhu
- East China Jiaotong University, Nanchang, China.
| | - Weiyi Xu
- East China Jiaotong University, Nanchang, China.
| | - Guoliang Luo
- East China Jiaotong University, Nanchang, China.
| | - Haolun Wang
- East China Jiaotong University, Nanchang, China.
| | | | - Wei Lu
- Jiangxi Provincial People's Hospital, Nanchang, China.
| |
Collapse
|
27
|
Machine learning approach to predict center of pressure trajectories in a complete gait cycle: a feedforward neural network vs. LSTM network. Med Biol Eng Comput 2019; 57:2693-2703. [PMID: 31650342 DOI: 10.1007/s11517-019-02056-0] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2019] [Accepted: 10/03/2019] [Indexed: 10/25/2022]
Abstract
Center of pressure (COP) trajectories of human can maintain regulation of forward progression and stability of lateral sway during walking. The insole pressure system can only detect COP trajectories of each foot during single stance. In this study, we developed artificial neural network models that could present COP trajectories in an integrated coordinate system during a complete gait cycle using pressure information of the insole system. A feed forward artificial neural network (FFANN) and a long short-term memory (LSTM) model were developed. For FFANN, among 198 pressure sensors from Pedar-X insoles, proper input variables were selected using sequential forward selection to reduce input dimension. The LSTM model used all 198 signals as inputs because of its self-learning characteristic. As results of cross-validation, the FFANN model showed correlation coefficients of 0.98-0.99 and 0.93-0.95 in anterior/posterior and medial/lateral directions, respectively. For the LSTM model, correlation coefficients were similar to those of FFANN. However, the relative root mean square error (12.5%) of the FFANN model was higher than that (9.8%) of the LSTM model in medial/lateral direction (p = 0.03). This study can be used for quantitative evaluation of clinical diagnosis and rehabilitation status for patient with various diseases through further training using varied databases. Graphical abstract Architectures of neural networks developed in this study (a feed forward artificial neural network; b LSTM network).
Collapse
|
28
|
Choi A, Jung H, Mun JH. Single Inertial Sensor-Based Neural Networks to Estimate COM-COP Inclination Angle During Walking. SENSORS (BASEL, SWITZERLAND) 2019; 19:E2974. [PMID: 31284482 PMCID: PMC6651410 DOI: 10.3390/s19132974] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/22/2019] [Revised: 06/29/2019] [Accepted: 07/04/2019] [Indexed: 11/16/2022]
Abstract
A biomechanical understanding of gait stability is needed to reduce falling risk. As a typical parameter, the COM-COP (center of mass-center of pressure) inclination angle (IA) could provide valuable insight into postural control and balance recovery ability. In this study, an artificial neural network (ANN) model was developed to estimate COM-COP IA based on signals using an inertial sensor. Also, we evaluated how different types of ANN and the cutoff frequency of the low-pass filter applied to input signals could affect the accuracy of the model. An inertial measurement unit (IMU) including an accelerometer, gyroscope, and magnetometer sensors was fabricated as a prototype. The COM-COP IA was calculated using a 3D motion analysis system including force plates. In order to predict the COM-COP IA, a feed-forward ANN and long-short term memory (LSTM) network was developed. As a result, the feed-forward ANN showed a relative root-mean-square error (rRMSE) of 15% while the LSTM showed an improved accuracy of 9% rRMSE. Additionally, the LSTM displayed a stable accuracy regardless of the cutoff frequency of the filter applied to the input signals. This study showed that estimating the COM-COP IA was possible with a cheap inertial sensor system. Furthermore, the neural network models in this study can be implemented in systems to monitor the balancing ability of the elderly or patients with impaired balancing ability.
Collapse
Affiliation(s)
- Ahnryul Choi
- Department of Biomedical Engineering, College of Medical Convergence, Catholic Kwandong University, 24, Beomilro 579beongil, Gangneung, Gangwon 25601, Korea
- Department of Bio-Mechatronic Engineering, College of Biotechnology and Bioengineering, Sungkyunkwan University, 2066 Seoburo, Jangan, Suwon, Gyeonggi 16419, Korea
| | - Hyunwoo Jung
- Department of Bio-Mechatronic Engineering, College of Biotechnology and Bioengineering, Sungkyunkwan University, 2066 Seoburo, Jangan, Suwon, Gyeonggi 16419, Korea
| | - Joung Hwan Mun
- Department of Bio-Mechatronic Engineering, College of Biotechnology and Bioengineering, Sungkyunkwan University, 2066 Seoburo, Jangan, Suwon, Gyeonggi 16419, Korea.
| |
Collapse
|