1
|
Schick D, Schick J, David JP, Neubauer R, Glaser M. Conceptualization of Cloud-Based Motion Analysis and Navigation for Wearable Robotic Applications. SENSORS (BASEL, SWITZERLAND) 2024; 24:4997. [PMID: 39124043 PMCID: PMC11314675 DOI: 10.3390/s24154997] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/21/2024] [Revised: 07/19/2024] [Accepted: 07/30/2024] [Indexed: 08/12/2024]
Abstract
The behavior of pedestrians in a non-constrained environment is difficult to predict. In wearable robotics, this poses a challenge, since devices like lower-limb exoskeletons and active orthoses need to support different walking activities, including level walking and climbing stairs. While a fixed movement trajectory can be easily supported, switches between these activities are difficult to predict. Moreover, the demand for these devices is expected to rise in the years ahead. In this work, we propose a cloud software system for use in wearable robotics, based on geographical mapping techniques and Human Activity Recognition (HAR). The system aims to give context to the surrounding pedestrians by providing hindsight information. The system was partially implemented and tested. The results indicate a viable concept with great extensibility prospects.
Collapse
Affiliation(s)
| | | | | | | | - Markus Glaser
- Institute for High Integrity Mechatronics Systems, Aalen University, 73430 Aalen, Germany; (D.S.)
| |
Collapse
|
2
|
Strick JA, Farris RJ, Sawicki JT. A Novel Gait Event Detection Algorithm Using a Thigh-Worn Inertial Measurement Unit and Joint Angle Information. J Biomech Eng 2024; 146:044502. [PMID: 38183222 DOI: 10.1115/1.4064435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Accepted: 12/27/2023] [Indexed: 01/07/2024]
Abstract
This paper describes the development and evaluation of a novel, threshold-based gait event detection algorithm utilizing only one thigh inertial measurement unit (IMU) and unilateral, sagittal plane hip and knee joint angles. The algorithm was designed to detect heel strike (HS) and toe off (TO) gait events, with the eventual goal of detection in a real-time exoskeletal control system. The data used in the development and evaluation of the algorithm were obtained from two gait databases, each containing synchronized IMU and ground reaction force (GRF) data. All database subjects were healthy individuals walking in either a level-ground, urban environment or a treadmill lab environment. Inertial measurements used were three-dimensional thigh accelerations and three-dimensional thigh angular velocities. Parameters for the TO algorithm were identified on a per-subject basis. The GRF data were utilized to validate the algorithm's timing accuracy and quantify the fidelity of the algorithm, measured by the F1-Score. Across all participants, the algorithm reported a mean timing error of -41±20 ms with an F1-Score of 0.988 for HS. For TO, the algorithm reported a mean timing error of -1.4±21 ms with an F1-Score of 0.991. The results of this evaluation suggest that this algorithm is a promising solution to inertial based gait event detection; however, further refinement and real-time evaluation are required for use in exoskeletal control.
Collapse
Affiliation(s)
- Jacob A Strick
- Center for Rotating Machinery Dynamics and Control (RoMaDyC), Washkewicz College of Engineering, Cleveland State University, 2121 Euclid Avenue, Cleveland, OH 44115
| | - Ryan J Farris
- Department of Engineering, Messiah University, One University Avenue, Mechanicsburg, PA 17055
| | - Jerzy T Sawicki
- Center for Rotating Machinery Dynamics and Control (RoMaDyC), Washkewicz College of Engineering, Cleveland State University, 2121 Euclid Avenue, Cleveland, OH 44115
| |
Collapse
|
3
|
Wang F, Chen F, Dong Y, Yong Q, Yang X, Zheng L, Zhang X, Su H. Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion. Biomimetics (Basel) 2023; 8:591. [PMID: 38132530 PMCID: PMC10741906 DOI: 10.3390/biomimetics8080591] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Revised: 11/08/2023] [Accepted: 12/01/2023] [Indexed: 12/23/2023] Open
Abstract
As human-robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method's effectiveness through a series of experiments, evaluating the system's robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion.
Collapse
Affiliation(s)
- Feilong Wang
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China; (F.W.); (F.C.); (X.Y.)
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| | - Furong Chen
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China; (F.W.); (F.C.); (X.Y.)
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| | - Yanling Dong
- School of Foreign Languages, Shandong University, Jinan 250100, China;
| | - Qi Yong
- ESIEE Paris, 93160 Noisy-le-Grand, France;
| | - Xiaolong Yang
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China; (F.W.); (F.C.); (X.Y.)
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| | - Long Zheng
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| | - Xinming Zhang
- College of Mechanical and Electrical Engineering, Changchun University of Science and Technology, Changchun 130022, China; (F.W.); (F.C.); (X.Y.)
| | - Hang Su
- Weihai Institute for Bionics, Jilin University, Weihai 264402, China
| |
Collapse
|
4
|
Kuo CT, Lin JJ, Jen KK, Hsu WL, Wang FC, Tsao TC, Yen JY. Human Posture Transition-Time Detection Based upon Inertial Measurement Unit and Long Short-Term Memory Neural Networks. Biomimetics (Basel) 2023; 8:471. [PMID: 37887602 PMCID: PMC10604330 DOI: 10.3390/biomimetics8060471] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 09/16/2023] [Accepted: 09/28/2023] [Indexed: 10/28/2023] Open
Abstract
As human-robot interaction becomes more prevalent in industrial and clinical settings, detecting changes in human posture has become increasingly crucial. While recognizing human actions has been extensively studied, the transition between different postures or movements has been largely overlooked. This study explores using two deep-learning methods, the linear Feedforward Neural Network (FNN) and Long Short-Term Memory (LSTM), to detect changes in human posture among three different movements: standing, walking, and sitting. To explore the possibility of rapid posture-change detection upon human intention, the authors introduced transition stages as distinct features for the identification. During the experiment, the subject wore an inertial measurement unit (IMU) on their right leg to measure joint parameters. The measurement data were used to train the two machine learning networks, and their performances were tested. This study also examined the effect of the sampling rates on the LSTM network. The results indicate that both methods achieved high detection accuracies. Still, the LSTM model outperformed the FNN in terms of speed and accuracy, achieving 91% and 95% accuracy for data sampled at 25 Hz and 100 Hz, respectively. Additionally, the network trained for one test subject was able to detect posture changes in other subjects, demonstrating the feasibility of personalized or generalized deep learning models for detecting human intentions. The accuracies for posture transition time and identification at a sampling rate of 100 Hz were 0.17 s and 94.44%, respectively. In summary, this study achieved some good outcomes and laid a crucial foundation for the engineering application of digital twins, exoskeletons, and human intention control.
Collapse
Affiliation(s)
- Chun-Ting Kuo
- Department of Mechanical Engineering, National Taiwan University, Taipei 106319, Taiwan
| | - Jun-Ji Lin
- Department of Mechanical Engineering, National Taiwan University, Taipei 106319, Taiwan
| | - Kuo-Kuang Jen
- Missile and Rocket Research Division, National Chung Shan Institute of Science and Technology, Taoyuan 325204, Taiwan
| | - Wei-Li Hsu
- School and Graduate Institute of Physical Therapy, National Taiwan University, Taipei 106319, Taiwan
| | - Fu-Cheng Wang
- Department of Mechanical Engineering, National Taiwan University, Taipei 106319, Taiwan
| | - Tsu-Chin Tsao
- Mechanical and Aerospace Engineering, Samueli School of Engineering, UCLA, Los Angeles, CA 90095, USA
| | - Jia-Yush Yen
- Department of Mechanical Engineering, National Taiwan University, Taipei 106319, Taiwan
- Department of Mechanical Engineering, National Taiwan University of Science and Technology, Taipei 106319, Taiwan
| |
Collapse
|
5
|
Son CS, Kang WS. Multivariate CNN Model for Human Locomotion Activity Recognition with a Wearable Exoskeleton Robot. Bioengineering (Basel) 2023; 10:1082. [PMID: 37760184 PMCID: PMC10525937 DOI: 10.3390/bioengineering10091082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Revised: 09/05/2023] [Accepted: 09/09/2023] [Indexed: 09/29/2023] Open
Abstract
This study introduces a novel convolutional neural network (CNN) architecture, encompassing both single and multi-head designs, developed to identify a user's locomotion activity while using a wearable lower limb robot. Our research involved 500 healthy adult participants in an activities of daily living (ADL) space, conducted from 1 September to 30 November 2022. We collected prospective data to identify five locomotion activities (level ground walking, stair ascent/descent, and ramp ascent/descent) across three terrains: flat ground, staircase, and ramp. To evaluate the predictive capabilities of the proposed CNN architectures, we compared its performance with three other models: one CNN and two hybrid models (CNN-LSTM and LSTM-CNN). Experiments were conducted using multivariate signals of various types obtained from electromyograms (EMGs) and the wearable robot. Our results reveal that the deeper CNN architecture significantly surpasses the performance of the three competing models. The proposed model, leveraging encoder data such as hip angles and velocities, along with postural signals such as roll, pitch, and yaw from the wearable lower limb robot, achieved superior performance with an inference speed of 1.14 s. Specifically, the F-measure performance of the proposed model reached 96.17%, compared to 90.68% for DDLMI, 94.41% for DeepConvLSTM, and 95.57% for LSTM-CNN, respectively.
Collapse
Affiliation(s)
- Chang-Sik Son
- Division of Intelligent Robot, Daegu Gyeongbuk Institute of Science & Technology (DGIST), Daegu 42988, Republic of Korea;
| | - Won-Seok Kang
- Division of Intelligent Robot, Daegu Gyeongbuk Institute of Science & Technology (DGIST), Daegu 42988, Republic of Korea;
- Department of Biomedical Science, Graduate School, Kyungpook National University, Daegu 41944, Republic of Korea
| |
Collapse
|
6
|
Su D, Hu Z, Wu J, Shang P, Luo Z. Review of adaptive control for stroke lower limb exoskeleton rehabilitation robot based on motion intention recognition. Front Neurorobot 2023; 17:1186175. [PMID: 37465413 PMCID: PMC10350518 DOI: 10.3389/fnbot.2023.1186175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Accepted: 06/13/2023] [Indexed: 07/20/2023] Open
Abstract
Stroke is a significant cause of disability worldwide, and stroke survivors often experience severe motor impairments. Lower limb rehabilitation exoskeleton robots provide support and balance for stroke survivors and assist them in performing rehabilitation training tasks, which can effectively improve their quality of life during the later stages of stroke recovery. Lower limb rehabilitation exoskeleton robots have become a hot topic in rehabilitation therapy research. This review introduces traditional rehabilitation assessment methods, explores the possibility of lower limb exoskeleton robots combining sensors and electrophysiological signals to assess stroke survivors' rehabilitation objectively, summarizes standard human-robot coupling models of lower limb rehabilitation exoskeleton robots in recent years, and critically introduces adaptive control models based on motion intent recognition for lower limb exoskeleton robots. This provides new design ideas for the future combination of lower limb rehabilitation exoskeleton robots with rehabilitation assessment, motion assistance, rehabilitation treatment, and adaptive control, making the rehabilitation assessment process more objective and addressing the shortage of rehabilitation therapists to some extent. Finally, the article discusses the current limitations of adaptive control of lower limb rehabilitation exoskeleton robots for stroke survivors and proposes new research directions.
Collapse
Affiliation(s)
- Dongnan Su
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Zhigang Hu
- School of Medical Technology and Engineering, Henan University of Science and Technology, Luoyang, China
- Henan Intelligent Rehabilitation Medical Robot Engineering Research Center, Henan University of Science and Technology, Luoyang, China
| | - Jipeng Wu
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Peng Shang
- Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Zhaohui Luo
- State-Owned Changhong Machinery Factory, Guilin, China
| |
Collapse
|
7
|
Mekritthikrai N, Yuenyongchaiwat K, Thanawattano C. Concurrent validity and reliability of new application for 6-min walk test in healthy adults. Heliyon 2023; 9:e17854. [PMID: 37539231 PMCID: PMC10395284 DOI: 10.1016/j.heliyon.2023.e17854] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Revised: 06/28/2023] [Accepted: 06/29/2023] [Indexed: 08/05/2023] Open
Abstract
Background Evaluation assessments for physical performance, such as walking tests, are important for measuring a person's well-being. As of current, medical technology is primarily used to administer these assessments. However, medical devices are not easily accessible and are intended for research purposes only, and hence inconvenient for clinical use. Therefore, we aimed to develop a prototype physical performance assessor device with a mobile application and explored concurrent validity and reliability between the standard 6-min walk test (6MWT) and wearable sensor 6MWT using 6-min walk distance in healthy adults. Methods Sixty healthy males and females, above 18 years of age, were required to attach a sensor to their dominant ankle while the standard protocol for 6MWT was performed. After completing the walking test, the distance from the wearable sensor 6MWT with a mobile application and the standard 6MWT were recorded and compared. Results There was no significant difference between the distance between the standard 6MWT (410.12 ± 74.03 m) and the distance obtained with the wearable sensor. Concurrent validity was found to be moderate, and Cronbach's alpha was 0.79, which indicated good internal consistency. Conclusion The innovative prototype wearable walking sensor with a mobile application can effectively evaluate physical performance in healthy individuals.Clinical trial registration number: TCTR20220801002.
Collapse
Affiliation(s)
- Nuttawuth Mekritthikrai
- Department of Physiotherapy, Faculty of Allied Health Sciences, Thammasat University, Thailand
| | - Kornanong Yuenyongchaiwat
- Department of Physiotherapy, Faculty of Allied Health Sciences, Thammasat University, Thailand
- Thammasat University Research Unit in Physical Therapy in Cardiovascular and Respiratory Systems, Thailand
| | | |
Collapse
|
8
|
Jiang J, Chen P, Peng J, Qiao X, Zhu F, Zhong J. Design and Optimization of Lower Limb Rehabilitation Exoskeleton with a Multiaxial Knee Joint. Biomimetics (Basel) 2023; 8:biomimetics8020156. [PMID: 37092408 PMCID: PMC10123689 DOI: 10.3390/biomimetics8020156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 04/07/2023] [Accepted: 04/11/2023] [Indexed: 04/25/2023] Open
Abstract
To facilitate rehabilitation training for patients, we proposed the implementation of an anthropomorphic exoskeleton structure that incorporates a variable instantaneous center of rotation (ICR). This design considers the variability in knee ICR among individuals, resulting from the irregular form of the human knee joint, and leverages a double-degrees-of-freedom (2DOF) five-bar mechanism to adapt to these differences. The walking gait of the human lower limb and the corresponding knee ICR were measured and calculated using an optical 3D motion capture system. The optimal dimension parameters of the five-bar mechanism were then obtained through the optimization of human movement position inputs and rod length constraints to minimize the error in knee ICR, gait angle, and ankle trajectory between the human and the exoskeleton. Finally, we established an exoskeleton prototype to conduct relevant experimental tests. The experiment results showed that the average errors of knee ICR trajectory, hip angle, knee angle, and ankle trajectory were 5.52 × 10-4 m, 0.010 rad, 0.014 rad, and 1.57 × 10-3 m, respectively. The experimental results demonstrated that the exoskeleton's movement trajectory was close to the human's, reducing the human-mechanism interaction force and improving patient comfort during rehabilitation training.
Collapse
Affiliation(s)
- Jiandong Jiang
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
- Key Laboratory of Special Purpose Equipment and Advanced Manufacturing Technology Ministry of Education, Zhejiang University of Technology, Hangzhou 310023, China
| | - Peisong Chen
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
| | - Jiyu Peng
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
- Key Laboratory of Special Purpose Equipment and Advanced Manufacturing Technology Ministry of Education, Zhejiang University of Technology, Hangzhou 310023, China
| | - Xin Qiao
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
- Key Laboratory of Special Purpose Equipment and Advanced Manufacturing Technology Ministry of Education, Zhejiang University of Technology, Hangzhou 310023, China
| | - Fengle Zhu
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
- Key Laboratory of Special Purpose Equipment and Advanced Manufacturing Technology Ministry of Education, Zhejiang University of Technology, Hangzhou 310023, China
| | - Jiang Zhong
- College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China
- Key Laboratory of Special Purpose Equipment and Advanced Manufacturing Technology Ministry of Education, Zhejiang University of Technology, Hangzhou 310023, China
| |
Collapse
|
9
|
Moreira L, Figueiredo J, Cerqueira J, Santos CP. A Review on Locomotion Mode Recognition and Prediction When Using Active Orthoses and Exoskeletons. SENSORS (BASEL, SWITZERLAND) 2022; 22:7109. [PMID: 36236204 PMCID: PMC9573198 DOI: 10.3390/s22197109] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 09/09/2022] [Accepted: 09/16/2022] [Indexed: 06/16/2023]
Abstract
Understanding how to seamlessly adapt the assistance of lower-limb wearable assistive devices (active orthosis (AOs) and exoskeletons) to human locomotion modes (LMs) is challenging. Several algorithms and sensors have been explored to recognize and predict the users' LMs. Nevertheless, it is not yet clear which are the most used and effective sensor and classifier configurations in AOs/exoskeletons and how these devices' control is adapted according to the decoded LMs. To explore these aspects, we performed a systematic review by electronic search in Scopus and Web of Science databases, including published studies from 1 January 2010 to 31 August 2022. Sixteen studies were included and scored with 84.7 ± 8.7% quality. Decoding focused on level-ground walking along with ascent/descent stairs tasks performed by healthy subjects. Time-domain raw data from inertial measurement unit sensors were the most used data. Different classifiers were employed considering the LMs to decode (accuracy above 90% for all tasks). Five studies have adapted the assistance of AOs/exoskeletons attending to the decoded LM, in which only one study predicted the new LM before its occurrence. Future research is encouraged to develop decoding tools considering data from people with lower-limb impairments walking at self-selected speeds while performing daily LMs with AOs/exoskeletons.
Collapse
Affiliation(s)
- Luís Moreira
- Center for Microelectromechanical Systems (CMEMS), University of Minho, 4800-058 Guimarães, Portugal
- LABBELS—Associate Laboratory, 4800-058 Guimarães, Portugal
| | - Joana Figueiredo
- Center for Microelectromechanical Systems (CMEMS), University of Minho, 4800-058 Guimarães, Portugal
- LABBELS—Associate Laboratory, 4800-058 Guimarães, Portugal
| | - João Cerqueira
- Life and Health Sciences Research Institute (ICVS), University of Minho, 4800-058 Guimarães, Portugal
- Clinical Academic Center (2CA-Braga), Hospital of Braga, 4700-099 Braga, Portugal
| | - Cristina P. Santos
- Center for Microelectromechanical Systems (CMEMS), University of Minho, 4800-058 Guimarães, Portugal
- LABBELS—Associate Laboratory, 4800-058 Guimarães, Portugal
- Clinical Academic Center (2CA-Braga), Hospital of Braga, 4700-099 Braga, Portugal
| |
Collapse
|
10
|
ZHENG JIANBIN, PENG MINGPENG, HUANG LIPING, GAO YIFAN, LI ZEFANG, WANG BINFENG, WANG YU. A CNN–SVM MODEL USING IMU FOR LOCOMOTION MODE RECOGNITION IN LOWER EXTREMITY EXOSKELETON. J MECH MED BIOL 2022. [DOI: 10.1142/s0219519422500439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Human activity intention is indispensable for wearable powered lower extremity exoskeleton such that ensuring the compliant control of the robot. Lots of researches have been done on gait phase detection, which served as a sub-module of locomotion mode recognition to support the follow-on task. Therefore, it is self-evident that locomotion mode recognition is of great importance. Many model-based recognition methods are usually applied in manual extraction of cumbersome features, such as the traditional neural network (NN), support vector machine (SVM), etc. In contrast, the feature mapping layer coming with the convolutional neural network (CNN) can effectively solve the above time-consuming problem. Given that the training of NN is prone to overfitting, SVM with optimal characteristics is considered. A hybrid CNN–SVM model is proposed to identify human locomotion modes by collecting multi-channel inertial measurement unit (IMU) signals and is integrated with the error correction function of the finite state machine (FSM). Therefore, the CNN–SVM model has great influence on the generalization performance and recognition accuracy. The recognition rates of five single locomotion modes and eight mixed locomotion modes reach 97.91% and 98.93%, respectively. The system meets the demand of real-time performance, and the recognition time exceeds 370[Formula: see text]ms on heel strike.
Collapse
Affiliation(s)
- JIANBIN ZHENG
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - MINGPENG PENG
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - LIPING HUANG
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - YIFAN GAO
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - ZEFANG LI
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - BINFENG WANG
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| | - YU WANG
- School of Information Engineering, Wuhan University of Technology, Wuhan, Hubei 430070, P. R. China
| |
Collapse
|
11
|
Measurement, Evaluation, and Control of Active Intelligent Gait Training Systems—Analysis of the Current State of the Art. ELECTRONICS 2022. [DOI: 10.3390/electronics11101633] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Gait recognition and rehabilitation has been a research hotspot in recent years due to its importance to medical care and elderly care. Active intelligent rehabilitation and assistance systems for lower limbs integrates mechanical design, sensing technology, intelligent control, and robotics technology, and is one of the effective ways to resolve the above problems. In this review, crucial technologies and typical prototypes of active intelligent rehabilitation and assistance systems for gait training are introduced. The limitations, challenges, and future directions in terms of gait measurement and intention recognition, gait rehabilitation evaluation, and gait training control strategies are discussed. To address the core problems of the sensing, evaluation and control technology of the active intelligent gait training systems, the possible future research directions are proposed. Firstly, different sensing methods need to be proposed for the decoding of human movement intention. Secondly, the human walking ability evaluation models will be developed by integrating the clinical knowledge and lower limb movement data. Lastly, the personalized gait training strategy for collaborative control of human–machine systems needs to be implemented in the clinical applications.
Collapse
|
12
|
Shi K, Mu F, Huang R, Huang K, Peng Z, Zou C, Yang X, Cheng H. Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism. Front Neurosci 2022; 16:796290. [PMID: 35546887 PMCID: PMC9082753 DOI: 10.3389/fnins.2022.796290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2021] [Accepted: 03/16/2022] [Indexed: 11/13/2022] Open
Abstract
A challenging task for the biological neural signal-based human-exoskeleton interface is to achieve accurate lower limb movement prediction of patients with hemiplegia in rehabilitation training scenarios. The human-exoskeleton interface based on single-modal biological signals such as electroencephalogram (EEG) is currently not mature in predicting movements, due to its unreliability. The multimodal human-exoskeleton interface is a very novel solution to this problem. This kind of interface normally combines the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction is still limited—the connection between sEMG and EEG signals and the deep feature fusion between them are ignored. In this article, a Dense con-attention mechanism-based Multimodal Enhance Fusion Network (DMEFNet) is proposed for predicting lower limb movement of patients with hemiplegia. The DMEFNet introduces the con-attention structure to extract the common attention between sEMG and EEG signal features. To verify the effectiveness of DMEFNet, an sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96 and 88.44%, respectively.
Collapse
Affiliation(s)
- Kecheng Shi
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Fengjun Mu
- School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Rui Huang
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Ke Huang
- Glasgow College, University of Electronic Science and Technology of China, Chengdu, China
| | - Zhinan Peng
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Chaobin Zou
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| | - Xiao Yang
- Department of Orthopedics, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| | - Hong Cheng
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
13
|
Lotfi M, Kedir-Talha M. Classification of gait patterns in the time domain using accelerometer signal. 2022 7TH INTERNATIONAL CONFERENCE ON IMAGE AND SIGNAL PROCESSING AND THEIR APPLICATIONS (ISPA) 2022. [DOI: 10.1109/ispa54004.2022.9786352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Affiliation(s)
- Madaoui Lotfi
- University of Sciences and Technology Houari Boumediene,Department of Electrical Engineering,Algiers,Algeria
| | - Malika Kedir-Talha
- University of Sciences and Technology Houari Boumediene,Department of Electrical Engineering,Algiers,Algeria
| |
Collapse
|
14
|
An sEMG based adaptive method for human-exoskeleton collaboration in variable walking environments. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103477] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
15
|
Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion. Appl Bionics Biomech 2022; 2022:9933018. [PMID: 35378794 PMCID: PMC8976668 DOI: 10.1155/2022/9933018] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2021] [Revised: 11/10/2021] [Accepted: 03/05/2022] [Indexed: 11/18/2022] Open
Abstract
In recent decades, although the research on gait recognition of lower limb exoskeleton robot has been widely developed, there are still limitations in rehabilitation training and clinical practice. The emergence of interactive information fusion technology provides a new research idea for the solution of this problem, and it is also the development trend in the future. In order to better explore the issue, this paper summarizes gait recognition based on interactive information fusion of lower limb exoskeleton robots. This review introduces the current research status, methods, and directions for information acquisition, interaction, fusion, and gait recognition of exoskeleton robots. The content involves the research progress of information acquisition methods, sensor placements, target groups, lower limb sports biomechanics, interactive information fusion, and gait recognition model. Finally, the current challenges, possible solutions, and promising prospects are analysed and discussed, which provides a useful reference resource for the study of interactive information fusion and gait recognition of rehabilitation exoskeleton robots.
Collapse
|
16
|
Cao W, Chen C, Wang D, Wu X, Chen L, Xu T, Liu J. A Lower Limb Exoskeleton With Rigid and Soft Structure for Loaded Walking Assistance. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2021.3125723] [Citation(s) in RCA: 10] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
17
|
Shi K, Huang R, Peng Z, Mu F, Yang X. MCSNet: Channel Synergy-Based Human-Exoskeleton Interface With Surface Electromyogram. Front Neurosci 2021; 15:704603. [PMID: 34867145 PMCID: PMC8636050 DOI: 10.3389/fnins.2021.704603] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 10/08/2021] [Indexed: 11/13/2022] Open
Abstract
The human-robot interface (HRI) based on biological signals can realize the natural interaction between human and robot. It has been widely used in exoskeleton robots recently to help predict the wearer's movement. Surface electromyography (sEMG)-based HRI has mature applications on the exoskeleton. However, the sEMG signals of paraplegic patients' lower limbs are weak, which means that most HRI based on lower limb sEMG signals cannot be applied to the exoskeleton. Few studies have explored the possibility of using upper limb sEMG signals to predict lower limb movement. In addition, most HRIs do not consider the contribution and synergy of sEMG signal channels. This paper proposes a human-exoskeleton interface based on upper limb sEMG signals to predict lower limb movements of paraplegic patients. The interface constructs an channel synergy-based network (MCSNet) to extract the contribution and synergy of different feature channels. An sEMG data acquisition experiment is designed to verify the effectiveness of MCSNet. The experimental results show that our method has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 94.51 and 80.75%, respectively. Furthermore, feature visualization and model ablation analysis show that the features extracted by MCSNet are physiologically interpretable.
Collapse
Affiliation(s)
- Kecheng Shi
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China.,Center for Robotics, University of Electronic Science and Technology of China, Chengdu, China.,Engineering Research Center of Human Robot Hybrid Intelligent Technologies and Systems, Ministry of Education, University of Electronic Science and Technology of China, Chengdu, China
| | - Rui Huang
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China.,Center for Robotics, University of Electronic Science and Technology of China, Chengdu, China.,Engineering Research Center of Human Robot Hybrid Intelligent Technologies and Systems, Ministry of Education, University of Electronic Science and Technology of China, Chengdu, China
| | - Zhinan Peng
- School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu, China.,Center for Robotics, University of Electronic Science and Technology of China, Chengdu, China.,Engineering Research Center of Human Robot Hybrid Intelligent Technologies and Systems, Ministry of Education, University of Electronic Science and Technology of China, Chengdu, China
| | - Fengjun Mu
- Center for Robotics, University of Electronic Science and Technology of China, Chengdu, China.,Engineering Research Center of Human Robot Hybrid Intelligent Technologies and Systems, Ministry of Education, University of Electronic Science and Technology of China, Chengdu, China
| | - Xiao Yang
- Department of Orthopedics, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
18
|
Shi K, Huang R, Mu F, Peng Z, Yin J, Cheng H. Channel Synergy-based Human-Robot Interface for a Lower Limb Walking Assistance Exoskeleton. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2021; 2021:1076-1081. [PMID: 34891474 DOI: 10.1109/embc46164.2021.9631040] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
The human-robot interface (HRI) based on surface electromyography(sEMG) can realize the natural interaction between human and robot. It has been widely used in exoskeleton robots recently to help predict the wearer's movement. The sEMG signal of the paraplegic patients' lower limbs is weak. How to achieve accurate prediction of the lower limb movement of patients with paraplegia has always been the focus of attention in the field of HRI. Few studies have explored the possibility of using upper limb sEMG signals to predict lower limb movement. In addition, most HRIs do not consider the contribution and synergy of sEMG signal channels. This paper proposes a human-exoskeleton interface based on upper limb sEMG signals to predict lower limb movements of paraplegic patients. The interface constructs a channel synergy-based network (MCSNet) to extract the contribution and synergy of different feature channels. An sEMG data acquisition experiment is designed to verify the effectiveness of MCSNet. The experimental results show that our method has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 94.51% and 80.75% respectively.
Collapse
|
19
|
Kinematics, Speed, and Anthropometry-Based Ankle Joint Torque Estimation: A Deep Learning Regression Approach. MACHINES 2021. [DOI: 10.3390/machines9080154] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
Powered Assistive Devices (PADs) have been proposed to enable repetitive, user-oriented gait rehabilitation. They may include torque controllers that typically require reference joint torque trajectories to determine the most suitable level of assistance. However, a robust approach able to automatically estimate user-oriented reference joint torque trajectories, namely ankle torque, while considering the effects of varying walking speed, body mass, and height on the gait dynamics, is needed. This study evaluates the accuracy and generalization ability of two Deep Learning (DL) regressors (Long-Short Term Memory and Convolutional Neural Network (CNN)) to generate user-oriented reference ankle torque trajectories by innovatively customizing them according to the walking speed (ranging from 1.0 to 4.0 km/h) and users’ body height and mass (ranging from 1.51 to 1.83 m and 52.0 to 83.7 kg, respectively). Furthermore, this study hypothesizes that DL regressors can estimate joint torque without resourcing electromyography signals. CNN was the most robust algorithm (Normalized Root Mean Square Error: 0.70 ± 0.06; Spearman Correlation: 0.89 ± 0.03; Coefficient of Determination: 0.91 ± 0.03). No statistically significant differences were found in CNN accuracy (p-value > 0.05) whether electromyography signals are included as inputs or not, enabling a less obtrusive and accurate setup for torque estimation.
Collapse
|