1
|
Suo J, Liu Y, Wang J, Chen M, Wang K, Yang X, Yao K, Roy VAL, Yu X, Daoud WA, Liu N, Wang J, Wang Z, Li WJ. AI-Enabled Soft Sensing Array for Simultaneous Detection of Muscle Deformation and Mechanomyography for Metaverse Somatosensory Interaction. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2024; 11:e2305025. [PMID: 38376001 DOI: 10.1002/advs.202305025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/23/2023] [Revised: 10/25/2023] [Indexed: 02/21/2024]
Abstract
Motion recognition (MR)-based somatosensory interaction technology, which interprets user movements as input instructions, presents a natural approach for promoting human-computer interaction, a critical element for advancing metaverse applications. Herein, this work introduces a non-intrusive muscle-sensing wearable device, that in conjunction with machine learning, enables motion-control-based somatosensory interaction with metaverse avatars. To facilitate MR, the proposed device simultaneously detects muscle mechanical activities, including dynamic muscle shape changes and vibrational mechanomyogram signals, utilizing a flexible 16-channel pressure sensor array (weighing ≈0.38 g). Leveraging the rich information from multiple channels, a recognition accuracy of ≈96.06% is achieved by classifying ten lower-limb motions executed by ten human subjects. In addition, this work demonstrates the practical application of muscle-sensing-based somatosensory interaction, using the proposed wearable device, for enabling the real-time control of avatars in a virtual space. This study provides an alternative approach to traditional rigid inertial measurement units and electromyography-based methods for achieving accurate human motion capture, which can further broaden the applications of motion-interactive wearable devices for the coming metaverse age.
Collapse
Affiliation(s)
- Jiao Suo
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Yifan Liu
- Dept. of Electrical and Computer Engineering, Michigan State University, MI, 48840, USA
| | - Jianfei Wang
- The Int. Research Centre for Nano Handling and Manufacturing of China, Changchun University of Science and Technology, Changchun, 130022, China
| | - Meng Chen
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Keer Wang
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Xiaomeng Yang
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Kuanming Yao
- Dept. of Biomedical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Vellaisamy A L Roy
- James Watt School of Engineering, University of Glasgow, Scotland, G12 8QQ, UK
| | - Xinge Yu
- Dept. of Biomedical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Walid A Daoud
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| | - Na Liu
- Sch. of Mechatronic Engineering and Automation, Shanghai University, Shanghai, 200444, China
| | - Jianping Wang
- Dept. of Computer Science, City University of Hong Kong, Hong Kong, 999077, China
| | - Zuobin Wang
- The Int. Research Centre for Nano Handling and Manufacturing of China, Changchun University of Science and Technology, Changchun, 130022, China
| | - Wen Jung Li
- Dept. of Mechanical Engineering, City University of Hong Kong, Hong Kong, 999077, China
| |
Collapse
|
2
|
Laubscher CA, Farris RJ, van den Bogert AJ, Sawicki JT. An Anthropometrically Parameterized Assistive Lower Limb Exoskeleton. J Biomech Eng 2021; 143:1109463. [PMID: 34008845 DOI: 10.1115/1.4051214] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2020] [Indexed: 01/04/2023]
Abstract
This paper presents an innovative design methodology for development of lower limb exoskeletons with the fabrication and experimental evaluation of prototype hardware. The proposed design approach is specifically conceived to be suitable for the pediatric population and uses additive manufacturing and a model parameterized in terms of subject anthropometrics to give a person-specific custom fit. The methodology is applied to create computer-aided design models using average anthropometrics of children 6-11 years old and using anthropometrics of an individual measured by the researchers. This demonstrates that the approach can scale to subject weight and height. A prototype exoskeleton is fabricated, which can actuate the hip and knee joints without restricting hip abduction-adduction motion. In order to test usability of the device and evaluate walking assistance, user effort is quantified in an assisted condition where the subject walks on a level treadmill with the exoskeleton powered. This is compared to an unassisted condition with the exoskeleton unpowered and a baseline condition with the subject not wearing the exoskeleton. Comparing assisted to baseline conditions, torque magnitudes increased at the hip and knee, mechanical energy generated increased at the hip but decreased at the knee, and muscle activations increased in the Vastus Lateralis but decreased in the Biceps Femoris. While the preliminary evidence for walking assistance is not entirely convincing for the tested conditions, the presented design methodology itself is promising as it has been successfully validated through the creation of computer-aided design models for children and fabrication of a serviceable exoskeleton prototype.
Collapse
Affiliation(s)
- Curt A Laubscher
- Department of Mechanical Engineering, Cleveland State University, Center for Rotating Machinery Dynamics and Control, Cleveland, OH 44115
| | - Ryan J Farris
- Human Motion and Control Division, Parker Hannifin Corporation, Macedonia, OH 44056
| | - Antonie J van den Bogert
- Department of Mechanical Engineering, Cleveland State University, Center for Human-Machine Systems, Cleveland, OH 44115
| | - Jerzy T Sawicki
- Department of Mechanical Engineering, Cleveland State University, Center for Rotating Machinery Dynamics and Control, Cleveland, OH 44115
| |
Collapse
|
3
|
Castillo CSM, Wilson S, Vaidyanathan R, Atashzar SF. Wearable MMG-Plus-One Armband: Evaluation of Normal Force on Mechanomyography (MMG) to Enhance Human-Machine Interfacing. IEEE Trans Neural Syst Rehabil Eng 2020; 29:196-205. [PMID: 33290226 DOI: 10.1109/tnsre.2020.3043368] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In this paper, we introduce a new mode of mechanomyography (MMG) signal capture for enhancing the performance of human-machine interfaces (HMIs) through modulation of normal pressure at the sensor location. Utilizing this novel approach, increased MMG signal resolution is enabled by a tunable degree of freedom normal to the sensor-skin contact area. We detail the mechatronic design, experimental validation, and user study of an armband with embedded acoustic sensors demonstrating this capacity. The design is motivated by the nonlinear viscoelasticity of the tissue, which increases with the normal surface pressure. This, in theory, results in higher conductivity of mechanical waves and hypothetically allows to interface with deeper muscle; thus, enhancing the discriminative information context of the signal space. Ten subjects (seven able-bodied and three trans-radial amputees) participated in a study consisting of the classification of hand gestures through MMG while increasing levels of contact force were administered. Four MMG channels were positioned around the forearm and placed over the flexor carpi radialis, brachioradialis, extensor digitorum communis, and flexor carpi ulnaris muscles. A total of 852 spectrotemporal features were extracted (213 features per each channel) and passed through a Neighborhood Component Analysis (NCA) technique to select the most informative neurophysiological subspace of the features for classification. A linear support vector machine (SVM) then classified the intended motion of the user. The results indicate that increasing the normal force level between the MMG sensor and the skin can improve the discriminative power of the classifier, and the corresponding pattern can be user-specific. These results have significant implications enabling embedding MMG sensors in sockets for prosthetic limb control and HMI.
Collapse
|
4
|
Segmenting Mechanomyography Measures of Muscle Activity Phases Using Inertial Data. Sci Rep 2019; 9:5569. [PMID: 30944380 PMCID: PMC6447582 DOI: 10.1038/s41598-019-41860-4] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2018] [Accepted: 03/18/2019] [Indexed: 11/21/2022] Open
Abstract
Electromyography (EMG) is the standard technology for monitoring muscle activity in laboratory environments, either using surface electrodes or fine wire electrodes inserted into the muscle. Due to limitations such as cost, complexity, and technical factors, including skin impedance with surface EMG and the invasive nature of fine wire electrodes, EMG is impractical for use outside of a laboratory environment. Mechanomyography (MMG) is an alternative to EMG, which shows promise in pervasive applications. The present study used an exerting squat-based task to induce muscle fatigue. MMG and EMG amplitude and frequency were compared before, during, and after the squatting task. Combining MMG with inertial measurement unit (IMU) data enabled segmentation of muscle activity at specific points: entering, holding, and exiting the squat. Results show MMG measures of muscle activity were similar to EMG in timing, duration, and magnitude during the fatigue task. The size, cost, unobtrusive nature, and usability of the MMG/IMU technology used, paired with the similar results compared to EMG, suggest that such a system could be suitable in uncontrolled natural environments such as within the home.
Collapse
|
5
|
Sanford J, Patterson R, Popa DO. Concurrent surface electromyography and force myography classification during times of prosthetic socket shift and user fatigue. J Rehabil Assist Technol Eng 2017; 4:2055668317708731. [PMID: 31186928 PMCID: PMC6453103 DOI: 10.1177/2055668317708731] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2016] [Accepted: 04/03/2017] [Indexed: 11/15/2022] Open
Abstract
Objective Surface electromyography has been a long-standing source of signals for control of powered prosthetic devices. By contrast, force myography is a more recent alternative to surface electromyography that has the potential to enhance reliability and avoid operational challenges of surface electromyography during use. In this paper, we report on experiments conducted to assess improvements in classification of surface electromyography signals through the addition of collocated force myography consisting of piezo-resistive sensors. Methods Force sensors detect intrasocket pressure changes upon muscle activation due to changes in muscle volume during activities of daily living. A heterogeneous sensor configuration with four surface electromyography-force myography pairs was investigated as a control input for a powered upper limb prosthetic. Training of two different multilevel neural perceptron networks was employed during classification and trained on data gathered during experiments simulating socket shift and muscle fatigue. Results Results indicate that intrasocket pressure data used in conjunction with surface EMG data can improve classification of human intent and control of a powered prosthetic device compared to traditional, surface electromyography only systems. Significance Additional sensors lead to significantly better signal classification during times of user fatigue, poor socket fit, as well as radial and ulnar wrist deviation. Results from experimentally obtained training data sets are presented.
Collapse
Affiliation(s)
- Joe Sanford
- Next Gen Systems Group, Department of Electrical Engineering, University of Texas-Arlington, Arlington, TX, USA
| | - Rita Patterson
- Department of Family and Osteopathic Manipulative Medicine, University of North Texas-Health Science Center, Fort Worth, TX, USA
| | - Dan O Popa
- Department of Electrical and Computer Engineering, University of Louisville, Louisville, KY, USA
| |
Collapse
|
6
|
LEI KINFONG, CHENG SHIHCHUNG, LEE MINGYIH, LIN WENYEN. MEASUREMENT AND ESTIMATION OF MUSCLE CONTRACTION STRENGTH USING MECHANOMYOGRAPHY BASED ON ARTIFICIAL NEURAL NETWORK ALGORITHM. BIOMEDICAL ENGINEERING: APPLICATIONS, BASIS AND COMMUNICATIONS 2013. [DOI: 10.4015/s1016237213500208] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Muscle contraction strength estimation using mechanomyographic (MMG) signal is typically calculated by the root mean square (RMS) amplitude. Raw MMG signal is processed by rectification, low-pass filtering, and mapping. In this work, beside RMS amplitude, another significant parameter of MMG signal, i.e. frequency variance (VAR), is introduced and used for constructing an algorithm for estimating the muscle contraction strength. Seven participants produced isometric contractions about the elbow while MMG signal and generated torque (resultant of muscle contraction strength) of biceps brachii were recorded. We found that MMG RMS increased monotonously and VAR decreased under incremental voluntary contractions. Based on these results, a two-layer neural network was utilized for the model of estimating the muscle contraction strength from MMG RMS and VAR. Experimental evaluation was performed under constant posture and sinusoidal contractions at 0.5 Hz, 0.25 Hz, 0.125 Hz, and random frequency. The results of the proposed algorithm and MMG RMS linear mapping were also compared. The proposed algorithm has better accuracy than linear mapping for all contraction frequencies. The mean absolute error decreased 6% for the 0.5Hz contraction, 43% for 0.25 Hz contraction, 52% for 0.125 Hz contraction, and 30% for random frequency contraction.
Collapse
Affiliation(s)
- KIN FONG LEI
- Graduate Institute of Medical Mechatronics, Chang Gung University, Tao-Yuan, Taiwan
- Healthy Aging Research Center, Chang Gung University, Tao-Yuan, Taiwan
| | - SHIH-CHUNG CHENG
- Graduate Institute of Athletics Coaching Science, National Taiwan Sport University, Tao-Yuan, Taiwan
| | - MING-YIH LEE
- Graduate Institute of Medical Mechatronics, Chang Gung University, Tao-Yuan, Taiwan
- Healthy Aging Research Center, Chang Gung University, Tao-Yuan, Taiwan
| | - WEN-YEN LIN
- Healthy Aging Research Center, Chang Gung University, Tao-Yuan, Taiwan
- Department of Electrical Engineering, Chang Gung University, Tao-Yuan, Taiwan
| |
Collapse
|
7
|
Abdoli-Eramaki M, Damecour C, Christenson J, Stevenson J. The effect of perspiration on the sEMG amplitude and power spectrum. J Electromyogr Kinesiol 2012; 22:908-13. [DOI: 10.1016/j.jelekin.2012.04.009] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2011] [Revised: 04/16/2012] [Accepted: 04/16/2012] [Indexed: 10/28/2022] Open
|