1
|
Molnar J, Agrawal V, Chernova S. Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis. Front Robot AI 2024; 11:1330812. [PMID: 38654756 PMCID: PMC11035831 DOI: 10.3389/frobt.2024.1330812] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 03/18/2024] [Indexed: 04/26/2024] Open
Abstract
Successful operation of a teleoperated robot depends on a well-designed control scheme to translate human motion into robot motion; however, a single control scheme may not be suitable for all users. On the other hand, individual personalization of control schemes may be infeasible for designers to produce. In this paper, we present a method by which users may be classified into groups with mutually compatible control scheme preferences. Users are asked to demonstrate freehand motions to control a simulated robot in a virtual reality environment. Hand pose data is captured and compared with other users using SLAM trajectory similarity analysis techniques. The resulting pairwise trajectory error metrics are used to cluster participants based on their control motions, without foreknowledge of the number or types of control scheme preferences that may exist. The clusters identified for two different robots shows that a small number of clusters form stably for each case, each with its own control scheme paradigm. Survey data from participants validates that the clusters identified through this method correspond to the participants' control scheme rationales, and also identify nuances in participant control scheme descriptions that may not be obvious to designers relying only on participant explanations of their preferences.
Collapse
Affiliation(s)
- Jennifer Molnar
- Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA, United States
| | - Varun Agrawal
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, United States
| | - Sonia Chernova
- School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, United States
| |
Collapse
|
2
|
Teng Z, Xu G, Zhang X, Chen X, Zhang S, Huang HY. Concurrent and continuous estimation of multi-finger forces by synergy mapping and reconstruction: a pilot study. J Neural Eng 2023; 20:066024. [PMID: 38029436 DOI: 10.1088/1741-2552/ad10d1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 11/29/2023] [Indexed: 12/01/2023]
Abstract
Objective.The absence of intuitive control in present myoelectric interfaces makes it a challenge for users to communicate with assistive devices efficiently in real-world conditions. This study aims to tackle this difficulty by incorporating neurophysiological entities, namely muscle and force synergies, onto multi-finger force estimation to allow intuitive myoelectric control.Approach. Eleven healthy subjects performed six isometric grasping tasks at three muscle contraction levels. The exerted fingertip forces were collected concurrently with the surface electromyographic (sEMG) signals from six extrinsic and intrinsic muscles of hand. Muscle synergies were then extracted from recorded sEMG signals, while force synergies were identified from measured force data. Afterwards, a linear regressor was trained to associate the two types of synergies. This would allow us to predict multi-finger forces simply by multiplying the activation signals derived from muscle synergies with the weighting matrix of initially identified force synergies. To mitigate the false activation of unintended fingers, the force predictions were finally corrected by a finger state recognition procedure.Main results. We found that five muscle synergies and four force synergies are able to make a tradeoff between the computation load and the prediction accuracy for the proposed model; When trained and tested on all six grasping tasks, our method (SYN-II) achieved better performance (R2= 0.80 ± 0.04, NRMSE = 0.19 ± 0.01) than conventional sEMG amplitude-based method; Interestingly, SYN-II performed better than all other methods when tested on two unknown tasks outside the four training tasks (R2= 0.74 ± 0.03, NRMSE = 0.22 ± 0.02), which indicated better generalization ability.Significance. This study shows the first attempt to link between muscle and force synergies to allow concurrent and continuous estimation of multi-finger forces from sEMG. The proposed approach may lay the foundation for high-performance myoelectric interfaces that allow users to control robotic hands in a more natural and intuitive manner.
Collapse
Affiliation(s)
- Zhicheng Teng
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Guanghua Xu
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Xun Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Xiaobi Chen
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Sicong Zhang
- School of Mechanical Engineering, Xi'an Jiaotong University, Xi'an, People's Republic of China
| | - Hsien-Yung Huang
- Department of Bioengineering, Imperial College London, London, United Kingdom
| |
Collapse
|
3
|
Mereu F, Morosato F, Cordella F, Zollo L, Gruppioni E. Exploring the EMG transient: the muscular activation sequences used as novel time-domain features for hand gestures classification. Front Neurorobot 2023; 17:1264802. [PMID: 38023447 PMCID: PMC10667427 DOI: 10.3389/fnbot.2023.1264802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Accepted: 10/26/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction Muscular activation sequences have been shown to be suitable time-domain features for classification of motion gestures. However, their clinical application in myoelectric prosthesis control was never investigated so far. The aim of the paper is to evaluate the robustness of these features extracted from the EMG signal in transient state, on the forearm, for classifying common hand tasks. Methods The signal associated to four hand gestures and the rest condition were acquired from ten healthy people and two persons with trans-radial amputation. A feature extraction algorithm allowed for encoding the EMG signals into muscular activation sequences, which were used to train four commonly used classifiers, namely Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Non-linear Logistic Regression (NLR) and Artificial Neural Network (ANN). The offline performances were assessed with the entire sample of recruited people. The online performances were assessed with the amputee subjects. Moreover, a comparison of the proposed method with approaches based on the signal envelope in the transient state and in the steady state was conducted. Results The highest performance were obtained with the NLR classifier. Using the sequences, the offline classification accuracy was higher than 93% for healthy and amputee subjects and always higher than the approach with the signal envelope in transient state. As regards the comparison with the steady state, the performances obtained with the proposed method are slightly lower (<4%), but the classification occurred at least 200 ms earlier. In the online application, the motion completion rate reached up to 85% of the total classification attempts, with a motion selection time that never exceeded 218 ms. Discussion Muscular activation sequences are suitable alternatives to the time-domain features commonly used in classification problems belonging to the sole EMG transient state and could be potentially exploited in control strategies of myoelectric prosthesis hands.
Collapse
Affiliation(s)
- Federico Mereu
- Centro Protesi Inail, Vigorso di Budrio, Bologna, Italy
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, Rome, Italy
| | | | - Francesca Cordella
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, Rome, Italy
| | - Loredana Zollo
- Unit of Advanced Robotics and Human-Centred Technologies, Università Campus Bio-Medico di Roma, Rome, Italy
| | | |
Collapse
|
4
|
Brambilla C, Atzori M, Müller H, d'Avella A, Scano A. Spatial and Temporal Muscle Synergies Provide a Dual Characterization of Low-dimensional and Intermittent Control of Upper-limb Movements. Neuroscience 2023; 514:100-122. [PMID: 36708799 DOI: 10.1016/j.neuroscience.2023.01.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2022] [Revised: 01/16/2023] [Accepted: 01/18/2023] [Indexed: 01/27/2023]
Abstract
Muscle synergy analysis investigates the neurophysiological mechanisms that the central nervous system employs to coordinate muscles. Several models have been developed to decompose electromyographic (EMG) signals into spatial and temporal synergies. However, using multiple approaches can complicate the interpretation of results. Spatial synergies represent invariant muscle weights modulated with variant temporal coefficients; temporal synergies are invariant temporal profiles that coordinate variant muscle weights. While non-negative matrix factorization allows to extract both spatial and temporal synergies, the comparison between the two approaches was rarely investigated targeting a large set of multi-joint upper-limb movements. Spatial and temporal synergies were extracted from two datasets with proximal (16 subjects, 10M, 6F) and distal upper-limb movements (30 subjects, 21M, 9F), focusing on their differences in reconstruction accuracy and inter-individual variability. We showed the existence of both spatial and temporal structure in the EMG data, comparing synergies with those from a surrogate dataset in which the phases were shuffled preserving the frequency content of the original data. The two models provide a compact characterization of motor coordination at the spatial or temporal level, respectively. However, a lower number of temporal synergies are needed to achieve the same reconstruction R2: spatial and temporal synergies may capture different hierarchical levels of motor control and are dual approaches to the characterization of low-dimensional coordination of the upper-limb. Last, a detailed characterization of the structure of the temporal synergies suggested that they can be related to intermittent control of the movement, allowing high flexibility and dexterity. These results improve neurophysiology understanding in several fields such as motor control, rehabilitation, and prosthetics.
Collapse
Affiliation(s)
- Cristina Brambilla
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), Lecco, Italy
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), CH-3960 Sierre, Switzerland; Department of Neuroscience, University of Padova, via Belzoni 160, 35121 Padova, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO Valais), CH-3960 Sierre, Switzerland; Medical Informatics, University of Geneva, Rue Gabrielle-Perret-Gentil 4, 1205 Geneva, Switzerland
| | - Andrea d'Avella
- Laboratory of Neuromotor Physiology, IRCCS Fondazione Santa Lucia, Rome, Italy; Department of Biomedical and Dental Sciences and Morphofunctional Imaging, University of Messina, Messina, Italy.
| | - Alessandro Scano
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), Lecco, Italy.
| |
Collapse
|
5
|
Brambilla C, Scano A. The Number and Structure of Muscle Synergies Depend on the Number of Recorded Muscles: A Pilot Simulation Study with OpenSim. SENSORS (BASEL, SWITZERLAND) 2022; 22:8584. [PMID: 36433182 PMCID: PMC9694016 DOI: 10.3390/s22228584] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 11/02/2022] [Accepted: 11/04/2022] [Indexed: 06/16/2023]
Abstract
The muscle synergy approach is used to evaluate motor control and to quantitatively determine the number and structure of the modules underlying movement. In experimental studies regarding the upper limb, typically 8 to 16 EMG probes are used depending on the application, although the number of muscles involved in motor generation is higher. Therefore, the number of motor modules may be underestimated and the structure altered with the standard spatial synergy model based on the non-negative matrix factorization (NMF). In this study, we compared the number and structure of muscle synergies when considering 12 muscles (an "average" condition that represents previous studies) and 32 muscles of the upper limb, also including multiple muscle heads and deep muscles. First, we estimated the muscle activations with an upper-limb model in OpenSim using data from multi-directional reaching movements acquired in experimental sessions; then, spatial synergies were extracted from EMG activations from 12 muscles and from 32 muscles and their structures were compared. Finally, we compared muscle synergies obtained from OpenSim and from real experimental EMG signals to assess the reliability of the results. Interestingly, we found that on average, an additional synergy is needed to reconstruct the same R2 level with 32 muscles with respect to 12 muscles; synergies have a very similar structure, although muscles with comparable physiological functions were added to the synergies extracted with 12 muscles. The additional synergies, instead, captured patterns that could not be identified with only 12 muscles. We concluded that current studies may slightly underestimate the number of controlled synergies, even though the main structure of synergies is not modified when adding more muscles. We also show that EMG activations estimated with OpenSim are in partial (but not complete) agreement with experimental recordings. These findings may have significative implications for motor control and clinical studies.
Collapse
Affiliation(s)
- Cristina Brambilla
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), 23900 Lecco, Italy
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), 20133 Milan, Italy
| | - Alessandro Scano
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), 23900 Lecco, Italy
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), Italian Council of National Research (CNR), 20133 Milan, Italy
| |
Collapse
|
6
|
Li Y, Wang P, Li R, Tao M, Liu Z, Qiao H. A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods. Front Neurorobot 2022; 16:843267. [PMID: 35574228 PMCID: PMC9097019 DOI: 10.3389/fnbot.2022.843267] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2021] [Accepted: 02/28/2022] [Indexed: 11/13/2022] Open
Abstract
Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.
Collapse
Affiliation(s)
- Yinlin Li
- State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Peng Wang
- State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences, Hong Kong, China
| | - Rui Li
- School of Automation, Chongqing University, Chongqing, China
| | - Mo Tao
- Science and Technology on Thermal Energy and Power Laboratory, Wuhan Second Ship Design and Research Institute, Wuhan, China
- School of Automation Science and Electrical Engineering, Beihang University, Beijing, China
| | - Zhiyong Liu
- State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Hong Qiao
- State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
7
|
Zhou Z, Sun J, Yu J, Liu K, Duan J, Chen L, Chen CLP. An Image-Based Benchmark Dataset and a Novel Object Detector for Water Surface Object Detection. Front Neurorobot 2021; 15:723336. [PMID: 34630064 PMCID: PMC8497741 DOI: 10.3389/fnbot.2021.723336] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Accepted: 08/24/2021] [Indexed: 11/13/2022] Open
Abstract
Water surface object detection is one of the most significant tasks in autonomous driving and water surface vision applications. To date, existing public large-scale datasets collected from websites do not focus on specific scenarios. As a characteristic of these datasets, the quantity of the images and instances is also still at a low level. To accelerate the development of water surface autonomous driving, this paper proposes a large-scale, high-quality annotated benchmark dataset, named Water Surface Object Detection Dataset (WSODD), to benchmark different water surface object detection algorithms. The proposed dataset consists of 7,467 water surface images in different water environments, climate conditions, and shooting times. In addition, the dataset comprises a total of 14 common object categories and 21,911 instances. Simultaneously, more specific scenarios are focused on in WSODD. In order to find a straightforward architecture to provide good performance on WSODD, a new object detector, named CRB-Net, is proposed to serve as a baseline. In experiments, CRB-Net was compared with 16 state-of-the-art object detection methods and outperformed all of them in terms of detection precision. In this paper, we further discuss the effect of the dataset diversity (e.g., instance size, lighting conditions), training set size, and dataset details (e.g., method of categorization). Cross-dataset validation shows that WSODD significantly outperforms other relevant datasets and that the adaptability of CRB-Net is excellent.
Collapse
Affiliation(s)
- Zhiguo Zhou
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
| | - Jiaen Sun
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
| | - Jiabao Yu
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
| | - Kaiyuan Liu
- School of Information and Electronics, Beijing Institute of Technology, Beijing, China
| | - Junwei Duan
- College of Information Science and Technology, Jinan University, Guangzhou, China
| | - Long Chen
- Faculty of Science and Technology, University of Macau, Taipa, Macau, SAR China
| | - C L Philip Chen
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
| |
Collapse
|
8
|
Averta G, Barontini F, Catrambone V, Haddadin S, Handjaras G, Held JPO, Hu T, Jakubowitz E, Kanzler CM, Kühn J, Lambercy O, Leo A, Obermeier A, Ricciardi E, Schwarz A, Valenza G, Bicchi A, Bianchi M. U-Limb: A multi-modal, multi-center database on arm motion control in healthy and post-stroke conditions. Gigascience 2021; 10:giab043. [PMID: 34143875 PMCID: PMC8212873 DOI: 10.1093/gigascience/giab043] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Revised: 04/26/2021] [Accepted: 05/14/2021] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Shedding light on the neuroscientific mechanisms of human upper limb motor control, in both healthy and disease conditions (e.g., after a stroke), can help to devise effective tools for a quantitative evaluation of the impaired conditions, and to properly inform the rehabilitative process. Furthermore, the design and control of mechatronic devices can also benefit from such neuroscientific outcomes, with important implications for assistive and rehabilitation robotics and advanced human-machine interaction. To reach these goals, we believe that an exhaustive data collection on human behavior is a mandatory step. For this reason, we release U-Limb, a large, multi-modal, multi-center data collection on human upper limb movements, with the aim of fostering trans-disciplinary cross-fertilization. CONTRIBUTION This collection of signals consists of data from 91 able-bodied and 65 post-stroke participants and is organized at 3 levels: (i) upper limb daily living activities, during which kinematic and physiological signals (electromyography, electro-encephalography, and electrocardiography) were recorded; (ii) force-kinematic behavior during precise manipulation tasks with a haptic device; and (iii) brain activity during hand control using functional magnetic resonance imaging.
Collapse
Affiliation(s)
- Giuseppe Averta
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
- Soft Robotics for Human Cooperation and Rehabilitation, Fondazione Istituto Italiano di Tecnologia, Via Morego 30, 16163 Genova, Italy
| | - Federica Barontini
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
- Soft Robotics for Human Cooperation and Rehabilitation, Fondazione Istituto Italiano di Tecnologia, Via Morego 30, 16163 Genova, Italy
| | - Vincenzo Catrambone
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
| | - Sami Haddadin
- RSI - Chair of Robotics and Systems Intelligence, Munich School of Robotics and Machine Intelligence, Technical University Munich (TUM), Heßstr. 134, 80797 München, Germany
| | - Giacomo Handjaras
- MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Piazza S. Francesco 19, 55100 Lucca, Italy
| | - Jeremia P O Held
- Division of Vascular Neurology and Neurorehabilitation, Department of Neurology, University of Zurich, Frauenklinikstrasse 26, 8006 Zürich, Switzerland
| | - Tingli Hu
- RSI - Chair of Robotics and Systems Intelligence, Munich School of Robotics and Machine Intelligence, Technical University Munich (TUM), Heßstr. 134, 80797 München, Germany
| | - Eike Jakubowitz
- Laboratory for Biomechanics and Biomaterials (LBB), Department of Orthopaedic Surgery, Hannover Medical School, L384, 30625 Hannover, Germany
| | - Christoph M Kanzler
- Rehabilitation Engineering Laboratory, Institute of Robotics and Intelligent Systems, Department of Health Sciences and Technology, CLA H 1.1 Tannenstrasse 3, 8092 Zurich, Switzerland
| | - Johannes Kühn
- RSI - Chair of Robotics and Systems Intelligence, Munich School of Robotics and Machine Intelligence, Technical University Munich (TUM), Heßstr. 134, 80797 München, Germany
| | - Olivier Lambercy
- Rehabilitation Engineering Laboratory, Institute of Robotics and Intelligent Systems, Department of Health Sciences and Technology, CLA H 1.1 Tannenstrasse 3, 8092 Zurich, Switzerland
| | - Andrea Leo
- MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Piazza S. Francesco 19, 55100 Lucca, Italy
| | - Alina Obermeier
- Laboratory for Biomechanics and Biomaterials (LBB), Department of Orthopaedic Surgery, Hannover Medical School, L384, 30625 Hannover, Germany
| | - Emiliano Ricciardi
- MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Piazza S. Francesco 19, 55100 Lucca, Italy
| | - Anne Schwarz
- Division of Vascular Neurology and Neurorehabilitation, Department of Neurology, University of Zurich, Frauenklinikstrasse 26, 8006 Zürich, Switzerland
| | - Gaetano Valenza
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
| | - Antonio Bicchi
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
- Soft Robotics for Human Cooperation and Rehabilitation, Fondazione Istituto Italiano di Tecnologia, Via Morego 30, 16163 Genova, Italy
| | - Matteo Bianchi
- Research Center “Enrico Piaggio” and Dipartimento di Ingegneria dell’Informazione, University of Pisa Largo Lucio Lazzarino 1, 56122 Pisa, Italy
| |
Collapse
|
9
|
A Novel Muscle Synergy Extraction Method Used for Motor Function Evaluation of Stroke Patients: A Pilot Study. SENSORS 2021; 21:s21113833. [PMID: 34205957 PMCID: PMC8199433 DOI: 10.3390/s21113833] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 05/21/2021] [Accepted: 05/28/2021] [Indexed: 12/12/2022]
Abstract
In this paper, we present a novel muscle synergy extraction method based on multivariate curve resolution–alternating least squares (MCR-ALS) to overcome the limitation of the nonnegative matrix factorization (NMF) method for extracting non-sparse muscle synergy, and we study its potential application for evaluating motor function of stroke survivors. Nonnegative matrix factorization (NMF) is the most widely used method for muscle synergy extraction. However, NMF is susceptible to components’ sparseness and usually provides inferior reliability, which significantly limits the promotion of muscle synergy. In this study, MCR-ALS was employed to extract muscle synergy from electromyography (EMG) data. Its performance was compared with two other matrix factorization algorithms, NMF and self-modeling mixture analysis (SMMA). Simulated data sets were utilized to explore the influences of the sparseness and noise on the extracted synergies. As a result, the synergies estimated by MCR-ALS were the most similar to true synergies as compared with SMMA and NMF. MCR-ALS was used to analyze the muscle synergy characteristics of upper limb movements performed by healthy (n = 11) and stroke (n = 5) subjects. The repeatability and intra-subject consistency were used to evaluate the performance of MCR-ALS. As a result, MCR-ALS provided much higher repeatability and intra-subject consistency as compared with NMF, which were important for the reliability of the motor function evaluation. The stroke subjects had lower intra-subject consistency and seemingly had more synergies as compared with the healthy subjects. Thus, MCR-ALS is a promising muscle synergy analysis method for motor function evaluation of stroke patients.
Collapse
|
10
|
Jarque-Bou NJ, Sancho-Bru JL, Vergara M. A Systematic Review of EMG Applications for the Characterization of Forearm and Hand Muscle Activity during Activities of Daily Living: Results, Challenges, and Open Issues. SENSORS 2021; 21:s21093035. [PMID: 33925928 PMCID: PMC8123433 DOI: 10.3390/s21093035] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 04/23/2021] [Accepted: 04/24/2021] [Indexed: 11/16/2022]
Abstract
The role of the hand is crucial for the performance of activities of daily living, thereby ensuring a full and autonomous life. Its motion is controlled by a complex musculoskeletal system of approximately 38 muscles. Therefore, measuring and interpreting the muscle activation signals that drive hand motion is of great importance in many scientific domains, such as neuroscience, rehabilitation, physiotherapy, robotics, prosthetics, and biomechanics. Electromyography (EMG) can be used to carry out the neuromuscular characterization, but it is cumbersome because of the complexity of the musculoskeletal system of the forearm and hand. This paper reviews the main studies in which EMG has been applied to characterize the muscle activity of the forearm and hand during activities of daily living, with special attention to muscle synergies, which are thought to be used by the nervous system to simplify the control of the numerous muscles by actuating them in task-relevant subgroups. The state of the art of the current results are presented, which may help to guide and foster progress in many scientific domains. Furthermore, the most important challenges and open issues are identified in order to achieve a better understanding of human hand behavior, improve rehabilitation protocols, more intuitive control of prostheses, and more realistic biomechanical models.
Collapse
|
11
|
Tieck JCV, Secker K, Kaiser J, Roennau A, Dillmann R. Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2020.3034067] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
12
|
Variability of Muscle Synergies in Hand Grasps: Analysis of Intra- and Inter-Session Data. SENSORS 2020; 20:s20154297. [PMID: 32752155 PMCID: PMC7435387 DOI: 10.3390/s20154297] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/01/2020] [Revised: 07/27/2020] [Accepted: 07/29/2020] [Indexed: 11/16/2022]
Abstract
Background. Muscle synergy analysis is an approach to understand the neurophysiological mechanisms behind the hypothesized ability of the Central Nervous System (CNS) to reduce the dimensionality of muscle control. The muscle synergy approach is also used to evaluate motor recovery and the evolution of the patients’ motor performance both in single-session and longitudinal studies. Synergy-based assessments are subject to various sources of variability: natural trial-by-trial variability of performed movements, intrinsic characteristics of subjects that change over time (e.g., recovery, adaptation, exercise, etc.), as well as experimental factors such as different electrode positioning. These sources of variability need to be quantified in order to resolve challenges for the application of muscle synergies in clinical environments. The objective of this study is to analyze the stability and similarity of extracted muscle synergies under the effect of factors that may induce variability, including inter- and intra-session variability within subjects and inter-subject variability differentiation. The analysis was performed using the comprehensive, publicly available hand grasp NinaPro Database, featuring surface electromyography (EMG) measures from two EMG electrode bracelets. Methods. Intra-session, inter-session, and inter-subject synergy stability was analyzed using the following measures: variance accounted for (VAF) and number of synergies (NoS) as measures of reconstruction stability quality and cosine similarity for comparison of spatial composition of extracted synergies. Moreover, an approach based on virtual electrode repositioning was applied to shed light on the influence of electrode position on inter-session synergy similarity. Results. Inter-session synergy similarity was significantly lower with respect to intra-session similarity, both considering coefficient of variation of VAF (approximately 0.2–15% for inter vs. approximately 0.1% to 2.5% for intra, depending on NoS) and coefficient of variation of NoS (approximately 6.5–14.5% for inter vs. approximately 3–3.5% for intra, depending on VAF) as well as synergy similarity (approximately 74–77% for inter vs. approximately 88–94% for intra, depending on the selected VAF). Virtual electrode repositioning revealed that a slightly different electrode position can lower similarity of synergies from the same session and can increase similarity between sessions. Finally, the similarity of inter-subject synergies has no significant difference from the similarity of inter-session synergies (both on average approximately 84–90% depending on selected VAF). Conclusion. Synergy similarity was lower in inter-session conditions with respect to intra-session. This finding should be considered when interpreting results from multi-session assessments. Lastly, electrode positioning might play an important role in the lower similarity of synergies over different sessions.
Collapse
|
13
|
Osuagwu BA, Timms S, Peachment R, Dowie S, Thrussell H, Cross S, Shirley R, Segura-Fragoso A, Taylor J. Home-based rehabilitation using a soft robotic hand glove device leads to improvement in hand function in people with chronic spinal cord injury:a pilot study. J Neuroeng Rehabil 2020; 17:40. [PMID: 32138780 PMCID: PMC7057671 DOI: 10.1186/s12984-020-00660-y] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2019] [Accepted: 02/13/2020] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Loss of hand function following high level spinal cord injury (SCI) is perceived as a high priority area for rehabilitation. Following discharge, it is often impractical for the specialist care centre to provide ongoing therapy for people living with chronic SCI at home, which can lead to further deterioration of hand function and a direct impact on an individual's capability to perform essential activities of daily living (ADL). OBJECTIVE This pilot study investigated the therapeutic effect of a self-administered home-based hand rehabilitation programme for people with cervical SCI using the soft extra muscle (SEM) Glove by Bioservo Technologies AB. METHODS Fifteen participants with chronic cervical motor incomplete (AIS C and D) SCI were recruited and provided with the glove device to use at home to complete a set task and perform their usual ADL for a minimum of 4 h a day for 12 weeks. Assessment was made at Week 0 (Initial), 6, 12 and 18 (6-week follow-up). The primary outcome measure was the Toronto Rehabilitation Institute hand function test (TRI-HFT), with secondary outcome measures including pinch dynamometry and the modified Ashworth scale. RESULTS The TRI-HFT demonstrated improvement in hand function at Week 6 of the therapy including improvement in object manipulation (58.3 ±3.2 to 66.9 ±1.8, p ≈ 0.01), and palmar grasp assessed as the length of the wooden bar that can be held using a pronated palmar grip (29.1 ±6.0 cm to 45.8 ±6.8 cm, p <0.01). A significant improvement in pinch strength, with reduced thumb muscle hypertonia was also detected. Improvements in function were present during the Week 12 assessment and also during the follow-up. CONCLUSIONS Self-administered rehabilitation using the SEM Glove is effective for improving and retaining gross and fine hand motor function for people living with chronic spinal cord injury at home. Retention of improved hand function suggests that an intensive activity-based rehabilitation programme in specific individuals is sufficient to improve long-term neuromuscular activity. Future studies should characterise the neuromuscular mechanism of action and the minimal rehabilitation programme necessary with the assistive device to improve ADL tasks following chronic cervical SCI. TRIAL REGISTRATION NUMBER Trial registration: ISRCTN, ISRCTN98677526, Registered 01/June/2017 - Retrospectively registered, http://www.isrctn.com/ISRCTN98677526.
Collapse
Affiliation(s)
- Bethel A.C. Osuagwu
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Sarah Timms
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Ruth Peachment
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Sarah Dowie
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Helen Thrussell
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Susan Cross
- National Spinal Injuries Centre, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | - Rebecca Shirley
- Bucks Healthcare Plastics, Stoke Mandeville Hospital, Mandeville Road, Aylesbury, HP21 8AL UK
| | | | - Julian Taylor
- Instituto de Ciencias de la Salud, Talavera de la Reina, Castilla-La Mancha, 45600 Spain
- Sensorimotor Function Group, Hospital Nacional de Parapléjicos, SESCAM, Toledo, 45071 Spain
| |
Collapse
|
14
|
Jarque-Bou NJ, Scano A, Atzori M, Müller H. Kinematic synergies of hand grasps: a comprehensive study on a large publicly available dataset. J Neuroeng Rehabil 2019; 16:63. [PMID: 31138257 PMCID: PMC6540541 DOI: 10.1186/s12984-019-0536-6] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2019] [Accepted: 05/14/2019] [Indexed: 11/29/2022] Open
Abstract
Background Hand grasp patterns require complex coordination. The reduction of the kinematic dimensionality is a key process to study the patterns underlying hand usage and grasping. It allows to define metrics for motor assessment and rehabilitation, to develop assistive devices and prosthesis control methods. Several studies were presented in this field but most of them targeted a limited number of subjects, they focused on postures rather than entire grasping movements and they did not perform separate analysis for the tasks and subjects, which can limit the impact on rehabilitation and assistive applications. This paper provides a comprehensive mapping of synergies from hand grasps targeting activities of daily living. It clarifies several current limits of the field and fosters the development of applications in rehabilitation and assistive robotics. Methods In this work, hand kinematic data of 77 subjects, performing up to 20 hand grasps, were acquired with a data glove (a 22-sensor CyberGlove II data glove) and analyzed. Principal Component Analysis (PCA) and hierarchical cluster analysis were used to extract and group kinematic synergies that summarize the coordination patterns available for hand grasps. Results Twelve synergies were found to account for > 80% of the overall variation. The first three synergies accounted for more than 50% of the total amount of variance and consisted of: the flexion and adduction of the Metacarpophalangeal joint (MCP) of fingers 3 to 5 (synergy #1), palmar arching and flexion of the wrist (synergy #2) and opposition of the thumb (synergy #3). Further synergies refine movements and have higher variability among subjects. Conclusion Kinematic synergies are extracted from a large number of subjects (77) and grasps related to activities of daily living (20). The number of motor modules required to perform the motor tasks is higher than what previously described. Twelve synergies are responsible for most of the variation in hand grasping. The first three are used as primary synergies, while the remaining ones target finer movements (e.g. independence of thumb and index finger). The results generalize the description of hand kinematics, better clarifying several limits of the field and fostering the development of applications in rehabilitation and assistive robotics.
Collapse
Affiliation(s)
- Néstor J Jarque-Bou
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Castellón de la Plana, Spain
| | - Alessandro Scano
- Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), National Research Council of Italy (CNR), Milan, Italy.,Institute of Intelligent Industrial Systems and Technologies for Advanced Manufacturing (STIIMA), National Research Council of Italy (CNR), Lecco, Italy
| | - Manfredo Atzori
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland. .,Medical Informatics, University of Geneva, Rue Gabrielle-Perret-Gentil 4, 1205, Geneva, Switzerland.
| |
Collapse
|
15
|
Dash A, Yadav A, Chauhan A, Lahiri U. Kinect-Assisted Performance-Sensitive Upper Limb Exercise Platform for Post-stroke Survivors. Front Neurosci 2019; 13:228. [PMID: 30967755 PMCID: PMC6438898 DOI: 10.3389/fnins.2019.00228] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2018] [Accepted: 02/26/2019] [Indexed: 11/23/2022] Open
Abstract
One's ability to use upper limbs is critical for performing activities of daily living necessary for enjoying quality community life. However, after stroke, such abilities becomes adversely affected and it often deprives one of their capability to perform tasks that need coordinated movement in the upper limbs. To address issues with upper limb dysfunction, patients typically undergo rehabilitative exercises. Given the high patient to doctor ratio particularly in developing countries like India, conventional rehabilitation with patients undergoing exercises under one-on-one therapist's supervision often becomes a challenge. Thus, investigators are exploring technology such as computer-based platforms coupled with cameras that can alleviate the need for the continuous presence of a therapist and can offer a powerful complementary tool in the hands of the clinicians. Such marker-based imaging systems used for rehabilitation can offer real-time processing and high accuracy of data. However, these systems often require dedicated lab space and high set-up time. Often this is very expensive and suffers from portability issues. Investigators have been exploring marker-less imaging techniques e.g., Kinect integrated computer-based graphical user interfaces in stroke-rehabilitation such as tracking one's limb movement during rehabilitation. In our present study, we have developed a Kinect-assisted computer-based system that offered Human Computer Interaction (HCI) tasks of varying challenge levels. Execution of the tasks required one to use reaching and coordination skills of the upper limbs. Also, the system was Performance-sensitive i.e., adaptive to the individualized residual movement ability of one's upper limb quantified in terms of task performance score. We tested for the usability of our system by exposing 15 healthy participants to our system. Subsequently, seven post-stroke patients interacted with our system over a few sessions spread over 2 weeks. Also, we studied patient's mean tonic activity corresponding to the HCI tasks as a possible indicator of one's post-stroke functional recovery suggesting its potential of our system to serve as a rehabilitation platform. Our results indicate the potential of such systems toward the improvement of task performance capability of post-stroke patients with possibilities of upper limb movement rehabilitation.
Collapse
Affiliation(s)
- Adyasha Dash
- Department of Electrical Engineering, Indian Institute of Technology Gandhinagar, Gandhinagar, India
| | - Anand Yadav
- Department of Electrical Engineering, Indian Institute of Technology Gandhinagar, Gandhinagar, India
| | - Anand Chauhan
- Department of Electrical Engineering, Indian Institute of Technology Gandhinagar, Gandhinagar, India
| | - Uttama Lahiri
- Department of Electrical Engineering, Indian Institute of Technology Gandhinagar, Gandhinagar, India
| |
Collapse
|
16
|
Bhatt N, SKM V. Posture similarity index: a method to compare hand postures in synergy space. PeerJ 2018; 6:e6078. [PMID: 30581672 PMCID: PMC6292379 DOI: 10.7717/peerj.6078] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2018] [Accepted: 11/06/2018] [Indexed: 11/20/2022] Open
Abstract
Background The human hand can perform a range of manipulation tasks, from holding a pen to holding a hammer. The central nervous system (CNS) uses different strategies in different manipulation tasks based on task requirements. Attempts to compare postures of the hand have been made for use in robotics and animation industries. In this study, we developed an index called the posture similarity index to quantify the similarity between two human hand postures. Methods Twelve right-handed volunteers performed 70 postures, and lifted and held 30 objects (total of 100 different postures, each performed five times). A 16-sensor electromagnetic tracking system captured the kinematics of individual finger phalanges (segments). We modeled the hand as a 21-DoF system and computed the corresponding joint angles. We used principal component analysis to extract kinematic synergies from this 21-DoF data. We developed a posture similarity index (PSI), that represents the similarity between posture in the synergy (Principal component) space. First, we tested the performance of this index using a synthetic dataset. After confirming that it performs well with the synthetic dataset, we used it to analyze the experimental data. Further, we used PSI to identify postures that are “representative” in the sense that they have a greater overlap (in synergy space) with a large number of postures. Results Our results confirmed that PSI is a relatively accurate index of similarity in synergy space both with synthetic data and real experimental data. Also, more special postures than common postures were found among “representative” postures. Conclusion We developed an index for comparing posture similarity in synergy space and demonstrated its utility by using synthetic dataset and experimental dataset. Besides, we found that “special” postures are actually “special” in the sense that there are more of them in the “representative” postures as identified by our posture similarity index.
Collapse
Affiliation(s)
- Nayan Bhatt
- Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai, India
| | - Varadhan SKM
- Department of Applied Mechanics, Indian Institute of Technology Madras, Chennai, India
| |
Collapse
|