51
|
Zanghieri M, Benatti S, Burrello A, Kartsch V, Conti F, Benini L. Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2020; 14:244-256. [PMID: 31831433 DOI: 10.1109/tbcas.2019.2959160] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Hand movement classification via surface electromyographic (sEMG) signal is a well-established approach for advanced Human-Computer Interaction. However, sEMG movement recognition has to deal with the long-term reliability of sEMG-based control, limited by the variability affecting the sEMG signal. Embedded solutions are affected by a recognition accuracy drop over time that makes them unsuitable for reliable gesture controller design. In this paper, we present a complete wearable-class embedded system for robust sEMG-based gesture recognition, based on Temporal Convolutional Networks (TCNs). Firstly, we developed a novel TCN topology (TEMPONet), and we tested our solution on a benchmark dataset (Ninapro), achieving 49.6% average accuracy, 7.8%, better than current State-Of-the-Art (SoA). Moreover, we designed an energy-efficient embedded platform based on GAP8, a novel 8-core IoT processor. Using our embedded platform, we collected a second 20-sessions dataset to validate the system on a setup which is representative of the final deployment. We obtain 93.7% average accuracy with the TCN, comparable with a SoA SVM approach (91.1%). Finally, we profiled the performance of the network implemented on GAP8 by using an 8-bit quantization strategy to fit the memory constraint of the processor. We reach a 4× lower memory footprint (460 kB) with a performance degradation of only 3% accuracy. We detailed the execution on the GAP8 platform, showing that the quantized network executes a single classification in 12.84 ms with a power envelope of 0.9 mJ, making it suitable for a long-lifetime wearable deployment.
Collapse
|
52
|
Yildiz KA, Shin AY, Kaufman KR. Interfaces with the peripheral nervous system for the control of a neuroprosthetic limb: a review. J Neuroeng Rehabil 2020; 17:43. [PMID: 32151268 PMCID: PMC7063740 DOI: 10.1186/s12984-020-00667-5] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2019] [Accepted: 02/17/2020] [Indexed: 12/22/2022] Open
Abstract
The field of prosthetics has been evolving and advancing over the past decade, as patients with missing extremities are expecting to control their prostheses in as normal a way as possible. Scientists have attempted to satisfy this expectation by designing a connection between the nervous system of the patient and the prosthetic limb, creating the field of neuroprosthetics. In this paper, we broadly review the techniques used to bridge the patient's peripheral nervous system to a prosthetic limb. First, we describe the electrical methods including myoelectric systems, surgical innovations and the role of nerve electrodes. We then describe non-electrical methods used alone or in combination with electrical methods. Design concerns from an engineering point of view are explored, and novel improvements to obtain a more stable interface are described. Finally, a critique of the methods with respect to their long-term impacts is provided. In this review, nerve electrodes are found to be one of the most promising interfaces in the future for intuitive user control. Clinical trials with larger patient populations, and for longer periods of time for certain interfaces, will help to evaluate the clinical application of nerve electrodes.
Collapse
Affiliation(s)
- Kadir A Yildiz
- Motion Analysis Laboratory, Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Alexander Y Shin
- Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Kenton R Kaufman
- Motion Analysis Laboratory, Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA.
- Motion Analysis Laboratory, W. Hall Wendel, Jr., Musculoskeletal Research, 200 First Street SW, Rochester, MN, 55905, USA.
| |
Collapse
|
53
|
Dai C, Hu X. Finger Joint Angle Estimation Based on Motoneuron Discharge Activities. IEEE J Biomed Health Inform 2020; 24:760-767. [DOI: 10.1109/jbhi.2019.2926307] [Citation(s) in RCA: 43] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
54
|
Segil JL, Kaliki R, Uellendahl J, Ff Weir RF. A Myoelectric Postural Control Algorithm for Persons With Transradial Amputations: A Consideration of Clinical Readiness. IEEE ROBOTICS & AUTOMATION MAGAZINE 2020; 27:77-86. [PMID: 32494115 PMCID: PMC7269158 DOI: 10.1109/mra.2019.2949688] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
BACKGROUND The bottleneck in upper limb prosthetic design is the myoelectric control algorithm. Here we studied the clinical readiness of the myoelectric postural control algorithm in a laboratory setting with two trans-radial amputees using a commercially available prosthetic limb system. TECHNIQUE The postural control algorithm was integrated into prosthetic limb systems using standard of care components. A comparison between a commercial state of the art system (the i-limb revolution state-based myoelectric controller) and the postural controller was performed with two people with trans-radial amputation using a self-contained prosthesis system. DISCUSSION The performance using the i-limb revolution state-based controller versus the postural controller was mixed based on the Southampton Hand Assessment Procedure. The SHAP scores indicate that the postural controller with i-limb revolution provided an average of 66% of hand function compared to an intact limb. Future work will study the advantages of the postural control algorithm in everyday use.
Collapse
Affiliation(s)
- Jacob L Segil
- Rocky Mountain Regional VA Medical Center and the Engineering Plus Program at the University of Colorado Boulder, Boulder CO, 80304
| | - Rahul Kaliki
- Infinite Biomedical Technologies, Baltimore, MD 21202
| | | | - Richard F Ff Weir
- Rocky Mountain Regional VA Medical Center and the University of Colorado Denver | Anschutz Medical Campus Aurora, CO 80045
| |
Collapse
|
55
|
Bumbaširević M, Lesic A, Palibrk T, Milovanovic D, Zoka M, Kravić-Stevović T, Raspopovic S. The current state of bionic limbs from the surgeon's viewpoint. EFORT Open Rev 2020; 5:65-72. [PMID: 32175092 PMCID: PMC7047902 DOI: 10.1302/2058-5241.5.180038] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
Amputations have a devastating impact on patients' health with consequent psychological distress, economic loss, difficult reintegration into society, and often low embodiment of standard prosthetic replacement.The main characteristic of bionic limbs is that they establish an interface between the biological residuum and an electronic device, providing not only motor control of prosthesis but also sensitive feedback.Bionic limbs can be classified into three main groups, according to the type of the tissue interfaced: nerve-transferred muscle interfacing (targeted muscular reinnervation), direct muscle interfacing and direct nerve interfacing.Targeted muscular reinnervation (TMR) involves the transfer of the remaining nerves of the amputated stump to the available muscles.With direct muscle interfacing, direct intramuscular implants record muscular contractions which are then wirelessly captured through a coil integrated in the socket to actuate prosthesis movement.The third group is the direct interfacing of the residual nerves using implantable electrodes that enable reception of electric signals from the prosthetic sensors. This can improve sensation in the phantom limb.The surgical procedure for electrode implantation consists of targeting the proximal nerve area, competently introducing, placing, and fixing the electrodes and cables, while retaining movement of the arm/leg and nerve, and avoiding excessive neural damage.Advantages of bionic limbs are: the improvement of sensation, improved reintegration/embodiment of the artificial limb, and better controllability. Cite this article: EFORT Open Rev 2020;5:65-72. DOI: 10.1302/2058-5241.5.180038.
Collapse
Affiliation(s)
- Marko Bumbaširević
- School of Medicine, University of Belgrade, Serbia
- University Clinic for Orthopaedic Surgery and Traumatology, Clinical Centre of Serbia, Serbia
| | - Aleksandar Lesic
- School of Medicine, University of Belgrade, Serbia
- University Clinic for Orthopaedic Surgery and Traumatology, Clinical Centre of Serbia, Serbia
| | - Tomislav Palibrk
- School of Medicine, University of Belgrade, Serbia
- University Clinic for Orthopaedic Surgery and Traumatology, Clinical Centre of Serbia, Serbia
| | - Darko Milovanovic
- School of Medicine, University of Belgrade, Serbia
- University Clinic for Orthopaedic Surgery and Traumatology, Clinical Centre of Serbia, Serbia
| | | | | | - Stanisa Raspopovic
- ETH Zürich, Department of Health Sciences and Technology, Institute for Robotics and Intelligent System, Zurich, Switzerland
| |
Collapse
|
56
|
Hajian G, Morin E, Etemad A. PCA-Based Channel Selection in High-Density EMG for Improving Force Estimation. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:652-655. [PMID: 31945982 DOI: 10.1109/embc.2019.8857118] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In this paper, a method for selecting channels to improve estimated force using fast orthogonal search (FOS) has been proposed. Surface electromyogram (sEMG) signals acquired from linear surface electrode arrays, placed on the long head and short head of biceps brachii, and brachioradialis during isometric contractions are used to estimate force induced at the wrist using the FOS algorithm. The method utilizes principle component analysis (PCA) in the frequency domain to select the channels with the highest contribution to the first principal component (PC). Our analysis demonstrates that our proposed method is capable of reducing the dimensionality of the data (the number of channels was reduced from 21 to 9) while improving the accuracy of the estimated force.
Collapse
|
57
|
Topalović I, Graovac S, Popović DB. EMG map image processing for recognition of fingers movement. J Electromyogr Kinesiol 2019; 49:102364. [PMID: 31654842 DOI: 10.1016/j.jelekin.2019.102364] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2019] [Revised: 10/06/2019] [Accepted: 10/08/2019] [Indexed: 10/25/2022] Open
Abstract
Electromyography (EMG) is the conventional noninvasive method for the estimation of muscle activities. We developed a new image processing method for the recognition of individual finger movements based on EMG maps. The maps were formed from the EMG recordings via an array electrode with 24 contacts connected to a multichannel wireless miniature digital amplifier. The task was to detect and quantify the high activity regions in the EMG maps in persons with no known motor impairment. The results show the temporal and spatial patterns within the images during well-defined finger movements. The average accuracy of the automatic recognition compared with the recognition by an expert clinician in persons involved in the tests was 97.87 ± 0.92%. The application of the technique is foreseen for control for an assistive system (hand prosthesis and exoskeleton) since the interface is wearable and the processing can be implemented on a microcomputer.
Collapse
Affiliation(s)
- Ivan Topalović
- Institute of Technical Sciences of SASA, Knez Mihailova 35/IV, Belgrade, Serbia.
| | - Stevica Graovac
- Faculty of Electrical Engineering, University of Belgrade, Bulevar kralja Aleksandra 73, Belgrade, Serbia
| | - Dejan B Popović
- Serbian Academy of Sciences and Arts (SASA), Knez Mihailova 35, Belgrade, Serbia; Aalborg University, Fredrik Bajers Vej 7, Aalborg, Denmark
| |
Collapse
|
58
|
Ravizza A, De Maria C, Di Pietro L, Sternini F, Audenino AL, Bignardi C. Comprehensive Review on Current and Future Regulatory Requirements on Wearable Sensors in Preclinical and Clinical Testing. Front Bioeng Biotechnol 2019; 7:313. [PMID: 31781554 PMCID: PMC6857326 DOI: 10.3389/fbioe.2019.00313] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Accepted: 10/23/2019] [Indexed: 11/13/2022] Open
Abstract
Medical devices are designed, tested, and placed on the market in a highly regulated environment. Wearable sensors are crucial components of various medical devices: design and validation of wearable sensors, if managed according to international standards, can foster innovation while respecting regulatory requirements. The purpose of this paper is to review the upcoming European Union (EU) Medical Device Regulations 2017/745 and 2017/746, the current and future International Electrotechnical Commission (IEC) and International Organization for Standardization (ISO) standards that set methods for design and validation of medical devices, with a focus on wearable sensors. Risk classification according to the regulation is described. The international standards IEC 62304, IEC 60601, ISO 14971, and ISO 13485 are reviewed to define regulatory restrictions during design, pre-clinical validation and clinical validation of devices that include wearable sensors as crucial components. This paper is not about any specific innovation but it is a toolbox for interpreting current and future regulatory restrictions; an integrated method for design planning, validation and clinical testing is proposed. Application of this method to design wearable sensors should be evaluated in the future in order to assess its potentially positive impact to fostering innovation and to ensure timely development.
Collapse
Affiliation(s)
| | - Carmelo De Maria
- Information Engineering Department, Research Center "Enrico Piaggio", University of Pisa, Pisa, Italy
| | - Licia Di Pietro
- Information Engineering Department, Research Center "Enrico Piaggio", University of Pisa, Pisa, Italy
| | - Federico Sternini
- PolitoBIOMed Lab, Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, Italy
| | - Alberto L Audenino
- PolitoBIOMed Lab, Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, Italy
| | - Cristina Bignardi
- PolitoBIOMed Lab, Department of Mechanical and Aerospace Engineering, Politecnico di Torino, Turin, Italy
| |
Collapse
|
59
|
Koch P, Phan H, Maass M, Katzberg F, Mertins A. Recurrent Neural Network Based Early Prediction of Future Hand Movements. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:4710-4713. [PMID: 30441401 DOI: 10.1109/embc.2018.8513145] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
This work focuses on a system for hand prostheses that can overcome the delay problem introduced by classical approaches while being reliable. The proposed approach based on a recurrent neural network enables us to incorporate the sequential nature of the surface electromyogram data and the proposed system can be used either for classification or early prediction of hand movements. Especially the latter is a key to a latency free steering of a prosthesis. The experiments conducted on the first three Ninapro databases reveal that the prediction up to 200 ms ahead in the future is possible without a significant drop in accuracy. Furthermore, for classification, our proposed approach outperforms the state of the art classifiers even though we used significantly shorter windows for feature extraction.
Collapse
|
60
|
Abstract
State-of-the-art high-end prostheses are electro-mechanically able to provide a great variety of movements. Nevertheless, in order to functionally replace a human limb, it is essential that each movement is properly controlled. This is the goal of prosthesis control, which has become a growing research field in the last decades, with the ultimate goal of reproducing biological limb control. Therefore, exploration and development of prosthesis control are crucial to improve many aspects of an amputee’s life. Nowadays, a large divergence between academia and industry has become evident in commercial systems. Although several studies propose more natural control systems with promising results, basic one degree of freedom (DoF), a control switching system is the most widely used option in industry because of simplicity, robustness and inertia. A few classification controlled prostheses have emerged in the last years but they are still a low percentage of the used ones. One of the factors that generate this situation is the lack of robustness of more advanced control algorithms in daily life activities outside of laboratory conditions. Because of this, research has shifted towards more functional prosthesis control. This work reviews the most recent literature in upper limb prosthetic control. It covers commonly used variants of possible biological inputs, its processing and translation to actual control, mostly focusing on electromyograms as well as the problems it will have to overcome in near future.
Collapse
|
61
|
Hagengruber A, Vogel J. Functional Tasks Performed by People with Severe Muscular Atrophy Using an sEMG Controlled Robotic Manipulator. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:1713-1718. [PMID: 30440725 DOI: 10.1109/embc.2018.8512703] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
For paralyzed people activities of daily living like eating or drinking are impossible without external assistance. Robotic assistance systems can give these people a part of their independence back. Especially if the operation with a joystick is not possible anymore due to a missing hand function, people need innovative interfaces to control assistive robots in 3D. Besides brain computer interfaces an approach based on surface electromyography (sEMG) can present an opportunity for people with a strong muscular atrophy. In this work we show that two people with proceeded spinal muscular atrophy can perform functional tasks using an sEMG controlled robotic manipulator. The interface provides a continuous control of three degrees of freedom of the endeffector of the robot. The performance was assessed with two clinical measures of upper limb functionality: the Box and Blocks Test and the Action Research Arm Test. Additionally, the participant could show that they can drink by themselves with the provided system.
Collapse
|
62
|
Wang Z, Fang Y, Li G, Liu H. Facilitate sEMG-Based Human–Machine Interaction Through Channel Optimization. INT J HUM ROBOT 2019. [DOI: 10.1142/s0219843619410019] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Electromyography (EMG) has been widely accepted to interact with prosthetic hands, but still limited to using few channels for the control of few degrees of freedom. The use of more channels can improve the controllability, but it also increases system’s complexity and reduces its wearability. It is yet clear if optimizely placing the EMG channel could provide a feasible solution to this challenge. This study customized a genetic algorithm to optimize the number of channels and its position on the forearm in inter-day hand gesture recognition scenario. Our experimental results demonstrate that optimally selected 14 channels out of 16 can reach a peak inter-day hand gesture recognition accuracy at 72.3%, and optimally selecting 9 and 11 channels would reduce the performance by 3% and 10%. The cross-validation results also demonstrate that the optimally selected EMG channels from five subjects also work on the rest of the subjects, improving the accuracies by 3.09% and 4.5% in 9- and 11-channel combination, respectively. In sum, this study demonstrates the feasibility of channel reduction through genetic algorithm, and preliminary proves the significance of EMG channel optimization for human–machine interaction.
Collapse
Affiliation(s)
- Zheng Wang
- College of Computer Science & Technology, Zhejiang University of Technology, 288 Liuhe Rd, Hangzhou 310023, P. R. China
| | - Yinfeng Fang
- School of Communication Engineering, Hangzhou Dianzi University, 1158, No. 2 Avenue, Xiasha, Hangzhou 310018, P. R. China
| | - Gongfa Li
- Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Institute of Precision Manufacturing, 947 Heping Avenue, Wuhan 430081, P. R. China
| | - Honghai Liu
- Intelligent Systems and Biomedical Robotics Group, School of Computing, University of Portsmouth, Portsmouth PO1 3HE, UK
- School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, P. R. China
| |
Collapse
|
63
|
Prosthetic hand control: A multidisciplinary review to identify strengths, shortcomings, and the future. Biomed Signal Process Control 2019. [DOI: 10.1016/j.bspc.2019.101588] [Citation(s) in RCA: 23] [Impact Index Per Article: 4.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
64
|
Ngan CGY, Kapsa RMI, Choong PFM. Strategies for neural control of prosthetic limbs: from electrode interfacing to 3D printing. MATERIALS (BASEL, SWITZERLAND) 2019; 12:E1927. [PMID: 31207952 PMCID: PMC6631966 DOI: 10.3390/ma12121927] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2019] [Revised: 06/03/2019] [Accepted: 06/12/2019] [Indexed: 01/28/2023]
Abstract
Limb amputation is a major cause of disability in our community, for which motorised prosthetic devices offer a return to function and independence. With the commercialisation and increasing availability of advanced motorised prosthetic technologies, there is a consumer need and clinical drive for intuitive user control. In this context, rapid additive fabrication/prototyping capacities and biofabrication protocols embrace a highly-personalised medicine doctrine that marries specific patient biology and anatomy to high-end prosthetic design, manufacture and functionality. Commercially-available prosthetic models utilise surface electrodes that are limited by their disconnect between mind and device. As such, alternative strategies of mind-prosthetic interfacing have been explored to purposefully drive the prosthetic limb. This review investigates mind to machine interfacing strategies, with a focus on the biological challenges of long-term harnessing of the user's cerebral commands to drive actuation/movement in electronic prostheses. It covers the limitations of skin, peripheral nerve and brain interfacing electrodes, and in particular the challenges of minimising the foreign-body response, as well as a new strategy of grafting muscle onto residual peripheral nerves. In conjunction, this review also investigates the applicability of additive tissue engineering at the nerve-electrode boundary, which has led to pioneering work in neural regeneration and bioelectrode development for applications at the neuroprosthetic interface.
Collapse
Affiliation(s)
- Catherine G Y Ngan
- Department of Surgery, The University of Melbourne, St Vincent's Hospital, Melbourne 3065, VIC, Australia.
- Biofab3D@ACMD, St Vincent's Hospital Melbourne, Melbourne 3065, VIC, Australia.
- ARC Centre of Excellence for Electromaterials Science, Intelligent Polymer Research Institute, Innovation Campus, University of Wollongong, Wollongong 2500, NSW, Australia.
| | - Rob M I Kapsa
- Biofab3D@ACMD, St Vincent's Hospital Melbourne, Melbourne 3065, VIC, Australia.
- ARC Centre of Excellence for Electromaterials Science, Intelligent Polymer Research Institute, Innovation Campus, University of Wollongong, Wollongong 2500, NSW, Australia.
- Department of Medicine, The University of Melbourne, Melbourne 3065, VIC, Australia.
- Department of Clinical Neurosciences, St Vincent's Hospital, Melbourne 3065, VIC, Australia.
| | - Peter F M Choong
- Department of Surgery, The University of Melbourne, St Vincent's Hospital, Melbourne 3065, VIC, Australia.
- Biofab3D@ACMD, St Vincent's Hospital Melbourne, Melbourne 3065, VIC, Australia.
- ARC Centre of Excellence for Electromaterials Science, Intelligent Polymer Research Institute, Innovation Campus, University of Wollongong, Wollongong 2500, NSW, Australia.
- Department of Orthopaedics, St Vincent's Hospital, Melbourne 3065, VIC, Australia.
| |
Collapse
|
65
|
Akhlaghi N, Dhawan A, Khan AA, Mukherjee B, Diao G, Truong C, Sikdar S. Sparsity Analysis of a Sonomyographic Muscle-Computer Interface. IEEE Trans Biomed Eng 2019; 67:688-696. [PMID: 31150331 DOI: 10.1109/tbme.2019.2919488] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Sonomyography has been shown to be a promising method for decoding volitional motor intent from analysis of ultrasound images of the forearm musculature. The objectives of this paper are to determine the optimal location for ultrasound transducer placement on the anterior forearm for imaging maximum muscle deformations during different hand motions, and to investigate the effect of using a sparse set of ultrasound scanlines for motion classification for ultrasound-based muscle-computer interfaces (MCIs). METHODS The optimal placement of the ultrasound transducer along the forearm was identified using freehand three-dimensional reconstructions of the muscle thickness during rest and motion completion. Based on the ultrasound images acquired from the optimally placed transducer, classification accuracy with equally spaced scanlines across the cross-sectional field of view was determined. Furthermore, the unique contribution of each scanline to class discrimination using Fisher criterion (FC) and mutual information (MI) with respect to motion discriminability was determined. RESULTS Experiments with five able-bodied subjects show that the maximum muscle deformation occurred between 40%-50% of the forearm length for multiple degrees-of-freedom. The average classification accuracy was 94% ± 6% with the entire 128-scanline image and 94% ± 5% with four equally spaced scanlines. However, no significant improvement in classification accuracy was observed with optimal scanline selection using FC and MI. CONCLUSION For an optimally placed transducer, a small subset of ultrasound scanlines can be used instead of a full imaging array without sacrificing performance in terms of classification accuracy for multiple degrees-of-freedom. SIGNIFICANCE The selection of a small subset of transducer elements can enable the reduction of computation, and simplification of the instrumentation and power consumption of wearable sonomyographic MCIs, particularly for rehabilitation and gesture recognition applications.
Collapse
|
66
|
Benatti S, Montagna F, Kartsch V, Rahimi A, Rossi D, Benini L. Online Learning and Classification of EMG-Based Gestures on a Parallel Ultra-Low Power Platform Using Hyperdimensional Computing. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2019; 13:516-528. [PMID: 31056519 DOI: 10.1109/tbcas.2019.2914476] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
This paper presents a wearable electromyographic gesture recognition system based on the hyperdimensional computing paradigm, running on a programmable parallel ultra-low-power (PULP) platform. The processing chain includes efficient on-chip training, which leads to a fully embedded implementation with no need to perform any offline training on a personal computer. The proposed solution has been tested on 10 subjects in a typical gesture recognition scenario achieving 85% average accuracy on 11 gestures recognition, which is aligned with the state-of-the-art, with the unique capability of performing online learning. Furthermore, by virtue of the hardware friendly algorithm and of the efficient PULP system-on-chip (Mr. Wolf) used for prototyping and evaluation, the energy budget required to run the learning part with 11 gestures is 10.04 mJ, and 83.2 μJ per classification. The system works with a average power consumption of 10.4 mW in classification, ensuring around 29 h of autonomy with a 100 mAh battery. Finally, the scalability of the system is explored by increasing the number of channels (up to 256 electrodes), demonstrating the suitability of our approach as universal, energy-efficient biopotential wearable recognition framework.
Collapse
|
67
|
Wearable and Flexible Textile Electrodes for Biopotential Signal Monitoring: A review. ELECTRONICS 2019. [DOI: 10.3390/electronics8050479] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Wearable electronics is a rapidly growing field that recently started to introduce successful commercial products into the consumer electronics market. Employment of biopotential signals in wearable systems as either biofeedbacks or control commands are expected to revolutionize many technologies including point of care health monitoring systems, rehabilitation devices, human–computer/machine interfaces (HCI/HMIs), and brain–computer interfaces (BCIs). Since electrodes are regarded as a decisive part of such products, they have been studied for almost a decade now, resulting in the emergence of textile electrodes. This study presents a systematic review of wearable textile electrodes in physiological signal monitoring, with discussions on the manufacturing of conductive textiles, metrics to assess their performance as electrodes, and an investigation of their application in the acquisition of critical biopotential signals for routine monitoring, assessment, and exploitation of cardiac (electrocardiography, ECG), neural (electroencephalography, EEG), muscular (electromyography, EMG), and ocular (electrooculography, EOG) functions.
Collapse
|
68
|
Ameri A, Akhaee MA, Scheme E, Englehart K. Regression convolutional neural network for improved simultaneous EMG control. J Neural Eng 2019; 16:036015. [DOI: 10.1088/1741-2552/ab0e2e] [Citation(s) in RCA: 68] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
|
69
|
A novel feature extraction method for machine learning based on surface electromyography from healthy brain. Neural Comput Appl 2019. [DOI: 10.1007/s00521-019-04147-3] [Citation(s) in RCA: 60] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
70
|
Luo X, Wu X, Chen L, Zhao Y, Zhang L, Li G, Hou W. Synergistic Myoelectrical Activities of Forearm Muscles Improving Robust Recognition of Multi-Fingered Gestures. SENSORS 2019; 19:s19030610. [PMID: 30717127 PMCID: PMC6387382 DOI: 10.3390/s19030610] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/12/2018] [Revised: 01/17/2019] [Accepted: 01/30/2019] [Indexed: 11/16/2022]
Abstract
Currently, surface electromyography (sEMG) features of the forearm multi-tendon muscles are widely used in gesture recognition, however, there are few investigations on the inherent physiological mechanism of muscle synergies. We aimed to study whether the muscle synergies could be used for gesture recognition. Five healthy participants executed five gestures of daily life (pinch, fist, open hand, grip, and extension) and the sEMG activity was acquired from six forearm muscles. A non-negative matrix factorization (NMF) algorithm was employed to decompose the pre-treated six-channel sEMG data to obtain the muscle synergy matrixes, in which the weights of each muscle channel determined the feature set for hand gesture classification. The results showed that the synergistic features of forearm muscles could be successfully clustered in the feature space, which enabled hand gestures to be recognized with high efficiency. By augmenting the number of participants, the mean recognition rate remained at more than 96% and reflected high robustness. We showed that muscle synergies can be well applied to gesture recognition.
Collapse
Affiliation(s)
- Xiuying Luo
- Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing 400044, China.
- Chongqing Engineering Research Center of Medical Electronics Technology, Chongqing 400044, China.
| | - Xiaoying Wu
- Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing 400044, China.
- Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing 400044, China.
| | - Lin Chen
- Chongqing Engineering Research Center of Medical Electronics Technology, Chongqing 400044, China.
- Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing 400044, China.
| | - Yun Zhao
- Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing 400044, China.
- Chongqing Engineering Research Center of Medical Electronics Technology, Chongqing 400044, China.
| | - Li Zhang
- Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing 400044, China.
- Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing 400044, China.
| | - Guanglin Li
- Key Lab of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China.
| | - Wensheng Hou
- Key Laboratory of Biorheological Science and Technology, Ministry of Education, Bioengineering College, Chongqing University, Chongqing 400044, China.
- Chongqing Key Laboratory of Artificial Intelligence and Service Robot Control Technology, Chongqing 400044, China.
- Collaborative Innovation Center for Brain Science, Chongqing University, Chongqing 400044, China.
| |
Collapse
|
71
|
Igual C, Igual J, Hahne JM, Parra LC. Adaptive Auto-Regressive Proportional Myoelectric Control. IEEE Trans Neural Syst Rehabil Eng 2019; 27:314-322. [PMID: 30676969 DOI: 10.1109/tnsre.2019.2894464] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
In proportional myographic control, one can control either position or velocity of movement. Here, we propose to use adaptive auto-regressive filters, so as to gradually adjust between the two. We implemented this in an adaptive system with closed-loop feedback, where both the user and the machine simultaneously attempt to follow a cursor on a 2-D arena. We tested this on 15 able-bodied and three limb-deficient participants using an eight-channel myoelectric armband. The human-machine pairs learn to perform smoother cursor movements with a larger range of motion when using the auto-regressive filters, as compared with our previous effortswithmoving-average filters. Importantly, the human-machine system converges to an approximate velocity control strategy resulting in faster and more accuratemovements with lessmuscle effort. The method is not specific tomyoelectriccontroland could be used equally well for motion control using high-dimensional signals from reinnervatedmuscles or direct brain recordings.
Collapse
|
72
|
Wu Y, Jiang D, Liu X, Bayford R, Demosthenous A. A Human-Machine Interface Using Electrical Impedance Tomography for Hand Prosthesis Control. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2018; 12:1322-1333. [PMID: 30371386 DOI: 10.1109/tbcas.2018.2878395] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
This paper presents a human-machine interface that establishes a link between the user and a hand prosthesis. It successfully uses electrical impedance tomography, a conventional bio-impedance imaging technique, using an array of electrodes contained in a wristband on the user's forearm. Using a high-performance analog front-end application specific integrated circuit (ASIC), the user's forearm inner bio-impedance redistribution is accurately assessed. These bio-signatures are strongly related to hand motions and using artificial neural networks, they can be learned so as to recognize the user's intention in real time for prosthesis operation. In this work, eleven hand motions are designed for prosthesis operation with a gesture switching enabled sub-grouping method. Experiments with five subjects show that the system can achieve 98.5% accuracy with a grouping of three gestures and an accuracy of 94.4% with two sets of five gestures. The ASIC comprises a current driver with common-mode reduction capability and a current feedback instrumentation amplifier (that occupy an area of 0.07 mm2). The ASIC operates from ±1.65 V power supplies and has a minimum bio-impedance sensitivity of 12.7 mΩp-p.
Collapse
|
73
|
Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration. Exp Brain Res 2018; 237:291-311. [PMID: 30506366 DOI: 10.1007/s00221-018-5441-x] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2017] [Accepted: 11/20/2018] [Indexed: 10/27/2022]
Abstract
The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.
Collapse
|
74
|
Jarque-Bou NJ, Vergara M, Sancho-Bru JL, Roda-Sales A, Gracia-Ibáñez V. Identification of forearm skin zones with similar muscle activation patterns during activities of daily living. J Neuroeng Rehabil 2018; 15:91. [PMID: 30373606 PMCID: PMC6206932 DOI: 10.1186/s12984-018-0437-0] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2018] [Accepted: 10/15/2018] [Indexed: 12/20/2022] Open
Abstract
Background A deeper knowledge of the activity of the forearm muscles during activities of daily living (ADL) could help to better understand the role of those muscles and allow clinicians to treat motor dysfunctions more effectively and thus improve patients’ ability to perform activities of daily living. Methods In this work, we recorded sEMG activity from 30 spots distributed over the skin of the whole forearm of six subjects during the performance of 21 representative simulated ADL from the Sollerman Hand Function Test. Functional principal component analysis and hierarchical cluster analysis (HCA) were used to identify forearm spots with similar muscle activation patterns. Results The best classification of spots with similar activity in simulated ADL consisted in seven muscular-anatomically coherent groups: (1) wrist flexion and ulnar deviation; (2) wrist flexion and radial deviation; (3) digit flexion; (4) thumb extension and abduction/adduction; (5) finger extension; (6) wrist extension and ulnar deviation; and (7) wrist extension and radial deviation. Conclusion The number of sEMG sensors could be reduced from 30 to 7 without losing any relevant information, using them as representative spots of the muscular activity of the forearm in simulated ADL. This may help to assess muscle function in rehabilitation while also simplifying the complexity of prosthesis control.
Collapse
Affiliation(s)
- Néstor J Jarque-Bou
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avinguda Vicent Sos Baynat, s/n., 12071, Castellón, Spain.
| | - Margarita Vergara
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avinguda Vicent Sos Baynat, s/n., 12071, Castellón, Spain
| | - Joaquín L Sancho-Bru
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avinguda Vicent Sos Baynat, s/n., 12071, Castellón, Spain
| | - Alba Roda-Sales
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avinguda Vicent Sos Baynat, s/n., 12071, Castellón, Spain
| | - Verónica Gracia-Ibáñez
- Department of Mechanical Engineering and Construction, Universitat Jaume I, Avinguda Vicent Sos Baynat, s/n., 12071, Castellón, Spain
| |
Collapse
|
75
|
Wan Y, Han Z, Zhong J, Chen G. Pattern recognition and bionic manipulator driving by surface electromyography signals using convolutional neural network. INT J ADV ROBOT SYST 2018. [DOI: 10.1177/1729881418802138] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
With the development of robotics, intelligent neuroprosthesis for amputees is more concerned. Research of robot controlling based on electrocardiogram, electromyography, and electroencephalogram is a hot spot. In medical research, electrode arrays are commonly used as sensors for surface electromyograms. Although these sensors collect more accurate data and sampling at higher frequencies, they have no advantage in terms of portability and ease of use. In recent years, there are also some small surface electromyography sensors for research. The portability of the sensor and the calculation speed of the calculation method directly affect the development of the bionic prosthesis. A consumer-grade surface electromyography device is selected as surface electromyography sensor in this study. We first proposed a data structure to convert raw surface electromyography signals from an array structure into a matrix structure (we called it surface electromyography graph). Then, a convolutional neural network was used to classify it. Discrete surface electromyography signals recorded from three persons 14 gestures (widely used in other research to evaluate the performance of classifier) have been applied to train the classifier and we get an accuracy of 97.27%. The impacts of different components used in convolutional neural network were tested with this data, and subsequently, the best results were selected to build the classifier used in this article. The NinaPro database 5 (one of the biggest surface electromyography data sets) was also used to evaluate our method, which comprises of hand movement data of 10 intact subjects with two myo armbands as sensors, and the classification accuracy increased by 13.76% on average when using double myo armbands and increased by 18.92% on average when using single myo armband. In order to driving the robot hand (bionic manipulator), a group of continuous surface electromyography signals was recorded to train the classifier, and an accuracy of 91.72% was acquired. We also used the same method to collect a set of surface electromyography data from a disabled with hand lost, then classified it using the abovementioned network and achieved an accuracy of 89.37%. Finally, the classifier was deployed to the microcontroller to drive the bionic manipulator, and the full video URL is given in the conclusion, with both the healthy man and the disabled tested with the bionic manipulator. The abovementioned results suggest that this method will help to facilitate the development and application of surface electromyography neuroprosthesis.
Collapse
Affiliation(s)
- Yuanfang Wan
- Beijing University of Chemical Technology, Beijing, China
| | - Zishan Han
- Beijing University of Chemical Technology, Beijing, China
| | - Jun Zhong
- Beijing Institute of Petrochemical Technology, Beijing, China
| | - Guohua Chen
- Beijing University of Chemical Technology, Beijing, China
| |
Collapse
|
76
|
Xu L, Chen X, Cao S, Zhang X, Chen X. A Fatigue Involved Modification Framework for Force Estimation in Fatiguing Contraction. IEEE Trans Neural Syst Rehabil Eng 2018; 26:2153-2164. [PMID: 30281465 DOI: 10.1109/tnsre.2018.2872554] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
To alleviate the negative impacts of muscle fatigue on a force estimation model, a modification framework taking use of fatigue index was put forward in this paper. Muscle force and surface electromyography were first collected using high-density electrode grid and dynamometer. Then, multi-step signal pre-processing and a nonnegative matrix factorization-based signal optimization were conducted, with fatigue indices being extracted in the same time. Next, a degree 4 polynomial fitting model was employed to undertake the training process, and the relationship between the generated model parameters and fatigue indices was built up. In the end, the parameter-index relationship was applied on different testing sets to complete fatigue-modified force estimation. Significant improvement was found in most testing cases across different sexes and ages. Relative decreases of 36.5%, 20.7%, and 20.4% in the percentage root mean square error were achieved by young males, young females, and elderly males. The proposed method can boost the performances of force estimation models, thereby contributing to the development of a variety of fields including biomechanical study, rehabilitation treatment, and prosthesis research.
Collapse
|
77
|
Kyranou I, Vijayakumar S, Erden MS. Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses. Front Neurorobot 2018; 12:58. [PMID: 30297994 PMCID: PMC6160857 DOI: 10.3389/fnbot.2018.00058] [Citation(s) in RCA: 34] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2018] [Accepted: 08/27/2018] [Indexed: 11/29/2022] Open
Abstract
Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.
Collapse
Affiliation(s)
- Iris Kyranou
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Informatics, Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, United Kingdom
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
| | - Sethu Vijayakumar
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Informatics, Institute of Perception, Action and Behaviour, University of Edinburgh, Edinburgh, United Kingdom
| | - Mustafa Suphi Erden
- Edinburgh Centre of Robotics, Edinburgh, United Kingdom
- School of Engineering and Physical Sciences, Heriot-Watt University, Edinburgh, United Kingdom
| |
Collapse
|
78
|
Ameri A, Akhaee MA, Scheme E, Englehart K. Real-time, simultaneous myoelectric control using a convolutional neural network. PLoS One 2018; 13:e0203835. [PMID: 30212573 PMCID: PMC6136764 DOI: 10.1371/journal.pone.0203835] [Citation(s) in RCA: 43] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2018] [Accepted: 08/28/2018] [Indexed: 11/18/2022] Open
Abstract
The evolution of deep learning techniques has been transformative as they have allowed complex mappings to be trained between control inputs and outputs without the need for feature engineering. In this work, a myoelectric control system based on convolutional neural networks (CNN) is proposed as a possible alternative to traditional approaches that rely on specifically designed features. This CNN-based system is validated using a real-time Fitts' law style target acquisition test requiring single and combined wrist motions. The performance of the proposed system is then compared to that of a standard support vector machine (SVM) based myoelectric system using a set of time-domain features. Despite the prevalence and demonstrated performance of these well-known features, no significant difference (p>0.05) was found between the two methods for any of the computed control metrics. This demonstrates the potential for automated learning approaches to extract complex and rich information from stochastic biological signals. This first evaluation of the usability of a CNN in a real-time myoelectric control environment provides a basis for further exploration.
Collapse
Affiliation(s)
- Ali Ameri
- Department of Biomedical Engineering, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Mohammad Ali Akhaee
- School of Electrical and Computer Engineering, University of Tehran, Tehran, Iran
| | - Erik Scheme
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| | - Kevin Englehart
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, NB, Canada
| |
Collapse
|
79
|
Chen X, Yuan Y, Cao S, Zhang X, Chen X. A SEMG-Force Estimation Framework Based on a Fast Orthogonal Search Method Coupled with Factorization Algorithms. SENSORS 2018; 18:s18072238. [PMID: 29997373 PMCID: PMC6069375 DOI: 10.3390/s18072238] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/23/2018] [Revised: 07/10/2018] [Accepted: 07/10/2018] [Indexed: 11/16/2022]
Abstract
A novel framework based on the fast orthogonal search (FOS) method coupled with factorization algorithms was proposed and implemented to realize high-accuracy muscle force estimation via surface electromyogram (SEMG). During static isometric elbow flexion, high-density SEMG (HD-SEMG) signals were recorded from upper arm muscles, and the generated elbow force was measured at the wrist. HD-SEMG signals were decomposed into time-invariant activation patterns and time-varying activation curves using three typical factorization algorithms including principal component analysis (PCA), independent component analysis (ICA), and nonnegative matrix factorization (NMF). The activation signal of the target muscle was obtained by summing the activation curves, and the FOS algorithm was used to create basis functions with activation signals and establish the force estimation model. Static isometric elbow flexion experiments at three target levels were performed on seven male subjects, and the force estimation performances were compared among three typical factorization algorithms as well as a conventional method for extracting the average signal envelope of all HD-SEMG channels (AVG-ENVLP method). The overall root mean square difference (RMSD) values between the measured forces and the estimated forces obtained by different methods were 11.79 ± 4.29% for AVG-ENVLP, 9.74 ± 3.77% for PCA, 9.59 ± 3.81% for ICA, and 9.51 ± 4.82% for NMF. The results demonstrated that, compared to the conventional AVG-ENVLP method, factorization algorithms could substantially improve the performance of force estimation. The FOS method coupled with factorization algorithms provides an effective way to estimate the combined force of multiple muscles and has potential value in the fields of sports biomechanics, gait analysis, prosthesis control strategy, and exoskeleton devices for assisted rehabilitation.
Collapse
Affiliation(s)
- Xiang Chen
- Department of Electronic Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| | - Yuan Yuan
- Department of Electronic Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| | - Shuai Cao
- Department of Electronic Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| | - Xu Zhang
- Department of Electronic Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| | - Xun Chen
- Department of Electronic Science and Technology, University of Science and Technology of China, Hefei 230027, China.
| |
Collapse
|
80
|
Huang C, Chen X, Cao S, Qiu B, Zhang X. An isometric muscle force estimation framework based on a high-density surface EMG array and an NMF algorithm. J Neural Eng 2018; 14:046005. [PMID: 28497771 DOI: 10.1088/1741-2552/aa63ba] [Citation(s) in RCA: 32] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE To realize accurate muscle force estimation, a novel framework is proposed in this paper which can extract the input of the prediction model from the appropriate activation area of the skeletal muscle. APPROACH Surface electromyographic (sEMG) signals from the biceps brachii muscle during isometric elbow flexion were collected with a high-density (HD) electrode grid (128 channels) and the external force at three contraction levels was measured at the wrist synchronously. The sEMG envelope matrix was factorized into a matrix of basis vectors with each column representing an activation pattern and a matrix of time-varying coefficients by a nonnegative matrix factorization (NMF) algorithm. The activation pattern with the highest activation intensity, which was defined as the sum of the absolute values of the time-varying coefficient curve, was considered as the major activation pattern, and its channels with high weighting factors were selected to extract the input activation signal of a force estimation model based on the polynomial fitting technique. MAIN RESULTS Compared with conventional methods using the whole channels of the grid, the proposed method could significantly improve the quality of force estimation and reduce the electrode number. SIGNIFICANCE The proposed method provides a way to find proper electrode placement for force estimation, which can be further employed in muscle heterogeneity analysis, myoelectric prostheses and the control of exoskeleton devices.
Collapse
Affiliation(s)
- Chengjun Huang
- Department of Electronic Science and Technology, University of Science and Technology of China (USTC), Hefei, People's Republic of China
| | | | | | | | | |
Collapse
|
81
|
He Y, Fukuda O, Bu N, Okumura H, Yamaguchi N. Surface EMG Pattern Recognition Using Long Short-Term Memory Combined with Multilayer Perceptron. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2018; 2018:5636-5639. [PMID: 30441614 DOI: 10.1109/embc.2018.8513595] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Motion classification based on pattern recognition of surface EMG (sEMG) signals is a promising approach for prosthetic control. We present a pattern recognition model that combines long short-term memory (LSTM) network with multiplayer perceptron (MLP) for sEMG signals feature learning and classification. The LSTM network captures temporal dependencies of the sEMG signals while the MLP has no inherent temporal dynamics but focuses on the static characteristics. The combination of the two networks would learn a feature space that contains both the dynamic and static information of the sEMG signals, which helps to improve the motion classification accuracy. The architecture of the proposed network was optimized by investigating the appropriate width and depth of the neural network as well as the dropout to achieve the best classification results. The performance of the proposed pattern recognition model was evaluated using Ninapro database. The results show that the proposed model can produce better classification accuracy than most of the well-known recognition techniques.
Collapse
|
82
|
Erickson JC, Hayes JA, Bustamante M, Joshi R, Rwagaju A, Paskaranandavadivel N, Angeli TR. Intsy: a low-cost, open-source, wireless multi-channel bioamplifier system. Physiol Meas 2018; 39:035008. [PMID: 29406314 DOI: 10.1088/1361-6579/aaad51] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
Abstract
OBJECTIVE Multi-channel electrical recordings of physiologically generated signals are common to a wide range of biomedical fields. The aim of this work was to develop, validate, and demonstrate the practical utility of a high-quality, low-cost 32/64-channel bioamplifier system with real-time wireless data streaming capability. APPROACH The new 'Intsy' system integrates three main off-the-shelf hardware components: (1) Intan RHD2132 bioamplifier; (2) Teensy 3.2 microcontroller; and (3) RN-42 Bluetooth 2.1 module with a custom LabView interface for real-time data streaming and visualization. Practical utility was validated by measuring serosal gastric slow waves and surface EMG on the forearm with various contraction force levels. Quantitative comparisons were made to a gold-standard commercial system (Biosemi ActiveTwo). MAIN RESULTS Intsy signal quality was quantitatively comparable to that of the ActiveTwo. Recorded slow wave signals had high SNR (24 ± 2.7 dB) and wavefront propagation was accurately mapped. EMG spike bursts were characterized by high SNR (⩾10 dB) and activation timing was readily identified. Stable data streaming rates achieved were 3.5 kS s-1 for wireless and 64 kS s-1 for USB-wired transmission. SIGNIFICANCE Intsy has the highest channel count of any existing open-source, wireless-enabled module. The flexibility, portability and low cost ($1300 for the 32-channel version, or $2500 for 64 channels) of this new hardware module reduce the entry barrier for a range of electrophysiological experiments, as are typical in the gastrointestinal (EGG), cardiac (ECG), neural (EEG), and neuromuscular (EMG) domains.
Collapse
Affiliation(s)
- Jonathan C Erickson
- Department of Physics and Engineering, Washington and Lee University, Lexington, VA 24450, United States of America
| | | | | | | | | | | | | |
Collapse
|
83
|
Palermo F, Cognolato M, Gijsberts A, Muller H, Caputo B, Atzori M. Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data. IEEE Int Conf Rehabil Robot 2018; 2017:1154-1159. [PMID: 28813977 DOI: 10.1109/icorr.2017.8009405] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Control methods based on sEMG obtained promising results for hand prosthetics. Control system robustness is still often inadequate and does not allow the amputees to perform a large number of movements useful for everyday life. Only few studies analyzed the repeatability of sEMG classification of hand grasps. The main goals of this paper are to explore repeatability in sEMG data and to release a repeatability database with the recorded experiments. The data are recorded from 10 intact subjects repeating 7 grasps 12 times, twice a day for 5 days. The data are publicly available on the Ninapro web page. The analysis for the repeatability is based on the comparison of movement classification accuracy in several data acquisitions and for different subjects. The analysis is performed using mean absolute value and waveform length features and a Random Forest classifier. The accuracy obtained by training and testing on acquisitions at different times is on average 27.03% lower than training and testing on the same acquisition. The results obtained by training and testing on different acquisitions suggest that previous acquisitions can be used to train the classification algorithms. The inter-subject variability is remarkable, suggesting that specific characteristics of the subjects can affect repeatibility and sEMG classification accuracy. In conclusion, the results of this paper can contribute to develop more robust control systems for hand prostheses, while the presented data allows researchers to test repeatability in further analyses.
Collapse
|
84
|
Fang Y, Zhou D, Li K, Liu H. Interface Prostheses With Classifier-Feedback-Based User Training. IEEE Trans Biomed Eng 2018; 64:2575-2583. [PMID: 28026744 DOI: 10.1109/tbme.2016.2641584] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.It is evident that user training significantly affects performance of pattern-recognition-based myoelectric prosthetic device control. Despite plausible classification accuracy on offline datasets, online accuracy usually suffers from the changes in physiological conditions and electrode displacement. The user ability in generating consistent electromyographic (EMG) patterns can be enhanced via proper user training strategies in order to improve online performance. This study proposes a clustering-feedback strategy that provides real-time feedback to users by means of a visualized online EMG signal input as well as the centroids of the training samples, whose dimensionality is reduced to minimal number by dimension reduction. Clustering feedback provides a criterion that guides users to adjust motion gestures and muscle contraction forces intentionally. The experiment results have demonstrated that hand motion recognition accuracy increases steadily along the progress of the clustering-feedback-based user training, while conventional classifier-feedback methods, i.e., label feedback, hardly achieve any improvement. The result concludes that the use of proper classifier feedback can accelerate the process of user training, and implies prosperous future for the amputees with limited or no experience in pattern-recognition-based prosthetic device manipulation.
Collapse
Affiliation(s)
- Yinfeng Fang
- Intelligent Systems and Biomedical Robotics Group, School of ComputingUniversity of Portsmouth
| | - Dalin Zhou
- Intelligent Systems and Biomedical Robotics Group, School of ComputingUniversity of Portsmouth
| | - Kairu Li
- Intelligent Systems and Biomedical Robotics Group, School of ComputingUniversity of Portsmouth
| | - Honghai Liu
- Intelligent Systems and Biomedical Robotics Group, School of ComputingUniversity of Portsmouth
| |
Collapse
|
85
|
Baldacchino T, Jacobs WR, Anderson SR, Worden K, Rowson J. Simultaneous Force Regression and Movement Classification of Fingers via Surface EMG within a Unified Bayesian Framework. Front Bioeng Biotechnol 2018; 6:13. [PMID: 29536005 PMCID: PMC5834453 DOI: 10.3389/fbioe.2018.00013] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2017] [Accepted: 01/23/2018] [Indexed: 11/13/2022] Open
Abstract
This contribution presents a novel methodology for myolectric-based control using surface electromyographic (sEMG) signals recorded during finger movements. A multivariate Bayesian mixture of experts (MoE) model is introduced which provides a powerful method for modeling force regression at the fingertips, while also performing finger movement classification as a by-product of the modeling algorithm. Bayesian inference of the model allows uncertainties to be naturally incorporated into the model structure. This method is tested using data from the publicly released NinaPro database which consists of sEMG recordings for 6 degree-of-freedom force activations for 40 intact subjects. The results demonstrate that the MoE model achieves similar performance compared to the benchmark set by the authors of NinaPro for finger force regression. Additionally, inherent to the Bayesian framework is the inclusion of uncertainty in the model parameters, naturally providing confidence bounds on the force regression predictions. Furthermore, the integrated clustering step allows a detailed investigation into classification of the finger movements, without incurring any extra computational effort. Subsequently, a systematic approach to assessing the importance of the number of electrodes needed for accurate control is performed via sensitivity analysis techniques. A slight degradation in regression performance is observed for a reduced number of electrodes, while classification performance is unaffected.
Collapse
Affiliation(s)
- Tara Baldacchino
- Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, United Kingdom
| | - William R. Jacobs
- Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, United Kingdom
| | - Sean R. Anderson
- Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, United Kingdom
| | - Keith Worden
- Dynamics Research Group, Department of Mechanical Engineering, University of Sheffield, Sheffield, United Kingdom
| | - Jennifer Rowson
- Dynamics Research Group, Department of Mechanical Engineering, University of Sheffield, Sheffield, United Kingdom
- Insigneo Institute for in silico Medicine, University of Sheffield, Sheffield, United Kingdom
| |
Collapse
|
86
|
Hwang HJ, Hahne JM, Müller KR. Real-time robustness evaluation of regression based myoelectric control against arm position change and donning/doffing. PLoS One 2017; 12:e0186318. [PMID: 29095846 PMCID: PMC5667774 DOI: 10.1371/journal.pone.0186318] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2017] [Accepted: 09/28/2017] [Indexed: 11/18/2022] Open
Abstract
There are some practical factors, such as arm position change and donning/doffing, which prevent robust myoelectric control. The objective of this study is to precisely characterize the impacts of the two representative factors on myoelectric controllability in practical control situations, thereby providing useful references that can be potentially used to find better solutions for clinically reliable myoelectric control. To this end, a real-time target acquisition task was performed by fourteen subjects including one individual with congenital upper-limb deficiency, where the impacts of arm position change, donning/doffing and a combination of both factors on control performance was systematically evaluated. The changes in online performance were examined with seven different performance metrics to comprehensively evaluate various aspects of myoelectric controllability. As a result, arm position change significantly affects offline prediction accuracy, but not online control performance due to real-time feedback, thereby showing no significant correlation between offline and online performance. Donning/doffing was still problematic in online control conditions. It was further observed that no benefit was attained when using a control model trained with multiple position data in terms of arm position change, and the degree of electrode shift caused by donning/doffing was not severely associated with the degree of performance loss under practical conditions (around 1 cm electrode shift). Since this study is the first to concurrently investigate the impacts of arm position change and donning/doffing in practical myoelectric control situations, all findings of this study provide new insights into robust myoelectric control with respect to arm position change and donning/doffing.
Collapse
Affiliation(s)
- Han-Jeong Hwang
- Department of Medical IT Convergence Engineering, Kumoh National Institute of Technology, Gyeongbuk-do, Gumi, Republic of Korea
- * E-mail: (HJH); (KRM)
| | - Janne Mathias Hahne
- Neurorehabilitaiton Systems Research Group, Department of Trauma Surgery, Orthopedic Surgery and Hand Surgery, Universiy Medical Center Goettingen, Goettingen, Germany
| | - Klaus-Robert Müller
- Machine Learning Group, Berlin Institute of Technology (TU Berlin), Berlin, Germany
- Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea
- Max Planck Institute for Informatics, Stuhlsatzenhausweg, Saarbrücken, Germany
- * E-mail: (HJH); (KRM)
| |
Collapse
|
87
|
Pizzolato S, Tagliapietra L, Cognolato M, Reggiani M, Müller H, Atzori M. Comparison of six electromyography acquisition setups on hand movement classification tasks. PLoS One 2017; 12:e0186132. [PMID: 29023548 PMCID: PMC5638457 DOI: 10.1371/journal.pone.0186132] [Citation(s) in RCA: 108] [Impact Index Per Article: 15.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2017] [Accepted: 09/26/2017] [Indexed: 11/26/2022] Open
Abstract
Hand prostheses controlled by surface electromyography are promising due to the non-invasive approach and the control capabilities offered by machine learning. Nevertheless, dexterous prostheses are still scarcely spread due to control difficulties, low robustness and often prohibitive costs. Several sEMG acquisition setups are now available, ranging in terms of costs between a few hundred and several thousand dollars. The objective of this paper is the relative comparison of six acquisition setups on an identical hand movement classification task, in order to help the researchers to choose the proper acquisition setup for their requirements. The acquisition setups are based on four different sEMG electrodes (including Otto Bock, Delsys Trigno, Cometa Wave + Dormo ECG and two Thalmic Myo armbands) and they were used to record more than 50 hand movements from intact subjects with a standardized acquisition protocol. The relative performance of the six sEMG acquisition setups is compared on 41 identical hand movements with a standardized feature extraction and data analysis pipeline aimed at performing hand movement classification. Comparable classification results are obtained with three acquisition setups including the Delsys Trigno, the Cometa Wave and the affordable setup composed of two Myo armbands. The results suggest that practical sEMG tests can be performed even when costs are relevant (e.g. in small laboratories, developing countries or use by children). All the presented datasets can be used for offline tests and their quality can easily be compared as the data sets are publicly available.
Collapse
Affiliation(s)
- Stefano Pizzolato
- Information Systems Institute at the University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
- Department of Management and Engineering, University of Padova, Padova, Italy
| | - Luca Tagliapietra
- Department of Management and Engineering, University of Padova, Padova, Italy
| | - Matteo Cognolato
- Information Systems Institute at the University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
- Rehabilitation Engineering Laboratory, Department of Health Sciences and Technology, ETH Zurich, Zurich, Switzerland
| | - Monica Reggiani
- Department of Management and Engineering, University of Padova, Padova, Italy
| | - Henning Müller
- Information Systems Institute at the University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
| | - Manfredo Atzori
- Information Systems Institute at the University of Applied Sciences Western Switzerland (HES-SO Valais), Sierre, Switzerland
- * E-mail:
| |
Collapse
|
88
|
Zhai X, Jelfs B, Chan RHM, Tin C. Self-Recalibrating Surface EMG Pattern Recognition for Neuroprosthesis Control Based on Convolutional Neural Network. Front Neurosci 2017; 11:379. [PMID: 28744189 PMCID: PMC5504564 DOI: 10.3389/fnins.2017.00379] [Citation(s) in RCA: 124] [Impact Index Per Article: 17.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2017] [Accepted: 06/19/2017] [Indexed: 11/21/2022] Open
Abstract
Hand movement classification based on surface electromyography (sEMG) pattern recognition is a promising approach for upper limb neuroprosthetic control. However, maintaining day-to-day performance is challenged by the non-stationary nature of sEMG in real-life operation. In this study, we propose a self-recalibrating classifier that can be automatically updated to maintain a stable performance over time without the need for user retraining. Our classifier is based on convolutional neural network (CNN) using short latency dimension-reduced sEMG spectrograms as inputs. The pretrained classifier is recalibrated routinely using a corrected version of the prediction results from recent testing sessions. Our proposed system was evaluated with the NinaPro database comprising of hand movement data of 40 intact and 11 amputee subjects. Our system was able to achieve ~10.18% (intact, 50 movement types) and ~2.99% (amputee, 10 movement types) increase in classification accuracy averaged over five testing sessions with respect to the unrecalibrated classifier. When compared with a support vector machine (SVM) classifier, our CNN-based system consistently showed higher absolute performance and larger improvement as well as more efficient training. These results suggest that the proposed system can be a useful tool to facilitate long-term adoption of prosthetics for amputees in real-life applications.
Collapse
Affiliation(s)
- Xiaolong Zhai
- Department of Mechanical and Biomedical Engineering, City University of Hong KongHong Kong, Hong Kong
| | - Beth Jelfs
- Department of Electronic Engineering, City University of Hong KongHong Kong, Hong Kong.,Centre for Biosystems, Neuroscience, and Nanotechnology, City University of Hong KongHong Kong, Hong Kong
| | - Rosa H M Chan
- Department of Electronic Engineering, City University of Hong KongHong Kong, Hong Kong.,Centre for Biosystems, Neuroscience, and Nanotechnology, City University of Hong KongHong Kong, Hong Kong
| | - Chung Tin
- Department of Mechanical and Biomedical Engineering, City University of Hong KongHong Kong, Hong Kong.,Centre for Biosystems, Neuroscience, and Nanotechnology, City University of Hong KongHong Kong, Hong Kong.,Centre for Robotics and Automation, City University of Hong KongHong Kong, Hong Kong
| |
Collapse
|
89
|
Koch P, Maass M, Katzberg F, Mertins A. Early prediction of future hand movements using sEMG data. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2017; 2017:54-57. [PMID: 29059809 DOI: 10.1109/embc.2017.8036761] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
We study in this work the feasibility of early prediction of hand movement based on sEMG signals to overcome the time delay issue of the conventional classification. Opposed to the classification task, the objective of early prediction is to predict a hand movement that is going to occur in the future given the information up to the current time point. The ability of early prediction may allow a hand prosthesis control system to compensate for the time delay and, as a result, improve the usability. Experimental results on the Ninapro database show that we can predict up to 300 ms ahead in the future while the prediction accuracy remains very close to that of the standard classification, i.e. it is just marginally lower. Furthermore, historical data prior the current time window is shown to be very important to improve performance, not only for the prediction but also the classification task.
Collapse
|
90
|
Meeker C, Park S, Bishop L, Stein J, Ciocarlie M. EMG pattern classification to control a hand orthosis for functional grasp assistance after stroke. IEEE Int Conf Rehabil Robot 2017; 2017:1203-1210. [PMID: 28813985 DOI: 10.1109/icorr.2017.8009413] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
Wearable orthoses can function both as assistive devices, which allow the user to live independently, and as rehabilitation devices, which allow the user to regain use of an impaired limb. To be fully wearable, such devices must have intuitive controls, and to improve quality of life, the device should enable the user to perform Activities of Daily Living. In this context, we explore the feasibility of using electromyography (EMG) signals to control a wearable exotendon device to enable pick and place tasks. We use an easy to don, commodity forearm EMG band with 8 sensors to create an EMG pattern classification control for an exotendon device. With this control, we are able to detect a user's intent to open, and can thus enable extension and pick and place tasks. In experiments with stroke survivors, we explore the accuracy of this control in both non-functional and functional tasks. Our results support the feasibility of developing wearable devices with intuitive controls which provide a functional context for rehabilitation.
Collapse
|
91
|
Hahne JM, Markovic M, Farina D. User adaptation in Myoelectric Man-Machine Interfaces. Sci Rep 2017; 7:4437. [PMID: 28667260 PMCID: PMC5493618 DOI: 10.1038/s41598-017-04255-x] [Citation(s) in RCA: 72] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2016] [Accepted: 05/11/2017] [Indexed: 11/09/2022] Open
Abstract
State of the art clinical hand prostheses are controlled in a simple and limited way that allows the activation of one function at a time. More advanced laboratory approaches, based on machine learning, offer a significant increase in functionality, but their clinical impact is limited, mainly due to lack of reliability. In this study, we analyse two conceptually different machine learning approaches, focusing on their robustness and performance in a closed loop application. A classification (finite number of classes) and a regression (continuous mapping) based projection of EMG into external commands were applied while artificially introducing non-stationarities in the EMG signals. When tested on ten able-bodied individuals and one transradial amputee, the two methods were similarly influenced by non-stationarities when tested offline. However, in online tests, where the user could adapt his muscle activation patterns to the changed conditions, the regression-based approach was significantly less influenced by the changes in signal features than the classification approach. This observation demonstrates, on the one hand, the importance of online tests with users in the loop for assessing the performance of myocontrol approaches. On the other hand, it also demonstrates that regression allows for a better user correction of control commands than classification.
Collapse
Affiliation(s)
- Janne M Hahne
- Neurorehabilitaiton Systems Research Group, Department of Trauma Surgery, Orthopedic Surgery and Hand Surgery, Universiy Medical Center Göttingen, Göttingen, Germany.
| | - Marko Markovic
- Neurorehabilitaiton Systems Research Group, Department of Trauma Surgery, Orthopedic Surgery and Hand Surgery, Universiy Medical Center Göttingen, Göttingen, Germany
| | - Dario Farina
- Neurorehabilitaiton Systems Research Group, Department of Trauma Surgery, Orthopedic Surgery and Hand Surgery, Universiy Medical Center Göttingen, Göttingen, Germany.,Department of Bioengineering, Imperial College London, London, UK
| |
Collapse
|
92
|
Weisz J, Allen PK, Barszap AG, Joshi SS. Assistive grasping with an augmented reality user interface. Int J Rob Res 2017. [DOI: 10.1177/0278364917707024] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Affiliation(s)
- Jonathan Weisz
- Department of Computer Science, Columbia University, New York, NY, USA
| | - Peter K Allen
- Department of Computer Science, Columbia University, New York, NY, USA
| | - Alexander G Barszap
- Department of Mechanical and Aerospace Engineering, University of California, Davis, USA
| | - Sanjay S Joshi
- Department of Mechanical and Aerospace Engineering, University of California, Davis, USA
| |
Collapse
|
93
|
Ghazaei G, Alameer A, Degenaar P, Morgan G, Nazarpour K. Deep learning-based artificial vision for grasp classification in myoelectric hands. J Neural Eng 2017; 14:036025. [PMID: 28467317 DOI: 10.1088/1741-2552/aa6802] [Citation(s) in RCA: 99] [Impact Index Per Article: 14.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Computer vision-based assistive technology solutions can revolutionise the quality of care for people with sensorimotor disorders. The goal of this work was to enable trans-radial amputees to use a simple, yet efficient, computer vision system to grasp and move common household objects with a two-channel myoelectric prosthetic hand. APPROACH We developed a deep learning-based artificial vision system to augment the grasp functionality of a commercial prosthesis. Our main conceptual novelty is that we classify objects with regards to the grasp pattern without explicitly identifying them or measuring their dimensions. A convolutional neural network (CNN) structure was trained with images of over 500 graspable objects. For each object, 72 images, at [Formula: see text] intervals, were available. Objects were categorised into four grasp classes, namely: pinch, tripod, palmar wrist neutral and palmar wrist pronated. The CNN setting was first tuned and tested offline and then in realtime with objects or object views that were not included in the training set. MAIN RESULTS The classification accuracy in the offline tests reached [Formula: see text] for the seen and [Formula: see text] for the novel objects; reflecting the generalisability of grasp classification. We then implemented the proposed framework in realtime on a standard laptop computer and achieved an overall score of [Formula: see text] in classifying a set of novel as well as seen but randomly-rotated objects. Finally, the system was tested with two trans-radial amputee volunteers controlling an i-limb UltraTM prosthetic hand and a motion controlTM prosthetic wrist; augmented with a webcam. After training, subjects successfully picked up and moved the target objects with an overall success of up to [Formula: see text]. In addition, we show that with training, subjects' performance improved in terms of time required to accomplish a block of 24 trials despite a decreasing level of visual feedback. SIGNIFICANCE The proposed design constitutes a substantial conceptual improvement for the control of multi-functional prosthetic hands. We show for the first time that deep-learning based computer vision systems can enhance the grip functionality of myoelectric hands considerably.
Collapse
Affiliation(s)
- Ghazal Ghazaei
- School of Electrical and Electronic Engineering, Newcastle University, Newcastle-upon-Tyne NE1 7RU, United Kingdom
| | | | | | | | | |
Collapse
|
94
|
Yang W, Wu X, Yu S. A Master–Slave Control Method for Dexterous Hands with Shaking Elimination Strategy. INT J HUM ROBOT 2017. [DOI: 10.1142/s021984361650016x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
With the limitations of the artificial intelligence, automatic control and sensor technologies, the dexterous hand in unstructured environments to achieve fully autonomous operations is still very difficult. This paper proposed a master–slave control method for dexterous hands with the combination of the data glove and the micro-stepper motor. The hardware of this method included CyberGloveII device, personal computer (PC), integrated control board (ICB), and YWZ dexterous hand (a multi-fingered robot hand with 20 active degrees of freedom (DOFs)). By the CyberGloveII device, we gained human finger joints motion data in real-time firstly, which were preprocessed by a shaking elimination algorithm to ensure the motion stability of the dexterous hand. Then, the motion data were mapped to the dexterous hand joints, respectively. A communication protocol was designed to transfer the motion data between the PC and the ICB. The motion data were transmitted into the ICB through a serial interface driving the corresponding dexterous hand joints. The experimental results showed that this method is feasible, can achieve the open-loop control of dexterous hands, and has excellent movement accuracy, real-time and stability.
Collapse
Affiliation(s)
- Wenzhen Yang
- Virtual Reality Laboratory, Zhejiang Sci-Tech University, 2# Avenue, Xiasha Higher Educational Zone, Hangzhou, Zhejiang 310018, P. R. China
- Mechanism and Automation Department, Zhejiang Sci-Tech University, Hangzhou, Zhejiang 310018, P. R. China
| | - Xinli Wu
- Virtual Reality Laboratory, Zhejiang Sci-Tech University, 2# Avenue, Xiasha Higher Educational Zone, Hangzhou, Zhejiang 310018, P. R. China
| | - Shiguang Yu
- Virtual Reality Laboratory, Zhejiang Sci-Tech University, 2# Avenue, Xiasha Higher Educational Zone, Hangzhou, Zhejiang 310018, P. R. China
| |
Collapse
|
95
|
Surface EMG-Based Inter-Session Gesture Recognition Enhanced by Deep Domain Adaptation. SENSORS 2017; 17:s17030458. [PMID: 28245586 PMCID: PMC5375744 DOI: 10.3390/s17030458] [Citation(s) in RCA: 115] [Impact Index Per Article: 16.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2016] [Revised: 02/03/2017] [Accepted: 02/21/2017] [Indexed: 11/17/2022]
Abstract
High-density surface electromyography (HD-sEMG) is to record muscles' electrical activity from a restricted area of the skin by using two dimensional arrays of closely spaced electrodes. This technique allows the analysis and modelling of sEMG signals in both the temporal and spatial domains, leading to new possibilities for studying next-generation muscle-computer interfaces (MCIs). sEMG-based gesture recognition has usually been investigated in an intra-session scenario, and the absence of a standard benchmark database limits the use of HD-sEMG in real-world MCI. To address these problems, we present a benchmark database of HD-sEMG recordings of hand gestures performed by 23 participants, based on an 8 × 16 electrode array, and propose a deep-learning-based domain adaptation framework to enhance sEMG-based inter-session gesture recognition. Experiments on NinaPro, CSL-HDEMG and our CapgMyo dataset validate that our approach outperforms state-of-the-arts methods on intra-session and effectively improved inter-session gesture recognition.
Collapse
|
96
|
Liu J, Chen W, Li M, Kang X. Continuous Recognition of Multifunctional Finger and Wrist Movements in Amputee Subjects Based on sEMG and Accelerometry. Open Biomed Eng J 2017; 10:101-110. [PMID: 28217178 PMCID: PMC5299557 DOI: 10.2174/1874120701610010101] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2016] [Revised: 10/31/2016] [Accepted: 11/02/2016] [Indexed: 11/22/2022] Open
Abstract
Background: While the classification of multifunctional finger and wrist movement based on surface electromyography (sEMG) signals in intact subjects can reach remarkable recognition rates, the performance obtained from amputated subjects remained low. Methods: In this paper, we proposed and evaluated the myoelectric control scheme of upper-limb prostheses by the continuous recognition of 17 multifunctional finger and wrist movements on 5 amputated subjects. Experimental validation was applied to select optimal features and classifiers for identifying sEMG and accelerometry (ACC) modalities under the windows-based analysis scheme. The majority vote is adopted to eliminate transient jumps and produces smooth output for window-based analysis scheme. Furthermore, principle component analysis was employed to reduce the dimension of features and to eliminate redundancy for ACC signal. Then a novel metric, namely movement error rate, was also employed to evaluate the performance of the continuous recognition framework proposed herein. Results: The average accuracy rates of classification were up to 88.7 ± 2.6% over 5 amputated subjects, which was an outstanding result in comparison with the previous literature. Conclusion: The proposed technique was proven to be a potential candidate for intelligent prosthetic systems, which would increase quality of life for amputated subjects.
Collapse
Affiliation(s)
- Junhong Liu
- Department of Communication Engineering, Jilin University, 130012 Changchun, China
| | - Wanzhong Chen
- Department of Communication Engineering, Jilin University, 130012 Changchun, China
| | - Mingyang Li
- Department of Communication Engineering, Jilin University, 130012 Changchun, China
| | - Xiaotao Kang
- Department of Communication Engineering, Jilin University, 130012 Changchun, China
| |
Collapse
|
97
|
Quitadamo LR, Cavrini F, Sbernini L, Riillo F, Bianchi L, Seri S, Saggio G. Support vector machines to detect physiological patterns for EEG and EMG-based human-computer interaction: a review. J Neural Eng 2017; 14:011001. [PMID: 28068295 DOI: 10.1088/1741-2552/14/1/011001] [Citation(s) in RCA: 47] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
Support vector machines (SVMs) are widely used classifiers for detecting physiological patterns in human-computer interaction (HCI). Their success is due to their versatility, robustness and large availability of free dedicated toolboxes. Frequently in the literature, insufficient details about the SVM implementation and/or parameters selection are reported, making it impossible to reproduce study analysis and results. In order to perform an optimized classification and report a proper description of the results, it is necessary to have a comprehensive critical overview of the applications of SVM. The aim of this paper is to provide a review of the usage of SVM in the determination of brain and muscle patterns for HCI, by focusing on electroencephalography (EEG) and electromyography (EMG) techniques. In particular, an overview of the basic principles of SVM theory is outlined, together with a description of several relevant literature implementations. Furthermore, details concerning reviewed papers are listed in tables and statistics of SVM use in the literature are presented. Suitability of SVM for HCI is discussed and critical comparisons with other classifiers are reported.
Collapse
Affiliation(s)
- L R Quitadamo
- Department of Electronic Engineering, University of Rome Tor Vergata, Rome, Italy. School of Life and Health Sciences, Aston Brain Center, Aston University, Birmingham, UK
| | | | | | | | | | | | | |
Collapse
|
98
|
Yang D, Gu Y, Jiang L, Osborn L, Liu H. Dynamic training protocol improves the robustness of PR-based myoelectric control. Biomed Signal Process Control 2017. [DOI: 10.1016/j.bspc.2016.08.017] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
99
|
Yang D, Yang W, Huang Q, Liu H. Classification of Multiple Finger Motions During Dynamic Upper Limb Movements. IEEE J Biomed Health Inform 2017; 21:134-141. [DOI: 10.1109/jbhi.2015.2490718] [Citation(s) in RCA: 47] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
100
|
Connan M, Ruiz Ramírez E, Vodermayer B, Castellini C. Assessment of a Wearable Force- and Electromyography Device and Comparison of the Related Signals for Myocontrol. Front Neurorobot 2016; 10:17. [PMID: 27909406 PMCID: PMC5112250 DOI: 10.3389/fnbot.2016.00017] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2016] [Accepted: 10/21/2016] [Indexed: 11/13/2022] Open
Abstract
In the frame of assistive robotics, multi-finger prosthetic hand/wrists have recently appeared, offering an increasing level of dexterity; however, in practice their control is limited to a few hand grips and still unreliable, with the effect that pattern recognition has not yet appeared in the clinical environment. According to the scientific community, one of the keys to improve the situation is multi-modal sensing, i.e., using diverse sensor modalities to interpret the subject's intent and improve the reliability and safety of the control system in daily life activities. In this work, we first describe and test a novel wireless, wearable force- and electromyography device; through an experiment conducted on ten intact subjects, we then compare the obtained signals both qualitatively and quantitatively, highlighting their advantages and disadvantages. Our results indicate that force-myography yields signals which are more stable across time during whenever a pattern is held, than those obtained by electromyography. We speculate that fusion of the two modalities might be advantageous to improve the reliability of myocontrol in the near future.
Collapse
Affiliation(s)
- Mathilde Connan
- Cognitive Robotics, Institute of Robotics and Mechatronics, German Aerospace Center (DLR) Wessling, Germany
| | - Eduardo Ruiz Ramírez
- Cognitive Robotics, Institute of Robotics and Mechatronics, German Aerospace Center (DLR) Wessling, Germany
| | - Bernhard Vodermayer
- Cognitive Robotics, Institute of Robotics and Mechatronics, German Aerospace Center (DLR) Wessling, Germany
| | - Claudio Castellini
- Cognitive Robotics, Institute of Robotics and Mechatronics, German Aerospace Center (DLR) Wessling, Germany
| |
Collapse
|