1
|
Xia H, Zhang Y, Rajabi N, Taleb F, Yang Q, Kragic D, Li Z. Shaping high-performance wearable robots for human motor and sensory reconstruction and enhancement. Nat Commun 2024; 15:1760. [PMID: 38409128 PMCID: PMC10897332 DOI: 10.1038/s41467-024-46249-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 02/19/2024] [Indexed: 02/28/2024] Open
Abstract
Most wearable robots such as exoskeletons and prostheses can operate with dexterity, while wearers do not perceive them as part of their bodies. In this perspective, we contend that integrating environmental, physiological, and physical information through multi-modal fusion, incorporating human-in-the-loop control, utilizing neuromuscular interface, employing flexible electronics, and acquiring and processing human-robot information with biomechatronic chips, should all be leveraged towards building the next generation of wearable robots. These technologies could improve the embodiment of wearable robots. With optimizations in mechanical structure and clinical training, the next generation of wearable robots should better facilitate human motor and sensory reconstruction and enhancement.
Collapse
Affiliation(s)
- Haisheng Xia
- School of Mechanical Engineering, Tongji University, Shanghai, 201804, China
- Translational Research Center, Shanghai YangZhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Tongji University, Shanghai, 201619, China
- Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230026, China
| | - Yuchong Zhang
- Robotics, Perception and Learning Lab, EECS at KTH Royal Institute of Technology Stockholm, 114 17, Stockholm, Sweden
| | - Nona Rajabi
- Robotics, Perception and Learning Lab, EECS at KTH Royal Institute of Technology Stockholm, 114 17, Stockholm, Sweden
| | - Farzaneh Taleb
- Robotics, Perception and Learning Lab, EECS at KTH Royal Institute of Technology Stockholm, 114 17, Stockholm, Sweden
| | - Qunting Yang
- Department of Automation, University of Science and Technology of China, Hefei, 230026, China
| | - Danica Kragic
- Robotics, Perception and Learning Lab, EECS at KTH Royal Institute of Technology Stockholm, 114 17, Stockholm, Sweden
| | - Zhijun Li
- School of Mechanical Engineering, Tongji University, Shanghai, 201804, China.
- Translational Research Center, Shanghai YangZhi Rehabilitation Hospital (Shanghai Sunshine Rehabilitation Center), Tongji University, Shanghai, 201619, China.
- Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, 230026, China.
| |
Collapse
|
2
|
Tian Y, Vaskov AK, Adidharma W, Cederna PS, Kemp SW. Merging Humans and Neuroprosthetics through Regenerative Peripheral Nerve Interfaces. Semin Plast Surg 2024; 38:10-18. [PMID: 38495064 PMCID: PMC10942838 DOI: 10.1055/s-0044-1779028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Limb amputations can be devastating and significantly affect an individual's independence, leading to functional and psychosocial challenges in nearly 2 million people in the United States alone. Over the past decade, robotic devices driven by neural signals such as neuroprostheses have shown great potential to restore the lost function of limbs, allowing amputees to regain movement and sensation. However, current neuroprosthetic interfaces have challenges in both signal quality and long-term stability. To overcome these limitations and work toward creating bionic limbs, the Neuromuscular Laboratory at University of Michigan Plastic Surgery has developed the Regenerative Peripheral Nerve Interface (RPNI). This surgical construct embeds a transected peripheral nerve into a free muscle graft, effectively amplifying small peripheral nerve signals to provide enhanced control signals for a neuroprosthetic limb. Furthermore, the RPNI has the potential to provide sensory feedback to the user and facilitate neuroprosthesis embodiment. This review focuses on the animal studies and clinical trials of the RPNI to recapitulate the promising trajectory toward neurobionics where the boundary between an artificial device and the human body becomes indistinct. This paper also sheds light on the prospects of the improvement and dissemination of the RPNI technology.
Collapse
Affiliation(s)
- Yucheng Tian
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan
| | - Alex K. Vaskov
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Widya Adidharma
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Paul S. Cederna
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, Michigan
| | - Stephen W.P. Kemp
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, Michigan
- Section of Plastic Surgery, Department of Surgery, University of Michigan, Ann Arbor, Michigan
| |
Collapse
|
3
|
Taghlabi KM, Cruz-Garza JG, Hassan T, Potnis O, Bhenderu LS, Guerrero JR, Whitehead RE, Wu Y, Luan L, Xie C, Robinson JT, Faraji AH. Clinical outcomes of peripheral nerve interfaces for rehabilitation in paralysis and amputation: a literature review. J Neural Eng 2024; 21:011001. [PMID: 38237175 DOI: 10.1088/1741-2552/ad200f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 01/18/2024] [Indexed: 02/02/2024]
Abstract
Peripheral nerve interfaces (PNIs) are electrical systems designed to integrate with peripheral nerves in patients, such as following central nervous system (CNS) injuries to augment or replace CNS control and restore function. We review the literature for clinical trials and studies containing clinical outcome measures to explore the utility of human applications of PNIs. We discuss the various types of electrodes currently used for PNI systems and their functionalities and limitations. We discuss important design characteristics of PNI systems, including biocompatibility, resolution and specificity, efficacy, and longevity, to highlight their importance in the current and future development of PNIs. The clinical outcomes of PNI systems are also discussed. Finally, we review relevant PNI clinical trials that were conducted, up to the present date, to restore the sensory and motor function of upper or lower limbs in amputees, spinal cord injury patients, or intact individuals and describe their significant findings. This review highlights the current progress in the field of PNIs and serves as a foundation for future development and application of PNI systems.
Collapse
Affiliation(s)
- Khaled M Taghlabi
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
| | - Jesus G Cruz-Garza
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
| | - Taimur Hassan
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- School of Medicine, Texas A&M University, Bryan, TX 77807, United States of America
| | - Ojas Potnis
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- School of Engineering Medicine, Texas A&M University, Houston, TX 77030, United States of America
| | - Lokeshwar S Bhenderu
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- School of Medicine, Texas A&M University, Bryan, TX 77807, United States of America
| | - Jaime R Guerrero
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
| | - Rachael E Whitehead
- Department of Academic Affairs, Houston Methodist Academic Institute, Houston, TX 77030, United States of America
| | - Yu Wu
- Rice Neuroengineering Initiative, Rice University, Houston, TX 77005, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, United States of America
| | - Lan Luan
- Rice Neuroengineering Initiative, Rice University, Houston, TX 77005, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, United States of America
| | - Chong Xie
- Rice Neuroengineering Initiative, Rice University, Houston, TX 77005, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, United States of America
| | - Jacob T Robinson
- Rice Neuroengineering Initiative, Rice University, Houston, TX 77005, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, United States of America
| | - Amir H Faraji
- Department of Neurological Surgery, Houston Methodist Hospital, Houston, TX 77030, United States of America
- Center for Neural Systems Restoration, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Clinical Innovations Laboratory, Houston Methodist Research Institute, Houston, TX 77030, United States of America
- Rice Neuroengineering Initiative, Rice University, Houston, TX 77005, United States of America
- Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, United States of America
| |
Collapse
|
4
|
Donati E, Valle G. Neuromorphic hardware for somatosensory neuroprostheses. Nat Commun 2024; 15:556. [PMID: 38228580 PMCID: PMC10791662 DOI: 10.1038/s41467-024-44723-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2022] [Accepted: 01/03/2024] [Indexed: 01/18/2024] Open
Abstract
In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.
Collapse
Affiliation(s)
- Elisa Donati
- Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland.
| | - Giacomo Valle
- Department of Organismal Biology and Anatomy, University of Chicago, Chicago, IL, USA.
| |
Collapse
|
5
|
Barberi F, Anselmino E, Mazzoni A, Goldfarb M, Micera S. Toward the Development of User-Centered Neurointegrated Lower Limb Prostheses. IEEE Rev Biomed Eng 2024; 17:212-228. [PMID: 37639425 DOI: 10.1109/rbme.2023.3309328] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/31/2023]
Abstract
The last few years witnessed radical improvements in lower-limb prostheses. Researchers have presented innovative solutions to overcome the limits of the first generation of prostheses, refining specific aspects which could be implemented in future prostheses designs. Each aspect of lower-limb prostheses has been upgraded, but despite these advances, a number of deficiencies remain and the most capable limb prostheses fall far short of the capabilities of the healthy limb. This article describes the current state of prosthesis technology; identifies a number of deficiencies across the spectrum of lower limb prosthetic components with respect to users' needs; and discusses research opportunities in design and control that would substantially improve functionality concerning each deficiency. In doing so, the authors present a roadmap of patients related issues that should be addressed in order to fulfill the vision of a next-generation, neurally-integrated, highly-functional lower limb prosthesis.
Collapse
|
6
|
Giannotti A, Lo Vecchio S, Musco S, Pollina L, Vallone F, Strauss I, Paggi V, Bernini F, Gabisonia K, Carlucci L, Lenzi C, Pirone A, Giannessi E, Miragliotta V, Lacour S, Del Popolo G, Moccia S, Micera S. Decoding bladder state from pudendal intraneural signals in pigs. APL Bioeng 2023; 7:046101. [PMID: 37811476 PMCID: PMC10558243 DOI: 10.1063/5.0156484] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 09/11/2023] [Indexed: 10/10/2023] Open
Abstract
Neuroprosthetic devices used for the treatment of lower urinary tract dysfunction, such as incontinence or urinary retention, apply a pre-set continuous, open-loop stimulation paradigm, which can cause voiding dysfunctions due to neural adaptation. In the literature, conditional, closed-loop stimulation paradigms have been shown to increase bladder capacity and voiding efficacy compared to continuous stimulation. Current limitations to the implementation of the closed-loop stimulation paradigm include the lack of robust and real-time decoding strategies for the bladder fullness state. We recorded intraneural pudendal nerve signals in five anesthetized pigs. Three bladder-filling states, corresponding to empty, full, and micturition, were decoded using the Random Forest classifier. The decoding algorithm showed a mean balanced accuracy above 86.67% among the three classes for all five animals. Our approach could represent an important step toward the implementation of an adaptive real-time closed-loop stimulation protocol for pudendal nerve modulation, paving the way for the design of an assisted-as-needed neuroprosthesis.
Collapse
Affiliation(s)
- A. Giannotti
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - S. Lo Vecchio
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - S. Musco
- Neuro-Urology Department, Careggi University Hospital, Firenze, Italy
| | - L. Pollina
- Bertarelli Foundation Chair in Translational NeuroEngineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - F. Vallone
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - I. Strauss
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering–IMTEK, IMBIT//NeuroProbes BrainLinks-BrainTools Cluster of Excellence, University of Freiburg, Freiburg, Germany
| | - V. Paggi
- Bertarelli Foundation Chair in Microengineering and Bioengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - F. Bernini
- BioMedLab, Scuola Superiore Sant'Anna, Pisa, Italy
| | - K. Gabisonia
- BioMedLab, Scuola Superiore Sant'Anna, Pisa, Italy
| | - L. Carlucci
- BioMedLab, Scuola Superiore Sant'Anna, Pisa, Italy
| | - C. Lenzi
- Department of Veterinary Sciences, University of Pisa, Pisa, Italy
| | - A. Pirone
- Department of Veterinary Sciences, University of Pisa, Pisa, Italy
| | - E. Giannessi
- Department of Veterinary Sciences, University of Pisa, Pisa, Italy
| | - V. Miragliotta
- Department of Veterinary Sciences, University of Pisa, Pisa, Italy
| | - S. Lacour
- Bertarelli Foundation Chair in Microengineering and Bioengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - G. Del Popolo
- Neuro-Urology Department, Careggi University Hospital, Firenze, Italy
| | - S. Moccia
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | - S. Micera
- Author to whom correspondence should be addressed:
| |
Collapse
|
7
|
Segas E, Mick S, Leconte V, Dubois O, Klotz R, Cattaert D, de Rugy A. Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality. eLife 2023; 12:RP87317. [PMID: 37847150 PMCID: PMC10581689 DOI: 10.7554/elife.87317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2023] Open
Abstract
Impressive progress is being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in people with transhumeral limb loss based on proximal residual limb motion and knowledge of the movement goal. This control was validated on 29 participants, including seven with above-elbow limb loss, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. This control also enabled 15 participants, including three with limb differences, to reach and grasp real objects with a robotic arm operated according to the same principle. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment the reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.
Collapse
Affiliation(s)
- Effie Segas
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
| | - Sébastien Mick
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
- ISIR UMR 7222, Sorbonne Université, CNRS, InsermParisFrance
| | | | - Océane Dubois
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
- ISIR UMR 7222, Sorbonne Université, CNRS, InsermParisFrance
| | | | | | | |
Collapse
|
8
|
Farina D, Vujaklija I, Brånemark R, Bull AMJ, Dietl H, Graimann B, Hargrove LJ, Hoffmann KP, Huang HH, Ingvarsson T, Janusson HB, Kristjánsson K, Kuiken T, Micera S, Stieglitz T, Sturma A, Tyler D, Weir RFF, Aszmann OC. Toward higher-performance bionic limbs for wider clinical use. Nat Biomed Eng 2023; 7:473-485. [PMID: 34059810 DOI: 10.1038/s41551-021-00732-x] [Citation(s) in RCA: 68] [Impact Index Per Article: 68.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2019] [Accepted: 04/01/2021] [Indexed: 12/19/2022]
Abstract
Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.
Collapse
Affiliation(s)
- Dario Farina
- Department of Bioengineering, Imperial College London, London, UK.
| | - Ivan Vujaklija
- Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland
| | - Rickard Brånemark
- Center for Extreme Bionics, Biomechatronics Group, MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Anthony M J Bull
- Department of Bioengineering, Imperial College London, London, UK
| | - Hans Dietl
- Ottobock Products SE & Co. KGaA, Vienna, Austria
| | | | - Levi J Hargrove
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Klaus-Peter Hoffmann
- Department of Medical Engineering & Neuroprosthetics, Fraunhofer-Institut für Biomedizinische Technik, Sulzbach, Germany
| | - He Helen Huang
- NCSU/UNC Joint Department of Biomedical Engineering, North Carolina State University, Raleigh, NC, USA
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Thorvaldur Ingvarsson
- Department of Research and Development, Össur Iceland, Reykjavík, Iceland
- Faculty of Medicine, University of Iceland, Reykjavík, Iceland
| | - Hilmar Bragi Janusson
- School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland
| | | | - Todd Kuiken
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Silvestro Micera
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Bertarelli Foundation Chair in Translational NeuroEngineering, Center for Neuroprosthetics and Institute of Bioengineering, School of Engineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| | - Thomas Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering-IMTEK, BrainLinks-BrainTools Center and Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany
| | - Agnes Sturma
- Department of Bioengineering, Imperial College London, London, UK
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| | - Dustin Tyler
- Case School of Engineering, Case Western Reserve University, Cleveland, OH, USA
- Louis Stokes Veterans Affairs Medical Centre, Cleveland, OH, USA
| | - Richard F Ff Weir
- Biomechatronics Development Laboratory, Bioengineering Department, University of Colorado Denver and VA Eastern Colorado Healthcare System, Aurora, CO, USA
| | - Oskar C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
9
|
Valle G. Peripheral neurostimulation for encoding artificial somatosensations. Eur J Neurosci 2022; 56:5888-5901. [PMID: 36097134 PMCID: PMC9826263 DOI: 10.1111/ejn.15822] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 08/08/2022] [Accepted: 09/08/2022] [Indexed: 01/11/2023]
Abstract
The direct neural stimulation of peripheral or central nervous systems has been shown as an effective tool to treat neurological conditions. The electrical activation of the nervous sensory pathway can be adopted to restore the artificial sense of touch and proprioception in people suffering from sensory-motor disorders. The modulation of the neural stimulation parameters has a direct effect on the electrically induced sensations, both when targeting the somatosensory cortex and the peripheral somatic nerves. The properties of the artificial sensations perceived, as their location, quality and intensity are strongly dependent on the direct modulation of pulse width, amplitude and frequency of the neural stimulation. Different sensory encoding schemes have been tested in patients showing distinct effects and outcomes according to their impact on the neural activation. Here, I reported the most adopted neural stimulation strategies to artificially encode somatosensation into the peripheral nervous system. The real-time implementation of these strategies in bionic devices is crucial to exploit the artificial sensory feedback in prosthetics. Thus, neural stimulation becomes a tool to directly communicate with the human nervous system. Given the importance of adding artificial sensory information to neuroprosthetic devices to improve their control and functionality, the choice of an optimal neural stimulation paradigm could increase the impact of prosthetic devices on the quality of life of people with sensorimotor disabilities.
Collapse
Affiliation(s)
- Giacomo Valle
- Laboratory for Neuroengineering, Department of Health Sciences and TechnologyInstitute for Robotics and Intelligent Systems, ETH ZürichZürichSwitzerland
| |
Collapse
|
10
|
Xu B, Zhang K, Yang X, Liu D, Hu C, Li H, Song A. Natural grasping movement recognition and force estimation using electromyography. Front Neurosci 2022; 16:1020086. [DOI: 10.3389/fnins.2022.1020086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Accepted: 10/06/2022] [Indexed: 11/13/2022] Open
Abstract
Electromyography (EMG) generated by human hand movements is usually used to decode different action types with high accuracy. However, the classifications of the gestures rarely consider the impact of force, and the estimation of the grasp force when performing natural grasping movements is so far overlooked. Decoding natural grasping movements and estimating the force generated by the associated movements can help patients to improve the accuracy of prosthesis control. This study mainly focused on two aspects: the classification of four natural grasping movements and the force estimation of these actions. For this purpose, we designed an experimental platform where subjects could perform four common natural grasping movements in daily life, including pinch, palmar, twist, and plug grasp, to complete target profiles. On the one hand, the results showed that, for natural grasping movements with different levels of force (three levels at 20, 50, and 80%), the average accuracy could reach from 91.43 to 97.33% under five classification schemes. On the other hand, the feasibility of force estimation for natural grasping movements was demonstrated. Furthermore, in the process of force estimation, we confirmed that the regression performance about plug grasp was the best, and the average R2 could reach 0.9082. Besides, we found that the regression results were affected by the speed of force application. These findings contribute to the natural control of myoelectric prosthesis and the EMG-based rehabilitation training system, improving the user’s experience and acceptance.
Collapse
|
11
|
Silveira C, Khushaba RN, Brunton E, Nazarpour K. Spatio-temporal feature extraction in sensory electroneurographic signals. PHILOSOPHICAL TRANSACTIONS. SERIES A, MATHEMATICAL, PHYSICAL, AND ENGINEERING SCIENCES 2022; 380:20210268. [PMID: 35658682 PMCID: PMC9289791 DOI: 10.1098/rsta.2021.0268] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/24/2021] [Accepted: 11/08/2021] [Indexed: 06/15/2023]
Abstract
The recording and analysis of peripheral neural signal can provide insight for various prosthetic and bioelectronics medicine applications. However, there are few studies that investigate how informative features can be extracted from population activity electroneurographic (ENG) signals. In this study, five feature extraction frameworks were implemented on sensory ENG datasets and their classification performance was compared. The datasets were collected in acute rat experiments where multi-channel nerve cuffs recorded from the sciatic nerve in response to proprioceptive stimulation of the hindlimb. A novel feature extraction framework, which incorporates spatio-temporal focus and dynamic time warping, achieved classification accuracies above 90% while keeping a low computational cost. This framework outperformed the remaining frameworks tested in this study and has improved the discrimination accuracy of the sensory signals. Thus, this study has extended the tools available to extract features from sensory population activity ENG signals. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.
Collapse
Affiliation(s)
- C. Silveira
- School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK
| | - R. N. Khushaba
- Australian Center for Field Robotics, The University of Sydney, New South Wales 2006, Australia
| | - E. Brunton
- National Vision Research Institute, Australian College of Optometry, Carlton, Victoria 3053, Australia
- Department of Optometry and Vision Sciences, Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Parkville, Victoria 3010, Australia
| | - K. Nazarpour
- Edinburgh Neuroprosthetics Laboratory, School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, UK
| |
Collapse
|
12
|
Lee H, Kim D, Park YL. Explainable Deep Learning Model for EMG-Based Finger Angle Estimation Using Attention. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1877-1886. [PMID: 35834448 DOI: 10.1109/tnsre.2022.3188275] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
Electromyography (EMG) is one of the most common methods to detect muscle activities and intentions. However, it has been difficult to estimate accurate hand motions represented by the finger joint angles using EMG signals. We propose an encoder-decoder network with an attention mechanism, an explainable deep learning model that estimates 14 finger joint angles from forearm EMG signals. This study demonstrates that the model trained by the single-finger motion data can be generalized to estimate complex motions of random fingers. The color map result of the after-training attention matrix shows that the proposed attention algorithm enables the model to learn the nonlinear relationship between the EMG signals and the finger joint angles, which is explainable. The highly activated entries in the color map of the attention matrix derived from model training are consistent with the experimental observations in which certain EMG sensors are highly activated when a particular finger moves. In summary, this study proposes an explainable deep learning model that estimates finger joint angles based on EMG signals of the forearm using the attention mechanism.
Collapse
|
13
|
Ahkami B, Mastinu E, Earley EJ, Ortiz-Catalan M. Extra-neural signals from severed nerves enable intrinsic hand movements in transhumeral amputations. Sci Rep 2022; 12:10218. [PMID: 35715459 PMCID: PMC9206000 DOI: 10.1038/s41598-022-13363-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 05/24/2022] [Indexed: 11/22/2022] Open
Abstract
Robotic prostheses controlled by myoelectric signals can restore limited but important hand function in individuals with upper limb amputation. The lack of individual finger control highlights the yet insurmountable gap to fully replacing a biological hand. Implanted electrodes around severed nerves have been used to elicit sensations perceived as arising from the missing limb, but using such extra-neural electrodes to record motor signals that allow for the decoding of phantom movements has remained elusive. Here, we showed the feasibility of using signals from non-penetrating neural electrodes to decode intrinsic hand and finger movements in individuals with above-elbow amputations. We found that information recorded with extra-neural electrodes alone was enough to decode phantom hand and individual finger movements, and as expected, the addition of myoelectric signals reduced classification errors both in offline and in real-time decoding.
Collapse
Affiliation(s)
- Bahareh Ahkami
- Center for Bionics and Pain Research, Mölndal, Sweden.,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Enzo Mastinu
- Center for Bionics and Pain Research, Mölndal, Sweden.,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Eric J Earley
- Center for Bionics and Pain Research, Mölndal, Sweden.,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Max Ortiz-Catalan
- Center for Bionics and Pain Research, Mölndal, Sweden. .,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden. .,Operational Area 3, Sahlgrenska University Hospital, Mölndal, Sweden. .,Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| |
Collapse
|
14
|
Ottaviani MM, Vallone F, Micera S, Recchia FA. Closed-Loop Vagus Nerve Stimulation for the Treatment of Cardiovascular Diseases: State of the Art and Future Directions. Front Cardiovasc Med 2022; 9:866957. [PMID: 35463766 PMCID: PMC9021417 DOI: 10.3389/fcvm.2022.866957] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 03/14/2022] [Indexed: 01/07/2023] Open
Abstract
The autonomic nervous system exerts a fine beat-to-beat regulation of cardiovascular functions and is consequently involved in the onset and progression of many cardiovascular diseases (CVDs). Selective neuromodulation of the brain-heart axis with advanced neurotechnologies is an emerging approach to corroborate CVDs treatment when classical pharmacological agents show limited effectiveness. The vagus nerve is a major component of the cardiac neuroaxis, and vagus nerve stimulation (VNS) is a promising application to restore autonomic function under various pathological conditions. VNS has led to encouraging results in animal models of CVDs, but its translation to clinical practice has not been equally successful, calling for more investigation to optimize this technique. Herein we reviewed the state of the art of VNS for CVDs and discuss avenues for therapeutic optimization. Firstly, we provided a succinct description of cardiac vagal innervation anatomy and physiology and principles of VNS. Then, we examined the main clinical applications of VNS in CVDs and the related open challenges. Finally, we presented preclinical studies that aim at overcoming VNS limitations through optimization of anatomical targets, development of novel neural interface technologies, and design of efficient VNS closed-loop protocols.
Collapse
Affiliation(s)
- Matteo Maria Ottaviani
- Institute of Life Sciences, Scuola Superiore Sant’Anna, Pisa, Italy
- Department of Excellence in Robotics and Artificial Intelligence, The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Fabio Vallone
- Department of Excellence in Robotics and Artificial Intelligence, The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
| | - Silvestro Micera
- Department of Excellence in Robotics and Artificial Intelligence, The BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy
- Bertarelli Foundation Chair in Translational Neural Engineering, Center for Neuroprosthetics, Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| | - Fabio A. Recchia
- Institute of Life Sciences, Scuola Superiore Sant’Anna, Pisa, Italy
- Fondazione Toscana Gabriele Monasterio, Pisa, Italy
- Department of Physiology, Cardiovascular Research Center, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, United States
| |
Collapse
|
15
|
Luu DK, Nguyen AT, Jiang M, Drealan MW, Xu J, Wu T, Tam WK, Zhao W, Lim BZH, Overstreet CK, Zhao Q, Cheng J, Keefer EW, Yang Z. Artificial Intelligence Enables Real-Time and Intuitive Control of Prostheses via Nerve Interface. IEEE Trans Biomed Eng 2022; 69:3051-3063. [PMID: 35302937 DOI: 10.1109/tbme.2022.3160618] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVE The next generation prosthetic hand that moves and feels like a real hand requires a robust neural interconnection between the human minds and machines. METHODS Here we present a neuroprosthetic system to demonstrate that principle by employing an artificial intelligence (AI) agent to translate the amputees movement intent through a peripheral nerve interface. The AI agent is designed based on the recurrent neural network (RNN) and could simultaneously decode six degree-of-freedom (DOF) from multichannel nerve data in real-time. The decoder's performance is characterized in motor decoding experiments with three human amputees. RESULTS First, we show the AI agent enables amputees to intuitively control a prosthetic hand with individual finger and wrist movements up to 97-98% accuracy. Second, we demonstrate the AI agent's real-time performance by measuring the reaction time and information throughput in a hand gesture matching task. Third, we investigate the AI agent's long-term uses and show the decoder's robust predictive performance over a 16-month implant duration. Conclusion & significance: Our study demonstrates the potential of AI-enabled nerve technology, underling the next generation of dexterous and intuitive prosthetic hands.
Collapse
|
16
|
Pasluosta C, Kiele P, Čvančara P, Micera S, Aszmann OC, Stieglitz T. Bidirectional bionic limbs: a perspective bridging technology and physiology. J Neural Eng 2022; 19. [PMID: 35132954 DOI: 10.1088/1741-2552/ac4bff] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 01/17/2022] [Indexed: 11/11/2022]
Abstract
Precise control of bionic limbs relies on robust decoding of motor commands from nerves or muscles signals and sensory feedback from artificial limbs to the nervous system by interfacing the afferent nerve pathways. Implantable devices for bidirectional communication with bionic limbs have been developed in parallel with research on physiological alterations caused by an amputation. In this perspective article, we question whether increasing our effort on bridging these technologies with a deeper understanding of amputation pathophysiology and human motor control may help to overcome pressing stalls in the next generation of bionic limbs.
Collapse
Affiliation(s)
- C Pasluosta
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany
| | - P Kiele
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany
| | - P Čvančara
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany.,BrainLinks-BrainTools, University of Freiburg, Freiburg, Germany
| | - S Micera
- School of Engineering, École Polytechnique Fédérale de Lausanne, Bertarelli Foundation Chair in Translational Neuroengineering, Centre for Neuroprosthetics and Institute of Bioengineering, Lausanne, Switzerland.,The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - O C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Medical University of Vienna; Division of Plastic and Reconstructive Surgery, Department of Surgery, Medical University of Vienna, Vienna, Austria
| | - T Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany.,Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany.,BrainLinks-BrainTools, University of Freiburg, Freiburg, Germany
| |
Collapse
|
17
|
Lee MW, Jang N, Choi N, Yang S, Jeong J, Nam HS, Oh S, Kim K, Hwang D. In Vivo Cellular-Level 3D Imaging of Peripheral Nerves Using a Dual-Focusing Technique for Intra-Neural Interface Implantation. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2102876. [PMID: 34845862 PMCID: PMC8787432 DOI: 10.1002/advs.202102876] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Revised: 11/09/2021] [Indexed: 06/13/2023]
Abstract
In vivo volumetric imaging of the microstructural changes of peripheral nerves with an inserted electrode could be key for solving the chronic implantation failure of an intra-neural interface necessary to provide amputated patients with natural motion and sensation. Thus far, no imaging devices can provide a cellular-level three-dimensional (3D) structural images of a peripheral nerve in vivo. In this study, an optical coherence tomography-based peripheral nerve imaging platform that employs a newly proposed depth of focus extension technique is reported. A point spread function with the finest transverse resolution of 1.27 µm enables the cellular-level volumetric visualization of the metal wire and microstructural changes in a rat sciatic nerve with the metal wire inserted in vivo. Further, the feasibility of applying the imaging platform to large animals for a preclinical study is confirmed through in vivo rabbit sciatic nerve imaging. It is expected that new possibilities for the successful chronic implantation of an intra-neural interface will open up by providing the 3D microstructural changes of nerves around the inserted electrode.
Collapse
Affiliation(s)
- Min Woo Lee
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Namseon Jang
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Nara Choi
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Sungwook Yang
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Jinwoo Jeong
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Hyeong Soo Nam
- Department of Mechanical EngineeringKorea Advanced Institute of Science and TechnologyDaejeon34141Republic of Korea
| | - Sang‐Rok Oh
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| | - Keehoon Kim
- Department of Mechanical EngineeringPohang University of Science and TechnologyGyeongbuk37673Republic of Korea
| | - Donghyun Hwang
- Center for Intelligent and Interactive RoboticsKorea Institute of Science and TechnologySeoul02792Republic of Korea
| |
Collapse
|
18
|
Shokur S, Mazzoni A, Schiavone G, Weber DJ, Micera S. A modular strategy for next-generation upper-limb sensory-motor neuroprostheses. MED 2021; 2:912-937. [DOI: 10.1016/j.medj.2021.05.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2021] [Revised: 04/28/2021] [Accepted: 05/10/2021] [Indexed: 02/06/2023]
|
19
|
Karczewski AM, Dingle AM, Poore SO. The Need to Work Arm in Arm: Calling for Collaboration in Delivering Neuroprosthetic Limb Replacements. Front Neurorobot 2021; 15:711028. [PMID: 34366820 PMCID: PMC8334559 DOI: 10.3389/fnbot.2021.711028] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 06/22/2021] [Indexed: 11/21/2022] Open
Abstract
Over the last few decades there has been a push to enhance the use of advanced prosthetics within the fields of biomedical engineering, neuroscience, and surgery. Through the development of peripheral neural interfaces and invasive electrodes, an individual's own nervous system can be used to control a prosthesis. With novel improvements in neural recording and signal decoding, this intimate communication has paved the way for bidirectional and intuitive control of prostheses. While various collaborations between engineers and surgeons have led to considerable success with motor control and pain management, it has been significantly more challenging to restore sensation. Many of the existing peripheral neural interfaces have demonstrated success in one of these modalities; however, none are currently able to fully restore limb function. Though this is in part due to the complexity of the human somatosensory system and stability of bioelectronics, the fragmentary and as-yet uncoordinated nature of the neuroprosthetic industry further complicates this advancement. In this review, we provide a comprehensive overview of the current field of neuroprosthetics and explore potential strategies to address its unique challenges. These include exploration of electrodes, surgical techniques, control methods, and prosthetic technology. Additionally, we propose a new approach to optimizing prosthetic limb function and facilitating clinical application by capitalizing on available resources. It is incumbent upon academia and industry to encourage collaboration and utilization of different peripheral neural interfaces in combination with each other to create versatile limbs that not only improve function but quality of life. Despite the rapidly evolving technology, if the field continues to work in divided "silos," we will delay achieving the critical, valuable outcome: creating a prosthetic limb that is right for the patient and positively affects their life.
Collapse
Affiliation(s)
| | - Aaron M. Dingle
- Division of Plastic Surgery, Department of Surgery, University of Wisconsin–Madison, Madison, WI, United States
| | | |
Collapse
|
20
|
Vallone F, Ottaviani MM, Dedola F, Cutrone A, Romeni S, Panarese AM, Bernini F, Cracchiolo M, Strauss I, Gabisonia K, Gorgodze N, Mazzoni A, Recchia FA, Micera S. Simultaneous decoding of cardiovascular and respiratory functional changes from pig intraneural vagus nerve signals. J Neural Eng 2021; 18. [PMID: 34153949 DOI: 10.1088/1741-2552/ac0d42] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2020] [Accepted: 06/21/2021] [Indexed: 12/15/2022]
Abstract
Objective. Bioelectronic medicine is opening new perspectives for the treatment of some major chronic diseases through the physical modulation of autonomic nervous system activity. Being the main peripheral route for electrical signals between central nervous system and visceral organs, the vagus nerve (VN) is one of the most promising targets. Closed-loop VN stimulation (VNS) would be crucial to increase effectiveness of this approach. Therefore, the extrapolation of useful physiological information from VN electrical activity would represent an invaluable source for single-target applications. Here, we present an advanced decoding algorithm novel to VN studies and properly detecting different functional changes from VN signals.Approach. VN signals were recorded using intraneural electrodes in anaesthetized pigs during cardiovascular and respiratory challenges mimicking increases in arterial blood pressure, tidal volume and respiratory rate. We developed a decoding algorithm that combines discrete wavelet transformation, principal component analysis, and ensemble learning made of classification trees.Main results. The new decoding algorithm robustly achieved high accuracy levels in identifying different functional changes and discriminating among them. Interestingly our findings suggest that electrodes positioning plays an important role on decoding performances. We also introduced a new index for the characterization of recording and decoding performance of neural interfaces. Finally, by combining an anatomically validated hybrid neural model and discrimination analysis, we provided new evidence suggesting a functional topographical organization of VN fascicles.Significance. This study represents an important step towards the comprehension of VN signaling, paving the way for the development of effective closed-loop VNS systems.
Collapse
Affiliation(s)
- Fabio Vallone
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Matteo Maria Ottaviani
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy.,Institute of Life Sciences, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Francesca Dedola
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Annarita Cutrone
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Simone Romeni
- Bertarelli Foundation Chair in Translational Neural Engineering, Center for Neuroprosthetics and Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| | - Adele Macrí Panarese
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Fabio Bernini
- Institute of Life Sciences, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Marina Cracchiolo
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Ivo Strauss
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Khatia Gabisonia
- Institute of Life Sciences, Scuola Superiore Sant'Anna, Pisa, Italy.,Fondazione Toscana Gabriele Monasterio, Pisa, Italy
| | - Nikoloz Gorgodze
- Institute of Life Sciences, Scuola Superiore Sant'Anna, Pisa, Italy.,Fondazione Toscana Gabriele Monasterio, Pisa, Italy
| | - Alberto Mazzoni
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Fabio A Recchia
- Institute of Life Sciences, Scuola Superiore Sant'Anna, Pisa, Italy.,Fondazione Toscana Gabriele Monasterio, Pisa, Italy.,Department of Physiology, Cardiovascular Research Center, Lewis Katz School of Medicine at Temple University, Philadelphia, PA, United States of America
| | - Silvestro Micera
- The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy.,Bertarelli Foundation Chair in Translational Neural Engineering, Center for Neuroprosthetics and Institute of Bioengineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
21
|
Gamal M, Mousa MH, Eldawlatly S, Elbasiouny SM. In-silico development and assessment of a Kalman filter motor decoder for prosthetic hand control. Comput Biol Med 2021; 132:104353. [PMID: 33831814 PMCID: PMC9887730 DOI: 10.1016/j.compbiomed.2021.104353] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2020] [Revised: 03/14/2021] [Accepted: 03/17/2021] [Indexed: 02/02/2023]
Abstract
Up to 50% of amputees abandon their prostheses, partly due to rapid degradation of the control systems, which require frequent recalibration. The goal of this study was to develop a Kalman filter-based approach to decoding motoneuron activity to identify movement kinematics and thereby provide stable, long-term, accurate, real-time decoding. The Kalman filter-based decoder was examined via biologically varied datasets generated from a high-fidelity computational model of the spinal motoneuron pool. The estimated movement kinematics controlled a simulated MuJoCo prosthetic hand. This clear-box approach showed successful estimation of hand movements under eight varied physiological conditions with no retraining. The mean correlation coefficient of 0.98 and mean normalized root mean square error of 0.06 over these eight datasets provide proof of concept that this decoder would improve long-term integrity of performance while performing new, untrained movements. Additionally, the decoder operated in real-time (~0.3 ms). Further results include robust performance of the Kalman filter when re-trained to more severe post-amputation limitations in the type and number of motoneurons remaining. An additional analysis shows that the decoder achieves better accuracy when using the firing of individual motoneurons as input, compared to using aggregate pool firing. Moreover, the decoder demonstrated robustness to noise affecting both the trained decoder parameters and the decoded motoneuron activity. These results demonstrate the utility of a proof of concept Kalman filter decoder that can support prosthetics' control systems to maintain accurate and stable real-time movement performance.
Collapse
Affiliation(s)
- Mai Gamal
- Center for Informatics Science, Nile University, Giza, Egypt,Computer Science and Engineering Department, Faculty of Media Engineering and Technology, German University in Cairo, Cairo, Egypt
| | - Mohamed H. Mousa
- Department of Biomedical, Industrial, and Human Factors Engineering, Wright State University, Dayton, OH, USA
| | - Seif Eldawlatly
- Computer Science and Engineering Department, Faculty of Media Engineering and Technology, German University in Cairo, Cairo, Egypt,Computer and Systems Engineering Department, Faculty of Engineering, Ain Shams University, Cairo, Egypt
| | - Sherif M. Elbasiouny
- Department of Biomedical, Industrial, and Human Factors Engineering, Wright State University, Dayton, OH, USA,Department of Neuroscience, Cell Biology, and Physiology, Wright State University, Dayton, OH, USA,Corresponding author. 3640 Colonel Glenn Hwy, Dayton, OH, 45435, USA., (S.M. Elbasiouny)
| |
Collapse
|
22
|
Cracchiolo M, Panarese A, Valle G, Strauss I, Granata G, Iorio RD, Stieglitz T, Rossini PM, Mazzoni A, Micera S. Computational approaches to decode grasping force and velocity level in upper-limb amputee from intraneural peripheral signals. J Neural Eng 2021; 18. [PMID: 33725672 DOI: 10.1088/1741-2552/abef3a] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 03/16/2021] [Indexed: 11/12/2022]
Abstract
Objective. Recent results have shown the potentials of neural interfaces to provide sensory feedback to subjects with limb amputation increasing prosthesis usability. However, their advantages for decoding motor control signals over current methods based on electromyography (EMG) are still debated. In this study we compared a standard EMG-based method with approaches that use peripheral intraneural data to infer distinct levels of grasping force and velocity in a trans-radial amputee.Approach. Surface EMG (three channels) and intraneural signals (collected with transverse intrafascicular multichannel electrodes, TIMEs, 56 channels) were simultaneously recorded during the amputee's intended grasping movements. We sorted single unit activity (SUA) from each neural signal and then we identified the most informative units. EMG envelopes were extracted from the recorded EMG signals. A reference support vector machine (SVM) classifier was used to map EMG envelopes into desired force and velocity levels. Two decoding approaches using SUA were then tested and compared to the EMG-based reference classifier: (a) SVM classification of firing rates into desired force and velocity levels; (b) reconstruction of covariates (the grasp cue level or EMG envelopes) from neural data and use of covariates for classification into desired force and velocity levels.Main results.Using EMG envelopes as reconstructed covariates from SUA yielded significantly better results than the other approaches tested, with performance similar to that of the EMG-based reference classifier, and stable over three different recording days. Of the two reconstruction algorithms used in this approach, a linear Kalman filter and a nonlinear point process adaptive filter, the nonlinear filter gave better results.Significance.This study presented a new effective approach for decoding grasping force and velocity from peripheral intraneural signals in a trans-radial amputee, which relies on using SUA to reconstruct EMG envelopes. Being dependent on EMG recordings only for the training phase, this approach can fully exploit the advantages of implanted neural interfaces and potentially overcome, in the medium to long term, current state-of-the-art methods. (Clinical trial's registration number: NCT02848846).
Collapse
Affiliation(s)
- Marina Cracchiolo
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Alessandro Panarese
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Giacomo Valle
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH, Zürich 8006, Switzerland
| | - Ivo Strauss
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Giuseppe Granata
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Riccardo Di Iorio
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Thomas Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering-IMTEK, BrainLinks-BrainTools Center of Excellence & Bernstein Center Freiburg, University of Freiburg, D-79110 Freiburg, Germany
| | - Paolo M Rossini
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Alberto Mazzoni
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Silvestro Micera
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy.,Bertarelli Foundation Chair in Translational Neuroengineering, Centre for Neuroprosthetics and Institute of Bioengineering, School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), Campus Biotech, Chemin des Mines 9, Geneva 1202, Switzerland
| |
Collapse
|
23
|
Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison. J Neuroeng Rehabil 2021; 18:45. [PMID: 33632237 PMCID: PMC7908731 DOI: 10.1186/s12984-021-00839-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2020] [Accepted: 02/14/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Advanced prostheses can restore function and improve quality of life for individuals with amputations. Unfortunately, most commercial control strategies do not fully utilize the rich control information from residual nerves and musculature. Continuous decoders can provide more intuitive prosthesis control using multi-channel neural or electromyographic recordings. Three components influence continuous decoder performance: the data used to train the algorithm, the algorithm, and smoothing filters on the algorithm's output. Individual groups often focus on a single decoder, so very few studies compare different decoders using otherwise similar experimental conditions. METHODS We completed a two-phase, head-to-head comparison of 12 continuous decoders using activities of daily living. In phase one, we compared two training types and a smoothing filter with three algorithms (modified Kalman filter, multi-layer perceptron, and convolutional neural network) in a clothespin relocation task. We compared training types that included only individual digit and wrist movements vs. combination movements (e.g., simultaneous grasp and wrist flexion). We also compared raw vs. nonlinearly smoothed algorithm outputs. In phase two, we compared the three algorithms in fragile egg, zipping, pouring, and folding tasks using the combination training and smoothing found beneficial in phase one. In both phases, we collected objective, performance-based (e.g., success rate), and subjective, user-focused (e.g., preference) measures. RESULTS Phase one showed that combination training improved prosthesis control accuracy and speed, and that the nonlinear smoothing improved accuracy but generally reduced speed. Phase one importantly showed simultaneous movements were used in the task, and that the modified Kalman filter and multi-layer perceptron predicted more simultaneous movements than the convolutional neural network. In phase two, user-focused metrics favored the convolutional neural network and modified Kalman filter, whereas performance-based metrics were generally similar among all algorithms. CONCLUSIONS These results confirm that state-of-the-art algorithms, whether linear or nonlinear in nature, functionally benefit from training on more complex data and from output smoothing. These studies will be used to select a decoder for a long-term take-home trial with implanted neuromyoelectric devices. Overall, clinical considerations may favor the mKF as it is similar in performance, faster to train, and computationally less expensive than neural networks.
Collapse
|
24
|
Fathi Y, Erfanian A. Decoding hindlimb kinematics from descending and ascending neural signals during cat locomotion. J Neural Eng 2021; 18. [PMID: 33395669 DOI: 10.1088/1741-2552/abd82a] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Accepted: 01/04/2021] [Indexed: 11/11/2022]
Abstract
OBJECTIVE The main objective of this research is to record both sensory and motor information from the ascending and descending tracts within the spinal cord for decoding the hindlimb kinematics during walking on the treadmill. APPROACH Two different experimental paradigms (i.e., active and passive) were used in the current study. During active experiments, five cats were trained to walk bipedally while their hands kept on the front frame of the treadmill for balance or to walk quadrupedally. During passive experiments, the limb was passively moved by the experimenter. Local field potential (LFP) activity was recorded using a microwire array implanted in the dorsal column (DC) and lateral column (LC) of the L3-L4 spinal segments. The amplitude and frequency components of the LFP formed the feature set and the elastic net regularization was used to decode the hindlimb joint angles. MAIN RESULTS The results show that there is no significant difference between the information content of the signals recorded from the DC and LC regions during walking on the treadmill, but the information content of the DC is significantly higher than that of the LC during passively applied movement of the hindlimb in the anesthetized cats. Moreover, the decoding performance obtained using the recorded signals from the DC is comparable with that from the LC during locomotion. But, the decoding performance obtained using the recording channels in the DC is significantly better than that obtained using the signals recorded from the LC. The long-term analysis shows that robust decoding performance can be achieved over 2-3 months without a significant decrease in performance. SIGNIFICANCE This work presents a promising approach to developing a natural and robust motor neuroprosthesis device using descending neural signals to execute the movement and ascending neural signals as the feedback information for control of the movement.
Collapse
Affiliation(s)
- Yaser Fathi
- Biomedical Engineering, Iran University of Science and Technology, Narmak, Resalat Square, Hengam Street, Iran University of Science and Technology, Tehran, Tehran, 16844, Iran (the Islamic Republic of)
| | - Abbas Erfanian
- Biomedical Engineering, Iran University of Science & Technology, Hengam Street, Narmak, Tehran 16844, Iran, Tehran, 16844, IRAN, ISLAMIC REPUBLIC OF
| |
Collapse
|