1
|
Farina D, Vujaklija I, Brånemark R, Bull AMJ, Dietl H, Graimann B, Hargrove LJ, Hoffmann KP, Huang HH, Ingvarsson T, Janusson HB, Kristjánsson K, Kuiken T, Micera S, Stieglitz T, Sturma A, Tyler D, Weir RFF, Aszmann OC. Toward higher-performance bionic limbs for wider clinical use. Nat Biomed Eng 2023; 7:473-485. [PMID: 34059810 DOI: 10.1038/s41551-021-00732-x] [Citation(s) in RCA: 94] [Impact Index Per Article: 94.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2019] [Accepted: 04/01/2021] [Indexed: 12/19/2022]
Abstract
Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.
Collapse
Affiliation(s)
- Dario Farina
- Department of Bioengineering, Imperial College London, London, UK.
| | - Ivan Vujaklija
- Department of Electrical Engineering and Automation, Aalto University, Espoo, Finland
| | - Rickard Brånemark
- Center for Extreme Bionics, Biomechatronics Group, MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Anthony M J Bull
- Department of Bioengineering, Imperial College London, London, UK
| | - Hans Dietl
- Ottobock Products SE & Co. KGaA, Vienna, Austria
| | | | - Levi J Hargrove
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Klaus-Peter Hoffmann
- Department of Medical Engineering & Neuroprosthetics, Fraunhofer-Institut für Biomedizinische Technik, Sulzbach, Germany
| | - He Helen Huang
- NCSU/UNC Joint Department of Biomedical Engineering, North Carolina State University, Raleigh, NC, USA
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Thorvaldur Ingvarsson
- Department of Research and Development, Össur Iceland, Reykjavík, Iceland
- Faculty of Medicine, University of Iceland, Reykjavík, Iceland
| | - Hilmar Bragi Janusson
- School of Engineering and Natural Sciences, University of Iceland, Reykjavík, Iceland
| | | | - Todd Kuiken
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, IL, USA
- Department of Physical Medicine & Rehabilitation, Northwestern University, Chicago, IL, USA
- Department of Biomedical Engineering, Northwestern University, Chicago, IL, USA
| | - Silvestro Micera
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pontedera, Italy
- Bertarelli Foundation Chair in Translational NeuroEngineering, Center for Neuroprosthetics and Institute of Bioengineering, School of Engineering, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
| | - Thomas Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering-IMTEK, BrainLinks-BrainTools Center and Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany
| | - Agnes Sturma
- Department of Bioengineering, Imperial College London, London, UK
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| | - Dustin Tyler
- Case School of Engineering, Case Western Reserve University, Cleveland, OH, USA
- Louis Stokes Veterans Affairs Medical Centre, Cleveland, OH, USA
| | - Richard F Ff Weir
- Biomechatronics Development Laboratory, Bioengineering Department, University of Colorado Denver and VA Eastern Colorado Healthcare System, Aurora, CO, USA
| | - Oskar C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic and Reconstructive Surgery, Medical University of Vienna, Vienna, Austria
| |
Collapse
|
2
|
Cracchiolo M, Panarese A, Valle G, Strauss I, Granata G, Iorio RD, Stieglitz T, Rossini PM, Mazzoni A, Micera S. Computational approaches to decode grasping force and velocity level in upper-limb amputee from intraneural peripheral signals. J Neural Eng 2021; 18. [PMID: 33725672 DOI: 10.1088/1741-2552/abef3a] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 03/16/2021] [Indexed: 11/12/2022]
Abstract
Objective. Recent results have shown the potentials of neural interfaces to provide sensory feedback to subjects with limb amputation increasing prosthesis usability. However, their advantages for decoding motor control signals over current methods based on electromyography (EMG) are still debated. In this study we compared a standard EMG-based method with approaches that use peripheral intraneural data to infer distinct levels of grasping force and velocity in a trans-radial amputee.Approach. Surface EMG (three channels) and intraneural signals (collected with transverse intrafascicular multichannel electrodes, TIMEs, 56 channels) were simultaneously recorded during the amputee's intended grasping movements. We sorted single unit activity (SUA) from each neural signal and then we identified the most informative units. EMG envelopes were extracted from the recorded EMG signals. A reference support vector machine (SVM) classifier was used to map EMG envelopes into desired force and velocity levels. Two decoding approaches using SUA were then tested and compared to the EMG-based reference classifier: (a) SVM classification of firing rates into desired force and velocity levels; (b) reconstruction of covariates (the grasp cue level or EMG envelopes) from neural data and use of covariates for classification into desired force and velocity levels.Main results.Using EMG envelopes as reconstructed covariates from SUA yielded significantly better results than the other approaches tested, with performance similar to that of the EMG-based reference classifier, and stable over three different recording days. Of the two reconstruction algorithms used in this approach, a linear Kalman filter and a nonlinear point process adaptive filter, the nonlinear filter gave better results.Significance.This study presented a new effective approach for decoding grasping force and velocity from peripheral intraneural signals in a trans-radial amputee, which relies on using SUA to reconstruct EMG envelopes. Being dependent on EMG recordings only for the training phase, this approach can fully exploit the advantages of implanted neural interfaces and potentially overcome, in the medium to long term, current state-of-the-art methods. (Clinical trial's registration number: NCT02848846).
Collapse
Affiliation(s)
- Marina Cracchiolo
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Alessandro Panarese
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Giacomo Valle
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH, Zürich 8006, Switzerland
| | - Ivo Strauss
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Giuseppe Granata
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Riccardo Di Iorio
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Thomas Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering-IMTEK, BrainLinks-BrainTools Center of Excellence & Bernstein Center Freiburg, University of Freiburg, D-79110 Freiburg, Germany
| | - Paolo M Rossini
- Institute of Neurology, Catholic University of The Sacred Heart, Policlinic A. Gemelli Foundation, Roma, Italy
| | - Alberto Mazzoni
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy
| | - Silvestro Micera
- The Biorobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, viale Rinaldo Piaggio 34, 56025 Pontedera, Italy.,Bertarelli Foundation Chair in Translational Neuroengineering, Centre for Neuroprosthetics and Institute of Bioengineering, School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), Campus Biotech, Chemin des Mines 9, Geneva 1202, Switzerland
| |
Collapse
|
3
|
Mikhaylov A, Pimashkin A, Pigareva Y, Gerasimova S, Gryaznov E, Shchanikov S, Zuev A, Talanov M, Lavrov I, Demin V, Erokhin V, Lobov S, Mukhina I, Kazantsev V, Wu H, Spagnolo B. Neurohybrid Memristive CMOS-Integrated Systems for Biosensors and Neuroprosthetics. Front Neurosci 2020; 14:358. [PMID: 32410943 PMCID: PMC7199501 DOI: 10.3389/fnins.2020.00358] [Citation(s) in RCA: 110] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2019] [Accepted: 03/24/2020] [Indexed: 11/18/2022] Open
Abstract
Here we provide a perspective concept of neurohybrid memristive chip based on the combination of living neural networks cultivated in microfluidic/microelectrode system, metal-oxide memristive devices or arrays integrated with mixed-signal CMOS layer to control the analog memristive circuits, process the decoded information, and arrange a feedback stimulation of biological culture as parts of a bidirectional neurointerface. Our main focus is on the state-of-the-art approaches for cultivation and spatial ordering of the network of dissociated hippocampal neuron cells, fabrication of a large-scale cross-bar array of memristive devices tailored using device engineering, resistive state programming, or non-linear dynamics, as well as hardware implementation of spiking neural networks (SNNs) based on the arrays of memristive devices and integrated CMOS electronics. The concept represents an example of a brain-on-chip system belonging to a more general class of memristive neurohybrid systems for a new-generation robotics, artificial intelligence, and personalized medicine, discussed in the framework of the proposed roadmap for the next decade period.
Collapse
Affiliation(s)
- Alexey Mikhaylov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Alexey Pimashkin
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Yana Pigareva
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | | | - Evgeny Gryaznov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
| | - Sergey Shchanikov
- Department of Information Technologies, Vladimir State University, Murom, Russia
| | - Anton Zuev
- Department of Information Technologies, Vladimir State University, Murom, Russia
| | - Max Talanov
- Neuroscience Laboratory, Kazan Federal University, Kazan, Russia
| | - Igor Lavrov
- Department of Neurologic Surgery, Mayo Clinic, Rochester, MN, United States
- Laboratory of Motor Neurorehabilitation, Kazan Federal University, Kazan, Russia
| | | | - Victor Erokhin
- Neuroscience Laboratory, Kazan Federal University, Kazan, Russia
- Kurchatov Institute, Moscow, Russia
- CNR-Institute of Materials for Electronics and Magnetism, Italian National Research Council, Parma, Italy
| | - Sergey Lobov
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Irina Mukhina
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Cell Technology Group, Privolzhsky Research Medical University, Nizhny Novgorod, Russia
| | - Victor Kazantsev
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Center for Technologies in Robotics and Mechatronics Components, Innopolis University, Innopolis, Russia
| | - Huaqiang Wu
- Institute of Microelectronics, Tsinghua University, Beijing, China
| | - Bernardo Spagnolo
- Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
- Dipartimento di Fisica e Chimica-Emilio Segrè, Group of Interdisciplinary Theoretical Physics, Università di Palermo and CNISM, Unità di Palermo, Palermo, Italy
- Istituto Nazionale di Fisica Nucleare, Sezione di Catania, Catania, Italy
| |
Collapse
|
4
|
Cracchiolo M, Valle G, Petrini F, Strauss I, Granata G, Stieglitz T, Rossini PM, Raspopovic S, Mazzoni A, Micera S. Decoding of grasping tasks from intraneural recordings in trans-radial amputee. J Neural Eng 2020; 17:026034. [PMID: 32207409 DOI: 10.1088/1741-2552/ab8277] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE A major challenge in neuroprosthetics is the restoration of sensory-motor hand functions in upper-limb amputees. Neuroprostheses based on the direct re-connection of the peripheral nerves may be an interesting approach for re-establishing the natural and effective bidirectional control of hand prostheses. Recent results have shown that transverse intrafascicular multi-channel electrodes (TIMEs) can restore natural and sophisticated sensory feedback. However, the potential of using TIME-recorded motor intraneural signals to decode grasping tasks has not as yet been explored. APPROACH In this study, we show that several hand-movement intentions can be decoded from intraneural signals recorded using four TIMEs implanted in the median and ulnar nerves of an upper limb amputee. Experimental sessions were performed over a week, from day 16 to day 23 after the surgical operation. Intraneural activity was recorded during several hand motor tasks imagined by the subject and processed offline. MAIN RESULTS We obtained a very high decoding accuracy considering 11 class states (up to 83%). These results confirm that neural signals recorded by multi-channel intraneural electrodes can be used to decode several movement intentions with high accuracy. Moreover, we were able to use same TIME channels for decoding over one week within the first month, even if the stability has to be confirmed during long-term experiments. SIGNIFICANCE Therefore, TIMEs could be used in the future to achieve a complete bidirectional approach exploiting neural pathways, to make a more natural and intuitive new generation of hand prostheses that have a closer resemblance to a healthy hand.
Collapse
Affiliation(s)
- Marina Cracchiolo
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| | | | | | | | | | | | | | | | | | | |
Collapse
|
5
|
Raspopovic S, Cimolato A, Panarese A, Vallone F, Del Valle J, Micera S, Navarro X. Neural signal recording and processing in somatic neuroprosthetic applications. A review. J Neurosci Methods 2020; 337:108653. [PMID: 32114143 DOI: 10.1016/j.jneumeth.2020.108653] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2019] [Revised: 11/30/2019] [Accepted: 02/26/2020] [Indexed: 12/11/2022]
Abstract
Neurointerfaces have acquired major relevance as both rehabilitative and therapeutic tools for patients with spinal cord injury, limb amputations and other neural disorders. Bidirectional neural interfaces are a key component for the functional control of neuroprosthetic devices. The two main neuroprosthetic applications of interfaces with the peripheral nervous system (PNS) are: the refined control of artificial prostheses with sensory neural feedback, and functional electrical stimulation (FES) systems attempting to generate motor or visceral responses in paralyzed organs. The results obtained in experimental and clinical studies with both, extraneural and intraneural electrodes are very promising in terms of the achieved functionality for the neural stimulation mode. However, the results of neural recordings with peripheral nerve interfaces are more limited. In this paper we review the different existing approaches for PNS signals recording, denoising, processing and classification, enabling their use for bidirectional interfaces. PNS recordings can provide three types of signals: i) population activity signals recorded by using extraneural electrodes placed on the outer surface of the nerve, which carry information about cumulative nerve activity; ii) spike activity signals recorded with intraneural electrodes placed inside the nerve, which carry information about the electrical activity of a set of individual nerve fibers; and iii) hybrid signals, which contain both spiking and cumulative signals. Finally, we also point out some of the main limitations, which are hampering clinical translation of neural decoding, and indicate possible solutions for improvement.
Collapse
Affiliation(s)
- Stanisa Raspopovic
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092, Zürich, Switzerland
| | - Andrea Cimolato
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092, Zürich, Switzerland; NEARLab - Neuroengineering and Medical Robotics Laboratory, DEIB Department of Electronics, Information and Bioengineering, Politecnico Di Milano, 20133, Milano, Italy; IIT Central Research Labs Genova, Istituto Italiano Tecnologia, 16163, Genova, Italy
| | | | - Fabio Vallone
- The BioRobotics Institute, Scuola Superiore Sant'Anna, I-56127, Pisa, Italy
| | - Jaume Del Valle
- Institute of Neurosciences and Department of Cell Biology, Physiology and Immunology, Universitat Autònoma De Barcelona, CIBERNED, 08193, Bellaterra, Spain
| | - Silvestro Micera
- The BioRobotics Institute, Scuola Superiore Sant'Anna, I-56127, Pisa, Italy; Translational Neural Engineering Laboratory, Center for Neuroprosthetics and Institute of Bioengineering, Ecole Polytechnique Federale De Lausanne, Lausanne, CH-1015, Switzerland.
| | - Xavier Navarro
- Institute of Neurosciences and Department of Cell Biology, Physiology and Immunology, Universitat Autònoma De Barcelona, CIBERNED, 08193, Bellaterra, Spain; Institut Guttmann De Neurorehabilitació, Badalona, Spain.
| |
Collapse
|
6
|
Bidirectional Control of Myoelectric Prostheses in Upper Limb Amputees: Current Results and Expectations. SERBIAN JOURNAL OF EXPERIMENTAL AND CLINICAL RESEARCH 2019. [DOI: 10.2478/sjecr-2019-0065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
Abstract
The most common causes of upper-limb amputations include traumatic etiology and malignity, followed by peripheral vascular diseases. Prosthetic fitting along with conducting a rehabilitation program provide the greatest possible degree of independence to the upper-limb amputees in performing their daily tasks, occupational, recreational and work activities. Despite recent advance in strategies of design and control, the lack of sensory feedback is, according to patients, one of the most important characteristics lacked by commercial myoelectric prostheses. This reason has led to the need for the development of comprehensive prosthetic part which would provide intuitive control and realistic sensory feedback to the amputees enabling them thus to more easily accomplish the tasks which are essential for easier performance of activities of daily life. Electromyography, and recently, electroneurography signals have been used for the development of more efficacious upper-limb prosthetic control. Several recent studies have demonstrated the efficacy of homologous and somatotopic approach in upper-limb amputees, by applying implanted and surface electrodes. This work presents novel methods for effective bidirectional control of myoelectric prostheses in patients with upper-limb amputations using motor control and sensory feedback. The above-mentioned approaches are applicable and have good prospects in further clinical use. The intraneural, extraneural and surface approach can be more or less applicable depending on the etiology and the level of amputation. From a clinical point of view, various approaches should be combined for obtaining more efficient control of bidirectional prostheses and corresponding sensory feedback.
Collapse
|