1
|
Huang HH, Hargrove LJ, Ortiz-Catalan M, Sensinger JW. Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction. Annu Rev Biomed Eng 2024; 26:503-528. [PMID: 38594922 DOI: 10.1146/annurev-bioeng-110222-095816] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/11/2024]
Abstract
Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.
Collapse
Affiliation(s)
- He Helen Huang
- Joint Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
- Joint Department of Biomedical Engineering, North Carolina State University, Raleigh, North Carolina, USA;
| | - Levi J Hargrove
- Department of Physical Medicine and Rehabilitation, Northwestern University, Chicago, Illinois, USA
- Center for Bionic Medicine, Shirley Ryan AbilityLab, Chicago, Illinois, USA
| | - Max Ortiz-Catalan
- Medical Bionics Department, University of Melbourne, Melbourne, Australia
- Bionics Institute, Melbourne, Australia
| | - Jonathon W Sensinger
- Institute of Biomedical Engineering, University of New Brunswick, Fredericton, New Brunswick, Canada;
| |
Collapse
|
2
|
Festin C, Ortmayr J, Maierhofer U, Tereshenko V, Blumer R, Schmoll M, Carrero-Rojas G, Luft M, Laengle G, Farina D, Bergmeister KD, Aszmann OC. Creation of a biological sensorimotor interface for bionic reconstruction. Nat Commun 2024; 15:5337. [PMID: 38914540 PMCID: PMC11196281 DOI: 10.1038/s41467-024-49580-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 06/12/2024] [Indexed: 06/26/2024] Open
Abstract
Neuromuscular control of bionic arms has constantly improved over the past years, however, restoration of sensation remains elusive. Previous approaches to reestablish sensory feedback include tactile, electrical, and peripheral nerve stimulation, however, they cannot recreate natural, intuitive sensations. Here, we establish an experimental biological sensorimotor interface and demonstrate its potential use in neuroprosthetics. We transfer a mixed nerve to a skeletal muscle combined with glabrous dermal skin transplantation, thus forming a bi-directional communication unit in a rat model. Morphological analyses indicate reinnervation of the skin, mechanoreceptors, NMJs, and muscle spindles. Furthermore, sequential retrograde labeling reveals specific sensory reinnervation at the level of the dorsal root ganglia. Electrophysiological recordings show reproducible afferent signals upon tactile stimulation and tendon manipulation. The results demonstrate the possibility of surgically creating an interface for both decoding efferent motor control, as well as encoding afferent tactile and proprioceptive feedback, and may indicate the way forward regarding clinical translation of biological communication pathways for neuroprosthetic applications.
Collapse
Affiliation(s)
- Christopher Festin
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Center for Biomedical Research, Medical University of Vienna, Vienna, Austria
| | - Joachim Ortmayr
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
| | - Udo Maierhofer
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Center for Biomedical Research, Medical University of Vienna, Vienna, Austria
| | - Vlad Tereshenko
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Center for Biomedical Research, Medical University of Vienna, Vienna, Austria
- Division of Plastic and Reconstructive Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Roland Blumer
- Center for Anatomy and Cell Biology, Medical University of Vienna, Vienna, Austria
| | - Martin Schmoll
- Center for Medical Physics and Biomedical Engineering, Medical University of Vienna, Vienna, Austria
| | - Génova Carrero-Rojas
- Center for Anatomy and Cell Biology, Medical University of Vienna, Vienna, Austria
| | - Matthias Luft
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Center for Biomedical Research, Medical University of Vienna, Vienna, Austria
- Department of Plastic, Aesthetic and Reconstructive Surgery, University Hospital St. Poelten, Karl Landsteiner University of Health Sciences, Krems, Austria
| | - Gregor Laengle
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Center for Biomedical Research, Medical University of Vienna, Vienna, Austria
- Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
| | - Dario Farina
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | - Konstantin D Bergmeister
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Department of Plastic, Aesthetic and Reconstructive Surgery, University Hospital St. Poelten, Karl Landsteiner University of Health Sciences, Krems, Austria
| | - Oskar C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria.
- Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria.
| |
Collapse
|
3
|
Sagastegui Alva PG, Boesendorfer A, Aszmann OC, Ibáñez J, Farina D. Excitation of natural spinal reflex loops in the sensory-motor control of hand prostheses. Sci Robot 2024; 9:eadl0085. [PMID: 38809994 DOI: 10.1126/scirobotics.adl0085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2023] [Accepted: 04/30/2024] [Indexed: 05/31/2024]
Abstract
Sensory feedback for prosthesis control is typically based on encoding sensory information in specific types of sensory stimuli that the users interpret to adjust the control of the prosthesis. However, in physiological conditions, the afferent feedback received from peripheral nerves is not only processed consciously but also modulates spinal reflex loops that contribute to the neural information driving muscles. Spinal pathways are relevant for sensory-motor integration, but they are commonly not leveraged for prosthesis control. We propose an approach to improve sensory-motor integration for prosthesis control based on modulating the excitability of spinal circuits through the vibration of tendons in a closed loop with muscle activity. We measured muscle signals in healthy participants and amputees during different motor tasks, and we closed the loop by applying vibration on tendons connected to the muscles, which modulated the excitability of motor neurons. The control signals to the prosthesis were thus the combination of voluntary control and additional spinal reflex inputs induced by tendon vibration. Results showed that closed-loop tendon vibration was able to modulate the neural drive to the muscles. When closed-loop tendon vibration was used, participants could achieve similar or better control performance in interfaces using muscle activation than without stimulation. Stimulation could even improve prosthetic grasping in amputees. Overall, our results indicate that closed-loop tendon vibration can integrate spinal reflex pathways in the myocontrol system and open the possibility of incorporating natural feedback loops in prosthesis control.
Collapse
Affiliation(s)
| | - Anna Boesendorfer
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
| | - Oskar C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
- Department of Plastic, Reconstructive and Aesthetic Surgery, Medical University of Vienna, Vienna, Austria
| | - Jaime Ibáñez
- Department of Bioengineering, Imperial College London, London, UK
- BSICoS group, I3A Institute, University of Zaragoza, IIS Aragón, Zaragoza, Spain
| | - Dario Farina
- Department of Bioengineering, Imperial College London, London, UK
| |
Collapse
|
4
|
Williams HE, Shehata AW, Cheng KY, Hebert JS, Pilarski PM. A multifaceted suite of metrics for comparative myoelectric prosthesis controller research. PLoS One 2024; 19:e0291279. [PMID: 38739557 PMCID: PMC11090368 DOI: 10.1371/journal.pone.0291279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 02/15/2024] [Indexed: 05/16/2024] Open
Abstract
Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop person-specific prosthesis controllers that can predict a user's intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users' intentions; the timing of specific wrist and hand control functions; and users' opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details-spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the "limb position effect". We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.
Collapse
Affiliation(s)
- Heather E. Williams
- Department of Biomedical Engineering, University of Alberta, Edmonton, AB, Canada
- Alberta Machine Intelligence Institute (Amii), Edmonton, AB, Canada
| | - Ahmed W. Shehata
- Department of Biomedical Engineering, University of Alberta, Edmonton, AB, Canada
| | - Kodi Y. Cheng
- Department of Biomedical Engineering, University of Alberta, Edmonton, AB, Canada
| | - Jacqueline S. Hebert
- Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta, Edmonton, AB, Canada
| | - Patrick M. Pilarski
- Alberta Machine Intelligence Institute (Amii), Edmonton, AB, Canada
- Division of Physical Medicine and Rehabilitation, Department of Medicine, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
5
|
Lavoie E, Hebert JS, Chapman CS. Comparing eye-hand coordination between controller-mediated virtual reality, and a real-world object interaction task. J Vis 2024; 24:9. [PMID: 38393742 PMCID: PMC10905649 DOI: 10.1167/jov.24.2.9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Accepted: 11/30/2023] [Indexed: 02/25/2024] Open
Abstract
Virtual reality (VR) technology has advanced significantly in recent years, with many potential applications. However, it is unclear how well VR simulations mimic real-world experiences, particularly in terms of eye-hand coordination. This study compares eye-hand coordination from a previously validated real-world object interaction task to the same task re-created in controller-mediated VR. We recorded eye and body movements and segmented participants' gaze data using the movement data. In the real-world condition, participants wore a head-mounted eye tracker and motion capture markers and moved a pasta box into and out of a set of shelves. In the VR condition, participants wore a VR headset and moved a virtual box using handheld controllers. Unsurprisingly, VR participants took longer to complete the task. Before picking up or dropping off the box, participants in the real world visually fixated the box about half a second before their hand arrived at the area of action. This 500-ms minimum fixation time before the hand arrived was preserved in VR. Real-world participants disengaged their eyes from the box almost immediately after their hand initiated or terminated the interaction, but VR participants stayed fixated on the box for much longer after it was picked up or dropped off. We speculate that the limited haptic feedback during object interactions in VR forces users to maintain visual fixation on objects longer than in the real world, altering eye-hand coordination. These findings suggest that current VR technology does not replicate real-world experience in terms of eye-hand coordination.
Collapse
Affiliation(s)
- Ewen Lavoie
- Faculty of Kinesiology, Sport, and Recreation, Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| | - Jacqueline S Hebert
- Division of Physical Medicine and Rehabilitation, Department of Biomedical Engineering, University of Alberta, Edmonton, AB, Canada
- Glenrose Rehabiliation Hospital, Alberta Health Services, Edmonton, AB, Canada
| | - Craig S Chapman
- Faculty of Kinesiology, Sport, and Recreation, Neuroscience and Mental Health Institute, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
6
|
Tao Q, Chao H, Fang D, Dou D. Progress in neurorehabilitation research and the support by the National Natural Science Foundation of China from 2010 to 2022. Neural Regen Res 2024; 19:226-232. [PMID: 37488871 PMCID: PMC10479845 DOI: 10.4103/1673-5374.375342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Revised: 03/27/2023] [Accepted: 04/14/2023] [Indexed: 07/26/2023] Open
Abstract
The National Natural Science Foundation of China is one of the major funding agencies for neurorehabilitation research in China. This study reviews the frontier directions and achievements in the field of neurorehabilitation in China and worldwide. We used data from the Web of Science Core Collection (WoSCC) database to analyze the publications and data provided by the National Natural Science Foundation of China to analyze funding information. In addition, the prospects for neurorehabilitation research in China are discussed. From 2010 to 2022, a total of 74,220 publications in neurorehabilitation were identified, with there being an overall upward tendency. During this period, the National Natural Science Foundation of China has funded 476 research projects with a total funding of 192.38 million RMB to support neurorehabilitation research in China. With the support of the National Natural Science Foundation of China, China has made some achievements in neurorehabilitation research. Research related to neurorehabilitation is believed to be making steady and significant progress in China.
Collapse
Affiliation(s)
- Qian Tao
- School of Medicine, Jinan University, Guangzhou, Guangdong Province, China
- School of Health and Life Science, University of Health and Rehabilitation Sciences, Qingdao, Shandong Province, China
- Department of Health Sciences, National Natural Science Foundation of China, Beijing, China
| | - Honglu Chao
- Department of Health Sciences, National Natural Science Foundation of China, Beijing, China
- Department of Neurosurgery, The First Affiliated Hospital of Nanjing Medical University, Nanjing, Jiangsu Province, China
| | - Dong Fang
- Department of Health Sciences, National Natural Science Foundation of China, Beijing, China
| | - Dou Dou
- Department of Health Sciences, National Natural Science Foundation of China, Beijing, China
| |
Collapse
|
7
|
Zhang J, Chou CH, Hao M, Li Y, Yu Y, Lan N. Fusion of dual modalities of non-invasive sensory feedback for object profiling with prosthetic hands. Front Neurorobot 2023; 17:1298176. [PMID: 38162892 PMCID: PMC10757719 DOI: 10.3389/fnbot.2023.1298176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2023] [Accepted: 11/24/2023] [Indexed: 01/03/2024] Open
Abstract
Introduction Either non-invasive somatotopic or substitute sensory feedback is capable of conveying a single modality of sensory information from prosthetic hands to amputees. However, the neurocognitive ability of amputees to integrate multi-modality sensory information for functional discrimination is unclear. The purpose of this study was to assess the fusion of non-invasive somatotopic tactile and substitute aperture feedbacks for profile perception of multiple physical features during grasping objects. Methods Two left transradial amputees with somatotopic evoked tactile sensation (ETS) of five fingers participated in the study. The tactile information of prosthetic hand was provided to amputees by the ETS feedback elicited on the stump projected finger map. Hand aperture information was conveyed to amputees with substitute electrotactile stimulation on the forearm or upper arm. Two types of sensory feedback were integrated to a commercial prosthetic hand. The efficacy of somatotopic ETS feedback on object length identification task was assessed with or without substitute aperture stimulation. The object size identification task was utilized to assess how ETS stimulation at the stump may affect aperture perception with stimulation on the ipsilateral upper arm or forearm. Finally, the task of identifying combined length and size was conducted to evaluate the ability of amputees to integrate the dual modalities of sensory feedback for perceiving profile features. Results The study revealed that amputee subjects can effectively integrate the ETS feedback with electrotactile substitutive feedback for object profile discrimination. Specifically, ETS was robust to provide object length information with electrotactile stimulation at either the forearm or upper arm. However, electrotactile stimulation at the upper arm for aperture perception was less susceptible to the interference of ETS stimulation than at the forearm. Discussion Amputee subjects are able to combine somatotopic ETS and aperture feedbacks for identifying multi-dimensional features in object profiling. The two sensory streams of information can be fused effectively without mutual interference for functional discrimination.
Collapse
Affiliation(s)
- Jie Zhang
- Laboratory of NeuroRehabilitation Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Chih-Hong Chou
- Laboratory of NeuroRehabilitation Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
- Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Manzhao Hao
- Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yan Li
- Laboratory of NeuroRehabilitation Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Yashuo Yu
- Laboratory of NeuroRehabilitation Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Ning Lan
- Laboratory of NeuroRehabilitation Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
- Institute of Medical Robotics, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| |
Collapse
|
8
|
Dominijanni G, Pinheiro DL, Pollina L, Orset B, Gini M, Anselmino E, Pierella C, Olivier J, Shokur S, Micera S. Human motor augmentation with an extra robotic arm without functional interference. Sci Robot 2023; 8:eadh1438. [PMID: 38091424 DOI: 10.1126/scirobotics.adh1438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 11/15/2023] [Indexed: 12/18/2023]
Abstract
Extra robotic arms (XRAs) are gaining interest in neuroscience and robotics, offering potential tools for daily activities. However, this compelling opportunity poses new challenges for sensorimotor control strategies and human-machine interfaces (HMIs). A key unsolved challenge is allowing users to proficiently control XRAs without hindering their existing functions. To address this, we propose a pipeline to identify suitable HMIs given a defined task to accomplish with the XRA. Following such a scheme, we assessed a multimodal motor HMI based on gaze detection and diaphragmatic respiration in a purposely designed modular neurorobotic platform integrating virtual reality and a bilateral upper limb exoskeleton. Our results show that the proposed HMI does not interfere with speaking or visual exploration and that it can be used to control an extra virtual arm independently from the biological ones or in coordination with them. Participants showed significant improvements in performance with daily training and retention of learning, with no further improvements when artificial haptic feedback was provided. As a final proof of concept, naïve and experienced participants used a simplified version of the HMI to control a wearable XRA. Our analysis indicates how the presented HMI can be effectively used to control XRAs. The observation that experienced users achieved a success rate 22.2% higher than that of naïve users, combined with the result that naïve users showed average success rates of 74% when they first engaged with the system, endorses the viability of both the virtual reality-based testing and training and the proposed pipeline.
Collapse
Affiliation(s)
- Giulia Dominijanni
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Daniel Leal Pinheiro
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- Neuroengineering and Neurocognition Laboratory, Escola Paulista de Medicina, Department of Neurology and Neurosurgery, Division of Neuroscience, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Leonardo Pollina
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Bastien Orset
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Martina Gini
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
- Neuroelectronic Interfaces, Faculty of Electrical Engineering and IT, Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen, Aachen 52074, Germany
| | - Eugenio Anselmino
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Camilla Pierella
- Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, and Maternal and Children's Sciences (DINOGMI), University of Genoa, Genoa, Italy
| | - Jérémy Olivier
- Institute for Industrial Sciences and Technologies, Haute Ecole du Paysage, d'Ingénierie et d'Architecture (HEPIA), HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland
| | - Solaiman Shokur
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Silvestro Micera
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| |
Collapse
|
9
|
Siegel JR, Battraw MA, Winslow EJ, James MA, Joiner WM, Schofield JS. Review and critique of current testing protocols for upper-limb prostheses: a call for standardization amidst rapid technological advancements. Front Robot AI 2023; 10:1292632. [PMID: 38035123 PMCID: PMC10684749 DOI: 10.3389/frobt.2023.1292632] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Accepted: 10/30/2023] [Indexed: 12/02/2023] Open
Abstract
This article provides a comprehensive narrative review of physical task-based assessments used to evaluate the multi-grasp dexterity and functional impact of varying control systems in pediatric and adult upper-limb prostheses. Our search returned 1,442 research articles from online databases, of which 25 tests-selected for their scientific rigor, evaluation metrics, and psychometric properties-met our review criteria. We observed that despite significant advancements in the mechatronics of upper-limb prostheses, these 25 assessments are the only validated evaluation methods that have emerged since the first measure in 1948. This not only underscores the lack of a consistently updated, standardized assessment protocol for new innovations, but also reveals an unsettling trend: as technology outpaces standardized evaluation measures, developers will often support their novel devices through custom, study-specific tests. These boutique assessments can potentially introduce bias and jeopardize validity. Furthermore, our analysis revealed that current validated evaluation methods often overlook the influence of competing interests on test success. Clinical settings and research laboratories differ in their time constraints, access to specialized equipment, and testing objectives, all of which significantly influence assessment selection and consistent use. Therefore, we propose a dual testing approach to address the varied demands of these distinct environments. Additionally, we found that almost all existing task-based assessments lack an integrated mechanism for collecting patient feedback, which we assert is essential for a holistic evaluation of upper-limb prostheses. Our review underscores the pressing need for a standardized evaluation protocol capable of objectively assessing the rapidly advancing prosthetic technologies across all testing domains.
Collapse
Affiliation(s)
- Joshua R. Siegel
- Department of Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA, United States
| | - Marcus A. Battraw
- Department of Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA, United States
| | - Eden J. Winslow
- Department of Biomedical Engineering, University of California, Davis, Davis, CA, United States
| | - Michelle A. James
- Shriners Hospital for Children, Northern California, Sacramento, Sacramento, CA, United States
| | - Wilsaan M. Joiner
- Department of Neurobiology, Physiology and Behavior, University of California, Davis, Davis, CA, United States
- Department of Neurology, University of California, Davis, Davis, CA, United States
| | - Jonathon S. Schofield
- Department of Mechanical and Aerospace Engineering, University of California, Davis, Davis, CA, United States
| |
Collapse
|
10
|
Lin C, Zhang C, Xu J, Liu R, Leng Y, Fu C. Neural Correlation of EEG and Eye Movement in Natural Grasping Intention Estimation. IEEE Trans Neural Syst Rehabil Eng 2023; 31:4329-4337. [PMID: 37883284 DOI: 10.1109/tnsre.2023.3327907] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2023]
Abstract
Decoding the user's natural grasp intent enhances the application of wearable robots, improving the daily lives of individuals with disabilities. Electroencephalogram (EEG) and eye movements are two natural representations when users generate grasp intent in their minds, with current studies decoding human intent by fusing EEG and eye movement signals. However, the neural correlation between these two signals remains unclear. Thus, this paper aims to explore the consistency between EEG and eye movement in natural grasping intention estimation. Specifically, six grasp intent pairs are decoded by combining feature vectors and utilizing the optimal classifier. Extensive experimental results indicate that the coupling between the EEG and eye movements intent patterns remains intact when the user generates a natural grasp intent, and concurrently, the EEG pattern is consistent with the eye movements pattern across the task pairs. Moreover, the findings reveal a solid connection between EEG and eye movements even when taking into account cortical EEG (originating from the visual cortex or motor cortex) and the presence of a suboptimal classifier. Overall, this work uncovers the coupling correlation between EEG and eye movements and provides a reference for intention estimation.
Collapse
|
11
|
Kim K, Yang H, Lee J, Lee WG. Metaverse Wearables for Immersive Digital Healthcare: A Review. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2023; 10:e2303234. [PMID: 37740417 PMCID: PMC10625124 DOI: 10.1002/advs.202303234] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 07/15/2023] [Indexed: 09/24/2023]
Abstract
The recent exponential growth of metaverse technology has been instrumental in reshaping a myriad of sectors, not least digital healthcare. This comprehensive review critically examines the landscape and future applications of metaverse wearables toward immersive digital healthcare. The key technologies and advancements that have spearheaded the metamorphosis of metaverse wearables are categorized, encapsulating all-encompassed extended reality, such as virtual reality, augmented reality, mixed reality, and other haptic feedback systems. Moreover, the fundamentals of their deployment in assistive healthcare (especially for rehabilitation), medical and nursing education, and remote patient management and treatment are investigated. The potential benefits of integrating metaverse wearables into healthcare paradigms are multifold, encompassing improved patient prognosis, enhanced accessibility to high-quality care, and high standards of practitioner instruction. Nevertheless, these technologies are not without their inherent challenges and untapped opportunities, which span privacy protection, data safeguarding, and innovation in artificial intelligence. In summary, future research trajectories and potential advancements to circumvent these hurdles are also discussed, further augmenting the incorporation of metaverse wearables within healthcare infrastructures in the post-pandemic era.
Collapse
Affiliation(s)
- Kisoo Kim
- Intelligent Optical Module Research CenterKorea Photonics Technology Institute (KOPTI)Gwangju61007Republic of Korea
| | - Hyosill Yang
- Department of NursingCollege of Nursing ScienceKyung Hee UniversitySeoul02447Republic of Korea
| | - Jihun Lee
- Department of Mechanical EngineeringCollege of EngineeringKyung Hee UniversityYongin17104Republic of Korea
| | - Won Gu Lee
- Department of Mechanical EngineeringCollege of EngineeringKyung Hee UniversityYongin17104Republic of Korea
| |
Collapse
|
12
|
Segas E, Mick S, Leconte V, Dubois O, Klotz R, Cattaert D, de Rugy A. Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality. eLife 2023; 12:RP87317. [PMID: 37847150 PMCID: PMC10581689 DOI: 10.7554/elife.87317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2023] Open
Abstract
Impressive progress is being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in people with transhumeral limb loss based on proximal residual limb motion and knowledge of the movement goal. This control was validated on 29 participants, including seven with above-elbow limb loss, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. This control also enabled 15 participants, including three with limb differences, to reach and grasp real objects with a robotic arm operated according to the same principle. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment the reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.
Collapse
Affiliation(s)
- Effie Segas
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
| | - Sébastien Mick
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
- ISIR UMR 7222, Sorbonne Université, CNRS, InsermParisFrance
| | | | - Océane Dubois
- Univ. Bordeaux, CNRS, INCIA, UMR 5287BordeauxFrance
- ISIR UMR 7222, Sorbonne Université, CNRS, InsermParisFrance
| | | | | | | |
Collapse
|
13
|
Kamsma TM, Boon WQ, Spitoni C, van Roij R. Unveiling the capabilities of bipolar conical channels in neuromorphic iontronics. Faraday Discuss 2023; 246:125-140. [PMID: 37404026 PMCID: PMC10568261 DOI: 10.1039/d3fd00022b] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Accepted: 03/07/2023] [Indexed: 07/06/2023]
Abstract
Conical channels filled with an aqueous electrolyte have been proposed as promising candidates for iontronic neuromorphic circuits. This is facilitated by a novel analytical model for the internal channel dynamics [T. M. Kamsma, W. Q. Boon, T. ter Rele, C. Spitoni and R. van Roij, Phys. Rev. Lett., 2023, 130(26), 268401], the relative ease of fabrication of conical channels, and the wide range of achievable memory retention times by varying the channel lengths. In this work, we demonstrate that the analytical model for conical channels can be generalized to channels with an inhomogeneous surface charge distribution, which we predict to exhibit significantly stronger current rectification and more pronounced memristive properties in the case of bipolar channels, i.e. channels where the tip and base carry a surface charge of opposite sign. Additionally, we show that the use of bipolar conical channels in a previously proposed iontronic circuit features hallmarks of neuronal communication, such as all-or-none action potentials and spike train generation. Bipolar channels allow, however, for circuit parameters in the range of their biological analogues, and exhibit membrane potentials that match well with biological mammalian action potentials, further supporting their potential biocompatibility.
Collapse
Affiliation(s)
- T M Kamsma
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands.
- Mathematical Institute, Utrecht University, Budapestlaan 6, 3584 CD Utrecht, The Netherlands
| | - W Q Boon
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands.
| | - C Spitoni
- Mathematical Institute, Utrecht University, Budapestlaan 6, 3584 CD Utrecht, The Netherlands
| | - R van Roij
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands.
| |
Collapse
|
14
|
Peternell G, Penasso H, Luttenberger H, Ronacher H, Schlintner R, Ashcraft K, Gardetto A, Ernst J, Kropiunig U. Vibrotactile Feedback for a Person with Transradial Amputation and Visual Loss: A Case Report. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1710. [PMID: 37893428 PMCID: PMC10608000 DOI: 10.3390/medicina59101710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 09/02/2023] [Accepted: 09/22/2023] [Indexed: 10/29/2023]
Abstract
Background and Objectives: After major upper-limb amputation, people face challenges due to losing tactile information and gripping function in their hands. While vision can confirm the success of an action, relying on it diverts attention from other sensations and tasks. This case report presents a 30-year-old man with traumatic, complete vision loss and transradial left forearm amputation. It emphasizes the importance of restoring tactile abilities when visual compensation is impossible. Materials and Methods: A prototype tactile feedback add-on system was developed, consisting of a sensor glove and upper arm cuff with related vibration actuators. Results: We found a 66% improvement in the Box and Blocks test and an overall functional score increase from 30% to 43% in the Southampton Hand Assessment Procedure with feedback. Qualitative improvements in bimanual activities, ergonomics, and reduced reliance on the unaffected hand were observed. Incorporating the tactile feedback system improved the precision of grasping and the utility of the myoelectric hand prosthesis, freeing the unaffected hand for other tasks. Conclusions: This case demonstrated improvements in prosthetic hand utility achieved by restoring peripheral sensitivity while excluding the possibility of visual compensation. Restoring tactile information from the hand and fingers could benefit individuals with impaired vision and somatosensation, improving acceptance, embodiment, social integration, and pain management.
Collapse
Affiliation(s)
- Gerfried Peternell
- Rehabilitation Clinic Tobelbad, Austrian Workers’ Compensation Board (AUVA), 8144 Tobelbad, Austria
- Ludwig Boltzmann Institute for Traumatology, 1200 Vienna, Austria
| | - Harald Penasso
- Ludwig Boltzmann Institute for Traumatology, 1200 Vienna, Austria
- Saphenus Medical Technology GmbH, 2500 Baden, Austria
| | - Henriette Luttenberger
- Rehabilitation Clinic Tobelbad, Austrian Workers’ Compensation Board (AUVA), 8144 Tobelbad, Austria
| | - Hildegard Ronacher
- Rehabilitation Clinic Tobelbad, Austrian Workers’ Compensation Board (AUVA), 8144 Tobelbad, Austria
| | | | - Kara Ashcraft
- Department of Integrative Physiology, University of Colorado Boulder, Boulder, CO 80309, USA;
| | - Alexander Gardetto
- Department of Plastic, Aesthetic and Reconstructive Surgery with Hand Surgery and Competence Center for Bionic Prosthetics, Brixsana Private Clinic, 39042 Bressanone, Italy;
| | - Jennifer Ernst
- Department of Trauma Surgery, Hannover Medical School, 30625 Hanover, Germany;
| | - Ursula Kropiunig
- Rehabilitation Clinic Tobelbad, Austrian Workers’ Compensation Board (AUVA), 8144 Tobelbad, Austria
| |
Collapse
|
15
|
Tchimino J, Dideriksen JL, Dosen S. EMG feedback improves grasping of compliant objects using a myoelectric prosthesis. J Neuroeng Rehabil 2023; 20:119. [PMID: 37705008 PMCID: PMC10500847 DOI: 10.1186/s12984-023-01237-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Accepted: 08/24/2023] [Indexed: 09/15/2023] Open
Abstract
BACKGROUND Closing the control loop in myoelectric prostheses by providing artificial somatosensory feedback is recognized as an important goal. However, designing a feedback interface that is effective in realistic conditions is still a challenge. Namely, in some situations, feedback can be redundant, as the information it provides can be readily obtained through hearing or vision (e.g., grasping force estimated from the deformation of a compliant object). EMG feedback is a non-invasive method wherein the tactile stimulation conveys to the user the level of their own myoelectric signal, hence a measurement intrinsic to the interface, which cannot be accessed incidentally. METHODS The present study investigated the efficacy of EMG feedback in prosthesis force control when 10 able-bodied participants and a person with transradial amputation used a myoelectric prosthesis to grasp compliant objects of different stiffness values. The performance with feedback was compared to that achieved when the participants relied solely on incidental cues. RESULTS The main outcome measures were the task success rate and completion time. EMG feedback resulted in significantly higher success rates regardless of pin stiffness, indicating that the feedback enhanced the accuracy of force application despite the abundance of incidental cues. Contrary to expectations, there was no difference in the completion time between the two feedback conditions. Additionally, the data revealed that the participants could produce smoother control signals when they received EMG feedback as well as more consistent commands across trials, signifying better control of the system by the participants. CONCLUSIONS The results presented in this study further support the efficacy of EMG feedback when closing the prosthesis control loop by demonstrating its benefits in particularly challenging conditions which maximized the utility of intrinsic feedback sources.
Collapse
Affiliation(s)
- Jack Tchimino
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Jakob Lund Dideriksen
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Strahinja Dosen
- Neurorehabilitation Systems, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
16
|
Papaleo ED, D'Alonzo M, Fiori F, Piombino V, Falato E, Pilato F, De Liso A, Di Lazzaro V, Di Pino G. Integration of proprioception in upper limb prostheses through non-invasive strategies: a review. J Neuroeng Rehabil 2023; 20:118. [PMID: 37689701 PMCID: PMC10493033 DOI: 10.1186/s12984-023-01242-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 08/24/2023] [Indexed: 09/11/2023] Open
Abstract
Proprioception plays a key role in moving our body dexterously and effortlessly. Nevertheless, the majority of investigations evaluating the benefits of providing supplemental feedback to prosthetics users focus on delivering touch restitution. These studies evaluate the influence of touch sensation in an attempt to improve the controllability of current robotic devices. Contrarily, investigations evaluating the capabilities of proprioceptive supplemental feedback have yet to be comprehensively analyzed to the same extent, marking a major gap in knowledge within the current research climate. The non-invasive strategies employed so far to restitute proprioception are reviewed in this work. In the absence of a clearly superior strategy, approaches employing vibrotactile, electrotactile and skin-stretch stimulation achieved better and more consistent results, considering both kinesthetic and grip force information, compared with other strategies or any incidental feedback. Although emulating the richness of the physiological sensory return through artificial feedback is the primary hurdle, measuring its effects to eventually support the integration of cumbersome and energy intensive hardware into commercial prosthetic devices could represent an even greater challenge. Thus, we analyze the strengths and limitations of previous studies and discuss the possible benefits of coupling objective measures, like neurophysiological parameters, as well as measures of prosthesis embodiment and cognitive load with behavioral measures of performance. Such insights aim to provide additional and collateral outcomes to be considered in the experimental design of future investigations of proprioception restitution that could, in the end, allow researchers to gain a more detailed understanding of possibly similar behavioral results and, thus, support one strategy over another.
Collapse
Affiliation(s)
- Ermanno Donato Papaleo
- Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (NeXTlab), Università Campus Bio-Medico Di Roma, Via Álvaro Del Portillo 21, 00128, Rome, Italy
| | - Marco D'Alonzo
- Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (NeXTlab), Università Campus Bio-Medico Di Roma, Via Álvaro Del Portillo 21, 00128, Rome, Italy
| | - Francesca Fiori
- Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (NeXTlab), Università Campus Bio-Medico Di Roma, Via Álvaro Del Portillo 21, 00128, Rome, Italy
| | - Valeria Piombino
- Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (NeXTlab), Università Campus Bio-Medico Di Roma, Via Álvaro Del Portillo 21, 00128, Rome, Italy
| | - Emma Falato
- Research Unit of Neurology, Department of Medicine and Surgery, Università Campus Bio-Medico Di Roma, Via Alvaro del Portillo, 21, 00128, Rome, Italy
- Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128, Rome, Italy
| | - Fabio Pilato
- Research Unit of Neurology, Department of Medicine and Surgery, Università Campus Bio-Medico Di Roma, Via Alvaro del Portillo, 21, 00128, Rome, Italy
- Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128, Rome, Italy
| | - Alfredo De Liso
- Research Unit of Neurology, Department of Medicine and Surgery, Università Campus Bio-Medico Di Roma, Via Alvaro del Portillo, 21, 00128, Rome, Italy
- Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128, Rome, Italy
| | - Vincenzo Di Lazzaro
- Research Unit of Neurology, Department of Medicine and Surgery, Università Campus Bio-Medico Di Roma, Via Alvaro del Portillo, 21, 00128, Rome, Italy
- Fondazione Policlinico Universitario Campus Bio-Medico, Via Alvaro del Portillo 200, 00128, Rome, Italy
| | - Giovanni Di Pino
- Research Unit of Neurophysiology and Neuroengineering of Human-Technology Interaction (NeXTlab), Università Campus Bio-Medico Di Roma, Via Álvaro Del Portillo 21, 00128, Rome, Italy.
| |
Collapse
|
17
|
Lanfranco RC, Chancel M, Ehrsson HH. Quantifying body ownership information processing and perceptual bias in the rubber hand illusion. Cognition 2023; 238:105491. [PMID: 37178590 DOI: 10.1016/j.cognition.2023.105491] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Revised: 05/02/2023] [Accepted: 05/04/2023] [Indexed: 05/15/2023]
Abstract
Bodily illusions have fascinated humankind for centuries, and researchers have studied them to learn about the perceptual and neural processes that underpin multisensory channels of bodily awareness. The influential rubber hand illusion (RHI) has been used to study changes in the sense of body ownership - that is, how a limb is perceived to belong to one's body, which is a fundamental building block in many theories of bodily awareness, self-consciousness, embodiment, and self-representation. However, the methods used to quantify perceptual changes in bodily illusions, including the RHI, have mainly relied on subjective questionnaires and rating scales, and the degree to which such illusory sensations depend on sensory information processing has been difficult to test directly. Here, we introduce a signal detection theory (SDT) framework to study the sense of body ownership in the RHI. We provide evidence that the illusion is associated with changes in body ownership sensitivity that depend on the information carried in the degree of asynchrony of correlated visual and tactile signals, as well as with perceptual bias and sensitivity that reflect the distance between the rubber hand and the participant's body. We found that the illusion's sensitivity to asynchrony is remarkably precise; even a 50 ms visuotactile delay significantly affected body ownership information processing. Our findings conclusively link changes in a complex bodily experience such as body ownership to basic sensory information processing and provide a proof of concept that SDT can be used to study bodily illusions.
Collapse
Affiliation(s)
- Renzo C Lanfranco
- Department of Neuroscience, Karolinska Institutet, Stockholm, Sweden.
| | - Marie Chancel
- Department of Neuroscience, Karolinska Institutet, Stockholm, Sweden; Psychology and Neurocognition Lab, Université Grenoble-Alpes, Grenoble, France
| | - H Henrik Ehrsson
- Department of Neuroscience, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
18
|
Gholinezhad S, Farina D, Dosen S, Dideriksen J. Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation. Sci Rep 2023; 13:12461. [PMID: 37528160 PMCID: PMC10393971 DOI: 10.1038/s41598-023-38753-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2023] [Accepted: 07/14/2023] [Indexed: 08/03/2023] Open
Abstract
Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r2 = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.
Collapse
Affiliation(s)
- Shima Gholinezhad
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Dario Farina
- Department of Bioengineering, Faculty of Engineering, Imperial College London, London, SW7 2AZ, UK
| | - Strahinja Dosen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Jakob Dideriksen
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| |
Collapse
|
19
|
Kamsma TM, Boon WQ, Ter Rele T, Spitoni C, van Roij R. Iontronic Neuromorphic Signaling with Conical Microfluidic Memristors. PHYSICAL REVIEW LETTERS 2023; 130:268401. [PMID: 37450821 DOI: 10.1103/physrevlett.130.268401] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/22/2022] [Accepted: 05/25/2023] [Indexed: 07/18/2023]
Abstract
Experiments have shown that the conductance of conical channels, filled with an aqueous electrolyte, can strongly depend on the history of the applied voltage. These channels hence have a memory and are promising elements in brain-inspired (iontronic) circuits. We show here that the memory of such channels stems from transient concentration polarization over the ionic diffusion time. We derive an analytic approximation for these dynamics which shows good agreement with full finite-element calculations. Using our analytic approximation, we propose an experimentally realizable Hodgkin-Huxley iontronic circuit where micrometer cones take on the role of sodium and potassium channels. Our proposed circuit exhibits key features of neuronal communication such as all-or-none action potentials upon a pulse stimulus and a spike train upon a sustained stimulus.
Collapse
Affiliation(s)
- T M Kamsma
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, Netherlands
- Mathematical Institute, Utrecht University, Budapestlaan 6, 3584 CD Utrecht, Netherlands
| | - W Q Boon
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, Netherlands
| | - T Ter Rele
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, Netherlands
- Soft Condensed Matter, Debye Institute for Nanomaterials Science, Utrecht University, Princetonplein 1, 3584 CC Utrecht, Netherlands
| | - C Spitoni
- Mathematical Institute, Utrecht University, Budapestlaan 6, 3584 CD Utrecht, Netherlands
| | - R van Roij
- Institute for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC Utrecht, Netherlands
| |
Collapse
|
20
|
Song S, Fallegger F, Trouillet A, Kim K, Lacour SP. Deployment of an electrocorticography system with a soft robotic actuator. Sci Robot 2023; 8:eadd1002. [PMID: 37163609 DOI: 10.1126/scirobotics.add1002] [Citation(s) in RCA: 13] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Electrocorticography (ECoG) is a minimally invasive approach frequently used clinically to map epileptogenic regions of the brain and facilitate lesion resection surgery and increasingly explored in brain-machine interface applications. Current devices display limitations that require trade-offs among cortical surface coverage, spatial electrode resolution, aesthetic, and risk consequences and often limit the use of the mapping technology to the operating room. In this work, we report on a scalable technique for the fabrication of large-area soft robotic electrode arrays and their deployment on the cortex through a square-centimeter burr hole using a pressure-driven actuation mechanism called eversion. The deployable system consists of up to six prefolded soft legs, and it is placed subdurally on the cortex using an aqueous pressurized solution and secured to the pedestal on the rim of the small craniotomy. Each leg contains soft, microfabricated electrodes and strain sensors for real-time deployment monitoring. In a proof-of-concept acute surgery, a soft robotic electrode array was successfully deployed on the cortex of a minipig to record sensory cortical activity. This soft robotic neurotechnology opens promising avenues for minimally invasive cortical surgery and applications related to neurological disorders such as motor and sensory deficits.
Collapse
Affiliation(s)
- Sukho Song
- Laboratory for Soft Bioelectronic Interfaces, Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1202 Geneva, Switzerland
- Laboratory of Sustainability Robotics, Swiss Federal Laboratories for Materials Science and Technology (Empa), 8600 Dübendorf, Switzerland
| | - Florian Fallegger
- Laboratory for Soft Bioelectronic Interfaces, Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1202 Geneva, Switzerland
| | - Alix Trouillet
- Laboratory for Soft Bioelectronic Interfaces, Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1202 Geneva, Switzerland
| | - Kyungjin Kim
- Laboratory for Soft Bioelectronic Interfaces, Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1202 Geneva, Switzerland
- Department of Mechanical Engineering, University of Connecticut, Storrs, CT 06269, USA
| | - Stéphanie P Lacour
- Laboratory for Soft Bioelectronic Interfaces, Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), 1202 Geneva, Switzerland
| |
Collapse
|
21
|
Abstract
Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.
Collapse
Affiliation(s)
- Guoying Gu
- Robotics Institute, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
- Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Ningbin Zhang
- Robotics Institute, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Chen Chen
- Robotics Institute, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Haipeng Xu
- Robotics Institute, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
| | - Xiangyang Zhu
- Robotics Institute, State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China
- Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai 200240, China
| |
Collapse
|
22
|
Cheng KY, Rehani M, Hebert JS. A scoping review of eye tracking metrics used to assess visuomotor behaviours of upper limb prosthesis users. J Neuroeng Rehabil 2023; 20:49. [PMID: 37095489 PMCID: PMC10127019 DOI: 10.1186/s12984-023-01180-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Accepted: 04/19/2023] [Indexed: 04/26/2023] Open
Abstract
Advanced upper limb prostheses aim to restore coordinated hand and arm function. However, this objective can be difficult to quantify as coordinated movements require an intact visuomotor system. Eye tracking has recently been applied to study the visuomotor behaviours of upper limb prosthesis users by enabling the calculation of eye movement metrics. This scoping review aims to characterize the visuomotor behaviours of upper limb prosthesis users as described by eye tracking metrics, to summarize the eye tracking metrics used to describe prosthetic behaviour, and to identify gaps in the literature and potential areas for future research. A review of the literature was performed to identify articles that reported eye tracking metrics to evaluate the visual behaviours of individuals using an upper limb prosthesis. Data on the level of amputation, type of prosthetic device, type of eye tracker, primary eye metrics, secondary outcome metrics, experimental task, aims, and key findings were extracted. Seventeen studies were included in this scoping review. A consistently reported finding is that prosthesis users have a characteristic visuomotor behaviour that differs from that of individuals with intact arm function. Visual attention has been reported to be directed more towards the hand and less towards the target during object manipulation tasks. A gaze switching strategy and delay to disengage gaze from the current target has also been reported. Differences in the type of prosthetic device and experimental task have revealed some distinct gaze behaviours. Control factors have been shown to be related to gaze behaviour, while sensory feedback and training interventions have been demonstrated to reduce the visual attention associated with prosthesis use. Eye tracking metrics have also been used to assess the cognitive load and sense of agency of prosthesis users. Overall, there is evidence that eye tracking is an effective tool to quantitatively assess the visuomotor behaviour of prosthesis users and the recorded eye metrics are sensitive to change in response to various factors. Additional studies are needed to validate the eye metrics used to assess cognitive load and sense of agency in upper limb prosthesis users.
Collapse
Affiliation(s)
- Kodi Y Cheng
- Division of Physical Medicine and Rehabilitation, Department of Medicine, Faculty of Medicine and Dentistry, College of Health Science, University of Alberta, Edmonton, AB, Canada
- Department of Biomedical Engineering, Faculty of Medicine and Dentistry, College of Health Science, University of Alberta, Edmonton, AB, Canada
| | - Mayank Rehani
- Division of Physical Medicine and Rehabilitation, Department of Medicine, Faculty of Medicine and Dentistry, College of Health Science, University of Alberta, Edmonton, AB, Canada
| | - Jacqueline S Hebert
- Division of Physical Medicine and Rehabilitation, Department of Medicine, Faculty of Medicine and Dentistry, College of Health Science, University of Alberta, Edmonton, AB, Canada.
- Department of Biomedical Engineering, Faculty of Medicine and Dentistry, College of Health Science, University of Alberta, Edmonton, AB, Canada.
- Glenrose Rehabilitation Hospital, Alberta Health Services, Edmonton, AB, Canada.
| |
Collapse
|
23
|
Cimolato A, Ciotti F, Kljajić J, Valle G, Raspopovic S. Symbiotic electroneural and musculoskeletal framework to encode proprioception via neurostimulation: ProprioStim. iScience 2023; 26:106248. [PMID: 36923003 PMCID: PMC10009292 DOI: 10.1016/j.isci.2023.106248] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Revised: 11/23/2022] [Accepted: 02/16/2023] [Indexed: 02/23/2023] Open
Abstract
Peripheral nerve stimulation in amputees achieved the restoration of touch, but not proprioception, which is critical in locomotion. A plausible reason is the lack of means to artificially replicate the complex activity of proprioceptors. To uncover this, we coupled neuromuscular models from ten subjects and nerve histologies from two implanted amputees to develop ProprioStim: a framework to encode proprioception by electrical evoking neural activity in close agreement with natural proprioceptive activity. We demonstrated its feasibility through non-invasive stimulation on seven healthy subjects comparing it with standard linear charge encoding. Results showed that ProprioStim multichannel stimulation was felt more natural, and hold promises for increasing accuracy in knee angle tracking, especially in future implantable solutions. Additionally, we quantified the importance of realistic 3D-nerve models against extruded models previously adopted for further design and validation of novel neurostimulation encoding strategies. ProprioStim provides clear guidelines for the development of neurostimulation policies restoring natural proprioception.
Collapse
Affiliation(s)
- Andrea Cimolato
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092 Zürich, Switzerland
- Rehab Technologies Lab, Fondazione Istituto Italiano di Tecnologia, 16163 Genova, Italy
- Neuroengineering and Medical Robotics Laboratory, Department of Electronics, Information and Bioengineering, Politecnico di Milano, 20133 Milan, Italy
| | - Federico Ciotti
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092 Zürich, Switzerland
| | - Jelena Kljajić
- Institute Mihajlo Pupin, Belgrade, 11060, Serbia
- School of Electrical Engineering, University of Belgrade, Belgrade, 11120, Serbia
| | - Giacomo Valle
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092 Zürich, Switzerland
| | - Stanisa Raspopovic
- Neuroengineering Lab, Department of Health Sciences and Technology, Institute for Robotics and Intelligent Systems, ETH Zürich, 8092 Zürich, Switzerland
| |
Collapse
|
24
|
Abstract
The generation of an internal body model and its continuous update is essential in sensorimotor control. Although known to rely on proprioceptive sensory feedback, the underlying mechanism that transforms this sensory feedback into a dynamic body percept remains poorly understood. However, advances in the development of genetic tools for proprioceptive circuit elements, including the sensory receptors, are beginning to offer new and unprecedented leverage to dissect the central pathways responsible for proprioceptive encoding. Simultaneously, new data derived through emerging bionic neural machine-interface technologies reveal clues regarding the relative importance of kinesthetic sensory feedback and insights into the functional proprioceptive substrates that underlie natural motor behaviors.
Collapse
Affiliation(s)
- Paul D Marasco
- Laboratory for Bionic Integration, Department of Biomedical Engineering, Lerner Research Institute, Cleveland Clinic, Cleveland, Ohio, USA;
- Charles Shor Epilepsy Center, Cleveland Clinic, Cleveland, Ohio, USA
- Advanced Platform Technology Center, Louis Stokes Cleveland Department of Veterans Affairs Medical Center, Cleveland, Ohio, USA
| | - Joriene C de Nooij
- Department of Neurology and the Columbia University Motor Neuron Center, Columbia University Medical Center, New York, NY, USA;
| |
Collapse
|
25
|
Su S, Chai G, Xu W, Meng J, Sheng X, Mouraux A, Zhu X. Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis. J Neural Eng 2023; 20. [PMID: 36595235 DOI: 10.1088/1741-2552/acab32] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Accepted: 12/13/2022] [Indexed: 12/15/2022]
Abstract
Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.
Collapse
Affiliation(s)
- Shiyong Su
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Guohong Chai
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Wei Xu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Jianjun Meng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - Xinjun Sheng
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| | - André Mouraux
- Institute of Neuroscience (IoNS), Université catholique de Louvain, Brussels, Belgium
| | - Xiangyang Zhu
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, People's Republic of China
| |
Collapse
|
26
|
Hunt CL, Sun Y, Wang S, Shehata AW, Hebert JS, Gonzalez-Fernandez M, Kaliki RR, Thakor NV. Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation. J Neuroeng Rehabil 2023; 20:16. [PMID: 36707817 PMCID: PMC9881335 DOI: 10.1186/s12984-023-01136-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 01/10/2023] [Indexed: 01/28/2023] Open
Abstract
BACKGROUND Virtual and augmented reality (AR) have become popular modalities for training myoelectric prosthesis control with upper-limb amputees. While some systems have shown moderate success, it is unclear how well the complex motor skills learned in an AR simulation transfer to completing the same tasks in physical reality. Limb loading is a possible dimension of motor skill execution that is absent in current AR solutions that may help to increase skill transfer between the virtual and physical domains. METHODS We implemented an immersive AR environment where individuals could operate a myoelectric virtual prosthesis to accomplish a variety of object relocation manipulations. Intact limb participants were separated into three groups, the load control (CGLD; [Formula: see text]), the AR control (CGAR; [Formula: see text]), and the experimental group (EG; [Formula: see text]). Both the CGAR and EG completed a 5-session prosthesis training protocol in AR while the CGLD performed simple muscle training. The EG attempted manipulations in AR while undergoing limb loading. The CGAR attempted the same manipulations without loading. All participants performed the same manipulations in physical reality while operating a real prosthesis pre- and post-training. The main outcome measure was the change in the number of manipulations completed during the physical reality assessments (i.e. completion rate). Secondary outcomes included movement kinematics and visuomotor behavior. RESULTS The EG experienced a greater increase in completion rate post-training than both the CGAR and CGLD. This performance increase was accompanied by a shorter motor learning phase, the EG's performance saturating in less sessions of AR training than the CGAR. CONCLUSION The results demonstrated that limb loading plays an important role in transferring complex motor skills learned in virtual spaces to their physical reality analogs. While participants who did not receive limb loading were able to receive some functional benefit from AR training, participants who received the loading experienced a greater positive change in motor performance with their performance saturating in fewer training sessions.
Collapse
Affiliation(s)
- Christopher L. Hunt
- grid.21107.350000 0001 2171 9311Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, USA
| | - Yinghe Sun
- grid.429997.80000 0004 1936 7531Department of Electrical and Computer Engineering, Tufts University, Medford, USA
| | - Shipeng Wang
- grid.21107.350000 0001 2171 9311Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, USA
| | - Ahmed W. Shehata
- grid.17089.370000 0001 2190 316XDivision of Physical Medicine & Rehabilitation, University of Alberta, Edmonton, Canada
| | - Jacqueline S. Hebert
- grid.17089.370000 0001 2190 316XDivision of Physical Medicine & Rehabilitation, University of Alberta, Edmonton, Canada
| | - Marlis Gonzalez-Fernandez
- grid.21107.350000 0001 2171 9311Department of Physical Medicine and Rehabilitation, The Johns Hopkins University, Baltimore, USA
| | - Rahul R. Kaliki
- grid.21107.350000 0001 2171 9311Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, USA ,grid.281272.cInfinite Biomedical Technologies, Baltimore, USA
| | - Nitish V. Thakor
- grid.21107.350000 0001 2171 9311Department of Biomedical Engineering, The Johns Hopkins University, Baltimore, USA
| |
Collapse
|
27
|
Zhao ZP, Nie C, Jiang CT, Cao SH, Tian KX, Yu S, Gu JW. Modulating Brain Activity with Invasive Brain-Computer Interface: A Narrative Review. Brain Sci 2023; 13:brainsci13010134. [PMID: 36672115 PMCID: PMC9856340 DOI: 10.3390/brainsci13010134] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 12/17/2022] [Accepted: 01/05/2023] [Indexed: 01/15/2023] Open
Abstract
Brain-computer interface (BCI) can be used as a real-time bidirectional information gateway between the brain and machines. In particular, rapid progress in invasive BCI, propelled by recent developments in electrode materials, miniature and power-efficient electronics, and neural signal decoding technologies has attracted wide attention. In this review, we first introduce the concepts of neuronal signal decoding and encoding that are fundamental for information exchanges in BCI. Then, we review the history and recent advances in invasive BCI, particularly through studies using neural signals for controlling external devices on one hand, and modulating brain activity on the other hand. Specifically, regarding modulating brain activity, we focus on two types of techniques, applying electrical stimulation to cortical and deep brain tissues, respectively. Finally, we discuss the related ethical issues concerning the clinical application of this emerging technology.
Collapse
Affiliation(s)
- Zhi-Ping Zhao
- School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China
| | - Chuang Nie
- Strategic Support Force Medical Center, Beijing 100101, China
| | - Cheng-Teng Jiang
- Savaid Medical School, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Sheng-Hao Cao
- Brainnetome Center and National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Beijing 100190, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Kai-Xi Tian
- Brainnetome Center and National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Beijing 100190, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China
| | - Shan Yu
- Brainnetome Center and National Laboratory of Pattern Recognition, Chinese Academy of Sciences, Beijing 100190, China
- School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China
- Correspondence: (S.Y.); (J.-W.G.); Tel.: +86-010-8254-4786 (S.Y.); +86-010-6635-6729 (J.-W.G.)
| | - Jian-Wen Gu
- School of Biological Science and Medical Engineering, Beihang University, Beijing 100191, China
- Strategic Support Force Medical Center, Beijing 100101, China
- Correspondence: (S.Y.); (J.-W.G.); Tel.: +86-010-8254-4786 (S.Y.); +86-010-6635-6729 (J.-W.G.)
| |
Collapse
|
28
|
A conversation with Dr. Paul Marasco, PhD, Laboratory for Bionic Integration, Department of Biomedical Engineering, Lerner Research Institute, Cleveland Clinic. J Clin Transl Sci 2023; 7:e56. [PMID: 37008603 PMCID: PMC10052430 DOI: 10.1017/cts.2022.528] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Accepted: 12/14/2022] [Indexed: 03/06/2023] Open
|
29
|
Segil JL, Roldan LM, Graczyk EL. Measuring embodiment: A review of methods for prosthetic devices. Front Neurorobot 2022; 16:902162. [PMID: 36590084 PMCID: PMC9797051 DOI: 10.3389/fnbot.2022.902162] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2022] [Accepted: 10/25/2022] [Indexed: 12/23/2022] Open
Abstract
The development of neural interfaces to provide improved control and somatosensory feedback from prosthetic limbs has initiated a new ability to probe the various dimensions of embodiment. Scientists in the field of neuroprosthetics require dependable measures of ownership, body representation, and agency to quantify the sense of embodiment felt by patients for their prosthetic limbs. These measures are critical to perform generalizable experiments and compare the utility of the new technologies being developed. Here, we review outcome measures used in the literature to evaluate the senses of ownership, body-representation, and agency. We categorize these existing measures based on the fundamental psychometric property measured and whether it is a behavioral or physiological measure. We present arguments for the efficacy and pitfalls of each measure to guide better experimental designs and future outcome measure development. The purpose of this review is to aid prosthesis researchers and technology developers in understanding the concept of embodiment and selecting metrics to assess embodiment in their research. Advances in the ability to measure the embodiment of prosthetic devices have far-reaching implications in the improvement of prosthetic limbs as well as promoting a broader understanding of ourselves as embodied agents.
Collapse
Affiliation(s)
- Jacob L. Segil
- Department of Mechanical Engineering, University of Colorado, Boulder, CO, United States
- Rocky Mountain Regional VA Medical Center, Aurora, CO, United States
| | - Leah Marie Roldan
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| | - Emily L. Graczyk
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
- Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States
| |
Collapse
|
30
|
Mathewson KW, Parker ASR, Sherstan C, Edwards AL, Sutton RS, Pilarski PM. Communicative capital: a key resource for human-machine shared agency and collaborative capacity. Neural Comput Appl 2022; 35:16805-16819. [PMID: 37455836 PMCID: PMC10338399 DOI: 10.1007/s00521-022-07948-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Accepted: 10/12/2022] [Indexed: 11/16/2022]
Abstract
In this work, we present a perspective on the role machine intelligence can play in supporting human abilities. In particular, we consider research in rehabilitation technologies such as prosthetic devices, as this domain requires tight coupling between human and machine. Taking an agent-based view of such devices, we propose that human-machine collaborations have a capacity to perform tasks which is a result of the combined agency of the human and the machine. We introduce communicative capital as a resource developed by a human and a machine working together in ongoing interactions. Development of this resource enables the partnership to eventually perform tasks at a capacity greater than either individual could achieve alone. We then examine the benefits and challenges of increasing the agency of prostheses by surveying literature which demonstrates that building communicative resources enables more complex, task-directed interactions. The viewpoint developed in this article extends current thinking on how best to support the functional use of increasingly complex prostheses, and establishes insight toward creating more fruitful interactions between humans and supportive, assistive, and augmentative technologies.
Collapse
Affiliation(s)
| | - Adam S. R. Parker
- University of Alberta, Edmonton, Canada
- Alberta Machine Intelligence Institute (Amii), Edmonton, Canada
| | | | | | - Richard S. Sutton
- DeepMind, Montreal, Canada
- University of Alberta, Edmonton, Canada
- Alberta Machine Intelligence Institute (Amii), Edmonton, Canada
- DeepMind, Edmonton, Canada
| | - Patrick M. Pilarski
- DeepMind, Montreal, Canada
- University of Alberta, Edmonton, Canada
- Alberta Machine Intelligence Institute (Amii), Edmonton, Canada
- DeepMind, Edmonton, Canada
| |
Collapse
|
31
|
Zbinden J, Lendaro E, Ortiz-Catalan M. A multi-dimensional framework for prosthetic embodiment: a perspective for translational research. J Neuroeng Rehabil 2022; 19:122. [PMID: 36369004 PMCID: PMC9652836 DOI: 10.1186/s12984-022-01102-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2022] [Accepted: 10/25/2022] [Indexed: 11/13/2022] Open
Abstract
The concept of embodiment has gained widespread popularity within prosthetics research. Embodiment has been claimed to be an indicator of the efficacy of sensory feedback and control strategies. Moreover, it has even been claimed to be necessary for prosthesis acceptance, albeit unfoundedly. Despite the popularity of the term, an actual consensus on how prosthetic embodiment should be used in an experimental framework has yet to be reached. The lack of consensus is in part due to terminological ambiguity and the lack of an exact definition of prosthetic embodiment itself. In a review published parallel to this article, we summarized the definitions of embodiment used in prosthetics literature and concluded that treating prosthetic embodiment as a combination of ownership and agency allows for embodiment to be quantified, and thus useful in translational research. Here, we review the potential mechanisms that give rise to ownership and agency considering temporal, spatial, and anatomical constraints. We then use this to propose a multi-dimensional framework where prosthetic embodiment arises within a spectrum dependent on the integration of volition and multi-sensory information as demanded by the degree of interaction with the environment. This framework allows for the different experimental paradigms on sensory feedback and prosthetic control to be placed in a common perspective. By considering that embodiment lays along a spectrum tied to the interactions with the environment, one can conclude that the embodiment of prosthetic devices should be assessed while operating in environments as close to daily life as possible for it to become relevant.
Collapse
|
32
|
Tchimino J, Dideriksen JL, Dosen S. EMG feedback outperforms force feedback in the presence of prosthesis control disturbance. Front Neurosci 2022; 16:952288. [PMID: 36203816 PMCID: PMC9530657 DOI: 10.3389/fnins.2022.952288] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 08/22/2022] [Indexed: 11/30/2022] Open
Abstract
Closing the prosthesis control loop by providing artificial somatosensory feedback can improve utility and user experience. Additionally, closed-loop control should be more robust with respect to disturbance, but this might depend on the type of feedback provided. Thus, the present study investigates and compares the performance of EMG and force feedback in the presence of control disturbances. Twenty able-bodied subjects and one transradial amputee performed delicate and power grasps with a prosthesis in a functional task, while the control signal gain was temporarily increased (high-gain disturbance) or decreased (low-gain disturbance) without their knowledge. Three outcome measures were considered: the percentage of trials successful in the first attempt (reaction to disturbance), the average number of attempts in trials where the wrong force was initially applied (adaptation to disturbance), and the average completion time of the last attempt in every trial. EMG feedback was shown to offer significantly better performance compared to force feedback during power grasping in terms of reaction to disturbance and completion time. During power grasping with high-gain disturbance, the median first-attempt success rate was significantly higher with EMG feedback (73.3%) compared to that achieved with force feedback (60%). Moreover, the median completion time for power grasps with low-gain disturbance was significantly longer with force feedback than with EMG feedback (3.64 against 2.48 s, an increase of 32%). Contrary to our expectations, there was no significant difference between feedback types with regards to adaptation to disturbances and the two feedback types performed similarly in delicate grasps. The results indicated that EMG feedback displayed better performance than force feedback in the presence of control disturbances, further demonstrating the potential of this approach to provide a reliable prosthesis-user interaction.
Collapse
|
33
|
Fang B, Wang C, Sun F, Chen Z, Shan J, Liu H, Ding W, Liang W. Simultaneous sEMG Recognition of Gestures and Force Levels for Interaction With Prosthetic Hand. IEEE Trans Neural Syst Rehabil Eng 2022; 30:2426-2436. [PMID: 35981072 DOI: 10.1109/tnsre.2022.3199809] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The natural interaction between the prosthetic hand and the upper limb amputation patient is important and directly affects the rehabilitation effect and operation ability. Most previous studies only focused on the interaction of gestures but ignored the force levels. This paper proposes a simultaneous recognition method of gestures and forces for interaction with a prosthetic hand. The multitask classification algorithm based on a convolutional neural network (CNN) is designed to improve recognition efficiency and ensure recognition accuracy. The offline experimental results show that the algorithm proposed in this study outperforms other methods in both training speed and accuracy. To prove the effectiveness of the proposed method, a myoelectric prosthetic hand integrated with tactile sensors is developed, and surface electromyography (sEMG) datasets of healthy persons and amputees are built. The online experimental results show that the amputee can control the prosthetic hand to continuously make gestures under different force levels, and the effect of hand coordination on the hand perception of amputees is explored. The results show that gesture classification operation tasks with different force levels based on sEMG signals can be accurately recognized and comfortably interact with prosthetic hands in real time. It improves the amputees' operation ability and relieves their muscle fatigue.
Collapse
|
34
|
Mamidanna P, Dideriksen JL, Dosen S. Estimating speed-accuracy trade-offs to evaluate and understand closed-loop prosthesis interfaces. J Neural Eng 2022; 19. [PMID: 35977526 DOI: 10.1088/1741-2552/ac8a78] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 08/17/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces. APPROACH Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a one-week washout period. Importantly, both interfaces used (identical) direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). Therefore, we estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions. MAIN RESULTS We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands. SIGNIFICANCE Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces.
Collapse
Affiliation(s)
- Pranav Mamidanna
- Department of Health Science and Technology, Aalborg Universitet, Frederik Bajers Vej 7, Aalborg, 9220, DENMARK
| | - Jakob L Dideriksen
- Department of Health Science and Technology, Aalborg University, Fredrik Bajersvej 7, DK-9220 Aalborg SE, Aalborg, 9100, DENMARK
| | - Strahinja Dosen
- Dept. of Health Science and Technology, Aalborg University, Fredrik Bajers Vej 7 D2, Aalborg, 9100, DENMARK
| |
Collapse
|
35
|
Cheng KY, Chapman CS, Hebert JS. Spatiotemporal Coupling of Hand and Eye Movements When Using a Myoelectric Prosthetic Hand. IEEE Int Conf Rehabil Robot 2022; 2022:1-6. [PMID: 36176081 DOI: 10.1109/icorr55369.2022.9896491] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
Upper limb prosthesis users have disruptions in hand-eye coordination, with increased fixations towards the hand and less visual allocation for feedforward planning. The purpose of this study was to explore whether improved motor planning, as reflected by eye gaze behaviour, was associated with more efficient hand movement patterns. Able-bodied participants wore a simulated prosthesis while performing a functional object movement task. Motion and eye tracking data were collected to quantify the eye gaze and hand movement during object interaction. The results of this study demonstrated that the latency of the eye to precede the hand at pick-up was correlated with measures of hand function, including hand variability, movement units, and grasp time, but not reach time. During transport and release, longer latency to disengage gaze from the grasped object and look ahead towards the target was correlated to hand kinematics of hand variability, distance travelled, and transport time. In addition, the latency of the eye to disengage the drop-off location was correlated to release time. Together these may point to control issues with opening and closing the prosthetic hand. Overall, increased feedforward fixations towards the target and reduced feedback fixations towards the hand were related to improved measures of hand function. Hence, coordination between eye and hand movements when using a myoelectric prosthesis may prove to be a useful metric to assess motor planning.
Collapse
|
36
|
Guo ZH, Wang HL, Shao J, Shao Y, Jia L, Li L, Pu X, Wang ZL. Bioinspired soft electroreceptors for artificial precontact somatosensation. SCIENCE ADVANCES 2022; 8:eabo5201. [PMID: 35622923 PMCID: PMC9140963 DOI: 10.1126/sciadv.abo5201] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/10/2023]
Abstract
Artificial haptic sensors form the basis of touch-based human-interfaced applications. However, they are unable to respond to remote events before physical contact. Some elasmobranch fishes, such as seawater sharks, use electroreception somatosensory system for remote environmental perception. Inspired by this ability, we design a soft artificial electroreceptor for sensing approaching targets. The electroreceptor, enabled by an elastomeric electret, is capable of encoding environmental precontact information into a series of voltage pulses functioning as unique precontact human interfaces. Electroceptor applications are demonstrated in a prewarning system, robotic control, game operation, and three-dimensional object recognition. These capabilities in perceiving proximal precontact events can lenrich the functionalities and applications of human-interfaced electronics.
Collapse
Affiliation(s)
- Zi Hao Guo
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - Hai Lu Wang
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
| | - Jiajia Shao
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - Yangshi Shao
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - Luyao Jia
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - Longwei Li
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
| | - Xiong Pu
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
- Center on Nanoenergy Research, School of Physical Science and Technology, GuangXi University, Nanning 530004, P. R. China
- Corresponding author. (X.P.); (Z.L.W.)
| | - Zhong Lin Wang
- Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing 100083, P. R. China
- School of Nanoscience and Technology, University of Chinese Academy of Sciences, Beijing 100049, P. R. China
- School of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA
- Corresponding author. (X.P.); (Z.L.W.)
| |
Collapse
|
37
|
Preliminary Evaluation of the Effect of Mechanotactile Feedback Location on Myoelectric Prosthesis Performance Using a Sensorized Prosthetic Hand. SENSORS 2022; 22:s22103892. [PMID: 35632311 PMCID: PMC9145984 DOI: 10.3390/s22103892] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/15/2022] [Revised: 05/13/2022] [Accepted: 05/16/2022] [Indexed: 02/01/2023]
Abstract
A commonly cited reason for the high abandonment rate of myoelectric prostheses is a lack of grip force sensory feedback. Researchers have attempted to restore grip force sensory feedback by stimulating the residual limb’s skin surface in response to the prosthetic hand’s measured grip force. Recent work has focused on restoring natural feedback to the missing digits directly through invasive surgical procedures. However, the functional benefit of utilizing somatotopically matching feedback has not been evaluated. In this paper, we propose an experimental protocol centered on a fragile object grasp and lift task using a sensorized myoelectric prosthesis to evaluate sensory feedback techniques. We formalized a suite of outcome measures related to task success, timing, and strategy. A pilot study (n = 3) evaluating the effect of utilizing a somatotopically accurate feedback stimulation location in able-bodied participants was conducted to evaluate the feasibility of the standardized platform, and to inform future studies on the role of feedback stimulation location in prosthesis use. Large between-participant effect sizes were observed in all outcome measures, indicating that the feedback location likely plays a role in myoelectric prosthesis performance. The success rate decreased, and task timing and task focus metrics increased, when using somatotopically-matched feedback compared to non-somatotopically-matched feedback. These results were used to conduct a power analysis, revealing that a sample size of n = 8 would be sufficient to achieve significance in all outcome measures.
Collapse
|
38
|
Multisensory Integration in Bionics: Relevance and Perspectives. CURRENT PHYSICAL MEDICINE AND REHABILITATION REPORTS 2022. [DOI: 10.1007/s40141-022-00350-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Abstract
Purpose of review
The goal of the review is to highlight the growing importance of multisensory integration processes connected to bionic limbs and somatosensory feedback restoration.
Recent findings
Restoring quasi-realistic sensations by means of neurostimulation has been shown to provide functional and motor benefits in limb amputees. In the recent past, cognitive processes linked to the artificial sense of touch seemed to play a crucial role for a full prosthesis integration and acceptance.
Summary
Artificial sensory feedback implemented in bionic limbs enhances the cognitive integration of the prosthetic device in amputees. The multisensory experience can be measured and must be considered in the design of novel somatosensory neural prostheses where the goal is to provide a realistic sensory experience to the prosthetic user. The correct integration of these sensory signals will guarantee higher-level cognitive benefits as a better prosthesis embodiment and a reduction of perceived limb distortions.
Collapse
|
39
|
Montero Aragón J, Thumser Z, Masiero F, Beckler D, Clemente F, Marasco P, Cipriani C. The myokinetic stimulation interface: activation of proprioceptive neural responses with remotely actuated magnets implanted in rodent forelimb muscles. J Neural Eng 2022; 19. [PMID: 35390778 DOI: 10.1088/1741-2552/ac6537] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Accepted: 04/06/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Proprioception is the sense of one's position, orientation, and movement in space, and it is of fundamental importance for motor control. When proprioception is impaired or absent, motor execution becomes error-prone, leading to poorly coordinated movements. The kinaesthetic illusion, which creates perceptions of limb movement in humans through non-invasively applying vibrations to muscles or tendons, provides an avenue for studying and restoring the sense of joint movement (kinaesthesia). This technique, however, leaves ambiguity between proprioceptive percepts that arise from muscles versus those that arise from skin receptors. Here we propose the concept of a stimulation system to activate kinaesthesia through the untethered application of localized vibration through implanted magnets. APPROACH In this proof-of-concept study, we use two simplified 1-DoF systems to show the feasibility of eliciting muscle-sensory responses in an animal model across multiple frequencies, including those that activate the kinaesthetic illusion (70 - 115 Hz). Furthermore, we generalized the concept by developing a 5-DoF prototype system capable of generating directional, frequency-selective vibrations with desired displacement profiles. MAIN RESULTS In-vivo tests with the 1-DoF systems demonstrated the feasibility to elicit muscle sensory neural responses in the median nerve of an animal model. Instead, in-vitro tests with the 5-DoF prototype demonstrated high accuracy in producing directional and frequency selective vibrations along different magnet axes. SIGNIFICANCE These results provide evidence for a new technique that interacts with the native neuro-muscular anatomy to study proprioception and eventually pave the way towards the development of advanced limb prostheses or assistive devices for the sensory impaired.
Collapse
Affiliation(s)
- Jordan Montero Aragón
- BioRobotics Institute, Scuola Superiore di Studi Universitari e di Perfezionamento Sant'Anna, Viale Rinaldo Piaggio, 34, Pisa, Toscana, 56025, ITALY
| | - Zachary Thumser
- Department of Biomedical Engineering, Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland, Ohio, 44195, UNITED STATES
| | - Federico Masiero
- BioRobotics Institute, Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio, 34, Pisa, 56025, ITALY
| | - Dylan Beckler
- Department of Biomedical Engineering, Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland, Ohio, 44195, UNITED STATES
| | - Francesco Clemente
- BioRobotics Institute, Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio, 34, Pontedera, 56025, ITALY
| | - Paul Marasco
- Department of Biomedical Engineering, Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland, Ohio, 44195, UNITED STATES
| | - Christian Cipriani
- BioRobotics Institute, Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio, 34, Pontedera, 56025, ITALY
| |
Collapse
|
40
|
Zbinden J, Lendaro E, Ortiz-Catalan M. Prosthetic embodiment: systematic review on definitions, measures, and experimental paradigms. J Neuroeng Rehabil 2022; 19:37. [PMID: 35346251 PMCID: PMC8962549 DOI: 10.1186/s12984-022-01006-6] [Citation(s) in RCA: 32] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Accepted: 03/04/2022] [Indexed: 11/25/2022] Open
Abstract
The term embodiment has become omnipresent within prosthetics research and is often used as a metric of the progress made in prosthetic technologies, as well as a hallmark for user acceptance. However, despite the frequent use of the term, the concept of prosthetic embodiment is often left undefined or described incongruently, sometimes even within the same article. This terminological ambiguity complicates the comparison of studies using embodiment as a metric of success, which in turn hinders the advancement of prosthetics research. To resolve these terminological ambiguities, we systematically reviewed the used definitions of embodiment in the prosthetics literature. We performed a thematic analysis of the definitions and found that embodiment is often conceptualized in either of two frameworks based on body representations or experimental phenomenology. We concluded that treating prosthetic embodiment within an experimental phenomenological framework as the combination of ownership and agency allows for embodiment to be a quantifiable metric for use in translational research. To provide a common reference and guidance on how to best assess ownership and agency, we conducted a second systematic review, analyzing experiments and measures involving ownership and agency. Together, we highlight a pragmatic definition of prosthetic embodiment as the combination of ownership and agency, and in an accompanying article, we provide a perspective on a multi-dimensional framework for prosthetic embodiment. Here, we concluded by providing recommendations on metrics that allow for outcome comparisons between studies, thereby creating a common reference for further discussions within prosthetics research.
Collapse
Affiliation(s)
- Jan Zbinden
- Center for Bionics and Pain Research, Mölndal, Sweden
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Eva Lendaro
- Center for Bionics and Pain Research, Mölndal, Sweden
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
| | - Max Ortiz-Catalan
- Center for Bionics and Pain Research, Mölndal, Sweden.
- Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden.
- Operational Area 3, Sahlgrenska University Hospital, Gothenburg, Sweden.
- Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| |
Collapse
|
41
|
Zhang J, Hao M, Yang F, Liang W, Sun A, Chou CH, Lan N. Evaluation of multiple perceptual qualities of transcutaneous electrical nerve stimulation for evoked tactile sensation in forearm amputees. J Neural Eng 2022; 19. [PMID: 35320789 DOI: 10.1088/1741-2552/ac6062] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2021] [Accepted: 03/23/2022] [Indexed: 11/11/2022]
Abstract
OBJECTIVE Evoked tactile sensation (ETS) elicited by transcutaneous electrical nerve stimulation (TENS) is promising to convey digit-specific sensory information to amputees naturally and non-invasively. Fitting ETS-based sensory feedback to amputees entails customizing coding of multiple sensory information for each stimulation site. This study was to elucidate the consistency of percepts and qualities by TENS at multiple stimulation sites in amputees retaining ETS. APPROACH Five transradial amputees with ETS and fourteen able-bodied subjects participated in this study. Surface electrodes with small size (10 mm in diameter) were adopted to fit the restricted projected finger map on the forearm stump of amputees. Effects of stimulus frequency on sensory types were assessed, and the map of perceptual threshold for each sensation was characterized. Sensitivity for vibration and buzz sensations was measured using distinguishable difference in stimulus pulse width. Rapid assessments for modulation ranges of pulse width at fixed amplitude and frequency were developed for coding sensory information. Buzz sensation was demonstrated for location discrimination relating to prosthetic fingers. MAIN RESULTS Vibration and buzz sensations were consistently evoked at 20 Hz and 50 Hz as dominant sensation types in all amputees and able-bodied subjects. Perceptual thresholds of different sensations followed a similar strength-duration curve relating stimulus amplitude to pulse width. The averaged distinguishable difference in pulse width was 12.84 ± 7.23 μs for vibration and 15.21 ± 6.47 μs for buzz in able-bodied subjects, and 14.91 ± 10.54 μs for vibration and 11.30 ± 3.42 μs for buzz in amputees. Buzz coding strategy enabled five amputees to discriminate contact of individual fingers with an overall accuracy of 77.85%. SIGNIFICANCE The consistency in perceptual qualities of dominant sensations can be exploited for coding multi-modality sensory feedback. A fast protocol of sensory coding is possible for fitting ETS-based, non-invasive sensory feedback to amputees.
Collapse
Affiliation(s)
- Jie Zhang
- Laboratory of Neurorehabilitaiton Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Room 404 South Building Med-X, No.1954 Rd. Huashan, Shanghai, Shanghai, 200030, CHINA
| | - Manzhao Hao
- Laboratory of Neurorehabilitaiton Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Room 401 South Building Med-X, No.1954 Rd. Huashan, Shanghai, Shanghai, 200030, CHINA
| | - Fei Yang
- Shanghai Jiao Tong University, Room 404 South Building Med-X, No. 1954 Rd. Huashan, Xuhui, Shanghai, Shanghai, 200030, CHINA
| | - Wenyuan Liang
- National Research Center for Rehabilitation Technical Aids, No.1 Rong Hua Zhong Road, Beijing Economic and Technological Development Area, Beijing, Beijing, 100176, CHINA
| | - Aiping Sun
- National Research Center for Rehabilitation Technical Aids, No.1 Rong Hua Zhong Road, Beijing Economic and Technological Development Area, Beijing, Beijing, 100176, CHINA
| | - Chi-Hong Chou
- Laboratory of Neurorehabilitaiton Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Room 401 South Building Med-X, No.1954 Rd. Huashan, Shanghai, Shanghai, 200030, CHINA
| | - Ning Lan
- Laboratory of Neurorehabilitaiton Engineering, School of Biomedical Engineering, Shanghai Jiao Tong University, Room 405 South Building Med-X, No.1954 Rd. Huashan, Shanghai, Shanghai, 200030, CHINA
| |
Collapse
|
42
|
Jabban L, Dupan S, Zhang D, Ainsworth B, Nazarpour K, Metcalfe BW. Sensory Feedback for Upper-Limb Prostheses: Opportunities and Barriers. IEEE Trans Neural Syst Rehabil Eng 2022; 30:738-747. [PMID: 35290188 DOI: 10.1109/tnsre.2022.3159186] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
The addition of sensory feedback to upper-limb prostheses has been shown to improve control, increase embodiment, and reduce phantom limb pain. However, most commercial prostheses do not incorporate sensory feedback due to several factors. This paper focuses on the major challenges of a lack of deep understanding of user needs, the unavailability of tailored, realistic outcome measures and the segregation between research on control and sensory feedback. The use of methods such as the Person-Based Approach and co-creation can improve the design and testing process. Stronger collaboration between researchers can integrate different prostheses research areas to accelerate the translation process.
Collapse
|
43
|
Pasluosta C, Kiele P, Čvančara P, Micera S, Aszmann OC, Stieglitz T. Bidirectional bionic limbs: a perspective bridging technology and physiology. J Neural Eng 2022; 19. [PMID: 35132954 DOI: 10.1088/1741-2552/ac4bff] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2021] [Accepted: 01/17/2022] [Indexed: 11/11/2022]
Abstract
Precise control of bionic limbs relies on robust decoding of motor commands from nerves or muscles signals and sensory feedback from artificial limbs to the nervous system by interfacing the afferent nerve pathways. Implantable devices for bidirectional communication with bionic limbs have been developed in parallel with research on physiological alterations caused by an amputation. In this perspective article, we question whether increasing our effort on bridging these technologies with a deeper understanding of amputation pathophysiology and human motor control may help to overcome pressing stalls in the next generation of bionic limbs.
Collapse
Affiliation(s)
- C Pasluosta
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany
| | - P Kiele
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany
| | - P Čvančara
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany.,BrainLinks-BrainTools, University of Freiburg, Freiburg, Germany
| | - S Micera
- School of Engineering, École Polytechnique Fédérale de Lausanne, Bertarelli Foundation Chair in Translational Neuroengineering, Centre for Neuroprosthetics and Institute of Bioengineering, Lausanne, Switzerland.,The BioRobotics Institute and Department of Excellence in Robotics and Artificial Intelligence, Scuola Superiore Sant'Anna, Pisa, Italy
| | - O C Aszmann
- Clinical Laboratory for Bionic Extremity Reconstruction, Medical University of Vienna; Division of Plastic and Reconstructive Surgery, Department of Surgery, Medical University of Vienna, Vienna, Austria
| | - T Stieglitz
- Laboratory for Biomedical Microtechnology, Department of Microsystems Engineering, University of Freiburg, Freiburg, Germany.,Bernstein Center Freiburg, University of Freiburg, Freiburg, Germany.,BrainLinks-BrainTools, University of Freiburg, Freiburg, Germany
| |
Collapse
|
44
|
Ortiz-Catalan M. Engineering and surgical advancements enable more cognitively integrated bionic arms. Sci Robot 2021; 6:eabk3123. [PMID: 34516750 DOI: 10.1126/scirobotics.abk3123] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Abstract
[Figure: see text].
Collapse
Affiliation(s)
- Max Ortiz-Catalan
- Center for Bionics and Pain Research, Mölndal, Sweden.,Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden.,Operational Area 3, Sahlgrenska University Hospital, Mölndal, Sweden.,Department of Orthopaedics, Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
| |
Collapse
|