1
|
Marucci M, Maddaluno O, Ryan CP, Perciballi C, Vasta S, Ciotti S, Moscatelli A, Betti V. Rewiring the evolution of the human hand: How the embodiment of a virtual bionic tool improves behavior. iScience 2024; 27:109937. [PMID: 39055602 PMCID: PMC11270032 DOI: 10.1016/j.isci.2024.109937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 11/02/2023] [Accepted: 05/06/2024] [Indexed: 07/27/2024] Open
Abstract
Humans are the most versatile tool users among animals. Accordingly, our manual skills evolved alongside the shape of the hand. In the future, further evolution may take place: humans may merge with their tools, and technology may integrate into our biology in a way that blurs the line between the two. So, the question is whether humans can embody a bionic tool (i.e., experience it as part of their body) and thus if this would affect behavior. We investigated in virtual reality how the substitution of the hand with a virtual grafting of an end-effector, either non-naturalistic (a bionic tool) or naturalistic (a hand), impacts embodiment and behavior. Across four experiments, we show that the virtual grafting of a bionic tool elicits a sense of embodiment similar to or even stronger than its natural counterpart. In conclusion, the natural usage of bionic tools can rewire the evolution of human behavior.
Collapse
Affiliation(s)
- Matteo Marucci
- Department of Psychology, Sapienza University of Rome, Rome, Italy
- Laboratory of Neuroscience and Applied Technology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Ottavia Maddaluno
- Department of Psychology, Sapienza University of Rome, Rome, Italy
- Laboratory of Neuroscience and Applied Technology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Colleen Patricia Ryan
- Department of Systems Medicine, University of Rome Tor Vergata, Rome, Italy
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Cristina Perciballi
- Department of Psychology, Sapienza University of Rome, Rome, Italy
- Laboratory of Neuroscience and Applied Technology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Simona Vasta
- Department of Psychology, Sapienza University of Rome, Rome, Italy
- Laboratory of Neuroscience and Applied Technology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Simone Ciotti
- Information Engineering Department and the Research Center “E. Piaggio”, University of Pisa, Pisa, Italy
| | - Alessandro Moscatelli
- Department of Systems Medicine, University of Rome Tor Vergata, Rome, Italy
- Laboratory of Neuromotor Physiology, Santa Lucia Foundation IRCCS, Rome, Italy
| | - Viviana Betti
- Department of Psychology, Sapienza University of Rome, Rome, Italy
- Laboratory of Neuroscience and Applied Technology, Santa Lucia Foundation IRCCS, Rome, Italy
| |
Collapse
|
2
|
Lin J, Ding B, Song Z, Li Z, Li S. A Model of Multi-Finger Coordination in Keystroke Movement. SENSORS (BASEL, SWITZERLAND) 2024; 24:1221. [PMID: 38400379 PMCID: PMC10892657 DOI: 10.3390/s24041221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/18/2024] [Revised: 02/04/2024] [Accepted: 02/13/2024] [Indexed: 02/25/2024]
Abstract
In multi-finger coordinated keystroke actions by professional pianists, movements are precisely regulated by multiple motor neural centers, exhibiting a certain degree of coordination in finger motions. This coordination enhances the flexibility and efficiency of professional pianists' keystrokes. Research on the coordination of keystrokes in professional pianists is of great significance for guiding the movements of piano beginners and the motion planning of exoskeleton robots, among other fields. Currently, research on the coordination of multi-finger piano keystroke actions is still in its infancy. Scholars primarily focus on phenomenological analysis and theoretical description, which lack accurate and practical modeling methods. Considering that the tendon of the ring finger is closely connected to adjacent fingers, resulting in limited flexibility in its movement, this study concentrates on coordinated keystrokes involving the middle and ring fingers. A motion measurement platform is constructed, and Leap Motion is used to collect data from 12 professional pianists. A universal model applicable to multiple individuals for multi-finger coordination in keystroke actions based on the backpropagation (BP) neural network is proposed, which is optimized using a genetic algorithm (GA) and a sparrow search algorithm (SSA). The angular rotation of the ring finger's MCP joint is selected as the model output, while the individual difference information and the angular data of the middle finger's MCP joint serve as inputs. The individual difference information used in this study includes ring finger length, middle finger length, and years of piano training. The results indicate that the proposed SSA-BP neural network-based model demonstrates superior predictive accuracy, with a root mean square error of 4.8328°. Based on this model, the keystroke motion of the ring finger's MCP joint can be accurately predicted from the middle finger's keystroke motion information, offering an evaluative method and scientific guidance for the training of multi-finger coordinated keystrokes in piano learners.
Collapse
Affiliation(s)
| | - Baihui Ding
- Key Laboratory of Mechanism Theory and Equipment Design, Ministry of Education, Tianjin University, Tianjin 300350, China; (J.L.); (Z.S.); (Z.L.); (S.L.)
| | | | | | | |
Collapse
|
3
|
Dominijanni G, Pinheiro DL, Pollina L, Orset B, Gini M, Anselmino E, Pierella C, Olivier J, Shokur S, Micera S. Human motor augmentation with an extra robotic arm without functional interference. Sci Robot 2023; 8:eadh1438. [PMID: 38091424 DOI: 10.1126/scirobotics.adh1438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 11/15/2023] [Indexed: 12/18/2023]
Abstract
Extra robotic arms (XRAs) are gaining interest in neuroscience and robotics, offering potential tools for daily activities. However, this compelling opportunity poses new challenges for sensorimotor control strategies and human-machine interfaces (HMIs). A key unsolved challenge is allowing users to proficiently control XRAs without hindering their existing functions. To address this, we propose a pipeline to identify suitable HMIs given a defined task to accomplish with the XRA. Following such a scheme, we assessed a multimodal motor HMI based on gaze detection and diaphragmatic respiration in a purposely designed modular neurorobotic platform integrating virtual reality and a bilateral upper limb exoskeleton. Our results show that the proposed HMI does not interfere with speaking or visual exploration and that it can be used to control an extra virtual arm independently from the biological ones or in coordination with them. Participants showed significant improvements in performance with daily training and retention of learning, with no further improvements when artificial haptic feedback was provided. As a final proof of concept, naïve and experienced participants used a simplified version of the HMI to control a wearable XRA. Our analysis indicates how the presented HMI can be effectively used to control XRAs. The observation that experienced users achieved a success rate 22.2% higher than that of naïve users, combined with the result that naïve users showed average success rates of 74% when they first engaged with the system, endorses the viability of both the virtual reality-based testing and training and the proposed pipeline.
Collapse
Affiliation(s)
- Giulia Dominijanni
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Daniel Leal Pinheiro
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- Neuroengineering and Neurocognition Laboratory, Escola Paulista de Medicina, Department of Neurology and Neurosurgery, Division of Neuroscience, Universidade Federal de São Paulo, São Paulo, Brazil
| | - Leonardo Pollina
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Bastien Orset
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Martina Gini
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
- Neuroelectronic Interfaces, Faculty of Electrical Engineering and IT, Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen, Aachen 52074, Germany
| | - Eugenio Anselmino
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Camilla Pierella
- Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, and Maternal and Children's Sciences (DINOGMI), University of Genoa, Genoa, Italy
| | - Jérémy Olivier
- Institute for Industrial Sciences and Technologies, Haute Ecole du Paysage, d'Ingénierie et d'Architecture (HEPIA), HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland
| | - Solaiman Shokur
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| | - Silvestro Micera
- Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy
| |
Collapse
|
4
|
Huang Y, Eden J, Ivanova E, Burdet E. Can Training Make Three Arms Better Than Two Heads for Trimanual Coordination? IEEE OPEN JOURNAL OF ENGINEERING IN MEDICINE AND BIOLOGY 2023; 4:148-155. [PMID: 38274776 PMCID: PMC10810309 DOI: 10.1109/ojemb.2023.3305808] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2023] [Revised: 06/05/2023] [Accepted: 08/08/2023] [Indexed: 01/27/2024] Open
Abstract
Supernumerary effectors have been proposed to enable users to perform tasks alone that normally require assistance from a partner. While various supernumerary robotic limbs have been developed in the last decade, the capability of users to operate them effectively has not yet been proven. Here we tested whether users (i) can complete a task that requires simultaneous and fine control of three effectors, and (ii) can be trained to do so with similar or superior performance as through collaboration with a human partner. As in previous studies, initial augmented capability was less than that of working with a partner. However, one hour of dedicated solo trimanual training across three days significantly increased task performance, so that participants became able to perform trimanual control alone as well as or better than they could with a new partner. This shows the viability of augmentation systems for applications such as in robotic surgery or industrial assembly, which can be further validated on real tasks with physical systems.
Collapse
Affiliation(s)
- Yanpei Huang
- Department of BioengineeringImperial College of Science Technology, MedicineSW7 2BXLondonU.K.
| | - Jonathan Eden
- Department of BioengineeringImperial College of Science Technology, MedicineSW7 2BXLondonU.K.
- Department of Mechanical EngineeringUniversity of MelbourneParkvilleVIC3052Australia
| | - Ekaterina Ivanova
- Department of BioengineeringImperial College of Science Technology, MedicineSW7 2BXLondonU.K.
- School of Electronic Engineering, Computer ScienceQueen Mary University of LondonE1 4NSLondonU.K.
| | - Etienne Burdet
- Department of BioengineeringImperial College of Science Technology, MedicineSW7 2BXLondonU.K.
| |
Collapse
|