1
|
Wei Y, Marshall AG, McGlone FP, Makdani A, Zhu Y, Yan L, Ren L, Wei G. Human tactile sensing and sensorimotor mechanism: from afferent tactile signals to efferent motor control. Nat Commun 2024; 15:6857. [PMID: 39127772 PMCID: PMC11316806 DOI: 10.1038/s41467-024-50616-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2023] [Accepted: 07/12/2024] [Indexed: 08/12/2024] Open
Abstract
In tactile sensing, decoding the journey from afferent tactile signals to efferent motor commands is a significant challenge primarily due to the difficulty in capturing population-level afferent nerve signals during active touch. This study integrates a finite element hand model with a neural dynamic model by using microneurography data to predict neural responses based on contact biomechanics and membrane transduction dynamics. This research focuses specifically on tactile sensation and its direct translation into motor actions. Evaluations of muscle synergy during in -vivo experiments revealed transduction functions linking tactile signals and muscle activation. These functions suggest similar sensorimotor strategies for grasping influenced by object size and weight. The decoded transduction mechanism was validated by restoring human-like sensorimotor performance on a tendon-driven biomimetic hand. This research advances our understanding of translating tactile sensation into motor actions, offering valuable insights into prosthetic design, robotics, and the development of next-generation prosthetics with neuromorphic tactile feedback.
Collapse
Affiliation(s)
- Yuyang Wei
- Department of Engineering Science, University of Oxford, Oxford, OX1 3PJ, UK
- Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester, Manchester, M13 9PL, UK
| | - Andrew G Marshall
- Institute of Life Course and Medical Sciences, University of Liverpool, Liverpool, L69 3BX, UK
| | - Francis P McGlone
- Department of Neuroscience and Biomedical Engineering, Aalto University, Otakaari 24, Helsinki, Finland
| | - Adarsh Makdani
- School of Natural Sciences and Psychology, Liverpool John Moores University, Liverpool, L3 5UX, UK
| | - Yiming Zhu
- Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester, Manchester, M13 9PL, UK
| | - Lingyun Yan
- Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester, Manchester, M13 9PL, UK
| | - Lei Ren
- Department of Mechanical, Aerospace and Civil Engineering, The University of Manchester, Manchester, M13 9PL, UK.
- Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Jilin, China.
| | - Guowu Wei
- School of Science, Engineering and Environment, University of Salford, Manchester, M5 4WT, UK.
| |
Collapse
|
2
|
Mandil W, Rajendran V, Nazari K, Ghalamzan-Esfahani A. Tactile-Sensing Technologies: Trends, Challenges and Outlook in Agri-Food Manipulation. SENSORS (BASEL, SWITZERLAND) 2023; 23:7362. [PMID: 37687818 PMCID: PMC10490130 DOI: 10.3390/s23177362] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 08/01/2023] [Accepted: 08/15/2023] [Indexed: 09/10/2023]
Abstract
Tactile sensing plays a pivotal role in achieving precise physical manipulation tasks and extracting vital physical features. This comprehensive review paper presents an in-depth overview of the growing research on tactile-sensing technologies, encompassing state-of-the-art techniques, future prospects, and current limitations. The paper focuses on tactile hardware, algorithmic complexities, and the distinct features offered by each sensor. This paper has a special emphasis on agri-food manipulation and relevant tactile-sensing technologies. It highlights key areas in agri-food manipulation, including robotic harvesting, food item manipulation, and feature evaluation, such as fruit ripeness assessment, along with the emerging field of kitchen robotics. Through this interdisciplinary exploration, we aim to inspire researchers, engineers, and practitioners to harness the power of tactile-sensing technology for transformative advancements in agri-food robotics. By providing a comprehensive understanding of the current landscape and future prospects, this review paper serves as a valuable resource for driving progress in the field of tactile sensing and its application in agri-food systems.
Collapse
Affiliation(s)
- Willow Mandil
- School of Computer Science, University of Lincoln, Lincoln LN6 7TS, UK
| | - Vishnu Rajendran
- Lincoln Institute for Agri-Food Technology, University of Lincoln, Lincoln LN6 7TS, UK
| | - Kiyanoush Nazari
- School of Computer Science, University of Lincoln, Lincoln LN6 7TS, UK
| | | |
Collapse
|
3
|
Texture recognition based on multi-sensory integration of proprioceptive and tactile signals. Sci Rep 2022; 12:21690. [PMID: 36522364 PMCID: PMC9755227 DOI: 10.1038/s41598-022-24640-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Accepted: 11/17/2022] [Indexed: 12/23/2022] Open
Abstract
The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors' outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.
Collapse
|
4
|
Macdonald FLA, Lepora NF, Conradt J, Ward-Cherrier B. Neuromorphic Tactile Edge Orientation Classification in an Unsupervised Spiking Neural Network. SENSORS (BASEL, SWITZERLAND) 2022; 22:6998. [PMID: 36146344 PMCID: PMC9500632 DOI: 10.3390/s22186998] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/31/2022] [Revised: 08/24/2022] [Accepted: 09/09/2022] [Indexed: 06/16/2023]
Abstract
Dexterous manipulation in robotic hands relies on an accurate sense of artificial touch. Here we investigate neuromorphic tactile sensation with an event-based optical tactile sensor combined with spiking neural networks for edge orientation detection. The sensor incorporates an event-based vision system (mini-eDVS) into a low-form factor artificial fingertip (the NeuroTac). The processing of tactile information is performed through a Spiking Neural Network with unsupervised Spike-Timing-Dependent Plasticity (STDP) learning, and the resultant output is classified with a 3-nearest neighbours classifier. Edge orientations were classified in 10-degree increments while tapping vertically downward and sliding horizontally across the edge. In both cases, we demonstrate that the sensor is able to reliably detect edge orientation, and could lead to accurate, bio-inspired, tactile processing in robotics and prosthetics applications.
Collapse
Affiliation(s)
- Fraser L. A. Macdonald
- Department of Engineering Mathematics, University of Bristol, Bristol BS8 1TW, UK
- Bristol Robotics Laboratory, University of the West of England, Bristol BS34 8QZ, UK
| | - Nathan F. Lepora
- Department of Engineering Mathematics, University of Bristol, Bristol BS8 1TW, UK
- Bristol Robotics Laboratory, University of the West of England, Bristol BS34 8QZ, UK
| | - Jörg Conradt
- School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, 114 28 Stockholm, Sweden
| | - Benjamin Ward-Cherrier
- Department of Engineering Mathematics, University of Bristol, Bristol BS8 1TW, UK
- Bristol Robotics Laboratory, University of the West of England, Bristol BS34 8QZ, UK
| |
Collapse
|
5
|
Powell H, Winkel M, Hopp AV, Linde H. A hybrid biological neural network model for solving problems in cognitive planning. Sci Rep 2022; 12:10628. [PMID: 35739285 PMCID: PMC9226121 DOI: 10.1038/s41598-022-11567-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 04/12/2022] [Indexed: 11/09/2022] Open
Abstract
A variety of behaviors, like spatial navigation or bodily motion, can be formulated as graph traversal problems through cognitive maps. We present a neural network model which can solve such tasks and is compatible with a broad range of empirical findings about the mammalian neocortex and hippocampus. The neurons and synaptic connections in the model represent structures that can result from self-organization into a cognitive map via Hebbian learning, i.e. into a graph in which each neuron represents a point of some abstract task-relevant manifold and the recurrent connections encode a distance metric on the manifold. Graph traversal problems are solved by wave-like activation patterns which travel through the recurrent network and guide a localized peak of activity onto a path from some starting position to a target state.
Collapse
Affiliation(s)
- Henry Powell
- Merck KGaA, Darmstadt, Germany. .,University of Glasgow, Glasgow, Scotland, UK.
| | | | | | - Helmut Linde
- Merck KGaA, Darmstadt, Germany.,Transylvanian Institute of Neuroscience, Cluj-Napoca, Romania
| |
Collapse
|
6
|
Sengupta D, Mastella M, Chicca E, Kottapalli AGP. Skin-Inspired Flexible and Stretchable Electrospun Carbon Nanofiber Sensors for Neuromorphic Sensing. ACS APPLIED ELECTRONIC MATERIALS 2022; 4:308-315. [PMID: 35098136 PMCID: PMC8793024 DOI: 10.1021/acsaelm.1c01010] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/18/2021] [Accepted: 12/20/2021] [Indexed: 06/14/2023]
Abstract
During the past few decades, a significant amount of research effort has been dedicated toward developing skin-inspired sensors for real-time human motion monitoring and next-generation robotic devices. Although several flexible and wearable sensors have been developed in the past, the need of the hour is developing accurate, reliable, sophisticated, facile yet inexpensive flexible sensors coupled with neuromorphic systems or spiking neural networks to encode tactile information without the need for complex digital architectures, thus achieving true skin-like sensing with limited resources. In this work, we propose an approach entailing carbon nanofiber-polydimethylsiloxane composite-based piezoresistive sensors, coupled with spiking neural networks, to mimic skin-like sensing. The strain and pressure sensors have been combined with appropriately designed neural networks to encode analog voltages to spikes to recreate bioinspired tactile sensing and proprioception. To further validate the proprioceptive capability of the system, a gesture tracking smart glove, combined with a spiking neural network, was demonstrated. Wearable and flexible sensors with accompanying neural networks such as the ones proposed in this work will pave the way for a future generation of skin-mimetic sensors for advanced prosthetic devices, apparel integrable smart sensors for human motion monitoring, and human-machine interfaces.
Collapse
Affiliation(s)
- Debarun Sengupta
- Department
of Advanced Production Engineering (APE), Engineering and Technology
Institute Groningen (ENTEG), University
of Groningen, Groningen 9747 AG, The Netherlands
| | - Michele Mastella
- Groningen
Cognitive Systems and Materials Center (CogniGron), University of Groningen, Groningen 9747 AG, The Netherlands
- Bio-Inspired
Circuits and Systems (BICS) Laboratory, Zernike Institute for Advanced
Materials (Zernike Inst Adv Mat), University
of Groningen, Nijenborgh
4, Groningen NL-9747 AG, Netherlands
| | - Elisabetta Chicca
- Groningen
Cognitive Systems and Materials Center (CogniGron), University of Groningen, Groningen 9747 AG, The Netherlands
- Bio-Inspired
Circuits and Systems (BICS) Laboratory, Zernike Institute for Advanced
Materials (Zernike Inst Adv Mat), University
of Groningen, Nijenborgh
4, Groningen NL-9747 AG, Netherlands
| | - Ajay Giri Prakash Kottapalli
- Department
of Advanced Production Engineering (APE), Engineering and Technology
Institute Groningen (ENTEG), University
of Groningen, Groningen 9747 AG, The Netherlands
- MIT
Sea Grant College Program, Massachusetts
Institute of Technology (MIT), 77 Massachusetts Avenue, NW98-151, Cambridge, Massachusetts 02139, United States
| |
Collapse
|
7
|
Gerasimova SA, Belov AI, Korolev DS, Guseinov DV, Lebedeva AV, Koryazhkina MN, Mikhaylov AN, Kazantsev VB, Pisarchik AN. Stochastic Memristive Interface for Neural Signal Processing. SENSORS 2021; 21:s21165587. [PMID: 34451027 PMCID: PMC8402302 DOI: 10.3390/s21165587] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/02/2021] [Revised: 08/09/2021] [Accepted: 08/16/2021] [Indexed: 11/16/2022]
Abstract
We propose a memristive interface consisting of two FitzHugh–Nagumo electronic neurons connected via a metal–oxide (Au/Zr/ZrO2(Y)/TiN/Ti) memristive synaptic device. We create a hardware–software complex based on a commercial data acquisition system, which records a signal generated by a presynaptic electronic neuron and transmits it to a postsynaptic neuron through the memristive device. We demonstrate, numerically and experimentally, complex dynamics, including chaos and different types of neural synchronization. The main advantages of our system over similar devices are its simplicity and real-time performance. A change in the amplitude of the presynaptic neurogenerator leads to the potentiation of the memristive device due to the self-tuning of its parameters. This provides an adaptive modulation of the postsynaptic neuron output. The developed memristive interface, due to its stochastic nature, simulates a real synaptic connection, which is very promising for neuroprosthetic applications.
Collapse
Affiliation(s)
- Svetlana A. Gerasimova
- Institute of Biology and Biomedicine, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (S.A.G.); (A.V.L.); (V.B.K.)
- Research Institute and Technology, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (A.I.B.); (D.S.K.); (D.V.G.); (A.N.M.)
| | - Alexey I. Belov
- Research Institute and Technology, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (A.I.B.); (D.S.K.); (D.V.G.); (A.N.M.)
| | - Dmitry S. Korolev
- Research Institute and Technology, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (A.I.B.); (D.S.K.); (D.V.G.); (A.N.M.)
| | - Davud V. Guseinov
- Research Institute and Technology, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (A.I.B.); (D.S.K.); (D.V.G.); (A.N.M.)
| | - Albina V. Lebedeva
- Institute of Biology and Biomedicine, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (S.A.G.); (A.V.L.); (V.B.K.)
| | - Maria N. Koryazhkina
- Research and Educational Center “Physics of Solid State Nanostructures”, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia;
| | - Alexey N. Mikhaylov
- Research Institute and Technology, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (A.I.B.); (D.S.K.); (D.V.G.); (A.N.M.)
| | - Victor B. Kazantsev
- Institute of Biology and Biomedicine, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia; (S.A.G.); (A.V.L.); (V.B.K.)
- Laboratory of Neuroscience and Cognitive Technology, Innopolis University, 420500 Innopolis, Russia
- Center for Neurotechnology and Machine Learning, Immanuel Kant Baltic Federal University, 236016 Kaliningrad, Russia
| | - Alexander N. Pisarchik
- Research and Educational Center “Physics of Solid State Nanostructures”, National Research Lobachevsky State University of Nizhny Novgorod, 603950 Nizhny Novgorod, Russia;
- Laboratory of Neuroscience and Cognitive Technology, Innopolis University, 420500 Innopolis, Russia
- Center for Biomedical Technology, Universidad Politécnica de Madrid, Pozuelo de Alarcón, 28223 Madrid, Spain
- Correspondence:
| |
Collapse
|