Human tactile sensing and sensorimotor mechanism: from afferent tactile signals to efferent motor control

In tactile sensing, decoding the journey from afferent tactile signals to efferent motor commands is a significant challenge primarily due to the difficulty in capturing population-level afferent nerve signals during active touch. This study integrates a finite element hand model with a neural dynamic model by using microneurography data to predict neural responses based on contact biomechanics and membrane transduction dynamics. This research focuses specifically on tactile sensation and its direct translation into motor actions. Evaluations of muscle synergy during in -vivo experiments revealed transduction functions linking tactile signals and muscle activation. These functions suggest similar sensorimotor strategies for grasping influenced by object size and weight. The decoded transduction mechanism was validated by restoring human-like sensorimotor performance on a tendon-driven biomimetic hand. This research advances our understanding of translating tactile sensation into motor actions, offering valuable insights into prosthetic design, robotics, and the development of next-generation prosthetics with neuromorphic tactile feedback.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

From Brain Models to Robotic Embodied Cognition: How Does Biological Plausibility Inform Neuromorphic Systems?

We examine the challenging "marriage" between computational efficiency and biological plausibility-A crucial node in the domain of spiking neural networks at the intersection of neuroscience, artificial intelligence, and robotics. Through a transdisciplinary review, we retrace the historical and most recent constraining influences that these parallel fields have exerted on descriptive analysis of the brain, construction of predictive brain models, and ultimately, the embodiment of neural networks in an enacted robotic agent. We study models of Spiking Neural Networks (SNN) as the central means enabling autonomous and intelligent behaviors in biological systems. We then provide a critical comparison of the available hardware and software to emulate SNNs for investigating biological entities and their application on artificial systems. Neuromorphics is identified as a promising tool to embody SNNs in real physical systems and different neuromorphic chips are compared. The concepts required for describing SNNs are dissected and contextualized in the new no man's land between cognitive neuroscience and artificial intelligence. Although there are recent reviews on the application of neuromorphic computing in various modules of the guidance, navigation, and control of robotic systems, the focus of this paper is more on closing the cognition loop in SNN-embodied robotics. We argue that biologically viable spiking neuronal models used for electroencephalogram signals are excellent candidates for furthering our knowledge of the explainability of SNNs. We complete our survey by reviewing different robotic modules that can benefit from neuromorphic hardware, e.g., perception (with a focus on vision), localization, and cognition. We conclude that the tradeoff between symbolic computational power and biological plausibility of hardware can be best addressed by neuromorphics, whose presence in neurorobotics provides an accountable empirical testbench for investigating synthetic and natural embodied cognition. We argue this is where both theoretical and empirical future work should converge in multidisciplinary efforts involving neuroscience, artificial intelligence, and robotics.

Effects of Sensing Tactile Arrays, Shear Force, and Proprioception of Robot on Texture Recognition

In robotics, tactile perception is important for fine control using robot grippers and hands. To effectively incorporate tactile perception in robots, it is essential to understand how humans use mechanoreceptors and proprioceptors to perceive texture. Thus, our study aimed to investigate the impact of tactile sensor arrays, shear force, and the positional information of the robot's end effector on its ability to recognize texture. A deep learning network was employed to classify tactile data from 24 different textures that were explored by a robot. The input values of the deep learning network were modified based on variations in the number of channels of the tactile signal, the arrangement of the tactile sensor, the presence or absence of shear force, and the positional information of the robot. By comparing the accuracy of texture recognition, our analysis revealed that tactile sensor arrays more accurately recognized the texture compared to a single tactile sensor. The utilization of shear force and positional information of the robot resulted in an improved accuracy of texture recognition when using a single tactile sensor. Furthermore, an equal number of sensors placed in a vertical arrangement led to a more accurate distinction of textures during exploration when compared to sensors placed in a horizontal arrangement. The results of this study indicate that the implementation of a tactile sensor array should be prioritized over a single sensor for enhanced accuracy in tactile sensing, and the use of integrated data should be considered for single tactile sensing.

Spike encoding techniques for IoT time-varying signals benchmarked on a neuromorphic classification task

Spiking Neural Networks (SNNs), known for their potential to enable low energy consumption and computational cost, can bring significant advantages to the realm of embedded machine learning for edge applications. However, input coming from standard digital sensors must be encoded into spike trains before it can be elaborated with neuromorphic computing technologies. We present here a detailed comparison of available spike encoding techniques for the translation of time-varying signals into the event-based signal domain, tested on two different datasets both acquired through commercially available digital devices: the Free Spoken Digit dataset (FSD), consisting of 8-kHz audio files, and the WISDM dataset, composed of 20-Hz recordings of human activity through mobile and wearable inertial sensors. We propose a complete pipeline to benchmark these encoding techniques by performing time-dependent signal classification through a Spiking Convolutional Neural Network (sCNN), including a signal preprocessing step consisting of a bank of filters inspired by the human cochlea, feature extraction by production of a sonogram, transfer learning via an equivalent ANN, and model compression schemes aimed at resource optimization. The resulting performance comparison and analysis provides a powerful practical tool, empowering developers to select the most suitable coding method based on the type of data and the desired processing algorithms, and further expands the applicability of neuromorphic computational paradigms to embedded sensor systems widely employed in the IoT and industrial domains.

Texture recognition based on multi-sensory integration of proprioceptive and tactile signals

The sense of touch plays a fundamental role in enabling us to interact with our surrounding environment. Indeed, the presence of tactile feedback in prostheses greatly assists amputees in doing daily tasks. In this line, the present study proposes an integration of artificial tactile and proprioception receptors for texture discrimination under varying scanning speeds. Here, we fabricated a soft biomimetic fingertip including an 8 × 8 array tactile sensor and a piezoelectric sensor to mimic Merkel, Meissner, and Pacinian mechanoreceptors in glabrous skin, respectively. A hydro-elastomer sensor was fabricated as an artificial proprioception sensor (muscle spindles) to assess the instantaneous speed of the biomimetic fingertip. In this study, we investigated the concept of the complex receptive field of RA-I and SA-I afferents for naturalistic textures. Next, to evaluate the synergy between the mechanoreceptors and muscle spindle afferents, ten naturalistic textures were manipulated by a soft biomimetic fingertip at six different speeds. The sensors' outputs were converted into neuromorphic spike trains to mimic the firing pattern of biological mechanoreceptors. These spike responses are then analyzed using machine learning classifiers and neural coding paradigms to explore the multi-sensory integration in real experiments. This synergy between muscle spindle and mechanoreceptors in the proposed neuromorphic system represents a generalized texture discrimination scheme and interestingly irrespective of the scanning speed.

Braille letter reading: A benchmark for spatio-temporal pattern recognition on neuromorphic hardware

Spatio-temporal pattern recognition is a fundamental ability of the brain which is required for numerous real-world activities. Recent deep learning approaches have reached outstanding accuracies in such tasks, but their implementation on conventional embedded solutions is still very computationally and energy expensive. Tactile sensing in robotic applications is a representative example where real-time processing and energy efficiency are required. Following a brain-inspired computing approach, we propose a new benchmark for spatio-temporal tactile pattern recognition at the edge through Braille letter reading. We recorded a new Braille letters dataset based on the capacitive tactile sensors of the iCub robot's fingertip. We then investigated the importance of spatial and temporal information as well as the impact of event-based encoding on spike-based computation. Afterward, we trained and compared feedforward and recurrent Spiking Neural Networks (SNNs) offline using Backpropagation Through Time (BPTT) with surrogate gradients, then we deployed them on the Intel Loihi neuromorphic chip for fast and efficient inference. We compared our approach to standard classifiers, in particular to the Long Short-Term Memory (LSTM) deployed on the embedded NVIDIA Jetson GPU, in terms of classification accuracy, power, and energy consumption together with computational delay. Our results show that the LSTM reaches ~97% of accuracy, outperforming the recurrent SNN by ~17% when using continuous frame-based data instead of event-based inputs. However, the recurrent SNN on Loihi with event-based inputs is ~500 times more energy-efficient than the LSTM on Jetson, requiring a total power of only ~30 mW. This work proposes a new benchmark for tactile sensing and highlights the challenges and opportunities of event-based encoding, neuromorphic hardware, and spike-based computing for spatio-temporal pattern recognition at the edge.

Neuromorphic Tactile Edge Orientation Classification in an Unsupervised Spiking Neural Network

Dexterous manipulation in robotic hands relies on an accurate sense of artificial touch. Here we investigate neuromorphic tactile sensation with an event-based optical tactile sensor combined with spiking neural networks for edge orientation detection. The sensor incorporates an event-based vision system (mini-eDVS) into a low-form factor artificial fingertip (the NeuroTac). The processing of tactile information is performed through a Spiking Neural Network with unsupervised Spike-Timing-Dependent Plasticity (STDP) learning, and the resultant output is classified with a 3-nearest neighbours classifier. Edge orientations were classified in 10-degree increments while tapping vertically downward and sliding horizontally across the edge. In both cases, we demonstrate that the sensor is able to reliably detect edge orientation, and could lead to accurate, bio-inspired, tactile processing in robotics and prosthetics applications.

A Biomimetic Circuit for Electronic Skin With Application in Hand Prosthesis

One major challenge in upper limb prostheses is providing sensory feedback to amputees. Reproducing the spiking patterns of human primary tactile afferents can be considered as the first step for this challenging problem. In this study, a novel biomimetic circuit for SA-I and RA-I afferents is proposed to functionally replicate the spiking response of the biological tactile afferents to indentation stimuli. The circuit has been designed, laid out, and simulated in TSMC 180nm CMOS technology with a 1.8V supply voltage. A pair of SA-I and RA-I afferent circuits consume [Formula: see text] of power. The occupied silicon area is [Formula: see text] for 32 afferents. To provide the inputs for circuit testing, a patch of skin with a grid of mechanoreceptors is simulated and tested by an edge stimulus presented at different orientations. Experimental data are collected using indentation of 3D-printed edges at different orientations on a tactile sensor mounted on a robotic arm. Inspired by innervation patterns observed in biology, the artificial afferents are connected to several neighboring mechanoreceptors with different weights to form complex receptive fields which cover the entire mechanoreceptor grid. Machine learning algorithms are applied offline to classify the edge orientations based on the pattern of neural responses. Our results show that the complex receptive fields arising from the innervation pattern led to smaller circuit area and lower power consumption, while facilitating data encoding from high-resolution sensors. The proposed biomimetic circuit and tactile encoding example demonstrate potential applications in modern tactile sensing modules for developing novel bio-robotic and prosthetic technologies.

A hybrid evolutionary learning classification for robot ground pattern recognition

In the field of intelligent robot engineering, whether it is humanoid, bionic or vehicle robots, the driving forms of standing, moving and walking, and the consciousness discrimination of the environment in which they are located have always been the focus and difficulty of research. Based on such problems, Naive Bayes Classifier (NBC), Support Vector Machine(SVM), k-Nearest-Neighbor (KNN), Decision Tree (DT), Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) were introduced to conduct experiments. The six individual classifiers have an obvious effect on a particular type of ground, but the overall performance is poor. Therefore, the paper proposes a “Novel Hybrid Evolutionary Learning” method (NHEL) which combines every single classifier by means of weighted voting and adopts an improved genetic algorithm (GA) to obtain the optimal weight. According to the fitness function and evolution times, this paper designs the adaptively changing crossover and mutation rate and applies the conjugate gradient (CG) to enhance GA. By making full use of the global search capabilities of GA and the fast local search ability of CG, the convergence speed is accelerated and the search precision is upgraded. The experimental results show that the performance of the proposed model is significantly better than individual machine learning and ensemble classifiers.

Sharpness recognition based on synergy between bio-inspired nociceptors and tactile mechanoreceptors

Touch and pain sensations are complementary aspects of daily life that convey crucial information about the environment while also providing protection to our body. Technological advancements in prosthesis design and control mechanisms assist amputees to regain lost function but often they have no meaningful tactile feedback or perception. In the present study, we propose a bio-inspired tactile system with a population of 23 digital afferents: 12 RA-I, 6 SA-I, and 5 nociceptors. Indeed, the functional concept of the nociceptor is implemented on the FPGA for the first time. One of the main features of biological tactile afferents is that their distal axon branches in the skin, creating complex receptive fields. Given these physiological observations, the bio-inspired afferents are randomly connected to the several neighboring mechanoreceptors with different weights to form their own receptive field. To test the performance of the proposed neuromorphic chip in sharpness detection, a robotic system with three-degree of freedom equipped with the tactile sensor indents the 3D-printed objects. Spike responses of the biomimetic afferents are then collected for analysis by rate and temporal coding algorithms. In this way, the impact of the innervation mechanism and collaboration of afferents and nociceptors on sharpness recognition are investigated. Our findings suggest that the synergy between sensory afferents and nociceptors conveys more information about tactile stimuli which in turn leads to the robustness of the proposed neuromorphic system against damage to the taxels or afferents. Moreover, it is illustrated that spiking activity of the biomimetic nociceptors is amplified as the sharpness increases which can be considered as a feedback mechanism for prosthesis protection. This neuromorphic approach advances the development of prosthesis to include the sensory feedback and to distinguish innocuous (non-painful) and noxious (painful) stimuli.

Embodied tactile perception and learning

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

Cuneate spiking neural network learning to classify naturalistic texture stimuli under varying sensing conditions

We implemented a functional neuronal network that was able to learn and discriminate haptic features from biomimetic tactile sensor inputs using a two-layer spiking neuron model and homeostatic synaptic learning mechanism. The first order neuron model was used to emulate biological tactile afferents and the second order neuron model was used to emulate biological cuneate neurons. We have evaluated 10 naturalistic textures using a passive touch protocol, under varying sensing conditions. Tactile sensor data acquired with five textures under five sensing conditions were used for a synaptic learning process, to tune the synaptic weights between tactile afferents and cuneate neurons. Using post-learning synaptic weights, we evaluated the individual and population cuneate neuron responses by decoding across 10 stimuli, under varying sensing conditions. This resulted in a high decoding performance. We further validated the decoding performance across stimuli, irrespective of sensing velocities using a set of 25 cuneate neuron responses. This resulted in a median decoding performance of 96% across the set of cuneate neurons. Being able to learn and perform generalized discrimination across tactile stimuli, makes this functional spiking tactile system effective and suitable for further robotic applications.

CORDIC-Astrocyte: Tripartite Glutamate-IP3-Ca 2+ Interaction Dynamics on FPGA

Real-time, large-scale simulation of biological systems is challenging due to different types of nonlinear functions describing biochemical reactions in the cells. The promise of the high speed, cost effectiveness, and power efficiency in addition to parallel processing has made application-specific hardware an attractive simulation platform. This paper proposes high-speed and low-cost digital hardware to emulate a biological-plausible astrocyte and glutamate-release mechanism. The nonlinear terms of these models were calculated using a high-precision and cost-effective algorithm. Subsequently, the modified models were simulated to study and validate their functions. We developed several hardware versions by setting different constraints to investigate trade-offs and find the best possible design. FPGA implementation results confirmed the ability of the design to emulate biological cell behaviours in detail with high accuracy. As for performance, the proposed design turned out to be faster and more efficient than previously published works that targeted digital hardware for biological-plausible astrocytes.

A Digital Hardware System for Spiking Network of Tactile Afferents

In the present research, we explore the possibility of utilizing a hardware-based neuromorphic approach to develop a tactile sensory system at the level of first-order afferents, which are slowly adapting type 1 (SA-I) and fast adapting type 1 (FA-I) afferents. Four spiking models are used to mimic neural signals of both SA-I and FA-I primary afferents. Next, a digital circuit is designed for each spiking model for both afferents to be implemented on the field-programmable gate array (FPGA). The four different digital circuits are then compared from source utilization point of view to find the minimum cost circuit for creating a population of digital afferents. In this way, the firing responses of both SA-I and FA-I afferents are physically measured in hardware. Finally, a population of 243 afferents consisting of 90 SA-I and 153 FA-I digital neuromorphic circuits are implemented on the FPGA. The FPGA also receives nine inputs from the force sensors through an interfacing board. Therefore, the data of multiple inputs are processed by the spiking network of tactile afferents, simultaneously. Benefiting from parallel processing capabilities of FPGA, the proposed architecture offers a low-cost neuromorphic structure for tactile information processing. Applying machine learning algorithms on the artificial spiking patterns collected from FPGA, we successfully classified three different objects based on the firing rate paradigm. Consequently, the proposed neuromorphic system provides the opportunity for development of new tactile processing component for robotic and prosthetic applications.

A Piezoresistive Tactile Sensor for a Large Area Employing Neural Network

Electronic skin is an important means through which robots can obtain external information. A novel flexible tactile sensor capable of simultaneously detecting the contact position and force was proposed in this paper. The tactile sensor had a three-layer structure. The upper layer was a specially designed conductive film based on indium-tin oxide polyethylene terephthalate (ITO-PET), which could be used for detecting contact position. The intermediate layer was a piezoresistive film used as the force-sensitive element. The lower layer was made of fully conductive material such as aluminum foil and was used only for signal output. In order to solve the inconsistencies and nonlinearity of the piezoresistive properties for large areas, a Radial Basis Function (RBF) neural network was used. This includes input, hidden, and output layers. The input layer has three nodes representing position coordinates, X, Y, and resistor, R. The output layer has one node representing force, F. A sensor sample was fabricated and experiments of contact position and force detection were performed on the sample. The results showed that the principal function of the tactile sensor was feasible. The sensor sample exhibited good consistency and linearity. The tactile sensor has only five lead wires and can provide the information support necessary for safe human—computer interactions.

A 3-D Surface Reconstruction with Shadow Processing for Optical Tactile Sensors

An optical tactile sensor technique with 3-dimension (3-D) surface reconstruction is proposed for robotic fingers. The hardware of the tactile sensor consists of a surface deformation sensing layer, an image sensor and four individually controlled flashing light emitting diodes (LEDs). The image sensor records the deformation images when the robotic finger touches an object. For each object, four deformation images are taken with the LEDs providing different illumination directions. Before the 3-D reconstruction, the look-up tables are built to map the intensity distribution to the image gradient data. The possible image shadow will be detected and amended. Then the 3-D depth distribution of the object surface can be reconstructed from the 2-D gradient obtained using the look-up tables. The architecture of the tactile sensor and the proposed signal processing flow have been presented in details. A prototype tactile sensor has been built. Both the simulation and experimental results have validated the effectiveness of the proposed 3-D surface reconstruction method for the optical tactile sensors. The proposed 3-D surface reconstruction method has the unique feature of image shadow detection and compensation, which differentiates itself from those in the literature.

A Digital Hardware Realization for Spiking Model of Cutaneous Mechanoreceptor

Inspired by the biology of human tactile perception, a hardware neuromorphic approach is proposed for spiking model of mechanoreceptors to encode the input force. In this way, a digital circuit is designed for a slowly adapting type I (SA-I) and fast adapting type I (FA-I) mechanoreceptors to be implemented on a low-cost digital hardware, such as field-programmable gate array (FPGA). This system computationally replicates the neural firing responses of both afferents. Then, comparative simulations are shown. The spiking models of mechanoreceptors are first simulated in MATLAB and next the digital neuromorphic circuits simulated in VIVADO are also compared to show that obtained results are in good agreement both quantitatively and qualitatively. Finally, we test the performance of the proposed digital mechanoreceptors in hardware using a prepared experimental set up. Hardware synthesis and physical realization on FPGA indicate that the digital mechanoreceptors are able to replicate essential characteristics of different firing patterns including bursting and spiking responses of the SA-I and FA-I mechanoreceptors. In addition to parallel computation, a main advantage of this method is that the mechanoreceptor digital circuits can be implemented in real-time through low-power neuromorphic hardware. This novel engineering framework is generally suitable for use in robotic and hand-prosthetic applications, so progressing the state of the art for tactile sensing.

An Extreme Learning Machine-Based Neuromorphic Tactile Sensing System for Texture Recognition

Despite significant advances in computational algorithms and development of tactile sensors, artificial tactile sensing is strikingly less efficient and capable than the human tactile perception. Inspired by efficiency of biological systems, we aim to develop a neuromorphic system for tactile pattern recognition. We particularly target texture recognition as it is one of the most necessary and challenging tasks for artificial sensory systems. Our system consists of a piezoresistive fabric material as the sensor to emulate skin, an interface that produces spike patterns to mimic neural signals from mechanoreceptors, and an extreme learning machine (ELM) chip to analyze spiking activity. Benefiting from intrinsic advantages of biologically inspired event-driven systems and massively parallel and energy-efficient processing capabilities of the ELM chip, the proposed architecture offers a fast and energy-efficient alternative for processing tactile information. Moreover, it provides the opportunity for the development of low-cost tactile modules for large-area applications by integration of sensors and processing circuits. We demonstrate the recognition capability of our system in a texture discrimination task, where it achieves a classification accuracy of 92% for categorization of ten graded textures. Our results confirm that there exists a tradeoff between response time and classification accuracy (and information transfer rate). A faster decision can be achieved at early time steps or by using a shorter time window. This, however, results in deterioration of the classification accuracy and information transfer rate. We further observe that there exists a tradeoff between the classification accuracy and the input spike rate (and thus energy consumption). Our work substantiates the importance of development of efficient sparse codes for encoding sensory data to improve the energy efficiency. These results have a significance for a wide range of wearable, robotic, prosthetic, and industrial applications.

Active Tactile Transfer Learning for Object Discrimination in an Unstructured Environment Using Multimodal Robotic Skin

In this paper, we propose a probabilistic active tactile transfer learning (ATTL) method to enable robotic systems to exploit their prior tactile knowledge while discriminating among objects via their physical properties (surface texture, stiffness, and thermal conductivity). Using the proposed method, the robot autonomously selects and exploits its most relevant prior tactile knowledge to efficiently learn about new unknown objects with a few training samples or even one. The experimental results show that using our proposed method, the robot successfully discriminated among new objects with [Formula: see text] discrimination accuracy using only one training sample (on-shot-tactile-learning). Furthermore, the results demonstrate that our method is robust against transferring irrelevant prior tactile knowledge (negative tactile knowledge transfer).

Improving Spiking Dynamical Networks: Accurate Delays, Higher-Order Synapses, and Time Cells

Researchers building spiking neural networks face the challenge of improving the biological plausibility of their model networks while maintaining the ability to quantitatively characterize network behavior. In this work, we extend the theory behind the neural engineering framework (NEF), a method of building spiking dynamical networks, to permit the use of a broad class of synapse models while maintaining prescribed dynamics up to a given order. This theory improves our understanding of how low-level synaptic properties alter the accuracy of high-level computations in spiking dynamical networks. For completeness, we provide characterizations for both continuous-time (i.e., analog) and discrete-time (i.e., digital) simulations. We demonstrate the utility of these extensions by mapping an optimal delay line onto various spiking dynamical networks using higher-order models of the synapse. We show that these networks nonlinearly encode rolling windows of input history, using a scale invariant representation, with accuracy depending on the frequency content of the input signal. Finally, we reveal that these methods provide a novel explanation of time cell responses during a delay task, which have been observed throughout hippocampus, striatum, and cortex.

Active Prior Tactile Knowledge Transfer for Learning Tactual Properties of New Objects

Reusing the tactile knowledge of some previously-explored objects (prior objects) helps us to easily recognize the tactual properties of new objects. In this paper, we enable a robotic arm equipped with multi-modal artificial skin, like humans, to actively transfer the prior tactile exploratory action experiences when it learns the detailed physical properties of new objects. These experiences, or prior tactile knowledge, are built by the feature observations that the robot perceives from multiple sensory modalities, when it applies the pressing, sliding, and static contact movements on objects with different action parameters. We call our method Active Prior Tactile Knowledge Transfer (APTKT), and systematically evaluated its performance by several experiments. Results show that the robot improved the discrimination accuracy by around 10% when it used only one training sample with the feature observations of prior objects. By further incorporating the predictions from the observation models of prior objects as auxiliary features, our method improved the discrimination accuracy by over 20%. The results also show that the proposed method is robust against transferring irrelevant prior tactile knowledge (negative knowledge transfer).

Neuromorphic photonic networks using silicon photonic weight banks

Photonic systems for high-performance information processing have attracted renewed interest. Neuromorphic silicon photonics has the potential to integrate processing functions that vastly exceed the capabilities of electronics. We report first observations of a recurrent silicon photonic neural network, in which connections are configured by microring weight banks. A mathematical isomorphism between the silicon photonic circuit and a continuous neural network model is demonstrated through dynamical bifurcation analysis. Exploiting this isomorphism, a simulated 24-node silicon photonic neural network is programmed using “neural compiler” to solve a differential system emulation task. A 294-fold acceleration against a conventional benchmark is predicted. We also propose and derive power consumption analysis for modulator-class neurons that, as opposed to laser-class neurons, are compatible with silicon photonic platforms. At increased scale, Neuromorphic silicon photonics could access new regimes of ultrafast information processing for radio, control, and scientific computing.