Biomimetic Active Touch with Fingertips and Whiskers

Identifying the Strength Level of Objects' Tactile Attributes Using a Multi-Scale Convolutional Neural Network

In order to solve the problem in which most currently existing research focuses on the binary tactile attributes of objects and ignores identifying the strength level of tactile attributes, this paper establishes a tactile data set of the strength level of objects' elasticity and hardness attributes to make up for the lack of relevant data, and proposes a multi-scale convolutional neural network to identify the strength level of object attributes. The network recognizes the different attributes and identifies differences in the strength level of the same object attributes by fusing the original features, i.e., the single-channel features and multi-channel features of the data. A variety of evaluation methods were used for comparison with multiple models in terms of strength levels of elasticity and hardness. The results show that our network has a more significant effect in accuracy. In the prediction results of the positive examples in the predicted value, the true value has a higher proportion of positive examples, that is, the precision is better. The prediction effect for the positive examples in the true value is better, that is, the recall is better. Finally, the recognition rate for all classes is higher in terms of f1_score. For the overall sample, the prediction of the multi-scale convolutional neural network has a higher recognition rate and the network's ability to recognize each strength level is more stable.

Embodied tactile perception and learning

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.

3D Contact Position Estimation of Image-Based Areal Soft Tactile Sensor with Printed Array Markers and Image Sensors

Tactile sensors have been widely used and researched in various fields of medical and industrial applications. Gradually, they will be used as new input devices and contact sensors for interactive robots. If a tactile sensor is to be applied to various forms of human–machine interactions, it needs to be soft to ensure comfort and safety, and it should be easily customizable and inexpensive. The purpose of this study is to estimate 3D contact position of a novel image-based areal soft tactile sensor (IASTS) using printed array markers and multiple cameras. First, we introduce the hardware structure of the prototype IASTS, which consists of a soft material with printed array markers and multiple cameras with LEDs. Second, an estimation algorithm for the contact position is proposed based on the image processing of the array markers and their Gaussian fittings. A series of basic experiments was conducted and their results were analyzed to verify the effectiveness of the proposed IASTS hardware and its estimation software. To ensure the stability of the estimated contact positions a Kalman filter was developed. Finally, it was shown that the contact positions on the IASTS were estimated with a reasonable error value for soft haptic applications.

A Deep-Learning Model for Underwater Position Sensing of a Wake's Source Using Artificial Seal Whiskers

Various marine animals possess the ability to track their preys and navigate dark aquatic environments using hydrodynamic sensing of the surrounding flow. In the present study, a deep-learning model is applied to a biomimetic sensor for underwater position detection of a wake-generating body. The sensor is composed of a bundle of spatially-distributed optical fibers that act as artificial seal-like whiskers and interact with the body's wake in the form of time-variant (bending) deflections. Supervised learning is employed to relate the vibrations of the artificial whiskers to the position of an upstream cylinder. The labeled training data are prepared based on the processing and reduction of the recorded bending responses of the artificial whiskers while the cylinder is placed at various locations. An iterative training algorithm is performed on two neural-network models while using the 10-fold cross-validation technique. The models are able to predict the coordinates of the cylinder in the two-dimensional (2D) space with a high degree of accuracy. The current implementation of the sensor can passively sense the wake generated by the cylinder at Re ≃ 6000 and estimate its position with an average error smaller than the characteristic diameter D of the cylinder and for inter-distances (in the water tunnel) up to 25-times D.

Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation

In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.