Slippage Detection with Piezoresistive Tactile Sensors

Mechanical and thermal stimulation for studying the somatosensory system: a review on devices and methods

The somatosensory system is widely studied to understand its functioning mechanisms. Multiple tests, based on different devices and methods, have been performed not only on humans but also on animals andex-vivomodels. Depending on the nature of the sample under analysis and on the scientific aims of interest, several solutions for experimental stimulation and for investigations on sensation or pain have been adopted. In this review paper, an overview of the available devices and methods has been reported, also analyzing the representative values adopted during literature experiments. Among the various physical stimulations used to study the somatosensory system, we focused only on mechanical and thermal ones. Based on the analysis of their main features and on literature studies, we pointed out the most suitable solution for humans, rodents, andex-vivomodels and investigation aims (sensation and pain).

Research on Adaptive Grasping with a Prosthetic Hand Based on Perceptual Information on Hardness and Surface Roughness

In order to solve the problems of methods that use a single form of sensing, the ease of causing deformation damage to the targets with a low hardness during grasping, and the slow sliding inhibition of a prosthetic hand when the grasping target slides, which are problems that exist in most current intelligent prosthetic hands, this study introduces an adaptive control strategy for prosthetic hands based on multi-sensor sensing. Using a force-sensing resistor (FSR) to collect changes in signals generated after contact with a target, a prosthetic hand can classify the target's hardness level and adaptively provide the desired grasping force so as to reduce the deformation of and damage to the target in the process of grasping. A fiber-optic sensor collects the light reflected by the object to identify its surface roughness, so that the prosthetic hand adaptively adjusts the sliding inhibition method according to the surface roughness information to improve the grasping efficiency. By integrating information on the hardness and surface roughness of the target, an adaptive control strategy for a prosthetic hand is proposed. The experimental results showed that the adaptive control strategy was able to reduce the damage to the target by enabling the prosthetic hand to achieve stable grasping; after grasping the target with an initial force and generating sliding, the efficiency of slippage inhibition was improved, the target could be stably grasped in a shorter time, and the hardness, roughness and weight ranges of targets that could be grasped by the prosthetic hand were enlarged, thus improving the success rate of stable grasping under extreme conditions.

Tactile-Sensing Technologies: Trends, Challenges and Outlook in Agri-Food Manipulation

Tactile sensing plays a pivotal role in achieving precise physical manipulation tasks and extracting vital physical features. This comprehensive review paper presents an in-depth overview of the growing research on tactile-sensing technologies, encompassing state-of-the-art techniques, future prospects, and current limitations. The paper focuses on tactile hardware, algorithmic complexities, and the distinct features offered by each sensor. This paper has a special emphasis on agri-food manipulation and relevant tactile-sensing technologies. It highlights key areas in agri-food manipulation, including robotic harvesting, food item manipulation, and feature evaluation, such as fruit ripeness assessment, along with the emerging field of kitchen robotics. Through this interdisciplinary exploration, we aim to inspire researchers, engineers, and practitioners to harness the power of tactile-sensing technology for transformative advancements in agri-food robotics. By providing a comprehensive understanding of the current landscape and future prospects, this review paper serves as a valuable resource for driving progress in the field of tactile sensing and its application in agri-food systems.

Faster Indentation Influences Skin Deformation To Reduce Tactile Discriminability of Compliant Objects

To discriminate the compliance of soft objects, we rely upon spatiotemporal cues in the mechanical deformation of the skin. However, we have few direct observations of skin deformation over time, in particular how its response differs with indentation velocities and depths, and thereby helps inform our perceptual judgments. To help fill this gap, we develop a 3D stereo imaging method to observe contact of the skin's surface with transparent, compliant stimuli. Experiments with human-subjects, in passive touch, are conducted with stimuli varying in compliance, indentation depth, velocity, and time duration. The results indicate that contact durations greater than 0.4 s are perceptually discriminable. Moreover, compliant pairs delivered at higher velocities are more difficult to discriminate because they induce smaller differences in deformation. In a detailed quantification of the skin's surface deformation, we find that several, independent cues aid perception. In particular, the rate of change of gross contact area best correlates with discriminability, across indentation velocities and compliances. However, cues associated with skin surface curvature and bulk force are also predictive, for stimuli more and less compliant than skin, respectively. These findings and detailed measurements seek to inform the design of haptic interfaces.

A Classification Method for Workers' Physical Risk

In Industry 4.0 scenarios, wearable sensing allows the development of monitoring solutions for workers' risk prevention. Current approaches aim to identify the presence of a risky event, such as falls, when it has already occurred. However, there is a need to develop methods capable of identifying the presence of a risk condition in order to prevent the occurrence of the damage itself. The measurement of vital and non-vital physiological parameters enables the worker's complex state estimation to identify risk conditions preventing falls, slips and fainting, as a result of physical overexertion and heat stress exposure. This paper aims at investigating classification approaches to identify risk conditions with respect to normal physical activity by exploiting physiological measurements in different conditions of physical exertion and heat stress. Moreover, the role played in the risk identification by specific sensors and features was investigated. The obtained results evidenced that k-Nearest Neighbors is the best performing algorithm in all the experimental conditions exploiting only information coming from cardiorespiratory monitoring (mean accuracy 88.7±7.3% for the model trained with max(HR), std(RR) and std(HR)).

Learning-Based Slip Detection for Robotic Fruit Grasping and Manipulation under Leaf Interference

Robotic harvesting research has seen significant achievements in the past decade, with breakthroughs being made in machine vision, robot manipulation, autonomous navigation and mapping. However, the missing capability of obstacle handling during the grasping process has severely reduced harvest success rate and limited the overall performance of robotic harvesting. This work focuses on leaf interference caused slip detection and handling, where solutions to robotic grasping in an unstructured environment are proposed. Through analysis of the motion and force of fruit grasping under leaf interference, the connection between object slip caused by leaf interference and inadequate harvest performance is identified for the first time in the literature. A learning-based perception and manipulation method is proposed to detect slip that causes problematic grasps of objects, allowing the robot to implement timely reaction. Our results indicate that the proposed algorithm detects grasp slip with an accuracy of 94%. The proposed sensing-based manipulation demonstrated great potential in robotic fruit harvesting, and could be extended to other pick-place applications.

Multimodal Fibrous Static and Dynamic Tactile Sensor

A highly versatile, low-cost, and robust tactile sensor capable of acquiring load measurements under static and dynamic modes employing a poly(vinylidene fluoride-co-trifluoroethylene) [P(VDF-TrFE)] micronanofiber element is presented. The sensor is comprised of three essential layers, a fibrous core P(VDF-TrFE) layer and two Ni/Cu conductive fabric electrode layers, with a total thickness of less than 300 μm. Using an in situ electrospinning process, the core fibers are deposited directly to a soft poly(dimethylsiloxane) (PDMS) fingertip. The core layer conforms to the surface and requires no additional processing, exhibiting the capability of the in situ electrospinning fabrication method to alleviate poor surface contacts and resolve issues associated with adhesion. The fabricated tactile sensor displayed a reliable and consistent measurement performance of static and instantaneous dynamic loads over a total of 30 000 test cycles. The capabilities and implications of the presented tactile sensor design for multimodal sensing in robot tactile sensing applications is further discussed and elucidated.

Neurophysiology of slip sensation and grip reaction: insights for hand prosthesis control of slippage

Sensory feedback is pivotal for a proficient dexterity of the hand. By modulating the grip force in function of the quick and not completely predictable change of the load force, grabbed objects are prevented to slip from the hand. Slippage control is an enabling achievement to all manipulation abilities. However, in hand prosthetics, the performance of even the most innovative research solutions proposed so far to control slippage remain distant from the human physiology. Indeed, slippage control involves parallel and compensatory activation of multiple mechanoceptors, spinal and supraspinal reflexes, and higher-order voluntary behavioral adjustments. In this work, we reviewed the literature on physiological correlates of slippage to propose a three-phases model for the slip sensation and reaction. Furthermore, we discuss the main strategies employed so far in the research studies that tried to restore slippage control in amputees. In the light of the proposed three-phase slippage model and from the weaknesses of already implemented solutions, we proposed several physiology-inspired solutions for slippage control to be implemented in the future hand prostheses. Understanding the physiological basis of slip detection and perception and implementing them in novel hand feedback system would make prosthesis manipulation more efficient and would boost its perceived naturalness, fostering the sense of agency for the hand movements.

DESIGN AND CHARACTERIZATION OF A MINIATURE THREE-AXIAL MEMS FORCE SENSOR

This paper reports the design, fabrication and calibration results of a miniature cross-shaped three-axial piezoresistive force sensor, which can simultaneously detect three force components in orthogonal directions. MEMS technology was used to fabricate the sensor structure and deposit a phosphosilicate layer on the silicon wafer to form piezoresistive resistors. Using the finite element simulation, the developed sensor performance characteristics, such as linearity, repeatability, sensitivity, and hysteresis, are analyzed for different arrangements of eight piezoresistors on the silicon beam surface. The sensor performance was experimentally validated by monitoring the voltage variation of Wheatstone bridge when a load-bearing rigid rod was loaded in three different directions by a set of weights. Calibration results exhibited linear output responses with the maximum linearity of 0.98 and small crosstalk below 7%. The MEMS sensor repeatability was tested with a commercial stepper motor by measuring a step function-varying profile force was applied to the sensor. Further optimization of the sensor design for sensing six degrees of freedom movement is envisaged with its sensitivity enhancement by the silicon substrate reduction.

Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation

In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.

Implantable Neural Interfaces and Wearable Tactile Systems for Bidirectional Neuroprosthetics Systems

Neuroprosthetics and neuromodulation represent a promising field for several related applications in the central and peripheral nervous system, such as the treatment of neurological disorders, the control of external robotic devices, and the restoration of lost tactile functions. These actions are allowed by the neural interface, a miniaturized implantable device that most commonly exploits electrical energy to fulfill these operations. A neural interface must be biocompatible, stable over time, low invasive, and highly selective; the challenge is to develop a safe, compact, and reliable tool for clinical applications. In case of anatomical impairments, neuroprosthetics is bound to the need of exploring the surrounding environment by fast-responsive and highly sensitive artificial tactile sensors that mimic the natural sense of touch. Tactile sensors and neural interfaces are closely interconnected since the readouts from the first are required to convey information to the neural implantable apparatus. The role of these devices is pivotal hence technical improvements are essential to ensure a secure system to be eventually adopted in daily life. This review highlights the fundamental criteria for the design and microfabrication of neural interfaces and artificial tactile sensors, their use in clinical applications, and future enhancements for the release of a second generation of devices.

Restoring Tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands

Despite previous studies on the restoration of tactile sensation on the fingers and the hand, there are no examples of use of the routed sensory information to finely control the prosthesis hand in complex grasp and manipulation tasks. Here it is shown that force and slippage sensations can be elicited in an amputee subject by means of biologically-inspired slippage detection and encoding algorithms, supported by a stick-slip model of the performed grasp. A combination of cuff and intraneural electrodes was implanted for eleven weeks in a young woman with hand amputation, and was shown to provide close-to-natural force and slippage sensations, paramount for significantly improving the subject's manipulative skills with the prosthesis. Evidence is provided about the improvement of the subject's grasping and manipulation capabilities over time, thanks to neural feedback. The elicited tactile sensations enabled the successful fulfillment of fine grasp and manipulation tasks with increasing complexity. Grasp performance was quantitatively assessed by means of instrumented objects and a purposely developed metrics. Closed-loop control capabilities enabled by the neural feedback were compared to those achieved without feedback. Further, the work investigates whether the described amelioration of motor performance in dexterous tasks had as central neurophysiological correlates changes in motor cortex plasticity and whether such changes were of purely motor origin, or else the effect of a strong and persistent drive of the sensory feedback.

Learning Spatio Temporal Tactile Features with a ConvLSTM for the Direction Of Slip Detection

Robotic manipulators have to constantly deal with the complex task of detecting whether a grasp is stable or, in contrast, whether the grasped object is slipping. Recognising the type of slippage-translational, rotational-and its direction is more challenging than detecting only stability, but is simultaneously of greater use as regards correcting the aforementioned grasping issues. In this work, we propose a learning methodology for detecting the direction of a slip (seven categories) using spatio-temporal tactile features learnt from one tactile sensor. Tactile readings are, therefore, pre-processed and fed to a ConvLSTM that learns to detect these directions with just 50 ms of data. We have extensively evaluated the performance of the system and have achieved relatively high results at the detection of the direction of slip on unseen objects with familiar properties (82.56% accuracy).

Recent Advances in Tactile Sensing Technology

Research on tactile sensing technology has been actively conducted in recent years to pave the way for the next generation of highly intelligent devices. Sophisticated tactile sensing technology has a broad range of potential applications in various fields including: (1) robotic systems with tactile sensors that are capable of situation recognition for high-risk tasks in hazardous environments; (2) tactile quality evaluation of consumer products in the cosmetic, automobile, and fabric industries that are used in everyday life; (3) robot-assisted surgery (RAS) to facilitate tactile interaction with the surgeon; and (4) artificial skin that features a sense of touch to help people with disabilities who suffer from loss of tactile sense. This review provides an overview of recent advances in tactile sensing technology, which is divided into three aspects: basic physiology associated with human tactile sensing, the requirements for the realization of viable tactile sensors, and new materials for tactile devices. In addition, the potential, hurdles, and major challenges of tactile sensing technology applications including artificial skin, medical devices, and analysis tools for human tactile perception are presented in detail. Finally, the review highlights possible routes, rapid trends, and new opportunities related to tactile devices in the foreseeable future.

Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human-Robot Interaction

The use of tactile perception can help first response robotic teams in disaster scenarios, where visibility conditions are often reduced due to the presence of dust, mud, or smoke, distinguishing human limbs from other objects with similar shapes. Here, the integration of the tactile sensor in adaptive grippers is evaluated, measuring the performance of an object recognition task based on deep convolutional neural networks (DCNNs) using a flexible sensor mounted in adaptive grippers. A total of 15 classes with 50 tactile images each were trained, including human body parts and common environment objects, in semi-rigid and flexible adaptive grippers based on the fin ray effect. The classifier was compared against the rigid configuration and a support vector machine classifier (SVM). Finally, a two-level output network has been proposed to provide both object-type recognition and human/non-human classification. Sensors in adaptive grippers have a higher number of non-null tactels (up to 37% more), with a lower mean of pressure values (up to 72% less) than when using a rigid sensor, with a softer grip, which is needed in physical human–robot interaction (pHRI). A semi-rigid implementation with 95.13% object recognition rate was chosen, even though the human/non-human classification had better results (98.78%) with a rigid sensor.

Evaluation of Pressure Capacitive Sensors for Application in Grasping and Manipulation Analysis

The analysis of the human grasping and manipulation capabilities is paramount for investigating human sensory-motor control and developing prosthetic and robotic hands resembling the human ones. A viable solution to perform this analysis is to develop instrumented objects measuring the interaction forces with the hand. In this context, the performance of the sensors embedded in the objects is crucial. This paper focuses on the experimental characterization of a class of capacitive pressure sensors suitable for biomechanical analysis. The analysis was performed in three loading conditions (Distributed load, 9 Tips load, and Wave-shaped load, thanks to three different inter-elements) via a traction/compression testing machine. Sensor assessment was also carried out under human- like grasping condition by placing a silicon material with the same properties of prosthetic cosmetic gloves in between the sensor and the inter-element in order to simulate the human skin. Data show that the input-output relationship of the analyzed, sensor is strongly influenced by both the loading condition (i.e., type of inter-element) and the grasping condition (with or without the silicon material). This needs to be taken into account to avoid significant measurement error. To go over this hurdle, the sensors have to be calibrated under each specific condition in order to apply suitable corrections to the sensor output and significantly improve the measurement accuracy.