Piezothermic Transduction of Functional Composite Materials

Multimodal tactile sensing fused with vision for dexterous robotic housekeeping

As robots are increasingly participating in our daily lives, the quests to mimic human abilities have driven the advancements of robotic multimodal senses. However, current perceptual technologies still unsatisfied robotic needs for home tasks/environments, particularly facing great challenges in multisensory integration and fusion, rapid response capability, and highly sensitive perception. Here, we report a flexible tactile sensor utilizing thin-film thermistors to implement multimodal perceptions of pressure, temperature, matter thermal property, texture, and slippage. Notably, the tactile sensor is endowed with an ultrasensitive (0.05 mm/s) and ultrafast (4 ms) slip sensing that is indispensable for dexterous and reliable grasping control to avoid crushing fragile objects or dropping slippery objects. We further propose and develop a robotic tactile-visual fusion architecture that seamlessly encompasses multimodal sensations from the bottom level to robotic decision-making at the top level. A series of intelligent grasping strategies with rapid slip feedback control and a tactile-visual fusion recognition strategy ensure dexterous robotic grasping and accurate recognition of daily objects, handling various challenging tasks, for instance grabbing a paper cup containing liquid. Furthermore, we showcase a robotic desktop-cleaning task, the robot autonomously accomplishes multi-item sorting and cleaning desktop, demonstrating its promising potential for smart housekeeping.

Multifunctional Electronic Skins Enable Robots to Safely and Dexterously Interact with Human

Human-robot collaboration is playing more and more important roles in current deployments of robotic systems in our lives. Haptic perception and intelligent control are essential to ensure safety and efficiency of human-robot interaction. However, existing robotic sensory and control systems are deficient in terms of performance issues, complexity, and cost. Here, the authors report a multifunctional electronic skin (e-skin) incorporating multiple perceptions with intelligent robotic control, by which robots can safely and dexterously interact with humans. The e-skin with a simple and cost-effective sensory structure has multimodal perceptions of proximity, temperature, contact force, and contact position with broad measuring range, high sensitivity, and fast response. The e-skin is applied onto robots to accomplish obstacle avoidance, safe and dexterous human-robot interaction, smart teaching, and playing Tai-Chi, which demonstrate a broad range of applications for intelligent robots equipped with e-skins.

Interface sensors with skin piezo-thermic transduction enable motion artifact removal for wearable physiological monitoring

Accurate measurements on physiological parameters using wearable monitoring devices during physical exercises are essential for personal healthcare and rehabilitation training, but still challenging owing to various motion artifacts (MA) caused by the interfacial dynamic change between wearable sensors and human skin. Here, we propose an interface sensor to detect noncontact proximity and contact pressure between wearable sensors and human skin. The interface sensor employs natural piezo-thermic transduction of human skin and enables direct interfacial proximity/pressure detection by using simple thin-film thermistors to detect the interfacial thermal field change. We develop a wearable watch-type heart rate (HR) monitor utilizing interface sensors to remove MA for a photoplethysmography (PPG) sensor through adaptive filtering. To validate the method, we conduct experiments for multiple subjects, who carry out HR monitoring using the wearable device while doing various physical exercises. The PPG-based HR estimations are corrected through MA removal using interface sensors and compared with that using conventional accelerometer-based MA removal. The experimental results verify that the interface sensors capture the interfacial dynamic change between the PPG sensor and skin better, and obtain more accurate HR estimations during irregular and muscle strength exercises. Utilizing natural transduction of human skin and simple thermometry, the interface sensor provides an advantageous way to overcome MA for wearable monitoring devices during physical activities and thus broadens wearable monitoring applications.

Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition

Robot hands with tactile perception can improve the safety of object manipulation and also improve the accuracy of object identification. Here, we report the integration of quadruple tactile sensors onto a robot hand to enable precise object recognition through grasping. Our quadruple tactile sensor consists of a skin-inspired multilayer microstructure. It works as thermoreceptor with the ability to perceive thermal conductivity of a material, measure contact pressure, as well as sense object temperature and environment temperature simultaneously and independently. By combining tactile sensing information and machine learning, our smart hand has the capability to precisely recognize different shapes, sizes, and materials in a diverse set of objects. We further apply our smart hand to the task of garbage sorting and demonstrate a classification accuracy of 94% in recognizing seven types of garbage.

Temperature and Strain Compensation for Flexible Sensors Based on Thermosensation

Flexible sensors have wide applications in wearable electronics, health monitoring, humanoid robotics, and smart prosthesis. Problems of temperature drift and bending/stretching strain are challenging and should not be neglected in practical applications of flexible sensors. Here, we report a novel temperature and strain compensation method for thermosensation-based flexible sensors. Thermosensation is human-skin-inspired perception, which inspires diverse flexible sensors (pressure sensor, flow sensor, temperature sensor, material sensor, proximity sensor, etc.) and multisensory electronic skin. Thermosensation-based flexible sensors utilize thin-film sensing thermistors to detect external physical stimuli through perceptions of the conductive and convective heat transfers toward the surroundings, which enables high-density integration of multisensations while minimizing complexity due to the uniform sensing principle of thermistors that have simple structures and easy operations. To overcome the negative effects of temperature drift and bending/stretching strain in these flexible sensors, we propose to monolithically integrate a compensating thermistor that has a similar geometric shape and is of the same material with the sensing thermistor into a Wheatstone-bridge feedback circuit. When the sensing and compensating thermistors meet geometric similarity, the compensations of temperature and strain are self-sustained by a feedback control of a circuit. The effectiveness is validated through theoretical analysis and experiment measurements. As examples, flexible pressure sensor and flexible flow sensor with temperature and strain compensations are demonstrated. Results indicate that the temperature and strain effects can be tremendously eliminated using the proposed compensation method, which is fast, self-sustained, and expedient to realize. The compensation method enriches competences of flexible sensors and demonstrates competitive advantages for diverse flexible and stretching applications of wearable electronics.

A Wearable Sensor Using Structured Silver-Particle Reinforced PDMS for Radial Arterial Pulse Wave Monitoring

Human pulse signals contain important and useful physiological information for the auxiliary diagnosis of cardiovascular disease. Here, a wearable pulse sensor based on piezo-thermic transduction is reported using a structured silver-particle reinforced polydimethylsiloxane (PDMS) membrane, for monitoring radial arterial pulse waves. The structured silver-particle reinforced PDMS membrane is optimally designed to meet the specific requirements on sensitivity, linearity, and effective preload measuring range for pulse detection by adjusting the air gap volume fraction and silver particle volume fraction of the structured material. The sensor is endowed with high sensitivity, good linearity in preload measuring range, allowing to detect the subtle pulse waveforms of subjects at different ages under different contact pressures, such as superficial (Fu), medium (Zhong) and deep (Chen). The developed pulse device provides a promising approach for homecare pulse monitoring.