Material recognition based on thermal cues: Mechanisms and applications

Flexible Ag2Se Thermoelectric Films Enable the Multifunctional Thermal Perception in Electronic Skins

Skin is critical for shaping our interactions with the environment. The electronic skin (E-skin) has emerged as a promising interface for medical devices to replicate the functions of damaged skin. However, exploration of thermal perception, which is crucial for physiological sensing, has been limited. In this work, a multifunctional E-skin based on flexible thermoelectric Ag2Se films is proposed, which utilizes the Seebeck effect to replicate the sensory functions of natural skin. The E-skin can enable capabilities including temperature perception, tactile perception, contactless perception, and material recognition by analyzing the thermal conduction behaviors of various materials. To further validate the capabilities of constructed E-skins, a wearable device with multiple sensory channels was fabricated and tested for gesture recognition. This work highlights the potential for using flexible thermoelectric materials in advanced biomedical applications including health monitoring and smart prosthetics.

The characterization of thermal perception in recreational surfers wearing wetsuits

The purpose of this study was to characterize the perception of heat loss, comfort, and wetness in recreational surfers wearing wetsuits, to compare these data with changes in skin temperature reported in prior studies, and to examine the impact of wetsuit thickness, zipper location, and accessory use on thermal sensation and comfort. Following their surf session, nine-hundred and three male (n = 735) and female (n = 168) recreational surfers responded to a series of questions regarding thermal comfort/sensation, wetsuit characteristics, and surfing history. Average whole body thermal sensation rating was 0.8 ± 3.6 on a scale of -10 to +10 and average whole body thermal comfort rating was 1.5 ± 1.2, midway between "just comfortable" and "comfortable." Overall, surfers felt coldest in their feet, hands, and head. Under their wetsuits, surfers felt the coldest, wettest, and least comfortable in their chest, lower legs, lower arms, and upper back. Wetsuit accessory use had the greatest impact on regions identified as coldest, least comfortable, and wettest. These data suggest that wetsuit design should focus on optimizing water access points and improving accessories for the feet, hands, and head.

Thermal Cues Composed of Sequences of Pulses for Transferring Data via a Haptic Thermal Interface

This research study is the preliminary phase of an effort to develop a generic data transfer method via human haptic thermal sensation (i.e., a coded language such as Morse or Braille). For the method to be effective, it must include a large variety of short, recognizable cues. Hence, we propose the concept of cues based on sequences of thermal pulses: combinations of warm and cool pulses with several levels of intensity. The objective of this study was to determine the feasibility of basing a generic data transfer method on haptic thermal cues using sequences of short pulses. The research included defining the basic characteristics of the stimuli parameters and developing practical methods for generating and measuring them. Several patterns of different sequences were designed considering the relevant data known to date and improved by implementing new insights acquired throughout the tests that were conducted. The final thermal cues presented to the participants were sensed by touch and clearly recognized. The results of this study indicate that developing this new method is feasible and that it could be applicable in various scenarios. In addition, the low impact measured on the user's skin temperature represents an inherent advantage for future implementation. This report presents promising findings and offers insights for further investigations.

Skin-interfaced electronics: A promising and intelligent paradigm for personalized healthcare

Skin-interfaced electronics (skintronics) have received considerable attention due to their thinness, skin-like mechanical softness, excellent conformability, and multifunctional integration. Current advancements in skintronics have enabled health monitoring and digital medicine. Particularly, skintronics offer a personalized platform for early-stage disease diagnosis and treatment. In this comprehensive review, we discuss (1) the state-of-the-art skintronic devices, (2) material selections and platform considerations of future skintronics toward intelligent healthcare, (3) device fabrication and system integrations of skintronics, (4) an overview of the skintronic platform for personalized healthcare applications, including biosensing as well as wound healing, sleep monitoring, the assessment of SARS-CoV-2, and the augmented reality-/virtual reality-enhanced human-machine interfaces, and (5) current challenges and future opportunities of skintronics and their potentials in clinical translation and commercialization. The field of skintronics will not only minimize physical and physiological mismatches with the skin but also shift the paradigm in intelligent and personalized healthcare and offer unprecedented promise to revolutionize conventional medical practices.

Tracking changes in touch desire and touch avoidance before and after the COVID-19 outbreak

Touch is essential for survival, social bonding, and overall health. However, the COVID-19 pandemic calls for an abrupt withdrawal from physical contact, and the prolonged lockdown has left many people in solitude without touch for months. This unprecedented dissociation from touch has cast a shadow on people's mental and physical well-being. Here we approached the issue by examining COVID-19's impact on people's touch attitudes. We analyzed people's desire and avoidance for animate and inanimate targets based on large-scale Japanese Twitter posts over an 8-year span. We analyzed the impact of the COVID-19 outbreak with the difference-in-differences estimation method, which can estimate the impact while accounting for other changes over time such as seasonality or long-term effects. As a result, we found that people's desire for touching the human body and pet animals increased significantly after the COVID-19 outbreak and remained high afterward. In contrast, the avoidance of touching everyday objects (e.g., doorknobs and money) increased immediately after the outbreak but gradually returned to the pre-COVID-19 levels. Our findings manifest the impact of COVID-19 on human touch behavior. Most importantly, they highlight the sign of "skin hunger," a public health crisis due to social distancing, and call attention to the trend that people are becoming less aware of infection control as COVID-19 persists.

Thermal Perception for Information Transmission: Theoretical Analysis, Device Design, and Experimental Verification

Information transmission is a fundamental issue for human-computer interaction (HCI). Traditional interaction methods through visual, voice, and force haptics are very mature. However, the thermal perception (TP) for HCI is not studied in depth. This work proposes the TP-based information transmission framework. Firstly, we investigated the human hand-object contact heat transfer model and the temperature perception resolution of the hand and verified the feasibility of spatiotemporal temperature stimulation for information transmission by simulations. Then, a thermal device was designed, which utilized a 7×5 Peltier array, a water cooler, temperature sensors, and a control module to realize various static and dynamic spatiotemporal temperature patterns stimulation. Finally, we implemented a device prototype and recruited 20 subjects for experimental studies. The results show that the device can display various temperature patterns and provide thermal stimulations with high precision and speed. Furthermore, the subjects can accurately recognize different temperature values, icons, codes, and waveforms with their palm and fingers after a few times of training, which validates the TP-based information transmission method. Therefore, people can apply this method to interact with machines for information feedback, virtual reality, augmented reality, etc.

Time Delay Affects Thermal Discrimination

Time order errors have been investigated in several fields, and the time delay between subsequent stimuli in discrimination tasks is one example of such errors. However, the effect of these types of errors in thermal discrimination tasks is understudied. To evaluate the effect of inter-stimulus interval (ISI) on thermal perception, we used a discrimination task with a staircase method between two non-zero thermal stimuli. We found that JND _ISI=0s was 3.10 and increased by 11.9% and 21.2% at JND _ISI=3s and JND _ISI=9s, respectively. Statistical analysis revealed that ISI was a statistically significant effect ( ) on thermal perception in our task. Future studies on thermal perception should keep the ISI consistent and report the time.

Flexible Pyroresistive Graphene Composites for Artificial Thermosensation Differentiating Materials and Solvent Types

When we touch an object, thermosensation allows us to perceive not only the temperature but also wetness and types of materials with different thermophysical properties (i.e., thermal conductivity and heat capacity) of objects. Emulation of such sensory abilities is important in robots, wearables, and haptic interfaces, but it is challenging because they are not directly perceptible sensations but rather learned abilities via sensory experiences. Emulating the thermosensation of human skin, we introduce an artificial thermosensation based on an intelligent thermo-/calorimeter (TCM) that can objectively differentiate types of contact materials and solvents with different thermophysical properties. We demonstrate a TCM based on pyroresistive composites with ultrahigh sensitivity (11.2% °C^-1) and high accuracy (<0.1 °C) by precisely controlling the melt-induced volume expansion of a semicrystalline polymer, as well as the negative temperature coefficient of reduced graphene oxide. In addition, the ultrathin TCM with coplanar electrode design shows deformation-insensitive temperature sensing, facilitating wearable skin temperature monitoring with accuracy higher than a commercial thermometer. Moreover, the TCM with a high pyroresistivity can objectively differentiate types of contact materials and solvents with different thermophysical properties. In a proof-of-principle application, our intelligent TCM, coupled with a machine-learning algorithm, enables objective evaluation of the thermal attributes (coolness and wetness) of skincare products.

Multifunctional Soft Robotic Finger Based on a Nanoscale Flexible Temperature-Pressure Tactile Sensor for Material Recognition

Robotic hands with tactile perception can perform more advanced and safer operations, such as material recognition. Nanowires with high sensitivity, fast response, and low power consumption are suitable for multifunctional flexible tactile sensors to provide the tactile perception of robotic hands. In this work, we designed a multifunctional soft robotic finger with a built-in nanoscale temperature-pressure tactile sensor for material recognition. The flexible multifunctional tactile sensor integrates a nanowire-based temperature sensor and a conductive sponge pressure sensor to measure the temperature change rate and contact pressure simultaneously. The developed nanoscale temperature and conductive sponge pressure sensor can reach a high sensitivity of 1.196%/°C and 13.29%/kPa, respectively. With this multifunctional tactile sensor, the soft finger can quickly recognize four metals within three contact pressure ranges and 13 materials within a high contact pressure range. By combining tactile information and artificial neural networks, the soft finger can recognize the materials precisely with a high recognition accuracy of 92.7 and 95.9%, respectively. This work proves the application potential of the multifunctional soft robot finger in material recognition.

Material Recognition via Heat Transfer Given Ambiguous Initial Conditions

Humans and robots can recognize materials with distinct thermal effusivities by making physical contact and observing temperatures during heat transfer. This works well with room temperature materials, yet research has shown that contact with distinct materials can result in similar temperatures and confusion when one material is heated or cooled. To thoroughly investigate this form of ambiguity, we designed a psychophysical experiment in which a participant discriminates between two materials given initial conditions that result in similar temperatures (i.e., ambiguous initial conditions). In this article, we conducted a study with 32 human participants and a robot. Humans and the robot confused the materials. We also found that robots can overcome this ambiguity using two temperature sensors with different temperatures prior to contact. We support this conclusion based on a mathematical proof using a heat transfer model and empirical results in which a robot achieved 100% accuracy compared to 5% human accuracy. Our results also indicate that robots with a single temperature sensor can use subtle cues to outperform humans. Overall, our work provides insights into challenging conditions for material recognition via heat transfer, and suggests methods by which robots can overcome these challenges.

Touch-Point Detection Using Thermal Video With Applications to Prevent Indirect Virus Spread

Viral and bacterial pathogens can be transmitted through direct contact with contaminated surfaces. Efficient decontamination of contaminated surfaces could lead to decreased disease transmission, if optimized methods for detecting contaminated surfaces can be developed. Here we describe such a method whereby thermal tracking technology is utilized to detect thermal signatures incurred by surfaces through direct contact. This is applicable in public places to assist with targeted sanitation and cleaning efforts to potentially reduce chance of disease transmission. In this study, we refer to the touched region of the surface as a "touch-point" and examine how the touch-point regions can be automatically localized with a computer vision pipeline of a thermal image sequence. The pipeline mainly comprises two components: a single-frame and a multi-frame analysis. The single-frame analysis consists of a Background subtraction method for image pre-processing and a U-net deep learning model for segmenting the touch-point regions. The multi-frame analysis performs a summation of the outputs from the single-frame analysis and creates a cumulative map of touch-points. Results show that the touch-point detection pipeline can achieve 75.0% precision and 81.5% F1-score for the testing experiments of predicting the touch-point regions. This preliminary study shows potential applications of preventing indirect pathogen spread in public spaces and improving the efficiency of cleaning sanitation.

Organic Haptics: Intersection of Materials Chemistry and Tactile Perception

The goal of the field of haptics is to create technologies that manipulate the sense of touch. In virtual and augmented reality, haptic devices are for touch what loudspeakers and RGB displays are for hearing and vision. Haptic systems that utilize micromotors or other miniaturized mechanical devices (e.g., for vibration and pneumatic actuation) produce interesting effects, but are quite far from reproducing the feeling of real materials. They are especially deficient in recapitulating surface properties: fine texture, friction, viscoelasticity, tack, and softness. The central argument of this Progress Report is that to reproduce the feel of everyday objects requires chemistry: molecular control over the properties of materials and ultimately design of materials which can change these properties in real time. Stimuli-responsive organic materials, such as polymers and composites, are a class of materials which can change their oxidation state, conductivity, shape, and rheological properties, and thus might be useful in future haptic technologies. Moreover, the use of such materials in research on tactile perception could help elucidate the limits of human tactile sensitivity. The work described represents the beginnings of this new area of inquiry, in which the defining approach is the marriage of materials science and psychology.

Cold and heavy: grasping the temperature-weight illusion

The apparent heaviness of weights placed on the skin depends on their temperature. We studied the effects of such a temperature–weight illusion (TWI) on perception and action in 21 healthy volunteers. Cold (18 °C), thermal-neutral (32 °C, skin temperature) and warm (41 °C) test objects were placed onto the palm of the non-dominant hand. Their veridical mass was 350 g (light) or 700 g (heavy). Perception of heaviness was assessed with two psychophysical experiments (magnitude estimation, cross modal matching). Cold heavy objects felt about 20% heavier than thermal-neutral objects of the same mass, shape and material. In a subsequent grip-lift experiment, the test objects were grasped with a precision grip of the dominant hand and lifted off the palm of the non-dominant hand. The grip and lift forces exerted by the fingertips were recorded. The temperature of the objects had significant effects (ANOVA, p < 0.05) on the peak grip and lift forces and on the peak grip force rate (i.e., the initial force incline). The peak grip force was about 10% higher when cold heavy objects were grasped and lifted, compared to lifts of otherwise identical thermal-neutral objects. The TWI was less pronounced when light objects or warm objects were handled. In conclusion, cooling of an object increases its apparent heaviness (perception) and influences scaling of the fingertip forces during grasping and lifting (action).

Thermal-Tactile Integration in Object Temperature Perception

The brain consistently faces a challenge of whether and how to combine the available information sources to estimate the properties of an object explored by hand. While object perception is an inference process involving multisensory inputs, thermal referral (TR) is an illusion demonstrating how the interaction between thermal and tactile systems can lead to deviations from physical reality-when observers touch three stimulators simultaneously with the middle three fingers of one hand but only the outer two stimulators are heated (or cooled), thermal uniformity is perceived across three fingers. Here, we used TR of warmth to examine the thermal-tactile interaction in object temperature perception. We show that TR is consistent with precision-weighted averaging of thermal sensation across tactile locations. Furthermore, we show that prolonged contact with TR stimulation results in adaptation to the local variations of veridical temperatures instead of the thermal uniformity perceived across three fingers. Our results illuminate the flexibility of processing that underlies thermal-tactile interactions and serve as a basis for thermal display design.