Stretchable and Self-Powered Mechanoluminescent Triboelectric Nanogenerator Fibers toward Wearable Amphibious Electro-Optical Sensor Textiles

Flexible electro-optical dual-mode sensor fibers with capability of the perceiving and converting mechanical stimuli into digital-visual signals show good prospects in smart human-machine interaction interfaces. However, heavy mass, low stretchability, and lack of non-contact sensing function seriously impede their practical application in wearable electronics. To address these challenges, a stretchable and self-powered mechanoluminescent triboelectric nanogenerator fiber (MLTENGF) based on lightweight carbon nanotube fiber is successfully constructed. Taking advantage of their mechanoluminescent-triboelectric synergistic effect, the well-designed MLTENGF delivers an excellent enhancement electrical signal of 200% and an evident optical signal whether on land or underwater. More encouragingly, the MLTENGF device possesses outstanding stability with almost unchanged sensitivity after stretching for 200%. Furthermore, an extraordinary non-contact sensing capability with a detection distance of up to 35 cm is achieved for the MLTENGF. As application demonstrations, MLTENGFs can be used for home security monitoring, intelligent zither, traffic vehicle collision avoidance, and underwater communication. Thus, this work accelerates the development of wearable electro-optical textile electronics for smart human-machine interaction interfaces.

Conformal Neuromorphic Bioelectronics for Sense Digitalization

Sense digitalization, the process of transforming sensory experiences into digital data, is an emerging research frontier that links the physical world with human perception and interaction. Inspired by the adaptability, fault tolerance, robustness, and energy efficiency of biological senses, this field drives the development of numerous innovative digitalization techniques. Neuromorphic bioelectronics, characterized by biomimetic adaptability, stand out for their seamless bidirectional interactions with biological entities through stimulus-response and feedback loops, incorporating bio-neuromorphic intelligence for information exchange. This review illustrates recent progress in sensory digitalization, encompassing not only the digital representation of physical sensations such as touch, light, and temperature, correlating to tactile, visual, and thermal perceptions, but also the detection of biochemical stimuli such as gases, ions, and neurotransmitters, mirroring olfactory, gustatory, and neural processes. It thoroughly examines the material design, device manufacturing, and system integration, offering detailed insights. However, the field faces significant challenges, including the development of new device/system paradigms, forging genuine connections with biological systems, ensuring compatibility with the semiconductor industry and overcoming the absence of standardization. Future ambition includes realization of biocompatible neural prosthetics, exoskeletons, soft humanoid robots, and cybernetic devices that integrate smoothly with both biological tissues and artificial components.

Superstrong Ionogel Enabled by Coacervation-Induced Nanofibril Assembly for Sustainable Moisture Energy Harvesting

Ionogels have grabbed significant interest in various applications, from sensors and actuators to wearable electronics and energy storage devices. However, current ionogels suffer from low strength and poor ionic conductivity, limiting their performance in practical applications. Here, inspired by the mechanical reinforcement of natural biomacromolecules through noncovalent aggregates, a strategy is proposed to construct nanofibril-based ionogels through complex coacervation-induced assembly. Cellulose nanofibrils (CNFs) can bundle together with poly(ionic liquid) (PIL) to form a superstrong nanofibrous network, in which the ionic liquid (IL) can be retained to form ionogels with high liquid inclusion and ionic conductivity. The strength of the CNF-PIL-IL ionogels can be tuned by the IL content over a wide range of up to 78 MPa. The optical transparency, high strength, and hygroscopicity enabled them to be promising candidates in moist-electricity generation and applications such as energy harvesting windows and wearable power generators. In addition, the ionogels are degradable and the ionogel-based generators can be recycled through dehydration. Our strategy suggests perspectives for the fabrication of high-strength and multifunctional ionogels for sustainable applications.

Biomimetic Electronic Skin for Robots Aiming at Superior Dynamic-Static Perception and Material Cognition Based on Triboelectric-Piezoresistive Effects

Empowering robots with tactile perception and even thinking as well as judgment capabilities similar to those of humans is an inevitable path for the development of future robots. Here, we propose a biomimetic electronic skin (BES) that truly serves and applies to robots to achieve superior dynamic-static perception and material cognition functionalities. First, the microstructured triboelectric and piezoresistive layers are fabricated by a facile template method followed by selected self-polymerization treatment, enabling BES with high sensitivity and a wide detection range. Further, through laminated-independent triboelectric and piezoresistive parts for perceiving dynamic and static pressures simultaneously, the BES is capable of supporting the robot hand to monitor the entire process during object grasping. Most importantly, by further combining a neural network model, an intelligent cognition system is constructed for real-time cognition of the object material species via one touch of the robot hand under arbitrary pressures, which goes beyond the human cognition ability.

Self-Adaptive Perception of Object's Deformability with Multiple Deformation Attributes Utilizing Biomimetic Mechanoreceptors

The perception of object's deformability in unstructured interactions relies on both kinesthetic and cutaneous cues to adapt the uncertainties of an object. However, the existing tactile sensors cannot provide adequate cutaneous cues to self-adaptively estimate the material softness, especially in non-standard contact scenarios where the interacting object deviates from the assumption of an elastic half-infinite body. This paper proposes an innovative design of a tactile sensor that integrates the capabilities of two slow-adapting mechanoreceptors within a soft medium, allowing self-decoupled sensing of local pressure and strain at specific locations within the contact interface. By leveraging these localized cutaneous cues, the sensor can accurately and self-adaptively measure the material softness of an object, accommodating variations in thicknesses and applied forces. Furthermore, when combined with a kinesthetic cue from the robot, the sensor can enhance tactile expression by the synergy of two relevant deformation attributes, including material softness and compliance. It is demonstrated that the biomimetic fusion of tactile information can fully comprehend the deformability of an object, hence facilitating robotic decision-making and dexterous manipulation.

A Biomimetic Nociceptor Using Centrosymmetric Crystals for Machine Intelligence

Pain sensation is a crucial aspect of perception in the body. Force-activated nociceptors encode electrochemical signals and yield multilevel information of pain, thus enabling smart feedback. Inspired by the natural template, multi-dimensional mechano-sensing materials provide promising approaches for biomimetic nociceptors in intelligent terminals. However, the reliance on non-centrosymmetric crystals has narrowed the range of these materials. Here centrosymmetric crystal Cr3+ -doped zinc gallogermanate (ZGGO:Cr) with multi-dimensional mechano-sensing is reported, eliminating the limitation of crystal structure. Under forces, ZGGO:Cr generates electrical signals imitating those of neuronal systems, and produces luminescence for spatial mapping of mechanical stimuli, suggesting a path toward bionic pain perception. On that basis, a wireless biomimetic nociceptor system is developed and a smart pain reflex in a robotic hand and robot-assisted biopsy surgery of rat and dog is achieved.

Self-Powered Sensing in Wearable Electronics─A Paradigm Shift Technology

With the advancements in materials science and micro/nanoengineering, the field of wearable electronics has experienced a rapid growth and significantly impacted and transformed various aspects of daily human life. These devices enable individuals to conveniently access health assessments without visiting hospitals and provide continuous, detailed monitoring to create comprehensive health data sets for physicians to analyze and diagnose. Nonetheless, several challenges continue to hinder the practical application of wearable electronics, such as skin compliance, biocompatibility, stability, and power supply. In this review, we address the power supply issue and examine recent innovative self-powered technologies for wearable electronics. Specifically, we explore self-powered sensors and self-powered systems, the two primary strategies employed in this field. The former emphasizes the integration of nanogenerator devices as sensing units, thereby reducing overall system power consumption, while the latter focuses on utilizing nanogenerator devices as power sources to drive the entire sensing system. Finally, we present the future challenges and perspectives for self-powered wearable electronics.

Non-rectangular neurostimulation waveforms elicit varied sensation quality and perceptive fields on the hand

Electrical stimulation of the nerves is known to elicit distinct sensations perceived in distal parts of the body. The stimulation is typically modulated in current with charge-balanced rectangular shapes that, although easily generated by stimulators available on the market, are not able to cover the entire range of somatosensory experiences from daily life. In this regard, we have investigated the effect of electrical neurostimulation with four non-rectangular waveforms in an experiment involving 11 healthy able-bodied subjects. Weiss curves were estimated and rheobase and chronaxie values were obtained showing increases in stimulation time required to elicit sensations for some waveforms. The localization of the sensations reported in the hand also appeared to differ between waveforms, although the total area did not vary significantly. Finally, the possibility of distinguishing different charge- and amplitude-matched stimuli was demonstrated through a two-alternative-forced-choice (2AFC) match-to-sample task, showing the ability of participants to successfully distinguish between waveforms with similar electrical characteristics but different shapes and charge transfer rates. This study provides evidence that, by using different waveforms to stimulate nerves, it is possible to affect not only the required charge to elicit sensations but also the sensation quality and its localization.

Implementing Monocular Visual-Tactile Sensors for Robust Manipulation

Tactile sensing is an essential capability for robots performing manipulation tasks. In this paper, we introduce a framework to build a monocular visual-tactile sensor for robotic manipulation tasks. Such a sensor is easy to manufacture with affordable ingredients and materials. Based on a marker-based detection method, the sensor can detect the contact positions on a flat or curved surface. In the case study, we have implemented a visual-tactile sensor design specifically through the framework proposed in this paper. The design is low cost and can be processed in a very short time, making it suitable for use as an exploratory study in the laboratory.

A review on sensory perception for dexterous robotic manipulation

Sensory perception for dexterous robotic hands is an active research area and recent progress in robotics. Effective dexterous manipulation requires robotic hands to accurately feedback their state or perceive the surrounding environment. This article reviews the state-of-the-art of sensory perception for dexterous robotic manipulation. Two types of sensors, such as intrinsic and extrinsic sensors, are introduced according to their function and layout in robotic hands. These sensors provide rich information to a robotic hand, which contains the posture, the contact information of objects, and the physical information of the environment. Then, a comprehensive analysis of perception methods including planning-level, control-level, and learning-level perceptions is presented. The information obtained from sensory perception can help robotic hands to make decisions effectively. Previously issued reviews mainly focus on the design of tactile senor, while we analyze and discuss the relationship among sensing, perception, and dexterous manipulation. Some potential research topics on sensory perception are also summarized and discussed.

Optical Microfibers for Sensing Proximity and Contact in Human-Machine Interfaces

The monitoring of proximity-contact events is essential for human-machine interactions, intelligent robots, and healthcare monitoring. We report a dual-modal sensor made with two functionalized optical microfibers (MFs), which is inspired by the somatosensory system of human skin. The integrated sensor with a hierarchical structure gradationally detects finger approaching and touching by measuring the relative humidity (RH) and force-triggered light intensity variations. Specifically, the RH sensory part shows enhanced evanescent absorption, achieving a sensitive RH measurement with a fast response (110 ms), a high resolution (0.11%RH), and a wide working range (10-100%RH). Enabled by the transition from guided modes into radiation modes of the waveguiding MF, the force sensory part exhibits a high sensitivity (6.2%/kPa) and a fast response (up to 1.5 kHz). By using a real-time data processing unit, the proximity-contact sensor (PCS) achieves continuous detection of the full-contact events, including finger approaching, contacting, pressing, releasing, and leaving. As a proof of concept, the electromagnetic-interference-free PCS enables a smart switch system to recognize the proximity and contact of bare/gloved fingers. Moreover, skin humidity detection and respiration monitoring are realized. These initial results pave the way toward a category of optical collaborative devices ranging from human-machine interfaces to multifunctional on-skin healthcare sensors.

Multi-modal self-adaptation during object recognition in an artificial cognitive system

The cognitive connection between the senses of touch and vision is probably the best-known case of multimodality. Recent discoveries suggest that the mapping between both senses is learned rather than innate. This evidence opens the door to a dynamic multimodality that allows individuals to adaptively develop within their environment. By mimicking this aspect of human learning, we propose a new multimodal mechanism that allows artificial cognitive systems (ACS) to quickly adapt to unforeseen perceptual anomalies generated by the environment or by the system itself. In this context, visual recognition systems have advanced remarkably in recent years thanks to the creation of large-scale datasets together with the advent of deep learning algorithms. However, this has not been the case for the haptic modality, where the lack of two-handed dexterous datasets has limited the ability of learning systems to process the tactile information of human object exploration. This data imbalance hinders the creation of synchronized datasets that would enable the development of multimodality in ACS during object exploration. In this work, we use a multimodal dataset recently generated from tactile sensors placed on a collection of objects that capture haptic data from human manipulation, together with the corresponding visual counterpart. Using this data, we create a multimodal learning transfer mechanism capable of both detecting sudden and permanent anomalies in the visual channel and maintaining visual object recognition performance by retraining the visual mode for a few minutes using haptic information. Our proposal for perceptual awareness and self-adaptation is of noteworthy relevance as can be applied by any system that satisfies two very generic conditions: it can classify each mode independently and is provided with a synchronized multimodal data set.

Bio-inspired flexible electronics for smart E-skin

"Learning from nature" provides endless inspiration for scientists to invent new materials and devices. Here, we review state-of-the-art technologies in flexible electronics, with a focus on bio-inspired smart skins. This review focuses on the development of E-skin for sensing a variety of parameters such as mechanical loads, temperature, light, and biochemical cues, with a trend of increased integration of multiple functions. It highlights the most recent advances in flexible electronics inspired by animals such as chameleons, squids, and octopi whose bodies have remarkable camouflage, mimicry, or self-healing attributes. Implantable devices, being overlapped with smart E-skin in a broad sense, are included in this review. This review outlines the remaining challenges in flexible electronics and the prospects for future development for biomedical applications. STATEMENT OF SIGNIFICANCE: This article reviews the state-of-the-art technologies of bio-inspired smart electronic skin (E-skin) developed in a "learning-mimicking-creating" (LMC) cycle. We emphasize the most recent innovations in the development of E-skin for sensing physical changes and biochemical cues, and for integrating multiple sensing modalities. We discuss the achievements in implantable materials, wireless communication, and device design pertaining to implantable flexible electronics. This review will provide prospective insights integrating material, electronics, and mechanical engineering viewpoints to foster new ideas for next-generation smart E-skin.

Self-Powered Interactive Fiber Electronics with Visual-Digital Synergies

Fiber electronics with mechanosensory functionality are highly desirable in healthcare, human-machine interfaces, and robotics. Most efforts are committed to optimize the electronically readable interface of fiber mechanoreceptor, while the user interface based on naked-eye readable output is rarely explored. Here, a scalable fiber electronics that can simultaneously visualize and digitize the mechanical stimulus without external power supply, named self-powered optoelectronic synergistic fiber sensors (SOEFSs), are reported. By coupling of space and surface charge polarization, a new mechanoluminescent (ML)-triboelectric synergistic effect is realized. It contributes to remarkable enhancement of both electrical (by 100%) and optical output (by 30%), as well as novel temporal-spatial resolution mode for motion capturing. Based on entirely new thermoplastic ML material system and spinning process, industrial-level continuously manufacture and recycling processes of SOEFS are realized. Furthermore, SOEFSs' application in human-machine interface, virtual reality, and underwater sensing, rescue, and information interaction is demonstrated.

Recent advances in wearable biosensing gloves and sensory feedback biosystems for enhancing rehabilitation, prostheses, healthcare, and virtual reality

Wearable sensing gloves and sensory feedback devices that record and enhance the sensations of the hand are used in healthcare, prosthetics, robotics, and virtual reality. Recent technological advancements in soft actuators, flexible bioelectronics, and wireless data acquisition systems have enabled the development of ergonomic, lightweight, and low-cost wearable devices. This review article includes the most up-to-date materials, sensors, actuators, and system-packaging technologies to develop wearable sensing gloves and sensory feedback devices. Furthermore, this review contemplates the use of wearable sensing gloves and sensory feedback devices together to advance their capabilities as assistive devices for people with prostheses and sensory impaired limbs. This review is divided into two sections: one detailing the technologies used to develop strain, pressure, and temperature sensors integrated with a multifunctional wearable sensing glove, and the other reviewing the devices and methods used for wearable sensory displays. We discuss the limitations of the current methods and technologies along with the future direction of the field. Overall, this paper presents an all-inclusive review of the technologies used to develop wearable sensing gloves and sensory feedback devices.

Flexible Artificial Sensory Systems Based on Neuromorphic Devices

Emerging flexible artificial sensory systems using neuromorphic electronics have been considered as a promising solution for processing massive data with low power consumption. The construction of artificial sensory systems with synaptic devices and sensing elements to mimic complicated sensing and processing in biological systems is a prerequisite for the realization. To realize high-efficiency neuromorphic sensory systems, the development of artificial flexible synapses with low power consumption and high-density integration is essential. Furthermore, the realization of efficient coupling between the sensing element and the synaptic device is crucial. This Review presents recent progress in the area of neuromorphic electronics for flexible artificial sensory systems. We focus on both the recent advances of artificial synapses, including device structures, mechanisms, and functions, and the design of intelligent, flexible perception systems based on synaptic devices. Additionally, key challenges and opportunities related to flexible artificial perception systems are examined, and potential solutions and suggestions are provided.