Intrinsic sense of touch for intuitive physical human-robot interaction

The sense of touch is a property that allows humans to interact delicately with their physical environment. This article reports on a technological advancement in intuitive human-robot interaction that enables an intrinsic robotic sense of touch without the use of artificial skin or tactile instrumentation. On the basis of high-resolution joint-force-torque sensing in a redundant arrangement, we were able to let the robot sensitively feel the surrounding environment and accurately localize touch trajectories in space and time that were applied on its surface by a human. Through an intertwined combination of manifold learning techniques and artificial neural networks, the robot identified and interpreted those touch trajectories as machine-readable letters, symbols, or numbers. This opens up unexplored opportunities in terms of intuitive and flexible interaction between human and robot. Furthermore, we showed that our concept of so-called virtual buttons can be used to straightforwardly implement a tactile communication link, including switches and slider bars, which are complementary to speech, hardware buttons, and control panels. These interaction elements could be freely placed, moved, and configured in arbitrary locations on the robot structure. The intrinsic sense of touch we proposed in this work can serve as the basis for an advanced category of physical human-robot interaction that has not been possible yet, enabling a shift from conventional modalities toward adaptability, flexibility, and intuitive handling.

U-TAG: Electromagnetic Wireless Sensing System for Robotic Hand Pre-Grasping

In order to perform complex manipulation and grasp tasks, robotic hands require sensors that can handle increasingly demanding functionality and degrees of freedom. This research paper proposes a radiofrequency sensor that uses a wireless connection between a probe and a tag. A compact and low-profile antenna is mounted on the hand and functions as a probe to read a printed passive resonator on the plastic object being targeted, operating within a pre-touch sensing range. The grasping strategy consists of four stages that involve planar alignment in up-to-down and left-to-right directions between the probe and tag, the search for an appropriate distance from the object, and rotational (angular) alignment. The real and imaginary components of the probe-input impedance are analyzed for different orientation strategies and positioning between the resonator on the object and the probe. These data are used to deduce the orientation of the hand relative to the target object and to determine the optimal position for grasping.

Detecting Transitions from Stability to Instability in Robotic Grasping Based on Tactile Perception

Robots execute diverse load operations, including carrying, lifting, tilting, and moving objects, involving load changes or transfers. This dynamic process can result in the shift of interactive operations from stability to instability. In this paper, we respond to these dynamic changes by utilizing tactile images captured from tactile sensors during interactions, conducting a study on the dynamic stability and instability in operations, and propose a real-time dynamic state sensing network by integrating convolutional neural networks (CNNs) for spatial feature extraction and long short-term memory (LSTM) networks to capture temporal information. We collect a dataset capturing the entire transition from stable to unstable states during interaction. Employing a sliding window, we sample consecutive frames from the collected dataset and feed them into the network for the state change predictions of robots. The network achieves both real-time temporal sequence prediction at 31.84 ms per inference step and an average classification accuracy of 98.90%. Our experiments demonstrate the network's robustness, maintaining high accuracy even with previously unseen objects.

Proximity Sensing Electronic Skin: Principles, Characteristics, and Applications

The research on proximity sensing electronic skin has garnered significant attention. This electronic skin technology enables detection without physical contact and holds vast application prospects in areas such as human-robot collaboration, human-machine interfaces, and remote monitoring. Especially in the context of the spread of infectious diseases like COVID-19, there is a pressing need for non-contact detection to ensure safe and hygienic operations. This article comprehensively reviews the significant advancements in the field of proximity sensing electronic skin technology in recent years. It covers the principles, as well as single-type proximity sensors with characteristics such as a large area, multifunctionality, strain, and self-healing capabilities. Additionally, it delves into the research progress of dual-type proximity sensors. Furthermore, the article places a special emphasis on the widespread applications of flexible proximity sensors in human-robot collaboration, human-machine interfaces, and remote monitoring, highlighting their importance and potential value across various domains. Finally, the paper provides insights into future advancements in flexible proximity sensor technology.

High-Performance Dual-Responsive Sensing Skin Enabled by Bioinspired Transduction of Coplanar Square-Loop Electrodes

Advancements in intelligent robots and human-machine interaction necessitate a shift in artificial skins toward multimodal perception. Dual-responsive skins that can detect proximity and pressure information are significant to establishing continuous sensing of interaction processes and extending interactive application scenarios. To address the current limitations of inadequate dual-mode performance, such as limited proximal response change and low tactile sensitivity, this paper presents a bioinspired complementary gradient architecture-enabled (CGA) transduction design and a high-performance dual-responsive skin based on coplanar square-loop electrodes. Through systematic investigation into the transduction of various electrode configurations, comparative results reveal the remarkable potential of coplanar electrodes to deliver superior dual-mode performance without compromise. Simulations and experiments prove that the proposed CGA response mechanism can capture local interface deformation and overall compression signals, further enhancing response sensitivity. The final developed artificial skin is sensitive to external pressure and the approach of objects simultaneously, exhibiting a long detection distance (∼40 mm), a high proximity response (>0.4), and outstanding touch sensitivity (0.131 kPa-1). Furthermore, we demonstrate proof-of-concept applications for the proposed sensing skin in a dual-mode teleoperation interface and adaptive grasping interactions.

Orange Dye and Silicone Glue Composite Gel-Based Optimized Impedimetric and Capacitive Surface-Type Proximity Sensors

Optimized surface-type impedimetric and capacitive proximity sensors have been fabricated on paper substrates by using rubbing-in technology. The orange dye (OD) and silicone glue (SG) composite-gel films were deposited on the zig-zag gap between two aluminum electrodes fixed on a paper (dielectric) substrate. The effect of proximity of various objects (receivers) on the impedance and the capacitance of the sensors was investigated. These objects were semi-cylindrical aluminum (metallic) foil, a cylindrical plastic tube filled with water, a kopeck-shaped plastic tube filled with carbon nanotubes and a human finger. The mechanism of sensing was based on the change in impedance and/or the capacitance of the sensors with variation of proximity between the surfaces of the sensor and the object. On decreasing proximity, the impedance of the sensors increased while the capacitance decreased. The impedimetric proximity sensitivities of CNT, water, metal-based receivers and the finger were up to 60 × 103 Ω/mm, 35 × 103 Ω/mm, 44 × 103 Ω/mm and 6.2 × 103 Ω/mm, respectively, while their capacitive sensitivities were -19.0 × 10-2 pF/mm, -16.0 × 10-2 pF/mm, -16.4 × 10-2 pF/mm and -1.8 × 10-2 pF/mm. If needed for practical application, the sensors can be built in to the Wheatstone bridge, which can also increase the sensitivity of the measurement. Moreover, the sensor's materials are low cost, while the fabrication technique is easy and ecologically friendly. The sensor can also be used for demonstrative purposes in school and college laboratories.

An artificial remote tactile device with 3D depth-of-field sensation

Flexible tactile neuromorphic devices are becoming important as the impetus for the development of human-machine collaboration. However, accomplishing and further transcending human intelligence with artificial intelligence still confront many barriers. Here, we present a self-powered stretchable three-dimensional remote tactile device (3D-RTD) that performs the depth-of-field (DOF) sensation of external mechanical motions through a conductive-dielectric heterogeneous structure. The device can build a logic relationship precisely between DOF motions of an external active object and sensory potential signals of bipolar sign, frequency, amplitude, etc. The sensory mechanism is revealed on the basis of the electrostatic theory and multiphysics modeling, and the performance is verified via an artificial-biological hybrid system with micro/macroscale interaction. The feasibility of the 3D-RTD as an obstacle-avoidance patch for the blind is systematically demonstrated with a rat. This work paves the way for multimodal neuromorphic device that transcends the function of a biological one toward a new modality for brain-like intelligence.

Lessons for Robotics From the Control Architecture of the Octopus

Biological and artificial agents are faced with many of the same computational and mechanical problems, thus strategies evolved in the biological realm can serve as inspiration for robotic development. The octopus in particular represents an attractive model for biologically-inspired robotic design, as has been recognized for the emerging field of soft robotics. Conventional global planning-based approaches to controlling the large number of degrees of freedom in an octopus arm would be computationally intractable. Instead, the octopus appears to exploit a distributed control architecture that enables effective and computationally efficient arm control. Here we will describe the neuroanatomical organization of the octopus peripheral nervous system and discuss how this distributed neural network is specialized for effectively mediating decisions made by the central brain and the continuous actuation of limbs possessing an extremely large number of degrees of freedom. We propose top-down and bottom-up control strategies that we hypothesize the octopus employs in the control of its soft body. We suggest that these strategies can serve as useful elements in the design and development of soft-bodied robotics.

The State-of-the-Art Sensing Techniques in Human Activity Recognition: A Survey

Human activity recognition (HAR) has become an intensive research topic in the past decade because of the pervasive user scenarios and the overwhelming development of advanced algorithms and novel sensing approaches. Previous HAR-related sensing surveys were primarily focused on either a specific branch such as wearable sensing and video-based sensing or a full-stack presentation of both sensing and data processing techniques, resulting in weak focus on HAR-related sensing techniques. This work tries to present a thorough, in-depth survey on the state-of-the-art sensing modalities in HAR tasks to supply a solid understanding of the variant sensing principles for younger researchers of the community. First, we categorized the HAR-related sensing modalities into five classes: mechanical kinematic sensing, field-based sensing, wave-based sensing, physiological sensing, and hybrid/others. Specific sensing modalities are then presented in each category, and a thorough description of the sensing tricks and the latest related works were given. We also discussed the strengths and weaknesses of each modality across the categorization so that newcomers could have a better overview of the characteristics of each sensing modality for HAR tasks and choose the proper approaches for their specific application. Finally, we summarized the presented sensing techniques with a comparison concerning selected performance metrics and proposed a few outlooks on the future sensing techniques used for HAR tasks.

TheraProx: Capacitive Proximity Sensing

This paper presents the development of a novel contactless omnidirectional capacitive proximity sensor. The presented device has been designed to be energy-efficient (≈5 mW power consumption) by means of duty-cycling the power supply. A comprehensive methodological experiment has been carried out to extensively evaluate the performance within the sensing range (5–10 cm). A simple boot-up self-adjustment mechanism has been implemented using a digital potentiometer. This feature allows for an effortless utilization of the proposed device in a wide variety of potential applications, including mobile robotics and human–machine interaction.