Active Haptic Perception in Robots: A Review

Advances in Piezoelectret Materials-Based Bidirectional Haptic Communication Devices

Bidirectional haptic communication devices accelerate the revolution of virtual/augmented reality and flexible/wearable electronics. As an emerging kind of flexible piezoelectric materials, piezoelectret materials can effortlessly convert mechanical force into electrical signals and respond to electrical fields in a deformation manner, exhibiting enormous potential in the construction of bidirectional haptic communication devices. Existing reviews on piezoelectret materials primarily focus on flexible energy harvesters and sensors, and the recent development of piezoelectret-based bidirectional haptic communication devices has not been comprehensively reviewed. Herein, a comprehensive overview of the materials construction, along with the recent advances in bidirectional haptic communication devices, is provided. First, the development timeline, key characteristics, and various fabrication methods of piezoelectret materials are introduced. Subsequently, following the underlying mechanisms of bidirectional electromechanical signal conversion of piezoelectret, strategies to improve the d33 coefficients of materials are proposed. The principles of haptic perception and feedback are also highlighted, and representative works and progress in this area are summarized. Finally, the challenges and opportunities associated with improving the overall practicability of piezoelectret materials-based bidirectional haptic communication devices are discussed.

Skin-Inspired Multi-Modal Mechanoreceptors for Dynamic Haptic Exploration

Active sensing is a fundamental aspect of human and animal interactions with the environment, providing essential information about the hardness, texture, and tackiness of objects. This ability stems from the presence of diverse mechanoreceptors in the skin, capable of detecting a wide range of stimuli and from the sensorimotor control of biological mechanisms. In contrast, existing tactile sensors for robotic applications typically excel in identifying only limited types of information, lacking the versatility of biological mechanoreceptors and the requisite sensing strategies to extract tactile information proactively. Here, inspired by human haptic perception, a skin-inspired artificial 3D mechanoreceptor (SENS) capable of detecting multiple mechanical stimuli is developed to bridge sensing and action in a closed-loop sensorimotor system for dynamic haptic exploration. A tensor-based non-linear theoretical model is established to characterize the 3D deformation (e.g., tensile, compressive, and shear deformation) of SENS, providing guidance for the design and optimization of multimode sensing properties with high fidelity. Based on SENS, a closed-loop robotic system capable of recognizing objects with improved accuracy (≈96%) is further demonstrated. This dynamic haptic exploration approach shows promise for a wide range of applications such as autonomous learning, healthcare, and space and deep-sea exploration.

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

Visuo-haptic object perception for robots: an overview

The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

A Comprehensive Study of Mobile Robot: History, Developments, Applications, and Future Research Perspectives

Intelligent mobile robots that can move independently were laid out in the real world around 100 years ago during the second world war after advancements in computer science. Since then, mobile robot research has transformed robotics and information engineering. For example, robots were crucial in military applications, especially in teleoperations, when they emerged during the second world war era. Furthermore, after the implementation of artificial intelligence (AI) in robotics, they became autonomous or more intelligent. Currently, mobile robots have been implemented in many applications like defense, security, freight, pattern recognition, medical treatment, mail delivery, infrastructure inspection and developments, passenger travel, and many more because they are more intelligent nowadays with artificial intelligence technology. To study the developments of mobile robots, we have studied an extensive literature survey of the last 50 years. In this article, we discuss a complete century of mobile robotics research, major sensors used in robotics, some major applications of mobile robots, and their impact on our lives and in applied engineering.

Hand-Object Interaction: From Human Demonstrations to Robot Manipulation

Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human demonstrations, highlighting the main challenges in this research area and suggesting desirable future developments. To this extent, the article presents a general definition of the hand-object interaction problem together with a concise review for each of the main subproblems involved, namely: sensing, perception, and learning. Furthermore, the article discusses the interplay between these subproblems and describes how their interaction in learning from demonstration contributes to the success of robot manipulation. In this way, the article provides a broad overview of the interdisciplinary approaches necessary for a robotic system to learn new manipulation skills by observing human behavior in the real world.

The interaction between motion and texture in the sense of touch

Besides providing information on elementary properties of objects, like texture, roughness, and softness, the sense of touch is also important in building a representation of object movement and the movement of our hands. Neural and behavioral studies shed light on the mechanisms and limits of our sense of touch in the perception of texture and motion, and of its role in the control of movement of our hands. The interplay between the geometrical and mechanical properties of the touched objects, such as shape and texture, the movement of the hand exploring the object, and the motion felt by touch, will be discussed in this article. Interestingly, the interaction between motion and textures can generate perceptual illusions in touch. For example, the orientation and the spacing of the texture elements on a static surface induces the illusion of surface motion when we move our hand on it or can elicit the perception of a curved trajectory during sliding, straight hand movements. In this work we present a multiperspective view that encompasses both the perceptual and the motor aspects, as well as the response of peripheral and central nerve structures, to analyze and better understand the complex mechanisms underpinning the tactile representation of texture and motion. Such a better understanding of the spatiotemporal features of the tactile stimulus can reveal novel transdisciplinary applications in neuroscience and haptics.

Biology and bioinspiration of soft robotics: Actuation, sensing, and system integration

Organisms in nature grow with senses, nervous, and actuation systems coordinated in ingenious ways to sustain metabolism and other essential life activities. The understanding of biological structures and functions guide the construction of soft robotics with unprecedented performances. However, despite the progress in soft robotics, there still remains a big gap between man-made soft robotics and natural lives in terms of autonomy, adaptability, self-repair, durability, energy efficiency, etc. Here, the actuation and sensing strategies in the natural biological world are summarized along with their man-made counterparts applied in soft robotics. The development trends of bioinspired soft robotics toward closed loop and embodiment are proposed. Challenges for obtaining autonomous soft robotics similar to natural organisms are outlined to provide a perspective in this field.

Tactile Object Recognition for Humanoid Robots Using New Designed Piezoresistive Tactile Sensor and DCNN

A tactile sensor array is a crucial component for applying physical sensors to a humanoid robot. This work focused on developing a palm-size tactile sensor array (56.0 mm × 56.0 mm) to apply object recognition for the humanoid robot hand. This sensor was based on a PCB technology operating with the piezoresistive principle. A conductive polymer composites sheet was used as a sensing element and the matrix array of this sensor was 16 × 16 pixels. The sensitivity of this sensor was evaluated and the sensor was installed on the robot hand. The tactile images, with resolution enhancement using bicubic interpolation obtained from 20 classes, were used to train and test 19 different DCNNs. InceptionResNetV2 provided superior performance with 91.82% accuracy. However, using the multimodal learning method that included InceptionResNetV2 and XceptionNet, the highest recognition rate of 92.73% was achieved. Moreover, this recognition rate improved when the object exploration was applied to demonstrate.

Grasp Stability Prediction for a Dexterous Robotic Hand Combining Depth Vision and Haptic Bayesian Exploration

Grasp stability prediction of unknown objects is crucial to enable autonomous robotic manipulation in an unstructured environment. Even if prior information about the object is available, real-time local exploration might be necessary to mitigate object modelling inaccuracies. This paper presents an approach to predict safe grasps of unknown objects using depth vision and a dexterous robot hand equipped with tactile feedback. Our approach does not assume any prior knowledge about the objects. First, an object pose estimation is obtained from RGB-D sensing; then, the object is explored haptically to maximise a given grasp metric. We compare two probabilistic methods (i.e. standard and unscented Bayesian Optimisation) against random exploration (i.e. uniform grid search). Our experimental results demonstrate that these probabilistic methods can provide confident predictions after a limited number of exploratory observations, and that unscented Bayesian Optimisation can find safer grasps, taking into account the uncertainty in robot sensing and grasp execution.

Overview and perspectives about the robotic surgical certification process in Brazil: the new statement and a national web-survey

OBJECTIVE

to appraise the general profile of the Brazilian robotic surgeon and the acknowledgment of the new certification process for robotic surgery upon the Associação Médica Brasileira (AMB - Brazilian Medical Association) statement. According to the AMB statement, medical societies and proctors have to achieve leading roles in training and certification of surgeons, acting in partnership with industry.

METHODS

a national web-based survey was promoted by the Colégio Brasileiro de Cirurgiões (CBC - Brazilian College of Surgeons) among their members.

RESULTS

the 294 answers were split into two groups: 133 (45.3%) who had robotic console certification, and 161 (54.8%) who did not have it. The overall median age was 46, but the non-robotic group presented more surgeons with at least 30 years of experience than to the robotic group (32.3% versus 23.3%, p=0.033). Surgeons with robotic certification more frequently work in a city with at least one million inhabitants than surgeons who were not certified (85.7 versus 63.4%, p<0.001). The majority of surgeons in both groups have similar positioning for all main points of the statement. However, the agreement proportions for the preceptors responsibility during the procedures were higher among non-robotic surgeons that expected the preceptor to assume co-responsibility for the procedure (85% versus 60.9%, p<0.001), and intervene during the procedure as much as necessary (97.5% versus 91.7%, p=0.033).

CONCLUSION

the overall agreement of the answers to the AMB statement seems to be a promising pathway to increase the participation of the medical entities into the robotic certification in Brazil.

Internet of Robotic Things Intelligent Connectivity and Platforms

The Internet of Things (IoT) and Industrial IoT (IIoT) have developed rapidly in the past few years, as both the Internet and "things" have evolved significantly. "Things" now range from simple Radio Frequency Identification (RFID) devices to smart wireless sensors, intelligent wireless sensors and actuators, robotic things, and autonomous vehicles operating in consumer, business, and industrial environments. The emergence of "intelligent things" (static or mobile) in collaborative autonomous fleets requires new architectures, connectivity paradigms, trustworthiness frameworks, and platforms for the integration of applications across different business and industrial domains. These new applications accelerate the development of autonomous system design paradigms and the proliferation of the Internet of Robotic Things (IoRT). In IoRT, collaborative robotic things can communicate with other things, learn autonomously, interact safely with the environment, humans and other things, and gain qualities like self-maintenance, self-awareness, self-healing, and fail-operational behavior. IoRT applications can make use of the individual, collaborative, and collective intelligence of robotic things, as well as information from the infrastructure and operating context to plan, implement and accomplish tasks under different environmental conditions and uncertainties. The continuous, real-time interaction with the environment makes perception, location, communication, cognition, computation, connectivity, propulsion, and integration of federated IoRT and digital platforms important components of new-generation IoRT applications. This paper reviews the taxonomy of the IoRT, emphasizing the IoRT intelligent connectivity, architectures, interoperability, and trustworthiness framework, and surveys the technologies that enable the application of the IoRT across different domains to perform missions more efficiently, productively, and completely. The aim is to provide a novel perspective on the IoRT that involves communication among robotic things and humans and highlights the convergence of several technologies and interactions between different taxonomies used in the literature.

Embodied tactile perception and learning

Various living creatures exhibit embodiment intelligence, which is reflected by a collaborative interaction of the brain, body, and environment. The actual behavior of embodiment intelligence is generated by a continuous and dynamic interaction between a subject and the environment through information perception and physical manipulation. The physical interaction between a robot and the environment is the basis for realizing embodied perception and learning. Tactile information plays a critical role in this physical interaction process. It can be used to ensure safety, stability, and compliance, and can provide unique information that is difficult to capture using other perception modalities. However, due to the limitations of existing sensors and perception and learning methods, the development of robotic tactile research lags significantly behind other sensing modalities, such as vision and hearing, thereby seriously restricting the development of robotic embodiment intelligence. This paper presents the current challenges related to robotic tactile embodiment intelligence and reviews the theory and methods of robotic embodied tactile intelligence. Tactile perception and learning methods for embodiment intelligence can be designed based on the development of new large‐scale tactile array sensing devices, with the aim to make breakthroughs in the neuromorphic computing technology of tactile intelligence.