Towards Living Machines: current and future trends of tactile sensing, grasping, and social robotics

The development of future technologies can be highly influenced by our deeper understanding of the principles that underlie living organisms. The Living Machines conference aims at presenting (among others) the interdisciplinary work of behaving systems based on such principles. Celebrating the 10 years of the conference, we present the progress and future challenges of some of the key themes presented in the robotics workshop of the Living Machines conference. More specifically, in this perspective paper, we focus on the advances in the field of biomimetics and robotics for the creation of artificial systems that can robustly interact with their environment, ranging from tactile sensing, grasping, and manipulation to the creation of psychologically plausible agents.

Grasp Pose Detection Based on Shape Simplification

For robots in an unstructured work environment, grasping unknown objects that have neither model data nor RGB data is very important. The key to robotic autonomous grasping is not only in the judgment of object type but also in the shape of the object. We present a new grasping approach based on the basic compositions of objects. The simplification of complex objects is conducive to the description of object shape and provides effective ideas for the selection of grasping strategies. First, the depth camera is used to obtain partial 3D data of the target object. Then the 3D data are segmented and the segmented parts are simplified to a cylinder, a sphere, an ellipsoid, and a parallelepiped according to the geometric and semantic shape characteristics. The grasp pose is constrained according to the simplified shape feature and the core part of the object is used for grasping training using deep learning. The grasping model was evaluated in a simulation experiment and robot experiment, and the experiment result shows that learned grasp score using simplified constraints is more robust to gripper pose uncertainty than without simplified constraint.

Haptic Manipulation of 3D Scans for Geometric Feature Enhancement

Localisation of geometric features like holes, edges, slots, etc. is vital to robotic planning in industrial automation settings. Low-cost 3D scanners are crucial in terms of improving accessibility, but pose a practical challenge to feature localisation because of poorer resolution and consequently affect robotic planning. In this work, we address the possibility of enhancing the quality of a 3D scan by a manual ’touch-up’ of task-relevant features, to ensure their automatic detection prior to automation. We propose a framework whereby the operator (i) has access to both the actual work-piece and its 3D scan; (ii) evaluates the missing salient features from the scan; (iii) uses a haptic stylus to physically interact with the actual work-piece, around such specific features; (iv) interactively updates the scan using the position and force information from the haptic stylus. The contribution of this work is the use of haptic mismatch for geometric update. Specifically, the geometry from the 3D scan is used to predict haptic feedback at a point on the work-piece surface. The haptic mismatch is derived as a measure of error between this prediction and the real interaction forces from physical contact at that point on the work-piece. The geometric update is driven until the haptic mismatch is minimised. Convergence of the proposed algorithm is first numerically verified on an analytical surface with simulated physical interaction. Error analysis of the surface position and orientations were also plotted. Experiments were conducted using a motion capture system providing sub-mm accuracy in position and a 6 axis F/T sensor. Missing features are successfully detected after the update of the scan using the proposed method in an experiment.

Braille and Morse Code Based Tactile Information Coding and its Applications

Tactile sensing has recently been used in pattern recognition technology for pilots’ posture information and environmental information. Human tactile sensing is limited, however, through control of the spatial distribution and vibration intensity of each contact in the tactile stimulation array, the accuracy, convenience, and comfort of the tactile device can be comprehensively improved. Moreover, the recognition rate of most current flight posture information methods is low. In this paper, the principle of vibration haptic coding is optimized. A combined coding scheme of “vibration [Formula: see text] sequence” is used to recognize pilot’s flight posture. A novel triangular coding scheme is proposed for the first time. Compared to other commonly used coding schemes such as “needle scheme“ and “rectangle scheme”, experimental results show that the triangular coding scheme is 1.5% more accuracy with response time reduced by nearly 1.25s in recognition pilots’ flight posture information.

Adaptive force and velocity control based on intrinsic contact sensing during surface exploration of dynamic objects

Haptic exploration is a process of using haptic feedback to interact and perceive an unknown object. It is an essential approach to understand the physical and geometrical properties of the object. While numerous research has been carried out for haptic exploration on static objects, haptic exploration on objects with dynamic movements has not been reported. It is due to the significant challenges to achieve robust force and velocity control when the object is nonstationary. In this work, a novel adaptive force and velocity control algorithm based on intrinsic contact sensing (ICS) for haptic surface exploration of dynamic objects is presented. A fuzzy-logic control framework making use of the information obtained from ICS has been developed. To validate the proposed control algorithm, extensive surface exploration experiments have been carried out on objects with different surface properties, geometries, stiffness, and concave or convex patterns. The validation results demonstrate the high accuracy and robustness of the proposed algorithm using different experimental platforms.