Dynamic Cloth Manipulation Considering Variable Stiffness and Material Change Using Deep Predictive Model With Parametric Bias

Dynamic manipulation of flexible objects such as fabric, which is difficult to modelize, is one of the major challenges in robotics. With the development of deep learning, we are beginning to see results in simulations and some actual robots, but there are still many problems that have not yet been tackled. Humans can move their arms at high speed using their flexible bodies skillfully, and even when the material to be manipulated changes, they can manipulate the material after moving it several times and understanding its characteristics. Therefore, in this research, we focus on the following two points: (1) body control using a variable stiffness mechanism for more dynamic manipulation, and (2) response to changes in the material of the manipulated object using parametric bias. By incorporating these two approaches into a deep predictive model, we show through simulation and actual robot experiments that Musashi-W, a musculoskeletal humanoid with a variable stiffness mechanism, can dynamically manipulate cloth while detecting changes in the physical properties of the manipulated object.

Learning-based cable coupling effect modeling for robotic manipulation of heavy industrial cables

The robotic manipulation of a heavy industrial cable is challenging to model and control because of the high number of degrees of freedom and the rigid-flexible coupling dynamics. In this paper, we report the development of modeling the cable effect and control methodology for robotic cable manipulation. Our cable effect model is based on the 2D convolutional neural network, which is a deep learning-based method uses the effective cable representation method to achieve the accurate, generalizable, and efficient estimation of the cable coupling forces and torques. Practical problems such as the measurement limits and time efficiency are considered in our method for real applications. With these approaches, we are the first to solve the problem of dynamic payload effect caused by heavy industrial cables in experimental cases. The used control methodology combines the active disturbance rejection control framework with the sliding mode control method, which can acquire promising tracking performance. We integrate our cable effect model into the control scheme, and demonstrate it satisfies the high-quality robotic manipulation of heavy cables. The performance of the proposed method is assessed with both a simulated system and real robot system. The results show that our method can estimate the cable coupling effect with over 85% accuracy and accomplish manipulation with a positioning error less than 0.01 mm. This reveals that our method is promising for robotic manipulation of heavy industrial cables and can accomplish the challenging cable insertion task.

Tracking linear deformable objects using slicing method

In this paper, an efficient novel method for tracking the linear deformable objects (LDOs) in real time is proposed. The method is developed based on recursively slicing a pointcloud into smaller pointclouds with sufficiently small variance. The performance of this method is investigated through a series of experiments with various camera resolutions in simulation when a robot end effector tracking an LDO using an RGBD camera, and in real word when the camera tracks a rope during a swing. The performance of the proposed method is compared with another state-of-the-art technique and the outcome is reported here.

Modeling of Deformable Objects for Robotic Manipulation: A Tutorial and Review

Manipulation of deformable objects has given rise to an important set of open problems in the field of robotics. Application areas include robotic surgery, household robotics, manufacturing, logistics, and agriculture, to name a few. Related research problems span modeling and estimation of an object's shape, estimation of an object's material properties, such as elasticity and plasticity, object tracking and state estimation during manipulation, and manipulation planning and control. In this survey article, we start by providing a tutorial on foundational aspects of models of shape and shape dynamics. We then use this as the basis for a review of existing work on learning and estimation of these models and on motion planning and control to achieve desired deformations. We also discuss potential future lines of work.

Multi-Modal Sensing and Robotic Manipulation of Non-Rigid Objects: A Survey

This paper aims to provide a comprehensive survey of recent advancements in modelling and autonomous manipulation of non-rigid objects. It first summarizes the recent advances in sensing and modelling of such objects with a focus on describing the methods and technologies used to measure their shape and estimate their material and physical properties. Formal representations considered to predict the deformation resulting from manipulation of non-rigid objects are then investigated. The third part provides a survey of planning and control strategies exploited to operate dexterous robotic systems while performing various tasks on objects made of different non-rigid materials.

A survey on dielectric elastomer actuators for soft robots

Conventional industrial robots with the rigid actuation technology have made great progress for humans in the fields of automation assembly and manufacturing. With an increasing number of robots needing to interact with humans and unstructured environments, there is a need for soft robots capable of sustaining large deformation while inducing little pressure or damage when maneuvering through confined spaces. The emergence of soft robotics offers the prospect of applying soft actuators as artificial muscles in robots, replacing traditional rigid actuators. Dielectric elastomer actuators (DEAs) are recognized as one of the most promising soft actuation technologies due to the facts that: i) dielectric elastomers are kind of soft, motion-generating materials that resemble natural muscle of humans in terms of force, strain (displacement per unit length or area) and actuation pressure/density; ii) dielectric elastomers can produce large voltage-induced deformation. In this survey, we first introduce the so-called DEAs emphasizing the key points of working principle, key components and electromechanical modeling approaches. Then, different DEA-driven soft robots, including wearable/humanoid robots, walking/serpentine robots, flying robots and swimming robots, are reviewed. Lastly, we summarize the challenges and opportunities for the further studies in terms of mechanism design, dynamics modeling and autonomous control.

Knot-tying with four-piece fixtures

We present a class of fixtures that can be disassembled into four pieces to extract the loosely tied knot. We prove that a fixture can be designed for any particular knot such that the knot can be extracted using only simple pure translations of the four fixture sections. We explore some of the issues raised by our experimental work with these fixtures, which show that simple knots can be tied extremely quickly (less than half a second) and reliably (99% repeatability) using four-piece fixtures.

On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments

In this paper, we address the active deformation control of compliant objects by robot manipulators. The control of deformations is needed to automate several important tasks, for example, the manipulation of soft tissues, shaping of food materials, or needle insertion. Note that in many of these applications, the object’s deformation properties are not known. To cope with this issue, in this paper we present two new visual servoing approaches to explicitly servo-control elastic deformations. The novelty of our kinematic controllers lies in its uncalibrated behavior; our adaptive methods do not require the prior identification of the object’s deformation model and the camera’s intrinsic/extrinsic parameters. This feature provides a way to automatically control deformations in a model-free manner. The experimental results that we report validate the feasibility of our controllers.

Estimation of Soft Tissue Mechanical Parameters from Robotic Manipulation Data

Robotic motion planning algorithms used for task automation in robotic surgical systems rely on availability of accurate models of target soft tissue's deformation. Relying on generic tissue parameters in constructing the tissue deformation models is problematic because, biological tissues are known to have very large (inter- and intra-subject) variability. A priori mechanical characterization (e.g., uniaxial bench test) of the target tissues before a surgical procedure is also not usually practical. In this paper, a method for estimating mechanical parameters of soft tissue from sensory data collected during robotic surgical manipulation is presented. The method uses force data collected from a multiaxial force sensor mounted on the robotic manipulator, and tissue deformation data collected from a stereo camera system. The tissue parameters are then estimated using an inverse finite element method. The effects of measurement and modeling uncertainties on the proposed method are analyzed in simulation. The results of experimental evaluation of the method are also presented.

Quasi-static manipulation of a Kirchhoff elastic rod based on a geometric analysis of equilibrium configurations

Consider a thin, flexible wire of fixed length that is held at each end by a robotic gripper. Any curve traced by this wire when in static equilibrium is a local solution to a geometric optimal control problem, with boundary conditions that vary with the position and orientation of each gripper. We prove that the set of all local solutions to this problem over all possible boundary conditions is a smooth manifold of finite dimension that can be parameterized by a single chart. We show that this chart makes it easy to implement a sampling-based algorithm for quasi-static manipulation planning. We characterize the performance of such an algorithm with experiments in simulation.

Topology-based representations for motion planning and generalization in dynamic environments with interactions

Motion can be described in several alternative representations, including joint configuration or end-effector spaces, but also more complex topology-based representations that imply a change of Voronoi bias, metric or topology of the motion space. Certain types of robot interaction problems, e.g. wrapping around an object, can suitably be described by so-called writhe and interaction mesh representations. However, considering motion synthesis solely in a topology-based space is insufficient since it does not account for additional tasks and constraints in other representations. In this paper, we propose methods to combine and exploit different representations for synthesis and generalization of motion in dynamic environments. Our motion synthesis approach is formulated in the framework of optimal control as an approximate inference problem. This allows for consistent combination of multiple representations (e.g. across task, end-effector and joint space). Motion generalization to novel situations and kinematics is similarly performed by projecting motion from topology-based to joint configuration space. We demonstrate the benefit of our methods on problems where direct path finding in joint configuration space is extremely hard whereas local optimal control exploiting a representation with different topology can efficiently find optimal trajectories. In real-world demonstrations, we highlight the benefits of using topology-based representations for online motion generalization in dynamic environments.

Estimation of Soft Tissue Mechanical Parameters from Robotic Manipulation Data

Robotic motion planning algorithms used for task automation in robotic surgical systems rely on availability of accurate models of target soft tissue's deformation. Relying on generic tissue parameters in constructing the tissue deformation models is problematic; because, biological tissues are known to have very large (inter- and intra-subject) variability. A priori mechanical characterization (e.g., uniaxial bench test) of the target tissues before a surgical procedure is also not usually practical. In this paper, a method for estimating mechanical parameters of soft tissue from sensory data collected during robotic surgical manipulation is presented. The method uses force data collected from a multiaxial force sensor mounted on the robotic manipulator, and tissue deformation data collected from a stereo camera system. The tissue parameters are then estimated using an inverse finite element method. The effects of measurement and modeling uncertainties on the proposed method are analyzed in simulation. The results of experimental evaluation of the method are also presented.

Development of a Wire Harness Assembly Motion Planner for Redundant Multiple Manipulators

This article discusses a motion planner for manipulating deformable linear objects with multiple manipulators. When multiple manipulators grip a rigid body, hand positions and postures of those manipulators are dependent variables of the positions and postures of the gripped rigid body. On the other hand, when multiple manipulators grip a deformable linear object, constraint conditions are eased compared to those for a rigid body. However, there is another problem: the need for consideration of deformation of a deformable linear object by the motion plan of a robot. In this research, in order to grip and operate such deformable linear objects with multiple manipulators, we developed a sampling-based robot motion planner. By combining basic motions generated by the developed robot motion planner, we will show that a complicated task, such as the assembly of a deformable linear object with the multiple manipulators, is possible. Using the example of a wire harness assembly work on an automobile production line, we perform motion planning using the developed motion planner, and we verify its effectiveness through simulations.

3-D Sensing for Flexible Linear Object Alignment in Robot Cell Production System

We have constructed 3-D sensing system for alignment of connector-fitted cables as flexible linear objects which used to be difficult to be automated at the production sites. In the system an industrial robot has a 3-D sensor and a monocular camera mounted at the hand. 3-D sensor, using space encoding method, allows the robot to make high-precision measurements of the order of sub-millimeters, but emphasis is placed on precision at the expense of fields of view. In addition, active sensing methods such as the space encoding method is hard to take measurements for black cables, as well as it has some difficulties with measurements of semitransparent plastic connectors depending on view-points. To cope with those problems, our system is such that the monocular camera on the robot is moved for motion stereo to take measurements on cable shapes; connector’s poses are coarsely estimated from the measurement results; and such view-points as will ensure stable measurements are computed by space encoding method to take precision measurements of connectors. Technical features of the system could be summarized as follows: 1) Determination of view-points to measure connectors, based on measurements of cable shapes, requires no more than two measurements, without repeated searches, to grab semitransparent plastic connectors. 2) Performance of stereo correspondence for plain or black cables, which tends to result in a failure with the aids of no more than irradiated slit patterns and epipolar constraints, has been improved through sequential correspondence inmotion image sequence and its stability evaluations. At the operation tests in the validation system, the robot is assigned a task to assemble the cables into industrial servo amplifiers available on the market, in which automatic alignment of 200 connector-fitted cables has successfully been accomplished in succession to confirm constant performance of the system.