Autonomous Visual Navigation for a Flower Pollination Drone

In this paper, we present the development of a visual navigation capability for a small drone enabling it to autonomously approach flowers. This is a very important step towards the development of a fully autonomous flower pollinating nanodrone. The drone we developed is totally autonomous and relies for its navigation on a small on-board color camera, complemented with one simple ToF distance sensor, to detect and approach the flower. The proposed solution uses a DJI Tello drone carrying a Maix Bit processing board capable of running all deep-learning-based image processing and navigation algorithms on-board. We developed a two-stage visual servoing algorithm that first uses a highly optimized object detection CNN to localize the flowers and fly towards it. The second phase, approaching the flower, is implemented by a direct visual steering CNN. This enables the drone to detect any flower in the neighborhood, steer the drone towards the flower and make the drone’s pollinating rod touch the flower. We trained all deep learning models based on an artificial dataset with a mix of images of real flowers, artificial (synthetic) flowers and virtually rendered flowers. Our experiments demonstrate that the approach is technically feasible. The drone is able to detect, approach and touch the flowers totally autonomously. Our 10 cm sized prototype is trained on sunflowers, but the methodology presented in this paper can be retrained for any flower type.

The Math of Tasks: A Domain Specific Language for Constraint-Based Task Specification

This paper proposes a Domain Specific Language to describe in a synthetic and comprehensive way, complex control problems for robotic systems. The proposed language, named Math of Tasks, abstracts from the mathematical description of the problem, which heavily depends on the particular algorithm chosen to solve it, relying on a set of operators and entities which instead, composed together, describes the behavior of the control problem. The Math of Tasks can describe any type of instantaneous controller regardless of the particular controlled variable (e.g., joint velocity, joint acceleration, contact forces). This paper shows the convenience of the proposed formalism using examples from classical control problems for different types of robotic platforms such as manipulators, humanoid bipeds, and quadrupeds.

Online Reconfiguration of Distributed Robot Control Systems for Modular Robot Behavior Implementation

The use of autonomous robots in areas that require executing a broad range of different tasks is currently hampered by the high complexity of the software that adapts the robot controller to different situations the robot would face. Current robot software frameworks facilitate implementing controllers for individual tasks with some variability, however, their possibilities for adapting the controllers at runtime are very limited and don’t scale with the requirements of a highly versatile autonomous robot. With the software presented in this paper, the behavior of robots is implemented modularly by composing individual controllers, between which it is possible to switch freely at runtime, since the required transitions are calculated automatically. Thereby the software developer is relieved of the task to manually implement and maintain the transitions between different operational modes of the robot, what largely reduces software complexity for larger amounts of different robot behaviors. The software is realized by a model-based development approach. We will present the metamodels enabling the modeling of the controllers as well as the runtime architecture for the management of the controllers on distributed computation hardware. Furthermore, this paper introduces an algorithm that calculates the transitions between two controllers. A series of technical experiments verifies the choice of the underlying middleware and the performance of online controller reconfiguration. A further experiment demonstrates the applicability of the approach to real robotics applications.

ControlIt! — A Software Framework for Whole-Body Operational Space Control

Whole Body Operational Space Control (WBOSC) enables floating-base highly redundant robots to achieve unified motion/force control of one or more operational space objectives while adhering to physical constraints. It is a pioneering algorithm in the field of human-centered Whole-Body Control (WBC). Although there are extensive studies on the algorithms and theory behind WBOSC, limited studies exist on the software architecture and APIs that enable WBOSC to perform and be integrated into a larger system. In this paper, we address this by presenting ControlIt!, a new open-source software framework for WBOSC. Unlike previous implementations, ControlIt! is multi-threaded to increase maximum servo frequencies using standard PC hardware. A new parameter binding mechanism enables tight integration between ControlIt! and external processes via an extensible set of transport protocols. To support a new robot, only two plugins and a URDF model is needed — the rest of ControlIt! remains unchanged. New WBC primitives can be added by writing Task or Constraint plugins. ControlIt!’s capabilities are demonstrated on Dreamer, a 16-DOF torque controlled humanoid upper body robot containing both series elastic and co-actuated joints, and using it to perform a product disassembly task. Using this testbed, we show that ControlIt! can achieve average servo latencies of about 0.5[Formula: see text]ms when configured with two Cartesian position tasks, two orientation tasks, and a lower priority posture task. This is 10 times faster than the 5[Formula: see text]ms that was achieved using UTA-WBC, the prototype implementation of WBOSC that is both application and platform-specific. Variations in the product’s position is handled by updating the goal of the Cartesian position task. ControlIt!’s source code is released under LGPL and we hope it will be adopted and maintained by the WBC community for the long term as a platform for WBC development and integration.

Whole Body Control of a Dual-Arm Mobile Robot Using a Virtual Kinematic Chain

Dual-arm manipulators have more advanced manipulation abilities compared to single-arm manipulators and manipulators mounted on a mobile base have additional mobility and a larger workspace. Combining these advantages, mobile dual-arm robots are expected to perform a variety of tasks in the future. Kinematically, the configuration of two arms that branches from the mobile base results in a serial-to-parallel kinematic structure. In order to respond to external disturbances, this serial-to-parallel kinematic structure makes inverse kinematic computations non-trivial, as the motion of the base has to take the needs of both arms into account. Instead of using the dual-arm kinematics directly, we propose to use a virtual kinematic chain (VKC) to specify the common motion of the two arms. We formulate a constraint-based programming solution which consists of two parts. In the first part, we use an extended serial kinematic chain including the mobile base and the VKC to formulate constraints that realize the desired orientation and translation expressed in the world frame. In the second part, we use the resolved VKC motion to constrain the common motion of the two arms. In order to explore the redundancy of the two arms in an optimization framework, we also provide a VKC-oriented manipulability measure as well as its closed-form gradient. We verify the proposed approach with simulations and experiments that are performed on a PR2 robot, which has two 7 degrees of freedom (DoF) arms and a 3 DoF mobile base.

G2-type SRMPC scheme for synchronous manipulation of two redundant robot arms

In this paper, to remedy the joint-angle drift phenomenon for manipulation of two redundant robot arms, a novel scheme for simultaneous repetitive motion planning and control (SRMPC) at the joint-acceleration level is proposed, which consists of two subschemes. To do so, the performance index of each SRMPC subscheme is derived and designed by employing the gradient dynamics twice, of which a convergence theorem and its proof are presented. In addition, for improving the accuracy of the motion planning and control, position error, and velocity, error feedbacks are incorporated into the forward kinematics equation and analyzed via Zhang neural-dynamics method. Then the two subschemes are simultaneously reformulated as two quadratic programs (QPs), which are finally unified into one QP problem. Furthermore, a piecewise-linear projection equation-based neural network (PLPENN) is used to solve the unified QP problem, which can handle the strictly convex QP problem in an inverse-free manner. More importantly, via such a unified QP formulation and the corresponding PLPENN solver, the synchronism of two redundant robot arms is guaranteed. Finally, two given tasks are fulfilled by 2 three-link and 2 five-link planar robot arms, respectively. Computer-simulation results validate the efficacy and accuracy of the SRMPC scheme and the corresponding PLPENN solver for synchronous manipulation of two redundant robot arms.

Algorithms for collision-free navigation of mobile robots in complex cluttered environments: a survey

We review a range of techniques related to navigation of unmanned vehicles through unknown environments with obstacles, especially those that rigorously ensure collision avoidance (given certain assumptions about the system). This topic continues to be an active area of research, and we highlight some directions in which available approaches may be improved. The paper discusses models of the sensors and vehicle kinematics, assumptions about the environment, and performance criteria. Methods applicable to stationary obstacles, moving obstacles and multiple vehicles scenarios are all reviewed. In preference to global approaches based on full knowledge of the environment, particular attention is given to reactive methods based on local sensory data, with a special focus on recently proposed navigation laws based on model predictive and sliding mode control.

Contact-state classification in human-demonstrated robot compliant motion tasks using the boosting algorithm

Robot programming by demonstration is a robot programming paradigm in which a human operator directly demonstrates the task to be performed. In this paper, we focus on programming by demonstration of compliant motion tasks, which are tasks that involve contacts between an object manipulated by the robot and the environment in which it operates. Critical issues in this paradigm are to distinguish essential actions from those that are not relevant for the correct execution of the task and to transform this information into a robot-independent representation. Essential actions in compliant motion tasks are the contacts that take place, and therefore, it is important to understand the sequence of contact states that occur during a demonstration, called contact classification or contact segmentation. We propose a contact classification algorithm based on a supervised learning algorithm, in particular on a stochastic gradient boosting algorithm. The approach described in this paper is accurate and does not depend on the geometric model of the objects involved in the demonstration. It neither relies on the kinestatic model of the contact interactions nor on the contact state graph, whose computation is usually of prohibitive complexity even for very simple geometric object models.

A Framework for the Simulation and Haptic Display of Dynamic Systems Subject to Holonomic Constraints

In this paper we present a framework that enables an operator to haptically and visually interact with a simulated dynamic environment subject to virtual holonomic constraints. The framework combines a geometric constraint solver with a constrained dynamics simulation engine that controls an admittance-type haptic display. This system takes on relevant issues in the context of assisted teleoperated tasks, from providing an intuitive interface for creating and combining virtual constraints, to haptically displaying rigid motion constraints in simulated environments subject to desired inertial dynamics. Two experiments carried out using the Cobotic Hand Controller haptic display are presented.

Inverse Kinematics and Control of a 7-DOF Redundant Manipulator Based on the Closed-Loop Algorithm

Closed-loop inverse kinematics (CLIK) algorithm mostly resolves the redundancy at the velocity level. In this paper we extend the CLIK algorithm to the acceleration level to meet some applications that require the joint accelerations. The redundancy resolutions at the velocities and acceleration levels via pseudoinverse method are analyzed respectively. The objective function of joint limits avoidance(JLA) is combined into the redundancy resolution as an optimization approach of the null space motion. A seven-DOF redundant manipulator is designed to do the computer simulations and the real experiments are carried out on a Powercube modular manipulator. Their results demonstrated the effectiveness of the proposed algorithm.

Hybrid Adaptive Vision—Force Control for Robot Manipulators Interacting with Unknown Surfaces

A hybrid control scheme based on adaptive visual servoing and direct force control is proposed for robot manipulators to perform interaction tasks on smooth surfaces. The camera parameters, as well as the constraint surface, are considered to be uncertain. A fixed camera with optical axis non-perpendicular to the robot workspace is used for position control, while a force sensor mounted on the robot wrist is used for force regulation. In order to solve the interaction problem on unknown surfaces, a method is developed to estimate the constraint geometry and keep the end-effector orthogonal to the surface at the contact point, during the task execution. Experimental results are presented to illustrate the performance and feasibility of the proposed scheme.