Versatile multicontact planning and control for legged loco-manipulation

Loco-manipulation planning skills are pivotal for expanding the utility of robots in everyday environments. These skills can be assessed on the basis of a system's ability to coordinate complex holistic movements and multiple contact interactions when solving different tasks. However, existing approaches have been merely able to shape such behaviors with hand-crafted state machines, densely engineered rewards, or prerecorded expert demonstrations. Here, we propose a minimally guided framework that automatically discovers whole-body trajectories jointly with contact schedules for solving general loco-manipulation tasks in premodeled environments. The key insight is that multimodal problems of this nature can be formulated and treated within the context of integrated task and motion planning (TAMP). An effective bilevel search strategy was achieved by incorporating domain-specific rules and adequately combining the strengths of different planning techniques: trajectory optimization and informed graph search coupled with sampling-based planning. We showcase emergent behaviors for a quadrupedal mobile manipulator exploiting both prehensile and nonprehensile interactions to perform real-world tasks such as opening/closing heavy dishwashers and traversing spring-loaded doors. These behaviors were also deployed on the real system using a two-layer whole-body tracking controller.

Adaptive assistive robotics: a framework for triadic collaboration between humans and robots

Robots and other assistive technologies have a huge potential to help society in domains ranging from factory work to healthcare. However, safe and effective control of robotic agents in these environments is complex, especially when it involves close interactions and multiple actors. We propose an effective framework for optimizing the behaviour of robots and complementary assistive technologies in systems comprising a mix of human and technological agents with numerous high-level goals. The framework uses a combination of detailed biomechanical modelling and weighted multi-objective optimization to allow for the fine tuning of robot behaviours depending on the specification of the task at hand. We illustrate our framework via two case studies across assisted living and rehabilitation scenarios, and conduct simulations and experiments of triadic collaboration in practice. Our results indicate a marked benefit to the triadic approach, showing the potential to improve outcome measures for human agents in robot-assisted tasks.

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

In human-robot collaboration scenarios with shared workspaces, a highly desired performance boost is offset by high requirements for human safety, limiting speed and torque of the robot drives to levels which cannot harm the human body. Especially for complex tasks with flexible human behavior, it becomes vital to maintain safe working distances and coordinate tasks efficiently. An established approach in this regard is reactive servo in response to the current human pose. However, such an approach does not exploit expectations of the human's behavior and can therefore fail to react to fast human motions in time. To adapt the robot's behavior as soon as possible, predicting human intention early becomes a factor which is vital but hard to achieve. Here, we employ a recently developed type of brain-computer interface (BCI) which can detect the focus of the human's overt attention as a predictor for impending action. In contrast to other types of BCI, direct projection of stimuli onto the workspace facilitates a seamless integration in workflows. Moreover, we demonstrate how the signal-to-noise ratio of the brain response can be used to adjust the velocity of the robot movements to the vigilance or alertness level of the human. Analyzing this adaptive system with respect to performance and safety margins in a physical robot experiment, we found the proposed method could improve both collaboration efficiency and safety distance.

Investigating Bi-Level Optimization for Learning and Vision From a Unified Perspective: A Survey and Beyond

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO.

Global motion planning and redundancy resolution for large objects manipulation by dual redundant robots with closed kinematics

The multi-arm robotic systems consisting of redundant robots are able to conduct more complex and coordinated tasks, such as manipulating large or heavy objects. The challenges of the motion planning and control for such systems mainly arise from the closed-chain constraint and redundancy resolution problem. The closed-chain constraint reduces the configuration space to lower-dimensional subsets, making it difficult for sampling feasible configurations and planning path connecting them. A global motion planner is proposed in this paper for the closed-chain systems, and motions in different disconnected manifolds are efficiently bridged by two type regrasping moves. The regrasping moves are automatically chosen by the planner based on cost-saving principle, which greatly improve the success rate and efficiency. Furthermore, to obtain the optional inverse kinematic solutions satisfying joint physical limits (e.g., joint position, velocity, acceleration limits) in the planning, the redundancy resolution problem for dual redundant robots is converted into a unified quadratic programming problem based on the combination of two diff erent-level optimizing criteria, i.e. the minimization velocity norm (MVN) and infinity norm torque-minimization (INTM). The Dual-MVN-INTM scheme guarantees smooth velocity, acceleration profiles, and zero final velocity at the end of motion. Finally, the planning results of three complex closed-chain manipulation task using two Franka Emika Panda robots and two Kinova Jaco2 robots in both simulation and experiment demonstrate the effectiveness and efficiency of the proposed method.

HapFIC: An Adaptive Force/Position Controller for Safe Environment Interaction in Articulated Systems

Haptic interaction is essential for the dynamic dexterity of animals, which seamlessly switch from an impedance to an admittance behaviour using the force feedback from their proprioception. However, this ability is extremely challenging to reproduce in robots, especially when dealing with complex interaction dynamics, distributed contacts, and contact switching. Current model-based controllers require accurate interaction modelling to account for contacts and stabilise the interaction. In this manuscript, we propose an adaptive force/position controller that exploits the fractal impedance controller's passivity and non-linearity to execute a finite search algorithm using the force feedback signal from the sensor at the end-effector. The method is computationally inexpensive, opening the possibility to deal with distributed contacts in the future. We evaluated the architecture in physics simulation and showed that the controller can robustly control the interaction with objects of different dynamics without violating the maximum allowable target forces or causing numerical instability even for very rigid objects. The proposed controller can also autonomously deal with contact switching and may find application in multiple fields such as legged locomotion, rehabilitation and assistive robotics.