Learning parametric dynamic movement primitives from multiple demonstrations

Phase-Synchronized Learning of Periodic Compliant Movement Primitives (P-CMPs)

Autonomous trajectory and torque profile synthesis through modulation and generalization require a database of motion with accompanying dynamics, which is typically difficult and time-consuming to obtain. Inspired by adaptive control strategies, this paper presents a novel method for learning and synthesizing Periodic Compliant Movement Primitives (P-CMPs). P-CMPs combine periodic trajectories encoded as Periodic Dynamic Movement Primitives (P-DMPs) with accompanying task-specific Periodic Torque Primitives (P-TPs). The state-of-the-art approach requires to learn TPs for each variation of the task, e.g., modulation of frequency. Comparatively, in this paper, we propose a novel P-TPs framework, which is both frequency and phase-dependent. Thereby, the executed P-CMPs can be easily modulated, and consequently, the learning rate can be improved. Moreover, both the kinematic and the dynamic profiles are parameterized, thus enabling the representation of skills using corresponding parameters. The proposed framework was evaluated on two robot systems, i.e., Kuka LWR-4 and Franka Emika Panda. The evaluation of the proposed approach on a Kuka LWR-4 robot performing a swinging motion and on Franka Emika Panda performing an exercise for elbow rehabilitation shows fast P-CTPs acquisition and accurate and compliant motion in real-world scenarios.

An Enhanced Robot Massage System in Smart Homes Using Force Sensing and a Dynamic Movement Primitive

With requirements to improve life quality, smart homes, and healthcare have gradually become a future lifestyle. In particular, service robots with human behavioral sensing for private or personal use in the home have attracted a lot of research attention thanks to their advantages in relieving high labor costs and the fatigue of human assistance. In this paper, a novel force-sensing- and robotic learning algorithm-based teaching interface for robot massaging has been proposed. For the teaching purposes, a human operator physically holds the end-effector of the robot to perform the demonstration. At this stage, the end position data are outputted and sent to be segmented via the Finite Difference (FD) method. A Dynamic Movement Primitive (DMP) is utilized to model and generalize the human-like movements. In order to learn from multiple demonstrations, Dynamic Time Warping (DTW) is used for the preprocessing of the data recorded on the robot platform, and a Gaussian Mixture Model (GMM) is employed for the evaluation of DMP to generate multiple patterns after the completion of the teaching process. After that, a Gaussian Mixture Regression (GMR) algorithm is applied to generate a synthesized trajectory to minimize position errors. Then a hybrid position/force controller is integrated to track the desired trajectory in the task space while considering the safety of human-robot interaction. The validation of our proposed method has been performed and proved by conducting massage tasks on a KUKA LBR iiwa robot platform.

Fast social-like learning of complex behaviors based on motor motifs

Social learning is widely observed in many species. Less experienced agents copy successful behaviors exhibited by more experienced individuals. Nevertheless, the dynamical mechanisms behind this process remain largely unknown. Here we assume that a complex behavior can be decomposed into a sequence of n motor motifs. Then a neural network capable of activating motor motifs in a given sequence can drive an agent. To account for (n-1)! possible sequences of motifs in a neural network, we employ the winnerless competition approach. We then consider a teacher-learner situation: one agent exhibits a complex movement, while another one aims at mimicking the teacher's behavior. Despite the huge variety of possible motif sequences we show that the learner, equipped with the provided learning model, can rewire "on the fly" its synaptic couplings in no more than (n-1) learning cycles and converge exponentially to the durations of the teacher's motifs. We validate the learning model on mobile robots. Experimental results show that the learner is indeed capable of copying the teacher's behavior composed of six motor motifs in a few learning cycles. The reported mechanism of learning is general and can be used for replicating different functions, including, for example, sound patterns or speech.

Bootstrapping of Parameterized Skills Through Hybrid Optimization in Task and Policy Spaces

Modern robotic applications create high demands on adaptation of actions with respect to variance in a given task. Reinforcement learning is able to optimize for these changing conditions, but relearning from scratch is hardly feasible due to the high number of required rollouts. We propose a parameterized skill that generalizes to new actions for changing task parameters, which is encoded as a meta-learner that provides parameters for task-specific dynamic motion primitives. Our work shows that utilizing parameterized skills for initialization of the optimization process leads to a more effective incremental task learning. In addition, we introduce a hybrid optimization method that combines a fast coarse optimization on a manifold of policy parameters with a fine grained parameter search in the unrestricted space of actions. The proposed algorithm reduces the number of required rollouts for adaptation to new task conditions. Application in illustrative toy scenarios, for a 10-DOF planar arm, and a humanoid robot point reaching task validate the approach.

Spatiotemporal synchronization of biped walking patterns with multiple external inputs by style-phase adaptation

In this paper, we propose a framework for generating coordinated periodic movements of robotic systems with multiple external inputs. We developed an adaptive pattern generator model that is composed of a two-factor observation model with a style parameter and phase dynamics with a phase variable. The style parameter controls the spatial patterns of the generated trajectories, and the phase variable manages its temporal profiles. By exploiting the style-phase separation in the pattern generation, we can independently design adaptation schemes for the spatial and temporal profiles of the pattern generator to multiple external inputs. To validate the effectiveness of our proposed method, we applied it to a user-exoskeleton model to achieve user-adaptive walking assistance for which the exoskeleton robot's movements need to be coordinated with the user walking patterns and environment. As a result, the exoskeleton robot successfully performed stable biped walking behaviors for walking assistance even when the style of the observed walking pattern and the period were suddenly changed.