1
|
Li J, Han H, Hu J, Lin J, Li P. Robot Learning Method for Human-like Arm Skills Based on the Hybrid Primitive Framework. SENSORS (BASEL, SWITZERLAND) 2024; 24:3964. [PMID: 38931748 PMCID: PMC11207368 DOI: 10.3390/s24123964] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2024] [Revised: 06/11/2024] [Accepted: 06/12/2024] [Indexed: 06/28/2024]
Abstract
This paper addresses the issue of how to endow robots with motion skills, flexibility, and adaptability similar to human arms. It innovatively proposes a hybrid-primitive-frame-based robot skill learning algorithm and utilizes the policy improvement with a path integral algorithm to optimize the parameters of the hybrid primitive framework, enabling robots to possess skills similar to human arms. Firstly, the end of the robot is dynamically modeled using an admittance control model to give the robot flexibility. Secondly, the dynamic movement primitives are employed to model the robot's motion trajectory. Additionally, novel stiffness primitives and damping primitives are introduced to model the stiffness and damping parameters in the impedance model. The combination of the dynamic movement primitives, stiffness primitives, and damping primitives is called the hybrid primitive framework. Simulated experiments are designed to validate the effectiveness of the hybrid-primitive-frame-based robot skill learning algorithm, including point-to-point motion under external force disturbance and trajectory tracking under variable stiffness conditions.
Collapse
Affiliation(s)
- Jiaxin Li
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
| | - Hasiaoqier Han
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Jinxin Hu
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Junwei Lin
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
- University of Chinese Academy of Sciences, Beijing 100049, China
| | - Peiyi Li
- Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China
- University of Chinese Academy of Sciences, Beijing 100049, China
| |
Collapse
|
2
|
Zang Y, Wang P, Zha F, Guo W, Zheng C, Sun L. Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation. Front Neurorobot 2023; 17:1320251. [PMID: 38023454 PMCID: PMC10666750 DOI: 10.3389/fnbot.2023.1320251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Accepted: 10/18/2023] [Indexed: 12/01/2023] Open
Abstract
Introduction Behavioral Cloning (BC) is a common imitation learning method which utilizes neural networks to approximate the demonstration action samples for task manipulation skill learning. However, in the real world, the demonstration trajectories from human are often sparse and imperfect, which makes it challenging to comprehensively learn directly from the demonstration action samples. Therefore, in this paper, we proposes a streamlined imitation learning method under the terse geometric representation to take good advantage of the demonstration data, and then realize the manipulation skill learning of assembly tasks. Methods We map the demonstration trajectories into the geometric feature space. Then we align the demonstration trajectories by Dynamic Time Warping (DTW) method to get the unified data sequence so we can segment them into several time stages. The Probability Movement Primitives (ProMPs) of the demonstration trajectories are then extracted, so we can generate a lot of task trajectories to be the global strategy action samples for training the neural networks. Notalby, we regard the current state of the assembly task as the via point of the ProMPs model to get the generated trajectories, while the time point of the via point is calculated according to the probability model of the different time stages. And we get the action of the current state according to the target position of the next time state. Finally, we train the neural network to obtain the global assembly strategy by Behavioral Cloning. Results We applied the proposed method to the peg-in-hole assembly task in the simulation environment based on Pybullet + Gym to test its task skill learning performance. And the learned assembly strategy was also executed on a real robotic platform to verify the feasibility of the method further. Discussion According to the result of the experiment, the proposed method achieves higher success rates compared to traditional imitation learning methods while exhibiting reasonable generalization capabilities. It shows that the ProMPs under geometric representation can help the BC method make better use of the demonstration trajectory and thus better learn the task skills.
Collapse
Affiliation(s)
- Yajing Zang
- School of Mechatronics Engineering, State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Pengfei Wang
- School of Mechatronics Engineering, State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Fusheng Zha
- School of Mechatronics Engineering, State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Wei Guo
- School of Mechatronics Engineering, State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| | - Chao Zheng
- Wuhan Second Ship Design and Research Institute, Wuhan, China
| | - Lining Sun
- School of Mechatronics Engineering, State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, China
| |
Collapse
|
3
|
Oikonomou P, Dometios A, Khamassi M, Tzafestas CS. Zero-shot model-free learning of periodic movements for a bio-inspired soft-robotic arm. Front Robot AI 2023; 10:1256763. [PMID: 37929074 PMCID: PMC10621048 DOI: 10.3389/frobt.2023.1256763] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 10/03/2023] [Indexed: 11/07/2023] Open
Abstract
In recent years, soft robots gain increasing attention as a result of their compliance when operating in unstructured environments, and their flexibility that ensures safety when interacting with humans. However, challenges lie on the difficulty to develop control algorithms due to various limitations induced by their soft structure. In this paper, we introduce a novel technique that aims to perform motion control of a modular bio-inspired soft-robotic arm, with the main focus lying on facilitating the qualitative reproduction of well-specified periodic trajectories. The introduced method combines the notion behind two previously developed methodologies both based on the Movement Primitive (MP) theory, by exploiting their capabilities while coping with their main drawbacks. Concretely, the requested actuation is initially computed using a Probabilistic MP (ProMP)-based method that considers the trajectory as a combination of simple movements previously learned and stored as a MP library. Subsequently, the key components of the resulting actuation are extracted and filtered in the frequency domain. These are eventually used as input to a Central Pattern Generator (CPG)-based model that takes over the generation of rhythmic patterns at the motor level. The proposed methodology is evaluated on a two-module soft arm. Results show that the first algorithmic component (ProMP) provides an immediate estimation of the requested actuation by avoiding time-consuming training, while the latter (CPG) further simplifies the execution by allowing its control through a low-dimensional parameterization. Altogether, these results open new avenues for the rapid acquisition of periodic movements in soft robots, and their compression into CPG parameters for long-term storage and execution.
Collapse
Affiliation(s)
- Paris Oikonomou
- Division of Signals, Control and Robotics, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | - Athanasios Dometios
- Division of Signals, Control and Robotics, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| | - Mehdi Khamassi
- Division of Signals, Control and Robotics, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
- Sorbonne Université, Centre National de la Recherche Scientifique, Institute of Intelligent Systems and Robotics, Paris, France
| | - Costas S Tzafestas
- Division of Signals, Control and Robotics, School of Electrical and Computer Engineering, National Technical University of Athens, Athens, Greece
| |
Collapse
|
4
|
Wang J, Wu D, Gao Y, Dong W. Interaction learning control with movement primitives for lower limb exoskeleton. Front Neurorobot 2022; 16:1086578. [PMID: 36605521 PMCID: PMC9807752 DOI: 10.3389/fnbot.2022.1086578] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Accepted: 11/23/2022] [Indexed: 12/24/2022] Open
Abstract
Research on robotic exoskeletons both in the military and medical fields has rapidly expanded over the previous decade. As a human-robot interaction system, it is a challenge to develop an assistive strategy that makes the exoskeleton supply efficient and natural assistance following the user's intention. This paper proposed a novel interaction learning control strategy for the lower extremity exoskeleton. A powerful representative tool probabilistic movement primitives (ProMPs) is adopted to model the motion and generate the desired trajectory in real-time. To adjust the trajectory by the user's real-time intention, a compensation term based on human-robot interaction force is designed and merged into the ProMPs model. Then, compliant impedance control is adopted as a low-level control where the desired trajectory is put into. Moreover, the model will be dynamically adapted online by penalizing both the interaction force and trajectory mismatch, with all the parameters that can be further learned by learning algorithm PIBB. The experimental results verified the effectiveness of the proposed control framework.
Collapse
|
5
|
Sutanto G, Rombach K, Chebotar Y, Su Z, Schaal S, Sukhatme GS, Meier F. Supervised learning and reinforcement learning of feedback models for reactive behaviors: Tactile feedback testbed. Int J Rob Res 2022. [DOI: 10.1177/02783649221143399] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Robots need to be able to adapt to unexpected changes in the environment such that they can autonomously succeed in their tasks. However, hand-designing feedback models for adaptation is tedious, if at all possible, making data-driven methods a promising alternative. In this paper, we introduce a full framework for learning feedback models for reactive motion planning. Our pipeline starts by segmenting demonstrations of a complete task into motion primitives via a semi-automated segmentation algorithm. Then, given additional demonstrations of successful adaptation behaviors, we learn initial feedback models through learning-from-demonstrations. In the final phase, a sample-efficient reinforcement learning algorithm fine-tunes these feedback models for novel task settings through few real system interactions. We evaluate our approach on a real anthropomorphic robot in learning a tactile feedback task.
Collapse
Affiliation(s)
- Giovanni Sutanto
- Autonomous Motion Department, Max Planck Institute for Intelligent Systems (AMD at MPI-IS), Tübingen, Germany
- Department of Computer Science, University of Southern California (CS at USC), Los Angeles, CA, USA
- Intrinsic Innovation LLC., Mountain View, CA, USA
| | | | | | - Zhe Su
- Dexterity Inc., Redwood City, CA, USA
| | | | - Gaurav S. Sukhatme
- Department of Computer Science, University of Southern California (CS at USC), Los Angeles, CA, USA
| | | |
Collapse
|
6
|
Sidiropoulos A, Papageorgiou D, Doulgeri Z. A novel framework for generalizing dynamic movement primitives under kinematic constraints. Auton Robots 2022. [DOI: 10.1007/s10514-022-10067-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Abstract
AbstractIn this work, we propose a novel framework for generalizing a desired trajectory pattern, encoded using Dynamic Movement Primitives (DMP), subject to kinematic constraints. DMP have been extensively used in robotics for encoding and reproducing kinematic behaviours, thanks to their generalization, stability and robustness properties. However, incorporating kinematic constraints has not yet been fully addressed. To this end, we design an optimization framework, based on the DMP formulation from our previous work, for generalizing trajectory patterns, encoded with DMP subject to kinematic constraints, considering also time-varying target and time duration, via-point and obstacle constraints. Simulations highlight these properties and comparisons are drawn with other approaches for enforcing constraints on DMP. The usefulness and applicability of the proposed framework is showcased in experimental scenarios, including a handover, where the target and time duration vary, and placing scenarios, where obstacles are dynamically introduced in the scene.
Collapse
|
7
|
Herzog R, Berger TM, Pauly MG, Xue H, Rueckert E, Münchau A, Bäumer T, Weissbach A. Cerebellar transcranial current stimulation – An intraindividual comparison of different techniques. Front Neurosci 2022; 16:987472. [PMID: 36188449 PMCID: PMC9521312 DOI: 10.3389/fnins.2022.987472] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2022] [Accepted: 08/22/2022] [Indexed: 11/13/2022] Open
Abstract
Transcranial current stimulation (tCS) techniques have been shown to induce cortical plasticity. As an important relay in the motor system, the cerebellum is an interesting target for plasticity induction using tCS, aiming to modulate its excitability and connectivity. However, until now it remains unclear, which is the most effective tCS method for inducing plasticity in the cerebellum. Thus, in this study, the effects of anodal transcranial direct current stimulation (tDCS), 50 Hz transcranial alternating current stimulation (50 Hz tACS), and high frequency transcranial random noise stimulation (tRNS) were compared with sham stimulation in 20 healthy subjects in a within-subject design. tCS was applied targeting the cerebellar lobe VIIIA using neuronavigation. We measured corticospinal excitability, short-interval intracortical inhibition (SICI), short-latency afferent inhibition (SAI), and cerebellar brain inhibition (CBI) and performed a sensor-based movement analysis at baseline and three times after the intervention (post1 = 15 min; post2 = 55 min; post3 = 95 min). Corticospinal excitability increased following cerebellar tACS and tRNS compared to sham stimulation. This effect was most pronounced directly after stimulation but lasted for at least 55 min after tACS. Cortico-cortical and cerebello-cortical conditioning protocols, as well as sensor-based movement analyses, did not change. Our findings suggest that cerebellar 50 Hz tACS is the most effective protocol to change corticospinal excitability.
Collapse
Affiliation(s)
- Rebecca Herzog
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
- Department of Neurology, University Hospital Schleswig Holstein, Lübeck, Germany
| | - Till M. Berger
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
| | - Martje G. Pauly
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
- Department of Neurology, University Hospital Schleswig Holstein, Lübeck, Germany
- Institute of Neurogenetics, University of Lübeck, Lübeck, Germany
| | - Honghu Xue
- Institute for Robotics and Cognitive Systems, University of Lübeck, Lübeck, Germany
| | | | - Alexander Münchau
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
| | - Tobias Bäumer
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
| | - Anne Weissbach
- Institute of Systems Motor Science, University of Lübeck, Lübeck, Germany
- Institute of Neurogenetics, University of Lübeck, Lübeck, Germany
- *Correspondence: Anne Weissbach,
| |
Collapse
|
8
|
Frank F, Paraschos A, van der Smagt P, Cseke B. Constrained Probabilistic Movement Primitives for Robot Trajectory Adaptation. IEEE T ROBOT 2022. [DOI: 10.1109/tro.2021.3127108] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
- Felix Frank
- Volkswagen Machine Learning Research Lab, Munich, Germany
| | | | | | - Botond Cseke
- Volkswagen Machine Learning Research Lab, Munich, Germany
| |
Collapse
|
9
|
Yervilla-Herrera H, Becerra I, Murrieta-Cid R, Sucar LE, Morales EF. Bayesian Probabilistic Stopping Test and Asymptotic Shortest Time Trajectories for Object Reconstruction with a Mobile Manipulator Robot. J INTELL ROBOT SYST 2022. [DOI: 10.1007/s10846-022-01696-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
10
|
Baumkircher A, Seme K, Munih M, Mihelj M. Collaborative Robot Precision Task in Medical Microbiology Laboratory. SENSORS 2022; 22:s22082862. [PMID: 35458847 PMCID: PMC9025832 DOI: 10.3390/s22082862] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2022] [Revised: 04/01/2022] [Accepted: 04/07/2022] [Indexed: 11/16/2022]
Abstract
This study focuses on the feasibility of collaborative robot implementation in a medical microbiology laboratory by demonstrating fine tasks using kinesthetic teaching. Fine tasks require sub-millimetre positioning accuracy. Bacterial colony picking and identification was used as a case study. Colonies were picked from Petri dishes and identified using matrix-assisted laser desorption/ionization (MALDI) time-of-flight (TOF) mass spectrometry. We picked and identified 56 colonies (36 colonies of Gram-negative Acinetobacter baumannii and 20 colonies of Gram-positive Staphylococcus epidermidis). The overall identification error rate was around 11%, although it was significantly lower for Gram-positive bacteria (5%) than Gram-negative bacteria (13.9%). Based on the identification scores, it was concluded that the system works similarly well as a manual operator. It was determined that tasks were successfully demonstrated using kinesthetic teaching and generalized using dynamic movement primitives (DMP). Further improvement of the identification error rate is possible by choosing a different deposited sample treatment method (e.g., semi-extraction, wet deposition).
Collapse
Affiliation(s)
- Aljaz Baumkircher
- Laboratory of Robotics, Faculty of Electrical Engineering, University of Ljubljana, Tržaška Cesta 25, 1000 Ljubljana, Slovenia
| | - Katja Seme
- Institute of Microbiology and Immunology, Faculty of Medicine, University of Ljubljana, Zaloška 4, 1000 Ljubljana, Slovenia
| | - Marko Munih
- Laboratory of Robotics, Faculty of Electrical Engineering, University of Ljubljana, Tržaška Cesta 25, 1000 Ljubljana, Slovenia
| | - Matjaž Mihelj
- Laboratory of Robotics, Faculty of Electrical Engineering, University of Ljubljana, Tržaška Cesta 25, 1000 Ljubljana, Slovenia
| |
Collapse
|
11
|
Jankowski J, Racca M, Calinon S. From Key Positions to Optimal Basis Functions for Probabilistic Adaptive Control. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3146614] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
12
|
Davoodi M, Iqbal A, Cloud JM, Beksi WJ, Gans NR. Safe Robot Trajectory Control Using Probabilistic Movement Primitives and Control Barrier Functions. Front Robot AI 2022; 9:772228. [PMID: 35368435 PMCID: PMC8965845 DOI: 10.3389/frobt.2022.772228] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/07/2021] [Accepted: 02/07/2022] [Indexed: 11/13/2022] Open
Abstract
In this paper, we present a novel means of control design for probabilistic movement primitives (ProMPs). Our proposed approach makes use of control barrier functions and control Lyapunov functions defined by a ProMP distribution. Thus, a robot may move along a trajectory within the distribution while guaranteeing that the system state never leaves more than a desired distance from the distribution mean. The control employs feedback linearization to handle nonlinearities in the system dynamics and real-time quadratic programming to ensure a solution exists that satisfies all safety constraints while minimizing control effort. Furthermore, we highlight how the proposed method may allow a designer to emphasize certain safety objectives that are more important than the others. A series of simulations and experiments demonstrate the efficacy of our approach and show it can run in real time.
Collapse
Affiliation(s)
- Mohammadreza Davoodi
- The University of Texas at Arlington Research Institute, Fort Worth, TX, United States
| | - Asif Iqbal
- The University of Texas at Arlington Research Institute, Fort Worth, TX, United States
| | - Joseph M. Cloud
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX, United States
| | - William J. Beksi
- Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX, United States
| | - Nicholas R. Gans
- The University of Texas at Arlington Research Institute, Fort Worth, TX, United States
- *Correspondence: Nicholas R. Gans,
| |
Collapse
|
13
|
Dou S, Xiao J, Zhao W, Yuan H, Liu H. A Robot Skill Learning Framework Based on Compliant Movement Primitives. J INTELL ROBOT SYST 2022. [DOI: 10.1007/s10846-022-01605-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
14
|
User intent estimation during robot learning using physical human robot interaction primitives. Auton Robots 2022. [DOI: 10.1007/s10514-021-10030-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
Abstract
AbstractAs robotic systems transition from traditional setups to collaborative work spaces, the prevalence of physical Human Robot Interaction has risen in both industrial and domestic environments. A popular representation for robot behavior is movement primitives which learn, imitate, and generalize from expert demonstrations. While there are existing works in context-aware movement primitives, they are usually limited to contact-free human robot interactions. This paper presents physical Human Robot Interaction Primitives (pHRIP), which utilize only the interaction forces between the human user and robot to estimate user intent and generate the appropriate robot response during physical human robot interactions. The efficacy of pHRIP is evaluated through multiple experiments based on target-directed reaching and obstacle avoidance tasks using a real seven degree of freedom robot arm. The results are validated against Interaction Primitives which use observations of robotic trajectories, with discussions of future pHRI applications utilizing pHRIP.
Collapse
|
15
|
Learning to Sequence and Blend Robot Skills via Differentiable Optimization. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3184003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
16
|
Gomes W, Maurice P, Dalin E, Mouret JB, Ivaldi S. Multi-Objective Trajectory Optimization to Improve Ergonomics in Human Motion. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2021.3125058] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
17
|
Ti B, Gao Y, Shi M, Fu L, Zhao J. Movement generalization of variable initial task state based on Euclidean transformation dynamical movement primitives. INT J ADV ROBOT SYST 2021. [DOI: 10.1177/17298814211065577] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Robots need the ability to tackle problems of movement generalization in variable task state and complex environment. Dynamical movement primitives can effectively endow robots with humanoid characteristics. However, when the initial state of tasks changes, the generalized trajectories by dynamical movement primitives cannot retain shape features of demonstration, resulting in the loss of imitation quality. In this article, a modified dynamical movement primitives based on Euclidean transformation is proposed to solve this problem. It transforms the initial task state to a virtual situation similar to the demonstration and then utilizes the dynamical movement primitive method to realize movement generalization. Finally, it reverses the movement back to the real situation. Besides, the information of obstacles is added to Euclidean transformation based dynamical movement primitives framework to endow robots with the ability of obstacle avoidance. The normalized root-mean-square error is proposed as the criterion to evaluate the imitation similarity. The feasibility of this method is verified through writing letters, wiping whiteboard in two-dimensional task, and stirring mixture in three-dimensional task. The results show that the similarity of movement imitation in the proposed method is higher than dynamical movement primitives when the initial state changes. Meanwhile, Euclidean transformation based dynamical movement primitives can still greatly retain shape feature of demonstration while avoiding obstacles in an unstructured environment.
Collapse
Affiliation(s)
- Boyang Ti
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, People’s Republic of China
| | - Yongsheng Gao
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, People’s Republic of China
| | - Ming Shi
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, People’s Republic of China
| | - Le Fu
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, People’s Republic of China
| | - Jie Zhao
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin, Heilongjiang, People’s Republic of China
| |
Collapse
|
18
|
Caserman P, Krug C, Göbel S. Recognizing Full-Body Exercise Execution Errors Using the Teslasuit. SENSORS (BASEL, SWITZERLAND) 2021; 21:8389. [PMID: 34960481 PMCID: PMC8703589 DOI: 10.3390/s21248389] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/29/2021] [Revised: 12/02/2021] [Accepted: 12/09/2021] [Indexed: 11/16/2022]
Abstract
Regular physical exercise is essential for overall health; however, it is also crucial to mitigate the probability of injuries due to incorrect exercise executions. Existing health or fitness applications often neglect accurate full-body motion recognition and focus on a single body part. Furthermore, they often detect only specific errors or provide feedback first after the execution. This lack raises the necessity for the automated detection of full-body execution errors in real-time to assist users in correcting motor skills. To address this challenge, we propose a method for movement assessment using a full-body haptic motion capture suit. We train probabilistic movement models using the data of 10 inertial sensors to detect exercise execution errors. Additionally, we provide haptic feedback, employing transcutaneous electrical nerve stimulation immediately, as soon as an error occurs, to correct the movements. The results based on a dataset collected from 15 subjects show that our approach can detect severe movement execution errors directly during the workout and provide haptic feedback at respective body locations. These results suggest that a haptic full-body motion capture suit, such as the Teslasuit, is promising for movement assessment and can give appropriate haptic feedback to the users so that they can improve their movements.
Collapse
Affiliation(s)
- Polona Caserman
- Research Group Serious Games, Technical University of Darmstadt, Rundeturmstrasse 10, 64283 Darmstadt, Germany; (C.K.); (S.G.)
| | | | | |
Collapse
|
19
|
Xue H, Herzog R, Berger TM, Bäumer T, Weissbach A, Rueckert E. Using Probabilistic Movement Primitives in Analyzing Human Motion Differences Under Transcranial Current Stimulation. Front Robot AI 2021; 8:721890. [PMID: 34595209 PMCID: PMC8476753 DOI: 10.3389/frobt.2021.721890] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Accepted: 08/19/2021] [Indexed: 11/13/2022] Open
Abstract
In medical tasks such as human motion analysis, computer-aided auxiliary systems have become the preferred choice for human experts for their high efficiency. However, conventional approaches are typically based on user-defined features such as movement onset times, peak velocities, motion vectors, or frequency domain analyses. Such approaches entail careful data post-processing or specific domain knowledge to achieve a meaningful feature extraction. Besides, they are prone to noise and the manual-defined features could hardly be re-used for other analyses. In this paper, we proposed probabilistic movement primitives (ProMPs), a widely-used approach in robot skill learning, to model human motions. The benefit of ProMPs is that the features are directly learned from the data and ProMPs can capture important features describing the trajectory shape, which can easily be extended to other tasks. Distinct from previous research, where classification tasks are mostly investigated, we applied ProMPs together with a variant of Kullback-Leibler (KL) divergence to quantify the effect of different transcranial current stimulation methods on human motions. We presented an initial result with 10 participants. The results validate ProMPs as a robust and effective feature extractor for human motions.
Collapse
Affiliation(s)
- Honghu Xue
- Institute for Robotics and Cognitive Systems, University of Luebeck, Luebeck, Germany
| | - Rebecca Herzog
- Institute of Systems Motor Science, University of Luebeck, Luebeck, Germany.,Department of Neurology, University Medical Center Schleswig-Holstein, Luebeck, Germany
| | - Till M Berger
- Institute of Systems Motor Science, University of Luebeck, Luebeck, Germany
| | - Tobias Bäumer
- Department of Neurology, University Medical Center Schleswig-Holstein, Luebeck, Germany
| | - Anne Weissbach
- Institute of Systems Motor Science, University of Luebeck, Luebeck, Germany.,Institute of Neurogenetics, University of Luebeck, Luebeck, Germany
| | - Elmar Rueckert
- Chair of Cyber-Physical-Systems, Montanuniversität Leoben, Leoben, Austria
| |
Collapse
|
20
|
Guan Y, Wang N, Yang C. An Improvement of Robot Stiffness-Adaptive Skill Primitive Generalization Using the Surface Electromyography in Human-Robot Collaboration. Front Neurosci 2021; 15:694914. [PMID: 34594181 PMCID: PMC8478287 DOI: 10.3389/fnins.2021.694914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Accepted: 08/06/2021] [Indexed: 11/29/2022] Open
Abstract
Learning from Demonstration in robotics has proved its efficiency in robot skill learning. The generalization goals of most skill expression models in real scenarios are specified by humans or associated with other perceptual data. Our proposed framework using the Probabilistic Movement Primitives (ProMPs) modeling to resolve the shortcomings of the previous research works; the coupling between stiffness and motion is inherently established in a single model. Such a framework can request a small amount of incomplete observation data to infer the entire skill primitive. It can be used as an intuitive generalization command sending tool to achieve collaboration between humans and robots with human-like stiffness modulation strategies on either side. Experiments (human–robot hand-over, object matching, pick-and-place) were conducted to prove the effectiveness of the work. Myo armband and Leap motion camera are used as surface electromyography (sEMG) signal and motion capture sensors respective in the experiments. Also, the experiments show that the proposed framework strengthened the ability to distinguish actions with similar movements under observation noise by introducing the sEMG signal into the ProMP model. The usage of the mixture model brings possibilities in achieving automation of multiple collaborative tasks.
Collapse
Affiliation(s)
- Yuan Guan
- Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom
| | - Ning Wang
- Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom
| | - Chenguang Yang
- Bristol Robotics Laboratory, University of the West of England, Bristol, United Kingdom
| |
Collapse
|
21
|
Tanneberg D, Ploeger K, Rueckert E, Peters J. SKID RAW: Skill Discovery From Raw Trajectories. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3068891] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
22
|
Vianello L, Mouret JB, Dalin E, Aubry A, Ivaldi S. Human Posture Prediction During Physical Human-Robot Interaction. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3086666] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
23
|
Jamsek M, Kunavar T, Bobek U, Rueckert E, Babic J. Predictive Exoskeleton Control for Arm-Motion Augmentation Based on Probabilistic Movement Primitives Combined With a Flow Controller. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3068892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
24
|
Ashith Shyam RB, Hao Z, Montanaro U, Dixit S, Rathinam A, Gao Y, Neumann G, Fallah S. Autonomous Robots for Space: Trajectory Learning and Adaptation Using Imitation. Front Robot AI 2021; 8:638849. [PMID: 34017860 PMCID: PMC8130759 DOI: 10.3389/frobt.2021.638849] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Accepted: 04/16/2021] [Indexed: 11/14/2022] Open
Abstract
This paper adds on to the on-going efforts to provide more autonomy to space robots and introduces the concept of programming by demonstration or imitation learning for trajectory planning of manipulators on free-floating spacecraft. A redundant 7-DoF robotic arm is mounted on small spacecraft dedicated for debris removal, on-orbit servicing and assembly, autonomous and rendezvous docking. The motion of robot (or manipulator) arm induces reaction forces on the spacecraft and hence its attitude changes prompting the Attitude Determination and Control System (ADCS) to take large corrective action. The method introduced here is capable of finding the trajectory that minimizes the attitudinal changes thereby reducing the load on ADCS. One of the critical elements in spacecraft trajectory planning and control is the power consumption. The approach introduced in this work carry out trajectory learning offline by collecting data from demonstrations and encoding it as a probabilistic distribution of trajectories. The learned trajectory distribution can be used for planning in previously unseen situations by conditioning the probabilistic distribution. Hence almost no power is required for computations after deployment. Sampling from a conditioned distribution provides several possible trajectories from the same start to goal state. To determine the trajectory that minimizes attitudinal changes, a cost term is defined and the trajectory which minimizes this cost is considered the optimal one.
Collapse
Affiliation(s)
- R B Ashith Shyam
- Department of Electrical and Electronic Engineering, Surrey Space Center, University of Surrey, Guildford, United Kingdom
| | - Zhou Hao
- Department of Electrical and Electronic Engineering, Surrey Space Center, University of Surrey, Guildford, United Kingdom
| | - Umberto Montanaro
- Department of Mechanical Engineering, University of Surrey, Guildford, United Kingdom
| | - Shilp Dixit
- Department of Mechanical Engineering, University of Surrey, Guildford, United Kingdom
| | - Arunkumar Rathinam
- Department of Electrical and Electronic Engineering, Surrey Space Center, University of Surrey, Guildford, United Kingdom
| | - Yang Gao
- Department of Electrical and Electronic Engineering, Surrey Space Center, University of Surrey, Guildford, United Kingdom
| | | | - Saber Fallah
- Department of Electrical and Electronic Engineering, Surrey Space Center, University of Surrey, Guildford, United Kingdom.,Department of Mechanical Engineering, University of Surrey, Guildford, United Kingdom
| |
Collapse
|
25
|
Kulak T, Girgin H, Odobez JM, Calinon S. Active Learning of Bayesian Probabilistic Movement Primitives. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3060414] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
26
|
Arduengo M, Colome A, Borras J, Sentis L, Torras C. Task-Adaptive Robot Learning From Demonstration With Gaussian Process Models Under Replication. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3056367] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
27
|
Jankowski J, Girgin H, Calinon S. Probabilistic Adaptive Control for Robust Behavior Imitation. IEEE Robot Autom Lett 2021. [DOI: 10.1109/lra.2021.3061310] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
28
|
Silva N, Zhang D, Kulvicius T, Gail A, Barreiros C, Lindstaedt S, Kraft M, Bölte S, Poustka L, Nielsen-Saines K, Wörgötter F, Einspieler C, Marschik PB. The future of General Movement Assessment: The role of computer vision and machine learning - A scoping review. RESEARCH IN DEVELOPMENTAL DISABILITIES 2021; 110:103854. [PMID: 33571849 PMCID: PMC7910279 DOI: 10.1016/j.ridd.2021.103854] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Revised: 01/03/2021] [Accepted: 01/05/2021] [Indexed: 05/03/2023]
Abstract
BACKGROUND The clinical and scientific value of Prechtl general movement assessment (GMA) has been increasingly recognised, which has extended beyond the detection of cerebral palsy throughout the years. With advancing computer science, a surging interest in developing automated GMA emerges. AIMS In this scoping review, we focused on video-based approaches, since it remains authentic to the non-intrusive principle of the classic GMA. Specifically, we aimed to provide an overview of recent video-based approaches targeting GMs; identify their techniques for movement detection and classification; examine if the technological solutions conform to the fundamental concepts of GMA; and discuss the challenges of developing automated GMA. METHODS AND PROCEDURES We performed a systematic search for computer vision-based studies on GMs. OUTCOMES AND RESULTS We identified 40 peer-reviewed articles, most (n = 30) were published between 2017 and 2020. A wide variety of sensing, tracking, detection, and classification tools for computer vision-based GMA were found. Only a small portion of these studies applied deep learning approaches. A comprehensive comparison between data acquisition and sensing setups across the reviewed studies, highlighting limitations and advantages of each modality in performing automated GMA is provided. CONCLUSIONS AND IMPLICATIONS A "method-of-choice" for automated GMA does not exist. Besides creating large datasets, understanding the fundamental concepts and prerequisites of GMA is necessary for developing automated solutions. Future research shall look beyond the narrow field of detecting cerebral palsy and open up to the full potential of applying GMA to enable an even broader application.
Collapse
Affiliation(s)
- Nelson Silva
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Know-Center GmbH, Graz, Austria
| | - Dajie Zhang
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany
| | - Tomas Kulvicius
- Department for Computational Neuroscience, Third Institute of Physics-Biophysics, Georg-August-University of Göttingen, Göttingen, Germany
| | - Alexander Gail
- Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; German Primate Center - Leibniz Institute for Primate Research, Göttingen, Germany
| | - Carla Barreiros
- Know-Center GmbH, Graz, Austria; Institute of Interactive Systems and Data Science, Graz University of Technology, Graz, Austria
| | - Stefanie Lindstaedt
- Know-Center GmbH, Graz, Austria; Institute of Interactive Systems and Data Science, Graz University of Technology, Graz, Austria
| | - Marc Kraft
- Department of Medical Engineering, Technical University Berlin, Berlin, Germany
| | - Sven Bölte
- Center of Neurodevelopmental Disorders (KIND), Centre for Psychiatry Research, Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden; Child and Adolescent Psychiatry, Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden; Curtin Autism Research Group, School of Occupational Therapy, Social Work and Speech Pathology, Curtin University, Perth, Western Australia, Australia
| | - Luise Poustka
- Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany
| | - Karin Nielsen-Saines
- Division of Pediatric Infectious Diseases, David Geffen UCLA School of Medicine, USA
| | - Florentin Wörgötter
- Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; Department for Computational Neuroscience, Third Institute of Physics-Biophysics, Georg-August-University of Göttingen, Göttingen, Germany; Institute of Physics, Department for Computational Neuroscience at the Bernstein Center Göttingen, Georg-August-University of Göttingen, Göttingen, Germany
| | - Christa Einspieler
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria
| | - Peter B Marschik
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; Center of Neurodevelopmental Disorders (KIND), Centre for Psychiatry Research, Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
29
|
Callens T, van der Have T, Rossom SV, De Schutter J, Aertbelien E. A Framework for Recognition and Prediction of Human Motions in Human-Robot Collaboration Using Probabilistic Motion Models. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.3005892] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
30
|
Ding G, Liu Y, Zang X, Zhang X, Liu G, Zhao J. A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations. SENSORS (BASEL, SWITZERLAND) 2020; 20:E5505. [PMID: 32992888 PMCID: PMC7583967 DOI: 10.3390/s20195505] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/21/2020] [Revised: 09/16/2020] [Accepted: 09/23/2020] [Indexed: 11/16/2022]
Abstract
In manufacturing, traditional task pre-programming methods limit the efficiency of human-robot skill transfer. This paper proposes a novel task-learning strategy, enabling robots to learn skills from human demonstrations flexibly and generalize skills under new task situations. Specifically, we establish a markerless vision capture system to acquire continuous human hand movements and develop a threshold-based heuristic segmentation algorithm to segment the complete movements into different movement primitives (MPs) which encode human hand movements with task-oriented models. For movement primitive learning, we adopt a Gaussian mixture model and Gaussian mixture regression (GMM-GMR) to extract the optimal trajectory encapsulating sufficient human features and utilize dynamical movement primitives (DMPs) to learn for trajectory generalization. In addition, we propose an improved visuo-spatial skill learning (VSL) algorithm to learn goal configurations concerning spatial relationships between task-relevant objects. Only one multioperation demonstration is required for learning, and robots can generalize goal configurations under new task situations following the task execution order from demonstration. A series of peg-in-hole experiments demonstrate that the proposed task-learning strategy can obtain exact pick-and-place points and generate smooth human-like trajectories, verifying the effectiveness of the proposed strategy.
Collapse
Affiliation(s)
| | | | - Xizhe Zang
- State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin 150001, China; (G.D.); (Y.L.); (X.Z.); (G.L.); (J.Z.)
| | | | | | | |
Collapse
|
31
|
Fabisch A, Langosz M, Kirchner F. BOLeRo: Behavior optimization and learning for robots. INT J ADV ROBOT SYST 2020. [DOI: 10.1177/1729881420913741] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Reinforcement learning and behavior optimization are becoming more and more popular in the field of robotics because algorithms are mature enough to tackle real problems in this domain. Robust implementations of state-of-the-art algorithms are often not publicly available though, and experiments are hardly reproducible because open-source implementations are often not available or are still in a stage of research code. Consequently, often it is infeasible to deploy these algorithms on robotic systems. BOLeRo closes this gap for policy search and evolutionary algorithms by delivering open-source implementations of behavior learning algorithms for robots. It is easy to integrate in robotic middlewares and it can be used to compare methods and develop prototypes in simulation.
Collapse
Affiliation(s)
- Alexander Fabisch
- Robotics Innovation Center, DFKI GmbH, Bremen, Germany
- Robotics Research Group, University of Bremen, Bremen, Germany
| | - Malte Langosz
- Robotics Innovation Center, DFKI GmbH, Bremen, Germany
| | - Frank Kirchner
- Robotics Innovation Center, DFKI GmbH, Bremen, Germany
- Robotics Research Group, University of Bremen, Bremen, Germany
| |
Collapse
|
32
|
Lockel S, Peters J, van Vliet P. A Probabilistic Framework for Imitating Human Race Driver Behavior. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2970620] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
33
|
Motokura K, Takahashi M, Ewerton M, Peters J. Plucking Motions for Tea Harvesting Robots Using Probabilistic Movement Primitives. IEEE Robot Autom Lett 2020. [DOI: 10.1109/lra.2020.2976314] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
34
|
Koert D, Trick S, Ewerton M, Lutter M, Peters J. Incremental Learning of an Open-Ended Collaborative Skill Library. INT J HUM ROBOT 2020. [DOI: 10.1142/s0219843620500012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Intelligent assistive robots can potentially contribute to maintaining an elderly person’s independence by supporting everyday life activities. However, the number of different and personalized activities to be supported renders pre-programming of all respective robot behaviors prohibitively difficult. Instead, to cope with a continuous and potentially open-ended stream of cooperative tasks, new collaborative robot behaviors need to be continuously learned and updated from demonstrations. To this end, we introduce an online learning method to incrementally build a cooperative skill library of probabilistic interaction primitives. The resulting model chooses a corresponding robot response to a human movement where the human intention is extracted from previously demonstrated movements. While existing batch learning methods for movement primitives usually learn such skill libraries only once for a pre-defined number of different skills, our approach enables extending the skill library in an open-ended and online fashion from new incoming demonstrations. The proposed approach is evaluated on a low-dimensional benchmark task and in a collaborative scenario with a 7DoF robot, where we also investigate the generalization of learned skills between different subjects.
Collapse
Affiliation(s)
- Dorothea Koert
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Susanne Trick
- Psychology of Information Processing Group, Department of Psychology, Technische Universität Darmstadt, Darmstadt, Germany
| | - Marco Ewerton
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Michael Lutter
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Jan Peters
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
- Max Planck Institute for Intelligent Systems, Tübingen, Germany
| |
Collapse
|
35
|
Lioutikov R, Maeda G, Veiga F, Kersting K, Peters J. Learning attribute grammars for movement primitive sequencing. Int J Rob Res 2019. [DOI: 10.1177/0278364919868279] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Movement primitives are a well studied and widely applied concept in modern robotics. However, composing primitives out of an existing library has shown to be a challenging problem. We propose the use of probabilistic context-free grammars to sequence a series of primitives to generate complex robot policies from a given library of primitives. The rule-based nature of formal grammars allows an intuitive encoding of hierarchically structured tasks. This hierarchical concept strongly connects with the way robot policies can be learned, organized, and re-used. However, the induction of context-free grammars has proven to be a complicated and yet unsolved challenge. We exploit the physical nature of robot movement primitives to restrict and efficiently search the grammar space. The grammar is learned by applying a Markov chain Monte Carlo optimization over the posteriors of the grammars given the observations. The proposal distribution is defined as a mixture over the probabilities of the operators connecting the search space. Moreover, we present an approach for the categorization of probabilistic movement primitives and discuss how the connectibility of two primitives can be determined. These characteristics in combination with restrictions to the operators guarantee continuous sequences while reducing the grammar space. In addition, a set of attributes and conditions is introduced that augments probabilistic context-free grammars in order to solve primitive sequencing tasks with the capability to adapt single primitives within the sequence. The method was validated on tasks that require the generation of complex sequences consisting of simple movement primitives using a seven-degree-of-freedom lightweight robotic arm.
Collapse
Affiliation(s)
| | | | - Filipe Veiga
- MIT Computer Science & Artificial Intelligence Lab, Cambridge, MA, USA
| | - Kristian Kersting
- CS Department and Centre for Cognitive Science, TU Darmstadt, Germany
| | - Jan Peters
- Intelligent Autonomous Systems, TU Darmstadt, Germany
- Max Planck Institute for Intelligent Systems, Tübingen, Germany
| |
Collapse
|
36
|
Koert D, Pajarinen J, Schotschneider A, Trick S, Rothkopf C, Peters J. Learning Intention Aware Online Adaptation of Movement Primitives. IEEE Robot Autom Lett 2019. [DOI: 10.1109/lra.2019.2928760] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
37
|
Ewerton M, Arenz O, Maeda G, Koert D, Kolev Z, Takahashi M, Peters J. Learning Trajectory Distributions for Assisted Teleoperation and Path Planning. Front Robot AI 2019; 6:89. [PMID: 33501104 PMCID: PMC7806068 DOI: 10.3389/frobt.2019.00089] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2019] [Accepted: 09/03/2019] [Indexed: 11/13/2022] Open
Abstract
Several approaches have been proposed to assist humans in co-manipulation and teleoperation tasks given demonstrated trajectories. However, these approaches are not applicable when the demonstrations are suboptimal or when the generalization capabilities of the learned models cannot cope with the changes in the environment. Nevertheless, in real co-manipulation and teleoperation tasks, the original demonstrations will often be suboptimal and a learning system must be able to cope with new situations. This paper presents a reinforcement learning algorithm that can be applied to such problems. The proposed algorithm is initialized with a probability distribution of demonstrated trajectories and is based on the concept of relevance functions. We show in this paper how the relevance of trajectory parameters to optimization objectives is connected with the concept of Pearson correlation. First, we demonstrate the efficacy of our algorithm by addressing the assisted teleoperation of an object in a static virtual environment. Afterward, we extend this algorithm to deal with dynamic environments by utilizing Gaussian Process regression. The full framework is applied to make a point particle and a 7-DoF robot arm autonomously adapt their movements to changes in the environment as well as to assist the teleoperation of a 7-DoF robot arm in a dynamic environment.
Collapse
Affiliation(s)
- Marco Ewerton
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany.,Perception and Activity Understanding Group/Robot Learning and Interaction Group, Idiap Research Institute, Martigny, Switzerland
| | - Oleg Arenz
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Guilherme Maeda
- ATR Computational Neuroscience Laboratory, Department of Brain Robot Interface, Kyoto, Japan.,Preferred Networks, Inc., Tokyo, Japan
| | - Dorothea Koert
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Zlatko Kolev
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany
| | - Masaki Takahashi
- Takahashi Laboratory, Department of System Design Engineering, Faculty of Science and Technology, Keio University, Yokohama, Japan
| | - Jan Peters
- Intelligent Autonomous Systems Group, Department of Computer Science, Technische Universität Darmstadt, Darmstadt, Germany.,Robot Learning Group, Max Planck Institute for Intelligent Systems, Tübingen, Germany
| |
Collapse
|
38
|
Maestre C, Mukhtar G, Gonzales C, Doncieux S. Action Generation Adapted to Low-Level and High-Level Robot-Object Interaction States. Front Neurorobot 2019; 13:56. [PMID: 31396071 PMCID: PMC6668554 DOI: 10.3389/fnbot.2019.00056] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2018] [Accepted: 07/09/2019] [Indexed: 11/20/2022] Open
Abstract
Our daily environments are complex, composed of objects with different features. These features can be categorized into low-level features, e.g., an object position or temperature, and high-level features resulting from a pre-processing of low-level features for decision purposes, e.g., a binary value saying if it is too hot to be grasped. Besides, our environments are dynamic, i.e., object states can change at any moment. Therefore, robots performing tasks in these environments must have the capacity to (i) identify the next action to execute based on the available low-level and high-level object states, and (ii) dynamically adapt their actions to state changes. We introduce a method named Interaction State-based Skill Learning (IS2L), which builds skills to solve tasks in realistic environments. A skill is a Bayesian Network that infers actions composed of a sequence of movements of the robot's end-effector, which locally adapt to spatio-temporal perturbations using a dynamical system. In the current paper, an external agent performs one or more kinesthetic demonstrations of an action generating a dataset of high-level and low-level states of the robot and the environment objects. First, the method transforms each interaction to represent (i) the relationship between the robot and the object and (ii) the next robot end-effector movement to perform at consecutive instants of time. Then, the skill is built, i.e., the Bayesian network is learned. While generating an action this skill relies on the robot and object states to infer the next movement to execute. This movement selection gets inspired by a type of predictive models for action selection usually called affordances. The main contribution of this paper is combining the main features of dynamical systems and affordances in a unique method to build skills that solve tasks in realistic scenarios. More precisely, combining the low-level movement generation of the dynamical systems, to adapt to local perturbations, with the next movement selection simultaneously based on high-level and low-level states. This contribution was assessed in three experiments in realistic environments using both high-level and low-level states. The built skills solved the respective tasks relying on both types of states, and adapting to external perturbations.
Collapse
Affiliation(s)
- Carlos Maestre
- UMR 7222, ISIR, Sorbonne Université and CNRS, Paris, France
| | - Ghanim Mukhtar
- UMR 7222, ISIR, Sorbonne Université and CNRS, Paris, France
| | | | | |
Collapse
|
39
|
Abstract
Imitation learning has been studied widely as a convenient way to transfer human skills to robots. This learning approach is aimed at extracting relevant motion patterns from human demonstrations and subsequently applying these patterns to different situations. Despite the many advancements that have been achieved, solutions for coping with unpredicted situations (e.g., obstacles and external perturbations) and high-dimensional inputs are still largely absent. In this paper, we propose a novel kernelized movement primitive (KMP), which allows the robot to adapt the learned motor skills and fulfill a variety of additional constraints arising over the course of a task. Specifically, KMP is capable of learning trajectories associated with high-dimensional inputs owing to the kernel treatment, which in turn renders a model with fewer open parameters in contrast to methods that rely on basis functions. Moreover, we extend our approach by exploiting local trajectory representations in different coordinate systems that describe the task at hand, endowing KMP with reliable extrapolation capabilities in broader domains. We apply KMP to the learning of time-driven trajectories as a special case, where a compact parametric representation describing a trajectory and its first-order derivative is utilized. In order to verify the effectiveness of our method, several examples of trajectory modulations and extrapolations associated with time inputs, as well as trajectory adaptations with high-dimensional inputs are provided.
Collapse
Affiliation(s)
- Yanlong Huang
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy
| | - Leonel Rozo
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy
| | - João Silvério
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy
| | - Darwin G Caldwell
- Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy
| |
Collapse
|