1
|
Learning Non-Parametric Models in Real Time via Online Generalized Product of Experts. IEEE Robot Autom Lett 2022. [DOI: 10.1109/lra.2022.3190809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
2
|
Tanwani AK, Yan A, Lee J, Calinon S, Goldberg K. Sequential robot imitation learning from observations. Int J Rob Res 2021. [DOI: 10.1177/02783649211032721] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
This paper presents a framework to learn the sequential structure in the demonstrations for robot imitation learning. We first present a family of task-parameterized hidden semi-Markov models that extracts invariant segments (also called sub-goals or options) from demonstrated trajectories, and optimally follows the sampled sequence of states from the model with a linear quadratic tracking controller. We then extend the concept to learning invariant segments from visual observations that are sequenced together for robot imitation. We present Motion2Vec that learns a deep embedding space by minimizing a metric learning loss in a Siamese network: images from the same action segment are pulled together while being pushed away from randomly sampled images of other segments, and a time contrastive loss is used to preserve the temporal ordering of the images. The trained embeddings are segmented with a recurrent neural network, and subsequently used for decoding the end-effector pose of the robot. We first show its application to a pick-and-place task with the Baxter robot while avoiding a moving obstacle from four kinesthetic demonstrations only, followed by suturing task imitation from publicly available suturing videos of the JIGSAWS dataset with state-of-the-art [Formula: see text]% segmentation accuracy and [Formula: see text] cm error in position per observation on the test set.
Collapse
Affiliation(s)
| | - Andy Yan
- University of California, Berkeley, CA, USA
| | | | | | | |
Collapse
|
3
|
Luo J, He W, Yang C. Combined perception, control, and learning for teleoperation: key technologies, applications, and challenges. COGNITIVE COMPUTATION AND SYSTEMS 2020. [DOI: 10.1049/ccs.2020.0005] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/15/2023] Open
Affiliation(s)
- Jing Luo
- Key Laboratory of Autonomous Systems and Networked ControlSchool of Automation Science and EngineeringSouth China University of TechnologyGuangzhou510640People's Republic of China
| | - Wei He
- School of Automation and Electrical EngineeringUniversity of Science and Technology BeijingBeijing100083People's Republic of China
| | - Chenguang Yang
- Key Laboratory of Autonomous Systems and Networked ControlSchool of Automation Science and EngineeringSouth China University of TechnologyGuangzhou510640People's Republic of China
| |
Collapse
|