1
|
Osorio P, Sagawa R, Abe N, Venture G. A Generative Model to Embed Human Expressivity into Robot Motions. SENSORS (BASEL, SWITZERLAND) 2024; 24:569. [PMID: 38257661 PMCID: PMC10819644 DOI: 10.3390/s24020569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 12/31/2023] [Accepted: 01/15/2024] [Indexed: 01/24/2024]
Abstract
This paper presents a model for generating expressive robot motions based on human expressive movements. The proposed data-driven approach combines variational autoencoders and a generative adversarial network framework to extract the essential features of human expressive motion and generate expressive robot motion accordingly. The primary objective was to transfer the underlying expressive features from human to robot motion. The input to the model consists of the robot task defined by the robot's linear velocities and angular velocities and the expressive data defined by the movement of a human body part, represented by the acceleration and angular velocity. The experimental results show that the model can effectively recognize and transfer expressive cues to the robot, producing new movements that incorporate the expressive qualities derived from the human input. Furthermore, the generated motions exhibited variability with different human inputs, highlighting the ability of the model to produce diverse outputs.
Collapse
Affiliation(s)
- Pablo Osorio
- Department of Mechanical Systems Engineering, Faculty of Engineering, Tokyo University of Agriculture and Technology, Koganei Campus, Tokyo 184-8588, Japan;
- CNRS-AIST JRL (Joint Robotics Laboratory) IRL, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan;
| | - Ryusuke Sagawa
- Department of Mechanical Systems Engineering, Faculty of Engineering, Tokyo University of Agriculture and Technology, Koganei Campus, Tokyo 184-8588, Japan;
- CNRS-AIST JRL (Joint Robotics Laboratory) IRL, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan;
| | - Naoko Abe
- Naver Labs Europe, 38240 Meylan, France;
| | - Gentiane Venture
- CNRS-AIST JRL (Joint Robotics Laboratory) IRL, National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba 305-8560, Japan;
- Department of Mechanical Engineering, Graduate School of Engineering, The University of Tokyo, Hongo Campus, Tokyo 113-8654, Japan
| |
Collapse
|
2
|
Korcsok B, Korondi P. How do you do the things that you do? Ethological approach to the description of robot behaviour. Biol Futur 2023; 74:253-279. [PMID: 37812380 DOI: 10.1007/s42977-023-00178-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2023] [Accepted: 09/08/2023] [Indexed: 10/10/2023]
Abstract
The detailed description of behaviour of the interacting parties is becoming more and more important in human-robot interaction (HRI), especially in social robotics (SR). With the rise in the number of publications, there is a substantial need for the objective and comprehensive description of implemented robot behaviours to ensure comparability and reproducibility of the studies. Ethograms and the meticulous analysis of behaviour was introduced long ago in animal behaviour research (cf. ethology). The adoption of this method in SR and HRI can ensure the desired clarity over robot behaviours, while also providing added benefits during robot development, behaviour modelling and analysis of HRI experiments. We provide an overview of the possible uses and advantages of ethograms in HRI, and propose a general framework for describing behaviour which can be adapted to the requirements of specific studies.
Collapse
Affiliation(s)
- Beáta Korcsok
- ELKH-ELTE Comparative Ethology Research Group, Budapest, Hungary.
- Department of Mechatronics, Optics and Mechanical Engineering Informatics, Faculty of Mechanical Engineering, Budapest University of Technology and Economics, Budapest, Hungary.
| | - Péter Korondi
- Department of Mechatronics, Faculty of Engineering, University of Debrecen, Debrecen, Hungary
| |
Collapse
|
3
|
Feng H, Zeng Y, Lu E. Brain-Inspired Affective Empathy Computational Model and Its Application on Altruistic Rescue Task. Front Comput Neurosci 2022; 16:784967. [PMID: 35923916 PMCID: PMC9341284 DOI: 10.3389/fncom.2022.784967] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2022] [Accepted: 06/22/2022] [Indexed: 11/23/2022] Open
Abstract
Affective empathy is an indispensable ability for humans and other species' harmonious social lives, motivating altruistic behavior, such as consolation and aid-giving. How to build an affective empathy computational model has attracted extensive attention in recent years. Most affective empathy models focus on the recognition and simulation of facial expressions or emotional speech of humans, namely Affective Computing. However, these studies lack the guidance of neural mechanisms of affective empathy. From a neuroscience perspective, affective empathy is formed gradually during the individual development process: experiencing own emotion—forming the corresponding Mirror Neuron System (MNS)—understanding the emotions of others through the mirror mechanism. Inspired by this neural mechanism, we constructed a brain-inspired affective empathy computational model, this model contains two submodels: (1) We designed an Artificial Pain Model inspired by the Free Energy Principle (FEP) to the simulate pain generation process in living organisms. (2) We build an affective empathy spiking neural network (AE-SNN) that simulates the mirror mechanism of MNS and has self-other differentiation ability. We apply the brain-inspired affective empathy computational model to the pain empathy and altruistic rescue task to achieve the rescue of companions by intelligent agents. To the best of our knowledge, our study is the first one to reproduce the emergence process of mirror neurons and anti-mirror neurons in the SNN field. Compared with traditional affective empathy computational models, our model is more biologically plausible, and it provides a new perspective for achieving artificial affective empathy, which has special potential for the social robots field in the future.
Collapse
Affiliation(s)
- Hui Feng
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
| | - Yi Zeng
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
- Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China
- National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China
- *Correspondence: Yi Zeng
| | - Enmeng Lu
- Research Center for Brain-Inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China
| |
Collapse
|
4
|
Matsumaru T. Methods of Generating Emotional Movements and Methods of Transmitting Behavioral Intentions: A Perspective on Human-Coexistence Robots. SENSORS (BASEL, SWITZERLAND) 2022; 22:4587. [PMID: 35746365 PMCID: PMC9227009 DOI: 10.3390/s22124587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/25/2022] [Revised: 06/14/2022] [Accepted: 06/16/2022] [Indexed: 06/15/2023]
Abstract
The purpose of this paper is to introduce and discuss the following two functions that are considered to be important in human-coexistence robots and human-symbiotic robots: the method of generating emotional movements, and the method of transmitting behavioral intentions. The generation of emotional movements is to design the bodily movements of robots so that humans can feel specific emotions. Specifically, the application of Laban movement analysis, the development from the circumplex model of affect, and the imitation of human movements are discussed. However, a general technique has not yet been established to modify any robot movement so that it contains a specific emotion. The transmission of behavioral intentions is about allowing the surrounding humans to understand the behavioral intentions of robots. Specifically, informative motions in arm manipulation and the transmission of the movement intentions of robots are discussed. In the former, the target position in the reaching motion, the physical characteristics in the handover motion, and the landing distance in the throwing motion are examined, but there are still few research cases. In the latter, no groundbreaking method has been proposed that is fundamentally different from earlier studies. Further research and development are expected in the near future.
Collapse
Affiliation(s)
- Takafumi Matsumaru
- Graduate School of Information, Production and Systems (IPS), Waseda University, Kitakyushu 808-0135, Japan
| |
Collapse
|
5
|
Hieida C, Nagai T. Survey and perspective on social emotions in robotics. Adv Robot 2022. [DOI: 10.1080/01691864.2021.2012512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Affiliation(s)
- Chie Hieida
- Division of Information Science, Graduate School of Science and Technology, Nara Institute of Science and Technology, Ikoma, Japan
| | - Takayuki Nagai
- Department of Systems Innovation, Graduate School of Engineering Science, Osaka University, Suita, Japan
- The University of Electro-Communications, Chofu, Japan
| |
Collapse
|
6
|
Savery R, Weinberg G. Robots and emotion: a survey of trends, classifications, and forms of interaction. Adv Robot 2021. [DOI: 10.1080/01691864.2021.1957014] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Richard Savery
- Georgia Tech Center for Music Technology, Georgia Institute of Technology, Atlanta, GA, USA
| | - Gil Weinberg
- Georgia Tech Center for Music Technology, Georgia Institute of Technology, Atlanta, GA, USA
| |
Collapse
|
7
|
Stock-Homburg R. Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research. Int J Soc Robot 2021. [DOI: 10.1007/s12369-021-00778-6] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
AbstractKnowledge production within the interdisciplinary field of human–robot interaction (HRI) with social robots has accelerated, despite the continued fragmentation of the research domain. Together, these features make it hard to remain at the forefront of research or assess the collective evidence pertaining to specific areas, such as the role of emotions in HRI. This systematic review of state-of-the-art research into humans’ recognition and responses to artificial emotions of social robots during HRI encompasses the years 2000–2020. In accordance with a stimulus–organism–response framework, the review advances robotic psychology by revealing current knowledge about (1) the generation of artificial robotic emotions (stimulus), (2) human recognition of robotic artificial emotions (organism), and (3) human responses to robotic emotions (response), as well as (4) other contingencies that affect emotions as moderators.
Collapse
|
8
|
Rezaeipanah A, Amiri P, Jafari S. Performing the Kick During Walking for RoboCup 3D Soccer Simulation League Using Reinforcement Learning Algorithm. Int J Soc Robot 2020. [DOI: 10.1007/s12369-020-00712-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
|
9
|
Abstract
Robots that have different forms and capabilities are used in a wide variety of situations; however, one common point to all robots interacting with humans is their ability to communicate with them. In addition to verbal communication or purely communicative movements, robots can also use their embodiment to generate expressive movements while achieving a task, to convey additional information to its human partner. This article surveys state-of-the-art techniques that generate whole-body expressive movements in robots and robot avatars. We consider different embodiments such as wheeled, legged, or flying systems and the different metrics used to evaluate the generated movements. Finally, we discuss future areas of improvement and the difficulties to overcome to develop truly expressive motions in artificial agents.
Collapse
Affiliation(s)
| | - Dana Kulić
- University of Waterloo and Monash University, VIC, Australia
| |
Collapse
|
10
|
Correlation Analysis for Predictive Models of Robot User’s Impression: A Study on Visual Medium and Mechanical Noise. Int J Soc Robot 2019. [DOI: 10.1007/s12369-019-00601-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
|