1
|
Xie H, Gao Z, Jia G, Shimoda S, Shi Q. Learning Rat-Like Behavioral Interaction Using a Small-Scale Robotic Rat. CYBORG AND BIONIC SYSTEMS 2023; 4:0032. [PMID: 37342211 PMCID: PMC10278959 DOI: 10.34133/cbsystems.0032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2023] [Accepted: 04/23/2023] [Indexed: 06/22/2023] Open
Abstract
In this paper, we propose a novel method for emulating rat-like behavioral interactions in robots using reinforcement learning. Specifically, we develop a state decision method to optimize the interaction process among 6 known behavior types that have been identified in previous research on rat interactions. The novelty of our method lies in using the temporal difference (TD) algorithm to optimize the state decision process, which enables the robots to make informed decisions about their behavior choices. To assess the similarity between robot and rat behavior, we use Pearson correlation. We then use TD-λ to update the state value function and make state decisions based on probability. The robots execute these decisions using our dynamics-based controller. Our results demonstrate that our method can generate rat-like behaviors on both short- and long-term timescales, with interaction information entropy comparable to that between real rats. Overall, our approach shows promise for controlling robots in robot-rat interactions and highlights the potential of using reinforcement learning to develop more sophisticated robotic systems.
Collapse
Affiliation(s)
- Hongzhao Xie
- Intelligent Robotics Institute, School of Mechatronical Engineering,
Beijing Institute of Technology, Beijing 100081, China
- Key Laboratory of Biomimetic Robots and Systems (Beijing Institute of Technology), Ministry of Education, Beijing 100081, China
| | - Zihang Gao
- Intelligent Robotics Institute, School of Mechatronical Engineering,
Beijing Institute of Technology, Beijing 100081, China
- Key Laboratory of Biomimetic Robots and Systems (Beijing Institute of Technology), Ministry of Education, Beijing 100081, China
| | - Guanglu Jia
- Intelligent Robotics Institute, School of Mechatronical Engineering,
Beijing Institute of Technology, Beijing 100081, China
- Key Laboratory of Biomimetic Robots and Systems (Beijing Institute of Technology), Ministry of Education, Beijing 100081, China
| | - Shingo Shimoda
- Nagoya University Graduate School of Medicine, Nagoya, Japan
| | - Qing Shi
- Intelligent Robotics Institute, School of Mechatronical Engineering,
Beijing Institute of Technology, Beijing 100081, China
- Key Laboratory of Biomimetic Robots and Systems (Beijing Institute of Technology), Ministry of Education, Beijing 100081, China
| |
Collapse
|
2
|
Zhou Z, Liu J, Pan J, Yu J. Proactivity of fish and leadership of self-propelled robotic fish during interaction. BIOINSPIRATION & BIOMIMETICS 2023; 18. [PMID: 37075759 DOI: 10.1088/1748-3190/acce87] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Accepted: 04/19/2023] [Indexed: 05/03/2023]
Abstract
Fish interacting with biomimetic robotic fish is beneficial for animal behavior research, particularly in the study of collective behavior. Compared with passive-dragging robotic fish, self-propelled robotic fish floats in water, and its movement matches the flow field formed by the caudal fin oscillation, leading to more realistic interaction with animals. In this paper, we propose a self-propelled koi-mimicking robotic fish entity, develop a system for robotic fish and koi fish interaction, and conduct extensive experiments on quantity variation and parameter variation. The results showed that fish exhibited significantly lower proactivity when alone, and the most proactive case is one robotic fish interacting with two real fish. The experiments on parameter variation indicated that fish may respond more proactivity to robotic fish that swim with high frequency and low amplitude, but may also move together with high-frequency and high-amplitude swimming robotic fish. These findings could provide insights into fish collective behavior, guide the design of further fish-robot interaction experiments, and suggest directions for future improvements in goal-oriented robotic fish platforms.
Collapse
Affiliation(s)
- Ziye Zhou
- State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University, Beijing 100871, People's Republic of China
| | - Jincun Liu
- College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, People's Republic of China
| | - Jie Pan
- State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University, Beijing 100871, People's Republic of China
| | - Junzhi Yu
- State Key Laboratory for Turbulence and Complex Systems, Department of Advanced Manufacturing and Robotics, College of Engineering, Peking University, Beijing 100871, People's Republic of China
- State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, People's Republic of China
| |
Collapse
|
3
|
Shi Q, Gao Z, Jia G, Li C, Huang Q, Ishii H, Takanishi A, Fukuda T. Implementing Rat-Like Motion for a Small-Sized Biomimetic Robot Based on Extraction of Key Movement Joints. IEEE T ROBOT 2021. [DOI: 10.1109/tro.2020.3033705] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
4
|
Abstract
In recent studies, robots are used to stimulate living systems in controlled experimental settings. This research strategy is here called interactive biorobotics, to distinguish it from classical biorobotics, in which robots are used to simulate, rather than to stimulate, living system behavior. This article offers a methodological analysis of interactive biorobotics and has two goals. The first one is to argue that interactive biorobotics is methodologically different, in some important respects, from classical biorobotics and from countless instances of model-based science. It will be shown that interactive biorobotics does not conform to the so-called "understanding by building" approach or synthetic method, and that it illustrates a novel use of models in science. The second goal is to reflect on the logic of interactive biorobotics. A distinction will be made between two classes of studies, which will be called "proximal" and "distal." In proximal studies, experiments involving robot-animal interaction are brought to bear on theoretical hypotheses on robot-animal interaction. In distal studies, experiments involving robot-animal interaction are brought to bear on theoretical hypotheses on animal-animal interaction. Distal studies involve logical steps which may be particularly hard to justify. This distinction, together with a methodological reflection on the relationship between the context in which the experiments are carried out and the context in which the conclusions are expected to hold, will lead to a checklist of questions which may be useful to justify and evaluate the validity of interactive biorobotics studies. The reconstruction of the logic of interactive biorobotics made here, though preliminary, may contribute to justifying the important role that robots, as tool for stimulating living systems, can play in the contemporary life sciences.
Collapse
Affiliation(s)
- Edoardo Datteri
- RobotiCSS Lab - Laboratory of Robotics for the Cognitive and Social Sciences, Department of Human Sciences for Education, University of Milano-Bicocca, Milan, Italy
| |
Collapse
|
5
|
Romano D, Donati E, Benelli G, Stefanini C. A review on animal-robot interaction: from bio-hybrid organisms to mixed societies. BIOLOGICAL CYBERNETICS 2019; 113:201-225. [PMID: 30430234 DOI: 10.1007/s00422-018-0787-5] [Citation(s) in RCA: 76] [Impact Index Per Article: 15.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/25/2017] [Accepted: 10/19/2018] [Indexed: 05/28/2023]
Abstract
Living organisms are far superior to state-of-the-art robots as they have evolved a wide number of capabilities that far encompass our most advanced technologies. The merging of biological and artificial world, both physically and cognitively, represents a new trend in robotics that provides promising prospects to revolutionize the paradigms of conventional bio-inspired design as well as biological research. In this review, a comprehensive definition of animal-robot interactive technologies is given. They can be at animal level, by augmenting physical or mental capabilities through an integrated technology, or at group level, in which real animals interact with robotic conspecifics. Furthermore, an overview of the current state of the art and the recent trends in this novel context is provided. Bio-hybrid organisms represent a promising research area allowing us to understand how a biological apparatus (e.g. muscular and/or neural) works, thanks to the interaction with the integrated technologies. Furthermore, by using artificial agents, it is possible to shed light on social behaviours characterizing mixed societies. The robots can be used to manipulate groups of living organisms to understand self-organization and the evolution of cooperative behaviour and communication.
Collapse
Affiliation(s)
- Donato Romano
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy.
| | - Elisa Donati
- The Institute of Neuroinformatics, University of Zurich/ETH, Winterthurerstrasse 190, 8057, Zurich, Switzerland
| | - Giovanni Benelli
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy
- Department of Agriculture, Food and Environment, University of Pisa, Via del Borghetto 80, 56124, Pisa, Italy
| | - Cesare Stefanini
- The BioRobotics Institute, Sant'Anna School of Advanced Studies, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy
- HEIC Center, BME Department, Khalifa University, PO Box 127788, Abu Dhabi, UAE
| |
Collapse
|
6
|
Bonnet F, Mills R, Szopek M, Schönwetter-Fuchs S, Halloy J, Bogdan S, Correia L, Mondada F, Schmickl T. Robots mediating interactions between animals for interspecies collective behaviors. Sci Robot 2019; 4:4/28/eaau7897. [PMID: 33137747 DOI: 10.1126/scirobotics.aau7897] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2018] [Accepted: 02/06/2019] [Indexed: 12/30/2022]
Abstract
Self-organized collective behavior has been analyzed in diverse types of gregarious animals. Such collective intelligence emerges from the synergy between individuals, which behave at their own time and spatial scales and without global rules. Recently, robots have been developed to collaborate with animal groups in the pursuit of better understanding their decision-making processes. These biohybrid systems make cooperative relationships between artificial systems and animals possible, which can yield new capabilities in the resulting mixed group. However, robots are currently tailor-made to successfully engage with one animal species at a time. This limits the possibilities of introducing distinct species-dependent perceptual capabilities and types of behaviors in the same system. Here, we show that robots socially integrated into animal groups of honeybees and zebrafish, each one located in a different city, allowing these two species to interact. This interspecific information transfer is demonstrated by collective decisions that emerge between the two autonomous robotic systems and the two animal groups. The robots enable this biohybrid system to function at any distance and operates in water and air with multiple sensorimotor properties across species barriers and ecosystems. These results demonstrate the feasibility of generating and controlling behavioral patterns in biohybrid groups of multiple species. Such interspecies connections between diverse robotic systems and animal species may open the door for new forms of artificial collective intelligence, where the unrivaled perceptual capabilities of the animals and their brains can be used to enhance autonomous decision-making, which could find applications in selective "rewiring" of ecosystems.
Collapse
Affiliation(s)
- Frank Bonnet
- Robotic Systems Laboratory, École Polytechnique Fédérale de Lausanne, EPFL STI IMT LSRO, ME B3 30 (Bâtiment ME), Station 9 1015 Lausanne, Switzerland.
| | - Rob Mills
- BioISI, Faculdade de Ciências da Universidade de Lisboa, 1749-016 Lisbon, Portugal
| | - Martina Szopek
- Artificial Life Laboratory of the Institute of Biology, Karl-Franzens University Graz, Universitätsplatz 2, 8010 Graz, Austria
| | - Sarah Schönwetter-Fuchs
- Artificial Life Laboratory of the Institute of Biology, Karl-Franzens University Graz, Universitätsplatz 2, 8010 Graz, Austria
| | - José Halloy
- Univ Paris Diderot, Sorbonne Paris Cité, LIED UMR 8236, 75013 Paris, France
| | - Stjepan Bogdan
- Laboratory for Robotics and Intelligent Control Systems, Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia
| | - Luís Correia
- BioISI, Faculdade de Ciências da Universidade de Lisboa, 1749-016 Lisbon, Portugal
| | - Francesco Mondada
- Robotic Systems Laboratory, École Polytechnique Fédérale de Lausanne, EPFL STI IMT LSRO, ME B3 30 (Bâtiment ME), Station 9 1015 Lausanne, Switzerland
| | - Thomas Schmickl
- Artificial Life Laboratory of the Institute of Biology, Karl-Franzens University Graz, Universitätsplatz 2, 8010 Graz, Austria
| |
Collapse
|
7
|
Gao Z, Shi Q, Fukuda T, Li C, Huang Q. An overview of biomimetic robots with animal behaviors. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2018.12.071] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
8
|
PiRat: An autonomous framework for studying social behaviour in rats and robots. PROCEEDINGS OF THE ... IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS. IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS 2018; 2018:7601-7608. [PMID: 34621592 DOI: 10.1109/iros.2018.8594060] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
The use of robots, as a social stimulus, provides several advantages over using another animal. In particular, for rat-robot studies, robots can produce social behaviour that is reproducible across trials. In the current work, we outline a framework for rat-robot interaction studies, that consists of a novel rat-sized robot (PiRat), models of robotic behavior, and a position tracking system for both robot and rat. We present the design of the framework, including constraints on autonomy, latency, and control. We pilot tested our framework by individually running the robot rat with eight different rats, first through a habituation stage, and then with PiRat performing two different types of behaviour - avoiding and frequently approaching. We evaluate the performance of the framework on latency and autonomy, and on the ability to influence the behaviour of individual rats. We find that the framework performs well on its constraints, engages some of the rats (according to the number of meetings), and features a control scheme that produces reproducible behaviour in rats. These features represent a first demonstration of a closed-loop rat-robot framework.
Collapse
|
9
|
|
10
|
Spatio-temporal organization during group formation in rats. Anim Cogn 2018; 21:513-529. [DOI: 10.1007/s10071-018-1185-z] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2017] [Revised: 04/08/2018] [Accepted: 04/25/2018] [Indexed: 01/29/2023]
|
11
|
Hybrid Locomotion Evaluation for a Novel Amphibious Spherical Robot. APPLIED SCIENCES-BASEL 2018. [DOI: 10.3390/app8020156] [Citation(s) in RCA: 51] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
12
|
Weiss O, Segev E, Eilam D. Social spatial cognition in rat tetrads: how they select their partners and their gathering places. Anim Cogn 2016; 20:409-418. [DOI: 10.1007/s10071-016-1063-5] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2016] [Revised: 11/23/2016] [Accepted: 12/03/2016] [Indexed: 12/11/2022]
|