1
|
Taniguchi A, Ito S, Taniguchi T. Hierarchical path planning from speech instructions with spatial concept-based topometric semantic mapping. Front Robot AI 2024; 11:1291426. [PMID: 39148580 PMCID: PMC11324419 DOI: 10.3389/frobt.2024.1291426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2023] [Accepted: 06/20/2024] [Indexed: 08/17/2024] Open
Abstract
Assisting individuals in their daily activities through autonomous mobile robots is a significant concern, especially for users without specialized knowledge. Specifically, the capability of a robot to navigate to destinations based on human speech instructions is crucial. Although robots can take different paths toward the same objective, the shortest path is not always the most suitable. A preferred approach would be to accommodate waypoint specifications flexibly for planning an improved alternative path even with detours. Furthermore, robots require real-time inference capabilities. In this sense, spatial representations include semantic, topological, and metric-level representations, each capturing different aspects of the environment. This study aimed to realize a hierarchical spatial representation using a topometric semantic map and path planning with speech instructions by including waypoints. Thus, we present a hierarchical path planning method called spatial concept-based topometric semantic mapping for hierarchical path planning (SpCoTMHP), which integrates place connectivity. This approach provides a novel integrated probabilistic generative model and fast approximate inferences with interactions among the hierarchy levels. A formulation based on "control as probabilistic inference" theoretically supports the proposed path planning algorithm. We conducted experiments in a home environment using the Toyota human support robot on the SIGVerse simulator and in a lab-office environment with the real robot Albert. Here, the user issues speech commands that specify the waypoint and goal, such as "Go to the bedroom via the corridor." Navigation experiments were performed using speech instructions with a waypoint to demonstrate the performance improvement of the SpCoTMHP over the baseline hierarchical path planning method with heuristic path costs (HPP-I) in terms of the weighted success rate at which the robot reaches the closest target (0.590) and passes the correct waypoints. The computation time was significantly improved by 7.14 s with the SpCoTMHP than the baseline HPP-I in advanced tasks. Thus, hierarchical spatial representations provide mutually understandable instruction forms for both humans and robots, thus enabling language-based navigation.
Collapse
Affiliation(s)
- Akira Taniguchi
- Emergent Systems Laboratory, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Shuya Ito
- Emergent Systems Laboratory, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Tadahiro Taniguchi
- Emergent Systems Laboratory, Ritsumeikan University, Kusatsu, Shiga, Japan
| |
Collapse
|
2
|
Miyazawa K, Nagai T. Concept formation through multimodal integration using multimodal BERT and VQ-VAE. Adv Robot 2022. [DOI: 10.1080/01691864.2022.2141583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Kazuki Miyazawa
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
| | - Takayuki Nagai
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
- Artificial Intelligence Exploration Research Center, The University of Electro-Communications, Tokyo, Japan
| |
Collapse
|
3
|
Sagara R, Taguchi R, Taniguchi A, Taniguchi T. Automatic selection of coordinate systems for learning relative and absolute spatial concepts. Front Robot AI 2022; 9:904751. [PMID: 36035866 PMCID: PMC9411740 DOI: 10.3389/frobt.2022.904751] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Accepted: 06/27/2022] [Indexed: 11/29/2022] Open
Abstract
Robots employed in homes and offices need to adaptively learn spatial concepts using user utterances. To learn and represent spatial concepts, the robot must estimate the coordinate system used by humans. For example, to represent spatial concept “left,” which is one of the relative spatial concepts (defined as a spatial concept depending on the object’s location), humans use a coordinate system based on the direction of a reference object. As another example, to represent spatial concept “living room,” which is one of the absolute spatial concepts (defined as a spatial concept that does not depend on the object’s location), humans use a coordinate system where a point on a map constitutes the origin. Because humans use these concepts in daily life, it is important for the robot to understand the spatial concepts in different coordinate systems. However, it is difficult for robots to learn these spatial concepts because humans do not clarify the coordinate system. Therefore, we propose a method (RASCAM) that enables a robot to simultaneously estimate the coordinate system and spatial concept. The proposed method is based on ReSCAM+O, which is a learning method for relative spatial concepts based on a probabilistic model. The proposed method introduces a latent variable that represents a coordinate system for simultaneous learning. This method can simultaneously estimate three types of unspecified information: coordinate systems, reference objects, and the relationship between concepts and words. No other method can estimate all these three types. Experiments using three different coordinate systems demonstrate that the proposed method can learn both relative and absolute spatial concepts while accurately selecting the coordinate system. The proposed approach can be beneficial for service robots to flexibly understand a new environment through the interactions with humans.
Collapse
Affiliation(s)
- Rikunari Sagara
- Taguchi Laboratory, Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
- *Correspondence: Rikunari Sagara,
| | - Ryo Taguchi
- Taguchi Laboratory, Department of Computer Science, Nagoya Institute of Technology, Nagoya, Japan
| | - Akira Taniguchi
- Emergent Systems Laboratory, College of Information Science and Engineering, Ritsumeikan University, Kyoto, Japan
| | - Tadahiro Taniguchi
- Emergent Systems Laboratory, College of Information Science and Engineering, Ritsumeikan University, Kyoto, Japan
| |
Collapse
|
4
|
Damgaard MR, Pedersen R, Bak T. Toward an idiomatic framework for cognitive robotics. PATTERNS (NEW YORK, N.Y.) 2022; 3:100533. [PMID: 35845837 PMCID: PMC9278519 DOI: 10.1016/j.patter.2022.100533] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/03/2021] [Revised: 04/20/2022] [Accepted: 05/27/2022] [Indexed: 11/25/2022]
Abstract
Inspired by the "cognitive hourglass" model presented by the researchers behind the cognitive architecture called Sigma, we propose a framework for developing cognitive architectures for cognitive robotics. The main purpose of the proposed framework is to ease development of cognitive architectures by encouraging cooperation and re-use of existing results. This is done by proposing a framework dividing development of cognitive architectures into a series of layers that can be considered partly in isolation, some of which directly relate to other research fields. Finally, we introduce and review some topics essential for the proposed framework. We also outline a set of applications.
Collapse
Affiliation(s)
- Malte Rørmose Damgaard
- Department of Electronic Systems, Automation and Control, Aalborg University, 9220 Aalborg, Denmark
| | - Rasmus Pedersen
- Department of Electronic Systems, Automation and Control, Aalborg University, 9220 Aalborg, Denmark
| | - Thomas Bak
- Department of Electronic Systems, Automation and Control, Aalborg University, 9220 Aalborg, Denmark
| |
Collapse
|
5
|
Taniguchi A, Fukawa A, Yamakawa H. Hippocampal formation-inspired probabilistic generative model. Neural Netw 2022; 151:317-335. [DOI: 10.1016/j.neunet.2022.04.001] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2021] [Revised: 03/09/2022] [Accepted: 04/03/2022] [Indexed: 11/25/2022]
|
6
|
Taniguchi T, Yamakawa H, Nagai T, Doya K, Sakagami M, Suzuki M, Nakamura T, Taniguchi A. A whole brain probabilistic generative model: Toward realizing cognitive architectures for developmental robots. Neural Netw 2022; 150:293-312. [PMID: 35339010 DOI: 10.1016/j.neunet.2022.02.026] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2021] [Revised: 02/25/2022] [Accepted: 02/28/2022] [Indexed: 01/08/2023]
Abstract
Building a human-like integrative artificial cognitive system, that is, an artificial general intelligence (AGI), is the holy grail of the artificial intelligence (AI) field. Furthermore, a computational model that enables an artificial system to achieve cognitive development will be an excellent reference for brain and cognitive science. This paper describes an approach to develop a cognitive architecture by integrating elemental cognitive modules to enable the training of the modules as a whole. This approach is based on two ideas: (1) brain-inspired AI, learning human brain architecture to build human-level intelligence, and (2) a probabilistic generative model (PGM)-based cognitive architecture to develop a cognitive system for developmental robots by integrating PGMs. The proposed development framework is called a whole brain PGM (WB-PGM), which differs fundamentally from existing cognitive architectures in that it can learn continuously through a system based on sensory-motor information. In this paper, we describe the rationale for WB-PGM, the current status of PGM-based elemental cognitive modules, their relationship with the human brain, the approach to the integration of the cognitive modules, and future challenges. Our findings can serve as a reference for brain studies. As PGMs describe explicit informational relationships between variables, WB-PGM provides interpretable guidance from computational sciences to brain science. By providing such information, researchers in neuroscience can provide feedback to researchers in AI and robotics on what the current models lack with reference to the brain. Further, it can facilitate collaboration among researchers in neuro-cognitive sciences as well as AI and robotics.
Collapse
Affiliation(s)
| | - Hiroshi Yamakawa
- The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo, Japan; The Whole Brain Architecture Initiative, 2-19-21 Nishikoiwa , Edogawa-ku, Tokyo, Japan; RIKEN, 6-2-3 Furuedai, Suita, Osaka, Japan
| | - Takayuki Nagai
- Osaka University, 1-3 Machikane-yama, Toyonaka, Osaka, Japan
| | - Kenji Doya
- Okinawa Institute of Science and Technology Graduate University, 1919-1 Tancha, Onna-son, Kunigami, Okinawa, Japan
| | | | - Masahiro Suzuki
- The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo, Japan
| | - Tomoaki Nakamura
- The University of Electro-Communications, 1-5-1 Chofugaoka, Chofu, Tokyo, Japan
| | | |
Collapse
|
7
|
Katsumata Y, Kanechika A, Taniguchi A, El Hafi L, Hagiwara Y, Taniguchi T. Map completion from partial observation using the global structure of multiple environmental maps. Adv Robot 2022. [DOI: 10.1080/01691864.2022.2029762] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/01/2022]
Affiliation(s)
- Yuki Katsumata
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Akinori Kanechika
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Akira Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Lotfi El Hafi
- Ritsumeikan Global Innovation Research Organization, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Yoshinobu Hagiwara
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Tadahiro Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| |
Collapse
|
8
|
Shimoda S, Jamone L, Ognibene D, Nagai T, Sciutti A, Costa-Garcia A, Oseki Y, Taniguchi T. What is the role of the next generation of cognitive robotics? Adv Robot 2021. [DOI: 10.1080/01691864.2021.2011780] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Shingo Shimoda
- RIKEN Center for Brain Science TOYOTA Collaboration Center, Nagoya, Japan
| | - Lorenzo Jamone
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK
| | - Dimitri Ognibene
- Computer Science and Artificial Intelligence, University of Milano Biccoca, Milano, Italy
| | - Takayuki Nagai
- Graduate School of Engineering Science, Osaka University, Osaka, Japan
| | - Alessandra Sciutti
- Cognitive Architecture for Collaborative Technologies Unit, Italian Institute of Technology, Genova, Italy
| | | | - Yohei Oseki
- Department of Language and Information Sciences, University of Tokyo, Tokyo, Japan
| | - Tadahiro Taniguchi
- Department of Human and Computer Intelligence, Ritsumeikan University, Shiga, Japan
| |
Collapse
|
9
|
Sagara R, Taguchi R, Taniguchi A, Taniguchi T, Hattori K, Hoguro M, Umezaki T. Unsupervised lexical acquisition of relative spatial concepts using spoken user utterances. Adv Robot 2021. [DOI: 10.1080/01691864.2021.2007168] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Rikunari Sagara
- Taguchi Laboratory, Department of Computer Science, Nagoya Institute of Technology, Aichi, Japan
| | - Ryo Taguchi
- Taguchi Laboratory, Department of Computer Science, Nagoya Institute of Technology, Aichi, Japan
| | - Akira Taniguchi
- Emergent Systems Laboratory, College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan
| | - Tadahiro Taniguchi
- Emergent Systems Laboratory, College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan
| | | | | | - Taizo Umezaki
- College of Engineering, Chubu University, Aichi, Japan
| |
Collapse
|
10
|
Hagiwara Y, Taguchi K, Ishibushi S, Taniguchi A, Taniguchi T. Hierarchical Bayesian model for the transfer of knowledge on spatial concepts based on multimodal information. Adv Robot 2021. [DOI: 10.1080/01691864.2021.2004224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Yoshinobu Hagiwara
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Keishiro Taguchi
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | | | - Akira Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| | - Tadahiro Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Kusatsu, Shiga, Japan
| |
Collapse
|
11
|
Friston K, Moran RJ, Nagai Y, Taniguchi T, Gomi H, Tenenbaum J. World model learning and inference. Neural Netw 2021; 144:573-590. [PMID: 34634605 DOI: 10.1016/j.neunet.2021.09.011] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2021] [Revised: 07/28/2021] [Accepted: 09/09/2021] [Indexed: 11/19/2022]
Abstract
Understanding information processing in the brain-and creating general-purpose artificial intelligence-are long-standing aspirations of scientists and engineers worldwide. The distinctive features of human intelligence are high-level cognition and control in various interactions with the world including the self, which are not defined in advance and are vary over time. The challenge of building human-like intelligent machines, as well as progress in brain science and behavioural analyses, robotics, and their associated theoretical formalisations, speaks to the importance of the world-model learning and inference. In this article, after briefly surveying the history and challenges of internal model learning and probabilistic learning, we introduce the free energy principle, which provides a useful framework within which to consider neuronal computation and probabilistic world models. Next, we showcase examples of human behaviour and cognition explained under that principle. We then describe symbol emergence in the context of probabilistic modelling, as a topic at the frontiers of cognitive robotics. Lastly, we review recent progress in creating human-like intelligence by using novel probabilistic programming languages. The striking consensus that emerges from these studies is that probabilistic descriptions of learning and inference are powerful and effective ways to create human-like artificial intelligent machines and to understand intelligence in the context of how humans interact with their world.
Collapse
Affiliation(s)
- Karl Friston
- Wellcome Centre for Human Neuroimaging, Institute of Neurology, University College London (UCL), WC1N 3BG, UK.
| | - Rosalyn J Moran
- Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King's College London, SE5 8AF, UK.
| | - Yukie Nagai
- International Research Center for Neurointelligence (IRCN), The University of Tokyo, Tokyo, Japan.
| | - Tadahiro Taniguchi
- College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan.
| | - Hiroaki Gomi
- NTT Communication Science Labs., Nippon Telegraph and Telephone, Kanawaga, Japan.
| | - Josh Tenenbaum
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA; The Center for Brains, Minds and Machines, MIT, Cambridge, MA, USA.
| |
Collapse
|