1
|
Deep Intelligence: What AI Should Learn from Nature’s Imagination. Cognit Comput 2023. [DOI: 10.1007/s12559-023-10124-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/08/2023]
|
2
|
Romero A, Bellas F, Duro RJ. A Perspective on Lifelong Open-Ended Learning Autonomy for Robotics through Cognitive Architectures. SENSORS (BASEL, SWITZERLAND) 2023; 23:1611. [PMID: 36772651 PMCID: PMC9920408 DOI: 10.3390/s23031611] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Revised: 01/26/2023] [Accepted: 01/29/2023] [Indexed: 06/18/2023]
Abstract
This paper addresses the problem of achieving lifelong open-ended learning autonomy in robotics, and how different cognitive architectures provide functionalities that support it. To this end, we analyze a set of well-known cognitive architectures in the literature considering the different components they address and how they implement them. Among the main functionalities that are taken as relevant for lifelong open-ended learning autonomy are the fact that architectures must contemplate learning, and the availability of contextual memory systems, motivations or attention. Additionally, we try to establish which of them were actually applied to real robot scenarios. It transpires that in their current form, none of them are completely ready to address this challenge, but some of them do provide some indications on the paths to follow in some of the aspects they contemplate. It can be gleaned that for lifelong open-ended learning autonomy, motivational systems that allow finding domain-dependent goals from general internal drives, contextual long-term memory systems that all allow for associative learning and retrieval of knowledge, and robust learning systems would be the main components required. Nevertheless, other components, such as attention mechanisms or representation management systems, would greatly facilitate operation in complex domains.
Collapse
|
3
|
Naya-Varela M, Faina A, Duro RJ. Morphological Development in Robotic Learning: A Survey. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3052548] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
4
|
Berberian N, Ross M, Chartier S. Embodied working memory during ongoing input streams. PLoS One 2021; 16:e0244822. [PMID: 33400724 PMCID: PMC7785253 DOI: 10.1371/journal.pone.0244822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 12/16/2020] [Indexed: 11/18/2022] Open
Abstract
Sensory stimuli endow animals with the ability to generate an internal representation. This representation can be maintained for a certain duration in the absence of previously elicited inputs. The reliance on an internal representation rather than purely on the basis of external stimuli is a hallmark feature of higher-order functions such as working memory. Patterns of neural activity produced in response to sensory inputs can continue long after the disappearance of previous inputs. Experimental and theoretical studies have largely invested in understanding how animals faithfully maintain sensory representations during ongoing reverberations of neural activity. However, these studies have focused on preassigned protocols of stimulus presentation, leaving out by default the possibility of exploring how the content of working memory interacts with ongoing input streams. Here, we study working memory using a network of spiking neurons with dynamic synapses subject to short-term and long-term synaptic plasticity. The formal model is embodied in a physical robot as a companion approach under which neuronal activity is directly linked to motor output. The artificial agent is used as a methodological tool for studying the formation of working memory capacity. To this end, we devise a keyboard listening framework to delineate the context under which working memory content is (1) refined, (2) overwritten or (3) resisted by ongoing new input streams. Ultimately, this study takes a neurorobotic perspective to resurface the long-standing implication of working memory in flexible cognition.
Collapse
Affiliation(s)
- Nareg Berberian
- Laboratory for Computational Neurodynamics and Cognition, School of Psychology, University of Ottawa, Ottawa, Ontario, Canada
| | - Matt Ross
- Laboratory for Computational Neurodynamics and Cognition, School of Psychology, University of Ottawa, Ottawa, Ontario, Canada
| | - Sylvain Chartier
- Laboratory for Computational Neurodynamics and Cognition, School of Psychology, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
5
|
|
6
|
Hangl S, Dunjko V, Briegel HJ, Piater J. Skill Learning by Autonomous Robotic Playing Using Active Learning and Exploratory Behavior Composition. Front Robot AI 2020; 7:42. [PMID: 33501210 PMCID: PMC7806109 DOI: 10.3389/frobt.2020.00042] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2019] [Accepted: 03/09/2020] [Indexed: 11/13/2022] Open
Abstract
We consider the problem of autonomous acquisition of manipulation skills where problem-solving strategies are initially available only for a narrow range of situations. We propose to extend the range of solvable situations by autonomous play with the object. By applying previously-trained skills and behaviors, the robot learns how to prepare situations for which a successful strategy is already known. The information gathered during autonomous play is additionally used to train an environment model. This model is exploited for active learning and the generation of novel preparatory behaviors compositions. We apply our approach to a wide range of different manipulation tasks, e.g., book grasping, grasping of objects of different sizes by selecting different grasping strategies, placement on shelves, and tower disassembly. We show that the composite behavior generation mechanism enables the robot to solve previously-unsolvable tasks, e.g., tower disassembly. We use success statistics gained during real-world experiments to simulate the convergence behavior of our system. Simulation experiments show that the learning speed can be improved by around 30% by using active learning.
Collapse
Affiliation(s)
- Simon Hangl
- Intelligent and Interactive Systems, Department of Informatics, University of Innsbruck, Innsbruck, Austria
| | | | - Hans J. Briegel
- Institute for Theoretical Physics, University of Innsbruck, Innsbruck, Austria
| | - Justus Piater
- Intelligent and Interactive Systems, Department of Informatics, University of Innsbruck, Innsbruck, Austria
| |
Collapse
|
7
|
Georgeon OL, Riegler A. CASH only: Constitutive autonomy through motorsensory self-programming. COGN SYST RES 2019. [DOI: 10.1016/j.cogsys.2019.08.006] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
8
|
Simplifying the creation and management of utility models in continuous domains for cognitive robotics. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2018.07.093] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
|
9
|
Gordon G. Social behaviour as an emergent property of embodied curiosity: a robotics perspective. Philos Trans R Soc Lond B Biol Sci 2019; 374:20180029. [PMID: 30853006 PMCID: PMC6452242 DOI: 10.1098/rstb.2018.0029] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/14/2018] [Indexed: 12/23/2022] Open
Abstract
Social interaction is an extremely complex yet vital component in daily life. We present a bottom-up approach for the emergence of social behaviours from the interaction of the curiosity drive, i.e. the intrinsic motivation to learn as much as possible, and the embedding environment of an agent. Implementing artificial curiosity algorithms in robots that explore human-like environments results in the emergence of a hierarchical structure of learning and behaviour. This structure resembles the sequential emergence of behavioural patterns in human babies, culminating in social behaviours, such as face detection, tracking and attention-grabbing facial expressions. These results suggest that an embodied curiosity drive may be the progenitor of many social behaviours if satiated by a social environment. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.
Collapse
Affiliation(s)
- Goren Gordon
- Curiosity Lab, Department of Industrial Engineering, Tel-Aviv University, Tel-Aviv, Israel
| |
Collapse
|
10
|
|
11
|
A Self-Verifying Cognitive Architecture for Robust Bootstrapping of Sensory-Motor Skills via Multipurpose Predictors. IEEE Trans Cogn Dev Syst 2018. [DOI: 10.1109/tcds.2018.2871857] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
12
|
Tanneberg D, Peters J, Rueckert E. Intrinsic motivation and mental replay enable efficient online adaptation in stochastic recurrent networks. Neural Netw 2018; 109:67-80. [PMID: 30408695 DOI: 10.1016/j.neunet.2018.10.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2018] [Revised: 08/22/2018] [Accepted: 10/09/2018] [Indexed: 11/29/2022]
Abstract
Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself are crucial. In this work, we propose a novel framework for probabilistic online motion planning with online adaptation based on a bio-inspired stochastic recurrent neural network. By using learning signals which mimic the intrinsic motivation signal cognitive dissonance in addition with a mental replay strategy to intensify experiences, the stochastic recurrent network can learn from few physical interactions and adapts to novel environments in seconds. We evaluate our online planning and adaptation framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is shown by learning unknown workspace constraints sample-efficiently from few physical interactions while following given way points.
Collapse
Affiliation(s)
- Daniel Tanneberg
- Intelligent Autonomous Systems, Technische Universität Darmstadt, Hochschulstr. 10, 64289 Darmstadt, Germany.
| | - Jan Peters
- Intelligent Autonomous Systems, Technische Universität Darmstadt, Hochschulstr. 10, 64289 Darmstadt, Germany; Robot Learning Group, Max-Planck Institute for Intelligent Systems, Max-Planck-Ring 4, 72076 Tübingen, Germany.
| | - Elmar Rueckert
- Institute for Robotics and Cognitive Systems, Universität zu Lübeck, Ratzeburger Allee 160, 23538 Lübeck, Germany; Intelligent Autonomous Systems, Technische Universität Darmstadt, Hochschulstr. 10, 64289 Darmstadt, Germany.
| |
Collapse
|
13
|
Wu R, Zhou C, Chao F, Zhu Z, Lin CM, Yang L. A Developmental Learning Approach of Mobile Manipulator via Playing. Front Neurorobot 2017; 11:53. [PMID: 29046632 PMCID: PMC5632655 DOI: 10.3389/fnbot.2017.00053] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2017] [Accepted: 09/19/2017] [Indexed: 11/13/2022] Open
Abstract
Inspired by infant development theories, a robotic developmental model combined with game elements is proposed in this paper. This model does not require the definition of specific developmental goals for the robot, but the developmental goals are implied in the goals of a series of game tasks. The games are characterized into a sequence of game modes based on the complexity of the game tasks from simple to complex, and the task complexity is determined by the applications of developmental constraints. Given a current mode, the robot switches to play in a more complicated game mode when it cannot find any new salient stimuli in the current mode. By doing so, the robot gradually achieves it developmental goals by playing different modes of games. In the experiment, the game was instantiated into a mobile robot with the playing task of picking up toys, and the game is designed with a simple game mode and a complex game mode. A developmental algorithm, "Lift-Constraint, Act and Saturate," is employed to drive the mobile robot move from the simple mode to the complex one. The experimental results show that the mobile manipulator is able to successfully learn the mobile grasping ability after playing simple and complex games, which is promising in developing robotic abilities to solve complex tasks using games.
Collapse
Affiliation(s)
- Ruiqi Wu
- Fujian Provincal Key Lab of Brain-Inspired Computing, Department of Cognitive Science, School of Informatics, Xiamen University, Xiamen, China
| | - Changle Zhou
- Fujian Provincal Key Lab of Brain-Inspired Computing, Department of Cognitive Science, School of Informatics, Xiamen University, Xiamen, China
| | - Fei Chao
- Fujian Provincal Key Lab of Brain-Inspired Computing, Department of Cognitive Science, School of Informatics, Xiamen University, Xiamen, China
| | - Zuyuan Zhu
- Department of Computer Science, School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom
| | - Chih-Min Lin
- Fujian Provincal Key Lab of Brain-Inspired Computing, Department of Cognitive Science, School of Informatics, Xiamen University, Xiamen, China.,Department of Electrical Engineering, Yuan Ze University, Tao-Yuan, Taiwan
| | - Longzhi Yang
- Department of Computer and Information Sciences, Faculty of Engineering and Environment, Northumbria University, Newcastle upon Tyne, United Kingdom
| |
Collapse
|
14
|
Min H, Yi C, Luo R, Zhu J, Bi S. Affordance Research in Developmental Robotics: A Survey. IEEE Trans Cogn Dev Syst 2016. [DOI: 10.1109/tcds.2016.2614992] [Citation(s) in RCA: 57] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
15
|
Abstract
Distant speaker recognition (DSR) system assumes the microphones are far away from the speaker’s mouth. Also, the position of microphones can vary. Furthermore, various challenges and limitation in terms of coloration, ambient noise and reverberation can bring some difficulties for recognition of the speaker. Although, applying speech enhancement techniques can attenuate speech distortion components, it may remove speaker-specific information and increase the processing time in real-time application. Currently, many efforts have been investigated to develop DSR for commercial viable systems. In this paper, state-of-the-art techniques in DSR such as robust feature extraction, feature normalization, robust speaker modeling, model compensation, dereverberation and score normalization are discussed to overcome the speech degradation components i.e., reverberation and ambient noise. Performance results on DSR show that whenever speaker to microphone distant increases, recognition rates decreases and equal error rate (EER) increases. Finally, the paper concludes that applying robust feature and robust speaker model varying lesser with distant, can improve the DSR performance.
Collapse
Affiliation(s)
- Mohammad Ali Nematollahi
- Department of Computer & Communication Systems Engineering, Faculty of Engineering, University Putra Malaysia, UPM Serdang, 43400 Selangor Darul Ehsan, Malaysia
| | - S. A. R. Al-Haddad
- Department of Computer & Communication Systems Engineering, Faculty of Engineering, University Putra Malaysia, UPM Serdang, 43400 Selangor Darul Ehsan, Malaysia
| |
Collapse
|
16
|
Dong D, Franklin S. A New Action Execution Module for the Learning Intelligent Distribution Agent (LIDA): The Sensory Motor System. Cognit Comput 2015. [DOI: 10.1007/s12559-015-9322-3] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
17
|
|
18
|
Learning and control of exploration primitives. J Comput Neurosci 2014; 37:259-80. [PMID: 24796479 DOI: 10.1007/s10827-014-0500-1] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2013] [Revised: 02/11/2014] [Accepted: 03/12/2014] [Indexed: 12/17/2022]
Abstract
Animals explore novel environments in a cautious manner, exhibiting alternation between curiosity-driven behavior and retreats. We present a detailed formal framework for exploration behavior, which generates behavior that maintains a constant level of novelty. Similar to other types of complex behaviors, the resulting exploratory behavior is composed of exploration motor primitives. These primitives can be learned during a developmental period, wherein the agent experiences repeated interactions with environments that share common traits, thus allowing transference of motor learning to novel environments. The emergence of exploration motor primitives is the result of reinforcement learning in which information gain serves as intrinsic reward. Furthermore, actors and critics are local and ego-centric, thus enabling transference to other environments. Novelty control, i.e. the principle which governs the maintenance of constant novelty, is implemented by a central action-selection mechanism, which switches between the emergent exploration primitives and a retreat policy, based on the currently-experienced novelty. The framework has only a few parameters, wherein time-scales, learning rates and thresholds are adaptive, and can thus be easily applied to many scenarios. We implement it by modeling the rodent's whisking system and show that it can explain characteristic observed behaviors. A detailed discussion of the framework's merits and flaws, as compared to other related models, concludes the paper.
Collapse
|
19
|
Petit M, Lallee S, Boucher JD, Pointeau G, Cheminade P, Ognibene D, Chinellato E, Pattacini U, Gori I, Martinez-Hernandez U, Barron-Gonzalez H, Inderbitzin M, Luvizotto A, Vouloutsi V, Demiris Y, Metta G, Dominey PF. The Coordinating Role of Language in Real-Time Multimodal Learning of Cooperative Tasks. ACTA ACUST UNITED AC 2013. [DOI: 10.1109/tamd.2012.2209880] [Citation(s) in RCA: 31] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
20
|
Gordon G, Dorfman N, Ahissar E. Reinforcement active learning in the vibrissae system: optimal object localization. ACTA ACUST UNITED AC 2012; 107:107-15. [PMID: 22789551 DOI: 10.1016/j.jphysparis.2012.06.004] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2012] [Revised: 06/12/2012] [Accepted: 06/29/2012] [Indexed: 10/28/2022]
Abstract
Rats move their whiskers to acquire information about their environment. It has been observed that they palpate novel objects and objects they are required to localize in space. We analyze whisker-based object localization using two complementary paradigms, namely, active learning and intrinsic-reward reinforcement learning. Active learning algorithms select the next training samples according to the hypothesized solution in order to better discriminate between correct and incorrect labels. Intrinsic-reward reinforcement learning uses prediction errors as the reward to an actor-critic design, such that behavior converges to the one that optimizes the learning process. We show that in the context of object localization, the two paradigms result in palpation whisking as their respective optimal solution. These results suggest that rats may employ principles of active learning and/or intrinsic reward in tactile exploration and can guide future research to seek the underlying neuronal mechanisms that implement them. Furthermore, these paradigms are easily transferable to biomimetic whisker-based artificial sensors and can improve the active exploration of their environment.
Collapse
Affiliation(s)
- Goren Gordon
- Department of Neurobiology, Weizmann Institute of Science, Rehovot 76100, Israel.
| | - Nimrod Dorfman
- Department of Computer Science and Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel
| | - Ehud Ahissar
- Department of Neurobiology, Weizmann Institute of Science, Rehovot 76100, Israel.
| |
Collapse
|
21
|
|
22
|
Gordon G, Ahissar E. Hierarchical curiosity loops and active sensing. Neural Netw 2012; 32:119-29. [PMID: 22386787 DOI: 10.1016/j.neunet.2012.02.024] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2011] [Revised: 01/01/2012] [Accepted: 02/07/2012] [Indexed: 11/19/2022]
Abstract
A curious agent acts so as to optimize its learning about itself and its environment, without external supervision. We present a model of hierarchical curiosity loops for such an autonomous active learning agent, whereby each loop selects the optimal action that maximizes the agent's learning of sensory-motor correlations. The model is based on rewarding the learner's prediction errors in an actor-critic reinforcement learning (RL) paradigm. Hierarchy is achieved by utilizing previously learned motor-sensory mapping, which enables the learning of other mappings, thus increasing the extent and diversity of knowledge and skills. We demonstrate the relevance of this architecture to active sensing using the well-studied vibrissae (whiskers) system, where rodents acquire sensory information by virtue of repeated whisker movements. We show that hierarchical curiosity loops starting from optimally learning the internal models of whisker motion and then extending to object localization result in free-air whisking and object palpation, respectively.
Collapse
Affiliation(s)
- Goren Gordon
- Department of Neurobiology, Weizmann Institute of Science, Rehovot, 76100, Israel.
| | | |
Collapse
|
23
|
Starzyk JA. Motivated Learning for Computational Intelligence. Mach Learn 2012. [DOI: 10.4018/978-1-60960-818-7.ch202] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
This chapter describes a motivated learning (ML) method that advances model building and learning techniques required for intelligent systems. Motivated learning addresses critical limitations of reinforcement learning (RL), the more common approach to coordinating a machine’s interaction with an unknown environment. RL maximizes the external reward by approximating multidimensional value functions; however, it does not work well in dynamically changing environments. The ML method overcomes RL problems by triggering internal motivations, and creating abstract goals and internal reward systems to stimulate learning. The chapter addresses the important question of how to motivate an agent to learn and enhance its own complexity? A mechanism is presented that extends low-level sensory-motor interactions towards advanced perception and motor skills, resulting in the emergence of desired cognitive properties. ML is compared to RL using a rapidly changing environment in which the agent needs to manage its motivations as well as choose and implement goals in order to succeed.
Collapse
|
24
|
KAWAMURA KAZUHIKO, GORDON STEPHENM, RATANASWASD PALIS, ERDEMIR ERDEM, HALL JOSEPHF. IMPLEMENTATION OF COGNITIVE CONTROL FOR A HUMANOID ROBOT. INT J HUM ROBOT 2011. [DOI: 10.1142/s0219843608001558] [Citation(s) in RCA: 34] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Engineers have long employed control systems utilizing models and feedback loops to control real-world systems. Limitations of model-based control led to a generation of intelligent control techniques such as adaptive and fuzzy control. The human brain, on the other hand, is known to process a variety of inputs in parallel, and shift between different levels of cognitive activities while ignoring distractions to focus on the task in hand. This process, known as cognitive control in psychology, is unique to humans and a handful of animals. We are interested in implementing such cognitive control functionalities for our humanoid robot ISAC. This paper outlines the features of multiagent-based cognitive architecture for a humanoid robot and the progress made toward the realization of cognitive control functionalities using attention, working memory and internal rehearsal. Several experiments have been conducted to show that the implementation of an integrated cognitive robot architecture is feasible.
Collapse
Affiliation(s)
- KAZUHIKO KAWAMURA
- Center for Intelligent Systems, Vanderbilt University, VU Box 350131 B, Nashville TN 37235-0131, USA
| | - STEPHEN M. GORDON
- Center for Intelligent Systems, Vanderbilt University, VU Box 350131 B, Nashville TN 37235-0131, USA
| | - PALIS RATANASWASD
- Center for Intelligent Systems, Vanderbilt University, VU Box 350131 B, Nashville TN 37235-0131, USA
| | - ERDEM ERDEMIR
- Center for Intelligent Systems, Vanderbilt University, VU Box 350131 B, Nashville TN 37235-0131, USA
| | - JOSEPH F. HALL
- Center for Intelligent Systems, Vanderbilt University, VU Box 350131 B, Nashville TN 37235-0131, USA
| |
Collapse
|
25
|
Abstract
The inherent value system of a developmental agent enables autonomous mental development to take place right after the agent's "birth." Biologically, it is not clear what basic components constitute a value system. In the computational model introduced here, we propose that inherent value systems should have at least three basic components: punishment, reward and novelty with decreasing weights from the first component to the last. Punishments and rewards are temporally sparse but novelty is temporally dense. We present a biologically inspired computational architecture that guides development of sensorimotor skills through real-time interactions with the environments, driven by an inborn value system. The inherent value system has been successfully tested on an artificial agent in a simulation environment and a robot in the real world.
Collapse
Affiliation(s)
- XIAO HUANG
- Embodied Intelligence Lab, Computer Science and Engineering Department, Michigan State University, East Lansing, MI, 48824, USA
| | - JUYANG WENG
- Embodied Intelligence Lab, Computer Science and Engineering Department, Michigan State University, East Lansing, MI, 48824, USA
| |
Collapse
|
26
|
SHULTZ THOMASR, RIVEST FRANÇOIS, EGRI LÁSZLÓ, THIVIERGE JEANPHILIPPE, DANDURAND FRÉDÉRIC. COULD KNOWLEDGE-BASED NEURAL LEARNING BE USEFUL IN DEVELOPMENTAL ROBOTICS? THE CASE OF KBCC. INT J HUM ROBOT 2011. [DOI: 10.1142/s0219843607001035] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
The new field of developmental robotics faces the formidable challenge of implementing effective learning mechanisms in complex, dynamic environments. We make a case that knowledge-based learning algorithms might help to meet this challenge. A constructive neural learning algorithm, knowledge-based cascade-correlation (KBCC), autonomously recruits previously-learned networks in addition to the single hidden units recruited by ordinary cascade-correlation. This enables learning by analogy when adequate prior knowledge is available, learning by induction from examples when there is no relevant prior knowledge, and various combinations of analogy and induction. A review of experiments with KBCC indicates that recruitment of relevant existing knowledge typically speeds learning and sometimes enables learning of otherwise impossible problems. Some additional domains of interest to developmental robotics are identified in which knowledge-based learning seems essential. The characteristics of KBCC in relation to other knowledge-based neural learners and analogical reasoning are summarized as is the neurological basis for learning from knowledge. Current limitations of this approach and directions for future work are discussed.
Collapse
Affiliation(s)
- THOMAS R. SHULTZ
- Department of Psychology, McGill University, 1205 Penfield Avenue, Montreal, QC H3A 1B1, Canada
- School of Computer Science, McGill University, 3480 University Street, Montreal, QC H3A 2B4, Canada
| | - FRANÇOIS RIVEST
- Département d'Informatique et de Recherche Opérationnelle, Université de Montréal, CP 6128 succursale Centre Ville, Montréal, QC H3C 3J7, Canada
| | - LÁSZLÓ EGRI
- School of Computer Science, McGill University, 3480 University Street, Montreal, QC H3A 2B4, Canada
| | - JEAN-PHILIPPE THIVIERGE
- Département de Physiologie, Université de Montréal, CP 6128 succursale Centre Ville, Montréal, QC H3T 1J4, Canada
| | - FRÉDÉRIC DANDURAND
- Department of Psychology, McGill University, 1205 Penfield Avenue, Montreal, QC H3A 1B1, Canada
| |
Collapse
|
27
|
CHOE YOONSUCK, YANG HUEIFANG, ENG DANIELCHERNYEOW. AUTONOMOUS LEARNING OF THE SEMANTICS OF INTERNAL SENSORY STATES BASED ON MOTOR EXPLORATION. INT J HUM ROBOT 2011. [DOI: 10.1142/s0219843607001102] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
What is available to developmental programs in autonomous mental development, and what should be learned at the very early stages of mental development? Our observation is that sensory and motor primitives are the most basic components present at the beginning, and what developmental agents need to learn from these resources is what their internal sensory states stand for. In this paper, we investigate the question in the context of a simple biologically motivated visuomotor agent. We observe and acknowledge, as many other researchers do, that action plays a key role in providing content to the sensory state. We propose a simple, yet powerful learning criterion, that of invariance, where invariance simply means that the internal state does not change over time. We show that after reinforcement learning based on the invariance criterion, the property of action sequence based on an internal sensory state accurately reflects the property of the stimulus that triggered that internal state. That way, the meaning of the internal sensory state can be firmly grounded on the property of that particular action sequence. We expect the framing of the problem and the proposed solution presented in this paper to help shed new light on autonomous understanding in developmental agents such as humanoid robots.
Collapse
Affiliation(s)
- YOONSUCK CHOE
- Department of Computer Science, Texas A&M University, College Station, TX 77843, USA
| | - HUEI-FANG YANG
- Department of Computer Science, Texas A&M University, College Station, TX 77843, USA
| | - DANIEL CHERN-YEOW ENG
- Department of Computer Science, Texas A&M University, College Station, TX 77843, USA
| |
Collapse
|
28
|
WENG JUYANG, LUWANG TIANYU, LU HONG, XUE XIANGYANG. A MULTILAYER IN-PLACE LEARNING NETWORK FOR DEVELOPMENT OF GENERAL INVARIANCES. INT J HUM ROBOT 2011. [DOI: 10.1142/s0219843607001072] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Currently, there is a lack of general-purpose, in-place learning engines that incrementally learn multiple tasks, to develop "soft" multi-task-shared invariances in the intermediate internal representation while a developmental robot interacts with its environment. In-place learning is a biologically inspired concept, rooted in the genomic equivalence principle, meaning that each neuron is responsible for its own development while interacting with its environment. With in-place learning, there is no need for a separate learning network. Computationally, biologically inspired, in-place learning provides unusually efficient learning algorithms whose simplicity, low computational complexity, and generality are set apart from typical conventional learning algorithms. We present in this paper the multiple-layer in-place learning network (MILN) for this ambitious goal. As a key requirement for autonomous mental development, the network enables both unsupervised and supervised learning to occur concurrently, depending on whether motor supervision signals are available or not at the motor end (the last layer) during the agent's interactions with the environment. We present principles based on which MILN automatically develops invariant neurons in different layers and why such invariant neuronal clusters are important for learning later tasks in open-ended development. From sequentially sensed sensory streams, the proposed MILN incrementally develops a hierarchy of internal representations. The global invariance achieved through multi-layer invariances, with increasing invariance from early layers to the later layers. Experimental results with statistical performance measures are presented to show the effects of the principles.
Collapse
Affiliation(s)
- JUYANG WENG
- Department of Computer Science and Engineering, Fudan University, Shanghai, China
- Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA
| | - TIANYU LUWANG
- Department of Computer Science and Engineering, Fudan University, Shanghai, China
| | - HONG LU
- Department of Computer Science and Engineering, Fudan University, Shanghai, China
| | - XIANGYANG XUE
- Department of Computer Science and Engineering, Fudan University, Shanghai, China
| |
Collapse
|
29
|
Ji Ryang Chung, Yoonsuck Choe. Emergence of Memory in Reactive Agents Equipped With Environmental Markers. ACTA ACUST UNITED AC 2011. [DOI: 10.1109/tamd.2011.2132800] [Citation(s) in RCA: 11] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
30
|
|
31
|
Cangelosi A, Metta G, Sagerer G, Nolfi S, Nehaniv C, Fischer K, Tani J, Belpaeme T, Sandini G, Nori F, Fadiga L, Wrede B, Rohlfing K, Tuci E, Dautenhahn K, Saunders J, Zeschel A. Integration of Action and Language Knowledge: A Roadmap for Developmental Robotics. ACTA ACUST UNITED AC 2010. [DOI: 10.1109/tamd.2010.2053034] [Citation(s) in RCA: 107] [Impact Index Per Article: 7.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
32
|
Massera G, Tuci E, Ferrauto T, Nolfi S. The Facilitatory Role of Linguistic Instructions on Developing Manipulation Skills. IEEE COMPUT INTELL M 2010. [DOI: 10.1109/mci.2010.937321] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
33
|
Task Muddiness, Intelligence Metrics, and the Necessity of Autonomous Mental Development. Minds Mach (Dordr) 2008. [DOI: 10.1007/s11023-008-9127-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
34
|
|
35
|
Abstract
This paper presents incremental hierarchical discriminant regression (IHDR) which incrementally builds a decision tree or regression tree for very high-dimensional regression or decision spaces by an online, real-time learning system. Biologically motivated, it is an approximate computational model for automatic development of associative cortex, with both bottom-up sensory inputs and top-down motor projections. At each internal node of the IHDR tree, information in the output space is used to automatically derive the local subspace spanned by the most discriminating features. Embedded in the tree is a hierarchical probability distribution model used to prune very unlikely cases during the search. The number of parameters in the coarse-to-fine approximation is dynamic and data-driven, enabling the IHDR tree to automatically fit data with unknown distribution shapes (thus, it is difficult to select the number of parameters up front). The IHDR tree dynamically assigns long-term memory to avoid the loss-of-memory problem typical with a global-fitting learning algorithm for neural networks. A major challenge for an incrementally built tree is that the number of samples varies arbitrarily during the construction process. An incrementally updated probability model, called sample-size-dependent negative-log-likelihood (SDNLL) metric is used to deal with large sample-size cases, small sample-size cases, and unbalanced sample-size cases, measured among different internal nodes of the IHDR tree. We report experimental results for four types of data: synthetic data to visualize the behavior of the algorithms, large face image data, continuous video stream from robot navigation, and publicly available data sets that use human defined features.
Collapse
Affiliation(s)
- Juyang Weng
- Department of Computer Science and Engineering, Michigan State University, East Lansing, MI 48824, USA.
| | | |
Collapse
|
36
|
Cognitively Inspired Anticipatory Adaptation and Associated Learning Mechanisms for Autonomous Agents. ANTICIPATORY BEHAVIOR IN ADAPTIVE LEARNING SYSTEMS 2007. [DOI: 10.1007/978-3-540-74262-3_7] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
|
37
|
Juyang Weng, Wey-Shiuan Hwang. From neural networks to the brain: autonomous mental development. IEEE COMPUT INTELL M 2006. [DOI: 10.1109/mci.2006.1672985] [Citation(s) in RCA: 25] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
38
|
McKinstry JL, Edelman GM, Krichmar JL. A cerebellar model for predictive motor control tested in a brain-based device. Proc Natl Acad Sci U S A 2006; 103:3387-92. [PMID: 16488974 PMCID: PMC1413924 DOI: 10.1073/pnas.0511281103] [Citation(s) in RCA: 47] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
The cerebellum is known to be critical for accurate adaptive control and motor learning. We propose here a mechanism by which the cerebellum may replace reflex control with predictive control. This mechanism is embedded in a learning rule (the delayed eligibility trace rule) in which synapses onto a Purkinje cell or onto a cell in the deep cerebellar nuclei become eligible for plasticity only after a fixed delay from the onset of suprathreshold presynaptic activity. To investigate the proposal that the cerebellum is a general-purpose predictive controller guided by a delayed eligibility trace rule, a computer model based on the anatomy and dynamics of the cerebellum was constructed. It contained components simulating cerebellar cortex and deep cerebellar nuclei, and it received input from a middle temporal visual area and the inferior olive. The model was incorporated in a real-world brain-based device (BBD) built on a Segway robotic platform that learned to traverse curved paths. The BBD learned which visual motion cues predicted impending collisions and used this experience to avoid path boundaries. During learning, the BBD adapted its velocity and turning rate to successfully traverse various curved paths. By examining neuronal activity and synaptic changes during this behavior, we found that the cerebellar circuit selectively responded to motion cues in specific receptive fields of simulated middle temporal visual areas. The system described here prompts several hypotheses about the relationship between perception and motor control and may be useful in the development of general-purpose motor learning systems for machines.
Collapse
Affiliation(s)
- Jeffrey L. McKinstry
- The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121
- *To whom correspondence may be addressed. E-mail: or
| | - Gerald M. Edelman
- The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121
- *To whom correspondence may be addressed. E-mail: or
| | - Jeffrey L. Krichmar
- The Neurosciences Institute, 10640 John Jay Hopkins Drive, San Diego, CA 92121
| |
Collapse
|