1
|
Heinrich S, Yao Y, Hinz T, Liu Z, Hummel T, Kerzel M, Weber C, Wermter S. Crossmodal Language Grounding in an Embodied Neurocognitive Model. Front Neurorobot 2020; 14:52. [PMID: 33154720 PMCID: PMC7591775 DOI: 10.3389/fnbot.2020.00052] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2019] [Accepted: 07/03/2020] [Indexed: 01/29/2023] Open
Abstract
Human infants are able to acquire natural language seemingly easily at an early age. Their language learning seems to occur simultaneously with learning other cognitive functions as well as with playful interactions with the environment and caregivers. From a neuroscientific perspective, natural language is embodied, grounded in most, if not all, sensory and sensorimotor modalities, and acquired by means of crossmodal integration. However, characterizing the underlying mechanisms in the brain is difficult and explaining the grounding of language in crossmodal perception and action remains challenging. In this paper, we present a neurocognitive model for language grounding which reflects bio-inspired mechanisms such as an implicit adaptation of timescales as well as end-to-end multimodal abstraction. It addresses developmental robotic interaction and extends its learning capabilities using larger-scale knowledge-based data. In our scenario, we utilize the humanoid robot NICO in obtaining the EMIL data collection, in which the cognitive robot interacts with objects in a children's playground environment while receiving linguistic labels from a caregiver. The model analysis shows that crossmodally integrated representations are sufficient for acquiring language merely from sensory input through interaction with objects in an environment. The representations self-organize hierarchically and embed temporal and spatial information through composition and decomposition. This model can also provide the basis for further crossmodal integration of perceptually grounded cognitive representations.
Collapse
Affiliation(s)
- Stefan Heinrich
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany.,International Research Center for Neurointelligence, The University of Tokyo, Tokyo, Japan
| | - Yuan Yao
- Natural Language Processing Lab, Department of Computer Science and Technology, Tsinghua University, Beijing, China
| | - Tobias Hinz
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany
| | - Zhiyuan Liu
- Natural Language Processing Lab, Department of Computer Science and Technology, Tsinghua University, Beijing, China
| | - Thomas Hummel
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany
| | - Matthias Kerzel
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany
| | - Cornelius Weber
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany
| | - Stefan Wermter
- Knowledge Technology Group, Department of Informatics, Universität Hamburg, Hamburg, Germany
| |
Collapse
|
2
|
Stepanova K, Klein FB, Cangelosi A, Vavrecka M. Mapping Language to Vision in a Real-World Robotic Scenario. IEEE Trans Cogn Dev Syst 2018. [DOI: 10.1109/tcds.2018.2819359] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
3
|
|
4
|
|
5
|
Towards Autonomous Robots Via an Incremental Clustering and Associative Learning Architecture. Cognit Comput 2014. [DOI: 10.1007/s12559-014-9311-y] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|