Developing the knowledge of number digits in a child-like robot

Emergence of number sense through the integration of multimodal information: developmental learning insights from neural network models

Introduction

Associating multimodal information is essential for human cognitive abilities including mathematical skills. Multimodal learning has also attracted attention in the field of machine learning, and it has been suggested that the acquisition of better latent representation plays an important role in enhancing task performance. This study aimed to explore the impact of multimodal learning on representation, and to understand the relationship between multimodal representation and the development of mathematical skills.

Methods

We employed a multimodal deep neural network as the computational model for multimodal associations in the brain. We compared the representations of numerical information, that is, handwritten digits and images containing a variable number of geometric figures learned through single- and multimodal methods. Next, we evaluated whether these representations were beneficial for downstream arithmetic tasks.

Results

Multimodal training produced better latent representation in terms of clustering quality, which is consistent with previous findings on multimodal learning in deep neural networks. Moreover, the representations learned using multimodal information exhibited superior performance in arithmetic tasks.

Discussion

Our novel findings experimentally demonstrate that changes in acquired latent representations through multimodal association learning are directly related to cognitive functions, including mathematical skills. This supports the possibility that multimodal learning using deep neural network models may offer novel insights into higher cognitive functions.

A Database for Learning Numbers by Visual Finger Recognition in Developmental Neuro-Robotics

Numerical cognition is a fundamental component of human intelligence that has not been fully understood yet. Indeed, it is a subject of research in many disciplines, e.g., neuroscience, education, cognitive and developmental psychology, philosophy of mathematics, linguistics. In Artificial Intelligence, aspects of numerical cognition have been modelled through neural networks to replicate and analytically study children behaviours. However, artificial models need to incorporate realistic sensory-motor information from the body to fully mimic the children's learning behaviours, e.g., the use of fingers to learn and manipulate numbers. To this end, this article presents a database of images, focused on number representation with fingers using both human and robot hands, which can constitute the base for building new realistic models of numerical cognition in humanoid robots, enabling a grounded learning approach in developmental autonomous agents. The article provides a benchmark analysis of the datasets in the database that are used to train, validate, and test five state-of-the art deep neural networks, which are compared for classification accuracy together with an analysis of the computational requirements of each network. The discussion highlights the trade-off between speed and precision in the detection, which is required for realistic applications in robotics.

Visual number sense in untrained deep neural networks

Number sense, the ability to estimate numerosity, is observed in naïve animals, but how this cognitive function emerges in the brain remains unclear. Here, using an artificial deep neural network that models the ventral visual stream of the brain, we show that number-selective neurons can arise spontaneously, even in the complete absence of learning. We also show that the responses of these neurons can induce the abstract number sense, the ability to discriminate numerosity independent of low-level visual cues. We found number tuning in a randomly initialized network originating from a combination of monotonically decreasing and increasing neuronal activities, which emerges spontaneously from the statistical properties of bottom-up projections. We confirmed that the responses of these number-selective neurons show the single- and multineuron characteristics observed in the brain and enable the network to perform number comparison tasks. These findings provide insight into the origin of innate cognitive functions.

A robot that counts like a child: a developmental model of counting and pointing

In this paper, a novel neuro-robotics model capable of counting real items is introduced. The model allows us to investigate the interaction between embodiment and numerical cognition. This is composed of a deep neural network capable of image processing and sequential tasks performance, and a robotic platform providing the embodiment-the iCub humanoid robot. The network is trained using images from the robot's cameras and proprioceptive signals from its joints. The trained model is able to count a set of items and at the same time points to them. We investigate the influence of pointing on the counting process and compare our results with those from studies with children. Several training approaches are presented in this paper, all of them use pre-training routine allowing the network to gain the ability of pointing and number recitation (from 1 to 10) prior to counting training. The impact of the counted set size and distance to the objects are investigated. The obtained results on counting performance show similarities with those from human studies.

Putting a Finger on Numerical Development - Reviewing the Contributions of Kindergarten Finger Gnosis and Fine Motor Skills to Numerical Abilities

The well-documented association between fingers and numbers is not only based on the observation that most children use their fingers for counting and initial calculation, but also on extensive behavioral and neuro-functional evidence. In this article, we critically review developmental studies evaluating the association between finger sensorimotor skills (i.e., finger gnosis and fine motor skills) and numerical abilities. In sum, reviewed studies were found to provide evidential value and indicated that both finger gnosis and fine motor skills predict measures of counting, number system knowledge, number magnitude processing, and calculation ability. Therefore, specific and unique contributions of both finger gnosis and fine motor skills to the development of numerical skills seem to be substantiated. Through critical consideration of the reviewed evidence, we suggest that the association of finger gnosis and fine motor skills with numerical abilities may emerge from a combination of functional and redeployment mechanisms, in which the early use of finger-based numerical strategies during childhood might be the developmental process by which number representations become intertwined with the finger sensorimotor system, which carries an innate predisposition for said association to unfold. Further research is nonetheless necessary to clarify the causal mechanisms underlying this association.