Can infants' sense of agency be found in their behavior? Insights from babybot simulations of the mobile-paradigm

Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies

A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering an infant's foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.

Methodological integrity assessment in the mobile paradigm literature: A lesson for understanding opportunistic use of researcher degrees of freedom in psychology

The mobile paradigm has played a fundamental role in memory development research. One key characteristic of the mobile paradigm literature is that across decades, researchers have faithfully followed a particular methodological protocol with its own unique definitions of learning and memory. To investigate the extent to which these methodological choices affected the results, the literature (77 publications and 505 statistical tests) was evaluated for four frequently encountered research biases. The results suggested that research using the paradigm was conducted with scientific rigor. However, methodological choices along with unique operational definitions of learning and memory accounted for more than half of the findings. Thus, the literature has been contaminated by methodological artifacts due to the opportunistic use of researcher degrees of freedom.

Meaning from movement and stillness: Signatures of coordination dynamics reveal infant agency

How do human beings make sense of their relation to the world and realize their ability to effect change? Applying modern concepts and methods of coordination dynamics, we demonstrate that patterns of movement and coordination in 3 to 4-mo-olds may be used to identify states and behavioral phenotypes of emergent agency. By means of a complete coordinative analysis of baby and mobile motion and their interaction, we show that the emergence of agency can take the form of a punctuated self-organizing process, with meaning found both in movement and stillness.

Dynamical systems model of development of the action differentiation in early infancy: a requisite of physical agency

Young infants are sensitive to whether their body movements cause subsequent events or not during the interaction with the environment. This ability has been revealed by empirical studies on the reinforcement of limb movements when a string is attached between an infant limb and a mobile toy suspended overhead. A previous study reproduced the experimental observation by modeling both the infant's limb and a mobile toy as a system of coupled oscillators. The authors then argued that emergence of agency could be explained by a phase transition in the dynamical system: from a weakly coupled state to a state where the both movements of the limb and the toy are highly coordinated. However, what remains unexplained is the following experimental observation: When the limb is connected to the mobile toy by a string, the infant increases the average velocity of the arm's movement. On the other hand, when the toy is controlled externally, the average arm's velocity is greatly reduced. Since young infants produce exuberant spontaneous movements even with no external stimuli, the inhibition of motor action to suppress the formation of spurious action-perception coupling should be also a crucial sign for the emergence of agency. Thus, we present a dynamical system model for the development of action differentiation, to move or not to move, in the mobile task. In addition to the pair of limb and mobile oscillators for providing positive feedback for reinforcement in the previous model, bifurcation dynamics are incorporated to enhance or inhibit self-movements in response to detecting contingencies between the limb and mobile movements. The results from computer simulations reproduce experimental observations on the developmental emergence of action differentiation between 2 and 3 months of age in the form of a bifurcation diagram. We infer that the emergence of physical agency entails young infants' ability not only to enhance a specific action-perception coupling, but also to decouple it and create a new mode of action-perception coupling based on the internal state dynamics with contingency detection between self-generated actions and environmental events.

Correlation between performance and quantity/variability of leg exploration in a contingency learning task during infancy

Quantity and quality of motor exploration are proposed to be fundamental for infant motor development. However, it is still not clear what types of motor exploration contribute to learning. To determine whether changes in quantity of leg movement and/or variability of leg acceleration are related to performance in a contingency learning task, twenty 6-8-month-old infants with typical development participated in a contingency learning task. During this task, a robot provided reinforcement when the infant's right leg peak acceleration was above an individualized threshold. The correlation coefficient between the infant's performance and the change in quantity of right leg movement, linear variability, and nonlinear variability of right leg movement acceleration from baseline were calculated. Simple linear regression and multiple linear regression were calculated to explain the contribution of each variable to the performance individually and collectively. We found significant correlation between the performance and the change in quantity of right leg movement (r = 0.86, p < 0.001), linear variability (r = 0.71, p < 0.001), and nonlinear variability (r = 0.62, p = 0.004) of right leg movement acceleration, respectively. However, multiple linear regression showed that only quantity and linear variability of leg movements were significant predicting factors for the performance ratio (p < 0.001, adjusted R² = 0.94). These results indicated that the quantity of exploration and variable exploratory strategies could be critical for the motor learning process during infancy.

Learning limb-specific contingencies in early infancy

Most research with the mobile paradigm has the underlying assumption that young infants can selectively move the limb causing the contingent feedback from the mobile while avoiding irrelevant motor responses. Contrary to this long-held belief, others have argued that such differentiation ability is not fully developed early in life. In the current study, we revisited the traditional mobile paradigm with a contemporary research approach (using high-precision motion capture techniques, a yoked-control design, and a large sample size) to investigate whether response differentiation ability emerges before 5 months of age. The data collected from 76 infants (aged between 115 and 159 days) revealed that infants can learn sensorimotor contingencies by increasing the movement of the connected leg relative to their baseline level. However, they did not differentially increase the movement of the leg causing an effect in the environment compared with that of other limbs. Our results illustrate that response differentiation ability emerges later than previously suggested.

Motivation(s) from control: response-effect contingency and confirmation of sensorimotor predictions reinforce different levels of selection

Humans and other animals live in dynamic environments. To reliably manipulate the environment and attain their goals they would benefit from a constant modification of motor-responding based on responses' current effect on the current environment. It is argued that this is exactly what is achieved by a mechanism that reinforces responses which have led to accurate sensorimotor predictions. We further show that evaluations of a response's effectiveness can occur simultaneously, driven by at least two different processes, each relying on different statistical properties of the feedback and affecting a different level of responding. Specifically, we show the continuous effect of (a) a sensorimotor process sensitive only to the conditional probability of effects given that the agent acted on the environment (i.e., action-effects) and of (b) a more abstract judgement or inference that is also sensitive to the conditional probabilities of occurrence of feedback given no action by the agent (i.e., inaction-effects). The latter process seems to guide action selection (e.g., should I act?) while the former the manner of the action's execution. This study is the first to show that different evaluation processes of a response's effectiveness influence different levels of responding.

Sensorimotor Representation Learning for an “Active Self” in Robots: A Model Survey

Safe human-robot interactions require robots to be able to learn how to behave appropriately in spaces populated by people and thus to cope with the challenges posed by our dynamic and unstructured environment, rather than being provided a rigid set of rules for operations. In humans, these capabilities are thought to be related to our ability to perceive our body in space, sensing the location of our limbs during movement, being aware of other objects and agents, and controlling our body parts to interact with them intentionally. Toward the next generation of robots with bio-inspired capacities, in this paper, we first review the developmental processes of underlying mechanisms of these abilities: The sensory representations of body schema, peripersonal space, and the active self in humans. Second, we provide a survey of robotics models of these sensory representations and robotics models of the self; and we compare these models with the human counterparts. Finally, we analyze what is missing from these robotics models and propose a theoretical computational framework, which aims to allow the emergence of the sense of self in artificial agents by developing sensory representations through self-exploration.

Making the World Behave: A New Embodied Account on Mobile Paradigm

In this review article, we describe the mobile paradigm, a method used for more than 50 years to assess how infants learn and remember sensorimotor contingencies. The literature on the mobile paradigm demonstrates that infants below 6 months of age can remember the learning environment weeks after when reminded periodically and integrate temporally distributed information across modalities. The latter ability is only possible if events occur within a temporal window of a few days, and the width of this required window changes as a function of age. A major critique of these conclusions is that the majority of this literature has neglected the embodied experience, such that motor behavior was considered an equivalent developmental substitute for verbal behavior. Over recent years, simulation and empirical work have highlighted the sensorimotor aspect and opened up a discussion for possible learning mechanisms and variability in motor preferences of young infants. In line with this recent direction, we present a new embodied account on the mobile paradigm which argues that learning sensorimotor contingencies is a core feature of development forming the basis for active exploration of the world and body. In addition to better explaining recent findings, this new framework aims to replace the dis-embodied approach to the mobile paradigm with a new understanding that focuses on variance and representations grounded in sensorimotor experience. Finally, we discuss a potential role for the dorsal stream which might be responsible for guiding action according to visual information, while infants learn sensorimotor contingencies in the mobile paradigm.

Neural processing of self-produced and externally generated events in 3-month-old infants

Did I make that sound? Differentiating whether sensory events are caused by us or the environment is pivotal for our sense of agency. Adults can predict the sensory effects of their actions, which results in attenuated processing of self-produced events compared with externally generated events. Yet, little is known about whether young infants predict and discriminate self-produced events from externally produced events. Using electroencephalography (EEG), 3-month-olds' neural response to the same audiovisual stimulus was compared between a Self-produced condition and externally generated conditions with predictable timing (External-Regular) and irregular timing (External-Irregular). We hypothesized that if 3-month-olds predict self-produced events, their event-related potentials should be smallest for the Self-produced condition, strongest for the External-Irregular condition, and in between for the External-Regular condition. Cluster-based permutation tests indicated a more positive deflection (300-470 ms) for irregular stimuli compared with regular stimuli over the vertex. Contrasting the Self-produced and External-Irregular conditions showed a statistical trend within the same time window. Although not fully conclusive, this might suggest the emerging differentiation between self-produced and less predictable external events. However, there was no statistical evidence that infants differentiated self-produced events from temporally predictable external events. Our findings shed light on the emerging sense of agency and suggest that 3-month-olds are transitioning toward predicting and discriminating the consequences of their actions.

Making Sense of the World: Infant Learning From a Predictive Processing Perspective

For human infants, the first years after birth are a period of intense exploration-getting to understand their own competencies in interaction with a complex physical and social environment. In contemporary neuroscience, the predictive-processing framework has been proposed as a general working principle of the human brain, the optimization of predictions about the consequences of one's own actions, and sensory inputs from the environment. However, the predictive-processing framework has rarely been applied to infancy research. We argue that a predictive-processing framework may provide a unifying perspective on several phenomena of infant development and learning that may seem unrelated at first sight. These phenomena include statistical learning principles, infants' motor and proprioceptive learning, and infants' basic understanding of their physical and social environment. We discuss how a predictive-processing perspective can advance the understanding of infants' early learning processes in theory, research, and application.

From movement to action: An EEG study into the emerging sense of agency in early infancy

Research into the developing sense of agency has traditionally focused on sensitivity to sensorimotor contingencies, but whether this implies the presence of a causal action-effect model has recently been called into question. Here, we investigated whether 3- to 4.5-month-old infants build causal action-effect models by focusing on behavioral and neural measures of violation of expectation. Infants had time to explore the causal link between their movements and audiovisual effects before the action-effect contingency was discontinued. We tested their ability to predict the consequences of their movements and recorded neural (EEG) and movement measures. If infants built a causal action-effect model, we expected to observe their violation of expectation in the form of a mismatch negativity (MMN) in the EEG and an extinction burst in their movement behavior after discontinuing the action-effect contingency. Our findings show that the group of infants who showed an MMN upon cessation of the contingent effect demonstrated a more pronounced limb-specific behavioral extinction burst, indicating a causal action-effect model, compared to the group of infants who did not show an MMN. These findings reveal that, in contrast to previous claims, the sense of agency is only beginning to emerge at this age.

The Differential Impact of a Response's Effectiveness and its Monetary Value on Response-Selection

While known reinforcers of behavior are outcomes that are valuable to the organism, recent research has demonstrated that the mere occurrence of an own-response effect can also reinforce responding. In this paper we begin investigating whether these two types of reinforcement occur via the same mechanism. To this end, we modified two different tasks, previously established to capture the influence of a response's effectiveness on the speed of motor-responses (indexed here by participants' reaction times). Specifically, in six experiments we manipulated both a response's 'pure' effectiveness and its outcome value (e.g., substantial versus negligible monetary reward) and measured the influence of both on the speed of responding. The findings strongly suggest that post action selection, responding is influenced only by pure effectiveness, as assessed by the motor system; thus, at these stages responding is not sensitive to abstract representations of the value of a response (e.g., monetary value). We discuss the benefit of distinguishing between these two necessary aspects of adaptive behavior namely, fine-tuning of motor-control and striving for desired outcomes. Finally, we embed the findings in the recently proposed Control-based response selection (CBRS) framework and elaborate on its potential for understanding motor-learning processes in developing infants.

A match does not make a sense: on the sufficiency of the comparator model for explaining the sense of agency

The development of a sense of agency is indispensable for a cognitive entity (biological or artificial) to become a cognitive agent. In developmental psychology, researchers have taken inspiration from adult cognitive psychology and neuroscience literature and use the comparator model to assess the presence of a sense of agency in early infancy. Similarly, robotics researchers have taken components of the proposed mechanism in attempts to build a sense of agency into artificial systems. In this article, we identify an invalidating theoretical flaw in the reasoning underlying this conversion from adult studies to developmental science and cognitive systems research, rooted in an oversight in the conceptualization of the comparator model as currently used in experimental practice. In these experiments, the emphasis has been put solely on testing for a match between predicted and observed sensory consequences. We argue that the match by itself can exclusively generate a simple categorization or a representation of equality between predicted and observed sensory consequences, both of which are insufficient to generate the causal representations required for a sense of agency. Consequently, the comparator model, as it has been described in the context of the sense of agency and as it is commonly used in experimental designs, is insufficient to generate the sense of agency: infants and robots require more than developing the ability to match predicted and observed sensory consequences for a sense of agency. We conclude with outlining possible solutions and future directions for researchers in developmental science and artificial intelligence.