Goal saliency boosts infants’ action prediction for human manual actions, but not for mechanical claws

What is in an action? Preschool children predict that agents take previous paths and not previous goals

Developmental science has experienced a vivid debate on whether young children prioritize goals over means in their prediction of others' actions. Influential developmental theories highlight the role of goal objects for action understanding. Yet, recent infant studies report evidence for the opposite. The empirical evidence is therefore inconclusive. The current study advanced this debate by assessing preschool children's verbal predictions of others' actions. In five experiments (N = 302), we investigated whether preschool children and adults predict agents to move towards their previous goal (that is, show goal-related predictions) or predict agents to move along the same movement path that they pursued before. While Experiments 1a, 1b and 1c presented young children and adults with animated agents, Experiments 2a and 2b presented participants with human grasping action. An integrative analysis across experiments revealed that children were more likely to predict the agent to move along the same movement path, Z = -4.574, p ≤ 0.0001 (r = 0.304). That is, preschool children were more likely to predict that agents would move along the same trajectory even though this action would lead to a new goal object. Thus, our findings suggest that young children's action prediction relies on the detection of spatial and movement information. Overall, we discuss our findings in terms of theoretical frameworks that conceive of action understanding as an umbrella term that comprises different forms and facets in which humans understand others' actions. RESEARCH HIGHLIGHTS: We investigated whether preschool children predict agents to move towards their previous goal or to move along the same movement path that they pursued before. Unlike adults, preschool children predicted that agents would move along the same trajectory even though this action would lead to a new goal. Adults' goal-based predictions were affected from contextual details, whereas children systematically made path-based predictions. Young children's action prediction relies on the detection of spatial and movement information.

Selective Action Prediction in Infancy Depending on Linguistic Cues: An EEG and Eyetracker Study

Humans' capacity to predict actions and to socially categorize individuals is at the basis of social cognition. Such capacities emerge in early infancy. By 6 months of age, infants predict others' reaching actions considering others' epistemic state. At a similar age, infants are biased to attend to and interact with more familiar individuals, considering adult-like social categories such as the language people speak. We report that these two core processes are interrelated early on in infancy. In a belief-based action prediction task, 6-month-old infants (males and females) presented with a native speaker generated online predictions about the agent's actions, as revealed by the activation of participants' sensorimotor areas before the agent's movement. However, infants who were presented with a foreign speaker did not recruit their motor system before the agent's action. The eyetracker analysis provided further evidence that linguistic group familiarity influences how infants predict others' actions, as only infants presented with a native speaker modified their attention to the stimuli as a function of the agent's forthcoming behavior. The current findings suggest that infants' emerging capacity to predict others' actions is modulated by social cues, such as others' linguistic group. A facilitation to predict and encode the actions of native speakers relative to foreign speakers may explain, in part, why infants preferentially attend to, imitate, and learn from the actions of native speakers.

Infants’ Prediction of Humanoid Robot’s Goal-Directed Action

Several studies have shown that infants anticipate human goal-directed actions, but not robot’s ones. However, the studies focusing on the robot goal-directed actions have mainly analyzed the effect of mechanical arms on infant’s attention. To date, the prediction of goal-directed actions in infants has not yet been studied when the agent is a humanoid robot. Given this lack of evidence in infancy research, the present study aims at analyzing infants’ action anticipation of both a human’s and a humanoid robot’s goal-directed action. Data were acquired on thirty 17-month-old infants, watching four video clips, where either a human or a humanoid robot performed a goal-directed action, i.e. reaching a target. Infants looking behavior was measured through the eye-tracking technique. The results showed that infants anticipated the goal-directed action of both the human and the robot and there were no differences in the anticipatory gaze behavior between the two agents. Furthermore, the findings indicated different attentional patterns for the human and the robot, showing a greater attention paid to the robot's face than the human’s face. Overall, the results suggest that 17-month-old infants may infer also humanoid robot’ underlying action goals.

The role of social signals in segmenting observed actions in 18-month-old children

Learning about actions requires children to identify the boundaries of an action and its units. Whereas some action units are easily identified, parents can support children's action learning by adjusting the presentation and using social signals. However, currently, little is understood regarding how children use these signals to learn actions. In the current study, we investigate the possibility that communicative signals are a particularly suitable cue for segmenting events. We investigated this hypothesis by presenting 18-month-old children (N = 60) with short action sequences consisting of toy animals either hopping or sliding across a board into a house, but interrupting this two-step sequence either (a) using an ostensive signal as a segmentation cue, (b) using a non-ostensive segmentation cue and (c) without additional segmentation information between the actions. Marking the boundary using communicative signals increased children's imitation of the less salient sliding action. Imitation of the hopping action remained unaffected. Crucially, marking the boundary of both actions using a non-communicative control condition did not increase imitation of either action. Communicative signals might be particularly suitable in segmenting non-salient actions that would otherwise be perceived as part of another action or as non-intentional. These results provide evidence of the importance of ostensive signals at event boundaries in scaffolding children's learning.

The Impact of Action Effects on Infants' Predictive Gaze Shifts for a Non-Human Grasping Action at 7, 11, and 18 Months

During the observation of goal-directed actions, infants usually predict the goal at an earlier age when the agent is familiar (e.g., human hand) compared to unfamiliar (e.g., mechanical claw). These findings implicate a crucial role of the developing agentive self for infants’ processing of others’ action goals. Recent theoretical accounts suggest that predictive gaze behavior relies on an interplay between infants’ agentive experience (top-down processes) and perceptual information about the agent and the action-event (bottom-up information; e.g., agency cues). The present study examined 7-, 11-, and 18-month-old infants’ predictive gaze behavior for a grasping action performed by an unfamiliar tool, depending on infants’ age-related action knowledge about tool-use and the display of the agency cue of producing a salient action effect. The results are in line with the notion of a systematic interplay between experience-based top-down processes and cue-based bottom-up information: Regardless of the salient action effect, predictive gaze shifts did not occur in the 7-month-olds (least experienced age group), but did occur in the 18-month-olds (most experienced age group). In the 11-month-olds, however, predictive gaze shifts occurred only when a salient action effect was presented. This sheds new light on how the developing agentive self, in interplay with available agency cues, supports infants’ action-goal prediction also for observed tool-use actions.

Emergent Goal-Anticipatory Gaze in Infants via Event-Predictive Learning and Inference

From about 7 months of age onward, infants start to reliably fixate the goal of an observed action, such as a grasp, before the action is complete. The available research has identified a variety of factors that influence such goal-anticipatory gaze shifts, including the experience with the shown action events and familiarity with the observed agents. However, the underlying cognitive processes are still heavily debated. We propose that our minds (i) tend to structure sensorimotor dynamics into probabilistic, generative event-predictive, and event boundary predictive models, and, meanwhile, (ii) choose actions with the objective to minimize predicted uncertainty. We implement this proposition by means of event-predictive learning and active inference. The implemented learning mechanism induces an inductive, event-predictive bias, thus developing schematic encodings of experienced events and event boundaries. The implemented active inference principle chooses actions by aiming at minimizing expected future uncertainty. We train our system on multiple object-manipulation events. As a result, the generation of goal-anticipatory gaze shifts emerges while learning about object manipulations: the model starts fixating the inferred goal already at the start of an observed event after having sampled some experience with possible events and when a familiar agent (i.e., a hand) is involved. Meanwhile, the model keeps reactively tracking an unfamiliar agent (i.e., a mechanical claw) that is performing the same movement. We qualitatively compare these modeling results to behavioral data of infants and conclude that event-predictive learning combined with active inference may be critical for eliciting goal-anticipatory gaze behavior in infants.

Adults, but not preschoolers or toddlers integrate situational constraints in their action anticipations: a developmental study on the flexibility of anticipatory gaze

Recent theories stress the role of situational information in understanding others' behaviour. For example, the predictive coding framework assumes that people take contextual information into account when anticipating other's actions. Likewise, the teleological stance theory assumes an early developing ability to consider situational constraints in action prediction. The current study investigates, over a wide age range, whether humans flexibly integrate situational constraints in their action anticipations. By means of an eye-tracking experiment, 2-year-olds, 5-year-olds, younger and older adults (together N = 181) observed an agent repeatedly taking one of two paths to reach a goal. Then, this path became blocked, and for test trials only the other path was passable. Results demonstrated that in test trials younger and older adults anticipated that the agent would take the continuous path, indicating that they took the situational constraints into account. In contrast, 2- and 5-year-olds anticipated that the agent would take the blocked path, indicating that they still relied on the agent's previous observed behaviour and-contrary to claims by the teleological stance theory-did not take the situational constraints into account. The results highlight developmental changes in human's ability to include situational constraints in their visual anticipations. Overall, the study contributes to theories on predictive coding and the development of action understanding.

Do Children and Adults with Autism Spectrum Condition Anticipate Others' Actions as Goal-Directed? A Predictive Coding Perspective

An action's end state can be anticipated by considering the agent's goal, or simply by projecting the movement trajectory. Theories suggest that individuals with autism spectrum condition (ASC) have difficulties anticipating other's goal-directed actions, caused by an impairment using prior information. We examined whether children, adolescents and adults with and without ASC visually anticipate another's action based on its goal or movement trajectory by presenting participants an agent repeatedly taking different paths to reach the same of two targets. The ASC group anticipated the goal and not just the movement pattern, but needed more time to perform goal-directed anticipations. Results are in line with predictive coding accounts, claiming that the use of prior information is impaired in ASC.

The impact of salient action effects on 6-, 7-, and 11-month-olds' goal-predictive gaze shifts for a human grasping action

When infants observe a human grasping action, experience-based accounts predict that all infants familiar with grasping actions should be able to predict the goal regardless of additional agency cues such as an action effect. Cue-based accounts, however, suggest that infants use agency cues to identify and predict action goals when the action or the agent is not familiar. From these accounts, we hypothesized that younger infants would need additional agency cues such as a salient action effect to predict the goal of a human grasping action, whereas older infants should be able to predict the goal regardless of agency cues. In three experiments, we presented 6-, 7-, and 11-month-olds with videos of a manual grasping action presented either with or without an additional salient action effect (Exp. 1 and 2), or we presented 7-month-olds with videos of a mechanical claw performing a grasping action presented with a salient action effect (Exp. 3). The 6-month-olds showed tracking gaze behavior, and the 11-month-olds showed predictive gaze behavior, regardless of the action effect. However, the 7-month-olds showed predictive gaze behavior in the action-effect condition, but tracking gaze behavior in the no-action-effect condition and in the action-effect condition with a mechanical claw. The results therefore support the idea that salient action effects are especially important for infants' goal predictions from 7 months on, and that this facilitating influence of action effects is selective for the observation of human hands.

Action Prediction Allows Hypothesis Testing via Internal Forward Models at 6 Months of Age

We propose that action prediction provides a cornerstone in a learning process known as internal forward models. According to this suggestion infants' predictions (looking to the mouth of someone moving a spoon upward) will moments later be validated or proven false (spoon was in fact directed toward a bowl), information that is directly perceived as the distance between the predicted and actual goal. Using an individual difference approach we demonstrate that action prediction correlates with the tendency to react with surprise when social interactions are not acted out as expected (action evaluation). This association is demonstrated across tasks and in a large sample (n = 118) at 6 months of age. These results provide the first indication that infants might rely on internal forward models to structure their social world. Additional analysis, consistent with prior work and assumptions from embodied cognition, demonstrates that the latency of infants' action predictions correlate with the infant's own manual proficiency.