Identifying others' informative intentions from movement kinematics

Communicative modulations of early action components support the prediction of distal goals

The successful unfolding of many social interactions relies on our capacity to predict other people's action goals, whether these are proximal (i.e., immediate) or distal (i.e., upcoming). The present set of studies asks whether observers can predict the distal goal of two-step action sequences when presented with communicative modulations of the first movement component of the sequence. We conducted three online experiments in which we presented participants with animations of a box moving to a first target location before moving onwards to a final, either near or far, target location. The second movement component and the target locations were occluded. After observing the first movement, participants were asked to select the most likely final target location, i.e., the distal goal of the sequence. Experiment 1 showed that participants relied on the velocity modulations of the first movement to infer the distal goal. The results of Experiment 2 indicated that such predictions of distal goals are possible even when the second movement in the sequence does not contain any velocity information, thus suggesting that the information present in the first movement plays the major role in the process of linking movements to their distal goals. However, Experiment 3 showed that under some circumstances the second movement can also contribute to how observers predict a distal goal. We discuss these results in terms of the underlying simulation processes that enable observers to predict a distal goal from the observation of proximal communicative modulations.

Action planning and execution cues influence economic partner choice

Prudently choosing who to interact with and who to avoid is an important ability to ensure that we benefit from a cooperative interaction. While the role of others' preferences, attributes, and values in partner choice have been established (Rossetti, Hilbe & Hauser, 2022), much less is known about whether the manner in which a potential partner plans and implements a decision provides helpful cues for partner choice. We used a partner choice paradigm in which participants chose who to interact with in the Prisoners' Dilemma. Before choosing a cooperation partner, participants were presented with information about the potential partners' decision-related actions in another round of the Prisoners' Dilemma. They received either information about the potential partners' planning during decision making (i.e., decision-time; Experiment 1) or action execution during decision implementation (i.e., movement directness; Experiment 2). Across both games, participants preferred to interact with those who planned actions quickly or executed actions with direct and smooth movements, indicating that they were cooperating confidently and without deliberation. This demonstrates that action cues present in either the planning or implementation of economic decisions influence partner choice. We discuss implications of this finding for human decision-making and perception-action coupling in action understanding.

Individual differences in cooperative and competitive play strategies

INTRODUCTION

Cooperation and competition are common in social interactions. It is not clear how individual differences in personality may predict performance strategies under these two contexts. We evaluated whether instructions to play cooperatively and competitively would differentially affect dyads playing a Pong video game. We hypothesized that instructions to play cooperatively would result in lower overall points scored and differences in paddle control kinematics relative to when participants were instructed to play competitively. We also predicted that higher scores in prosociality and Sportspersonship would be related to better performance during cooperative than competitive conditions.

METHODS

Pairs of participants played a Pong video game under cooperative and competitive instructions. During competitive trials, participants were instructed to score more points against one another to win the game. During the cooperative trials, participants were instructed to work together to score as few points against one another as possible. After game play, each participant completed surveys so we could measure their trait prosociality and Sportspersonship.

RESULTS

Condition was a significant predictor of where along the paddle participants hit the ball, which controlled ball exit angles. Specifically, during cooperation participants concentrated ball contacts on the paddle towards the center to produce more consistent rebound angles. We found a significant correlation of Sex and the average points scored by participants during cooperative games, competitive games, and across all trials. Sex was also significantly correlated with paddle kinematics during cooperative games. The overall scores on the prosociality and Sportspersonship surveys were not significantly correlated with the performance outcomes in cooperative and competitive games. The dimension of prosociality assessing empathic concern was significantly correlated with performance outcomes during cooperative video game play.

DISCUSSION

No Sportspersonship survey score was able to predict cooperative or competitive game performance, suggesting that Sportspersonship personality assessments are not reliable predictors of cooperative or competitive behaviors translated to a virtual game setting. Survey items and dimensions probing broader empathic concern may be more effective predictors of cooperative and competitive performance during interactive video game play. Further testing is encouraged to assess the efficacy of prosocial personality traits as predictors of cooperative and competitive video game behavior.

What's your point? Insights from virtual reality on the relation between intention and action in the production of pointing gestures

Human communication involves the process of translating intentions into communicative actions. But how exactly do our intentions surface in the visible communicative behavior we display? Here we focus on pointing gestures, a fundamental building block of everyday communication, and investigate whether and how different types of underlying intent modulate the kinematics of the pointing hand and the brain activity preceding the gestural movement. In a dynamic virtual reality environment, participants pointed at a referent to either share attention with their addressee, inform their addressee, or get their addressee to perform an action. Behaviorally, it was observed that these different underlying intentions modulated how long participants kept their arm and finger still, both prior to starting the movement and when keeping their pointing hand in apex position. In early planning stages, a neurophysiological distinction was observed between a gesture that is used to share attitudes and knowledge with another person versus a gesture that mainly uses that person as a means to perform an action. Together, these findings suggest that our intentions influence our actions from the earliest neurophysiological planning stages to the kinematic endpoint of the movement itself.

Kinematic priming of action predictions

The ability to anticipate what others will do next is crucial for navigating social, interactive environments. Here, we develop an experimental and analytical framework to measure the implicit readout of prospective intention information from movement kinematics. Using a primed action categorization task, we first demonstrate implicit access to intention information by establishing a novel form of priming, which we term kinematic priming: subtle differences in movement kinematics prime action prediction. Next, using data collected from the same participants in a forced-choice intention discrimination task 1 h later, we quantify single-trial intention readout-the amount of intention information read by individual perceivers in individual kinematic primes-and assess whether it can be used to predict the amount of kinematic priming. We demonstrate that the amount of kinematic priming, as indexed by both response times (RTs) and initial fixations to a given probe, is directly proportional to the amount of intention information read by the individual perceiver at the single-trial level. These results demonstrate that human perceivers have rapid, implicit access to intention information encoded in movement kinematics and highlight the potential of our approach to reveal the computations that permit the readout of this information with single-subject, single-trial resolution.

Learning from others is good, with others is better: the role of social interaction in human acquisition of new knowledge

Learning in humans is highly embedded in social interaction: since the very early stages of our lives, we form memories and acquire knowledge about the world from and with others. Yet, within cognitive science and neuroscience, human learning is mainly studied in isolation. The focus of past research in learning has been either exclusively on the learner or (less often) on the teacher, with the primary aim of determining developmental trajectories and/or effective teaching techniques. In fact, social interaction has rarely been explicitly taken as a variable of interest, despite being the medium through which learning occurs, especially in development, but also in adulthood. Here, we review behavioural and neuroimaging research on social human learning, specifically focusing on cognitive models of how we acquire semantic knowledge from and with others, and include both developmental as well as adult work. We then identify potential cognitive mechanisms that support social learning, and their neural correlates. The aim is to outline key new directions for experiments investigating how knowledge is acquired in its ecological niche, i.e. socially, within the framework of the two-person neuroscience approach. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Understanding others' distal goals from proximal communicative actions

Many social interactions require individuals to coordinate their actions and to inform each other about their goals. Often these goals concern an immediate (i.e., proximal) action, as when people give each other a brief handshake, but they sometimes also refer to a future (i.e. distal) action, as when football players perform a passing sequence. The present study investigates whether observers can derive information about such distal goals by relying on kinematic modulations of an actor's instrumental actions. In Experiment 1 participants were presented with animations of a box being moved at different velocities towards an apparent endpoint. The distal goal, however, was for the object to be moved past this endpoint, to one of two occluded target locations. Participants then selected the location which they considered the likely distal goal of the action. As predicted, participants were able to detect differences in movement velocity and, based on these differences, systematically mapped the movements to the two distal goal locations. Adding a distal goal led to more variation in the way participants mapped the observed movements onto different target locations. The results of Experiments 2 and 3 indicated that this cannot be explained by difficulties in perceptual discrimination. Rather, the increased variability likely reflects differences in interpreting the underlying connection between proximal communicative actions and distal goals. The present findings extend previous research on sensorimotor communication by demonstrating that communicative action modulations are not restricted to predicting proximal goals but can also be used to infer more distal goals.

Measuring the prediction of observed actions using an occlusion paradigm: Comparing autistic and non-autistic adults

Action prediction involves observing and predicting the actions of others and plays an important role in social cognition and interacting with others. It is thought to use simulation, whereby the observers use their own motor system to predict the observed actions. As individuals diagnosed with autism are characterized by difficulties understanding the actions of others and motor coordination issues, it is possible that action prediction ability is altered in this population. This study compared action prediction ability between 20 autistic and 22 non-autistic adults using an occlusion paradigm. Participants watched different videos of a female actor carrying out everyday actions. During each video, the action was transiently occluded by a gray rectangle for 1000 ms. During occlusions, the video was allowed to continue as normal or was moved forward (i.e., appearing to continue too far ahead) or moved backwards (i.e., appearing to continue too far behind). Participants were asked to indicate after each occlusion whether the action continued with the correct timing or was too far ahead/behind. Autistic individuals were less accurate than non-autistic individuals, particularly when the video was too far behind. A trend analysis suggested that autistic participants were more likely to judge too far behind occlusions as being in time. These preliminary results suggest that prediction ability may be altered in autistic adults, potentially due to slower simulation or a delayed onset of these processes. LAY SUMMARY: When we observe other people performing everyday actions, we use their movements to help us understand and predict what they are doing. In this study, we found that autistic compared to non-autistic adults were slightly less accurate at predicting other people's actions. These findings help to unpick the different ways that social understanding is affected in autism.

Expression unleashed: The evolutionary and cognitive foundations of human communication

Human expression is open-ended, versatile, and diverse, ranging from ordinary language use to painting, from exaggerated displays of affection to micro-movements that aid coordination. Here we present and defend the claim that this expressive diversity is united by an interrelated suite of cognitive capacities, the evolved functions of which are the expression and recognition of informative intentions. We describe how evolutionary dynamics normally leash communication to narrow domains of statistical mutual benefit, and how expression is unleashed in humans. The relevant cognitive capacities are cognitive adaptations to living in a partner choice social ecology; and they are, correspondingly, part of the ordinarily developing human cognitive phenotype, emerging early and reliably in ontogeny. In other words, we identify distinctive features of our species' social ecology to explain how and why humans, and only humans, evolved the cognitive capacities that, in turn, lead to massive diversity and open-endedness in means and modes of expression. Language use is but one of these modes of expression, albeit one of manifestly high importance. We make cross-species comparisons, describe how the relevant cognitive capacities can evolve in a gradual manner, and survey how unleashed expression facilitates not only language use, but also novel behaviour in many other domains too, focusing on the examples of joint action, teaching, punishment, and art, all of which are ubiquitous in human societies but relatively rare in other species. Much of this diversity derives from graded aspects of human expression, which can be used to satisfy informative intentions in creative and new ways. We aim to help reorient cognitive pragmatics, as a phenomenon that is not a supplement to linguistic communication and on the periphery of language science, but rather the foundation of the many of the most distinctive features of human behaviour, society, and culture.

Moral dynamics: Grounding moral judgment in intuitive physics and intuitive psychology

When holding others morally responsible, we care about what they did, and what they thought. Traditionally, research in moral psychology has relied on vignette studies, in which a protagonist's actions and thoughts are explicitly communicated. While this research has revealed what variables are important for moral judgment, such as actions and intentions, it is limited in providing a more detailed understanding of exactly how these variables affect moral judgment. Using dynamic visual stimuli that allow for a more fine-grained experimental control, recent studies have proposed a direct mapping from visual features to moral judgments. We embrace the use of visual stimuli in moral psychology, but question the plausibility of a feature-based theory of moral judgment. We propose that the connection from visual features to moral judgments is mediated by an inference about what the observed action reveals about the agent's mental states, and what causal role the agent's action played in bringing about the outcome. We present a computational model that formalizes moral judgments of agents in visual scenes as computations over an intuitive theory of physics combined with an intuitive theory of mind. We test the model's quantitative predictions in three experiments across a wide variety of dynamic interactions.

Social epistemic actions

We consider the ways humans engage in social epistemic actions, to guide each other's attention, prediction, and learning processes towards salient information, at the timescale of online social interaction and joint action. This parallels the active guidance of other's attention, prediction, and learning processes at the longer timescale of niche construction and cultural practices, as discussed in the target article.

Adults imitate to send a social signal

Humans frequently imitate each other's actions with high fidelity, and different reasons have been proposed for why they do so. Here we test the hypothesis that imitation can act as a social signal, with imitation occurring with greater fidelity when a participant is being watched. In a preregistered study, 30 pairs of naïve participants played an augmented-reality game involving moving blocks in space. We compared imitation fidelity between trials where the leader watched the followers' action, and trials where the leader did not watch. Followers imitated the trajectory height demonstrated by the leader, and critically, the strength of this correlation was greater in trials where the follower knew the leader was watching them. This suggests that followers spontaneously used imitation as a social signal in a nonverbal interaction task, supporting socio-communicative hypotheses of imitation.