Action-space coding in social contexts

Abstract and concrete concepts in conversation

Concepts allow us to make sense of the world. Most evidence on their acquisition and representation comes from studies of single decontextualized words and focuses on the opposition between concrete and abstract concepts (e.g., "bottle" vs. "truth"). A significant step forward in research on concepts consists in investigating them in online interaction during their use. Our study examines linguistic exchanges analyzing the differences between sub-kinds of concepts. Participants were submitted to an online task in which they had to simulate a conversational exchange by responding to sentences involving sub-kinds of concrete (tools, animals, food) and abstract concepts (PS, philosophical-spiritual; EMSS, emotional-social, PSTQ, physical-spatio-temporal-quantitative). We found differences in content: foods evoked interoception; tools and animals elicited materials, spatial, auditive features, confirming their sensorimotor grounding. PS and EMSS yielded inner experiences (e.g., emotions, cognitive states, introspections) and opposed PSTQ, tied to visual properties and concrete agency. More crucially, the various concepts elicited different interactional dynamics: more abstract concepts generated higher uncertainty and more interactive exchanges than concrete ones. Investigating concepts in situated interactions opens new possibilities for studying conceptual knowledge and its pragmatic and social aspects.

Wearing the face mask affects our social attention over space

Recent studies suggest that covering the face inhibits the recognition of identity and emotional expressions. However, it might also make the eyes more salient, since they are a reliable index to orient our social and spatial attention. This study investigates (1) whether the pervasive interaction with people with face masks fostered by the COVID-19 pandemic modulates the processing of spatial information essential to shift attention according to other’s eye-gaze direction (i.e., gaze-cueing effect: GCE), and (2) whether this potential modulation interacts with motor responses (i.e., Simon effect). Participants were presented with face cues orienting their gaze to a congruent or incongruent target letter location (gaze-cueing paradigm) while wearing a surgical mask (Mask), a patch (Control), or nothing (No-Mask). The task required to discriminate the identity of the lateralized target letters by pressing one of two lateralized response keys, in a corresponding or a non-corresponding position with respect to the target. Results showed that GCE was not modulated by the presence of the Mask, but it occurred in the No-Mask condition, confirming previous studies. Crucially, the GCE interacted with Simon effect in the Mask and Control conditions, though in different ways. While in the Mask condition the GCE emerged only when target and response positions corresponded (i.e., Simon-corresponding trials), in the Control condition it emerged only when they did not correspond (i.e., Simon-non-corresponding trials). These results indicate that people with face masks induce us to jointly orient our visual attention in the direction of the seen gaze (GCE) in those conditions resembling (or associated with) a general approaching behavior (Simon-corresponding trials). This is likely promoted by the fact that we tend to perceive wearing the mask as a personal safety measure and, thus, someone wearing the face mask is perceived as a trustworthy person. In contrast, people with a patch on their face can be perceived as more threatening, therefore inducing a GCE in those conditions associated with a general avoidance behavior (Simon-non-corresponding trials).

Human-like behavioral variability blurs the distinction between a human and a machine in a nonverbal Turing test

Variability is a property of biological systems, and in animals (including humans), behavioral variability is characterized by certain features, such as the range of variability and the shape of its distribution. Nevertheless, only a few studies have investigated whether and how variability features contribute to the ascription of humanness to robots in a human-robot interaction setting. Here, we tested whether two aspects of behavioral variability, namely, the standard deviation and the shape of distribution of reaction times, affect the ascription of humanness to robots during a joint action scenario. We designed an interactive task in which pairs of participants performed a joint Simon task with an iCub robot placed by their side. Either iCub could perform the task in a preprogrammed manner, or its button presses could be teleoperated by the other member of the pair, seated in the other room. Under the preprogrammed condition, the iCub pressed buttons with reaction times falling within the range of human variability. However, the distribution of the reaction times did not resemble a human-like shape. Participants were sensitive to humanness, because they correctly detected the human agent above chance level. When the iCub was controlled by the computer program, it passed our variation of a nonverbal Turing test. Together, our results suggest that hints of humanness, such as the range of behavioral variability, might be used by observers to ascribe humanness to a humanoid robot.

Hierarchical Integration of Communicative and Spatial Perspective‐Taking Demands in Sensorimotor Control of Referential Pointing

Recognized as a simple communicative behavior, referential pointing is cognitively complex because it invites a communicator to consider an addressee's knowledge. Although we know referential pointing is affected by addressees’ physical location, it remains unclear whether and how communicators’ inferences about addressees’ mental representation of the interaction space influence sensorimotor control of referential pointing. The communicative perspective‐taking task requires a communicator to point at one out of multiple referents either to instruct an addressee which one should be selected (communicative, COM) or to predict which one the addressee will select (non‐communicative, NCOM), based on either which referents can be seen (Level‐1 perspective‐taking, PT1) or how the referents were perceived (Level‐2 perspective‐taking, PT2) by the addressee. Communicators took longer to initiate the movements in PT2 than PT1 trials, and they held their pointing fingers for longer at the referent in COM than NCOM trials. The novel findings of this study pertain to trajectory control of the pointing movements. Increasing both communicative and perspective‐taking demands led to longer pointing trajectories, with an under‐additive interaction between those two experimental factors. This finding suggests that participants generate communicative behaviors that are as informative as required rather than overly exaggerated displays, by integrating communicative and perspective‐taking information hierarchically during sensorimotor control. This observation has consequences for models of human communication. It implies that the format of communicative and perspective‐taking knowledge needs to be commensurate with the movement dynamics controlled by the sensorimotor system.

ERP markers of action planning and outcome monitoring in human - robot interaction

The present study aimed to examine event-related potentials (ERPs) of action planning and outcome monitoring in human-robot interaction. To this end, participants were instructed to perform costly actions (i.e. losing points) to stop a balloon from inflating and to prevent its explosion. They performed the task alone (individual condition) or with a robot (joint condition). Similar to findings from human-human interactions, results showed that action planning was affected by the presence of another agent, robot in this case. Specifically, the early readiness potential (eRP) amplitude was larger in the joint, than in the individual, condition. The presence of the robot affected also outcome perception and monitoring. Our results showed that the P1/N1 complex was suppressed in the joint, compared to the individual condition when the worst outcome was expected, suggesting that the presence of the robot affects attention allocation to negative outcomes of one's own actions. Similarly, results also showed that larger losses elicited smaller feedback-related negativity (FRN) in the joint than in the individual condition. Taken together, our results indicate that the social presence of a robot may influence the way we plan our actions and also the way we monitor their consequences. Implications of the study for the human-robot interaction field are discussed.

The influence of the location of ordered symbols on the ordinal position effect: The involvement of the task performed

Since the ordinal position effect was identified, several studies have investigated its mechanism in various contexts; however, how the space location of ordinal symbols influences this effect remains unclear. Thus, the present study explored Chinese words representing the day before yesterday, yesterday, tomorrow, and the day after tomorrow as ordinal symbols to investigate how the stimulus space location influences the ordinal position effect across different task contexts. We randomly and equally presented days on the left or right location of a display and asked participants to perform a stimulus space location, a stimulus colour and a stimulus order classification task in three consecutive experiments, respectively. The results revealed that the spatial stimulus-response compatibility effect and Simon effect prevailed in the stimulus space location and colour classification task. Conversely, the ordinal position effect prevailed in the stimulus order classification task. These results suggested that (1) the spatial stimulus-response compatibility effect (or Simon effect) and the ordinal position effect cannot appear simultaneously in some experimental contexts and that (2) the task context predicted which of these effects prevailed. From these results, we conclude that the ordinal symbols could be coded depending on multiple reference frames, including spatial and non-spatial reference frame, and the use of the reference frame was mediated by the task context.

The role of the co-actor's response reachability in the joint Simon effect: remapping of working space by tool use

The Simon effect, that is the advantage of the spatial correspondence between stimulus and response locations when stimulus location is task irrelevant, occurs even when the task is performed by two participants, each performing a go/no-go task. This effect, known as the joint Simon effect, does not emerge when participants sit outside each other's peripersonal space, thus suggesting that the presence of an active confederate in peripersonal space might provide a reference for response coding. The present study investigated whether this finding is due to the distance separating the participants and/or to the distance separating each participant and the other agent's response. In two experiments, pairs of participants performed a social detection task sitting outside each other's arm reach, with response keys located close to the participants or outside arm reach. When the response key was located outside the participant's arm reach, he/she could reach it by means of a tool. In Experiment 1, by means of a tool, participants could reach their response key only, while in Experiment 2, they could reach also their co-agent's response key. The joint Simon effect did not emerge when participants could not reach the co-actor's response, while it emerged when they could potentially reach the other participant's response using the tool, but only when turn taking was required. These results may be taken as evidence that the possibility to reach and act upon the co-actor's response key may be at the bases of compatibility effects observed in joint action contexts requiring complementary responses.

Sharing Different Reference Frames: How Stimulus Setup and Task Setup Shape Egocentric and Allocentric Simon Effects

Different reference frames are used in daily life in order to structure the environment. The two-choice Simon task setting has been used to investigate how task-irrelevant spatial information influences human cognitive control. In recent studies, a Go/NoGo Simon task setting was used in order to divide the Simon task between a pair of participants. Yet, not only a human co-actor, but also even an attention-grabbing object can provide sufficient reference in order to reintroduce a Simon effect (SE) indicating cognitive conflict in Go/NoGo task settings. Interestingly, the SE could only occur when a reference point outside of the stimulus setup was available. The current studies exploited the dependency between different spatial reference frames (egocentric and allocentric) offered by the stimulus setup itself and the task setup (individual vs. joint Go/NoGot task setting). Two studies (Experiments 1 and 2) were carried out along with a human co-actor. Experiment 3 used an attention-grabbing object instead. The egocentric and allocentric SEs triggered by different features of the stimulus setup (global vs. local) were modulated by the task setup. When interacting with a human co-actor, an egocentric SE was found for global features of the stimulus setup (i.e., stimulus position on the screen). In contrast, an allocentric SE was yielded in the individual task setup illustrating the relevance of more local features of the stimulus setup (i.e., the manikin’s ball position). Results point toward salience shifts between different spatial reference frames depending on the nature of the task setup.

Response Coordination Emerges in Cooperative but Not Competitive Joint Task

Effective social interactions rely on humans' ability to attune to others within social contexts. Recently, it has been proposed that the emergence of shared representations, as indexed by the Joint Simon effect (JSE), might result from interpersonal coordination (Malone et al., 2014). The present study aimed at examining interpersonal coordination in cooperative and competitive joint tasks. To this end, in two experiments we investigated response coordination, as reflected in instantaneous cross-correlation, when co-agents cooperate (Experiment 1) or compete against each other (Experiment 2). In both experiments, participants performed a go/no-go Simon task alone and together with another agent in two consecutive sessions. In line with previous studies, we found that social presence differently affected the JSE under cooperative and competitive instructions. Similarly, cooperation and competition were reflected in co-agents response coordination. For the cooperative session (Experiment 1), results showed higher percentage of interpersonal coordination for the joint condition, relative to when participants performed the task alone. No difference in the coordination of responses occurred between the individual and the joint conditions when co-agents were in competition (Experiment 2). Finally, results showed that interpersonal coordination between co-agents implies the emergence of the JSE. Taken together, our results suggest that shared representations seem to be a necessary, but not sufficient, condition for interpersonal coordination.

Joint Action: Mental Representations, Shared Information and General Mechanisms for Coordinating with Others

In joint action, multiple people coordinate their actions to perform a task together. This often requires precise temporal and spatial coordination. How do co-actors achieve this? How do they coordinate their actions toward a shared task goal? Here, we provide an overview of the mental representations involved in joint action, discuss how co-actors share sensorimotor information and what general mechanisms support coordination with others. By deliberately extending the review to aspects such as the cultural context in which a joint action takes place, we pay tribute to the complex and variable nature of this social phenomenon.