Kinematics and observer-animator kinematic similarity predict mental state attribution from Heider-Simmel style animations

Revisiting the Heider and Simmel experiment for social meaning attribution in virtual reality

In their seminal experiment in 1944, Heider and Simmel revealed that humans have a pronounced tendency to impose narrative meaning even in the presence of simple animations of geometric shapes. Despite the shapes having no discernible features or emotions, participants attributed strong social context, meaningful interactions, and even emotions to them. This experiment, run on traditional 2D displays has since had a significant impact on fields ranging from psychology to narrative storytelling. Virtual Reality (VR), on the other hand, offers a significantly new viewing paradigm, a fundamentally different type of experience with the potential to enhance presence, engagement and immersion. In this work, we explore and analyze to what extent the findings of the original experiment by Heider and Simmel carry over into a VR setting. We replicate such experiment in both traditional 2D displays and with a head mounted display (HMD) in VR, and use both subjective (questionnaire-based) and objective (eye-tracking) metrics to record the observers' visual behavior. We perform a thorough analysis of this data, and propose novel metrics for assessing the observers' visual behavior. Our questionnaire-based results suggest that participants who viewed the animation through a VR headset developed stronger emotional connections with the geometric shapes than those who viewed it on a traditional 2D screen. Additionally, the analysis of our eye-tracking data indicates that participants who watched the animation in VR exhibited fewer shifts in gaze, suggesting greater engagement with the action. However, we did not find evidence of differences in how subjects perceived the roles of the shapes, with both groups interpreting the animation's plot at the same level of accuracy. Our findings may have important implications for future psychological research using VR, especially regarding our understanding of social cognition and emotions.

Disruption of dopamine D2/D3 system function impairs the human ability to understand the mental states of other people

Difficulties in reasoning about others' mental states (i.e., mentalising/Theory of Mind) are highly prevalent among disorders featuring dopamine dysfunctions (e.g., Parkinson's disease) and significantly affect individuals' quality of life. However, due to multiple confounding factors inherent to existing patient studies, currently little is known about whether these sociocognitive symptoms originate from aberrant dopamine signalling or from psychosocial changes unrelated to dopamine. The present study, therefore, investigated the role of dopamine in modulating mentalising in a sample of healthy volunteers. We used a double-blind, placebo-controlled procedure to test the effect of the D2/D3 antagonist haloperidol on mental state attribution, using an adaptation of the Heider and Simmel (1944) animations task. On 2 separate days, once after receiving 2.5 mg haloperidol and once after receiving placebo, 33 healthy adult participants viewed and labelled short videos of 2 triangles depicting mental state (involving mentalistic interaction wherein 1 triangle intends to cause or act upon a particular mental state in the other, e.g., surprising) and non-mental state (involving reciprocal interaction without the intention to cause/act upon the other triangle's mental state, e.g., following) interactions. Using Bayesian mixed effects models, we observed that haloperidol decreased accuracy in labelling both mental and non-mental state animations. Our secondary analyses suggest that dopamine modulates inference from mental and non-mental state animations via independent mechanisms, pointing towards 2 putative pathways underlying the dopaminergic modulation of mental state attribution: action representation and a shared mechanism supporting mentalising and emotion recognition. We conclude that dopaminergic pathways impact Theory of Mind, at least indirectly. Our results have implications for the neurochemical basis of sociocognitive difficulties in patients with dopamine dysfunctions and generate new hypotheses about the specific dopamine-mediated mechanisms underlying social cognition.

Interactionally Embedded Gestalt Principles of Multimodal Human Communication

Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g., parallel or temporally misaligned), must be rapidly processed as a coherent message by the receiver. In this contribution, we introduce the notion of interactionally embedded, affordance-driven gestalt perception as a framework that can explain how this rapid processing of multimodal signals is achieved as efficiently as it is. We discuss empirical evidence showing how basic principles of gestalt perception can explain some aspects of unimodal phenomena such as verbal language processing and visual scene perception but require additional features to explain multimodal human communication. We propose a framework in which high-level gestalt predictions are continuously updated by incoming sensory input, such as unfolding speech and visual signals. We outline the constituent processes that shape high-level gestalt perception and their role in perceiving relevance and prägnanz. Finally, we provide testable predictions that arise from this multimodal interactionally embedded gestalt-perception framework. This review and framework therefore provide a theoretically motivated account of how we may understand the highly complex, multimodal behaviors inherent in natural social interaction.