Concurrent Cortical Representations of Function- and Size-Related Object Affordances: An fMRI Study

Human brain dynamics dissociate early perceptual and late motor-related stages of affordance processing

Affordances, the opportunities for action offered by the environment to an agent, are vital for meaningful behaviour and exist in every interaction with the environment. There is an ongoing debate in the field about whether the perception of affordances is an automated process. Some studies suggest that affordance perception is an automated process that is independent from the visual context and bodily interaction with the environment, whereas others argue that it is modulated by the visual and motor context in which affordances are perceived. The present paper aims to resolve this debate by examining affordance automaticity from the perspective of sensorimotor time windows. To investigate the impact of different forms of bodily interactions with an environment, that is, the movement context (physical vs. joystick movement), we replicated a previous study on affordance perception in which participants actively moved through differently wide doors in an immersive 3D virtual environment. In the present study, we displayed the same environment on a 2D screen with participants moving through doors of different widths using the keys on a standard keyboard. We compared components of the event-related potential (ERP) from the continuously recorded electroencephalogram (EEG) that were previously reported to be related to affordance perception of architectural transitions (passable and impassable doors). Comparing early sensory and later motor-related ERPs, our study replicated ERPs reflecting early affordance perception but found differences in later motor-related components. These results indicate a shift from automated perception of affordances during early sensorimotor time windows to movement context dependence of affordance perception at later stages, suggesting that affordance perception is a dynamic and flexible process that changes over sensorimotor stages.

Body size as a metric for the affordable world

The physical body of an organism serves as a vital interface for interactions with its environment. Here, we investigated the impact of human body size on the perception of action possibilities (affordances) offered by the environment. We found that the body size delineated a distinct boundary on affordances, dividing objects of continuous real-world sizes into two discrete categories with each affording distinct action sets. Additionally, the boundary shifted with imagined body sizes, suggesting a causal link between body size and affordance perception. Intriguingly, ChatGPT, a large language model lacking physical embodiment, exhibited a modest yet comparable affordance boundary at the scale of human body size, suggesting the boundary is not exclusively derived from organism-environment interactions. A subsequent fMRI experiment offered preliminary evidence of affordance processing exclusively for objects within the body size range, but not for those beyond. This suggests that only objects capable of being manipulated are the objects capable of offering affordance in the eyes of an organism. In summary, our study suggests a novel definition of object-ness in an affordance-based context, advocating the concept of embodied cognition in understanding the emergence of intelligence constrained by an organism's physical attributes.

Sustainable materials: a linking bridge between material perception, affordance, and aesthetics

The perception of material properties, which refers to the way in which individuals perceive and interpret materials through their sensory experiences, plays a crucial role in our interaction with the environment. Affordance, on the other hand, refers to the potential actions and uses that materials offer to users. In turn, the perception of the affordances is modulated by the aesthetic appreciation that individuals experience when interacting with the environment. Although material perception, affordances, and aesthetic appreciation are recognized as essential to fostering sustainability in society, only a few studies have investigated this subject matter systematically and their reciprocal influences. This scarcity is partially due to the challenges offered by the complexity of combining interdisciplinary topics that explore interactions between various disciplines, such as psychophysics, neurophysiology, affective science, aesthetics, and social and environmental sciences. Outlining the main findings across disciplines, this review highlights the pivotal role of material perception in shaping sustainable behaviors. It establishes connections between material perception, affordance, aesthetics, and sustainability, emphasizing the need for interdisciplinary research and integrated approaches in environmental psychology. This integration is essential as it can provide insight into how to foster sustainable and durable changes.

A focus on the multiple interfaces between action and perception and their neural correlates

Successful behaviour relies on the appropriate interplay between action and perception. The well-established dorsal and ventral stream theories depicted two distinct functional pathways for the processes of action and perception, respectively. In physiological conditions, the two pathways closely cooperate in order to produce successful adaptive behaviour. As the coupling between perception and action exists, this requires an interface that is responsible for a common reading of the two functions. Several studies have proposed different types of perception and action interfaces, suggesting their role in the creation of the shared interaction channel. In the present review, we describe three possible perception and action interfaces: i) the motor code, including common coding approaches, ii) attention, and iii) object affordance; we highlight their potential neural correlates. From this overview, a recurrent neural substrate that underlies all these interface functions appears to be crucial: the parieto-frontal circuit. This network is involved in the mirror mechanism which underlies the perception and action interfaces identified as common coding and motor code theories. The same network is also involved in the spotlight of attention and in the encoding of potential action towards objects; these are manifested in the perception and action interfaces for common attention and object affordance, respectively. Within this framework, most studies were dedicated to the description of the role of the inferior parietal lobule; growing evidence, however, suggests that the superior parietal lobule also plays a crucial role in the interplay between action and perception. The present review proposes a novel model that is inclusive of the superior parietal regions and their relative contribution to the different action and perception interfaces.

The roles of motion, gesture, and embodied action in the processing of mathematical concepts

This article discusses perspective and frame of reference in the metaphorical description of mathematical concepts in terms of motions, gestures, and embodied actions. When a mathematical concept is described metaphorically in terms of gestures, embodied actions, or fictive motions, the motor system comes into play to ground and understand that concept. Every motion, gesture, or embodied action involves a perspective and a frame of reference. The flexibility in taking perspective and frame of reference allows people to embody a mathematical concept or idea in various ways. Based on the findings of past studies, it is suggested that the graphical representation of a mathematical concept may activate those areas of the motor system that are involved in the production of that graphical representation. This is supported by studies showing that when observers look at a painting or handwritten letters, they simulate the painter's or writer's hand movements during painting or writing. Likewise, the motor system can contribute to the grounding of abstract mathematical concepts, such as functions, numbers, and arithmetic operations.

Sound-Action Symbolism

Recent evidence has shown linkages between actions and segmental elements of speech. For instance, close-front vowels are sound symbolically associated with the precision grip, and front vowels are associated with forward-directed limb movements. The current review article presents a variety of such sound-action effects and proposes that they compose a category of sound symbolism that is based on grounding a conceptual knowledge of a referent in articulatory and manual action representations. In addition, the article proposes that even some widely known sound symbolism phenomena such as the sound-magnitude symbolism can be partially based on similar sensorimotor grounding. It is also discussed that meaning of suprasegmental speech elements in many instances is similarly grounded in body actions. Sound symbolism, prosody, and body gestures might originate from the same embodied mechanisms that enable a vivid and iconic expression of a meaning of a referent to the recipient.

Hazardous tools: the emergence of reasoning in human tool use

Humans are unique in the way they understand the causal relationships between the use of tools and achieving a goal. The idea at the core of the present research is that tool use can be considered as an instance of problem-solving situations supported by technical reasoning. In an eye-tracking study, we investigated the fixation patterns of participants (N = 32) looking at 3D images of thematically consistent (e.g., nail-steel hammer) and thematically inconsistent (e.g., scarf-steel hammer) object-tool pairs that could be either "hazardous" (accidentally electrified) or not. Results showed that under thematically consistent conditions, participants focused on the tool's manipulation area (e.g., the handle of a steel hammer). However, when electrified tools were present or when the visual scene was not action-prompting, regardless of the presence of electricity, the tools' functional/identity areas (e.g., the head of a steel hammer) were fixated longer than the tools' manipulation areas. These results support an integrated and reasoning-based approach to human tool use and document, for the first time, the crucial role of mechanical/semantic knowledge in tool visual exploration.

It's not all about looks: The role of object shape in parietal representations of manual tools

The ability to build and expertly manipulate manual tools sets humans apart from other animals. Watching images of manual tools has been shown to elicit a distinct pattern of neural activity in a network of parietal areas, assumingly because tools entail a potential for action-a unique feature related to their functional use and not shared with other manipulable objects. However, a question has been raised whether this selectivity reflects a processing of low-level visual properties-such as elongated shape that is idiosyncratic to most tool-objects-rather than action-specific features. To address this question, we created and behaviourally validated a stimulus set that dissociates objects that are manipulable and nonmanipulable, as well as objects with different degrees of body extension property (tools and non-tools), while controlling for object shape and low-level image properties. We tested the encoding of action-related features by investigating neural representations in two parietal regions of interest (intraparietal sulcus and superior parietal lobule) using functional MRI. Univariate differences between tools and non-tools were not observed when controlling for visual properties, but strong evidence for the action account was nevertheless revealed when using a multivariate approach. Overall, this study provides further evidence that the representational content in the dorsal visual stream reflects encoding of action-specific properties.

The Sound of Grasp Affordances: Influence of Grasp-Related Size of Categorized Objects on Vocalization

Previous research shows that simultaneously executed grasp and vocalization responses are faster when the precision grip is performed with the vowel [i] and the power grip is performed with the vowel [ɑ]. Research also shows that observing an object that is graspable with a precision or power grip can activate the grip congruent with the object. Given the connection between vowel articulation and grasping, this study explores whether grasp-related size of observed objects can influence not only grasp responses but also vowel pronunciation. The participants had to categorize small and large objects into natural and manufactured categories by pronouncing the vowel [i] or [ɑ]. As predicted, [i] was produced faster when the object's grasp-related size was congruent with the precision grip while [ɑ] was produced faster when the size was congruent with the power grip (Experiment 1). The effect was not, however, observed when the participants were presented with large objects that are not typically grasped by the power grip (Experiment 2). This study demonstrates that vowel production is systematically influenced by grasp-related size of a viewed object, supporting the account that sensory-motor processes related to grasp planning and representing grasp-related properties of viewed objects interact with articulation processes. The paper discusses these findings in the context of size-sound symbolism, suggesting that mechanisms that transform size-grasp affordances into corresponding grasp- and articulation-related motor programs might provide a neural basis for size-sound phenomena that links small objects with closed-front vowels and large objects with open-back vowels.