Dimensions of Perception: 3D Real-Life Objects Are More Readily Detected Than Their 2D Images

Taking consciousness for real: Increasing the ecological validity of the study of conscious vs. unconscious processes

The study of consciousness has developed well-controlled, rigorous methods for manipulating and measuring consciousness. Yet, in the process, experimental paradigms grew farther away from everyday conscious and unconscious processes, which raises the concern of ecological validity. In this review, we suggest that the field can benefit from adopting a more ecological approach, akin to other fields of cognitive science. There, this approach challenged some existing hypotheses, yielded stronger effects, and enabled new research questions. We argue that such a move is critical for studying consciousness, where experimental paradigms tend to be artificial and small effect sizes are relatively prevalent. We identify three paths for doing so-changing the stimuli and experimental settings, changing the measures, and changing the research questions themselves-and review works that have already started implementing such approaches. While acknowledging the inherent challenges, we call for increasing ecological validity in consciousness studies.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Object-based inhibition of return in three-dimensional space: From simple drawings to real objects

Cued to an object in space, inhibition of the attended location can spread to the entire object. Although object-based inhibition of return (IOR) studies in a two-dimensional plane have been documented, the IOR has not been explored when objects cross depth in three-dimensional (3D) space. In the present study, we used a virtual reality technique to adapt the double-rectangle paradigm to a 3D space, and manipulated the cue validity and target location to examine the difference in object-based IOR between far and near spaces under different object representations. The study showed that the object-based IOR of simple drawings existed only in near space, whereas object-based IOR of real objects existed only in far space at first, and as the object similarity decreases, it appeared in both far and near spaces.

Nature in the Heart and Mind of the Beholder: Psycho-Emotional and EEG Differences in Perception of Virtual Nature Due to Gender

Natural environment experiences in virtual reality (VR) can be a feasible option for people unable to connect with real nature. Existing research mostly focuses on health and emotional advantages of the "virtual nature" therapy, but studies of its neuropsychological effects related to visual perception are rare. In our experiment, 20 subjects watched nature-related video content in VR headsets (3D condition) and on a computer screen (2D condition). In addition to the gender factor, we considered the individual Environmental Identity Index (EID) and collected the self-assessment of the emotional state per the components of Valence, Arousal, and Dominance in each experimental condition. Besides the psychometric data, we also registered brainwave activity (EEG) and analyzed it with the 7 frequency bands. For EID, which was considerably higher in women, we found significant positive correlation with Valence (i.e., beneficial effect of the natural stimuli on the psycho-emotional status). At the same time, the analysis of the EEG data suggests a considerable impact of the VR immersion itself, with higher relaxation alpha effect in 3D vs. 2D condition in men. The novel and most pronounced effect of the gender factor was found in the relation between the EID and the EEG powers in the high-frequency bands-that is, positive correlation of these variables in women (0.64 < Rs < 0.74) but negative correlation in men (-0.66 < Rs < -0.72). Our results imply individually different and gender-dependent effects of the natural stimulus in VR. Correspondingly, the video and VR content development should consider this and aim to provide a user characteristics-tailored experience.

Tool use acquisition induces a multifunctional interference effect during object processing: evidence from the sensorimotor mu rhythm

A fundamental characteristic of human development is acquiring and accumulating tool use knowledge through observation and sensorimotor experience. Recent studies showed that, in children and adults, different action possibilities to grasp-to-move and grasp-to-use objects generate a conflict that extinguishes neural motor resonance phenomena during visual object processing. In this study, a training protocol coupled with EEG recordings was administered in virtual reality to healthy adults to evaluate whether a similar conflict occurs between novel tool use knowledge. Participants perceived and manipulated two novel 3D tools trained beforehand with either single or double-usage. A weaker reduction of mu-band (10-13 Hz) power, accompanied by a reduced inter-trial phase coherence, was recorded during the perception of the tool associated with the double-usage. These effects started within the first 200 ms of visual object processing and were predominantly recorded over the left motor system. Furthermore, interacting with the double usage tool delayed grasp-to-reach movements. The results highlight a multifunctional interference effect, such as tool use acquisition reduces the neural motor resonance phenomenon and inhibits the activation of the motor system during subsequent object recognition. These results imply that learned tool use information guides sensorimotor processes of objects.

Using Extended Reality to Study the Experience of Presence

Extended reality (XR), encompassing various forms of virtual reality (VR) and augmented reality (AR), has become a powerful experimental tool in consciousness research due to its capability to create holistic and immersive experiences of oneself and surrounding environments through simulation. One hallmark of a successful XR experience is when it elicits a strong sense of presence, which can be thought of as a subjective sense of reality of the self and the world. Although XR research has shed light on many factors that may influence presence (or its absence) in XR environments, there remains much to be discovered about the detailed and diverse phenomenology of presence, and the neurocognitive mechanisms that underlie it. In this chapter, we analyse the concept of presence and relate it to the way in which humans may generate and maintain a stable sense of reality during both natural perception and virtual experiences. We start by reviewing the concept of presence as developed in XR research, covering both factors that may influence presence and potential ways of measuring presence. We then discuss the phenomenological characteristics of presence in human consciousness, drawing on clinical examples where presence is disturbed. Next, we describe two experiments using XR that investigated the effects of sensorimotor contingency and affordances on a specific form of presence related to the sense of objects as really existing in the world, referred to as 'objecthood'. We then go beyond perceptual presence to discuss the concept of 'conviction about reality', which corresponds to people's beliefs about the reality status of their perceptual experiences. We finish by exploring how the novel XR method of 'Substitutional Reality' can allow experimental investigation of these topics, opening new experimental directions for studying presence beyond the 'as-if' experience of fully simulated environments.