Steady-state visually evoked potentials reveal partial size constancy in early visual cortex

Distance mis-estimations can be reduced with specific shadow locations

Shadows in physical space are copious, yet the impact of specific shadow placement and their abundance is yet to be determined in virtual environments. This experiment aimed to identify whether a target's shadow was used as a distance indicator in the presence of binocular distance cues. Six lighting conditions were created and presented in virtual reality for participants to perform a perceptual matching task. The task was repeated in a cluttered and sparse environment, where the number of cast shadows (and their placement) varied. Performance in this task was measured by the directional bias of distance estimates and variability of responses. No significant difference was found between the sparse and cluttered environments, however due to the large amount of variance, one explanation is that some participants utilised the clutter objects as anchors to aid them, while others found them distracting. Under-setting of distances was found in all conditions and environments, as predicted. Having an ambient light source produced the most variable and inaccurate estimates of distance, whereas lighting positioned above the target reduced the mis-estimation of distances perceived.

Linear perspective cues have a greater effect on the perceptual rescaling of distant stimuli than textures in the virtual environment

The presence of pictorial depth cues in virtual environments is important for minimising distortions driven by unnatural viewing conditions (e.g., vergence-accommodation conflict). Our aim was to determine how different pictorial depth cues affect size constancy in virtual environments under binocular and monocular viewing conditions. We systematically removed linear perspective cues and textures of a hallway in a virtual environment. The experiment was performed using the method of constant stimuli. The task required participants to compare the size of 'far' (10 m) and 'near' (5 m) circles displayed inside a virtual environment with one or both or none of the pictorial depth cues. Participants performed the experiment under binocular and monocular viewing conditions while wearing a virtual reality headset. ANOVA revealed that size constancy was greater for both the far and the near circles in the virtual environment with pictorial depth cues compared to the one without cues. However, the effect of linear perspective cues was stronger than textures, especially for the far circle. We found no difference between the binocular and monocular viewing conditions across the different virtual environments. We conclude that linear perspective cues exert a stronger effect than textures on the perceptual rescaling of far stimuli placed in the virtual environment, and that this effect does not vary between binocular and monocular viewing conditions.

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Behavioral examination of the role of the primary visual cortex in the perceived size representation

Previous research has shown that neural activity in the primary visual cortex (V1) and V1 surface area may be linked with subjective experience of size illusions. Here, we behaviorally measured the hallway illusion with experimental manipulations as a proxy of V1's influence on size perception. We first tested whether the hallway illusion can persist without further recurrent processing by using backward masking. Next, we examined relations among the hallway illusion magnitude and other perceptual measures that have been suggested to be correlated with V1 surface area. In Experiment 1, the magnitude of the hallway illusion was not affected by the stimulus duration and visual masking when the hallway context was previewed (i.e., complex depth information is already processed). It suggests that V1 activity could support the size illusion to some extent even when recurrent processing between V1 and higher areas is disturbed. In Experiment 2, the hallway illusion magnitude was correlated with the Vernier acuity threshold, but not with physical size discriminability. Our results provide converging evidence with the previous findings in that neural activity in V1 may contribute to size illusions and that V1 surface area is not the sole factor that mediates size perception and visual precision.

Laplacian reference is optimal for steady-state visual-evoked potentials

Steady-state visual-evoked potentials (SSVEPs) are widely used in human neuroscience studies and applications such as brain-computer interfaces (BCIs). Surprisingly, no previous study has systematically evaluated different reference methods for SSVEP analysis, despite that signal reference is crucial for the proper assessment of neural activities. In the present study, using four datasets from our previous SSVEP studies (Chen J, Valsecchi M, Gegenfurtner KR. J Neurophysiol 118: 749-754, 2017; Chen J, Valsecchi M, Gegenfurtner KR. Neuropsychologia 102: 206-216, 2017; Chen J, McManus M, Valsecchi M, Harris LR, Gegenfurtner KR. J Vis 19: 8, 2019) and three public datasets from other studies (Baker DH, Vilidaite G, Wade AR. PLoS Comput Biol 17: e1009507, 2021; Lygo FA, Richard B, Wade AR, Morland AB, Baker DH. NeuroImage 230: 117780, 2021; Vilidaite G, Norcia AM, West RJH, Elliott CJH, Pei F, Wade AR, Baker DH. Proc R Soc B 285: 20182255, 2018), we compared four reference methods: monopolar reference, common average reference, averaged-mastoids reference, and Laplacian reference. The quality of the resulting SSVEP signals was compared in terms of both signal-to-noise ratios (SNRs) and reliability. The results showed that Laplacian reference, which uses signals at the maximally activated electrode after subtracting the average of the nearby electrodes to reduce common noise, gave rise to the highest SNRs. Furthermore, the Laplacian reference resulted in SSVEP signals that were highly reliable across recording sessions or trials. These results suggest that Laplacian reference is optimal for SSVEP studies and applications. Laplacian reference is especially advantageous for SSVEP experiments where short preparation time is preferred as it requires only data from the maximally activated electrode and a few surrounding electrodes.NEW & NOTEWORTHY The present study provides a comprehensive evaluation of the use of different reference methods for steady-state visual-evoked potentials (SSVEPs) and has found that Laplacian reference increases signal-to-noise ratios (SNRs) and enhances reliabilities of SSVEP signals. Thus, the results suggest that Laplacian reference is optimal for SSVEP analysis.

A Systematic Review of Using Deep Learning Technology in the Steady-State Visually Evoked Potential-Based Brain-Computer Interface Applications: Current Trends and Future Trust Methodology

The significance of deep learning techniques in relation to steady-state visually evoked potential- (SSVEP-) based brain-computer interface (BCI) applications is assessed through a systematic review. Three reliable databases, PubMed, ScienceDirect, and IEEE, were considered to gather relevant scientific and theoretical articles. Initially, 125 papers were found between 2010 and 2021 related to this integrated research field. After the filtering process, only 30 articles were identified and classified into five categories based on their type of deep learning methods. The first category, convolutional neural network (CNN), accounts for 70% (n = 21/30). The second category, recurrent neural network (RNN), accounts for 10% (n = 3/30). The third and fourth categories, deep neural network (DNN) and long short-term memory (LSTM), account for 6% (n = 30). The fifth category, restricted Boltzmann machine (RBM), accounts for 3% (n = 1/30). The literature's findings in terms of the main aspects identified in existing applications of deep learning pattern recognition techniques in SSVEP-based BCI, such as feature extraction, classification, activation functions, validation methods, and achieved classification accuracies, are examined. A comprehensive mapping analysis was also conducted, which identified six categories. Current challenges of ensuring trustworthy deep learning in SSVEP-based BCI applications were discussed, and recommendations were provided to researchers and developers. The study critically reviews the current unsolved issues of SSVEP-based BCI applications in terms of development challenges based on deep learning techniques and selection challenges based on multicriteria decision-making (MCDM). A trust proposal solution is presented with three methodology phases for evaluating and benchmarking SSVEP-based BCI applications using fuzzy decision-making techniques. Valuable insights and recommendations for researchers and developers in the SSVEP-based BCI and deep learning are provided.

Can People Infer Distance in a 2D Scene Using the Visual Size and Position of an Object?

Depth information is limited in a 2D scene and for people to perceive the distance of an object, they need to rely on pictorial cues such as perspective, size constancy and elevation in the scene. In this study, we tested whether people could use an object’s size and its position in a 2D image to determine its distance. In a series of online experiments, participants viewed a target representing their smartphone rendered within a 2D scene. They either positioned it in the scene at the distance they thought was correct based on its size or adjusted the target to the correct size based on its position in the scene. In all experiments, the adjusted target size and positions were not consistent with their initially presented positions and sizes and were made larger and moved further away on average. Familiar objects influenced adjusted position from size but not adjusted size from position. These results suggest that in a 2D scene, (1) people cannot use an object’s visual size and position relative to the horizon to infer distance reliably and (2) familiar objects in the scene affect perceived size and distance differently. The differences found demonstrate that size and distance perception processes may be independent.

V1 as an egocentric cognitive map

We typically distinguish between V1 as an egocentric perceptual map and the hippocampus as an allocentric cognitive map. In this article, we argue that V1 also functions as a post-perceptual egocentric cognitive map. We argue that three well-documented functions of V1, namely (i) the estimation of distance, (ii) the estimation of size, and (iii) multisensory integration, are better understood as post-perceptual cognitive inferences. This argument has two important implications. First, we argue that V1 must function as the neural correlates of the visual perception/cognition distinction and suggest how this can be accommodated by V1's laminar structure. Second, we use this insight to propose a low-level account of visual consciousness in contrast to mid-level accounts (recurrent processing theory; integrated information theory) and higher-level accounts (higher-order thought; global workspace theory). Detection thresholds have been traditionally used to rule out such an approach, but we explain why it is a mistake to equate visibility (and therefore the presence/absence of visual experience) with detection thresholds.

Familiar size affects the perceived size and distance of real objects even with binocular vision

Although the familiar size of real-world objects affects size and distance perception, evidence is mixed about whether this is the case when oculomotor cues are available. We examined the familiar size effect (FSE) on both size and distance perception for real objects under two viewing conditions with full or restricted oculomotor cues (binocular viewing, which provides vergence and accommodation cues, and monocular viewing through a 1-mm pinhole, which removes those cues). Familiar objects (a playing die versus a Rubik's cube) were manufactured in their typical (1.6-cm die and 5.7-cm Rubik's cube) and reverse (5.7-cm die and 1.6-cm Rubik's cube) sizes and shown at two distances (25 cm versus 91 cm) in isolation. Small near and large far objects subtended equal retinal angles. Participants provided manual estimates of perceived size and distance. For every combination of size and distance, Rubik's cubes were perceived as larger and farther than the dice, even during binocular viewing at near distances (<1 meter), when oculomotor cues are particularly strong. For size perception but not distance perception, the familiar size effect was significantly stronger under monocular pinhole viewing than binocular viewing. These results suggest that (1) familiar size affects the accuracy of perception, not just the speed; (2) the effect occurs even when oculomotor cues are available; and (3) size and distance perception are not perfectly yoked.

Electrophysiological evidence for higher-level chromatic mechanisms in humans

Color vision in humans starts with three types of cones (short [S], medium [M], and long [L] wavelengths) in the retina and three retinal and subcortical cardinal mechanisms, which linearly combine cone signals into the luminance channel (L + M), the red-green channel (L - M), and the yellow-blue channel (S-(L + M)). Chromatic mechanisms at the cortical level, however, are less well characterized. The present study investigated such higher-order chromatic mechanisms by recording electroencephalograms (EEGs) on human observers in a noise masking paradigm. Observers viewed colored stimuli that consisted of a target embedded in noise. Color directions of the target and noise varied independently and systematically in an isoluminant plane of color space. The target was flickering on-off at 3 Hz, eliciting steady-state visual evoked potential (SSVEP) responses. As a result, the masking strength could be estimated from the SSVEP amplitude in the presence of 6 Hz noise. Masking was strongest (i.e. target eliciting smallest SSVEPs) when the target and noise were along the same color direction, and was weakest (i.e. target eliciting highest SSVEPs) when the target and noise were along orthogonal directions. This pattern of results was observed both when the target color varied along the cardinal and intermediate directions, which is evidence for higher-order chromatic mechanisms tuned to intermediate axes. The SSVEP result can be well predicted by a model with multiple broadly tuned chromatic mechanisms. In contrast, a model with only cardinal mechanisms failed to account for the data. These results provide strong electrophysiological evidence for multiple chromatic mechanisms in the early visual cortex of humans.

Holistic Integration in the Processing of Chinese Characters as Revealed by Electroencephalography Frequency Tagging

It is debated whether perceptual expertise of nonface objects, such as visual words, is indicated by holistic processing, which is regarded as a marker of perceptual expertise of faces. We address this question by frequency-tagged electroencephalography. Different parts of real or pseudo Chinese characters are presented at distinctive frequencies (6 or 7.2 Hz), which induce frequency-tagged steady-state visual-evoked potentials at occipital brain areas. The intermodulation response (e.g., 6 + 7.2 = 13.2 Hz) would emerge when holistic integration takes place. Our results suggest that the intermodulation response to the real characters is left lateralized, which is contralateral to previous findings with faces. Furthermore, at the left occipital area, the intermodulation response to real characters is more prominent than pseudo characters, suggesting that holistic integration is enhanced for real characters than for pseudo ones. Taken together, our findings suggest that holistic integration is potentially a general expertise marker for both faces and non-face objects.

Opposite effects of motion dynamics on the Ebbinghaus and corridor illusions

We recently showed that motion dynamics greatly enhance the magnitude of certain size contrast illusions, such as the Ebbinghaus and Delboeuf illusions. Here, we extend our study of the effect of motion dynamics on size illusions through a novel dynamic corridor illusion, in which a single target translates along a corridor background. Across three psychophysical experiments, we quantify the effects of stimulus dynamics on the Ebbinghaus and corridor illusions across different viewing conditions. The results revealed that stimulus dynamics had opposite effects on these different classes of size illusions. Whereas dynamic motion enhanced the magnitude of the Ebbinghaus illusion, it attenuated the magnitude the corridor illusion. Our results highlight precision-driven weighting of visual cues by neural circuits computing perceived object size. This hypothesis is consistent with observations beyond size perception and may represent a more general principle of cue integration in the visual system.