Neural correlates of size illusions: an event-related potential study

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Behavioral examination of the role of the primary visual cortex in the perceived size representation

Previous research has shown that neural activity in the primary visual cortex (V1) and V1 surface area may be linked with subjective experience of size illusions. Here, we behaviorally measured the hallway illusion with experimental manipulations as a proxy of V1's influence on size perception. We first tested whether the hallway illusion can persist without further recurrent processing by using backward masking. Next, we examined relations among the hallway illusion magnitude and other perceptual measures that have been suggested to be correlated with V1 surface area. In Experiment 1, the magnitude of the hallway illusion was not affected by the stimulus duration and visual masking when the hallway context was previewed (i.e., complex depth information is already processed). It suggests that V1 activity could support the size illusion to some extent even when recurrent processing between V1 and higher areas is disturbed. In Experiment 2, the hallway illusion magnitude was correlated with the Vernier acuity threshold, but not with physical size discriminability. Our results provide converging evidence with the previous findings in that neural activity in V1 may contribute to size illusions and that V1 surface area is not the sole factor that mediates size perception and visual precision.

The conceptual understanding of depth rather than the low-level processing of spatial frequencies drives the corridor illusion

Our objective was to determine how different spatial frequencies affect the perceptual size rescaling of stimuli in the corridor illusion. Two experiments were performed using the method of constant stimuli. In experiment 1, the task required participants to compare the size of comparison and standard rings displayed over the same background image. ANOVA on the points of subject equality (PSEs) revealed that the perceived size of the top and bottom standard rings changed as a function of the availability of the high, medium, and low spatial frequency information. In experiment 2, the task required participants to compare the size of a comparison ring presented outside of the background image with a standard ring presented inside it. ANOVA on the PSEs revealed that the apparent size of the top and not the bottom standard ring changed depending on the availability of medium spatial frequency information. Eye-tracking revealed that the spatial frequency range of the background image in the periphery affected participants' eye positioning, which may explain why the effects of different spatial frequencies fluctuated across experiments. Nonetheless, when we consider these findings together, we propose that the conceptual understanding of depth plays a more important role in explaining the corridor illusion than the low-level processing of depth information extracted from different spatial frequencies along separate channels.

Opposite effects of motion dynamics on the Ebbinghaus and corridor illusions

We recently showed that motion dynamics greatly enhance the magnitude of certain size contrast illusions, such as the Ebbinghaus and Delboeuf illusions. Here, we extend our study of the effect of motion dynamics on size illusions through a novel dynamic corridor illusion, in which a single target translates along a corridor background. Across three psychophysical experiments, we quantify the effects of stimulus dynamics on the Ebbinghaus and corridor illusions across different viewing conditions. The results revealed that stimulus dynamics had opposite effects on these different classes of size illusions. Whereas dynamic motion enhanced the magnitude of the Ebbinghaus illusion, it attenuated the magnitude the corridor illusion. Our results highlight precision-driven weighting of visual cues by neural circuits computing perceived object size. This hypothesis is consistent with observations beyond size perception and may represent a more general principle of cue integration in the visual system.

Temporal dynamics of the flash-induced bouncing effect

Two identical visual disks moving toward each other on a two‐dimensional (2D) display are more likely to be perceived as “streaming through” than “bouncing off” each other after their coincidence. However, either a brief auditory tone or visual flash presented at the coincident moment of the disks can strikingly increase the incidence of the bouncing percept. Despite the neural substrates underlying the sound‐induced bouncing effect have been widely investigated, little is known about the neural mechanisms underlying the flash‐induced bouncing effect. The present study used event‐related potential recordings to explore the temporal dynamics of the flash‐induced bouncing effect. The results showed that the amplitude of the postcoincidence parietooccipital P2 component (190–230 ms after coincidence) elicited by the visual motion was significantly smaller on bouncing relative to streaming trials only when the flash was presented but not when absent. In addition, the parietal P3 component (330–430 ms) was found to be larger on bouncing than streaming trials when the flash was presented, but the opposite was true when no flash was presented. These electrophysiological findings suggest that the flash‐induced bouncing effect may occur at both perceptual and postperceptual stages of processing.

Grip Constancy but Not Perceptual Size Constancy Survives Lesions of Early Visual Cortex

Object constancies are central constructs in theories of visual phenomenology. A powerful example is "size constancy," in which the perceived size of an object remains stable despite changes in viewing distance [1-4]. Evidence from neuropsychology [5], neuroimaging [6-11], transcranial magnetic stimulation [12, 13], single-unit and lesion studies in monkey [14-20], and computational modeling [21] suggests that re-entrant processes involving reciprocal interactions between primary visual cortex (V1) and extrastriate visual areas [22-26] play an essential role in mediating size constancy. It is seldom appreciated, however, that object constancies must also operate for the visual guidance of goal-directed action. For example, when reaching out to pick up an object, the hand's in-flight aperture scales with size of the goal object [27-30] and is refractory to the decrease in retinal-image size with increased viewing distance [31-41] (Figure 1), a phenomenon we call "grip constancy." Does grip constancy, like perceptual constancy, depend on V1 or can it be mediated by pathways that bypass it altogether? We tested these possibilities in an individual, M.C., who has bilateral lesions encompassing V1 and much of the ventral visual stream. We show that her perceptual estimates of object size co-vary with retinal-image size rather than real-world size as viewing distance varies. In contrast, M.C. shows near-normal scaling of in-flight grasp aperture to object size despite changes in viewing distance. Thus, although early visual cortex is necessary for perceptual object constancy, it is unnecessary for grip constancy, which is mediated instead by separate visual inputs to dorsal-stream visuomotor areas [42-48].

Visual Size Processing in Early Visual Cortex Follows Lateral Occipital Cortex Involvement

Neural activation in the early visual cortex (EVC) reflects the perceived rather than retinal size of stimuli, suggesting that feedback possibly from extrastriate regions modulates retinal size information in EVC. Meanwhile, the lateral occipital cortex (LOC) has been suggested to be critically involved in object size processing. To test for the potential contributions of feedback modulations on size representations in EVC, we investigated the dynamics of relevant processes using transcranial magnetic stimulation (TMS). Specifically, we briefly disrupted the neural activity of EVC and LOC at early, intermediate, and late time windows while participants performed size judgment tasks in either an illusory or neutral context. TMS over EVC and LOC allowed determining whether these two brain regions are relevant for generating phenomenological size impressions. Furthermore, the temporal order of TMS effects allowed inferences on the dynamics of information exchange between the two areas. Particularly, if feedback signals from LOC to EVC are crucial for generating altered size representations in EVC, then TMS effects over EVC should be observed simultaneously or later than the effects following LOC stimulation. The data from 20 humans (13 females) revealed that TMS over both EVC and LOC impaired illusory size perception. However, the strongest effects of TMS applied over EVC occurred later than those of LOC, supporting a functionally relevant feedback modulation from LOC to EVC for scaling size information. Our results suggest that context integration and the concomitant change of perceived size require LOC and result in modulating representations in EVC via recurrent processing.SIGNIFICANCE STATEMENT How we perceive an object's size is not entirely determined by its physical size or the size of its retinal representation but also the spatial context. Using transcranial magnetic stimulation, we investigated the role of the early visual cortex (EVC) and the higher-level visual area, lateral occipital cortex (LOC), known to be critically involved in object processing, in transforming an initial retinal representation into one that reflects perceived size. Transcranial magnetic stimulation altered size perception earlier over LOC compared with EVC, suggesting that context integration and the concomitant change in perceived size representations in EVC rely on feedback from LOC.

Changing the Real Viewing Distance Reveals the Temporal Evolution of Size Constancy in Visual Cortex

Our visual system provides a distance-invariant percept of object size by integrating retinal image size with viewing distance (size constancy). Single-unit studies with animals have shown that some distance cues, especially oculomotor cues such as vergence and accommodation, can modulate the signals in the thalamus or V1 at the initial processing stage [1-7]. Accordingly, one might predict that size constancy emerges much earlier in time [8-10], even as visual signals are being processed in the thalamus. So far, the studies that have looked directly at size coding have either used fMRI (poor temporal resolution [11-13]) or relied on inadequate stimuli (pictorial illusions presented on a monitor at a fixed distance [11, 12, 14, 15]). Here, we physically moved the monitor to different distances, a more ecologically valid paradigm that emulates what happens in everyday life and is an example of the increasing trend of "bringing the real world into the lab." Using this paradigm in combination with electroencephalography (EEG), we examined the computation of size constancy in real time with real-world viewing conditions. Our study provides strong evidence that, even though oculomotor distance cues have been shown to modulate the spiking rate of neurons in the thalamus and in V1, the integration of viewing distance cues and retinal image size takes at least 150 ms to unfold, which suggests that the size-constancy-related activation patterns in V1 reported in previous fMRI studies (e.g., [12, 13]) reflect the later processing within V1 and/or top-down input from other high-level visual areas.

The moon size illusion does not improve perceptual judgments

Recent studies suggest that the accuracy of perceptual judgments can be influenced by the perceived illusory size of a stimulus, with judgments being more accurate for increased illusory size. This phenomenon seems consistent with recent neuroscientific findings that representations in early visual areas reflect the perceived (illusory) size of stimuli rather than the physical size. We further explored this idea with the moon illusion, in which the moon appears larger when it is close to the horizon and smaller when it is higher in the sky. Participants (n=230) adjusted the orientation of an image of the moon on a smartphone to match the perceived orientation of the moon in the sky. Contrary to previous studies that investigated accuracy and size illusions, we found slightly lower perceptual judgment accuracy when the moon appeared large (close to the horizon) compared to when it appeared small (high in the sky).

Pre-coincidence brain activity predicts the perceptual outcome of streaming/bouncing motion display

When two identical visual discs move toward each other on a two-dimensional visual display, they can be perceived as either "streaming through" or "bouncing off" each other after their coincidence. Previous studies have observed a strong bias toward the streaming percept. Additionally, the incidence of the bouncing percept in this ambiguous display could be increased by various factors, such as a brief sound at the moment of coincidence and a momentary pause of the two discs. The streaming/bouncing bistable motion phenomenon has been studied intensively since its discovery. However, little is known regarding the neural basis underling the perceptual ambiguity in the classic version of the streaming/bouncing motion display. The present study investigated the neural basis of the perception disambiguating underling the processing of the streaming/bouncing bistable motion display using event-related potential (ERP) recordings. Surprisingly, the amplitude of frontal central P2 (220-260 ms) that was elicited by the moving discs ~200 ms before the coincidence of the two discs was observed to be predictive of subsequent streaming or bouncing percept. A larger P2 amplitude was observed for streaming percept than the bouncing percept. These findings suggest that the streaming/bouncing bistable perception may have been disambiguated unconsciously ~200 ms before the coincidence of the two discs.

The Role of Parieto-Occipital Junction in the Interaction between Dorsal and Ventral Streams in Disparity-Defined Near and Far Space Processing

Neuropsychological and functional MRI data suggest that two functionally and anatomically dissociable streams of visual processing exist: a ventral perception-related stream and a dorsal action-related stream. However, relatively little is known about how the two streams interact in the intact brain during the production of adaptive behavior. Using functional MRI and a virtual three-dimensional paradigm, we aimed at examining whether the parieto-occipital junction (POJ) acts as an interface for the integration and processing of information between the dorsal and ventral streams in the near and far space processing. Virtual reality three-dimensional near and far space was defined by manipulating binocular disparity, with -68.76 arcmin crossed disparity for near space and +68.76 arcmin uncrossed disparity for near space. Our results showed that the POJ and bilateral superior occipital gyrus (SOG) showed relative increased activity when responded to targets presented in the near space than in the far space, which was independent of the retinotopic and perceived sizes of target. Furthermore, the POJ showed the enhanced functional connectivity with both the dorsal and ventral streams during the far space processing irrespective of target sizes, supporting that the POJ acts as an interface between the dorsal and ventral streams in disparity-defined near and far space processing. In contrast, the bilateral SOG showed the enhanced functional connectivity only with the ventral stream if retinotopic sizes of targets in the near and far spaces were matched, which suggested there was a functional dissociation between the POJ and bilateral SOG.

A Neural Model of Distance-Dependent Percept of Object Size Constancy

Size constancy is one of the well-known visual phenomena that demonstrates perceptual stability to account for the effect of viewing distance on retinal image size. Although theories involving distance scaling to achieve size constancy have flourished based on psychophysical studies, its underlying neural mechanisms remain unknown. Single cell recordings show that distance-dependent size tuned cells are common along the ventral stream, originating from V1, V2, and V4 leading to IT. In addition, recent research employing fMRI demonstrates that an object's perceived size, associated with its perceived egocentric distance, modulates its retinotopic representation in V1. These results suggest that V1 contributes to size constancy, and its activity is possibly regulated by feedback of distance information from other brain areas. Here, we propose a neural model based on these findings. First, we construct an egocentric distance map in LIP by integrating horizontal disparity and vergence through gain-modulated MT neurons. Second, LIP neurons send modulatory feedback of distance information to size tuned cells in V1, resulting in a spread of V1 cortical activity. This process provides V1 with distance-dependent size representations. The model supports that size constancy is preserved by scaling retinal image size to compensate for changes in perceived distance, and suggests a possible neural circuit capable of implementing this process.