The human primary visual cortex (V1) encodes the perceived position of static but not moving objects

Brain-like illusion produced by Skye's Oblique Grating in deep neural networks

The analogy between the brain and deep neural networks (DNNs) has sparked interest in neuroscience. Although DNNs have limitations, they remain valuable for modeling specific brain characteristics. This study used Skye's Oblique Grating illusion to assess DNNs' relevance to brain neural networks. We collected data on human perceptual responses to a series of visual illusions. This data was then used to assess how DNN responses to these illusions paralleled or differed from human behavior. We performed two analyses:(1) We trained DNNs to perform horizontal vs. non-horizontal classification on images with bars tilted different degrees (non-illusory images) and tested them on images with horizontal bars with different illusory strengths measured by human behavior (illusory images), finding that DNNs showed human-like illusions; (2) We performed representational similarity analysis to assess whether illusory representation existed in different layers within DNNs, finding that DNNs showed illusion-like responses to illusory images. The representational similarity between real tilted images and illusory images was calculated, which showed the highest values in the early layers and decreased layer-by-layer. Our findings suggest that DNNs could serve as potential models for explaining the mechanism of visual illusions in human brain, particularly those that may originate in early visual areas like the primary visual cortex (V1). While promising, further research is necessary to understand the nuanced differences between DNNs and human visual pathways.

Bounded contribution of human early visual cortex to the topographic anisotropy in spatial extent perception

To interact successfully with objects, it is crucial to accurately perceive their spatial extent, an enclosed region they occupy in space. Although the topographic representation of space in the early visual cortex (EVC) has been favored as a neural correlate of spatial extent perception, its exact nature and contribution to perception remain unclear. Here, we inspect the topographic representations of human individuals' EVC and perception in terms of how much their anisotropy is influenced by the orientation (co-axiality) and radial position (radiality) of stimuli. We report that while the anisotropy is influenced by both factors, its direction is primarily determined by radiality in EVC but by co-axiality in perception. Despite this mismatch, the individual differences in both radial and co-axial anisotropy are substantially shared between EVC and perception. Our findings suggest that spatial extent perception builds on EVC's spatial representation but requires an additional mechanism to transform its topographic bias.

Reconstructing visual illusory experiences from human brain activity

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

Predictions and rewards affect decision-making but not subjective experience

To survive, organisms constantly make decisions to avoid danger and maximize rewards in information-rich environments. As a result, decisions about sensory input are not only driven by sensory information but also by other factors, such as the expected rewards of a decision (known as the payoff matrix) or by information about temporal regularities in the environment (known as cognitive priors or predictions). However, it is unknown to what extent these different types of information affect subjective experience or whether they merely result in nonperceptual response criterion shifts. To investigate this question, we used three carefully matched manipulations that typically result in behavioral shifts in decision criteria: a visual illusion (Müller-Lyer condition), a punishment scheme (payoff condition), and a change in the ratio of relevant stimuli (base rate condition). To gauge shifts in subjective experience, we introduce a task in which participants not only make decisions about what they have just seen but are also asked to reproduce their experience of a target stimulus. Using Bayesian ordinal modeling, we show that each of these three manipulations affects the decision criterion as intended but that the visual illusion uniquely affects sensory experience as measured by reproduction. In a series of follow-up experiments, we use computational modeling to show that although the visual illusion results in a distinct drift-diffusion (DDM) parameter profile relative to nonsensory manipulations, reliance on DDM parameter estimates alone is not sufficient to ascertain whether a given manipulation is perceptual or nonperceptual.

The role of awareness in shaping responses in human visual cortex

The visual cortex contains information about stimuli even when they are not consciously perceived. However, it remains unknown whether the visual system integrates local features into global objects without awareness. Here, we tested this by measuring brain activity in human observers viewing fragmented shapes that were either visible or rendered invisible by fast counterphase flicker. We then projected measured neural responses to these stimuli back into visual space. Visible stimuli caused robust responses reflecting the positions of their component fragments. Their neural representations also strongly resembled one another regardless of local features. By contrast, representations of invisible stimuli differed from one another and, crucially, also from visible stimuli. Our results demonstrate that even the early visual cortex encodes unconscious visual information differently from conscious information, presumably by only encoding local features. This could explain previous conflicting behavioural findings on unconscious visual processing.