Predictive Coding: A Possible Explanation of Filling-In at the Blind Spot

The role of extrinsic and intrinsic factors in perceptual filling-in of the blind-spot with variegated color and texture stimuli

Vision scientists dedicated their efforts to unraveling the mechanism of filling-in at the blind-spot (BS) through numerous psychophysical experiments. The prevalent interpretation, emphasizing active filling-in, has spurred extensive research endeavors. In a parallel vein, a pertinent study highlighted the predominance of the nasal Visual Field (VF) over the temporal one and postulated the role of the Cortical Magnification Factor (CMF) in explaining the asymmetry of filling-in. In this study, we first replicated this experiment and then conducted BS-specific psychophysical experiments employing various bi-colored and bi-textured (patterned) stimuli. We observed that nasal dominance is not persistent in the context of the spread of perception for BS filling-in. We posit that the visual information processing priority index (VIPPI), comprising the CMF (an intrinsic factor unaffected by stimulus characteristics) and relative luminance (an extrinsic factor dependent on stimulus characteristics), governs the spread of perception for filling-in in case of diverse neighborhoods of the BS.

Fine-scale measurement of the blind spot borders

The blind spot is both a necessity and a nuisance for seeing. It is the portion of the visual field projecting to where the optic nerve crosses the retina, a region devoid of photoreceptors and hence visual input. The precise way in which vision transitions into blindness at the blind spot border is to date unknown. A chief challenge to map this transition is the incessant movement of the eye, which unavoidably smears measurements across space. In this study, we used high-resolution eye-tracking and state-of-the-art retinal stabilization to finely map the blind spot borders. Participants reported the onset of tiny high-contrast probes that were briefly flashed at precise positions around the blind spot. This method has sufficient resolution to enable mapping of blood vessels from psychophysical measurements. Our data show that, even after accounting for eye movements, the transition zones at the edges of the blind spot are considerable. On the horizontal meridian, the regions with detection rates between 80% and 20% span approximately 25% of the overall width of the blind spot. These borders also vary considerably in size across different axes. These data show that the transition from full visibility to blindness at the blind spot border is not abrupt but occurs over a broad area.

Predictive coding feedback results in perceived illusory contours in a recurrent neural network

Modern feedforward convolutional neural networks (CNNs) can now solve some computer vision tasks at super-human levels. However, these networks only roughly mimic human visual perception. One difference from human vision is that they do not appear to perceive illusory contours (e.g. Kanizsa squares) in the same way humans do. Physiological evidence from visual cortex suggests that the perception of illusory contours could involve feedback connections. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we equip a deep feedforward convolutional network with brain-inspired recurrent dynamics. The network was first pretrained with an unsupervised reconstruction objective on a natural image dataset, to expose it to natural object contour statistics. Then, a classification decision head was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar "illusory contour" configuration: inducer shapes oriented to form an illusory square. Compared with feedforward baselines, the iterative "predictive coding" feedback resulted in more illusory contours being classified as physical squares. The perception of the illusory contour was measurable in the luminance profile of the image reconstructions produced by the model, demonstrating that the model really "sees" the illusion. Ablation studies revealed that natural image pretraining and feedback error correction are both critical to the perception of the illusion. Finally we validated our conclusions in a deeper network (VGG): adding the same predictive coding feedback dynamics again leads to the perception of illusory contours.

Reactive, Proactive, and Inductive Agents: An Evolutionary Path for Biological and Artificial Spiking Networks

Complex environments provide structured yet variable sensory inputs. To best exploit information from these environments, organisms must evolve the ability to anticipate consequences of new stimuli, and act on these predictions. We propose an evolutionary path for neural networks, leading an organism from reactive behavior to simple proactive behavior and from simple proactive behavior to induction-based behavior. Based on earlier in-vitro and in-silico experiments, we define the conditions necessary in a network with spike-timing dependent plasticity for the organism to go from reactive to proactive behavior. Our results support the existence of specific evolutionary steps and four conditions necessary for embodied neural networks to evolve predictive and inductive abilities from an initial reactive strategy.

Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction

The cerebral cortex predicts visual motion to adapt human behavior to surrounding objects moving in real time. Although the underlying mechanisms are still unknown, predictive coding is one of the leading theories. Predictive coding assumes that the brain's internal models (which are acquired through learning) predict the visual world at all times and that errors between the prediction and the actual sensory input further refine the internal models. In the past year, deep neural networks based on predictive coding were reported for a video prediction machine called PredNet. If the theory substantially reproduces the visual information processing of the cerebral cortex, then PredNet can be expected to represent the human visual perception of motion. In this study, PredNet was trained with natural scene videos of the self-motion of the viewer, and the motion prediction ability of the obtained computer model was verified using unlearned videos. We found that the computer model accurately predicted the magnitude and direction of motion of a rotating propeller in unlearned videos. Surprisingly, it also represented the rotational motion for illusion images that were not moving physically, much like human visual perception. While the trained network accurately reproduced the direction of illusory rotation, it did not detect motion components in negative control pictures wherein people do not perceive illusory motion. This research supports the exciting idea that the mechanism assumed by the predictive coding theory is one of basis of motion illusion generation. Using sensory illusions as indicators of human perception, deep neural networks are expected to contribute significantly to the development of brain research.

Significance of Natural Scene Statistics in Understanding the Anisotropies of Perceptual Filling-in at the Blind Spot

Psychophysical experiments reveal our horizontal preference in perceptual filling-in at the blind spot. On the other hand, tolerance in filling-in exhibit vertical preference. What causes this anisotropy in our perception? Building upon the general notion that the functional properties of the early visual system are shaped by the innate specification as well as the statistics of the environment, we reasoned that the anisotropy in filling-in could be understood in terms of anisotropy in orientation distribution inherent in natural scene statistics. We examined this proposition by investigating filling-in of bar stimuli in a Hierarchical Predictive Coding model network. The model network, trained with natural images, exhibited anisotropic filling-in performance at the blind spot, which is similar to the findings of psychophysical experiments. We suggest that the over-representation of horizontal contours in the natural scene contributes to the observed horizontal superiority in filling-in and the broader distribution of vertical contours contributes to the observed vertical superiority of tolerance in filling-in. These results indicate that natural scene statistics plays a significant role in determining the filling-in performance at the blind spot and shaping the associated anisotropies.