The Role of Bottom-Up and Top-Down Cortical Interactions in Adaptation to Natural Scene Statistics

Canonical circuit computations for computer vision

Advanced computer vision mechanisms have been inspired by neuroscientific findings. However, with the focus on improving benchmark achievements, technical solutions have been shaped by application and engineering constraints. This includes the training of neural networks which led to the development of feature detectors optimally suited to the application domain. However, the limitations of such approaches motivate the need to identify computational principles, or motifs, in biological vision that can enable further foundational advances in machine vision. We propose to utilize structural and functional principles of neural systems that have been largely overlooked. They potentially provide new inspirations for computer vision mechanisms and models. Recurrent feedforward, lateral, and feedback interactions characterize general principles underlying processing in mammals. We derive a formal specification of core computational motifs that utilize these principles. These are combined to define model mechanisms for visual shape and motion processing. We demonstrate how such a framework can be adopted to run on neuromorphic brain-inspired hardware platforms and can be extended to automatically adapt to environment statistics. We argue that the identified principles and their formalization inspires sophisticated computational mechanisms with improved explanatory scope. These and other elaborated, biologically inspired models can be employed to design computer vision solutions for different tasks and they can be used to advance neural network architectures of learning.

Cell-type-specific integration of feedforward and feedback synaptic inputs in the posterior parietal cortex

The integration of feedforward (sensory) and feedback (top-down) neuronal signals is a principal function of the neocortex. Yet, we have limited insight into how these information streams are combined by individual neurons. Using a two-color optogenetic strategy, we found that layer 5 pyramidal neurons in the posterior parietal cortex receive monosynaptic dual innervation, combining sensory inputs with top-down signals. Subclasses of layer 5 pyramidal neurons integrated these synapses with distinct temporal dynamics. Specifically, regular spiking cells exhibited supralinear enhancement of delayed-but not coincident-inputs, while intrinsic burst-firing neurons selectively boosted coincident synaptic events. These subthreshold integration characteristics translated to a nonlinear summation of action potential firing. Complementing electrophysiology with computational modeling, we found that distinct integration profiles arose from a cell-type-specific interaction of ionic mechanisms and feedforward inhibition. These data provide insight into the cellular properties that guide the nonlinear interaction of distinct long-range afferents in the neocortex.

Parallel Adaptation to Spatially Distinct Distortions

Optical distortions as a visual disturbance are inherent in many optical devices such as spectacles or virtual reality headsets. In such devices, distortions vary spatially across the visual field. In progressive addition lenses, for example, the left and right regions of the lens skew the peripheral parts of the wearers visual field in opposing directions. The human visual system adapts to homogeneous distortions and the respective aftereffects are transferred to non-retinotopic locations. This study investigates simultaneous adaptation to two opposing distortions at different retinotopic locations. Two oppositely skewed natural image sequences were presented to 10 subjects as adaptation stimuli at two distinct locations in the visual field. To do so, subjects were instructed to keep fixation on a target. Eye tracking was used for gaze control. Change of perceived motion direction was measured in a direction identification task. The point of subjective equality (PSE), that is, the angle at which a group of coherently moving dots was perceived as moving horizontal, was determined for both retinal locations. The shift of perceived motion direction was evaluated by comparing PSE before and after adaptation. A significant shift at both retinal locations in the direction of the skew distortion of the corresponding adaptation stimulus is demonstrated. Consequently, parallel adaptation to two opposing distortions in a retinotopic reference frame was confirmed by this study.

Top-down influence affects the response adaptation of V1 neurons in cats

The visual system lowers its perceptual sensitivity to a prolonged presentation of the same visual signal. This brain plasticity, called visual adaptation, is generally attributed to the response adaptation of neurons in the visual cortex. Although well-studied in the neurons of the primary visual cortex (V1), the contribution of high-level visual cortical regions to the response adaptation of V1 neurons is unclear. In the present study, we measured the response adaptation strength of V1 neurons before and after the top-down influence of the area 21a (A21a), a higher-order visual cortex homologous to the primate V4 area, was modulated with a noninvasive tool of transcranial direct current stimulation (tDCS). Our results showed that the response adaptation of V1 neurons enhanced significantly after applying anode (a-) tDCS in A21a when compared with that before a-tDCS, whereas the response adaptation of V1 neurons weakened after cathode (c-) tDCS relative to before c-tDCS in A21a. By contrast, sham (s-) tDCS in A21a had no significant impact on the response adaptation of V1 neurons. Further analysis indicated that a-tDCS in A21a significantly increased both the initial response (IR) of V1 neurons to the first several (five) trails of visual stimulation and the plateau response (PR) to the prolonged visual stimulation; the increase in PR was lower than in IR, which caused an enhancement in response adaptation. Conversely, c-tDCS significantly decreased both IR and PR of V1 neurons; the reduction in PR was smaller than in IR, which resulted in a weakness in response adaptation. Furthermore, the tDCS-induced changes of V1 neurons in response and response adaptation could recover after tDCS effect vanished, but did not occur after the neuronal activity in A21a was silenced by electrolytic lesions. These results suggest that the top-down influence of A21a may alter the response adaptation of V1 neurons through activation of local inhibitory circuitry, which enhances network inhibition in the V1 area upon an increased top-down input, weakens inhibition upon a decreased top-down input, and thus maintains homeostasis of V1 neurons in response to the long-presenting visual signals.

Motion-form interaction: Motion and form aftereffects induced by distorted static natural scenes

Spatially varying distortions (SVDs) are common artifacts of spectacles like progressive additional lenses (PALs). To habituate to distortions of PALs, the visual system has to adapt to distortion-induced image alterations, termed skew adaptation. But how this visual adjustment is achieved is largely unknown. This study examines the properties of visual adaptation to distortions of PALs in natural scenes. The visual adaptation in response to altered form and motion features of the natural stimuli were probed in two different psychophysical experiments. Observers were exposed to distortions in natural images, and form and motion aftereffects were tested subsequently in a constant stimuli procedure where subjects were asked to judge the skew, or the motion direction of an according test stimulus. Exposure to skewed natural stimuli induced a shift in perceived undistorted form as well as motion direction, when viewing distorted dynamic natural scenes, and also after exposure to static distorted natural images. Therefore, skew adaptation occurred in form and motion for dynamic visual scenes as well as static images. Thus, specifically in the condition of static skewed images and the test feature of motion direction, cortical interactions between motion-form processing presumably contributed to the adaptation process. In a nutshell, interfeature cortical interactions constituted the adaptation process to distortion of PALs. Thus, comprehensive investigation of adaptation to distortions of PALs would benefit from taking into account content richness of the stimuli to be used, like natural images.