A geometric framework for nonlinear visual coding

FP-nets as novel deep networks inspired by vision

Feature-product networks (FP-nets) are inspired by end-stopped cortical cells with FP-units that multiply the outputs of two filters. We enhance state-of-the-art deep networks, such as the ResNet and MobileNet, with FP-units and show that the resulting FP-nets perform better on the Cifar-10 and ImageNet benchmarks. Moreover, we analyze the hyperselectivity of the FP-net model neurons and show that this property makes FP-nets less sensitive to adversarial attacks and JPEG artifacts. We then show that the learned model neurons are end-stopped to different degrees and that they provide sparse representations with an entropy that decreases with hyperselectivity.

Binocular Integration of Perceptually Suppressed Visual Information in Amblyopia

Purpose

The purpose of this study was to assess whether motion information from suppressed amblyopic eyes can influence visual perception.

Methods

Participants with normal vision (n = 20) and with amblyopia (n = 20; 11 anisometropic and 9 strabismic/mixed) viewed dichoptic, orthogonal drifting gratings through a mirror stereoscope. Participants continuously reported form and motion percepts as gratings rivaled for 60 seconds. Responses were binned into categories ranging from binocular integration to complete suppression. Periods when the grating presented to the nondominant/amblyopic eye was suppressed were analyzed further to determine the extent of binocular integration of motion.

Results

Individuals with amblyopia experienced longer periods of non-preferred eye suppression than controls. When the non-preferred eye grating was suppressed, binocular integration of motion occurred 48.1 ± 6.2% and 31.2 ± 5.8% of the time in control and amblyopic participants, respectively. Periods of motion integration from the suppressed eye were significantly non-zero for both groups.

Conclusions

Visual information seen only by a suppressed amblyopic eye can be binocularly integrated and influence the overall visual percept. These findings reveal that visual information subjected to interocular suppression can still contribute to binocular vision and suggest the use of appropriate optical correction for the amblyopic eye to improve image quality for binocular combination.

Sensitivity to gaze-contingent contrast increments in naturalistic movies: An exploratory report and model comparison

Sensitivity to luminance contrast is a prerequisite for all but the simplest visual systems. To examine contrast increment detection performance in a way that approximates the natural environmental input of the human visual system, we presented contrast increments gaze-contingently within naturalistic video freely viewed by observers. A band-limited contrast increment was applied to a local region of the video relative to the observer's current gaze point, and the observer made a forced-choice response to the location of the target (≈25,000 trials across five observers). We present exploratory analyses showing that performance improved as a function of the magnitude of the increment and depended on the direction of eye movements relative to the target location, the timing of eye movements relative to target presentation, and the spatiotemporal image structure at the target location. Contrast discrimination performance can be modeled by assuming that the underlying contrast response is an accelerating nonlinearity (arising from a nonlinear transducer or gain control). We implemented one such model and examined the posterior over model parameters, estimated using Markov-chain Monte Carlo methods. The parameters were poorly constrained by our data; parameters constrained using strong priors taken from previous research showed poor cross-validated prediction performance. Atheoretical logistic regression models were better constrained and provided similar prediction performance to the nonlinear transducer model. Finally, we explored the properties of an extended logistic regression that incorporates both eye movement and image content features. Models of contrast transduction may be better constrained by incorporating data from both artificial and natural contrast perception settings.

Intrinsic dimensionality predicts the saliency of natural dynamic scenes

Since visual attention-based computer vision applications have gained popularity, ever more complex, biologically inspired models seem to be needed to predict salient locations (or interest points) in naturalistic scenes. In this paper, we explore how far one can go in predicting eye movements by using only basic signal processing, such as image representations derived from efficient coding principles, and machine learning. To this end, we gradually increase the complexity of a model from simple single-scale saliency maps computed on grayscale videos to spatiotemporal multiscale and multispectral representations. Using a large collection of eye movements on high-resolution videos, supervised learning techniques fine-tune the free parameters whose addition is inevitable with increasing complexity. The proposed model, although very simple, demonstrates significant improvement in predicting salient locations in naturalistic videos over four selected baseline models and two distinct data labeling scenarios.

Efficient coding and multiple motions

Based on the principle of efficient coding, we present a theoretical framework for how to categorize the basic types of changes that can occur in a spatio-temporal signal. First, theoretical results for the problem of estimating multiple transparent motions are reviewed. Then, confidence measures for the presence of multiple motions are used to derive a basic alphabet of local signal variation that includes motion layers. To better understand and visualize this alphabet, a representation of motions in the projective plane is used. A further, practical contribution is an interactive tool that allows generating multiple motion patterns and displaying them in various apertures. In our framework, we can explain some well-known results on coherent motion and a few more complex perceptual phenomena such as the 2D-1D entrainment effect, but the focus of this paper is on the methods. Our working hypothesis is that efficient representations can be obtained by suppressing all the redundancies that arise if the visual input does not change in a particular direction, or a set of directions. Finally, we assume that human eye movements will tend to avoid the redundant parts of the visual input and report results where our framework has been used to obtain very good predictions of eye movements made on overlaid natural videos.

Component and intrinsic motion integrate in 'dancing bar' illusion

We introduce a new illusion that contradicts common assumptions in the field of visual motion perception. When an unoccluded bar moves at certain speeds and oscillates at certain frequencies, the perceived direction of the bar is not predicted by its intrinsic terminators but is biased to move in the direction orthogonal to its orientation. It appears that the veridical terminator motions are integrated with spurious component motion signals, generating an at times complex pattern of motion around an apparently closed loop path. In the absence of oscillation the effect does not occur. Several factors, including optimal angle, speed, and oscillating distance of the bar, are quantified and possible mechanisms are discussed. In a model, we suggest that the effect arises because of the failure to inhibit spurious component motion signals arising from contours that are nearly oriented along the direction of true motion.