Active efficient coding explains the development of binocular vision and its failure in amblyopia

Sequential Effects in Reaching Reveal Efficient Coding in Motor Planning

The nervous system utilizes prior information to enhance the accuracy of perception and action. Prevailing models of motor control emphasize Bayesian models, which suggest that the system adjusts the current motor plan by integrating information from previous observations. While Bayesian integration has been extensively examined, those studies usually applied a highly stable and predictable environment. In contrast, in many real-life situations, motor goals change rapidly over time in a relatively unpredictable way, leaving it unclear whether Bayesian integration is useful in those natural environments. An alternative model that leverages prior information to improve performance is efficient coding, which suggests that the motor system maximizes the accuracy by dynamically tuning the allocation of the encoding resources based on environmental statistics. To investigate whether this adaptive mechanism operates in motor planning, we employed center-out reaching tasks with motor goals changing in a relatively unpredictable way, where Bayesian and efficient coding models predict opposite sequential effects. Consistent with the efficient coding model, we found that current movements were biased in the opposite direction of previous movements. These repulsive biases were amplified by intrinsic motor variability. Moreover, movement variability decreased when successive reaches were similar to each other. Together, these effects support the presence of efficient coding in motor planning, a novel mechanism with which the motor system maintains flexibility and high accuracy in dynamic environments.

Models of vision need some action

Bowers et al. focus their criticisms on research that compares behavioral and brain data from the ventral stream with a class of deep neural networks for object recognition. While they are right to identify issues with current benchmarking research programs, they overlook a much more fundamental limitation of this literature: Disregarding the importance of action and interaction for perception.

Efficacy of binocular vision training and Fresnel press-on prism on children with esotropia and amblyopia

PURPOSE

In the process of clinical diagnosis and treatment of amblyopia, we have found that the treatment time of this disease was significantly different among different patients. The purpose of this study was to compare the efficacy of binocular vision training (BVT) and Fresnel press-on prism (FPP) on children with esotropia combined with amblyopia.

METHODS

From May 2015 to December 2018, a total of 101 children aged 3-9 years with esotropia and amblyopia who were in our hospital were enrolled in this randomized clinical trial. They were randomly divided into combined group (48 cases) and prism group (53 cases): the children in the prism group received FPP treatment, and those in the combined group received the combined treatment of BVT and FPP. The visual acuity, the binocular function and the strabismic therapeutic effects were compared between two groups.

RESULTS

After treatment, the visual acuity in both groups was both significantly improved compared with that before treatment (P = 0.0079). The binocular-monocular function, including synoptophore visual function and the Titmus stereopsis, in both groups was significantly improved compared with those before treatment (P < 0.05), and it was more significant in the combined group compared with the prism group (P < 0.05). The cure rate of strabismus was 87.50% (42/48) and 30.19% (16/53) in the combined group and the prism group, respectively, and there was significant difference between groups (P = 0.0036). The cure time was shortened with the lower of the degree of esotropia.

CONCLUSION

BVT combined with FPP can effectively promote the recovery of binocular vision in children with esotropia combined with amblyopia, and some children can achieve complete cure of strabismus.

Predictive coding is a consequence of energy efficiency in recurrent neural networks

Predictive coding is a promising framework for understanding brain function. It postulates that the brain continuously inhibits predictable sensory input, ensuring preferential processing of surprising elements. A central aspect of this view is its hierarchical connectivity, involving recurrent message passing between excitatory bottom-up signals and inhibitory top-down feedback. Here we use computational modeling to demonstrate that such architectural hardwiring is not necessary. Rather, predictive coding is shown to emerge as a consequence of energy efficiency. When training recurrent neural networks to minimize their energy consumption while operating in predictive environments, the networks self-organize into prediction and error units with appropriate inhibitory and excitatory interconnections and learn to inhibit predictable sensory input. Moving beyond the view of purely top-down-driven predictions, we demonstrate, via virtual lesioning experiments, that networks perform predictions on two timescales: fast lateral predictions among sensory units and slower prediction cycles that integrate evidence over time.

Efficient Temporal Coding in the Early Visual System: Existing Evidence and Future Directions

While it is universally accepted that the brain makes predictions, there is little agreement about how this is accomplished and under which conditions. Accurate prediction requires neural circuits to learn and store spatiotemporal patterns observed in the natural environment, but it is not obvious how such information should be stored, or encoded. Information theory provides a mathematical formalism that can be used to measure the efficiency and utility of different coding schemes for data transfer and storage. This theory shows that codes become efficient when they remove predictable, redundant spatial and temporal information. Efficient coding has been used to understand retinal computations and may also be relevant to understanding more complicated temporal processing in visual cortex. However, the literature on efficient coding in cortex is varied and can be confusing since the same terms are used to mean different things in different experimental and theoretical contexts. In this work, we attempt to provide a clear summary of the theoretical relationship between efficient coding and temporal prediction, and review evidence that efficient coding principles explain computations in the retina. We then apply the same framework to computations occurring in early visuocortical areas, arguing that data from rodents is largely consistent with the predictions of this model. Finally, we review and respond to criticisms of efficient coding and suggest ways that this theory might be used to design future experiments, with particular focus on understanding the extent to which neural circuits make predictions from efficient representations of environmental statistics.

Unsupervised learning of haptic material properties

When touching the surface of an object, its spatial structure translates into a vibration on the skin. The perceptual system evolved to translate this pattern into a representation that allows to distinguish between different materials. Here, we show that perceptual haptic representation of materials emerges from efficient encoding of vibratory patterns elicited by the interaction with materials. We trained a deep neural network with unsupervised learning (Autoencoder) to reconstruct vibratory patterns elicited by human haptic exploration of different materials. The learned compressed representation (i.e., latent space) allows for classification of material categories (i.e., plastic, stone, wood, fabric, leather/wool, paper, and metal). More importantly, classification performance is higher with perceptual category labels as compared to ground truth ones, and distances between categories in the latent space resemble perceptual distances, suggesting a similar coding. Crucially, the classification performance and the similarity between the perceptual and the latent space decrease with decreasing compression level. We could further show that the temporal tuning of the emergent latent dimensions is similar to properties of human tactile receptors.

Recurrent processing improves occluded object recognition and gives rise to perceptual hysteresis

Over the past decades, object recognition has been predominantly studied and modelled as a feedforward process. This notion was supported by the fast response times in psychophysical and neurophysiological experiments and the recent success of deep feedforward neural networks for object recognition. Recently, however, this prevalent view has shifted and recurrent connectivity in the brain is now believed to contribute significantly to object recognition — especially under challenging conditions, including the recognition of partially occluded objects. Moreover, recurrent dynamics might be the key to understanding perceptual phenomena such as perceptual hysteresis. In this work we investigate if and how artificial neural networks can benefit from recurrent connections. We systematically compare architectures comprised of bottom-up, lateral, and top-down connections. To evaluate the impact of recurrent connections for occluded object recognition, we introduce three stereoscopic occluded object datasets, which span the range from classifying partially occluded hand-written digits to recognizing three-dimensional objects. We find that recurrent architectures perform significantly better than parameter-matched feedforward models. An analysis of the hidden representation of the models suggests that occluders are progressively discounted in later time steps of processing. We demonstrate that feedback can correct the initial misclassifications over time and that the recurrent dynamics lead to perceptual hysteresis. Overall, our results emphasize the importance of recurrent feedback for object recognition in difficult situations.

Recent understanding of binocular vision in the natural environment with clinical implications

Technological advances in recent decades have allowed us to measure both the information available to the visual system in the natural environment and the rich array of behaviors that the visual system supports. This review highlights the tasks undertaken by the binocular visual system in particular and how, for much of human activity, these tasks differ from those considered when an observer fixates a static target on the midline. The everyday motor and perceptual challenges involved in generating a stable, useful binocular percept of the environment are discussed, together with how these challenges are but minimally addressed by much of current clinical interpretation of binocular function. The implications for new technology, such as virtual reality, are also highlighted in terms of clinical and basic research application.

Performance of biologically grounded models of the early visual system on standard object recognition tasks

Computational neuroscience models of vision and neural network models for object recognition are often framed by different research agendas. Computational neuroscience mainly aims at replicating experimental data, while (artificial) neural networks target high performance on classification tasks. However, we propose that models of vision should be validated on object recognition tasks. At some point, mechanisms of realistic neuro-computational models of the visual cortex have to convince in object recognition as well. In order to foster this idea, we report the recognition accuracy for two different neuro-computational models of the visual cortex on several object recognition datasets. The models were trained using unsupervised Hebbian learning rules on natural scene inputs for the emergence of receptive fields comparable to their biological counterpart. We assume that the emerged receptive fields result in a general codebook of features, which should be applicable to a variety of visual scenes. We report the performances on datasets with different levels of difficulty, ranging from the simple MNIST to the more complex CIFAR-10 or ETH-80. We found that both networks show good results on simple digit recognition, comparable with previously published biologically plausible models. We also observed that our deeper layer neurons provide for naturalistic datasets a better recognition codebook. As for most datasets, recognition results of biologically grounded models are not available yet, our results provide a broad basis of performance values to compare methodologically similar models.

Binocular Integration of Perceptually Suppressed Visual Information in Amblyopia

Purpose

The purpose of this study was to assess whether motion information from suppressed amblyopic eyes can influence visual perception.

Methods

Participants with normal vision (n = 20) and with amblyopia (n = 20; 11 anisometropic and 9 strabismic/mixed) viewed dichoptic, orthogonal drifting gratings through a mirror stereoscope. Participants continuously reported form and motion percepts as gratings rivaled for 60 seconds. Responses were binned into categories ranging from binocular integration to complete suppression. Periods when the grating presented to the nondominant/amblyopic eye was suppressed were analyzed further to determine the extent of binocular integration of motion.

Results

Individuals with amblyopia experienced longer periods of non-preferred eye suppression than controls. When the non-preferred eye grating was suppressed, binocular integration of motion occurred 48.1 ± 6.2% and 31.2 ± 5.8% of the time in control and amblyopic participants, respectively. Periods of motion integration from the suppressed eye were significantly non-zero for both groups.

Conclusions

Visual information seen only by a suppressed amblyopic eye can be binocularly integrated and influence the overall visual percept. These findings reveal that visual information subjected to interocular suppression can still contribute to binocular vision and suggest the use of appropriate optical correction for the amblyopic eye to improve image quality for binocular combination.

The development of active binocular vision under normal and alternate rearing conditions

The development of binocular vision is an active learning process comprising the development of disparity tuned neurons in visual cortex and the establishment of precise vergence control of the eyes. We present a computational model for the learning and self-calibration of active binocular vision based on the Active Efficient Coding framework, an extension of classic efficient coding ideas to active perception. Under normal rearing conditions with naturalistic input, the model develops disparity tuned neurons and precise vergence control, allowing it to correctly interpret random dot stereograms. Under altered rearing conditions modeled after neurophysiological experiments, the model qualitatively reproduces key experimental findings on changes in binocularity and disparity tuning. Furthermore, the model makes testable predictions regarding how altered rearing conditions impede the learning of precise vergence control. Finally, the model predicts a surprising new effect that impaired vergence control affects the statistics of orientation tuning in visual cortical neurons.

Active head rolls enhance sonar-based auditory localization performance

Animals utilize a variety of active sensing mechanisms to perceive the world around them. Echolocating bats are an excellent model for the study of active auditory localization. The big brown bat (Eptesicus fuscus), for instance, employs active head roll movements during sonar prey tracking. The function of head rolls in sound source localization is not well understood. Here, we propose an echolocation model with multi-axis head rotation to investigate the effect of active head roll movements on sound localization performance. The model autonomously learns to align the bat's head direction towards the target. We show that a model with active head roll movements better localizes targets than a model without head rolls. Furthermore, we demonstrate that active head rolls also reduce the time required for localization in elevation. Finally, our model offers key insights to sound localization cues used by echolocating bats employing active head movements during echolocation.

Active efficient coding explains the development of binocular vision and its failure in amblyopia

Brains must operate in an energy-efficient manner. The efficient coding hypothesis states that sensory systems achieve this by adapting neural representations to the statistics of sensory input signals. Importantly, however, these statistics are shaped by the organism’s behavior and how it samples information from the environment. Therefore, optimal performance requires jointly optimizing neural representations and behavior, a theory called active efficient coding. Here, we test the plausibility of this theory by proposing a computational model of the development of binocular vision. The model explains the development of accurate binocular vision under healthy conditions. In the case of refractive errors, however, the model develops an amblyopia-like state and suggests conditions for successful treatment.

The development of vision during the first months of life is an active process that comprises the learning of appropriate neural representations and the learning of accurate eye movements. While it has long been suspected that the two learning processes are coupled, there is still no widely accepted theoretical framework describing this joint development. Here, we propose a computational model of the development of active binocular vision to fill this gap. The model is based on a formulation of the active efficient coding theory, which proposes that eye movements as well as stimulus encoding are jointly adapted to maximize the overall coding efficiency. Under healthy conditions, the model self-calibrates to perform accurate vergence and accommodation eye movements. It exploits disparity cues to deduce the direction of defocus, which leads to coordinated vergence and accommodation responses. In a simulated anisometropic case, where the refraction power of the two eyes differs, an amblyopia-like state develops in which the foveal region of one eye is suppressed due to inputs from the other eye. After correcting for refractive errors, the model can only reach healthy performance levels if receptive fields are still plastic, in line with findings on a critical period for binocular vision development. Overall, our model offers a unifying conceptual framework for understanding the development of binocular vision.