Evidence for Optimal Integration of Visual Feature Representations across Saccades

We explore the visual world through saccadic eye movements, but saccades also present a challenge to visual processing by shifting externally stable objects from one retinal location to another. The brain could solve this problem in two ways: by overwriting preceding input and starting afresh with each fixation or by maintaining a representation of presaccadic visual features in working memory and updating it with new information from the remapped location. Crucially, when multiple objects are present in a scene the planning of eye movements profoundly affects the precision of their working memory representations, transferring limited memory resources from fixation toward the saccade target. Here we show that when humans make saccades, it results in an update of not just the precision of representations but also their contents. When multiple item colors are shifted imperceptibly during a saccade the perceived colors are found to fall between presaccadic and postsaccadic values, with the weight given to each input varying continuously with item location, and fixed relative to saccade parameters. Increasing sensory uncertainty, by adding color noise, biases updating toward the more reliable input, which is consistent with an optimal integration of presaccadic working memory with a postsaccadic updating signal. We recover this update signal and show it to be tightly focused on the vicinity of the saccade target. These results reveal how the nervous system accumulates detailed visual information from multiple views of the same object or scene.

SIGNIFICANCE STATEMENT

This study examines the consequences of saccadic eye movements for the internal representation of visual objects. A saccade shifts the image of a stable visual object from one part of the retina to another. We show that visual representations are built up over these different views of the same object, by combining information obtained before and after each saccade. The weights given to presaccadic and postsaccadic information are determined by the relative reliability of each input. This provides evidence that the visual system combines inputs over time in a statistically optimal way.

A decentralised neural model explaining optimal integration of navigational strategies in insects

Insect navigation arises from the coordinated action of concurrent guidance systems but the neural mechanisms through which each functions, and are then coordinated, remains unknown. We propose that insects require distinct strategies to retrace familiar routes (route-following) and directly return from novel to familiar terrain (homing) using different aspects of frequency encoded views that are processed in different neural pathways. We also demonstrate how the Central Complex and Mushroom Bodies regions of the insect brain may work in tandem to coordinate the directional output of different guidance cues through a contextually switched ring-attractor inspired by neural recordings. The resultant unified model of insect navigation reproduces behavioural data from a series of cue conflict experiments in realistic animal environments and offers testable hypotheses of where and how insects process visual cues, utilise the different information that they provide and coordinate their outputs to achieve the adaptive behaviours observed in the wild.

Signal Integration in Human Visual Speed Perception

Object motion in natural scenes results in visual stimuli with a rich and broad spatiotemporal frequency spectrum. While the question of how the visual system detects and senses motion energies at different spatial and temporal frequencies has been fairly well studied, it is unclear how the visual system integrates this information to form coherent percepts of object motion. We applied a combination of tailored psychophysical experiments and predictive modeling to address this question with regard to perceived motion in a given direction (i.e., stimulus speed). We tested human subjects in a discrimination experiment using stimuli that selectively targeted four distinct spatiotemporally tuned channels with center frequencies consistent with a common speed. We first characterized subjects' responses to stimuli that targeted only individual channels. Based on these measurements, we then predicted subjects' psychometric functions for stimuli that targeted multiple channels simultaneously. Specifically, we compared predictions of three Bayesian observer models that either optimally integrated the information across all spatiotemporal channels, or only used information from the most reliable channel, or formed an average percept across channels. Only the model with optimal integration was successful in accounting for the data. Furthermore, the proposed channel model provides an intuitive explanation for the previously reported spatial frequency dependence of perceived speed of coherent object motion. Finally, our findings indicate that a prior expectation for slow speeds is added to the inference process only after the sensory information is combined and integrated.

Parallel updating and weighting of multiple spatial maps for visual stability during whole body motion

It is known that the brain uses multiple reference frames to code spatial information, including eye-centered and body-centered frames. When we move our body in space, these internal representations are no longer in register with external space, unless they are actively updated. Whether the brain updates multiple spatial representations in parallel, or whether it restricts its updating mechanisms to a single reference frame from which other representations are constructed, remains an open question. We developed an optimal integration model to simulate the updating of visual space across body motion in multiple or single reference frames. To test this model, we designed an experiment in which participants had to remember the location of a briefly presented target while being translated sideways. The behavioral responses were in agreement with a model that uses a combination of eye- and body-centered representations, weighted according to the reliability in which the target location is stored and updated in each reference frame. Our findings suggest that the brain simultaneously updates multiple spatial representations across body motion. Because both representations are kept in sync, they can be optimally combined to provide a more precise estimate of visual locations in space than based on single-frame updating mechanisms.