Disruption of Foveal Space Impairs Discrimination of Peripheral Objects

Frequency-specific and periodic masking of peripheral characters by delayed foveal input

The foveal-feedback mechanism supports peripheral object recognition by processing information about peripheral objects in foveal retinotopic visual cortex. When a foveal object is asynchronously presented with a peripheral target, peripheral discrimination performance is affected differently depending on the relationship between the foveal and peripheral objects. However, it is not clear whether the delayed foveal input competes for foveal resources with the information processed by foveal-feedback or masks it. In the current study, we tested these hypotheses by measuring the effect of foveal noise at different spatial frequencies on peripheral discrimination of familiar and novel characters. Our results showed that the impairment of foveal-feedback was strongest for low-spatial frequency noise. A control experiment revealed that for spatially overlapping noise, low-spatial frequencies were more effective than medium-spatial frequencies in the periphery, but vice versa in the fovea. This suggests that the delayed foveal input selectively masks foveal-feedback when it is sufficiently similar to the peripheral information. Additionally, this foveal masking was periodic as evidenced by behavioral oscillations at around 5 Hz. Thus, we conclude that foveal-feedback supports peripheral discrimination of familiar and novel objects by periodically processing peripheral object information.

Perceptual learning across saccades: Feature but not location specific

Perceptual learning is the ability to enhance perception through practice. The hallmark of perceptual learning is its specificity for the trained location and stimulus features, such as orientation. For example, training in discriminating a grating's orientation improves performance only at the trained location but not in other untrained locations. Perceptual learning has mostly been studied using stimuli presented briefly while observers maintained gaze at one location. However, in everyday life, stimuli are actively explored through eye movements, which results in successive projections of the same stimulus at different retinal locations. Here, we studied perceptual learning of orientation discrimination across saccades. Observers were trained to saccade to a peripheral grating and to discriminate its orientation change that occurred during the saccade. The results showed that training led to transsaccadic perceptual learning (TPL) and performance improvements which did not generalize to an untrained orientation. Remarkably, however, for the trained orientation, we found a complete transfer of TPL to the untrained location in the opposite hemifield suggesting high flexibility of reference frame encoding in TPL. Three control experiments in which participants were trained without saccades did not show such transfer, confirming that the location transfer was contingent upon eye movements. Moreover, performance at the trained location, but not at the untrained location, was also improved in an untrained fixation task. Our results suggest that TPL has both, a location-specific component that occurs before the eye movement and a saccade-related component that involves location generalization.

The Role of Foveal Cortex in Discriminating Peripheral Stimuli: The Sketchpad Hypothesis

Foveal (central) and peripheral vision are strongly interconnected to provide an integrated experience of the world around us. Recently, it has been suggested that there is a feedback mechanism that links foveal and peripheral vision. This peripheral-to-foveal feedback differs from other feedback mechanisms in that during visual processing a novel representation of a stimulus is formed in a different cortical region than that of the feedforward representation. The functional role of foveal feedback is not yet completely understood, but some evidence from neuroimaging studies suggests a link with peripheral shape processing. Behavioural and transcranial magnetic stimulation studies show impairment in peripheral shape discrimination when the foveal retinotopic cortex is disrupted post stimulus presentation. This review aims to link these findings to the visual sketchpad hypothesis. According to this hypothesis, foveal retinotopic cortex stores task-relevant information to aid identification of peripherally presented objects. We discuss how the characteristics of foveal feedback support this hypothesis and rule out other possible explanations. We also discuss the possibility that the foveal feedback may be independent of the sensory modality of the stimulation.

Investigating the role of the foveal cortex in peripheral object discrimination

Peripheral object discrimination is hindered by a central dynamic mask presented between 150 and 300 ms after stimulus onset. The mask is thought to interfere with task-relevant feedback coming from higher visual areas to the foveal cortex in V1. Fan et al. (2016) supported this hypothesis by showing that the effect of mask can be further delayed if the task requires mental manipulation of the peripheral target. The main purpose of this study was to better characterize the temporal dynamics of foveal feedback. Specifically, in two experiments we have shown that (1) the effect of foveal noise mask is sufficiently robust to be replicated in an online data collection (2) in addition to a change in sensitivity the mask affects also the criterion, which becomes more conservative; (3) the expected dipper function for sensitivity approximates a quartic with a global minimum at 94 ms, while the best fit for criterion is a quintic with a global maximum at 174 ms; (4) the power spectrum analysis of perceptual oscillations in sensitivity data shows a cyclic effect of mask at 3 and 12 Hz. Overall, our results show that foveal noise affects sensitivity in a cyclic manner, with a global dip emerging earlier than previously found. The noise also affects the response bias, even though with a different temporal profile. We, therefore, suggest that foveal noise acts on two distinct feedback mechanisms, a faster perceptual feedback followed by a slower cognitive feedback.

Attention impedes neural representation of interpolated orientation during perceptual completion

One of the most remarkable functional feats accomplished by visual system is the interpolation of missing retinal inputs based on surrounding information, a process known as perceptual completion. Perceptual completion enables the active construction of coherent, vivid percepts from spatially discontinuous visual information that is prevalent in real-life visual scenes. Despite mounting evidence linking sensory activity enhancement and perceptual completion, surprisingly little is known about whether and how attention, a fundamental modulator of sensory activities, affects perceptual completion. Using EEG-based time-resolved inverted encoding model (IEM), we reconstructed the moment-to-moment representation of the illusory grating that resulted from spatially interpolating the orientation of surrounding inducers. We found that, despite manipulation of observers' attentional focus, the illusory grating representation unfolded in time in a similar manner. Critically, attention to the surrounding inducers simultaneously attenuated the illusory grating representation and delayed its temporal development. Our findings disclosed, for the first time, the suppressive role of selective attention in perceptual completion and were suggestive of a fast, automatic neural machinery that implements the interpolation of missing visual information.

Voxel-to-voxel predictive models reveal unexpected structure in unexplained variance

Encoding models based on deep convolutional neural networks (DCNN) predict BOLD responses to natural scenes in the human visual system more accurately than many other currently available models. However, DCNN-based encoding models fail to predict a significant amount of variance in the activity of most voxels in all visual areas. This failure could reflect limitations in the data (e.g., a noise ceiling), or could reflect limitations of the DCNN as a model of computation in the brain. Understanding the source and structure of the unexplained variance could therefore provide helpful clues for improving models of brain computation. Here, we characterize the structure of the variance that DCNN-based encoding models cannot explain. Using a publicly available dataset of BOLD responses to natural scenes, we determined if the source of unexplained variance was shared across voxels, individual brains, retinotopic locations, and hierarchically distant visual brain areas. We answered these questions using voxel-to-voxel (vox2vox) models that predict activity in a target voxel given activity in a population of source voxels. We found that simple linear vox2vox models increased within-subject prediction accuracy over DCNN-based models for any pair of source/target visual areas, clearly demonstrating that the source of unexplained variance is widely shared within and across visual brain areas. However, vox2vox models were not more accurate than DCNN-based encoding models when source and target voxels came from different brains, demonstrating that the source of unexplained variance was not shared across brains. Importantly, control analyses demonstrated that the source of unexplained variance was not encoded in the mean activity of source voxels, or the activity of voxels in white matter. Interestingly, the weights of vox2vox models revealed preferential connection of target voxel activity to source voxels with adjacent receptive fields, even when source and target voxels were in different functional brain areas. Finally, we found that the prediction accuracy of the vox2vox models decayed with hierarchical distance between the source and target voxels but showed detailed patterns of dependence on hierarchical relationships that we did not observe in DCNNs. Given these results, we argue that the structured variance unexplained by DCNN-based encoding models is unlikely to be entirely caused by non-neural artifacts (e.g., spatially correlated measurement noise) or a failure of DCNNs to approximate the features encoded in brain activity; rather, our results point to a need for brain models that provide both mechanistic and computational explanations for structured ongoing activity in the brain. Keywords: fMRI, encoding models, deep neural networks, functional connectivity.

A review of interactions between peripheral and foveal vision

Visual processing varies dramatically across the visual field. These differences start in the retina and continue all the way to the visual cortex. Despite these differences in processing, the perceptual experience of humans is remarkably stable and continuous across the visual field. Research in the last decade has shown that processing in peripheral and foveal vision is not independent, but is more directly connected than previously thought. We address three core questions on how peripheral and foveal vision interact, and review recent findings on potentially related phenomena that could provide answers to these questions. First, how is the processing of peripheral and foveal signals related during fixation? Peripheral signals seem to be processed in foveal retinotopic areas to facilitate peripheral object recognition, and foveal information seems to be extrapolated toward the periphery to generate a homogeneous representation of the environment. Second, how are peripheral and foveal signals re-calibrated? Transsaccadic changes in object features lead to a reduction in the discrepancy between peripheral and foveal appearance. Third, how is peripheral and foveal information stitched together across saccades? Peripheral and foveal signals are integrated across saccadic eye movements to average percepts and to reduce uncertainty. Together, these findings illustrate that peripheral and foveal processing are closely connected, mastering the compromise between a large peripheral visual field and high resolution at the fovea.

Late disruption of central visual field disrupts peripheral perception of form and color

Evidence from neuroimaging and brain stimulation studies suggest that visual information about objects in the periphery is fed back to foveal retinotopic cortex in a separate representation that is essential for peripheral perception. The characteristics of this phenomenon have important theoretical implications for the role fovea-specific feedback might play in perception. In this work, we employed a recently developed behavioral paradigm to explore whether late disruption to central visual space impaired perception of color. In the first experiment, participants performed a shape discrimination task on colored novel objects in the periphery while fixating centrally. Consistent with the results from previous work, a visual distractor presented at fixation ~100ms after presentation of the peripheral stimuli impaired sensitivity to differences in peripheral shapes more than a visual distractor presented at other stimulus onset asynchronies. In a second experiment, participants performed a color discrimination task on the same colored objects. In a third experiment, we further tested for this foveal distractor effect with stimuli restricted to a low-level feature by using homogenous color patches. These two latter experiments resulted in a similar pattern of behavior: a central distractor presented at the critical stimulus onset asynchrony impaired sensitivity to peripheral color differences, but, importantly, the magnitude of the effect was stronger when peripheral objects contained complex shape information. These results show a behavioral effect consistent with disrupting feedback to the fovea, in line with the foveal feedback suggested by previous neuroimaging studies.

Object categorization in visual periphery is modulated by delayed foveal noise

Behavioral studies in humans indicate that peripheral vision can do object recognition to some extent. Moreover, recent studies have shown that some information from brain regions retinotopic to visual periphery is somehow fed back to regions retinotopic to the fovea and disrupting this feedback impairs object recognition in human. However, it is unclear to what extent the information in visual periphery contributes to human object categorization. Here, we designed two series of rapid object categorization tasks to first investigate the performance of human peripheral vision in categorizing natural object images at different eccentricities and abstraction levels (superordinate, basic, and subordinate). Then, using a delayed foveal noise mask, we studied how modulating the foveal representation impacts peripheral object categorization at any of the abstraction levels. We found that peripheral vision can quickly and accurately accomplish superordinate categorization, while its performance in finer categorization levels dramatically drops as the object presents further in the periphery. Also, we found that a 300-ms delayed foveal noise mask can significantly disturb categorization performance in basic and subordinate levels, while it has no effect on the superordinate level. Our results suggest that human peripheral vision can easily process objects at high abstraction levels, and the information is fed back to foveal vision to prime foveal cortex for finer categorizations when a saccade is made toward the target object.

Perceptual training yields rapid improvements in visually impaired youth

Visual function demands coordinated responses to information over a wide field of view, involving both central and peripheral vision. Visually impaired individuals often seem to underutilize peripheral vision, even in absence of obvious peripheral deficits. Motivated by perceptual training studies with typically sighted adults, we examined the effectiveness of perceptual training in improving peripheral perception of visually impaired youth. Here, we evaluated the effectiveness of three training regimens: (1) an action video game, (2) a psychophysical task that combined attentional tracking with a spatially and temporally unpredictable motion discrimination task, and (3) a control video game. Training with both the action video game and modified attentional tracking yielded improvements in visual performance. Training effects were generally larger in the far periphery and appear to be stable 12 months after training. These results indicate that peripheral perception might be under-utilized by visually impaired youth and that this underutilization can be improved with only ~8 hours of perceptual training. Moreover, the similarity of improvements following attentional tracking and action video-game training suggest that well-documented effects of action video-game training might be due to the sustained deployment of attention to multiple dynamic targets while concurrently requiring rapid attending and perception of unpredictable events.

Temporally flexible feedback signal to foveal cortex for peripheral object recognition

Recent studies have shown that information from peripherally presented images is present in the human foveal retinotopic cortex, presumably because of feedback signals. We investigated this potential feedback signal by presenting noise in fovea at different object-noise stimulus onset asynchronies (SOAs), whereas subjects performed a discrimination task on peripheral objects. Results revealed a selective impairment of performance when foveal noise was presented at 250-ms SOA, but only for tasks that required comparing objects' spatial details, suggesting a task- and stimulus-dependent foveal processing mechanism. Critically, the temporal window of foveal processing was shifted when mental rotation was required for the peripheral objects, indicating that the foveal retinotopic processing is not automatically engaged at a fixed time following peripheral stimulation; rather, it occurs at a stage when detailed information is required. Moreover, fMRI measurements using multivoxel pattern analysis showed that both image and object category-relevant information of peripheral objects was represented in the foveal cortex. Taken together, our results support the hypothesis of a temporally flexible feedback signal to the foveal retinotopic cortex when discriminating objects in the visual periphery.