Dynamic noise backgrounds facilitate target fading

Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention

Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.

Neural correlates of lateral modulation and perceptual filling-in in center-surround radial sinusoidal gratings: an fMRI study

We investigated lateral modulation effects with functional magnetic resonance imaging. We presented radial sinusoidal gratings in random sequence: a scotoma grating with two arc-shaped blank regions (scotomata) in the periphery, one in the left and one in the right visual field, a center grating containing pattern only in the scotoma regions, and a full-field grating where the pattern occupied the whole screen. On each trial, one of the three gratings flickered in counterphase for 10 s, followed by a blank period. Observers were instructed to perform a fixation task and report whether filling-in was experienced during the scotoma condition. The results showed that the blood-oxygen-level-dependent signal was reduced in areas corresponding to the scotoma regions in the full-field compared to the center condition in V1 to V3 areas, indicating a lateral inhibition effect when the surround was added to the center pattern. The univariate analysis results showed no difference between the filling-in and no-filling-in trials. However, multivariate pattern analysis results showed that classifiers trained on activation pattern in V1 to V3 could differentiate between filling-in and no-filling-in trials, suggesting that the neural activation pattern in visual cortex correlated with the subjective percept.

Predictive masking of an artificial scotoma is associated with a system-wide reconfiguration of neural populations in the human visual cortex

The visual brain has the remarkable capacity to complete our percept of the world even when the information extracted from the visual scene is incomplete. This ability to predict missing information based on information from spatially adjacent regions is an intriguing attribute of healthy vision. Yet, it gains particular significance when it masks the perceptual consequences of a retinal lesion, leaving patients unaware of their partial loss of vision and ultimately delaying diagnosis and treatment. At present, our understanding of the neural basis of this masking process is limited which hinders both quantitative modeling as well as translational application. To overcome this, we asked the participants to view visual stimuli with and without superimposed artificial scotoma (AS). We used fMRI to record the associated cortical activity and applied model-based analyzes to track changes in cortical population receptive fields and connectivity in response to the introduction of the AS. We found that throughout the visual field and cortical hierarchy, pRFs shifted their preferred position towards the AS border. Moreover, extrastriate areas biased their sampling of V1 towards sections outside the AS projection zone, thereby effectively masking the AS with signals from spared portions of the visual field. We speculate that the signals that drive these system-wide population modifications originate in extrastriate visual areas and, through feedback, also reconfigure the neural populations in the earlier visual areas.

Microsaccades mediate perceptual alternations in Monet's "Impression, sunrise"

Troxler fading, the perceptual disappearance of stationary images upon sustained fixation, is common for objects with equivalent luminance to that of the background. Previous work showed that variations in microsaccadic rates underlie the perceptual vanishing and intensification of simple stimuli, such as Gabor patches. Here, we demonstrate that microsaccade dynamics also contribute to Troxler fading and intensification during the viewing of representational art. Participants fixated a small spot while viewing either a Gabor patch on a blank background, or Monet's painting "Impression, Sunrise." They continuously reported, via button press/release, whether the Gabor patch, or the sun in Monet's painting, was fading versus intensifying, while their eye movements were recorded with high precision. Microsaccade rates peaked before reports of increased visibility, and dropped before reports of decreased visibility or fading, both when viewing Gabor patches and Monet's sun. These results reveal that the relationship between microsaccade production and the reversal and prevention of Troxler fading applies not only to the viewing of contrived stimuli, but also to the observation of "Impression, Sunrise." Whether or not perceptual fading was consciously intended by Monet, our findings indicate that observers' oculomotor dynamics are a contributor to the cornerstone of Impressionism.

The SSVEP tracks attention, not consciousness, during perceptual filling-in

Research on the neural basis of conscious perception has almost exclusively shown that becoming aware of a stimulus leads to increased neural responses. By designing a novel form of perceptual filling-in (PFI) overlaid with a dynamic texture display, we frequency-tagged multiple disappearing targets as well as their surroundings. We show that in a PFI paradigm, the disappearance of a stimulus and subjective invisibility is associated with increases in neural activity, as measured with steady-state visually evoked potentials (SSVEPs), in electroencephalography (EEG). We also find that this increase correlates with alpha-band activity, a well-established neural measure of attention. These findings cast doubt on the direct relationship previously reported between the strength of neural activity and conscious perception, at least when measured with current tools, such as the SSVEP. Instead, we conclude that SSVEP strength more closely measures changes in attention.

Stronger perceptual filling-in of spatiotemporal information in the blind spot compared with artificial gaps

Complete visual information about a scene and the objects within it is often not available to us. For example, objects may be partly occluded by other objects or have sections missing. In the retinal blind spot, there are no photoreceptors and visual input is not detected. However, owing to perceptual filling-in by the visual system we often do not perceive these gaps. There is a lack of consensus on how much of the mechanism for perceptual filling-in is similar in the case of a natural scotoma, such as the blind spot, and artificial scotomata, such as sections of the stimulus being physically removed. Part of the difficulty in assessing this relationship arises from a lack of direct comparisons between the two cases, with artificial scotomata being tested in different locations in the visual field compared with the blind spot. The peripheral location of the blind spot may explain its enhanced filling-in compared with artificial scotomata, as reported in previous studies. In the present study, we directly compared perceptual filling-in of spatiotemporal information in the blind spot and artificial gaps of the same size and eccentricity. We found stronger perceptual filling-in in the blind spot, suggesting improved filling-in for the blind spot reported in previous studies cannot be simply attributed to its peripheral location.

Changes in visibility as a function of spatial frequency and microsaccade occurrence

Fixational eye movements (FEMs), including microsaccades, drift, and tremor, shift our eye position during ocular fixation, producing retinal motion that is thought to help visibility by counteracting neural adaptation to unchanging stimulation. Yet, how each FEM type influences this process is still debated. Recent studies found little to no relationship between microsaccades and visual perception of spatial frequencies (SF). However, these conclusions were based on coarse analyses that make it hard to appreciate the actual effects of microsaccades on target visibility as a function of SF. Thus, how microsaccades contribute to the visibility of stimuli of different SFs remains unclear. Here, we asked how the visibility of targets of various SFs changed over time, in relationship with concurrent microsaccade production. Participants continuously reported on changes in target visibility, allowing us to time-lock ongoing changes in microsaccade parameters to perceptual transitions in visibility. Microsaccades restored/increased the visibility of low SF targets more efficiently than that of high SF targets. Yet, microsaccade rates rose before periods of increased visibility, and dropped before periods of diminished visibility, for all the SFs tested, suggesting that microsaccades boosted target visibility across a wide range of SFs. Our data also indicate that visual stimuli fade/become harder to see less often in the presence of microsaccades. In addition, larger microsaccades restored/increased target visibility more effectively than smaller microsaccades. These combined results support the proposal that microsaccades enhance visibility across a broad variety of SFs.

Motion-Dependent Filling-In of Spatiotemporal Information at the Blind Spot

We usually do not notice the blind spot, a receptor-free region on the retina. Stimuli extending through the blind spot appear filled in. However, if an object does not reach through but ends in the blind spot, it is perceived as “cut off” at the boundary. Here we show that even when there is no corresponding stimulation at opposing edges of the blind spot, well known motion-induced position shifts also extend into the blind spot and elicit a dynamic filling-in process that allows spatial structure to be extrapolated into the blind spot. We presented observers with sinusoidal gratings that drifted into or out of the blind spot, or flickered in counterphase. Gratings moving into the blind spot were perceived to be longer than those moving out of the blind spot or flickering, revealing motion-dependent filling-in. Further, observers could perceive more of a grating’s spatial structure inside the blind spot than would be predicted from simple filling-in of luminance information from the blind spot edge. This is evidence for a dynamic filling-in process that uses spatiotemporal information from the motion system to extrapolate visual percepts into the scotoma of the blind spot. Our findings also provide further support for the notion that an explicit spatial shift of topographic representations contributes to motion-induced position illusions.

Individually different weighting of multiple processes underlies effects of metacontrast masking

Metacontrast masking occurs when a mask follows a target stimulus in close spatial proximity. Target visibility varies with stimulus onset asynchrony (SOA) between target and mask in individually different ways leading to different masking functions with corresponding phenomenological reports. We used individual differences to determine the processes that underlie metacontrast masking. We assessed individual masking functions in a masked target discrimination task using different masking conditions and applied factor-analytical techniques on measures of sensitivity. Results yielded two latent variables that (1) contribute to performance with short and long SOA, respectively, (2) relate to specific stimulus features, and (3) differentially correlate with specific subjective percepts. We propose that each latent variable reflects a specific process. Two additional processes may contribute to performance with short and long SOAs, respectively. Discrimination performance in metacontrast masking results from individually different weightings of two to four processes, each of which contributes to specific subjective percepts.

Microsaccades restore the visibility of minute foveal targets

Stationary targets can fade perceptually during steady visual fixation, a phenomenon known as Troxler fading. Recent research found that microsaccades—small, involuntary saccades produced during attempted fixation—can restore the visibility of faded targets, both in the visual periphery and in the fovea. Because the targets tested previously extended beyond the foveal area, however, the ability of microsaccades to restore the visibility of foveally-contained targets remains unclear. Here, subjects reported the visibility of low-to-moderate contrast targets contained entirely within the fovea during attempted fixation. The targets did not change physically, but their visibility varied intermittently during fixation, in an illusory fashion (i.e., foveal Troxler fading). Microsaccade rates increased significantly before the targets became visible, and decreased significantly before the targets faded, for a variety of target contrasts. These results support previous research linking microsaccade onsets to the visual restoration of peripheral and foveal targets, and extend the former conclusions to minute targets contained entirely within the fovea. Our findings suggest that the involuntary eye movements produced during attempted fixation do not always prevent fading—in either the fovea or the periphery—and that microsaccades can restore perception, when fading does occur. Therefore, microsaccades are relevant to human perception of foveal stimuli.

Contrast magnitude and polarity effects on color filling-in along cardinal color axes

Color filling-in is the phenomenon in which the color of a visual area is perceived as the color that is only presented in an adjacent area. In a stimulus with multiple edges, color filling-in can occur along any edge and in both centripetal and centrifugal directions when maintaining steady fixation. The current study aimed to investigate the role of chromatic contrast magnitude and polarity along the two chromaticity cardinal axes and the interaction of the axes in the color filling-in process. In Experiment 1, the color filling-in process was examined using stimuli with three different regions and two edges. The three regions had chromaticities that varied only in one of the chromaticity axes. In Experiment 2, the regions along both edges differed in chromaticity along both axes. The results showed that the contrast magnitudes and polarity relationship of the two edges worked together to determine the filled-in direction and time course of the filled-in percepts. Further, the results pointed to a common mechanism mediating the color filling-in process along the two cardinal axes, and the two axes did not act independently in this process.

Simulations of induced visual scene fading with boundary offset and filling-in

Blurred images can appear to fade to uniform brightness and color when viewed with some types of visual transient stimuli. Simons et al. (2006) identified the conditions where such scene fading occurs and noted that their findings were inconsistent with mechanisms that have been used to explain other fading effects. We show that their empirical findings are consistent with a neural model of visual perception that hypothesizes filling-in of brightness and color that is constrained by signals from a boundary contour system. Certain types of transients can weaken the boundary responses and thereby induce scene fading. The simulations explain how even small transient changes can produce scene fading effects across large parts of an image.

Microsaccades and blinks trigger illusory rotation in the "rotating snakes" illusion

Certain repetitive arrangements of luminance gradients elicit the perception of strong illusory motion. Among them, the "Rotating Snakes Illusion" has generated a large amount of interest in the visual neurosciences, as well as in the public. Prior evidence indicates that the Rotating Snakes illusion depends critically on eye movements, yet the specific eye movement types involved and their associated neural mechanisms remain controversial. According to recent reports, slow ocular drift--a nonsaccadic type of fixational eye movement--drives the illusion, whereas microsaccades produced during attempted fixation fail to do so. Here, we asked human subjects to indicate the presence or absence of rotation during the observation of the illusion while we simultaneously recorded their eye movements with high precision. We found a strong quantitative link between microsaccade and blink production and illusory rotation. These results suggest that transient oculomotor events such as microsaccades, saccades, and blinks, rather than continuous drift, act to trigger the illusory motion in the Rotating Snakes illusion.

Illusory contours over pathological retinal scotomas

Our visual percepts are not fully determined by physical stimulus inputs. Thus, in visual illusions such as the Kanizsa figure, inducers presented at the corners allow one to perceive the bounding contours of the figure in the absence of luminance-defined borders. We examined the discrimination of the curvature of these illusory contours that pass across retinal scotomas caused by macular degeneration. In contrast with previous studies with normal-sighted subjects that showed no perception of these illusory contours in the region of physiological scotomas at the optic nerve head, we demonstrated perfect discrimination of the curvature of the illusory contours over the pathological retinal scotoma. The illusion occurred despite the large scar around the macular lesion, strongly reducing discrimination of whether the inducer openings were acute or obtuse and suggesting that the coarse information in the inducers (low spatial frequency) sufficed. The result that subjective contours can pass through the pathological retinal scotoma suggests that the visual cortex, despite the loss of bottom-up input, can use low-spatial frequency information from the inducers to form a neural representation of new complex geometrical shapes inside the scotoma.

Opposite effects of perceptual and working memory load on perceptual filling-in of an artificial scotoma

A target presented on a background of dynamic noise disappears from awareness after a few seconds of maintained peripheral viewing. Whereas the effects of bottom-up factors in such filling-in are well documented, the roles of different top-down functions remain relatively unexplored. Here, we investigated the roles of attention and working memory (WM) by manipulating load in concurrent tasks while participants reported filling-in of a peripheral target. In Experiment 1, increasing perceptual load reduced the probability of filling-in and increased the latency of its occurrence. In Experiment 2, increasing WM load shortened the time before filling-in occurred--the opposite effect to increasing perceptual load. These results demonstrate that different top-down functions may have dissociable effects on filling-in.

Seeing grating-textured surface begins at the border

Two experiments were conducted to reveal that the human visual system represents grating texture surface using a border-to-interior strategy. This strategy dictates that the visual system first registers the surface boundary contour and then sequentially spreads texture from the border to the interior of the image. Our experiments measured the perceived grating texture surface at various stimulus durations after the onset of a grating texture image. We found that the grating texture is initially seen near the boundary contours, with eventual spreading inward to the center of the image. To quantify the observation, the extent of the texture spreading from the boundary contour is measured as a function of the stimulus duration (30-500 ms). This allows us to analyze the texture spreading in retinal and cortical distances, based on human fMRI studies of the cortical magnification factor in cortical areas V1-V4, and to derive the spreading speed. We found that the spreading speed is constant when scaled according to the cortical distance. Similar findings are obtained no matter whether the grating texture image is presented monocularly or dichoptically, suggesting the generality of the border-to-interior strategy for representing surfaces.

The Role of Mask Coherence in Motion-Induced Blindness

Motion-induced blindness (MIB) is the perceived disappearance of a salient target when surrounded by a moving mask. Much research has focused on the role of target characteristics on perceived disappearance by a coherently moving mask. However, we asked a different question: mainly, are there certain characteristics about the mask that can impact disappearance? To address this, we behaviorally tested whether MIB is enhanced or reduced by the property of common fate. In experiments 1, 2, and 3, we systematically manipulated the motion coherence of the mask and measured the amount of target disappearance. Results showed that, as mask coherence increased, perceived target disappearance decreased. This pattern was unaffected by the lifetime of the moving dots, the dot density of the motion stimulus, or the target eccentricity. In experiment 4, we investigated whether the number of motion directions contained in an incoherent mask could account for our findings. Using masks containing 1, 3, and 5 motion directions, we found that disappearance did not increase proportionally to the number of motion directions. We discuss our findings in line with current proposed mechanisms of MIB.

A new taxonomy for perceptual filling-in

Perceptual filling-in occurs when structures of the visual system interpolate information across regions of visual space where that information is physically absent. It is a ubiquitous and heterogeneous phenomenon, which takes place in different forms almost every time we view the world around us, such as when objects are occluded by other objects or when they fall behind the blind spot. Yet, to date, there is no clear framework for relating these various forms of perceptual filling-in. Similarly, whether these and other forms of filling-in share common mechanisms is not yet known. Here we present a new taxonomy to categorize the different forms of perceptual filling-in. We then examine experimental evidence for the processes involved in each type of perceptual filling-in. Finally, we use established theories of general surface perception to show how contextualizing filling-in using this framework broadens our understanding of the possible shared mechanisms underlying perceptual filling-in. In particular, we consider the importance of the presence of boundaries in determining the phenomenal experience of perceptual filling-in.

Eye movements under various conditions of image fading

Under normal viewing conditions, the image on the retina is always in motion. Images fade and may eventually disappear when the physiological motion of the retinal stimulus is reduced or eliminated. According to a widespread theory, microsaccades are responsible for maintaining visibility during fixation. However, while it is clear that the sudden changes in visual input caused by microsaccades are sufficient to restore visibility, it has long been questioned whether this effect might be an epiphenomenon, rather than an important function of microsaccades. In this study, we compared the eye movements measured under conditions that either simulated or induced loss of visibility to those recorded when fading did not occur. Both drifts and microsaccades were unaffected by changes in the stimulus contrast and bandwidth that recreated the percept experienced during image fading. Under retinal stabilization, a condition in which observers reported fading, microsaccade rates decreased, instead of increasing as predicted by the fading prevention hypothesis. While image fading had no influence on oculomotor activity, eye movements were instead strongly modulated by the onset of the stimulus and by the requested precision of fixation. Microsaccades occurred more frequently and were more corrective for preceding drifts during accurate fixation on a cue than during relaxed fixation on a region of the screen. These results do not support a causal relationship between image fading and microsaccade production and show that the precision of required fixation is a major contributor to microsaccades.

Eye movement inhibits the facilitation of perceptual filling-in

When a small figure is presented in human peripheral vision, it becomes invisible and invaded by surrounding texture, within a few seconds. This visual illusion is called perceptual filling-in. Time to filling-in (filling-in time) is varied by the properties of small figure, surround texture and some experimental conditions. In our preliminary study (Yokota, IEEE/IC-EMBS 2005), we found that incomplete fixation distributes filling-in time. Furthermore, that we can see nothing by restraining eye movement artificially is well known. Therefore, we can consider that filling-in time is influenced by eye movement. Although it has been recently reported that eye movement influences the filling-in occurrence (Martinez-Conde, Neuron 2006), the relation between eye movement and the filling-in time has rarely been reported. For this study, we measured the filling-in time for three subjects, for four surrounding textures, with simultaneous recording of eye movement. The results show that the filling-in time correlates to the standard deviation of the power of the eye distance from the fixation point. Furthermore, we found relatively strong correlation between the filling-in time and the power of high frequency component 50-200 (Hz) in the eye movement, though the correlation of the power of low frequency component 10-50 (Hz) is not so high. Thus we suppose that filling-in is inhibited by small involuntary eye movement.

Feature mixing rather than feature replacement during perceptual filling-in

'Filling-in' occurs when a retinally stabilized object subjectively appears to vanish following perceptual fading of its boundaries. The term 'filling-in' literally means that information about the apparently vanished object is lost and replaced solely by information arising from the surrounding background. However, we find evidence that the mechanism of 'filling-in' can actually involve a process of 'feature mixing' rather than 'feature replacement,' whereby features on either side of a perceptually faded boundary merge. Here we investigate the properties of feature mixing and specify certain conditions under which such mixing occurs. Our results show that, when using visual stimuli composed of spatially alternating stripes containing different luminances or motion signals, and when using the neon-color-spreading paradigm, the filled-in luminance, motion, or color is approximately the area and magnitude weighted average of the background and the foreground luminance, motion, or color, respectively. Together, these results demonstrate that, under at least certain conditions, 'filling-in' may involve a process of feature mixing or feature averaging rather than one of feature replacement.

Microsaccades drive illusory motion in the Enigma illusion

Visual images consisting of repetitive patterns can elicit striking illusory motion percepts. For almost 200 years, artists, psychologists, and neuroscientists have debated whether this type of illusion originates in the eye or in the brain. For more than a decade, the controversy has centered on the powerful illusory motion perceived in the painting Enigma, created by op-artist Isia Leviant. However, no previous study has directly correlated the Enigma illusion to any specific physiological mechanism, and so the debate rages on. Here, we show that microsaccades, a type of miniature eye movement produced during visual fixation, can drive illusory motion in Enigma. We asked subjects to indicate when illusory motion sped up or slowed down during the observation of Enigma while we simultaneously recorded their eye movements with high precision. Before "faster" motion periods, the rate of microsaccades increased. Before "slower/no" motion periods, the rate of microsaccades decreased. These results reveal a direct link between microsaccade production and the perception of illusory motion in Enigma and rule out the hypothesis that the origin of the illusion is purely cortical.

The twinkle aftereffect is pre-cortical and is independent of filling-in

A real or artificial scotoma within a dynamic noise field fills in within a few seconds. When the dynamic noise is replaced with a homogenous field, a twinkling after effect (TwAE) is induced exclusively in the location of the former scotoma. We are employing the appearance of the TwAE to perform rapid perimetry in patients with retinal scotomas. To analyze the loci within the visual system and the mechanisms of filling-in and the TwAE, we examined their orientation tuning, inter-ocular transfer, and threshold versus contrast functions by measuring contrast detection thresholds for stimuli presented in areas that were filled-in or contained the TwAE. For filling-in, detection thresholds were narrowly tuned for orientation, transferred interocularly, and rose monotonically with the contrast of a surround pattern. These results indicate that surround suppression and filling-in involve inhibitory processes originating at cortical stages of visual processing. Threshold versus contrast functions were weakly dipper-shaped for the TwAE, did not transfer inter-ocularly, and were not tuned for orientation. These results indicate that the TwAE involves additive noise that is pre-cortical in origin and that it is distinct from filling-in.

“Perceptual Scotomas”

In motion-induced blindness (MIB), salient objects in full view can repeatedly fluctuate into and out of conscious awareness when superimposed onto certain global moving patterns. Here we suggest a new account of this striking phenomenon: Rather than being a failure of visual processing, MIB may be a functional product of the visual system's attempt to separate distal stimuli from artifacts of damage to the visual system itself. When a small object is invariant despite changes that are occurring to a global region of the surrounding visual field, the visual system may discount that stimulus as akin to a scotoma, and may thus expunge it from awareness. We describe three experiments demonstrating new phenomena predicted by this account and discuss how it can also explain several previous results.

Induced internal noise in perceptual artificial scotomas created by surrounding dynamic noise

Research has shown that exposure to a homogeneous gray patch surrounded by a dynamic noise background causes filling-in of the artificial scotoma by the twinkling noise from the surround. When the background is switched off, observers report perception of a prolonged patch of twinkling noise in the unstimulated area. We studied the effects of exposure to a centrally presented artificial scotoma and the twinkling aftereffect on the threshold for detecting a foveal Gabor patch embedded in external scotoma noise. The detection thresholds were mainly elevated in the absence of scotoma noise and less affected at higher levels of scotoma noise. The analysis of the experimental data using the equivalent input noise approach revealed that the reduced contrast sensitivity is due to induced internal noise whose variance is proportional to the strength of the surrounding noise. We did not find significant effects on the internal noise in a control experiment using flickering Gaussian noise samples of 1.6 Hz which did not cause filling-in and dynamic afterimage. These findings suggest that the perceptual phenomena caused by artificial scotomas may reflect increased variability of neural activity due to long-range interactions between the surrounding noise and unstimulated region of the artificial scotoma.

[Adaptation to central scotoma. Part II. Perceptual filling-in phenomenon]

The visual cortex may reorganize after occurrence of a scotoma. Different experimental studies have shown that there is an attempt to minimize the impact of the scotoma, the missing information being filled-in by surrounding information. The clinical consequences of this filling-in phenomenon have been extensively studied by Safran and co-workers. The present review summarizes the current literature on this phenomenon and its clinical consequences. Furthermore, the authors present their own experience with the filling-in phenomenon in patients with age-related macular degeneration. Their study shows that, with few exceptions, the phenomenon only occurs in patients with bilateral central scotoma, in their better eye.

Subjective disappearance of a target by flickering flankers

This study examined the subjective disappearance of a visual object induced by a neighboring flickering ring (Experiments 1 and 2), a set of four flickering dots (Experiment 3), and apparent motion (Experiment 4) as flickering flankers. Observers were asked to report whether a target disappeared during 10 s of stimulus presentation. We used the proportion of disappearance as a measure of performance. Interestingly, subjective disappearance was rarely observed when flickering flankers were presented with a separation of less than 0.5 degrees from the target. However, disappearance was observed when dynamic random-dot patterns were presented with a separation of less than 0.5 degrees from the target border (Experiment 5). Our results indicate that the flicker of flankers near the target disturbs target adaptation or attentional inhibition, causing persistent target representation in higher-order object selection, and resulting in non-disappearance of the target.

Facilitation of perceptual filling-in for spatio-temporal frequency of dynamic textures

Objects are perceived to fade and disappear within a few seconds under certain conditions when a small object surrounded by a moving texture is presented in human peripheral vision. This phenomenon is called perceptual filling-in or fading. Investigation of filling-in properties is important to understand visual information capture and processing. Previous studies have adopted filling-in time to evaluate the facilitation of filling-in. From this viewpoint, we propose a model of the filling-in process to address the phenomenon by which a small homogeneous area (filling-in target), which is surrounded by spatio-temporal frequency limited random-dot dynamic textures, is presented to an observer's peripheral vision (Proc.IC-EMBS2003). The model expresses target distinguishability from the surrounding texture. This study measured time to filling-in for various spatio-temporal frequencies of target-surrounding dynamic textures. Spatio-temporal frequency sensitivity of human vision was also estimated. Applying these results to the proposed model, it was suggested that M-channel pathway of LGN facilitates perceptual filling-in. In contrast, the P-channel pathway is assumed not to facilitate, but rather inhibit, filling-in.

Influence of small eye movement on perceptual filling-in time

When a small area that has a different texture from its surroundings is presented to a subject's peripheral vision, that person perceives that the area is filled by its surrounding texture. It disappears within a few seconds under certain circumstances. This illusion is called filling-in. The filling-in time depends on textural properties, the area's size, the eccentricity with which the small area is projected, and so on. Filling-in characteristics must be elucidated to understand the mode of information processing in human vision because filling-in has been considered to contribute greatly to capturing external visual information. Facilitation of filling-in is generally evaluated using the filling-in time. Furthermore, it is well-known that we can see nothing by restraining eye movement artificially. Eye movement is important to acquire visual information. Therefore, we can suppose that facilitation of filling-in is influenced by eye movement. Although it has been recently indicated that eye movement influences the filling-in time while measuring time to filling-in, the relationship between eye movement and the filling-in time has rarely been reported. In this study, we measured the filling-in time, with simultaneous recording of eye movement. Results showed that the filling-in time correlates moderately or weakly with eye movement, under the condition that complete fixation is achieved.

Peripheral fading with monocular and binocular viewing

This study measured the fading times of peripheral targets as a function of whether viewing was monocular or binocular, and of brightness contrast. Data from a binocularly normal group showed Troxler fading to be significantly faster with monocular (i.e., patched) than with binocular viewing. In contrast, one-eyed observers showed significantly longer fading times than the two-eyed observers viewing monocularly and equivalent times to their binocular viewing. A control experiment showed that these findings were not due to worse fixation stability, larger pupil sizes, or an unusually large blinking rate in the enucleated group. The enucleated group actually exhibited a slight miosis, equivalent fixation stability, and a normal blinking rate. In both experiments, the times to fading of all observers were a function of brightness contrast. We conclude that in binocularly normal observers patching or closing one eye does not produce monocular vision but rather a condition of weak binocular rivalry, and that the absence of inhibitory binocular interactions in the enucleated group may explain, in part, their resistance to fading and their superior performance in other contrast-defined tasks.

In honour of Lothar Spillmann - filling-in, wiggly lines, adaptation, and aftereffects

I have studied a number of visual phenomena that Lothar Spillmann has already elucidated. These include: Neon spreading: when a small red cross is superimposed on intersecting black lines, the red cross seems to spread out into an illusory disk. Unlike the Hermann grid, neon spreading is relatively unaffected when the black lines are curved or wiggly. This suggests that the Hermann grid, but not neon spreading, involves long-range interactions. Neon spreading can be shown in random-dot patterns, even without intersections. It is strongest when the red crosses are equiluminous with the gray background. Adaptation, aftereffects, and filling-in: direct and induced aftereffects of color, motion, and dimming. Artificial scotomata and filling-in: the "dam" theory is false. Staring at wiggly lines or irregularly scattered dots makes them gradually appear straighter, or more regularly spaced. I present evidence that irregularity is actually a visual dimension to which the visual system can adapt. Conjectures on the nature of peripheral fading and of motion-induced blindness. Some failed experiments on correlated visual inputs and cortical plasticity.

Contrast dependency in perceptual filling-in

Two experiments were conducted to investigate how stimulus contrast affected the time required for perceptual filling-in. The stimuli consisted of a Gabor patch (target) and a circular grating region (surround). In Experiment 1, the target contrast was manipulated, and the surround contrast was fixed. Filling-in was significantly delayed with higher target contrast, but this delay was observed only when the target contrast exceeded the surround contrast. In Experiment 2, however, a much smaller effect of changing the surround contrast occurred. Possible reasons for this asymmetric effect are discussed.

The neural mechanisms of perceptual filling-in

Filling-in is a perceptual phenomenon in which a visual attribute such as colour, brightness, texture or motion is perceived in a region of the visual field even though such an attribute exists only in the surround. Filling-in dramatically reveals the dissociation between the retinal input and the percept, and raises fundamental questions about how these two relate to each other. Filling-in is observed in various situations, and is an essential part of our normal surface perception. Here, I review recent experiments examining brain activities associated with filling-in, and discuss possible neural mechanisms underlying this remarkable perceptual phenomenon. The evidence shows that neuronal activities in early visual cortical areas are involved in filling-in, providing new insights into visual cortical functions.

Effects of Selective Attention on Perceptual Filling-in

After few seconds, a figure steadily presented in peripheral vision becomes perceptually filled-in by its background, as if it “disappeared”. We report that directing attention to the color, shape, or location of a figure increased the probability of perceiving filling-in compared to unattended figures, without modifying the time required for filling-in. This effect could be augmented by boosting attention. Furthermore, the frequency distribution of filling-in response times for attended figures could be predicted by multiplying the frequencies of response times for unattended figures with a constant. We propose that, after failure of figure–ground segregation, the neural interpolation processes that produce perceptual filling-in are enhanced in attended figure regions. As filling-in processes are involved in surface perception, the present study demonstrates that even very early visual processes are subject to modulation by cognitive factors.

Fixational eye movements in normal and pathological vision

Most of our visual experience is driven by the eye movements we produce while we fixate our gaze. In a sense, our visual system thus has a built-in contradiction: when we direct our gaze at an object of interest, our eyes are never still. Therefore the perception, physiology, and computational modeling of fixational eye movements is critical to our understanding of vision in general, and also to the understanding of the neural computations that work to overcome neural adaptation in normal subjects as well as in clinical patients. Moreover, because we are not aware of our fixational eye movements, they can also help us understand the underpinnings of visual awareness. Research in the field of fixational eye movements faded in importance for several decades during the late 20th century. However, new electrophysiological and psychophysical data have now rejuvenated the field. The last decade has brought significant advances to our understanding of the neuronal and perceptual effects of fixational eye movements, with crucial implications for neural coding, visual awareness, and perception in normal and pathological vision. This chapter will review the type of neural activity generated by fixational eye movements at different levels in the visual system, as well as the importance of fixational eye movements for visual perception in normal vision and in visual disease. Special attention will be given to microsaccades, the fastest and largest type of fixational eye movement.

Perceptual filling-in: More than the eye can see

When a gray figure is surrounded by a background of dynamic texture, fixating away from the figure for several seconds will result in an illusory replacement of the figure by its background. This visual illusion is referred to as perceptual filling-in. The study of filling-in is important, because the underlying neural processes compensate for imperfections in our visual system (e.g., the blind spot) and contribute to normal surface perception. A long-standing question has been whether perceptual filling-in results from symbolic tagging of surface regions in higher order cortex (ignoring the absence of information), or from active neural interpolation in lower order visual areas (active filling-in of information). The present chapter reviews a number of psychophysical studies in human subjects and physiological experiments in monkeys to evaluate the above two hypotheses. The data combined show that there is strong evidence for neural interpolation processes in retinotopically organized, lower order areas, but that there is also a role for higher order perceptual and cognitive factors such as attention.

From perceptive fields to Gestalt

Studies on visual psychophysics and perception conducted in the Freiburg psychophysics laboratory during the last 35 years are reviewed. Many of these were inspired by single-cell neurophysiology in cat and monkey. The aim was to correlate perceptual phenomena and their effects to possible neuronal mechanisms from retina to visual cortex and beyond. Topics discussed include perceptive field organization, figure-ground segregation and grouping, fading and filling-in, and long-range color interaction. While some of these studies succeeded in linking perception to neuronal response patterns, others require further investigation. The task of probing the human brain with perceptual phenomena continues to be a challenge for the future.

Microsaccades Counteract Visual Fading during Fixation

Our eyes move continually, even while we fixate our gaze on an object. If fixational eye movements are counteracted, our perception of stationary objects fades completely, due to neural adaptation. Some studies have suggested that fixational microsaccades refresh retinal images, thereby preventing adaptation and fading. However, other studies disagree, and so the role of microsaccades remains unclear. Here, we correlate visibility during fixation to the occurrence of microsaccades. We asked subjects to indicate when Troxler fading of a peripheral target occurs, while simultaneously recording their eye movements with high precision. We found that before a fading period, the probability, rate, and magnitude of microsaccades decreased. Before transitions toward visibility, the probability, rate, and magnitude of microsaccades increased. These results reveal a direct link between suppression of microsaccades and fading and suggest a causal relationship between microsaccade production and target visibility during fixation.

A common mechanism for perceptual filling-in and motion-induced blindness

Perceptual-filling-in (PFI) and motion-induced-blindness (MIB) are two phenomena of temporary blindness in which, after prolonged viewing, perceptually salient targets repeatedly disappear and reappear, amidst a field of distracters (i.e., non-targets). Past studies have shown that boundary adaptation is important in PFI, and that depth ordering between target and distracter pattern is important in MIB. Here we show that the reverse is also true; that boundary adaptation is important in MIB, and that depth ordering is important in PFI. Results corroborate our earlier conjecture that PFI and MIB are highly related phenomena that share a common underlying mechanism. We argue that this mechanism involves boundary adaptation, but also that the depth effect shows that boundary adaptation can be no more than a sufficient cause of PFI and MIB, and not a necessary one.

Linking motion-induced blindness to perceptual filling-in

"Motion-induced blindness" and "perceptual filing-in" are two phenomena in which perceptually salient stimuli repeatedly disappear and reappear after prolonged viewing. Despite the many similarities between MIB and PFI, two differences suggest that they could be unrelated phenomena: (1) An area surrounded by background stimuli can be perceived to disappear completely in PFI but not in MIB and (2) high contrast stimuli are perceived to disappear less easily in PFI but, remarkably enough, more easily in MIB. In this article we show that the apparent differences between MIB and PFI disappear when eccentricity, contrast, and perceptual grouping are taken into account and that both are most likely caused by the same underlying mechanism.

A role for cortical crosstalk in the binding problem: stimulus-driven correlations that link color, form, and motion

The putative independence of cortical mechanisms for color, form, and motion raises the binding problem-how is neural activity coordinated to create unified and correctly segmented percepts? Binding could be guided by stimulus-driven correlations between mechanisms, but the nature of these correlations is largely unexplored and no one has (intentionally) studied effects on binding if this joint information is compromised. Here, we develop a theoretical framework which: (1) describes crosstalk-generated correlations between cortical mechanisms for color, achromatic form, and motion, which arise from retinogeniculate encoding; (2) shows how these correlations can facilitate synchronization, segmentation, and binding; (3) provides a basis for understanding perceptual oddities and binding failures that occur for equiluminant and stabilized images. These ideas can be tested by measuring both perceptual events and neural activity while achromatic border contrast or stabilized image velocity is manipulated.

Filling-in phenomenon in patients with age-related macular degeneration: differences regarding uni- or bilaterality of central scotoma

PURPOSE

The purpose of this study was to explore the presence of the filling-in phenomenon in patients with uni- or bilateral central scotoma (CS) resulting from natural history or laser photocoagulation of choroidal neovascularization in age-related macular degeneration (AMD).

METHODS

Sixteen consecutive patients with unilateral CS and 14 patients with bilateral CS were assessed (44 eyes) with a scanning laser ophthalmoscope (SLO). Scotoma was delineated by scotometry with a point (1 degree x1 degree) moving radially from the periphery to the center of the lesion. In addition, patients underwent a line test, consisting of a horizontal line moving vertically and a vertical line moving horizontally, from the periphery to the center. The lines were longer than the macular lesion and were projected onto the retina. Patients were asked to indicate when the lines seemed interrupted. The perceptual filling-in phenomenon was considered to be present when limits of the perceived scotoma, determined by the line test, were smaller than those assessed by scotometry. In patients with bilateral CS, the results were analyzed to distinguish the less or more severely affected eye.

RESULTS

In all eyes, the limits of the scotoma obtained with the scotometry test corresponded to the anatomic edges of the macular lesion. In patients with bilateral CS, the filling-in phenomenon was observed in 12 out of 14 (85%) less severely affected eyes, but only in one (7%) of their more severely affected eyes. In patients with unilateral CS, the phenomenon was observed in only one out of 16 (6%) eyes.

CONCLUSION

These results suggest that the filling-in phenomenon mostly occurs in patients with bilateral central scotoma, and almost always in their less affected eye. Thus, it did usually not occur in an eye if the fellow eye was better.

Filling-in of retinal scotomas

In this study we examined the perception of one- and two-dimensional patterns across central retinal scotomas, caused by age-related macular degeneration. In contrast with previous studies of disrupted visual input that used the blind spot and artificial scotomas, the current study used large central scotomas caused by physical retinal damage. Such damage is associated with atrophy and long-term cortical reorganization, and it was therefore unclear whether perceptual completion in the damaged system will be similar to that reported for artificial scotomas and the blind spot. In addition, the scotomas under study were much larger and more central than artificial scotomas for which perceptual completion has been reported. For 1-D line and grating patterns, we found perceptual completion across large central scotomas (up to radius of 7 degrees ), which is significantly beyond the range of perceptual completion in artificial scotomas. Gratings completion was better than that of a single line, and increased with bars density. The use of central scotomas allowed us to test the completion of 2-D patterns that are difficult to study in peripheral vision. We found completion of two-dimensional dot arrays over large regions that improved with pattern density and regularity. The results show that in the physically damaged system the range of perceptual completion is increased compared with artificial scotomas, they strongly support the view of an active filling-in process rather than simply ignoring the damaged location, and they show that perceptual completion of physical scotomas is likely to involve cortical processing at multiple levels. We finally discuss implications of the results to the possible use of image enhancement techniques to facilitate the perception of low-vision individuals.

Is neural filling-in necessary to explain the perceptual completion of motion and depth information?

Retinal activity is the first stage of visual perception. Retinal sampling is non-uniform and not continuous, yet visual experience is not characterized by holes and discontinuities in the world. How does the brain achieve this perceptual completion? Fifty years ago, it was suggested that visual perception involves a two-stage process of (i) edge detection followed by (ii) neural filling-in of surface properties. We examine whether this general hypothesis can account for the specific example of perceptual completion of a small target surrounded by dynamic dots (an 'artificial scotoma'), a phenomenon argued to provide insight into the mechanisms responsible for perception. We degrade the target's borders using first blur and then depth continuity, and find that border degradation does not influence time to target disappearance. This indicates that important information for the continuity of target perception is conveyed at a coarse spatial scale. We suggest that target disappearance could result from adaptation that is not specific to borders, and question the need to hypothesize an active filling-in process to explain this phenomenon.

Cortical plasticity revealed by circumscribed retinal lesions or artificial scotomas

We review the work of others in which the effects of circumscribed, topographically corresponding binocular retinal lesions on the topographic organization of the visual cortex revealed that there is a substantial degree of topographical plasticity in the primary visual cortices of adult cats and macaque monkeys. Despite the evidence indicating that the reorganization of the topographic map in primary visual cortices of adult cats and macaques related to the input from one eye could be suppressed for a long time by inputs related to the other eye, we observed a substantial degree of topographical plasticity in the primary visual cortices of adult cats in which we have made circumscribed monocular retinal lesions. Overall, in both binocularly and monocularly lesioned adult animals, most cells recorded in the cortical projection zone of the retinal lesion (LPZ), several hours, several weeks or several months after placement of the lesions exhibited 'ectopic' excitatory visual receptive fields (RFs) which were displaced to the normal retina in the immediate vicinity of the lesion. The presence of ectopic RFs in cells recorded in the cortical LPZ, combined with the presence of normal cortical representation of the part of the retina in the vicinity of the lesion, indicate a clear expansion of the cortical representation of the part of the retina surrounding the lesion. When stimulated via the ectopic RFs, cortical cells exhibited normal orientation tuning and in the case of animals with monocular lesions, the orientation tuning of binocular cells when stimulated via ectopic RFs appeared to be very similar to that when the cells were stimulated via the RFs in the normal, unlesioned eye. In both binocularly and monocularly lesioned animals, the responses evoked by optimal visual stimuli from the ectopic RFs were substantially weaker than those evoked from their normal counterparts. Similarly, upper velocity limits were significantly lower when visual stimuli were presented via the ectopic RFs. In contrast to cats in which the retinal lesions were made in adulthood, in cats lesioned monocularly in adolescence (8-11 weeks postnatal), both the peak discharge rates and upper velocity limits of responses to photic stimuli presented via the ectopic RFs were very similar to those to stimuli presented via the normal eye. The intracortical mechanism(s) underlying the long-term cortical plasticity revealed by retinal lesions are likely to be closely linked to the mechanism(s) underlying the short-term reversible enlargement of cortical receptive fields observed with artificial scotomas. Furthermore, a similar putative intracortical mechanism(s) appears to underlie psychophysical phenomena observed in studies of retinal scotomas in humans. Overall, the research reviewed here strongly challenges the view that receptive fields of neurons in mammalian visual cortices are 'hard-wired'.

Texture fading correlates with stimulus salience

Perceptual fading of texture targets on similarly textured backgrounds was studied in relation to stimulus salience using texture patterns defined by orientation contrast, shape contrast, and order contrast. In two independent experiments, perceptual salience of the targets was determined. In the first, the textural contrast of the stimuli was varied and their salience quantified using magnitude estimation; in the second, reaction time was measured for the same stimulus patterns. In a third experiment, stimulus fading time was determined. Whereas magnitude estimates and fading time increased, reaction time decreased with increasing textural contrast strength, the shape of the curves depending on the kind of texture pattern used. When fading time was plotted against target salience, the slopes of the regression lines for shape and order contrast were similar, while the slope for orientation contrast was steeper, indicating longer fading times at equal stimulus salience. A control experiment using short oriented bars instead of gratings revealed that this difference may be attributed to the abutting contour between the target and its surround. With this contour removed, the fading time was largely the same for all three kinds of texture patterns. In the absence of a border (no cancellation), the unconnected target areas appeared to change gradually in orientation, shape, and spatial arrangement, thereby assuming the properties of the background (substitution).

Filling-in the details on perceptual fading

We examined the perceptual disappearance (or 'filling in') of a peripheral target surrounded by dynamic texture. Targets defined by different visual attributes were used to explore the importance of target properties in determining the time-course of fading. Introducing luminance-, motion- or direction-contrast between the target and background increased the time-to-fade. For motion contrast, this was related to target visibility. Targets defined by a difference of texture from the background took longer to fade than those defined by a difference of motion. This might correspond to activity in different visual areas, or could be due to different visibilities in each case.

Target/surround asymmetry in perceptual filling-in

Four experiments examined how differences in the properties of the target and surround affect the time required for perceptual filling-in. They examined differences in luminance, orientation, spatial frequency, and color. A larger target/surround difference delayed filling-in ('feature difference effect'). Interestingly, exchanging the target and surround properties significantly varied the time ('target/surround asymmetry'). Filling-in was facilitated when the target was brighter and closer to the vertical or horizontal than the surround. Little asymmetry was found in the frequency domain, while significant asymmetry was observed for specific color combinations. These effects are discussed with respect to edge adaptation, feature adaptation, balance of neural activities, and contextual modulation.

Dynamic random noise shrinks the twinkling aftereffect induced by artificial scotomas

Physiological alterations in cortical neurons are induced during adaptation to an artificial scotoma, a small homogeneous patch within a dynamic random noise or patterned background. When the dynamic noise is replaced by an equiluminant gray background, a twinkling aftereffect can be seen in the location of the artificial scotoma. Following binocular adaptation, we discovered that the perceived size of the twinkling aftereffect was dramatically smaller than the inducing artificial scotoma. Dichoptic adaptation induced shrinkage in the twinkling aftereffect that was similar to that found after binocular adaptation, suggesting that the twinkling aftereffect and its shrinkage both have cortical origins. We speculate that this perceptual shrinkage may reflect the interaction between two cortical mechanisms: a twinkling aftereffect mechanism that spreads throughout the artificial scotoma, and a filling-in mechanism that has a greater influence at the edges of the artificial scotoma and spreads inwards.