The abutting grating illusion

The object-based shift direction anisotropy is modulated by the horizontal visual field meridian

Reallocating object-based attention across the visual field meridians is significantly faster horizontally than vertically (termed the shift direction anisotropy; SDA), implicating the meridians in reorienting object-based attention. Here, we tested the modulatory role of the meridians in the emergence of the SDA by manipulating meridian local feature contrast. Considering the notion of separate pools of attentional resources in each cortical hemisphere, we hypothesised that manipulating the horizontal meridian would selectively modulate the SDA. In four experiments, participants were presented with an "L"-shaped object and detected a target that appeared at either a cued location or at one of two equidistant non-cued locations at the far end of the horizontal or vertical object arm. Meridian local feature contrast was manipulated with perceptually strong enhancements (visible lines and colour contrast borders) and perceptually weak enhancements (illusory borders from line texture patterns and inducers). Weak enhancements of the meridians did not significantly modulate SDA magnitude; however, during perceptually strong enhancements of the horizontal meridian, the SDA was significantly reduced compared with both vertical meridian enhancement and no-enhancement conditions. Moreover, horizontal and vertical shift RTs were statistically equivalent when the horizontal meridian was enhanced with a visible line, our strongest manipulation, indicating the SDA was eliminated. These results suggest that the SDA emerges due to reallocating object-based attention across the horizontal meridian. We interpret this finding as evidence in support of the theory by which anatomical segregations of the visual system determine how pools of attentional resources resolve competition between and within cortical hemispheres.

Reconstructing visual illusory experiences from human brain activity

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

Visual and haptic cues in processing occlusion

Introduction

Although shape is effective in processing occlusion, ambiguities in segmentation can also be addressed using depth discontinuity given visually and haptically. This study elucidates the contribution of visual and haptic cues to depth discontinuity in processing occlusion.

Methods

A virtual reality experiment was conducted with 15 students as participants. Word stimuli were presented on a head-mounted display for recognition. The central part of the words was masked with a virtual ribbon placed at different depths so that the ribbon appeared as an occlusion. The visual depth cue was either present with binocular stereopsis or absent with monocular presentation. The haptic cue was either missing, provided consecutively, or concurrently, by actively tracing a real off-screen bar edge that was positionally aligned with the ribbon in the virtual space. Recognition performance was compared between depth cue conditions.

Results

We found that word recognition was better with the stereoscopic cue but not with the haptic cue, although both cues contributed to greater confidence in depth estimation. The performance was better when the ribbon was at the farther depth plane to appear as a hollow, rather than when it was at the nearer depth plane to cover the word.

Discussion

The results indicate that occlusion is processed in the human brain by visual input only despite the apparent effectiveness of haptic space perception, reflecting a complex set of natural constraints.

Challenging deep learning models with image distortion based on the abutting grating illusion

Even state-of-the-art deep learning models lack fundamental abilities compared with humans. While many image distortions have been proposed to compare deep learning with humans, they depend on mathematical transformations instead of human cognitive functions. Here, we propose an image distortion based on the abutting grating illusion, which is a phenomenon discovered in humans and animals. The distortion generates illusory contour perception using line gratings abutting each other. We applied the method to MNIST, high-resolution MNIST, and "16-class-ImageNet" silhouettes. Many models, including models trained from scratch and 109 models pretrained with ImageNet or various data augmentation techniques, were tested. Our results show that abutting grating distortion is challenging even for state-of-the-art deep learning models. We discovered that DeepAugment models outperformed other pretrained models. Visualization of early layers indicates that better-performing models exhibit the endstopping property, which is consistent with neuroscience discoveries. Twenty-four human subjects classified distorted samples to validate the distortion.

Testing the effect of display organization in the collinear search impairment

Previous studies established that a salient collinear structure impairs local visual search. A display organization hypothesis states that the vertical grouping of elemental bars in the search display may selectively increase the salience of the local target in the background than that in the collinear distractor, leading to the collinear search impairment. Three displays were designed to test this hypothesis. A classical search display was adopted as a baseline. A diagonal search display was created with tilted bars, making perceptual organization diagonal and should reduce collinear search impairment. An illusory search display was designed by using abutting line illusion to emphasize the vertical grouping direction, which should increase collinear search impairment. A manipulation check was conducted with an online survey to understand the perceptual organization of the three displays. Results showed that the probability to perceive the stimuli grouping in the vertical direction was strongest in the illusory display and the least in the diagonal display. Nevertheless, the collinear search impairment did not vary with these manipulations, argue against the display organization hypothesis. We speculate that the search impairment might associate with the perceptual organization of the collinear distractor per se, rather than the perceptual organization of the background.

The Serpentine Illusion: A Visual Motion Illusion Induced by Phase-Shifted Line Gratings

In a pattern of horizontal lines containing ± 45° zigzagging phase-shifted strips, vivid illusory motion is perceived when the pattern is translated up or down at a moderate speed. Two forms of illusory motion are seen: [i] a motion "racing" along the diagonal interface between the strips and [ii] lateral (sideways) motion of the strip sections. We found the relative salience of these two illusory motions to be strongly influenced by the vertical spacing and length of the line gratings, and the period length of the zigzag strips. Both illusory motions are abolished when the abutting strips are interleaved, separated by a gap or when a real line is superimposed at the interface. Illusory motion is also severely weakened when equiluminant colored grating lines are used. Illusory motion perception is fully restored at < 20% luminance contrast. Using adaptation, we find that line-ends alone are insufficient for illusory motion perception, and that both physical carrier motion and line orientation are required. We finally test a classical spatiotemporal energy model of V1 cells that exhibit direction tuning changes that are consistent with the direction of illusory motion. Taking this data together, we constructed a new visual illusion and surmise its origin to interactions of spatial and temporal energy of the lines and line-ends preferentially driving the magnocellular pathway.

Vanishing Girls, Mysterious Blacks

Participants had to indicate the location of points on what might be called "amodal contours" in some works of art. The works represented mutually quite different cases. In one case, there were not even scattered modal cues, thus the amodal contour had to be hallucinated on the basis of generic familiarity. Here, observers indicated coherent geometrical structures (to a good approximation a smooth curve), although at idiosyncratic locations. In another case, we presented an ambiguous image that led to much more "fuzzy" amodal completions. We also presented an image that had at least some similarity to a configuration treated by Kanizsa. Here, observers were coherent and they mutually agreed, so the scarce modal cues apparently largely dictated the awareness.

Contour interpolation: A case study in Modularity of Mind

In his monograph Modularity of Mind (1983), philosopher Jerry Fodor argued that mental architecture can be partly decomposed into computational organs termed modules, which were characterized as having nine co-occurring features such as automaticity, domain specificity, and informational encapsulation. Do modules exist? Debates thus far have been framed very generally with few, if any, detailed case studies. The topic is important because it has direct implications on current debates in cognitive science and because it potentially provides a viable framework from which to further understand and make hypotheses about the mind's structure and function. Here, the case is made for the modularity of contour interpolation, which is a perceptual process that represents non-visible edges on the basis of how surrounding visible edges are spatiotemporally configured. There is substantial evidence that interpolation is domain specific, mandatory, fast, and developmentally well-sequenced; that it produces representationally impoverished outputs; that it relies upon a relatively fixed neural architecture that can be selectively impaired; that it is encapsulated from belief and expectation; and that its inner workings cannot be fathomed through conscious introspection. Upon differentiating contour interpolation from a higher-order contour representational ability ("contour abstraction") and upon accommodating seemingly inconsistent experimental results, it is argued that interpolation is modular to the extent that the initiating conditions for interpolation are strong. As interpolated contours become more salient, the modularity features emerge. The empirical data, taken as a whole, show that at least certain parts of the mind are modularly organized.

Judging Distance across Texture Discontinuities

Sinai et al (1998 Nature 395 497-500) showed that less distance is perceived along a ground surface that spans two differently textured regions than along a surface that is uniformly textured. We examined the effect of texture continuity on judged distance using computer-generated displays of simulated surfaces in five experiments. Discontinuities were produced by using different textures, the same texture reversed in contrast, or the same texture shifted horizontally. The simulated surface was either a ground plane or a frontoparallel plane. For all textures and both orientations, less distance was judged in the discontinuous conditions than in continuous conditions. We propose that when a surface contains a texture discontinuity, a small area adjacent to the perceived boundary is excluded from judged distances.

Orientation-Selective Adaptation to Crowded Illusory Lines

Visual adaptation has been successfully used as a psychophysical tool for studying the functional organisation of visual awareness. It has been shown that orientation-selective adaptation to a grating pattern occurs in crowded conditions. In such conditions, simultaneous presentation of flanking distractors pushes the target stimulus out of conscious perception and severely impairs orientation discrimination in the periphery of the visual field. In the present study, orientation-selective adaptation to illusory lines induced by two line gratings abutting each other with a phase shift was examined in crowded and non-crowded conditions. To rule out the effects of lower level adaptations we used an animation paradigm in which the orientations of the two line gratings were altered repeatedly during adaptation phase without any change in the orientation of the resulting illusory line. Although performance of subjects in reporting the orientation of crowded illusory lines was at chance level, orientation-selective adaptation was preserved for crowded as well as non-crowded adapting targets. Two control experiments demonstrated that adaptation to endpoints of real lines at the location of abutting grating lines had minimal effect on the adaptation to illusory lines; and changes in the configuration of endpoints could not be responsible for better performance when adapting and test stimuli were different. We conclude that a crowding effect occurs after illusory lines have been processed in the visual stream. Since illusory lines seem to be represented at relatively early stages of visual processing (e.g. area V2), adaptation to crowded illusory stimuli suggests that neuronal activation in those early stages is not necessarily correlated with conscious perception.

Second-Order Footsteps Illusions

In the “footsteps illusion”, light and dark squares travel at constant speed across black and white stripes. The squares appear to move faster and slower as their contrast against the stripes varies. We now demonstrate some second-order footsteps illusions, in which all edges are defined by colors or textures—even though luminance-based neural motion detectors are blind to such edges.

A New Variant of the Ouchi Illusion Reveals Fourier-Component-Based Processing

We report that anomalous motion illusion in a new variant of the Ouchi figure is well predicted by the strength of its Fourier fundamentals and harmonics. The original Ouchi figure consists of a rectangular checkerboard pattern surrounded by an orthogonal rectangular checkerboard pattern, in which illusory relative motion between the two regions is perceived. Although this illusion has been explained in terms of biases in integrating one-dimensional motion signals to determine the two-dimensional motion direction, the physiological mechanism has not been clarified. With our new stimuli, which consisted of thin lines instead of rectangles, we found that the perceived illusion is drastically reduced when the position of each line element is randomly shifted. This is not predicted by simple models of local motion integration along the visible edges. We demonstrate that the relative amplitude of the relevant Fourier fundamentals and harmonics leads to a quantitative prediction. Our analysis was successfully applied to other variants of the Ouchi figure (Khang and Essock 1997 Perception26 585–597), closely predicting the reported rating. The results indicate that the underlying physiological mechanism is sensitive to the Fourier components of the stimuli rather than the visible edges.

Visual Illusions: An Interesting Tool to Investigate Developmental Dyslexia and Autism Spectrum Disorder

A visual illusion refers to a percept that is different in some aspect from the physical stimulus. Illusions are a powerful non-invasive tool for understanding the neurobiology of vision, telling us, indirectly, how the brain processes visual stimuli. There are some neurodevelopmental disorders characterized by visual deficits. Surprisingly, just a few studies investigated illusory perception in clinical populations. Our aim is to review the literature supporting a possible role for visual illusions in helping us understand the visual deficits in developmental dyslexia and autism spectrum disorder. Future studies could develop new tools - based on visual illusions - to identify an early risk for neurodevelopmental disorders.

Comparing subjective contours for Kanizsa squares and linear edge alignments ('New York Titanic' figures)

One current view is that subjective contours may involve high-level detection of a salient shape with back propagation to early visual areas where small receptive fields allow for scrutiny of relevant details. This idea applies to Kanizsa-type figures. However, Gillam and Chan (2002 Psychological Science, 13, 279-282) using figures based on Gillam's graphic 'New York Titanic' (Gillam, 1997 Thresholds: Limits of perception. New York: Arts Magazine) showed that strong subjective contours can be seen along the linearly aligned edges of a set of shapes if occlusion cues of 'extrinsic edge' and 'entropy contrast' are strong. Here we compared ratings of the strength of subjective contours along linear alignments with those seen in Kanizsa figures. The strongest subjective contour for a single set of linearly aligned shapes was similar in strength to the edges of a Kanizsa square (controlling for support ratio) despite the lack of a salient region. The addition of a second set of linearly aligned inducers consistent with a common surface increased subjective-contour strength, as did having four rather than two 'pacmen' in the Kanizsa figure, indicating a role for surface support. We argue that linear subjective contours allow for the investigation of certain occlusion cues and the interactions between them that are not easily explored with Kanizsa figures.

Orientation domain diversity in macaque area V2

Single orientation domains in primary (V1) and second (V2) visual cortical areas are known to encode the orientation of visual contours. However, the visual world contains multiple and complex contour types. How do these domains handle such complexity? Using optical imaging methods, we have examined orientation response to two types of contours: real (luminance-defined) and illusory (inferred). We find that, unlike area V1, there are multiple types of orientation domain in V2. These include “real only” domains, “higher-order” domains (which respond to an orientation whether real or illusory), and other domains with nonmatching real/illusory orientation preference. We suggest that this plurality of orientation domains in V2 enables the complexities of local and global contour extraction.

Equivalent representation of real and illusory contours in macaque V4

The cortical processing of illusory contours provides a unique window for exploring the brain mechanisms underlying visual perception. Previous electrophysiological single-cell recordings demonstrate that a subgroup of cells in macaque V1 and V2 signal the presence of illusory contours, whereas recent human brain imaging studies reveal higher-order visual cortices playing a central role in illusory figure processing. It seems that the processing of illusory contours/figures may engage multiple cortical interactions between hierarchically organized processing stages in the ventral visual pathway of primates. However, it is not yet known in which brain areas illusory contours are represented in the same manner as real contours at both the population and single-cell levels. Here, by combining intrinsic optical imaging in anesthetized rhesus macaques with single-cell recordings in awake ones, we found a complete overlap of orientation domains in visual cortical area V4 for processing real and illusory contours. In contrast, the orientation domains mapped in early visual areas V1 and V2 mainly encoded the local physical stimulus features inducing the subjective perception of global illusory contours. Our results indicate that real and illusory contours are encoded equivalently by the same functional domains in V4, suggesting that V4 is a key cortical locus for integration of local features into global contours.

Properties of long-range illusory contours produced by offset-arcs

Researchers have used several different types of illusory contours to investigate properties of human perception. One rarely used illusory contour is a combination of the abutting grating and Kanizsa illusions. We call this the offset-arcs illusion and provide an empirical investigation of the illusion. Through a series of four experiments, using different methods of measurement, we show that changes to the phase of the abutting-grating part of the inducing stimulus can dramatically change the perceived strength and clarity of the long-range illusory contour. The easy manipulation of illusion strength should make the offset-arcs illusion applicable to a wide range of studies that utilize long-range illusory contours. The lack of a brightness component to the illusion should allow the offset-arcs illusion to help separate perceptual grouping from surface brightness effects that are often confounded in other illusory contours.

Global visual processing in macaques studied using Kanizsa illusory shapes

The ability to extract form information from a visual scene, for object recognition or figure-ground segregation, is a fundamental visual system function. Many studies of nonhuman primates have addressed the neural mechanisms involved in global form processing, but few have sought to demonstrate this ability behaviorally. In this study, we probed global visual processing in macaque monkeys (Macaca nemestrina) using classical Kanizsa illusory shapes as an assay of global form perception. We trained three monkeys on a "similarity match-to-sample" form discrimination task, first with complete forms embedded in fields of noncontour-inducing "pacman" elements. We then tested them with classic Kanizsa illusory shapes embedded in fields of randomly oriented elements. Two of the three subjects reached our criterion performance level of 80% correct or better on four of five illusory test conditions, demonstrating clear evidence of Kanizsa illusory form perception; the third subject mastered three of five conditions. Performance limits for illusory form discrimination were obtained by manipulating support ratio and by measuring threshold for discriminating "fat" and "thin" illusory squares. Our results indicate that macaque monkeys are capable of global form processing similarly to humans and that the perceptual mechanisms for "filling-in" contour gaps exist in macaques as they do in humans.

Curvature coding in illusory contours

We have employed the shape frequency and shape-amplitude after-effects (SFAE and SAAE) to investigate: (i) whether the shapes of illusory and real curves are processed by the same or different mechanisms, and (ii) the carrier-tuning properties of illusory curvature mechanisms. The SFAE and SAAE are the phenomena in which adaptation to a sinusoidal-shaped contour results in a shift in, respectively, the perceived shape-frequency and perceived shape-amplitude of a test contour in a direction away from that of the adapting stimulus. Both after-effects are believed to be mediated by mechanisms sensitive to curvature (Gheorghiu & Kingdom, 2007a, 2009; see also Hancock & Peirce, 2008). We observed both shape after-effects in sinusoidally-shaped illusory contours defined by phase-shifted line-grating carriers. We tested whether illusory and real contours were mediated by the same or different mechanisms by comparing same adaptor-and-test with different adaptor-and-test combinations of real and illusory contours. Real contour adaptors produced after-effects in illusory contour tests that were as great as, or even greater than those produced by illusory contour adaptors. However, illusory contour adaptors produced much weaker after-effects in real contour tests than did real contour adaptors. This asymmetry suggests that illusory contour curves are encoded by a sub-set of mechanisms sensitive to real contour curves. We also examined the carrier-tuning properties of illusory-contour curvature processing using adaptor and test illusory contours that differed in the luminance contrast-polarity, luminance scale and orientation of the carriers. We found no selectivity to any of these dimensions for either even-symmetric or odd-symmetric line-gratings carriers, even though selectivity to these dimensions was found for real contours.

Phenomenology and neurophysiological correlations: two approaches to perception research

This article argues that phenomenological description and neurophysiological correlation complement each other in perception research. Whilst phenomena constitute the material, neuronal mechanisms are indispensable for their explanation. Numerous examples of neurophysiological correlates show that the correlation of phenomenology and neurophysiology is fruitful. Phenomena for which neuronal mechanism have been found include: (in area V1) filling-in of real and artificial scotomata, contour integration, figure-ground segregation by orientation contrast, amodal completion, and motion transparency; (in V2) modal completion, border ownership, surface transparency, and cyclopean perception; (in V3) alignment in dotted contours, and filling-in with dynamic texture; (in V4) colour constancy; (in MT) shape by accretion/deletion, grouping by coherent motion, apparent motion in motion quartets, motion in apertures, and biological motion. Results suggest that in monkey visual cortex, occlusion cues, including stereo depth, are predominantly processed in lower areas, whereas mechanisms for grouping and motion are primarily represented in higher areas. More correlations are likely to emerge as neuroscientists strive for a better understanding of visual perception. The paper concludes with a review of major achievements in visual neuroscience pertinent to the study of the phenomena under consideration.

Three-dimensional shape from second-order orientation flows

In images of textured surfaces, orientation flows formed by perspective convergence invariably convey 3D shape. We show that orientation flows formed by contrast-modulated (CM) and illusory contours (IC) convey 3D shape, and that both stimulus types induce 3D shape aftereffects on CM and IC test stimuli. Adaptation to luminance-modulated (LM) orientation flows induce robust 3D shape aftereffects on CM and IC tests, however, aftereffects using CM/IC adapting stimuli on LM tests were substantially weaker. These results can be explained by the adaptation of 3D shape-selective neurons that invariantly extract first- and second-order orientation flows from striate and extra-striate signals, which receive stronger input from neurons selective for first-order orientation flows.

Orientation-selective adaptation to illusory contours in human visual cortex

Humans can perceive illusory or subjective contours in the absence of any real physical boundaries. We used an adaptation protocol to look for orientation-selective neural responses to illusory contours defined by phase-shifted abutting line gratings in the human visual cortex. We measured functional magnetic resonance imaging (fMRI) responses to illusory-contour test stimuli after adapting to an illusory-contour adapter stimulus that was oriented parallel or orthogonal to the test stimulus. We found orientation-selective adaptation to illusory contours in early (V1 and V2) and higher-tier visual areas (V3, hV4, VO1, V3A/B, V7, LO1, and LO2). That is, fMRI responses were smaller for test stimuli parallel to the adapter than for test stimuli orthogonal to the adapter. In two control experiments using spatially jittered and phase-randomized stimuli, we demonstrated that this adaptation was not just in response to differences in the distribution of spectral power in the stimuli. Orientation-selective adaptation to illusory contours increased from early to higher-tier visual areas. Thus, both early and higher-tier visual areas contain neurons selective for the orientation of this type of illusory contour.

Neural mechanisms mediating responses to abutting gratings: luminance edges vs. illusory contours

The discontinuities of phase-shifted abutting line gratings give rise to perception of an "illusory contour" (IC) along the line terminations. Neuronal responses to such ICs have been interpreted as evidence for a specialized visual mechanism, since such responses cannot be predicted from conventional linear receptive fields. However, when the spatial scale of the component gratings (carriers) is large compared to the neuron's luminance passband, these IC responses might be evoked simply by the luminance edges at the line terminations. Thus by presenting abutting gratings at a series of carrier spatial scales to cat A18 neurons, we were able to distinguish genuine nonlinear responses from those due to luminance edges. Around half of the neurons (both simple and complex types) showed a bimodal response pattern to abutting gratings: one peak at a low carrier spatial frequency range that overlapped with the luminance passband, and a second distinct peak at much higher frequencies beyond the neuron's grating resolution. For those bimodally responding neurons, the low-frequency responses were sensitive to carrier phase, but the high-frequency responses were phase-invariant. Thus the responses at low carrier spatial frequencies could be understood via a linear model, while the higher frequency responses represented genuine nonlinear IC processing. IC responsive neurons also demonstrated somewhat lower spatial preference to the periodic contours (envelopes) compared to gratings, but the optimal orientation and motion direction for both were quite similar. The nonlinear responses to ICs could be explained by the same energy mechanism underlying responses to second-order stimuli such as contrast-modulated gratings. Similar neuronal preferences for ICs and for gratings may contribute to the form-cue invariant perception of moving contours.

From perceptive fields to Gestalt

Studies on visual psychophysics and perception conducted in the Freiburg psychophysics laboratory during the last 35 years are reviewed. Many of these were inspired by single-cell neurophysiology in cat and monkey. The aim was to correlate perceptual phenomena and their effects to possible neuronal mechanisms from retina to visual cortex and beyond. Topics discussed include perceptive field organization, figure-ground segregation and grouping, fading and filling-in, and long-range color interaction. While some of these studies succeeded in linking perception to neuronal response patterns, others require further investigation. The task of probing the human brain with perceptual phenomena continues to be a challenge for the future.

Investigating the neural basis of the auditory continuity illusion

In this study, we investigated one type of auditory perceptual grouping phenomena--the auditory continuity illusion (also called temporal induction). We employed a previously developed, neurobiologically realistic, large-scale neural network model of the auditory processing pathway in the cortex, ranging from the primary auditory cortex to the prefrontal cortex, and simulated temporal induction without changing any model parameters. The model processes tonal contour stimuli, composed of combinations of upward and downward FM sweeps and tones, in a delayed match-to-sample task. The local electrical activities of the neuronal units of the model simulated accurately the experimentally observed electrophysiological data, where available, and the model's simulated BOLD-fMRI data were quantitatively matched with experimental fMRI data. In the present simulations, intact stimuli were matched with fragmented versions (i.e., with inserted silent gaps). The ability of the model to match fragmented stimuli declined as the duration of the gaps increased. However, when simulated broadband noise was inserted into these gaps, the matching response was restored, indicating that a continuous stimulus was perceived. The electrical activities of the neuronal units of the model agreed with electrophysiological data, and the behavioral activity of the model matched human behavioral data. In the model, the predominant mechanism implementing temporal induction is the divergence of the feedforward connections along the auditory processing pathway in the temporal cortex. These simulation results not only attest to the robustness of the model, but further predict the primary role of the anatomical connectivity of the auditory processing areas in mediating the continuity illusion.

Illusory contours and surfaces without amodal completion and depth stratification

Cognitive and figural cues were studied in modified Ehrenstein figures made from letters of the alphabet instead of radial lines. Capital letters with and without terminators (L, J vs O, D) were used, oriented towards or away from the central gap. Three groups, of 14 subjects each, estimated the magnitude of either (i) the illusory contour, (ii) brightness enhancement, or (iii) apparent depth. Strong illusory contour formation and brightness enhancement, but no depth stratification, were perceived in figures devoid of apparent occlusion and amodal completion. These results demonstrate that the Ehrenstein illusion can arise from line ends--with no need for perceptual completion, showing that illusory boundaries and surfaces can be dissociated from apparent depth. Results support a bottom-up explanation in terms of end-stopped neurons in the visual cortex. Conversely, top-down processes appear to be responsible for depth stratification.

Stereoscopic illusory contours--cortical neuron responses and human perception

In human perception, figure-ground segregation suggests that stereoscopic cues are grouped over wide areas of the visual field. For example, two abutting rectangles of equal luminance and size are seen as a uniform surface when presented at the same depth, but appear as two surfaces separated by an illusory contour and a step in depth when presented with different retinal disparities. Here, we describe neurons in the monkey visual cortex that signal such illusory contours and can be selective for certain figure-ground directions that human observers perceive at these contours. The results suggest that these neurons group stereoscopic cues over distances up to 8 degrees. In addition, we compare these results with human perception and show that the mean stimulus parameters required by these neurons also induce optimal percepts of illusory contours in human observers.

Neural dynamics of motion integration and segmentation within and across apertures

A neural model is developed of how motion integration and segmentation processes, both within and across apertures, compute global motion percepts. Figure-ground properties, such as occlusion, influence which motion signals determine the percept. For visible apertures, a line's terminators do not specify true line motion. For invisible apertures, a line's intrinsic terminators create veridical feature-tracking signals. Sparse feature-tracking signals can be amplified before they propagate across position and are integrated with ambiguous motion signals within line interiors. This integration process determines the global percept. It is the result of several processing stages: directional transient cells respond to image transients and input to a directional short-range filter that selectively boosts feature-tracking signals with the help of competitive signals. Then, a long-range filter inputs to directional cells that pool signals over multiple orientations, opposite contrast polarities, and depths. This all happens no later than cortical area MT. The directional cells activate a directional grouping network, proposed to occur within cortical area MST, within which directions compete to determine a local winner. Enhanced feature-tracking signals typically win over ambiguous motion signals. Model MST cells that encode the winning direction feed back to model MT cells, where they boost directionally consistent cell activities and suppress inconsistent activities over the spatial region to which they project. This feedback accomplishes directional and depthful motion capture within that region. Model simulations include the barberpole illusion, motion capture, the spotted barberpole, the triple barberpole, the occluded translating square illusion, motion transparency and the chopsticks illusion. Qualitative explanations of illusory contours from translating terminators and plaid adaptation are also given.

Real and illusory contour processing in area V1 of the primate: a cortical balancing act

It is known that neurons in area V2 (the second visual area) can signal the orientation of illusory contours in the primate. Whether area V1 (primary visual cortex) can signal illusory contour orientation is more controversial. While some electrophysiology studies have ruled out illusory signaling in V1, other reports suggest that V1 shows some illusory-specific response. Here, using optical imaging and single unit electrophysiology, we report that primate V1 does show an orientation-specific response to the 'abutting line grating' illusory contour. However, this response does not signal an illusory contour in the conventional sense. Rather, we find that illusory contour stimulation leads to an activation map that, after appropriate subtraction of real line signal, is inversely related to the real orientation map. The illusory contour orientation is thus negatively signaled or de-emphasized in V1. This 'activation reversal' is robust, is not due merely to presence of line ends, is not dependent on inducer orientation, and is not due to precise position of line end stimulation of V1 cells. These data suggest a resolution for previous apparently contradictory experimental findings. We propose that the de-emphasis of illusory contour orientation in V1 may be an important signal of contour identity and may, together with illusory signal from V2, provide a unique signature for illusory contour representation.

Role of motion integration in contour perception

Contours are believed to play a key role in the visual analysis of scenes by the primate brain. In dynamic scenes, the presence of contours is often signaled by discontinuities in motion fields. However, it is unclear whether the motion fields over which the visual system extracts discontinuities, correspond to the local optic-flow or the pattern motion fields obtained by integrating local estimates. A resolution of this issue would provide important clues about the organization of visual motion and form analysis processes. In this paper, we present experimental evidence which suggests that the perception of motion defined contours is strongly dependent on motion integration - an operation that is believed to take place relatively late in the visual stream.

A neural model of how horizontal and interlaminar connections of visual cortex develop into adult circuits that carry out perceptual grouping and learning

A neural model suggests how horizontal and interlaminar connections in visual cortical areas V1 and V2 develop within a laminar cortical architecture and give rise to adult visual percepts. The model suggests how mechanisms that control cortical development in the infant lead to properties of adult cortical anatomy, neurophysiology and visual perception. The model clarifies how excitatory and inhibitory connections can develop stably by maintaining a balance between excitation and inhibition. The growth of long-range excitatory horizontal connections between layer 2/3 pyramidal cells is balanced against that of short-range disynaptic interneuronal connections. The growth of excitatory on-center connections from layer 6-to-4 is balanced against that of inhibitory interneuronal off-surround connections. These balanced connections interact via intracortical and intercortical feedback to realize properties of perceptual grouping, attention and perceptual learning in the adult, and help to explain the observed variability in the number and temporal distribution of spikes emitted by cortical neurons. The model replicates cortical point spread functions and psychophysical data on the strength of real and illusory contours. The on-center, off-surround layer 6-to-4 circuit enables top-down attentional signals from area V2 to modulate, or attentionally prime, layer 4 cells in area V1 without fully activating them. This modulatory circuit also enables adult perceptual learning within cortical area V1 and V2 to proceed in a stable way.

Visual cortical mechanisms of perceptual grouping: interacting layers, networks, columns, and maps

The visual cortex has a laminar organization whose circuits form functional columns in cortical maps. How this laminar architecture supports visual percepts is not well understood. A neural model proposes how the laminar circuits of V1 and V2 generate perceptual groupings that maintain sensitivity to the contrasts and spatial organization of scenic cues. The model can decisively choose which groupings cohere and survive, even while balanced excitatory and inhibitory interactions preserve contrast-sensitive measures of local boundary likelihood or strength. In the model, excitatory inputs from lateral geniculate nucleus (LGN) activate layers 4 and 6 of V1. Layer 6 activates an on-center off-surround network of inputs to layer 4. Together these layer 4 inputs preserve analog sensitivity to LGN input contrasts. Layer 4 cells excite pyramidal cells in layer 2/3, which activate monosynaptic long-range horizontal excitatory connections between layer 2/3 pyramidal cells, and short-range disynaptic inhibitory connections mediated by smooth stellate cells. These interactions support inward perceptual grouping between two or more boundary inducers, but not outward grouping from a single inducer. These boundary signals feed back to layer 4 via the layer 6-to-4 on-center off-surround network. This folded feedback joins cells in different layers into functional columns while selecting winning groupings. Layer 6 in V1 also sends top-down signals to LGN using an on-center off-surround network, which suppresses LGN cells that do not receive feedback, while selecting, enhancing, and synchronizing activity of those that do. The model is used to simulate psychophysical and neurophysiological data about perceptual grouping, including various Gestalt grouping laws.

From elements to perception: local and global processing in visual neurons

Gestalt psychologists in the early part of the century challenged psychophysical notions that perceptual phenomena can be understood from a punctate (atomistic) analysis of the elements present in the stimulus. Their ideas slowed later attempts to explain vision in terms of single-cell recordings from individual neurons. A rapprochement between Gestalt phenomenology and neurophysiology seemed unlikely when the first ECVP was held in Marburg, Germany, in 1978. Since that time, response properties of neurons have been discovered that invite an interpretation of visual phenomena (including illusions) in terms of neuronal processing by long-range interactions, as first proposed by Mach and Hering in the last century. This article traces a personal journey into the early days of neurophysiological vision research to illustrate the progress that has taken place from the first attempts to correlate single-cell responses with visual perceptions. Whereas initially the receptive-field properties of individual classes of cells--e.g., contrast, wavelength, orientation, motion, disparity, and spatial-frequency detectors--were used to account for relatively simple visual phenomena, nowadays complex perceptions are interpreted in terms of long-range interactions, involving many neurons. This change in paradigm from local to global processing was made possible by recent findings, in the cortex, on horizontal interactions and backward propagation (feedback loops) in addition to classical feedforward processing. These mechanisms are exemplified by studies of the tilt effect and tilt aftereffect, direction-specific motion adaptation, illusory contours, filling-in and fading, figure--ground segregation by orientation and motion contrast, and pop-out in dynamic visual-noise patterns. Major questions for future research and a discussion of their epistemological implications conclude the article.

Neon colour spreading in three-dimensional illusory objects in humans

We studied whether neon spreading can be induced within three-dimensional illusory triangles. Kanizsa triangles were induced by black pacman disks consisting of red sectors with curved sides. Viewing our stimuli monocularly produced two-dimensional illusory contours and surfaces as well as neon spreading in each figure. Triangles appeared concave or convex under stereoscopical viewing. Neon colour spreading was induced within illusory figures bending in three-dimensional space, suggesting that neural contour completion and surface filling-in interact across depth. Surprisingly, neon spreading was induced above the intervening surface even when the inducers were below the surface. Neon colour and illusory configuration were preserved behind the intervening surface only when it appeared transparent.

Development of illusory-contour perception in infants

We investigated whether infants from 8-22 weeks of age were sensitive to the illusory contour created by aligned line terminators. Previous reports of illusory-contour detection in infants under 4 months old could be due to infants' preference for the presence of terminators rather than their configuration. We generated preferential-looking stimuli containing sinusoidal lines whose oscillating, abutting terminators give a strong illusory contour in adult perception. Our experiments demonstrated a preference in infants 8 weeks old and above for an oscillating illusory contour compared with a stimulus containing equal terminator density and movement. Control experiments excluded local line density, or attention to alignment in general, as the basis for this result. In the youngest age group (8-10 weeks) stimulus velocity appears to be critical in determining the visibility of illusory contours, which is consistent with other data on motion processing at this age. We conclude that, by 2 months of age, the infant's visual system contains the nonlinear mechanisms necessary to extract an illusory contour from aligned terminators.

Neuronal correlates of real and illusory contour perception: functional anatomy with PET

Illusory contours provide a striking example of the visual system's ability to extract a meaningful representation of the surroundings from fragmented visual stimuli. Psychophysical and neurophysiological data suggest that illusory contours are processed in early visual cortical areas, and neuroimaging studies in humans have shown that Kanizsa-type illusory contours activate early retinotopic visual areas that are also activated by real contours. It is not known whether other types of illusory contours are processed by the same mechanisms, nor is it clear to what extent attentional effects may have influenced these results, as no attempt was made to match the salience of real and illusory stimuli in previous imaging studies. It therefore remains an open question whether there are any brain regions specifically involved in the perception of illusory contours. To address these questions, we have used 15O-butanol positron emission tomography (PET) and a novel kind of illusory contour stimulus that is induced only by aligned line ends. By employing a form discrimination task that was matched for attention and stimulus salience across conditions we were able to directly contrast perception of real and illusory contours. We found that the regions activated by illusory contour perception were the same as those activated by real contours. Only one region, located in the right fusiform gyrus, was significantly more strongly activated by perception of illusory contours than by real contours. In addition, a principal component analysis suggested that illusory contour perception is associated with a change in the correlation between V1 and V2. We conclude that different kinds of illusory contours are processed by the same cortical regions and that these regions overlap extensively with those involved in processing of real contours. At the regional level, perception of illusory contours thus appears to differ from perception of real contours by the degree of involvement of higher visual areas as well as by the nature of interaction between early visual areas.

Detection and discrimination of subjective contours defined by offset gratings

Three experiments were conducted to refine our understanding of the mechanisms that encode subjective contours. In Experiment 1, discrimination thresholds (stimulus onset asynchronies [SOAs] yielding 81% correct) were measured in a backward masking paradigm for subjective contours defined by offset gratings. For large apertures, thresholds increased as carrier frequency increased. For the smallest aperture, thresholds were a U-shaped function of carrier frequency. Experiment 2 showed that these threshold results were generally consistent with the rated strength of the subjective contours. Experiment 3 showed that detection thresholds (SOAs yielding 81% correct) again increased with carrier spatial frequency, increased for obliquely oriented carriers, and, for a particular frequency and orientation of the carrier, were lower when the subjective contour was orthogonal to the carrier. All of these results are well explained by a two-stage process in which a second-layer filter integrates the responses of end-stopped mechanisms to the terminators defining the subjective contour. In the model, the end-stopped mechanisms have low-pass sensitivity to carrier spatial frequency, and the sizes of the second-layer filters are proportional to the scale of the end-stopped mechanisms from which they draw their input.

Illusory motion capture and release in rotating 3-D figure in humans

An illusory bar emerges in a cleft between two opposing gratings. When the gratings rotated around the vertical axis in three-dimensional (3-D) space, the illusory bar was seen either (i) rotating with the inducing gratings or (ii) as a stationary and opaque tape located in front of gratings. This illusion seems to be caused by the different temporal dynamics of the illusion and its inducers, especially by the slower extinction rate for the illusory bar than its inducers. The illusion is a psychophysical demonstration of an illusory figure becoming spatially and temporally loose from its inducers, suggesting that they are processed separately in the brain. This indicates that illusory figures are not only by-products of normal vision but have their own important function.

Interpolation processes in the perception of real and illusory contours

The spatial and temporal characteristics of mechanisms that bridge gaps between line segments were determined. The presentation time that was necessary for localisation and identification of a triangular shape made up of pacmen, pacmen with lines, lines, line segments (corners), or pacmen with circles (amodal completion) was measured. The triangle was embedded in a field of distractors made up of the same components but at random orientations. Subjects had to indicate whether the triangle was on the left or on the right of the display (localisation) and whether it was pointing upward or downward (identification). Poststimulus masks consisted of pinwheels for the pacmen stimuli or wheels defined by lines. Stimuli were presented on a grey background and defined by luminance or isoluminant contrast. Thresholds were fastest when the triangle was defined by real contours, as for the pacmen with lines (105 ms) and the lines only (92 ms), slightly slower for corners (118 ms) and pacmen (136 ms), and much slower for the amodally completed pacmen (285 ms). For all inducer types localisation was about 20 ms faster than identification. In a second experiment the relative length of the gap between inducers was varied. Thresholds increased as a function of gap length, indicating that the gaps between the inducers need to be interpolated. There was no significant difference in the speed of this interpolation process between the pacman stimuli and the line-segment stimuli. About 40 ms were required to interpolate 1 deg of visual angle, corresponding to about one third of the distance between inducers. In a third experiment, it was found that processing of isoluminant stimuli was as fast as for low-contrast luminance stimuli, when targets were defined by real contours (lines), but much slower for illusory contours (pacmen). The conclusion is that the time necessary to interpolate a contour depends greatly on the spatial configuration of the stimulus. Since interpolation is faster for the line-segment stimuli, which do not elicit the percept of an illusory contour, the interpolation process seems to be independent of the formation of illusory contours.

Figure-ground segregation at contours: a neural mechanism in the visual cortex of the alert monkey

An important task of vision is the segregation of figure and ground in situations of spatial occlusion. Psychophysical evidence suggests that the depth order at contours is defined early in visual processing. We have analysed this process in the visual cortex of the alert monkey. The animals were trained on a visual fixation task which reinforced foveal viewing. During periods of active visual fixation, we recorded the responses of single neurons in striate and prestriate cortex (areas V1, V2, and V3/V3A). The stimuli mimicked situations of spatial occlusion, usually a uniform light (or dark) rectangle overlaying a grating texture of opposite contrast. The direction of figure and ground at the borders of these rectangles was defined by the direction of the terminating grating lines (occlusion cues). Neuronal responses were analysed with respect to figure-ground direction and contrast polarity at such contours. Striate neurons often failed to respond to such stimuli, or were selective for contrast polarity; others were non-selective. Some neurons preferred a certain combination of figure-ground direction and contrast polarity. These neurons were rare both in striate and prestriate cortex. The majority of neurons signalled figure-ground direction independent of contrast polarity. These neurons were only found in prestriate cortex. We explain these responses in terms of a model which also explains neuronal signals of illusory contours. These results suggest that occlusion cues are used at an early level of processing to segregate figure and ground at contours.

Illusory contour-motion arising from translating terminators

Periodic grating patterns were created by phase shifting or eliminating vertical columns of a fine line carrier grating oriented 45 deg. Motion was created by translating the patterns parallel to the carrier grating. This veridical motion was seen when terminators (i) were created in low-frequency carriers; (ii) terminated short lines; and (iii) moved slowly. In the complementary conditions an illusory contour-motion was seen perpendicular to the orientation of the terminator-defined contours. A model involving a competition between second-layer filters (encoding the orientation and motions of the terminator defined contours) and double endstopped mechanisms (signalling the presence of terminators) was developed and found to be in quantitative agreement with these data. Experiments with plaids composed of two such patterns were generally consistent with the results of the one-dimensional cases. Coherent "subjective contour plaid" motion was almost always seen when the two subjective contours had the same orientation and were perfectly phase aligned.

Visual brain and visual perception: how does the cortex do perceptual grouping?

How the brain generates visual percepts is a central problem in neuroscience. We propose a detailed neural model of how lateral geniculate nuclei and the interblob cortical stream through V1 and V2 generate context-sensitive perceptual groupings from visual inputs. The model suggests a functional role for cortical layers, columns, maps and networks, and proposes homologous circuits for V1 and V2 with larger-scale processing in V2. An integrated treatment of interlaminar, horizontal, orientational and endstopping cortical interactions and a role for corticogeniculate feedback in grouping are proposed. Modeled circuits simulate parametric psychophysical data about boundary grouping and illusory contour formation.