From elements to perception: local and global processing in visual neurons

Gestalt neurons and emergent properties in visual perception: A novel concept for the transformation from local to global processing

Gestalten in visual perception are defined by emergent properties of the whole, which cannot be predicted from the sum of its parts; rather, they arise by virtue of inherent principles, the Laws of Seeing. This review attempts to assign neurophysiological correlates to select emergent properties in motion and contour perception and proposes parallels to the processing of local versus global attributes by classical versus contextual receptive fields. The aim is to identify Gestalt neurons in the visual system to account for the Laws of Seeing in causal terms and to explain "Why do things look as they do" (Koffka, 1935, p. 76).

PS-Net: human perception-guided segmentation network for EM cell membrane

MOTIVATION

Cell membrane segmentation in electron microscopy (EM) images is a crucial step in EM image processing. However, while popular approaches have achieved performance comparable to that of humans on low-resolution EM datasets, they have shown limited success when applied to high-resolution EM datasets. The human visual system, on the other hand, displays consistently excellent performance on both low and high resolutions. To better understand this limitation, we conducted eye movement and perceptual consistency experiments. Our data showed that human observers are more sensitive to the structure of the membrane while tolerating misalignment, contrary to commonly used evaluation criteria. Additionally, our results indicated that the human visual system processes images in both global-local and coarse-to-fine manners.

RESULTS

Based on these observations, we propose a computational framework for membrane segmentation that incorporates these characteristics of human perception. This framework includes a novel evaluation metric, the perceptual Hausdorff distance (PHD), and an end-to-end network called the PHD-guided segmentation network (PS-Net) that is trained using adaptively tuned PHD loss functions and a multiscale architecture. Our subjective experiments showed that the PHD metric is more consistent with human perception than other criteria, and our proposed PS-Net outperformed state-of-the-art methods on both low- and high-resolution EM image datasets as well as other natural image datasets.

AVAILABILITY AND IMPLEMENTATION

The code and dataset can be found at https://github.com/EmmaSRH/PS-Net.

Binocular summation is affected by crowding and tagging

In perceptual crowding, a letter easily recognized on its own, becomes unrecognizable if it is surrounded by other letters, an effect that confers a limit on the visual processing. Models assume that crowding is a hallmark of the periphery but that it is almost absent in the fovea. However, recently it was shown that crowding occurs in the fovea of people with an abnormal development of functional vision (amblyopia), when the stimulus is presented for a very short time. When targets and flankers are dissimilar, the crowding is reduced (tagging). Since a combination of binocular inputs increases the processing load, we investigated whether color tagging the target reduces crowding in the fovea of subjects with normal vision and determined how crowding is combined with binocular vision. The crowding effect at the fovea was significantly reduced by tagging with a color target. Interestingly, whereas binocular summation for a single letter was expected to be about 40%, it was significantly reduced and almost absent under crowding conditions. Our results are consistent with the notion that the crowding effect produces a high processing load on visual processing, which interferes with other processes such as binocular summation. We assume that the tagging effect in our experiment improved the subject's abilities (sensitivity and RT) by creating a "segmentation", i.e., a visual simulated separation between the target letter and the background. Interestingly, tagging the target with a distinct color can eliminate or reduce the crowding effect and consequently, binocular summation recovers.

Illusory Bands in Orientation and Spatial Frequency: A Cortical Analog to Mach Bands

Illusory bands at a luminance transition in space (ie an edge) are well known. Here we demonstrate illusory bands of enhanced orientations or spatial frequencies at transitions between higher-contrast and lower-contrast image content along the orientation and spatial-frequency dimensions—the dimensions of cortical spatial coding. We conclude that this illusion is a consequence of cortical-level suppression of units of similar orientations and spatial frequencies and serves to aid texture segmentation while providing efficient neural coding.

Reliability and validity of the Leuven Perceptual Organization Screening Test (L-POST)

Neuropsychological tests of visual perception mostly assess high-level processes like object recognition. Object recognition, however, relies on distinct mid-level processes of perceptual organization that are only implicitly tested in classical tests. Furthermore, the psychometric properties of the existing instruments are limited. To fill this gap, the Leuven perceptual organization screening test (L-POST) was developed, in which a wide range of mid-level phenomena are measured in 15 subtests. In this study, we evaluated reliability and validity of the L-POST. Performance on the test is evaluated relative to a norm sample of more than 1,500 healthy control participants. Cronbach's alpha of the norm sample and test-retest correlations for 20 patients provide evidence for adequate reliability of L-POST performance. The convergent and discriminant validity of the test was assessed in 40 brain-damaged patients, whose performance on the L-POST was compared with standard clinical tests of visual perception and other measures of cognitive function. The L-POST showed high sensitivity to visual dysfunction and decreased performance was specific to visual problems. In conclusion, the L-POST is a reliable and valid screening test for perceptual organization. It offers a useful online tool for researchers and clinicians to get a broader overview of the mid-level processes that are preserved or disrupted in a given patient.

A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations

Our first review article (Wagemans et al., 2012) on the occasion of the centennial anniversary of Gestalt psychology focused on perceptual grouping and figure-ground organization. It concluded that further progress requires a reconsideration of the conceptual and theoretical foundations of the Gestalt approach, which is provided here. In particular, we review contemporary formulations of holism within an information-processing framework, allowing for operational definitions (e.g., integral dimensions, emergent features, configural superiority, global precedence, primacy of holistic/configural properties) and a refined understanding of its psychological implications (e.g., at the level of attention, perception, and decision). We also review 4 lines of theoretical progress regarding the law of Prägnanz-the brain's tendency of being attracted towards states corresponding to the simplest possible organization, given the available stimulation. The first considers the brain as a complex adaptive system and explains how self-organization solves the conundrum of trading between robustness and flexibility of perceptual states. The second specifies the economy principle in terms of optimization of neural resources, showing that elementary sensors working independently to minimize uncertainty can respond optimally at the system level. The third considers how Gestalt percepts (e.g., groups, objects) are optimal given the available stimulation, with optimality specified in Bayesian terms. Fourth, structural information theory explains how a Gestaltist visual system that focuses on internal coding efficiency yields external veridicality as a side effect. To answer the fundamental question of why things look as they do, a further synthesis of these complementary perspectives is required.

A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization

In 1912, Max Wertheimer published his paper on phi motion, widely recognized as the start of Gestalt psychology. Because of its continued relevance in modern psychology, this centennial anniversary is an excellent opportunity to take stock of what Gestalt psychology has offered and how it has changed since its inception. We first introduce the key findings and ideas in the Berlin school of Gestalt psychology, and then briefly sketch its development, rise, and fall. Next, we discuss its empirical and conceptual problems, and indicate how they are addressed in contemporary research on perceptual grouping and figure-ground organization. In particular, we review the principles of grouping, both classical (e.g., proximity, similarity, common fate, good continuation, closure, symmetry, parallelism) and new (e.g., synchrony, common region, element and uniform connectedness), and their role in contour integration and completion. We then review classic and new image-based principles of figure-ground organization, how it is influenced by past experience and attention, and how it relates to shape and depth perception. After an integrated review of the neural mechanisms involved in contour grouping, border ownership, and figure-ground perception, we conclude by evaluating what modern vision science has offered compared to traditional Gestalt psychology, whether we can speak of a Gestalt revival, and where the remaining limitations and challenges lie. A better integration of this research tradition with the rest of vision science requires further progress regarding the conceptual and theoretical foundations of the Gestalt approach, which is the focus of a second review article.

Equivalent representation of real and illusory contours in macaque V4

The cortical processing of illusory contours provides a unique window for exploring the brain mechanisms underlying visual perception. Previous electrophysiological single-cell recordings demonstrate that a subgroup of cells in macaque V1 and V2 signal the presence of illusory contours, whereas recent human brain imaging studies reveal higher-order visual cortices playing a central role in illusory figure processing. It seems that the processing of illusory contours/figures may engage multiple cortical interactions between hierarchically organized processing stages in the ventral visual pathway of primates. However, it is not yet known in which brain areas illusory contours are represented in the same manner as real contours at both the population and single-cell levels. Here, by combining intrinsic optical imaging in anesthetized rhesus macaques with single-cell recordings in awake ones, we found a complete overlap of orientation domains in visual cortical area V4 for processing real and illusory contours. In contrast, the orientation domains mapped in early visual areas V1 and V2 mainly encoded the local physical stimulus features inducing the subjective perception of global illusory contours. Our results indicate that real and illusory contours are encoded equivalently by the same functional domains in V4, suggesting that V4 is a key cortical locus for integration of local features into global contours.

Phenomenology and neurophysiological correlations: two approaches to perception research

This article argues that phenomenological description and neurophysiological correlation complement each other in perception research. Whilst phenomena constitute the material, neuronal mechanisms are indispensable for their explanation. Numerous examples of neurophysiological correlates show that the correlation of phenomenology and neurophysiology is fruitful. Phenomena for which neuronal mechanism have been found include: (in area V1) filling-in of real and artificial scotomata, contour integration, figure-ground segregation by orientation contrast, amodal completion, and motion transparency; (in V2) modal completion, border ownership, surface transparency, and cyclopean perception; (in V3) alignment in dotted contours, and filling-in with dynamic texture; (in V4) colour constancy; (in MT) shape by accretion/deletion, grouping by coherent motion, apparent motion in motion quartets, motion in apertures, and biological motion. Results suggest that in monkey visual cortex, occlusion cues, including stereo depth, are predominantly processed in lower areas, whereas mechanisms for grouping and motion are primarily represented in higher areas. More correlations are likely to emerge as neuroscientists strive for a better understanding of visual perception. The paper concludes with a review of major achievements in visual neuroscience pertinent to the study of the phenomena under consideration.

Visual perception and consciousness in dermatopathology: mechanisms of figure-ground segregation account for errors in diagnosis

Visual perception has been the object of research in psychology for almost a century. Little has been written, however, about the effects of perceptive phenomena on methods in medicine that utilize interpretation of two-dimensional images for diagnosis. Starting from the work by Edgar Rubin in the beginning of the last century, this article gives a summary of observations of psychologists who investigated the mechanisms of so-called "figure-ground segregation." These unconscious mechanisms follow rules that explain why certain structures are perceived consciously as a figure, whereas other structures surrounding such a figure are neglected and not perceived consciously in detail. Perception of a structure as a figure can be due to, for example, a convex shape of its contour, proximity of lines around it, closed contours, a simple shape, and attribution of meaning to a structure. In examples from the practice of dermatopathology, those unconscious mechanisms of figure-ground segregation will be shown to be relevant to diagnosis of sections of tissue. The mechanisms help to explain why, for example, ill-defined and concave-shaped structures, stromal differences of neoplasms, interstitial infiltrates and deposits, and simulators of common diseases are often difficult to recognize at first sight. Teachers of dermatopathology need to be aware of these unconscious mechanisms of visual perception because they explain why novices struggle with certain diagnoses and differential diagnoses. Proper instruction about these phenomena, early in the process of training, will prevent a student from being frustrated with misperceptions.

Identification of everyday objects on the basis of fragmented outline versions

Although Attneave (1954 Psychological Review 61 183 193) and Biederman (1987 Psychological Review 94 115-147) have argued that curved contour segments are most important in shape perception, Kennedy and Domander (1985 Perception 14 367-370) showed that fragmented object contours are better identifiable when straight segments are shown. We used the set of line drawings published by Snodgrass and Vanderwart (1980 Journal of Experimental Psychology: Human Learning and Memory 6 174-215), to make outline versions that could be used to investigate this issue with a larger and more heterogeneous stimulus set. Fragments were placed either around the 'salient' points or around the midpoints (points midway between two salient points), creating curved versus relatively straight fragments when the original outline was fragmented (experiment 1), or angular and straight fragments when straight-line versions were fragmented (experiment 2). We manipulated fragment length in each experiment except the last one, in which we presented only selected points (experiment 3). While fragmented versions were on average more identifiable when straight fragments were shown, certain objects were more identifiable when the curved segments or the angles were shown. A tentative explanation of these results is presented in terms of an advantage for straight segments during grouping processes for outlines with high part salience, and an advantage for curved segments during matching processes for outlines with low part salience.

Dynamic Grouping and Interpolation Induced by Flickering Stimuli

A novel grouping and interpolation effect induced by flickering stimuli is described: a matrix of flickering elements forms stick-like clusters, then the clusters gradually dissociate back into the discrete elements within a few seconds. On continuous viewing, this flicker-induced grouping and interpolation repeatedly disappeared and reappeared. The perceived strength of this phenomenon peaked when the luminance of the flickering elements was alternately darker and lighter than the background; thus, the reversal of the luminance contrast polarity was responsible for flicker-induced grouping. The temporal dynamics of flicker-induced grouping showed the stochastic nature of perceptual alternation, which depended on the global structure of the stimulus. From these results, it is concluded that flicker-induced grouping reflects multiple stages of visual processing.

In honour of Lothar Spillmann - filling-in, wiggly lines, adaptation, and aftereffects

I have studied a number of visual phenomena that Lothar Spillmann has already elucidated. These include: Neon spreading: when a small red cross is superimposed on intersecting black lines, the red cross seems to spread out into an illusory disk. Unlike the Hermann grid, neon spreading is relatively unaffected when the black lines are curved or wiggly. This suggests that the Hermann grid, but not neon spreading, involves long-range interactions. Neon spreading can be shown in random-dot patterns, even without intersections. It is strongest when the red crosses are equiluminous with the gray background. Adaptation, aftereffects, and filling-in: direct and induced aftereffects of color, motion, and dimming. Artificial scotomata and filling-in: the "dam" theory is false. Staring at wiggly lines or irregularly scattered dots makes them gradually appear straighter, or more regularly spaced. I present evidence that irregularity is actually a visual dimension to which the visual system can adapt. Conjectures on the nature of peripheral fading and of motion-induced blindness. Some failed experiments on correlated visual inputs and cortical plasticity.

From perceptive fields to Gestalt

Studies on visual psychophysics and perception conducted in the Freiburg psychophysics laboratory during the last 35 years are reviewed. Many of these were inspired by single-cell neurophysiology in cat and monkey. The aim was to correlate perceptual phenomena and their effects to possible neuronal mechanisms from retina to visual cortex and beyond. Topics discussed include perceptive field organization, figure-ground segregation and grouping, fading and filling-in, and long-range color interaction. While some of these studies succeeded in linking perception to neuronal response patterns, others require further investigation. The task of probing the human brain with perceptual phenomena continues to be a challenge for the future.

Two competing mechanisms underlying neon color spreading, visual phantoms and grating induction

Neon color spreading is closely related to the photopic visual phantom illusion, since these two completion phenomena are characterized by in-phase lightness induction, and the only difference in the stimulus configuration is the difference in the inducer height. This idea was supported by the present study. Neon color spreading showed almost the same function of critical spatial frequency as photopic visual phantoms (Experiment 1), and the critical spatial frequency was constant as the inducer height was changed (Experiment 2). We also examined the relationship between neon color spreading and grating induction (characterized by counterphase lightness induction) in critical spatial frequency (Experiment 3) and in magnitudes of lightness induction (Experiment 4) as a function of the inducer height. The inducer height at which in-phase (neon color spreading) appearance gave way to counterphase (grating) induction was approximately 0.1 deg. These results suggest that neon color spreading shares a common neural mechanism with the photopic visual phantom illusion and that this mechanism is different from, and competes with, the mechanism of grating induction.

Frequency dependence and gender effects in visual cortical regions involved in temporal frequency dependent pattern processing

Neural response to flickering stimuli has been shown to be frequency dependent in the primary visual cortex. Controversial gender differences in blood oxygen level dependent (BOLD) amplitude upon 6 and 8 Hz visual stimulation have been reported. In order to analyze frequency and gender effects in early visual processing we employed a passive graded task paradigm with a dartboard stimulus combining eight temporal frequencies from 0 to 22 Hz in one run. Activation maps were calculated within Statistical Parametric Mapping, and BOLD amplitudes were estimated for each frequency within the striate and extrastriate visual cortex. The BOLD amplitude was found to steadily rise up to 8 Hz in BA 17 and 18 with an activation plateau at higher frequencies. In addition, we observed a laterality effect in the striate cortex with higher BOLD contrasts in the right hemisphere in men and in women. BOLD response rises similarly in men and women up to 8 Hz but with lower amplitudes in women at 4, 8, and 12 Hz (30% lower). No frequency effect above 1 Hz was found in the extrastriate visual cortex. There was also a regional specific gender difference. Men activated more in the right lingual gyrus (BA 18) and the right cerebellum compared with women, whereas women showed more activation in the right inferior temporal gyrus (BA 17). The study indicates that frequency dependent processing at the cortical level is limited to the striate cortex and may be associated with a more global information processing (right hemisphere dominance), particularly in men. The finding of significantly lower BOLD amplitudes in women despite previously shown larger VEP (visual evoked potential) amplitudes might suggest gender differences in cerebral hemodynamics (baseline rCBV, rCBF, or neurovascular coupling). The regional distinction points at additional differences in psychological processing even when using a simple visual stimulus.