From perceptive fields to Gestalt

Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention

Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.

Fine-scale measurement of the blind spot borders

The blind spot is both a necessity and a nuisance for seeing. It is the portion of the visual field projecting to where the optic nerve crosses the retina, a region devoid of photoreceptors and hence visual input. The precise way in which vision transitions into blindness at the blind spot border is to date unknown. A chief challenge to map this transition is the incessant movement of the eye, which unavoidably smears measurements across space. In this study, we used high-resolution eye-tracking and state-of-the-art retinal stabilization to finely map the blind spot borders. Participants reported the onset of tiny high-contrast probes that were briefly flashed at precise positions around the blind spot. This method has sufficient resolution to enable mapping of blood vessels from psychophysical measurements. Our data show that, even after accounting for eye movements, the transition zones at the edges of the blind spot are considerable. On the horizontal meridian, the regions with detection rates between 80% and 20% span approximately 25% of the overall width of the blind spot. These borders also vary considerably in size across different axes. These data show that the transition from full visibility to blindness at the blind spot border is not abrupt but occurs over a broad area.

Microsaccades mediate perceptual alternations in Monet's "Impression, sunrise"

Troxler fading, the perceptual disappearance of stationary images upon sustained fixation, is common for objects with equivalent luminance to that of the background. Previous work showed that variations in microsaccadic rates underlie the perceptual vanishing and intensification of simple stimuli, such as Gabor patches. Here, we demonstrate that microsaccade dynamics also contribute to Troxler fading and intensification during the viewing of representational art. Participants fixated a small spot while viewing either a Gabor patch on a blank background, or Monet's painting "Impression, Sunrise." They continuously reported, via button press/release, whether the Gabor patch, or the sun in Monet's painting, was fading versus intensifying, while their eye movements were recorded with high precision. Microsaccade rates peaked before reports of increased visibility, and dropped before reports of decreased visibility or fading, both when viewing Gabor patches and Monet's sun. These results reveal that the relationship between microsaccade production and the reversal and prevention of Troxler fading applies not only to the viewing of contrived stimuli, but also to the observation of "Impression, Sunrise." Whether or not perceptual fading was consciously intended by Monet, our findings indicate that observers' oculomotor dynamics are a contributor to the cornerstone of Impressionism.

The empirical characteristics of human pattern vision defy theoretically-driven expectations

Contrast is the most fundamental property of images. Consequently, any comprehensive model of biological vision must incorporate this attribute and provide a veritable description of its impact on visual perception. Current theoretical and computational models predict that vision should modify its characteristics at low contrast: for example, it should become broader (more lowpass) to protect from noise, as often demonstrated by individual neurons. We find that the opposite is true for human discrimination of elementary image elements: vision becomes sharper, not broader, as contrast approaches threshold levels. Furthermore, it suffers from increased internal variability at low contrast and it transitions from a surprisingly linear regime at high contrast to a markedly nonlinear processing mode in the low-contrast range. These characteristics are hard-wired in that they happen on a single trial without memory or expectation. Overall, the empirical results urge caution when attempting to interpret human vision from the standpoint of optimality and related theoretical constructs. Direct measurements of this phenomenon indicate that the actual constraints derive from intrinsic architectural features, such as the co-existence of complex-cell-like and simple-cell-like components. Small circuits built around these elements can indeed account for the empirical results, but do not appear to operate in a manner that conforms to optimality even approximately. More generally, our results provide a compelling demonstration of how far we still are from securing an adequate computational account of the most basic operations carried out by human vision.

The Elementary Operations of Human Vision Are Not Reducible to Template-Matching

It is generally acknowledged that biological vision presents nonlinear characteristics, yet linear filtering accounts of visual processing are ubiquitous. The template-matching operation implemented by the linear-nonlinear cascade (linear filter followed by static nonlinearity) is the most widely adopted computational tool in systems neuroscience. This simple model achieves remarkable explanatory power while retaining analytical tractability, potentially extending its reach to a wide range of systems and levels in sensory processing. The extent of its applicability to human behaviour, however, remains unclear. Because sensory stimuli possess multiple attributes (e.g. position, orientation, size), the issue of applicability may be asked by considering each attribute one at a time in relation to a family of linear-nonlinear models, or by considering all attributes collectively in relation to a specified implementation of the linear-nonlinear cascade. We demonstrate that human visual processing can operate under conditions that are indistinguishable from linear-nonlinear transduction with respect to substantially different stimulus attributes of a uniquely specified target signal with associated behavioural task. However, no specific implementation of a linear-nonlinear cascade is able to account for the entire collection of results across attributes; a satisfactory account at this level requires the introduction of a small gain-control circuit, resulting in a model that no longer belongs to the linear-nonlinear family. Our results inform and constrain efforts at obtaining and interpreting comprehensive characterizations of the human sensory process by demonstrating its inescapably nonlinear nature, even under conditions that have been painstakingly fine-tuned to facilitate template-matching behaviour and to produce results that, at some level of inspection, do conform to linear filtering predictions. They also suggest that compliance with linear transduction may be the targeted outcome of carefully crafted nonlinear circuits, rather than default behaviour exhibited by basic components.

Photopic and scotopic spatiotemporal tuning of adult zebrafish vision

Sensitivity to spatial and temporal patterns is a fundamental aspect of vision. Herein, we investigated this sensitivity in adult zebrafish for a wide range of spatial (0.014 to 0.511 cycles/degree [c/d]) and temporal frequencies (0.025 to 6 cycles/s) to better understand their visual system. Measurements were performed at photopic (1.8 log cd m(-2)) and scotopic (-4.5 log cd m(-2)) light levels to assess the optokinetic response (OKR). The resulting spatiotemporal contrast sensitivity (CS) functions revealed that the OKR of zebrafish is tuned to spatial frequency and speed but not to temporal frequencies. Thereby, optimal test parameters for CS measurements were identified. At photopic light levels, a spatial frequency of 0.116 ± 0.01 c/d (mean ± SD) and a grating speed of 8.42 ± 2.15 degrees/second (d/s) was ideal; at scotopic light levels, these values were 0.110 ± 0.02 c/d and 5.45 ± 1.31 d/s, respectively. This study allows to better characterize zebrafish mutants with altered vision and to distinguish between defects of rod and cone photoreceptors as measurements were performed under different light conditions.

Semantic control of feature extraction from natural scenes

In the early stages of image analysis, visual cortex represents scenes as spatially organized maps of locally defined features (e.g., edge orientation). As image reconstruction unfolds and features are assembled into larger constructs, cortex attempts to recover semantic content for object recognition. It is conceivable that higher level representations may feed back onto early processes and retune their properties to align with the semantic structure projected by the scene; however, there is no clear evidence to either support or discard the applicability of this notion to the human visual system. Obtaining such evidence is challenging because low and higher level processes must be probed simultaneously within the same experimental paradigm. We developed a methodology that targets both levels of analysis by embedding low-level probes within natural scenes. Human observers were required to discriminate probe orientation while semantic interpretation of the scene was selectively disrupted via stimulus inversion or reversed playback. We characterized the orientation tuning properties of the perceptual process supporting probe discrimination; tuning was substantially reshaped by semantic manipulation, demonstrating that low-level feature detectors operate under partial control from higher level modules. The manner in which such control was exerted may be interpreted as a top-down predictive strategy whereby global semantic content guides and refines local image reconstruction. We exploit the novel information gained from data to develop mechanistic accounts of unexplained phenomena such as the classic face inversion effect.

Whole person-evoked fMRI activity patterns in human fusiform gyrus are accurately modeled by a linear combination of face- and body-evoked activity patterns

Visual cues from the face and the body provide information about another's identity, emotional state, and intentions. Previous neuroimaging studies that investigated neural responses to (bodiless) faces and (headless) bodies have reported overlapping face- and body-selective brain regions in right fusiform gyrus (FG). In daily life, however, faces and bodies are typically perceived together and are effortlessly integrated into the percept of a whole person, raising the possibility that neural responses to whole persons are qualitatively different than responses to isolated faces and bodies. The present study used fMRI to examine how FG activity in response to a whole person relates to activity in response to the same face and body but presented in isolation. Using multivoxel pattern analysis, we modeled person-evoked response patterns in right FG through a linear combination of face- and body-evoked response patterns. We found that these synthetic patterns were able to accurately approximate the response patterns to whole persons, with face and body patterns each adding unique information to the response patterns evoked by whole person stimuli. These results suggest that whole person responses in FG primarily arise from the coactivation of independent face- and body-selective neural populations.

Patients with schizophrenia do not preserve automatic grouping when mentally re-grouping figures: shedding light on an ignored difficulty

Looking at a pair of objects is easy when automatic grouping mechanisms bind these objects together, but visual exploration can also be more flexible. It is possible to mentally “re-group” two objects that are not only separate but belong to different pairs of objects. “Re-grouping” is in conflict with automatic grouping, since it entails a separation of each item from the set it belongs to. This ability appears to be impaired in patients with schizophrenia. Here we check if this impairment is selective, which would suggest a dissociation between grouping and “re-grouping,” or if it impacts on usual, automatic grouping, which would call for a better understanding of the interactions between automatic grouping and “re-grouping.” Sixteen outpatients with schizophrenia and healthy controls had to identify two identical and contiguous target figures within a display of circles and squares alternating around a fixation point. Eye-tracking was used to check central fixation. The target pair could be located in the same or separate hemifields. Identical figures were grouped by a connector (grouped automatically) or not (to be re-grouped). Attention modulation of automatic grouping was tested by manipulating the proportion of connected and unconnected targets, thus prompting subjects to focalize on either connected or unconnected pairs. Both groups were sensitive to automatic grouping in most conditions, but patients were unusually slowed down for connected targets while focalizing on unconnected pairs. In addition, this unusual effect occurred only when targets were presented within the same hemifield. Patients and controls differed on this asymmetry between within- and across-hemifield presentation, suggesting that patients with schizophrenia do not re-group figures in the same way as controls do. We discuss possible implications on how “re-grouping” ties in with ongoing, automatic perception in healthy volunteers.

"Brain-reading" of perceived colors reveals a feature mixing mechanism underlying perceptual filling-in in cortical area V1

Visual filling-in occurs when a retinally stabilized object undergoes perceptual fading. As the term "filling-in" implies, it is commonly believed that information about the apparently vanished object is lost and replaced solely by information arising from the surrounding background. Here we report multivoxel pattern analysis fMRI data that challenge this long-held belief. When subjects view blue disks on a red background while fixating, the stimulus and background appear to turn a uniform purple upon perceptual fading, suggesting that a feature mixing mechanism may underlie color filling-in. We find that ensemble fMRI signals in retinotopic visual areas reliably predict (i) which of three colors a subject reports seeing; (ii) whether a subject is in a perceptually filled-in state or not; and (iii) furthermore, while subjects are in the perceptual state of filling-in, the BOLD signal activation pattern in the sub-areas of V1 corresponding to the location of the blue disks behaves as if subjects are in fact viewing a perceptually mixed color (purple), rather than the color of the disks (blue) or the color of the background (red). These results imply that the mechanism of filling-in in stimuli in which figure and background surfaces are equated is a process of "feature mixing", not "feature replacement". These data indicate that feature mixing may involve cortical areas as early as V1.

Spatial organisation in passive tactile perception: is there a tactile field?

The perceptual field is a cardinal concept of sensory psychology. 'Field' refers to a representation in which perceptual contents have spatial properties and relations which derive from the spatial properties and relations of corresponding stimuli. It is a matter of debate whether a perceptual field exists in touch analogous to the visual field. To study this issue, we investigated whether tactile stimuli on the palm can be perceived as complex stimulus patterns, according to basic spatial principles. Subjects judged the intensity of a target stimulus to the palm, ignoring two brief preceding touches at nearby flanker locations. We found that the judgements of the target intensity were boosted by flankers when the target lay on the line joining the flankers in comparison to when the target lay away from this line. Therefore, we suggest that a tactile spatial organisation, i.e. a tactile field, exists; the field supports the relation of collinearity; it is automatically and implicitly activated by touch, and it groups spatially coherent perceptual contents.

A new psychophysical estimation of the receptive field size

When a line extends beyond the width of an aperture, its direction of motion cannot be detected correctly. Only the component of motion perpendicular to the line is detectable (aperture problem). Early visual areas face the same aperture problem because receptive field sizes are relatively small. The susceptibility of early visual areas to the aperture problem opens an opportunity to measure the aperture width of a receptive field psychophysically that can be used to estimate the receptive field size. We found an already established visual illusion (the rotating tilted lines illusion or RTLI) can be used to measure the aperture size and hence estimate the receptive field size. To estimate the receptive field size, we conducted a psychophysical experiment in which the radii and tilted line length of RTLI were systematically changed. Our psychophysical estimation of receptive field size strongly corresponds with the previous measures of receptive field size using electrophysiological and fMRI methods.