Contrast negation and texture synthesis differentially disrupt natural texture appearance

Natural Contrast Statistics Facilitate Human Face Categorization

The ability to detect faces in the environment is of utmost ecological importance for human social adaptation. While face categorization is efficient, fast and robust to sensory degradation, it is massively impaired when the facial stimulus does not match the natural contrast statistics of this visual category, i.e., the typically experienced ordered alternation of relatively darker and lighter regions of the face. To clarify this phenomenon, we characterized the contribution of natural contrast statistics to face categorization. Specifically, 31 human adults viewed various natural images of nonface categories at a rate of 12 Hz, with highly variable images of faces occurring every eight stimuli (1.5 Hz). As in previous studies, neural responses at 1.5 Hz as measured with high-density electroencephalography (EEG) provided an objective neural index of face categorization. Here, when face images were shown in their naturally experienced contrast statistics, the 1.5-Hz face categorization response emerged over occipito-temporal electrodes at very low contrast [5.1%, or 0.009 root-mean-square (RMS) contrast], quickly reaching optimal amplitude at 22.6% of contrast (i.e., RMS contrast of 0.041). Despite contrast negation preserving an image's spectral and geometrical properties, negative contrast images required twice as much contrast to trigger a face categorization response, and three times as much to reach optimum. These observations characterize how the internally stored natural contrast statistics of the face category facilitate visual processing for the sake of fast and efficient face categorization.

Visual perception and cuttlefish camouflage

Visual perception is inherently statistical: brains exploit repeating features of natural scenes to disambiguate images that could, in principle, have many causes. A clear case for the relevance of statistical inference in vision is animal camouflage. Although visual scenes are each composed of unique arrangements of pixels, they are usually perceived mainly as groupings of statistically defined patches (sandy/leafy/smooth etc…); this fact is exploited by camouflaging animals. The unique ability of certain cephalopods to camouflage actively within many different surroundings provides a rare and direct behavioral readout for texture perception. In addition, because cephalopods and chordates each arose after a phylogenetic split that occurred some 600M years ago, the apparent convergence of texture perception across these groups suggests common principles. Studying cephalopod camouflage may thus help us resolve general problems of visual perception.

Image content is more important than Bouma's Law for scene metamers

We subjectively perceive our visual field with high fidelity, yet peripheral distortions can go unnoticed and peripheral objects can be difficult to identify (crowding). Prior work showed that humans could not discriminate images synthesised to match the responses of a mid-level ventral visual stream model when information was averaged in receptive fields with a scaling of about half their retinal eccentricity. This result implicated ventral visual area V2, approximated ‘Bouma’s Law’ of crowding, and has subsequently been interpreted as a link between crowding zones, receptive field scaling, and our perceptual experience. However, this experiment never assessed natural images. We find that humans can easily discriminate real and model-generated images at V2 scaling, requiring scales at least as small as V1 receptive fields to generate metamers. We speculate that explaining why scenes look as they do may require incorporating segmentation and global organisational constraints in addition to local pooling.

As you read this digest, your eyes move to follow the lines of text. But now try to hold your eyes in one position, while reading the text on either side and below: it soon becomes clear that peripheral vision is not as good as we tend to assume. It is not possible to read text far away from the center of your line of vision, but you can see ‘something’ out of the corner of your eye. You can see that there is text there, even if you cannot read it, and you can see where your screen or page ends. So how does the brain generate peripheral vision, and why does it differ from what you see when you look straight ahead?

One idea is that the visual system averages information over areas of the peripheral visual field. This gives rise to texture-like patterns, as opposed to images made up of fine details. Imagine looking at an expanse of foliage, gravel or fur, for example. Your eyes cannot make out the individual leaves, pebbles or hairs. Instead, you perceive an overall pattern in the form of a texture. Our peripheral vision may also consist of such textures, created when the brain averages information over areas of space.

Wallis, Funke et al. have now tested this idea using an existing computer model that averages visual input in this way. By giving the model a series of photographs to process, Wallis, Funke et al. obtained images that should in theory simulate peripheral vision. If the model mimics the mechanisms that generate peripheral vision, then healthy volunteers should be unable to distinguish the processed images from the original photographs. But in fact, the participants could easily discriminate the two sets of images. This suggests that the visual system does not solely use textures to represent information in the peripheral visual field. Wallis, Funke et al. propose that other factors, such as how the visual system separates and groups objects, may instead determine what we see in our peripheral vision.

This knowledge could ultimately benefit patients with eye diseases such as macular degeneration, a condition that causes loss of vision in the center of the visual field and forces patients to rely on their peripheral vision.

Children's use of visual summary statistics for material categorization

Although adults' ability to recognize materials from complex natural images has been well characterized, we still know very little about the development of material perception. When do children exhibit adult-like abilities to categorize materials? What visual features do they use to do so as a function of age and material category? In the present study, we attempted to address both of these issues in two experiments that we administered to school-age children (5–10 years old) and adults. In both tasks, we asked our participants to categorize natural materials (metal, stone, water, and wood) using original images of these materials as well as synthetic images made with the Portilla–Simoncelli algorithm. By including synthetic images in our stimulus set, we were able to assess both how material categorization develops during childhood and how visual summary statistics are recruited for material perception across age groups. We observed that when asked to provide category labels for individual images (Experiment 1), young children were disproportionately bad at categorizing some materials after they were synthesized, suggesting material-specific changes in information use over the course of development. However, when asked to match real and synthetic images according to material category without labeling (Experiment 2), these effects were weakened. We conclude that while children have adult-like abilities to encode and compare images based on summary statistics, the mapping between summary statistics and category labels undergoes prolonged development during childhood.

The Visual N1 Is Sensitive to Deviations from Natural Texture Appearance

Disruptions of natural texture appearance are known to negatively impact performance in texture discrimination tasks, for example, such that contrast-negated textures, synthetic textures, and textures depicting abstract art are processed less efficiently than natural textures. Presently, we examined how visual ERP responses (the P1 and the N1 in particular) were affected by violations of natural texture appearance. We presented participants with images depicting either natural textures or synthetic textures made from the original stimuli. Both stimulus types were additionally rendered either in positive or negative contrast. These appearance manipulations (negation and texture synthesis) preserve a range of low-level features, but also disrupt higher-order aspects of texture appearance. We recorded continuous EEG while participants completed a same/different image discrimination task using these images and measured both the P1 and N1 components over occipital recording sites. While the P1 exhibited no sensitivity to either contrast polarity or real/synthetic appearance, the N1 was sensitive to both deviations from natural appearance. Polarity reversal and synthetic appearance affected the N1 latency differently, however, suggesting a differential impact on processing. Our results suggest that stages of visual processing indexed by the P1 and N1 are sensitive to high-order statistical regularities in natural textures and also suggest that distinct violations of natural appearance impact neural responses differently.

Invariant texture perception is harder with synthetic textures: Implications for models of texture processing

Texture synthesis models have become a popular tool for studying the representations supporting texture processing in human vision. In particular, the summary statistics implemented in the Portilla-Simoncelli (P-S) model support high-quality synthesis of natural textures, account for performance in crowding and search tasks, and may account for the response properties of V2 neurons. We chose to investigate whether or not these summary statistics are also sufficient to support texture discrimination in a task that required illumination invariance. Our observers performed a match-to-sample task using natural textures photographed with either diffuse overhead lighting or lighting from the side. Following a briefly presented sample texture, participants identified which of two test images depicted the same texture. In the illumination change condition, illumination differed between the sample and the matching test image. In the no change condition, sample textures and matching test images were identical. Critically, we generated synthetic versions of these images using the P-S model and also tested participants with these. If the statistics in the P-S model are sufficient for invariant texture perception, performance with synthetic images should not differ from performance in the original task. Instead, we found a significant cost of applying texture synthesis in both lighting conditions. We also observed this effect when power-spectra were matched across images (Experiment 2) and when sample and test images were drawn from unique locations in the parent textures to minimize the contribution of image-based processing (Experiment 3). Invariant texture processing thus depends upon measurements not implemented in the P-S algorithm.

Face animacy is not all in the eyes: evidence from contrast chimeras

Observers are capable of distinguishing real faces from artificial faces of various types (eg dolls, computer-generated faces) relatively easily. While a number of diagnostic cues are potentially available to observers to accomplish this task, the appearance of the eyes has been shown to be critically important. However, eye appearance appears to interact with other cues, like the appearance of the skin, in some settings. The 'uncanny' appearance of some artificial faces appears to result from multiple visual features and their departure from typical face norms, for example, and recent results investigating how real and artificial features are perceived in chimeric faces also suggest that observers use multiple cues to measure face animacy. Presently, we examined the cues that support real-artificial face discrimination by using contrast negation and so-called 'contrast chimeras' to selectively disrupt the appearance of the eyes and the remainder of the face pattern. First, we demonstrate that, like other aspects of face perception, perceived animacy is significantly impacted by contrast negation. Second, by selectively manipulating the contrast of the eyes relative to the rest of the face, we demonstrate that these face regions are of approximately equal use to observers for animacy discrimination.

Infant Preference for Natural Texture Statistics is Modulated by Contrast Polarity

Adult observers are sensitive to statistical regularities present in natural images. Developmentally, research has shown that children do not show sensitivity to these natural regularities until approximately 8-10 years of age. This finding is surprising given that even infants gradually encode a range of high-level statistical regularities of their visual environment in the first year of life, We suggest that infants may in fact exhibit sensitivity to natural image statistics under circumstances where images of complex, natural textures, such as a photograph of rocks, are used as experimental stimuli and natural appearance is substantially manipulated. We tested this hypothesis by examining how infants' visual preference for real versus computer-generated synthetic textures was modulated by contrast negation, which produces an image similar to a photographic negative. We observed that older infants' (9-months of age) preferential looking behavior in this task was affected by contrast polarity, suggesting that the infant visual system is sensitive to deviations from natural texture appearance, including (1) discrepancies in appearance that differentiate natural and synthetic textures from one another and (2) the disruption of contrast polarity following negation. We discuss our results in the context of adult texture processing and the "perceptual narrowing" of visual recognition during the first year of life.