Spotting animals in natural scenes: efficiency of humans and monkeys at very low contrasts

Deeper neural network models better reflect how humans cope with contrast variation in object recognition

Visual inputs are far from ideal in everyday situations such as in the fog where the contrasts of input stimuli are low. However, human perception remains relatively robust to contrast variations. To provide insights about the underlying mechanisms of contrast invariance, we addressed two questions. Do contrast effects disappear along the visual hierarchy? Do later stages of the visual hierarchy contribute to contrast invariance? We ran a behavioral experiment where we manipulated the level of stimulus contrast and the involvement of higher-level visual areas through immediate and delayed backward masking of the stimulus. Backward masking led to significant drop in performance in our visual categorization task, supporting the role of higher-level visual areas in contrast invariance. To obtain mechanistic insights, we ran the same categorization task on three state-of the-art computational models of human vision each with a different depth in visual hierarchy. We found contrast effects all along the visual hierarchy, no matter how far into the hierarchy. Moreover, that final layers of deeper hierarchical models, which had been shown to be best models of final stages of the visual system, coped with contrast effects more effectively. These results suggest that, while contrast effects reach the final stages of the hierarchy, those stages play a significant role in compensating for contrast variations in the visual system.

Rapid category selectivity for animals versus man-made objects: An N2pc study

Visual recognition occurs rapidly at multiple categorization levels, including the superordinate level (e.g., animal), basic level (e.g., cat), or exemplar level (e.g., my cat). Visual search for animals is faster than for man-made objects, even when the images from those categories have comparable gist statistics (i.e., low- or mid-level visual information), which suggests that higher-level, conceptual influences may support this search advantage for animals. However, it remains unclear whether the search advantage can be explained in part by early visual search processes via the N2pc ERP component, which emerges earlier than behavioral responses, across different categorization levels. Participants searched for 1) an exact image (e.g., a specific squirrel image, Exemplar-level Search), 2) any images of an item (e.g., any squirrels, Basic-level Search), or 3) any items in a category (e.g., any animals, Superordinate-level Search). In addition to Target Present trials, Foil trials measured involuntary attentional selection of task-irrelevant images related to the targets (e.g., other squirrel images when searching for a specific squirrel image, or other animals when searching for squirrels). ERP results revealed 1) a larger N2pc amplitude during Foil trials in Exemplar-level Search for animals than man-made objects, and 2) faster onset latencies for animal search than man-made object search across all categorization levels. These results suggest that the search advantage for animals over man-made objects emerges early, and that attentional selection is more biased toward the basic-level (e.g., squirrel) for animals than for man-made objects during visual search.

Partial awareness can be induced by independent cognitive access to different spatial frequencies

Partial awareness-an intermediate state between complete consciousness and unconsciousness-has been explained by independent cognitive access to different levels of representation in hierarchical visual processing. This account, however, cannot explain graded visual experiences in low levels. We aimed to explain partial awareness in low levels of visual processing by independent cognitive access to different spatial frequencies. To observe partial awareness stably, we used a novel method. Stimuli were presented briefly (12 ms) and repeatedly with a specific inter-stimulus interval, ranging from 0 to 235 ms. By using various stimuli containing high and low spatial frequencies (superimposed sinusoidal gratings, Navon letters, and scenes), we found that conscious percept was degraded with increasing inter-stimulus intervals. However, the degree of degradation was smaller for low spatial frequency than for high spatial frequency information. Our results reveal that cognitive access to different spatial frequencies can occur independently and this can explain partial awareness in low levels of visual processing.

Asymmetric Compression of Representational Space for Object Animacy Categorization under Degraded Viewing Conditions

Animacy is a robust organizing principle among object category representations in the human brain. Using multivariate pattern analysis methods, it has been shown that distance to the decision boundary of a classifier trained to discriminate neural activation patterns for animate and inanimate objects correlates with observer RTs for the same animacy categorization task [Ritchie, J. B., Tovar, D. A., & Carlson, T. A. Emerging object representations in the visual system predict reaction times for categorization. PLoS Computational Biology, 11, e1004316, 2015; Carlson, T. A., Ritchie, J. B., Kriegeskorte, N., Durvasula, S., & Ma, J. Reaction time for object categorization is predicted by representational distance. Journal of Cognitive Neuroscience, 26, 132–142, 2014]. Using MEG decoding, we tested if the same relationship holds when a stimulus manipulation (degradation) increases task difficulty, which we predicted would systematically decrease the distance of activation patterns from the decision boundary and increase RTs. In addition, we tested whether distance to the classifier boundary correlates with drift rates in the linear ballistic accumulator [Brown, S. D., & Heathcote, A. The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178, 2008]. We found that distance to the classifier boundary correlated with RT, accuracy, and drift rates in an animacy categorization task. Split by animacy, the correlations between brain and behavior were sustained longer over the time course for animate than for inanimate stimuli. Interestingly, when examining the distance to the classifier boundary during the peak correlation between brain and behavior, we found that only degraded versions of animate, but not inanimate, objects had systematically shifted toward the classifier decision boundary as predicted. Our results support an asymmetry in the representation of animate and inanimate object categories in the human brain.

Transcranial Stimulation of the Orbitofrontal Cortex Affects Decisions about Magnocellular Optimized Stimuli

Visual categorization plays an important role in fast and efficient information processing; still the neuronal basis of fast categorization has not been established yet. There are two main hypotheses known; both agree that primary, global impressions are based on the information acquired through the magnocellular pathway (MC). It is unclear whether this information is available through the MC that provides information (also) for the ventral pathway or through top-down mechanisms by connections between the dorsal pathway and the ventral pathway via the frontal cortex. To clarify this, a categorization task was performed by 48 subjects; they had to make decisions about objects' sizes. We created stimuli specific to the magno- and parvocellular pathway (PC) on the basis of their spatial frequency content. Transcranial direct-current stimulation was used to assess the role of frontal areas, a target of the MC. Stimulation did not bias the accuracy of decisions when stimuli optimized for the PC were used. In the case of stimuli optimized for the MC, anodal stimulation improved the subjects' accuracy in the behavioral test, while cathodal stimulation impaired accuracy. Our results support the hypothesis that fast visual categorization processes rely on top-down mechanisms that promote fast predictions through coarse information carried by MC via the orbitofrontal cortex.

The DIsgust-RelaTed-Images (DIRTI) database: Validation of a novel standardized set of disgust pictures

Selecting appropriate stimuli is a major challenge of affective research. Although several standardized databases for affective pictures exist, none of them focus on discrete emotions such as disgust. Validated pictures inducing discrete emotions are still limited, and this presents a problem for researchers interested in studying different facets of disgust. In this paper, we introduce the DIsgust-RelaTed-Images (DIRTI) picture set. The set consists of 240 disgust-inducing pictures divided into six categories (food, animals, body products, injuries/infections, death, and hygiene). Additionally, we included 60 matched neutral pictures (10 per category). All pictures were rated by 200 participants on nine-point rating scales measuring disgust, fear, valence, and arousal. The present validation study covered a wide age range (18-75 years) with a balanced number of participants in each decade of life. For each picture, we provide separate ratings on the four scales for men and women. In addition to the original pictures, we also provide a luminance-matched version for experiments that require control of the physical properties of the pictures. The standardized DIRTI picture set allows researchers to chose from a wide set of disgust-inducing pictures and may enhance researchers' ability to draw comparisons between studies on disgust. (Download DIRTI picture set: http://dx.doi.org/10.5281/zenodo.167037).

The Time-Course of Ultrarapid Categorization: The Influence of Scene Congruency and Top-Down Processing

Although categorization can take place at different levels of abstraction, classic studies on semantic labeling identified the basic level, for example, dog, as entry point for categorization. Ultrarapid categorization tasks have contradicted these findings, indicating that participants are faster at detecting superordinate-level information, for example, animal, in a complex visual image. We argue that both seemingly contradictive findings can be reconciled within the framework of parallel distributed processing and its successor Leabra (Local, Error-driven and Associative, Biologically Realistic Algorithm). The current study aimed at verifying this prediction in an ultrarapid categorization task with a dynamically changing presentation time (PT) for each briefly presented object, followed by a perceptual mask. Furthermore, we manipulated two defining task variables: level of categorization (basic vs. superordinate categorization) and object presentation mode (object-in-isolation vs. object-in-context). In contradiction with previous ultrarapid categorization research, focusing on reaction time, we used accuracy as our main dependent variable. Results indicated a consistent superordinate processing advantage, coinciding with an overall improvement in performance with longer PT and a significantly more accurate detection of objects in isolation, compared with objects in context, at lower stimulus PT. This contextual disadvantage disappeared when PT increased, indicating that figure-ground separation with recurrent processing is vital for meaningful contextual processing to occur.

How semantic category modulates preschool children's visual memory

The dynamic interplay between perception and memory has been explored in preschool children by presenting filtered stimuli regarding animals and artifacts. The identification of filtered images was markedly influenced by both prior exposure and the semantic nature of the stimuli. The identification of animals required less physical information than artifacts did. Our results corroborate the notion that the human attention system evolves to reliably develop definite category-specific selection criteria by which living entities are monitored in different ways.

Attention in natural scenes: contrast affects rapid visual processing and fixations alike

For natural scenes, attention is frequently quantified either by performance during rapid presentation or by gaze allocation during prolonged viewing. Both paradigms operate on different time scales, and tap into covert and overt attention, respectively. To compare these, we ask some observers to detect targets (animals/vehicles) in rapid sequences, and others to freely view the same target images for 3 s, while their gaze is tracked. In some stimuli, the target's contrast is modified (increased/decreased) and its background modified either in the same or in the opposite way. We find that increasing target contrast relative to the background increases fixations and detection alike, whereas decreasing target contrast and simultaneously increasing background contrast has little effect. Contrast increase for the whole image (target + background) improves detection, decrease worsens detection, whereas fixation probability remains unaffected by whole-image modifications. Object-unrelated local increase or decrease of contrast attracts gaze, but less than actual objects, supporting a precedence of objects over low-level features. Detection and fixation probability are correlated: the more likely a target is detected in one paradigm, the more likely it is fixated in the other. Hence, the link between overt and covert attention, which has been established in simple stimuli, transfers to more naturalistic scenarios.

Object recognition in congruent and incongruent natural scenes: a life-span study

Efficient processing of our complex visual environment is essential and many daily visual tasks rely on accurate and fast object recognition. It is therefore important to evaluate how object recognition performance evolves during the course of adulthood. Surprisingly, this ability has not yet been investigated in the aged population, although several neuroimaging studies have reported altered activity in high-level visual ventral regions when elderly subjects process natural stimuli. In the present study, color photographs of various objects embedded in contextual scenes were used to assess object categorization performance in 97 participants aged from 20 to 91. Objects were either animals or pieces of furniture, embedded in either congruent or incongruent contexts. In every age group, subjects showed reduced categorization performance, both in terms of accuracy and speed, when objects were seen in incongruent vs. congruent contexts. In subjects over 60 years old, object categorization was greatly slowed down when compared to young and middle-aged subjects. Moreover, subjects over 75 years old evidenced a significant decrease in categorization accuracy when objects were seen in incongruent contexts. This indicates that incongruence of the scene may be particularly disturbing in late adulthood, therefore impairing object recognition. Our results suggest that daily visual processing of complex natural environments may be less efficient with age, which might impact performance in everyday visual tasks.

Informatics in radiology: what can you see in a single glance and how might this guide visual search in medical images?

Diagnostic accuracy for radiologists is above that expected by chance when they are exposed to a chest radiograph for only one-fifth of a second, a period too brief for more than a single voluntary eye movement. How do radiologists glean information from a first glance at an image? It is thought that this expert impression of the gestalt of an image is related to the everyday, immediate visual understanding of the gist of a scene. Several high-speed mechanisms guide our search of complex images. Guidance by basic features (such as color) requires no learning, whereas guidance by complex scene properties is learned. It is probable that both hardwired guidance by basic features and learned guidance by scene structure become part of radiologists' expertise. Search in scenes may be best explained by a two-pathway model: Object recognition is performed via a selective pathway in which candidate targets must be individually selected for recognition. A second, nonselective pathway extracts information from global or statistical information without selecting specific objects. An appreciation of the role of nonselective processing may be particularly useful for understanding what separates novice from expert radiologists and could help establish new methods of physician training based on medical image perception.

Similarity relations in visual search predict rapid visual categorization

How do we perform rapid visual categorization?It is widely thought that categorization involves evaluating the similarity of an object to other category items, but the underlying features and similarity relations remain unknown. Here, we hypothesized that categorization performance is based on perceived similarity relations between items within and outside the category. To this end, we measured the categorization performance of human subjects on three diverse visual categories (animals, vehicles, and tools) and across three hierarchical levels (superordinate, basic, and subordinate levels among animals). For the same subjects, we measured their perceived pair-wise similarities between objects using a visual search task. Regardless of category and hierarchical level, we found that the time taken to categorize an object could be predicted using its similarity to members within and outside its category. We were able to account for several classic categorization phenomena, such as (a) the longer times required to reject category membership; (b) the longer times to categorize atypical objects; and (c) differences in performance across tasks and across hierarchical levels. These categorization times were also accounted for by a model that extracts coarse structure from an image. The striking agreement observed between categorization and visual search suggests that these two disparate tasks depend on a shared coarse object representation.

Scene perception in age-related macular degeneration: the effect of contrast

PURPOSE

To investigate the effect of contrast on scene perception in people with age-related macular degeneration (AMD) and to examine the relationship between task performance and macular function.

METHODS

Nineteen patients with AMD and visual acuity below 20/50 were compared with 16 normally sighted, age-matched controls. Complete ophthalmologic examination (visual acuity, intraocular pressure measurement, and funduscopy) was performed in both patients and controls. In addition, Pelli-Robson contrast sensitivity, fluorescein angiography, and visual field size were assessed in the AMD study patients. The stimuli were photographs of natural scenes containing or lacking an animal (the target). For each scene, the contrast of the original photograph was divided by 2, 4, and 8 to yield versions with a residual contrast of 50, 25, and 12.5%, respectively. The four levels of contrast were presented randomly and participants were asked to press a key when they saw an animal.

RESULTS

AMD patients exhibited a larger drop in target detection performance with the decrease in contrast than controls. We found a correlation between visual acuity and performance when the contrast was reduced to 50, 25, and 12.5% of the original value but not in the normal contrast condition. There were no correlations between letter contrast sensitivity, visual field lesion size, and performance.

CONCLUSIONS

Our results suggest that optimal, stable contrast conditions would facilitate object recognition in everyday life for people with AMD.

The characteristics and limits of rapid visual categorization

Visual categorization appears both effortless and virtually instantaneous. The study by Thorpe et al. (1996) was the first to estimate the processing time necessary to perform fast visual categorization of animals in briefly flashed (20 ms) natural photographs. They observed a large differential EEG activity between target and distracter correct trials that developed from 150 ms after stimulus onset, a value that was later shown to be even shorter in monkeys! With such strong processing time constraints, it was difficult to escape the conclusion that rapid visual categorization was relying on massively parallel, essentially feed-forward processing of visual information. Since 1996, we have conducted a large number of studies to determine the characteristics and limits of fast visual categorization. The present chapter will review some of the main results obtained. I will argue that rapid object categorizations in natural scenes can be done without focused attention and are most likely based on coarse and unconscious visual representations activated with the first available (magnocellular) visual information. Fast visual processing proved efficient for the categorization of large superordinate object or scene categories, but shows its limits when more detailed basic representations are required. The representations for basic objects (dogs, cars) or scenes (mountain or sea landscapes) need additional processing time to be activated. This finding is at odds with the widely accepted idea that such basic representations are at the entry level of the system. Interestingly, focused attention is still not required to perform these time consuming basic categorizations. Finally we will show that object and context processing can interact very early in an ascending wave of visual information processing. We will discuss how such data could result from our experience with a highly structured and predictable surrounding world that shaped neuronal visual selectivity.

Chinese characters elicit face-like N170 inversion effects

Recognition of both faces and Chinese characters is commonly believed to rely on configural information. While faces typically exhibit behavioral and N170 inversion effects that differ from non-face stimuli (Rossion, Joyce, Cottrell, & Tarr, 2003), the current study examined whether a similar reliance on configural processing may result in similar inversion effects for faces and Chinese characters. Participants were engaged in an orientation judgment task (Experiment 1) and a one-back identity matching task (Experiment 2). Across two experiments, the N170 was delayed and enhanced in magnitude for upside-down faces and compound Chinese characters, compared to upright stimuli. The inversion effects for these two stimulus categories were bilateral for latency and right-lateralized for amplitudes. For simple Chinese characters, only the latency inversion effects were significant. Moreover, the size of the right-hemisphere inversion effects in N170 amplitude was larger for faces than Chinese characters. These findings show the N170 inversion effects from non-face stimuli closely parallel effects seen with faces. Face-like N170 inversion effects elicited by Chinese compound characters were attributed to the difficulty of part-whole integration as well as the disrupted regularity in relational information due to inversion. Hemispheric difference in Chinese character processing is also discussed.

Semantic category effects modulate visual priming in neglect patients

Previous studies indicate that extinguished stimuli can still be unconsciously processed, leading to implicit priming effects. Here we investigated whether these implicit effects might be modulated by the semantic nature of the stimuli. Five neglect patients and ten controls performed an identification task of items belonging to living and non-living categories. In the study phase photographs of animals and artifacts were presented either to the left visual field (LVF) or to the right visual field (RVF). In the identification phase, each stimulus was displayed centrally and was revealed in a sequence of frames where the item was represented by an increasingly less and less filtered image up to a complete version. The results showed that lateralized stimuli differentially affected controls' and neglect patients' memory retrieval. In controls memory traces from the study phase served as efficient primes, thereby reducing the amount of information necessary for the identification of both stimulus categories. Moreover, hemispheric differences emerged with an advantage of the RVF/left hemisphere for artifact items, while no difference was found for living things. Neglect patients showed a priming effect for artifact items presented either to the RVF/left hemisphere or LVF/right hemisphere, as well as for living items presented to the RVF/left hemisphere, but not for living items presented to the LVF/right hemisphere. The priming effect observed for extinguished artifacts is consistent with the evidence of the existence of a specific mechanism destined to analyze, in an automatic and implicit fashion, motor-relevant information of manipulable objects and tools, which are important for identification process. Results are discussed in relation to current models of organization of conceptual knowledge within the framework of different processes performed by the two hemispheres.

Humans and monkeys share visual representations

Conceptual abilities in animals have been shown at several levels of abstraction, but it is unclear whether the analogy with humans results from convergent evolution or from shared brain mechanisms inherited from a common origin. Macaque monkeys can access "non-similarity-based concepts," such as when sorting pictures containing a superordinate target category (animal, tree, etc.) among other scenes. However, such performances could result from low-level visual processing based on learned regularities of the photographs, such as for scene categorization by artificial systems. By using pictures of man-made objects or animals embedded in man-made or natural contexts, the present study clearly establishes that macaque monkeys based their categorical decision on the presence of the animal targets regardless of the scene backgrounds. However, as is found with humans, monkeys performed better with categorically congruent object/context associations, especially when small object sizes favored background information. The accuracy improvements and the response-speed gains attributable to superordinate category congruency in monkeys were strikingly similar to those of human subjects tested with the same task and stimuli. These results suggest analogous processing of visual information during the activation of abstract representations in both humans and monkeys; they imply a large overlap between superordinate visual representations in humans and macaques as well as the implicit use of experienced associations between object and context.

Ultra-rapid categorization of fourier-spectrum equalized natural images: macaques and humans perform similarly

BACKGROUND

Comparative studies of cognitive processes find similarities between humans and apes but also monkeys. Even high-level processes, like the ability to categorize classes of object from any natural scene under ultra-rapid time constraints, seem to be present in rhesus macaque monkeys (despite a smaller brain and the lack of language and a cultural background). An interesting and still open question concerns the degree to which the same images are treated with the same efficacy by humans and monkeys when a low level cue, the spatial frequency content, is controlled.

METHODOLOGY/PRINCIPAL FINDINGS

We used a set of natural images equalized in Fourier spectrum and asked whether it is still possible to categorize them as containing an animal and at what speed. One rhesus macaque monkey performed a forced-choice saccadic task with a good accuracy (67.5% and 76% for new and familiar images respectively) although performance was lower than with non-equalized images. Importantly, the minimum reaction time was still very fast (100 ms). We compared the performances of human subjects with the same setup and the same set of (new) images. Overall mean performance of humans was also lower than with original images (64% correct) but the minimum reaction time was still short (140 ms).

CONCLUSION

Performances on individual images (% correct but not reaction times) for both humans and the monkey were significantly correlated suggesting that both species use similar features to perform the task. A similar advantage for full-face images was seen for both species. The results also suggest that local low spatial frequency information could be important, a finding that fits the theory that fast categorization relies on a rapid feedforward magnocellular signal.

Key visual features for rapid categorization of animals in natural scenes

In speeded categorization tasks, decisions could be based on diagnostic target features or they may need the activation of complete representations of the object. Depending on task requirements, the priming of feature detectors through top-down expectation might lower the threshold of selective units or speed up the rate of information accumulation. In the present paper, 40 subjects performed a rapid go/no-go animal/non-animal categorization task with 400 briefly flashed natural scenes to study how performance depends on physical scene characteristics, target configuration, and the presence or absence of diagnostic animal features. Performance was evaluated both in terms of accuracy and speed and d' curves were plotted as a function of reaction time (RT). Such d' curves give an estimation of the processing dynamics for studied features and characteristics over the entire subject population. Global image characteristics such as color and brightness do not critically influence categorization speed, although they slightly influence accuracy. Global critical factors include the presence of a canonical animal posture and animal/background size ratio suggesting the role of coarse global form. Performance was best for both accuracy and speed, when the animal was in a typical posture and when it occupied about 20-30% of the image. The presence of diagnostic animal features was another critical factor. Performance was significantly impaired both in accuracy (drop 3.3-7.5%) and speed (median RT increase 7-16 ms) when diagnostic animal parts (eyes, mouth, and limbs) were missing. Such animal features were shown to influence performance very early when only 15-25% of the response had been produced. In agreement with other experimental and modeling studies, our results support fast diagnostic recognition of animals based on key intermediate features and priming based on the subject's expertise.