Recognition of natural scenes from global properties: seeing the forest without representing the trees

What makes icons appealing? The role of processing fluency in predicting icon appeal in different task contexts

Although icons appear on almost all interfaces, there is a paucity of research examining the determinants of icon appeal. The experiments reported here examined the icon characteristics determining appeal and the extent to which processing fluency - the subjective ease with which individuals process information - was used as a heuristic to guide appeal evaluations. Participants searched for, and identified, icons in displays. The initial appeal of icons was held constant while ease of processing was manipulated by systematically varying the complexity and familiarity of the icons presented and the type of task participants were asked to carry out. Processing fluency reliably influenced users' appeal ratings and appeared to be based on users' unconscious awareness of the ease with which they carried out experimental tasks.

Contour junctions underlie neural representations of scene categories in high-level human visual cortex

Humans efficiently grasp complex visual environments, making highly consistent judgments of entry-level category despite their high variability in visual appearance. How does the human brain arrive at the invariant neural representations underlying categorization of real-world environments? We here show that the neural representation of visual environments in scene-selective human visual cortex relies on statistics of contour junctions, which provide cues for the three-dimensional arrangement of surfaces in a scene. We manipulated line drawings of real-world environments such that statistics of contour orientations or junctions were disrupted. Manipulated and intact line drawings were presented to participants in an fMRI experiment. Scene categories were decoded from neural activity patterns in the parahippocampal place area (PPA), the occipital place area (OPA) and other visual brain regions. Disruption of junctions but not orientations led to a drastic decrease in decoding accuracy in the PPA and OPA, indicating the reliance of these areas on intact junction statistics. Accuracy of decoding from early visual cortex, on the other hand, was unaffected by either image manipulation. We further show that the correlation of error patterns between decoding from the scene-selective brain areas and behavioral experiments is contingent on intact contour junctions. Finally, a searchlight analysis exposes the reliance of visually active brain regions on different sets of contour properties. Statistics of contour length and curvature dominate neural representations of scene categories in early visual areas and contour junctions in high-level scene-selective brain regions.

Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study

Perceiving the visual world around us requires the brain to represent the features of stimuli and to categorize the stimulus based on these features. Incorrect categorization can result either from errors in visual representation or from errors in processes that lead to categorical choice. To understand the temporal relationship between the neural signatures of such systematic errors, we recorded whole-scalp magnetoencephalography (MEG) data from human subjects performing a rapid-scene categorization task. We built scene category decoders based on (1) spatiotemporally resolved neural activity, (2) spatial envelope (SpEn) image features, and (3) behavioral responses. Using confusion matrices, we tracked how well the pattern of errors from neural decoders could be explained by SpEn decoders and behavioral errors, over time and across cortical areas. Across the visual cortex and the medial temporal lobe, we found that both SpEn and behavioral errors explained unique variance in the errors of neural decoders. Critically, these effects were nearly simultaneous, and most prominent between 100 and 250ms after stimulus onset. Thus, during rapid-scene categorization, neural processes that ultimately result in behavioral categorization are simultaneous and co-localized with neural processes underlying visual information representation.

Memory and learning with rapid audiovisual sequences

We examined short-term memory for sequences of visual stimuli embedded in varying multisensory contexts. In two experiments, subjects judged the structure of the visual sequences while disregarding concurrent, but task-irrelevant auditory sequences. Stimuli were eight-item sequences in which varying luminances and frequencies were presented concurrently and rapidly (at 8 Hz). Subjects judged whether the final four items in a visual sequence identically replicated the first four items. Luminances and frequencies in each sequence were either perceptually correlated (Congruent) or were unrelated to one another (Incongruent). Experiment 1 showed that, despite encouragement to ignore the auditory stream, subjects' categorization of visual sequences was strongly influenced by the accompanying auditory sequences. Moreover, this influence tracked the similarity between a stimulus's separate audio and visual sequences, demonstrating that task-irrelevant auditory sequences underwent a considerable degree of processing. Using a variant of Hebb's repetition design, Experiment 2 compared musically trained subjects and subjects who had little or no musical training on the same task as used in Experiment 1. Test sequences included some that intermittently and randomly recurred, which produced better performance than sequences that were generated anew for each trial. The auditory component of a recurring audiovisual sequence influenced musically trained subjects more than it did other subjects. This result demonstrates that stimulus-selective, task-irrelevant learning of sequences can occur even when such learning is an incidental by-product of the task being performed.

Perceived egocentric distance sensitivity and invariance across scene-selective cortex

Behavioral studies in many species and studies in robotics have demonstrated two sources of information critical for visually-guided navigation: sense (left-right) information and egocentric distance (proximal-distal) information. A recent fMRI study found sensitivity to sense information in two scene-selective cortical regions, the retrosplenial complex (RSC) and the occipital place area (OPA), consistent with hypotheses that these regions play a role in human navigation. Surprisingly, however, another scene-selective region, the parahippocampal place area (PPA), was not sensitive to sense information, challenging hypotheses that this region is directly involved in navigation. Here we examined how these regions encode egocentric distance information (e.g., a house seen from close up versus far away), another type of information crucial for navigation. Using fMRI adaptation and a regions-of-interest analysis approach in human adults, we found sensitivity to egocentric distance information in RSC and OPA, while PPA was not sensitive to such information. These findings further support that RSC and OPA are directly involved in navigation, while PPA is not, consistent with the hypothesis that scenes may be processed by distinct systems guiding navigation and recognition.

Lateral and Medial Ventral Occipitotemporal Regions Interact During the Recognition of Images Revealed from Noise

Several studies suggest different functional roles for the medial and the lateral sections of the ventral visual cortex in object recognition. Texture and surface information is processed in medial sections, while shape information is processed in lateral sections. This begs the question whether and how these functionally specialized sections interact with each other and with early visual cortex to facilitate object recognition. In the current research, we set out to answer this question. In an fMRI study, 13 subjects viewed and recognized images of objects and animals that were gradually revealed from noise while their brains were being scanned. We applied dynamic causal modeling (DCM)-a method to characterize network interactions-to determine the modulatory effect of object recognition on a network comprising the primary visual cortex (V1), the lingual gyrus (LG) in medial ventral cortex and the lateral occipital cortex (LO). We found that object recognition modulated the bilateral connectivity between LG and LO. Moreover, the feed-forward connectivity from V1 to LG and LO was modulated, while there was no evidence for feedback from these regions to V1 during object recognition. In particular, the interaction between medial and lateral areas supports a framework in which visual recognition of objects is achieved by networked regions that integrate information on image statistics, scene content and shape-rather than by a single categorically specialized region-within the ventral visual cortex.

Concepts, contents, and consciousness

In his paper 'Are we ever aware of concepts? A critical question for the Global Neuronal Workspace, Integrated Information, and Attended Intermediate-Level Representation theories of consciousness' (2015, this journal), Kemmerer defends a conservative account of consciousness, according to which concepts and thoughts do not characterize the contents of consciousness, and then uses that account to argue against both the Global Neuronal Workspace theory of consciousness and Integrated Information Theory of Consciousness, and as a point in favour of Prinz's Attended Intermediate-level Representations theory. We argue that there are a number of respects in which the contrast between conservative and liberal conceptions of the admissible contents of consciousness is more complex than Kemmerer's discussion suggests. We then consider Kemmerer's case for conservatism, arguing that it lumbers liberals with commitments that they need not - and in our view should not - endorse. We also argue that Kemmerer's attempt to use his case for conservatism against the Global Neuronal Workspace and Integrated Information theories of consciousness on the one hand, and as a point in favour of Prinz's Attended Intermediate Representations theory on the other hand, is problematic. Finally, we consider Kemmerer's overall strategy of using an account of the admissible contents of consciousness to evaluate theories of consciousness, and suggest that here too there are complications that Kemmerer's discussion overlooks.

Effects of presentation duration on measures of complexity in affective environmental scenes and representational paintings

Complexity constitutes an integral part of humans' environment and is inherent to information processing. However, little is known about the dynamics of visual complexity perception of affective environmental scenes (IAPS pictures) and artworks, such as affective representational paintings. In three experiments, we studied the time course of visual complexity perception by varying presentation duration and comparing subjective ratings with objective measures of complexity. In Experiment 1, 60 females rated 96 IAPS pictures, presented either for 1, 5, or 25s, for familiarity, complexity, pleasantness and arousal. In Experiment 2, another 60 females rated 96 representational paintings. Mean ratings of complexity and pleasantness changed according to presentation duration in a similar vein in both experiments, suggesting an inverted U-shape. No common pattern of results was observed for arousal and familiarity ratings across the two picture sets. The correlations between subjective and objective measures of complexity increased with longer exposure durations for IAPS pictures, but results were more ambiguous for paintings. Experiment 3 explored the time course of the multidimensionality of visual complexity perception. Another 109 females rated the number of objects, their disorganization and the differentiation between a figure-ground vs. complex scene composition of pictures presented for 1 and 5s. The multidimensionality of visual complexity only clearly emerged in the 5-s condition. In both picture sets, the strength of the correlations with objective measures depended on the type of subdimension of complexity and was less affected by presentation duration than correlations with general complexity in Experiments 1 and 2. These results have clear implications for perceptual and cognitive theories, especially for those of esthetic experiences, in which the dynamical changes of complexity perception need to be integrated.

Does object view influence the scene consistency effect?

Traditional research on the scene consistency effect only used clearly recognizable object stimuli to show mutually interactive context effects for both the object and background components on scene perception (Davenport & Potter in Psychological Science, 15, 559-564, 2004). However, in real environments, objects are viewed from multiple viewpoints, including an accidental, hard-to-recognize one. When the observers named target objects in scenes (Experiments 1a and 1b, object recognition task), we replicated the scene consistency effect (i.e., there was higher accuracy for the objects with consistent backgrounds). However, there was a significant interaction effect between consistency and object viewpoint, which indicated that the scene consistency effect was more important for identifying objects in the accidental view condition than in the canonical view condition. Therefore, the object recognition system may rely more on the scene context when the object is difficult to recognize. In Experiment 2, the observers identified the background (background recognition task) while the scene consistency and object views were manipulated. The results showed that object viewpoint had no effect, while the scene consistency effect was observed. More specifically, the canonical and accidental views both equally provided contextual information for scene perception. These findings suggested that the mechanism for conscious recognition of objects could be dissociated from the mechanism for visual analysis of object images that were part of a scene. The "context" that the object images provided may have been derived from its view-invariant, relatively low-level visual features (e.g., color), rather than its semantic information.

Associative Processing Is Inherent in Scene Perception

How are complex visual entities such as scenes represented in the human brain? More concretely, along what visual and semantic dimensions are scenes encoded in memory? One hypothesis is that global spatial properties provide a basis for categorizing the neural response patterns arising from scenes. In contrast, non-spatial properties, such as single objects, also account for variance in neural responses. The list of critical scene dimensions has continued to grow—sometimes in a contradictory manner—coming to encompass properties such as geometric layout, big/small, crowded/sparse, and three-dimensionality. We demonstrate that these dimensions may be better understood within the more general framework of associative properties. That is, across both the perceptual and semantic domains, features of scene representations are related to one another through learned associations. Critically, the components of such associations are consistent with the dimensions that are typically invoked to account for scene understanding and its neural bases. Using fMRI, we show that non-scene stimuli displaying novel associations across identities or locations recruit putatively scene-selective regions of the human brain (the parahippocampal/lingual region, the retrosplenial complex, and the transverse occipital sulcus/occipital place area). Moreover, we find that the voxel-wise neural patterns arising from these associations are significantly correlated with the neural patterns arising from everyday scenes providing critical evidence whether the same encoding principals underlie both types of processing. These neuroimaging results provide evidence for the hypothesis that the neural representation of scenes is better understood within the broader theoretical framework of associative processing. In addition, the results demonstrate a division of labor that arises across scene-selective regions when processing associations and scenes providing better understanding of the functional roles of each region within the cortical network that mediates scene processing.

Selective Neural Synchrony Suppression as a Forward Gatekeeper to Piecemeal Conscious Perception

The emergence of conscious visual perception is assumed to ignite late (∼250 ms) gamma-band oscillations shortly after an initial (∼100 ms) forward sweep of neural sensory (nonconscious) information. However, this neural evidence is not utterly congruent with rich behavioral data which rather point to piecemeal (i.e., graded) perceptual processing. To address the unexplored neural mechanisms of piecemeal ignition of conscious perception, hierarchical script sensitivity of the putative visual word form area (VWFA) was exploited to signal null (i.e., sensory), partial (i.e., letter-level), and full (i.e., word-level) conscious perception. Two magnetoencephalography experiments were conducted in which healthy human participants viewed masked words (Experiment I: active task, Dutch words; Experiment II: passive task, Hebrew words) while high-frequency (broadband gamma) brain activity was measured. Findings revealed that piecemeal conscious perception did not ignite a linear piecemeal increase in oscillations. Instead, whereas late (∼250 ms) gamma-band oscillations signaled full conscious perception (i.e., word-level), partial conscious perception (i.e., letter-level) was signaled via the inhibition of the early (∼100 ms) forward sweep. This inhibition regulates the downstream broadcast to filter out irrelevant (i.e., masks) information. The findings thus highlight a local (VWFA) gatekeeping mechanism for conscious perception, operating by filtering out and in selective percepts.

Natural Scene Recognition with Increasing Time-On-Task: The Role of Typicality and Global Image Properties

Human observers can recognize natural images very effectively. Yet, in the literature there is a debate about the extent to which the recognition of natural images requires controlled attentional processing. In the present study we address this topic by testing whether natural scene recognition is affected by mental fatigue. Mental fatigue is known to particularly compromise high-level, controlled attentional processing of local features. Effortless, automatic processing of more global features of an image stays relatively intact, however. We conducted a natural image categorization experiment ( N = 20) in which mental fatigue was induced by time-on-task (ToT). Stimuli were images from 5 natural scene categories. Semantic typicality (high or low) and the magnitude of 7 global image properties were determined for each image in separate rating experiments. Significant performance effects of typicality and global properties on scene recognition were found, but, despite a general decline in performance, these effects remained unchanged with increasing ToT. The findings support the importance of the global property processing in natural scene recognition and suggest that this process is insensitive to mental fatigue.

Visual, haptic and bimodal scene perception: evidence for a unitary representation

Participants studied seven meaningful scene-regions bordered by removable boundaries (30s each). In Experiment 1 (N = 80) participants used visual or haptic exploration and then minutes later, reconstructed boundary position using the same or the alternate modality. Participants in all groups shifted boundary placement outward (boundary extension), but visual study yielded the greater error. Critically, this modality-specific difference in boundary extension transferred without cost in the cross-modal conditions, suggesting a functionally unitary scene representation. In Experiment 2 (N = 20), bimodal study led to boundary extension that did not differ from haptic exploration alone, suggesting that bimodal spatial memory was constrained by the more "conservative" haptic modality. In Experiment 3 (N = 20), as in picture studies, boundary memory was tested 30s after viewing each scene-region and as with pictures, boundary extension still occurred. Results suggest that scene representation is organized around an amodal spatial core that organizes bottom-up information from multiple modalities in combination with top-down expectations about the surrounding world.

The role of memory for visual search in scenes

Many daily activities involve looking for something. The ease with which these searches are performed often allows one to forget that searching represents complex interactions between visual attention and memory. Although a clear understanding exists of how search efficiency will be influenced by visual features of targets and their surrounding distractors or by the number of items in the display, the role of memory in search is less well understood. Contextual cueing studies have shown that implicit memory for repeated item configurations can facilitate search in artificial displays. When searching more naturalistic environments, other forms of memory come into play. For instance, semantic memory provides useful information about which objects are typically found where within a scene, and episodic scene memory provides information about where a particular object was seen the last time a particular scene was viewed. In this paper, we will review work on these topics, with special emphasis on the role of memory in guiding search in organized, real-world scenes.

Objects influence the shape of remembered views: examining global and local aspects of boundary extension

Boundary extension is a constructive memory error for views of scenes in which viewx tendto be remembered as more spatially expansive than they appeared. In seven experiments xwe examinedwhether local differences in boundary extension within views might exist by presenting par ticipants ith overhead views of single, elongated objects or object pairs on textured ground surfaces i:n whichobjects were oriented either vertically or horizontally. Memory for views' spatial expanse xwas testedwith either a view-recognition test or a border-adjustment test in which participants could ad just thespatial expanse of a test view using the computer's mouse. The border-adjustment test was uused tossess local boundary extension (primarily); the view-recognition test was used to assess paparticipants'emories for the overall spatial expanse of views (ie global boundary extension). Across exexperiments,n the border-adjustment tests specifically, participants showed more spatial expanse along the obobject'songer axis, in some cases restricting the view along the object's shorter axis. In addition, the rerecognition-t data revealed greater boundary extension for views with vertically oriented objects than for x.iviewsth horizontally oriented objects. Taken together, the results suggest that objects in scene x iviews canfect both local and global aspects of memory for spatial expanse of scene views.

Applying artificial vision models to human scene understanding

How do we understand the complex patterns of neural responses that underlie scene understanding? Studies of the network of brain regions held to be scene-selective—the parahippocampal/lingual region (PPA), the retrosplenial complex (RSC), and the occipital place area (TOS)—have typically focused on single visual dimensions (e.g., size), rather than the high-dimensional feature space in which scenes are likely to be neurally represented. Here we leverage well-specified artificial vision systems to explicate a more complex understanding of how scenes are encoded in this functional network. We correlated similarity matrices within three different scene-spaces arising from: (1) BOLD activity in scene-selective brain regions; (2) behavioral measured judgments of visually-perceived scene similarity; and (3) several different computer vision models. These correlations revealed: (1) models that relied on mid- and high-level scene attributes showed the highest correlations with the patterns of neural activity within the scene-selective network; (2) NEIL and SUN—the models that best accounted for the patterns obtained from PPA and TOS—were different from the GIST model that best accounted for the pattern obtained from RSC; (3) The best performing models outperformed behaviorally-measured judgments of scene similarity in accounting for neural data. One computer vision method—NEIL (“Never-Ending-Image-Learner”), which incorporates visual features learned as statistical regularities across web-scale numbers of scenes—showed significant correlations with neural activity in all three scene-selective regions and was one of the two models best able to account for variance in the PPA and TOS. We suggest that these results are a promising first step in explicating more fine-grained models of neural scene understanding, including developing a clearer picture of the division of labor among the components of the functional scene-selective brain network.

Effective and efficient subjective testing of texture similarity metrics

The development and testing of objective texture similarity metrics that agree with human judgments of texture similarity require, in general, extensive subjective tests. The effectiveness and efficiency of such tests depend on a careful analysis of the abilities of human perception and the application requirements. The focus of this paper is on defining performance requirements and testing procedures for objective texture similarity metrics. We identify three operating domains for evaluating the performance of a similarity metric: the ability to retrieve "identical" textures; the top of the similarity scale, where a monotonic relationship between metric values and subjective scores is desired; and the ability to distinguish between perceptually similar and dissimilar textures. Each domain has different performance goals and requires different testing procedures. For the third domain, we propose ViSiProG, a new Visual Similarity by Progressive Grouping procedure for conducting subjective experiments that organizes a texture database into clusters of visually similar images. The grouping is based on visual blending and greatly simplifies labeling image pairs as similar or dissimilar. ViSiProG collects subjective data in an efficient and effective manner, so that a relatively large database of textures can be accommodated. Experimental results and comparisons with structural texture similarity metrics demonstrate both the effectiveness of the proposed subjective testing procedure and the performance of the metrics.

Scene categorization in Alzheimer's disease: a saccadic choice task

AIMS

We investigated the performance in scene categorization of patients with Alzheimer's disease (AD) using a saccadic choice task.

METHOD

24 patients with mild AD, 28 age-matched controls and 26 young people participated in the study. The participants were presented pairs of coloured photographs and were asked to make a saccadic eye movement to the picture corresponding to the target scene (natural vs. urban, indoor vs. outdoor).

RESULTS

The patients' performance did not differ from chance for natural scenes. Differences between young and older controls and patients with AD were found in accuracy but not saccadic latency.

CONCLUSIONS

The results are interpreted in terms of cerebral reorganization in the prefrontal and temporo-occipital cortex of patients with AD, but also in terms of impaired processing of visual global properties of scenes.

The Processing Speed of Scene Categorization at Multiple Levels of Description: The Superordinate Advantage Revisited

Recent studies have sought to determine which levels of categories are processed first in visual scene categorization and have shown that the natural and man-made superordinate-level categories are understood faster than are basic-level categories. The current study examined the robustness of the superordinate-level advantage in a visual scene categorization task. A go/no-go categorization task was evaluated with response time distribution analysis using an ex-Gaussian template. A visual scene was categorized as either superordinate or basic level, and two basic-level categories forming a superordinate category were judged as either similar or dissimilar to each other. First, outdoor/ indoor groups and natural/man-made were used as superordinate categories to investigate whether the advantage could be generalized beyond the natural/man-made boundary. Second, a set of images forming a superordinate category was manipulated. We predicted that decreasing image set similarity within the superordinate-level category would work against the speed advantage. We found that basic-level categorization was faster than outdoor/indoor categorization when the outdoor category comprised dissimilar basic-level categories. Our results indicate that the superordinate-level advantage in visual scene categorization is labile across different categories and category structures.

The nature of expertise in fingerprint matching: experts can do a lot with a little

Expert decision making often seems impressive, even miraculous. People with genuine expertise in a particular domain can perform quickly and accurately, and with little information. In the series of experiments presented here, we manipulate the amount of “information” available to a group of experts whose job it is to identify the source of crime scene fingerprints. In Experiment 1, we reduced the amount of information available to experts by inverting fingerprint pairs and adding visual noise. There was no evidence for an inversion effect—experts were just as accurate for inverted prints as they were for upright prints—but expert performance with artificially noisy prints was impressive. In Experiment 2, we separated matching and nonmatching print pairs in time. Experts were conservative, but they were still able to discriminate pairs of fingerprints that were separated by five-seconds, even though the task was quite different from their everyday experience. In Experiment 3, we separated the print pairs further in time to test the long-term memory of experts compared to novices. Long-term recognition memory for experts and novices was the same, with both performing around chance. In Experiment 4, we presented pairs of fingerprints quickly to experts and novices in a matching task. Experts were more accurate than novices, particularly for similar nonmatching pairs, and experts were generally more accurate when they had more time. It is clear that experts can match prints accurately when there is reduced visual information, reduced opportunity for direct comparison, and reduced time to engage in deliberate reasoning. These findings suggest that non-analytic processing accounts for a substantial portion of the variance in expert fingerprint matching accuracy. Our conclusion is at odds with general wisdom in fingerprint identification practice and formal training, and at odds with the claims and explanations that are offered in court during expert testimony.

Visual categorization of natural movies by rats

Visual categorization of complex, natural stimuli has been studied for some time in human and nonhuman primates. Recent interest in the rodent as a model for visual perception, including higher-level functional specialization, leads to the question of how rodents would perform on a categorization task using natural stimuli. To answer this question, rats were trained in a two-alternative forced choice task to discriminate movies containing rats from movies containing other objects and from scrambled movies (ordinate-level categorization). Subsequently, transfer to novel, previously unseen stimuli was tested, followed by a series of control probes. The results show that the animals are capable of acquiring a decision rule by abstracting common features from natural movies to generalize categorization to new stimuli. Control probes demonstrate that they did not use single low-level features, such as motion energy or (local) luminance. Significant generalization was even present with stationary snapshots from untrained movies. The variability within and between training and test stimuli, the complexity of natural movies, and the control experiments and analyses all suggest that a more high-level rule based on more complex stimulus features than local luminance-based cues was used to classify the novel stimuli. In conclusion, natural stimuli can be used to probe ordinate-level categorization in rats.

Space representation in age-related macular degeneration

PURPOSE

To investigate the effect of age-related macular degeneration (AMD) on memory for spatial representations in realistic environments.

METHODS

Participants were 19 patients with AMD and 13 age-matched observers. In a short-term spatial memory task, observers were first presented with one view of a scene (the prime view), and their task was to change the viewpoint forward or backward to match the prime view. Memory performance was measured as the number of snapshots between the selected view and the prime view.

RESULTS

When selecting a match to the prime view, both people with AMD and those in the control group showed systematic biases toward the middle view of the range of snapshots. People with AMD exhibited a stronger middle bias after presentation of close and far prime views while navigating accurately after a middle prime view. No relation was found between visual acuity, visual field defect, or lesion size and the memory performance.

CONCLUSIONS

Memory tasks using indoor scenes can be accomplished when central vision is impoverished, as with AMD. Stronger center bias for a scene location suggests that people with AMD rely more on their memory of a canonical view.

Do simultaneously viewed objects influence scene recognition individually or as groups? Two perceptual studies

The ability to quickly categorize visual scenes is critical to daily life, allowing us to identify our whereabouts and to navigate from one place to another. Rapid scene categorization relies heavily on the kinds of objects scenes contain; for instance, studies have shown that recognition is less accurate for scenes to which incongruent objects have been added, an effect usually interpreted as evidence of objects' general capacity to activate semantic networks for scene categories they are statistically associated with. Essentially all real-world scenes contain multiple objects, however, and it is unclear whether scene recognition draws on the scene associations of individual objects or of object groups. To test the hypothesis that scene recognition is steered, at least in part, by associations between object groups and scene categories, we asked observers to categorize briefly-viewed scenes appearing with object pairs that were semantically consistent or inconsistent with the scenes. In line with previous results, scenes were less accurately recognized when viewed with inconsistent versus consistent pairs. To understand whether this reflected individual or group-level object associations, we compared the impact of pairs composed of mutually related versus unrelated objects; i.e., pairs, which, as groups, had clear associations to particular scene categories versus those that did not. Although related and unrelated object pairs equally reduced scene recognition accuracy, unrelated pairs were consistently less capable of drawing erroneous scene judgments towards scene categories associated with their individual objects. This suggests that scene judgments were influenced by the scene associations of object groups, beyond the influence of individual objects. More generally, the fact that unrelated objects were as capable of degrading categorization accuracy as related objects, while less capable of generating specific alternative judgments, indicates that the process by which objects interfere with scene recognition is separate from the one through which they inform it.

The gist of the abnormal: above-chance medical decision making in the blink of an eye

Very fast extraction of global structural and statistical regularities allows us to access the "gist"--the basic meaning--of real-world images in as little as 20 ms. Gist processing is central to efficient assessment and orienting in complex environments. This ability is probably based on our extensive experience with the regularities of the natural world. If that is so, would experts develop an ability to extract the gist from the artificial stimuli (e.g., medical images) with which they have extensive visual experience? Anecdotally, experts report some ability to categorize images as normal or abnormal before actually finding an abnormality. We tested the reality of this perception in two expert populations: radiologists and cytologists. Observers viewed brief (250- to 2,000-ms) presentations of medical images. The presence of abnormality was randomized across trials. The task was to rate the abnormality of an image on a 0-100 analog scale and then to attempt to localize that abnormality on a subsequent screen showing only the outline of the image. Both groups of experts had above-chance performance for detecting subtle abnormalities at all stimulus durations (cytologists d' ≈ 1.2 and radiologists d' ≈ 1), whereas the nonexpert control groups did not differ from chance (d' ≈ 0.23, d' ≈ 0.25). Furthermore, the experts' ability to localize these abnormalities was at chance levels, suggesting that categorization was based on a global signal, and not on fortuitous attention to a localized target. It is possible that this global signal could be exploited to improve clinical performance.

The Impact of Density and Ratio on Object-Ensemble Representation in Human Anterior-Medial Ventral Visual Cortex

Behavioral research has demonstrated that observers can extract summary statistics from ensembles of multiple objects. We recently showed that a region of anterior-medial ventral visual cortex, overlapping largely with the scene-sensitive parahippocampal place area (PPA), participates in object-ensemble representation. Here we investigated the encoding of ensemble density in this brain region using fMRI-adaptation. In Experiment 1, we varied density by changing the spacing between objects and found no sensitivity in PPA to such density changes. Thus, density may not be encoded in PPA, possibly because object spacing is not perceived as an intrinsic ensemble property. In Experiment 2, we varied relative density by changing the ratio of 2 types of objects comprising an ensemble, and observed significant sensitivity in PPA to such ratio change. Although colorful ensembles were shown in Experiment 2, Experiment 3 demonstrated that sensitivity to object ratio change was not driven mainly by a change in the ratio of colors. Thus, while anterior-medial ventral visual cortex is insensitive to density (object spacing) changes, it does code relative density (object ratio) within an ensemble. Object-ensemble processing in this region may thus depend on high-level visual information, such as object ratio, rather than low-level information, such as spacing/spatial frequency.

Drawing skill is related to the efficiency of encoding object structure

Accurate drawing calls on many skills beyond simple motor coordination. A good internal representation of the target object's structure is necessary to capture its proportion and shape in the drawing. Here, we assess two aspects of the perception of object structure and relate them to participants' drawing accuracy. First, we assessed drawing accuracy by computing the geometrical dissimilarity of their drawing to the target object. We then used two tasks to evaluate the efficiency of encoding object structure. First, to examine the rate of temporal encoding, we varied presentation duration of a possible versus impossible test object in the fovea using two different test sizes (8° and 28°). More skilled participants were faster at encoding an object's structure, but this difference was not affected by image size. A control experiment showed that participants skilled in drawing did not have a general advantage that might have explained their faster processing for object structure. Second, to measure the critical image size for accurate classification in the periphery, we varied image size with possible versus impossible object tests centered at two different eccentricities (3° and 8°). More skilled participants were able to categorise object structure at smaller sizes, and this advantage did not change with eccentricity. A control experiment showed that the result could not be attributed to differences in visual acuity, leaving attentional resolution as a possible explanation. Overall, we conclude that drawing accuracy is related to faster encoding of object structure and better access to crowded details.

Infant Preference for Natural Texture Statistics is Modulated by Contrast Polarity

Adult observers are sensitive to statistical regularities present in natural images. Developmentally, research has shown that children do not show sensitivity to these natural regularities until approximately 8-10 years of age. This finding is surprising given that even infants gradually encode a range of high-level statistical regularities of their visual environment in the first year of life, We suggest that infants may in fact exhibit sensitivity to natural image statistics under circumstances where images of complex, natural textures, such as a photograph of rocks, are used as experimental stimuli and natural appearance is substantially manipulated. We tested this hypothesis by examining how infants' visual preference for real versus computer-generated synthetic textures was modulated by contrast negation, which produces an image similar to a photographic negative. We observed that older infants' (9-months of age) preferential looking behavior in this task was affected by contrast polarity, suggesting that the infant visual system is sensitive to deviations from natural texture appearance, including (1) discrepancies in appearance that differentiate natural and synthetic textures from one another and (2) the disruption of contrast polarity following negation. We discuss our results in the context of adult texture processing and the "perceptual narrowing" of visual recognition during the first year of life.

The contribution of central and peripheral vision in scene categorization: a study on people with central vision loss

Studies in normally sighted people suggest that scene recognition is based on global physical properties and can be accomplished by the low resolution of peripheral vision. We examine the contribution of peripheral and central vision in scene gist recognition in patients with central vision loss and age-matched controls. Twenty-one patients with neovascular age related macular degeneration (AMD), with a visual acuity lower than 20/50, and 15 age-matched normally sighted controls participated in a natural/urban scene categorization task. The stimuli were colored photographs of natural scenes presented randomly at one of five spatial locations of a computer screen: centre, top left, top right, bottom left and bottom right at 12° eccentricity. Sensitivity (d') and response times were recorded. Normally sighted people exhibited higher sensitivity and shorter response times when the scene was presented centrally than for peripheral pictures. Sensitivity was lower and response times were longer for people with AMD than for controls at all spatial location. In contrast to controls patients were not better for central than for peripheral pictures. The results of normally sighted controls indicate that scene categorization can be accomplished by the low resolution of peripheral vision but central vision remains more efficient than peripheral vision for scene gist recognition. People with central vision loss likely categorized scenes on the basis of low frequency information both in normal peripheral vision and in low acuity central vision.

Attention in the real world: toward understanding its neural basis

The efficient selection of behaviorally relevant objects from cluttered environments supports our everyday goals. Attentional selection has typically been studied in search tasks involving artificial and simplified displays. Although these studies have revealed important basic principles of attention, they do not explain how the brain efficiently selects familiar objects in complex and meaningful real-world scenes. Findings from recent neuroimaging studies indicate that real-world search is mediated by 'what' and 'where' attentional templates that are implemented in high-level visual cortex. These templates represent target-diagnostic properties and likely target locations, respectively, and are shaped by object familiarity, scene context, and memory. We propose a framework for real-world search that incorporates these recent findings and specifies directions for future study.

Encoding-Stage Crosstalk Between Object- and Spatial Property-Based Scene Processing Pathways

Scene categorization draws on 2 information sources: The identities of objects scenes contain and scenes' intrinsic spatial properties. Because these resources are formally independent, it is possible for them to leads to conflicting judgments of scene category. We tested the hypothesis that the potential for such conflicts is mitigated by a system of "crosstalk" between object- and spatial layout-processing pathways, under which the encoded spatial properties of scenes are biased by scenes' object contents. Specifically, we show that the presence of objects strongly associated with a given scene category can bias the encoded spatial properties of scenes containing them toward the average of that category, an effect which is evident both in behavioral measures of scenes' perceived spatial properties and in scene-evoked multivoxel patterns recorded with functional magnetic resonance imaging from the parahippocampal place area (PPA), a region associated with the processing of scenes' spatial properties. These results indicate that harmonization of object- and spatial property-based estimates of scene identity begins when spatial properties are encoded, and that the PPA plays a central role in this process.

Beyond gist: strategic and incremental information accumulation for scene categorization

Research on scene categorization generally concentrates on gist processing, particularly the speed and minimal features with which the "story" of a scene can be extracted. However, this focus has led to a paucity of research into how scenes are categorized at specific hierarchical levels (e.g., a scene could be a road or more specifically a highway); consequently, research has disregarded a potential diagnostically driven feedback process. We presented participants with scenes that were low-pass filtered so only their gist was revealed, while a gaze-contingent window provided the fovea with full-resolution details. By recording where in a scene participants fixated prior to making a basic- or subordinate-level judgment, we identified the scene information accrued when participants made either categorization. We observed a feedback process, dependent on categorization level, that systematically accrues sufficient and detailed diagnostic information from the same scene. Our results demonstrate that during scene processing, a diagnostically driven bidirectional interplay between top-down and bottom-up information facilitates relevant category processing.

Angular declination and the dynamic perception of egocentric distance

The extraction of the distance between an object and an observer is fast when angular declination is informative, as it is with targets placed on the ground. To what extent does angular declination drive performance when viewing time is limited? Participants judged target distances in a real-world environment with viewing durations ranging from 36-220 ms. An important role for angular declination was supported by experiments showing that the cue provides information about egocentric distance even on the very first glimpse, and that it supports a sensitive response to distance in the absence of other useful cues. Performance was better at 220-ms viewing durations than for briefer glimpses, suggesting that the perception of distance is dynamic even within the time frame of a typical eye fixation. Critically, performance in limited viewing trials was better when preceded by a 15-s preview of the room without a designated target. The results indicate that the perception of distance is powerfully shaped by memory from prior visual experience with the scene. A theoretical framework for the dynamic perception of distance is presented.

Parametric Coding of the Size and Clutter of Natural Scenes in the Human Brain

Estimating the size of a space and its degree of clutter are effortless and ubiquitous tasks of moving agents in a natural environment. Here, we examine how regions along the occipital-temporal lobe respond to pictures of indoor real-world scenes that parametrically vary in their physical "size" (the spatial extent of a space bounded by walls) and functional "clutter" (the organization and quantity of objects that fill up the space). Using a linear regression model on multivoxel pattern activity across regions of interest, we find evidence that both properties of size and clutter are represented in the patterns of parahippocampal cortex, while the retrosplenial cortex activity patterns are predominantly sensitive to the size of a space, rather than the degree of clutter. Parametric whole-brain analyses confirmed these results. Importantly, this size and clutter information was represented in a way that generalized across different semantic categories. These data provide support for a property-based representation of spaces, distributed across multiple scene-selective regions of the cerebral cortex.

Detecting unidentified changes

Does becoming aware of a change to a purely visual stimulus necessarily cause the observer to be able to identify or localise the change or can change detection occur in the absence of identification or localisation? Several theories of visual awareness stress that we are aware of more than just the few objects to which we attend. In particular, it is clear that to some extent we are also aware of the global properties of the scene, such as the mean luminance or the distribution of spatial frequencies. It follows that we may be able to detect a change to a visual scene by detecting a change to one or more of these global properties. However, detecting a change to global property may not supply us with enough information to accurately identify or localise which object in the scene has been changed. Thus, it may be possible to reliably detect the occurrence of changes without being able to identify or localise what has changed. Previous attempts to show that this can occur with natural images have produced mixed results. Here we use a novel analysis technique to provide additional evidence that changes can be detected in natural images without also being identified or localised. It is likely that this occurs by the observers monitoring the global properties of the scene.

From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category

The visual system processes natural scenes in a split second. Part of this process is the extraction of "gist," a global first impression. It is unclear, however, how the human visual system computes this information. Here, we show that, when human observers categorize global information in real-world scenes, the brain exhibits strong sensitivity to low-level summary statistics. Subjects rated a specific instance of a global scene property, naturalness, for a large set of natural scenes while EEG was recorded. For each individual scene, we derived two physiologically plausible summary statistics by spatially pooling local contrast filter outputs: contrast energy (CE), indexing contrast strength, and spatial coherence (SC), indexing scene fragmentation. We show that behavioral performance is directly related to these statistics, with naturalness rating being influenced in particular by SC. At the neural level, both statistics parametrically modulated single-trial event-related potential amplitudes during an early, transient window (100-150 ms), but SC continued to influence activity levels later in time (up to 250 ms). In addition, the magnitude of neural activity that discriminated between man-made versus natural ratings of individual trials was related to SC, but not CE. These results suggest that global scene information may be computed by spatial pooling of responses from early visual areas (e.g., LGN or V1). The increased sensitivity over time to SC in particular, which reflects scene fragmentation, suggests that this statistic is actively exploited to estimate scene naturalness.

Statistics of high-level scene context

CONTEXT IS CRITICAL FOR RECOGNIZING ENVIRONMENTS AND FOR SEARCHING FOR OBJECTS WITHIN THEM: contextual associations have been shown to modulate reaction time and object recognition accuracy, as well as influence the distribution of eye movements and patterns of brain activations. However, we have not yet systematically quantified the relationships between objects and their scene environments. Here I seek to fill this gap by providing descriptive statistics of object-scene relationships. A total of 48, 167 objects were hand-labeled in 3499 scenes using the LabelMe tool (Russell et al., 2008). From these data, I computed a variety of descriptive statistics at three different levels of analysis: the ensemble statistics that describe the density and spatial distribution of unnamed "things" in the scene; the bag of words level where scenes are described by the list of objects contained within them; and the structural level where the spatial distribution and relationships between the objects are measured. The utility of each level of description for scene categorization was assessed through the use of linear classifiers, and the plausibility of each level for modeling human scene categorization is discussed. Of the three levels, ensemble statistics were found to be the most informative (per feature), and also best explained human patterns of categorization errors. Although a bag of words classifier had similar performance to human observers, it had a markedly different pattern of errors. However, certain objects are more useful than others, and ceiling classification performance could be achieved using only the 64 most informative objects. As object location tends not to vary as a function of category, structural information provided little additional information. Additionally, these data provide valuable information on natural scene redundancy that can be exploited for machine vision, and can help the visual cognition community to design experiments guided by statistics rather than intuition.

Scene perception in posterior cortical atrophy: categorization, description and fixation patterns

Partial or complete Balint's syndrome is a core feature of the clinico-radiological syndrome of posterior cortical atrophy (PCA), in which individuals experience a progressive deterioration of cortical vision. Although multi-object arrays are frequently used to detect simultanagnosia in the clinical assessment and diagnosis of PCA, to date there have been no group studies of scene perception in patients with the syndrome. The current study involved three linked experiments conducted in PCA patients and healthy controls. Experiment 1 evaluated the accuracy and latency of complex scene perception relative to individual faces and objects (color and grayscale) using a categorization paradigm. PCA patients were both less accurate (faces < scenes < objects) and slower (scenes < objects < faces) than controls on all categories, with performance strongly associated with their level of basic visual processing impairment; patients also showed a small advantage for color over grayscale stimuli. Experiment 2 involved free description of real world scenes. PCA patients generated fewer features and more misperceptions than controls, though perceptual errors were always consistent with the patient's global understanding of the scene (whether correct or not). Experiment 3 used eye tracking measures to compare patient and control eye movements over initial and subsequent fixations of scenes. Patients' fixation patterns were significantly different to those of young and age-matched controls, with comparable group differences for both initial and subsequent fixations. Overall, these findings describe the variability in everyday scene perception exhibited by individuals with PCA, and indicate the importance of exposure duration in the perception of complex scenes.

Auditory and cognitive effects of aging on perception of environmental sounds in natural auditory scenes

PURPOSE

Previously, Gygi and Shafiro (2011) found that when environmental sounds are semantically incongruent with the background scene (e.g., horse galloping in a restaurant), they can be identified more accurately by young normal-hearing listeners (YNH) than sounds congruent with the scene (e.g., horse galloping at a racetrack). This study investigated how age and high-frequency audibility affect this Incongruency Advantage (IA) effect.

METHOD

In Experiments 1a and 1b, elderly listeners ( N = 18 for 1a; N = 10 for 1b) with age-appropriate hearing (EAH) were tested on target sounds and auditory scenes in 5 sound-to-scene ratios (So/Sc) between -3 and -18 dB. Experiment 2 tested 11 YNH on the same sound-scene pairings lowpass-filtered at 4 kHz (YNH-4k).

RESULTS

The EAH and YNH-4k groups exhibited an almost identical pattern of significant IA effects, but both were at approximately 3.9 dB higher So/Sc than the previously tested YNH listeners. However, the psychometric functions revealed a shallower slope for EAH listeners compared with YNH listeners for the congruent stimuli only, suggesting a greater difficulty for the EAH listeners in attending to sounds expected to occur in a scene.

CONCLUSIONS

These findings indicate that semantic relationships between environmental sounds in soundscapes are mediated by both audibility and cognitive factors and suggest a method for dissociating these factors.