A stimulus-driven approach reveals vertical luminance gradient as a stimulus feature that drives human cortical scene selectivity

Human neuroimaging studies have revealed a dedicated cortical system for visual scene processing. But what is a "scene"? Here, we use a stimulus-driven approach to identify a stimulus feature that selectively drives cortical scene processing. Specifically, using fMRI data from BOLD5000, we examined the images that elicited the greatest response in the cortical scene processing system, and found that there is a common "vertical luminance gradient" (VLG), with the top half of a scene image brighter than the bottom half; moreover, across the entire set of images, VLG systematically increases with the neural response in the scene-selective regions (Study 1). Thus, we hypothesized that VLG is a stimulus feature that selectively engages cortical scene processing, and directly tested the role of VLG in driving cortical scene selectivity using tightly controlled VLG stimuli (Study 2). Consistent with our hypothesis, we found that the scene-selective cortical regions-but not an object-selective region or early visual cortex-responded significantly more to images of VLG over control stimuli with minimal VLG. Interestingly, such selectivity was also found for images with an "inverted" VLG, resembling the luminance gradient in night scenes. Finally, we also tested the behavioral relevance of VLG for visual scene recognition (Study 3); we found that participants even categorized tightly controlled stimuli of both upright and inverted VLG to be a place more than an object, indicating that VLG is also used for behavioral scene recognition. Taken together, these results reveal that VLG is a stimulus feature that selectively engages cortical scene processing, and provide evidence for a recent proposal that visual scenes can be characterized by a set of common and unique visual features.

Two scene navigation systems dissociated by deliberate versus automatic processing

Successfully navigating the world requires avoiding boundaries and obstacles in one's immediately-visible environment, as well as finding one's way to distant places in the broader environment. Recent neuroimaging studies suggest that these two navigational processes involve distinct cortical scene processing systems, with the occipital place area (OPA) supporting navigation through the local visual environment, and the retrosplenial complex (RSC) supporting navigation through the broader spatial environment. Here we hypothesized that these systems are distinguished not only by the scene information they represent (i.e., the local visual versus broader spatial environment), but also based on the automaticity of the process they involve, with navigation through the broader environment (including RSC) operating deliberately, and navigation through the local visual environment (including OPA) operating automatically. We tested this hypothesis using fMRI and a maze-navigation paradigm, where participants navigated two maze structures (complex or simple, testing representation of the broader spatial environment) under two conditions (active or passive, testing deliberate versus automatic processing). Consistent with the hypothesis that RSC supports deliberate navigation through the broader environment, RSC responded significantly more to complex than simple mazes during active, but not passive navigation. By contrast, consistent with the hypothesis that OPA supports automatic navigation through the local visual environment, OPA responded strongly even during passive navigation, and did not differentiate between active versus passive conditions. Taken together, these findings suggest the novel hypothesis that navigation through the broader spatial environment is deliberate, whereas navigation through the local visual environment is automatic, shedding new light on the dissociable functions of these systems.

Concavity as a diagnostic feature of visual scenes

Despite over two decades of research on the neural mechanisms underlying human visual scene, or place, processing, it remains unknown what exactly a “scene” is. Intuitively, we are always inside a scene, while interacting with the outside of objects. Hence, we hypothesize that one diagnostic feature of a scene may be concavity, portraying “inside”, and predict that if concavity is a scene-diagnostic feature, then: 1) images that depict concavity, even non-scene images (e.g., the “inside” of an object – or concave object), will be behaviorally categorized as scenes more often than those that depict convexity, and 2) the cortical scene-processing system will respond more to concave images than to convex images. As predicted, participants categorized concave objects as scenes more often than convex objects, and, using functional magnetic resonance imaging (fMRI), two scene-selective cortical regions (the parahippocampal place area, PPA, and the occipital place area, OPA) responded significantly more to concave than convex objects. Surprisingly, we found no behavioral or neural differences between images of concave versus convex buildings. However, in a follow-up experiment, using tightly-controlled images, we unmasked a selective sensitivity to concavity over convexity of scene boundaries (i.e., walls) in PPA and OPA. Furthermore, we found that even highly impoverished line drawings of concave shapes are behaviorally categorized as scenes more often than convex shapes. Together, these results provide converging behavioral and neural evidence that concavity is a diagnostic feature of visual scenes.

Contour junctions underlie neural representations of scene categories in high-level human visual cortex

Humans efficiently grasp complex visual environments, making highly consistent judgments of entry-level category despite their high variability in visual appearance. How does the human brain arrive at the invariant neural representations underlying categorization of real-world environments? We here show that the neural representation of visual environments in scene-selective human visual cortex relies on statistics of contour junctions, which provide cues for the three-dimensional arrangement of surfaces in a scene. We manipulated line drawings of real-world environments such that statistics of contour orientations or junctions were disrupted. Manipulated and intact line drawings were presented to participants in an fMRI experiment. Scene categories were decoded from neural activity patterns in the parahippocampal place area (PPA), the occipital place area (OPA) and other visual brain regions. Disruption of junctions but not orientations led to a drastic decrease in decoding accuracy in the PPA and OPA, indicating the reliance of these areas on intact junction statistics. Accuracy of decoding from early visual cortex, on the other hand, was unaffected by either image manipulation. We further show that the correlation of error patterns between decoding from the scene-selective brain areas and behavioral experiments is contingent on intact contour junctions. Finally, a searchlight analysis exposes the reliance of visually active brain regions on different sets of contour properties. Statistics of contour length and curvature dominate neural representations of scene categories in early visual areas and contour junctions in high-level scene-selective brain regions.

Conceptual size representation in ventral visual cortex

Recent findings suggest that visual objects may be mapped along the ventral occipitotemporal cortex according to their real-world size (Konkle and Oliva, 2012). It has been argued that such mapping does not reflect an abstract, conceptual size representation, but rather the visual or functional properties associated with small versus big real-world objects. To determine whether a more abstract conceptual size representation may affect visual cortical activation we used meaningless geometrical shapes, devoid of semantic or functional associations, which were associated with specific size representations by virtue of extensive training. Following training, participants underwent functional magnetic resonance imaging (fMRI) scanning while performing a conceptual size comparison task on the geometrical shapes. In addition, a size comparison task was conducted for numeral digits denoting small and big numbers. A region-of-interest analysis revealed larger blood oxygenation level dependent (BOLD) responses for conceptually 'big' than for conceptually 'small' shapes, as well as for big versus small numbers, within medial (parahippocampal place area, PPA) and lateral (occipital place area, OPA) place-selective regions. Processing of the 'big' visual shapes further elicited enhanced activation in early visual cortex, possibly reflecting top-down projections from PPA. By using arbitrary shapes and numbers we minimized visual, categorical, or functional influences on fMRI measurement, providing evidence for a possible neural mechanism underlying the representation of abstract conceptual size within the ventral visual stream.

Spatial frequency processing in scene-selective cortical regions

Visual analysis begins with the parallel extraction of different attributes at different spatial frequencies. Low spatial frequencies (LSF) convey coarse information and are characterized by high luminance contrast, while high spatial frequencies (HSF) convey fine details and are characterized by low luminance contrast. In the present fMRI study, we examined how scene-selective regions-the parahippocampal place area (PPA), the retrosplenial cortex (RSC) and the occipital place area (OPA)-responded to spatial frequencies when contrast was either equalized or not equalized across spatial frequencies. Participants performed a categorization task on LSF, HSF and non-filtered scenes belonging to two different categories (indoors and outdoors). We either left contrast across scenes untouched, or equalized it using a root-mean-square contrast normalization. We found that when contrast remained unmodified, LSF and NF scenes elicited greater activation than HSF scenes in the PPA. However, when contrast was equalized across spatial frequencies, the PPA was selective to HFS. This suggests that PPA activity relies on an interaction between spatial frequency and contrast in scenes. In the RSC, LSF and NF elicited greater response than HSF scenes when contrast was not modified, while no effect of spatial frequencies appeared when contrast was equalized across filtered scenes, suggesting that the RSC is sensitive to high-contrast information. Finally, we observed selective activation of the OPA in response to HSF, irrespective of contrast manipulation. These results provide new insights into how scene-selective areas operate during scene processing.