BOLD fMRI and psychophysical measurements of contrast response to broadband images

Attention deployment in natural scenes: Higher-order scene statistics rather than semantics modulate the N2pc component

Which properties of a natural scene affect visual search? We consider the alternative hypotheses that low-level statistics, higher-level statistics, semantics, or layout affect search difficulty in natural scenes. Across three experiments (n = 20 each), we used four different backgrounds that preserve distinct scene properties: (a) natural scenes (all experiments); (b) 1/f noise (pink noise, which preserves only low-level statistics and was used in Experiments 1 and 2); (c) textures that preserve low-level and higher-level statistics but not semantics or layout (Experiments 2 and 3); and (d) inverted (upside-down) scenes that preserve statistics and semantics but not layout (Experiment 2). We included "split scenes" that contained different backgrounds left and right of the midline (Experiment 1, natural/noise; Experiment 3, natural/texture). Participants searched for a Gabor patch that occurred at one of six locations (all experiments). Reaction times were faster for targets on noise and slower on inverted images, compared to natural scenes and textures. The N2pc component of the event-related potential, a marker of attentional selection, had a shorter latency and a higher amplitude for targets in noise than for all other backgrounds. The background contralateral to the target had an effect similar to that on the target side: noise led to faster reactions and shorter N2pc latencies than natural scenes, although we observed no difference in N2pc amplitude. There were no interactions between the target side and the non-target side. Together, this shows that-at least when searching simple targets without own semantic content-natural scenes are more effective distractors than noise and that this results from higher-order statistics rather than from semantics or layout.

Uncovering local aggregated air quality index with smartphone captured images leveraging efficient deep convolutional neural network

The prevalence and mobility of smartphones make these a widely used tool for environmental health research. However, their potential for determining aggregated air quality index (AQI) based on PM2.5 concentration in specific locations remains largely unexplored in the existing literature. In this paper, we thoroughly examine the challenges associated with predicting location-specific PM2.5 concentration using images taken with smartphone cameras. The focus of our study is on Dhaka, the capital of Bangladesh, due to its significant air pollution levels and the large population exposed to it. Our research involves the development of a Deep Convolutional Neural Network (DCNN), which we train using over a thousand outdoor images taken and annotated. These photos are captured at various locations in Dhaka, and their labels are based on PM2.5 concentration data obtained from the local US consulate, calculated using the NowCast algorithm. Through supervised learning, our model establishes a correlation index during training, enhancing its ability to function as a Picture-based Predictor of PM2.5 Concentration (PPPC). This enables the algorithm to calculate an equivalent daily averaged AQI index from a smartphone image. Unlike, popular overly parameterized models, our model shows resource efficiency since it uses fewer parameters. Furthermore, test results indicate that our model outperforms popular models like ViT and INN, as well as popular CNN-based models such as VGG19, ResNet50, and MobileNetV2, in predicting location-specific PM2.5 concentration. Our dataset is the first publicly available collection that includes atmospheric images and corresponding PM2.5 measurements from Dhaka. Our codes and dataset are available at  https://github.com/lepotatoguy/aqi .

A mechanistic account of visual discomfort

Much of the neural machinery of the early visual cortex, from the extraction of local orientations to contextual modulations through lateral interactions, is thought to have developed to provide a sparse encoding of contour in natural scenes, allowing the brain to process efficiently most of the visual scenes we are exposed to. Certain visual stimuli, however, cause visual stress, a set of adverse effects ranging from simple discomfort to migraine attacks, and epileptic seizures in the extreme, all phenomena linked with an excessive metabolic demand. The theory of efficient coding suggests a link between excessive metabolic demand and images that deviate from natural statistics. Yet, the mechanisms linking energy demand and image spatial content in discomfort remain elusive. Here, we used theories of visual coding that link image spatial structure and brain activation to characterize the response to images observers reported as uncomfortable in a biologically based neurodynamic model of the early visual cortex that included excitatory and inhibitory layers to implement contextual influences. We found three clear markers of aversive images: a larger overall activation in the model, a less sparse response, and a more unbalanced distribution of activity across spatial orientations. When the ratio of excitation over inhibition was increased in the model, a phenomenon hypothesised to underlie interindividual differences in susceptibility to visual discomfort, the three markers of discomfort progressively shifted toward values typical of the response to uncomfortable stimuli. Overall, these findings propose a unifying mechanistic explanation for why there are differences between images and between observers, suggesting how visual input and idiosyncratic hyperexcitability give rise to abnormal brain responses that result in visual stress.

The Psychosis Human Connectome Project: Design and rationale for studies of visual neurophysiology

Visual perception is abnormal in psychotic disorders such as schizophrenia. In addition to hallucinations, laboratory tests show differences in fundamental visual processes including contrast sensitivity, center-surround interactions, and perceptual organization. A number of hypotheses have been proposed to explain visual dysfunction in psychotic disorders, including an imbalance between excitation and inhibition. However, the precise neural basis of abnormal visual perception in people with psychotic psychopathology (PwPP) remains unknown. Here, we describe the behavioral and 7 tesla MRI methods we used to interrogate visual neurophysiology in PwPP as part of the Psychosis Human Connectome Project (HCP). In addition to PwPP (n = 66) and healthy controls (n = 43), we also recruited first-degree biological relatives (n = 44) in order to examine the role of genetic liability for psychosis in visual perception. Our visual tasks were designed to assess fundamental visual processes in PwPP, whereas MR spectroscopy enabled us to examine neurochemistry, including excitatory and inhibitory markers. We show that it is feasible to collect high-quality data across multiple psychophysical, functional MRI, and MR spectroscopy experiments with a sizable number of participants at a single research site. These data, in addition to those from our previously described 3 tesla experiments, will be made publicly available in order to facilitate further investigations by other research groups. By combining visual neuroscience techniques and HCP brain imaging methods, our experiments offer new opportunities to investigate the neural basis of abnormal visual perception in PwPP.

Contextual Information Modulates Pupil Size in Autistic Children

Recent Bayesian models suggest that perception is more “data-driven” and less dependent on contextual information in autistic individuals than others. However, experimental tests of this hypothesis have given mixed results, possibly due to the lack of objectivity of the self-report methods typically employed. Here we introduce an objective no-report paradigm based on pupillometry to assess the processing of contextual information in autistic children, together with a comparison clinical group. After validating in neurotypical adults a child-friendly pupillometric paradigm, in which we embedded test images within an animation movie that participants watched passively, we compared pupillary response to images of the sun and meaningless control images in children with autism vs. age- and IQ-matched children presenting developmental disorders unrelated to the autistic spectrum. Both clinical groups showed stronger pupillary constriction for the sun images compared with control images, like the neurotypical adults. However, there was no detectable difference between autistic children and the comparison group, despite a significant difference in pupillary light responses, which were enhanced in the autistic group. Our report introduces an objective technique for studying perception in clinical samples and children. The lack of statistically significant group differences in our tests suggests that autistic children and the comparison group do not show large differences in perception of these stimuli. This opens the way to further studies testing contextual processing at other levels of perception.

The missing N1 or jittered P2: Electrophysiological correlates of pattern glare in the time and frequency domain

Excessive sensitivity to certain visual stimuli (cortical hyperexcitability) is associated with a number of neurological disorders including migraine, epilepsy, multiple sclerosis, autism and possibly dyslexia. Others show disruptive sensitivity to visual stimuli with no other obvious pathology or symptom profile (visual stress) which can extend to discomfort and nausea. We used event‐related potentials (ERPs) to explore the neural correlates of visual stress and headache proneness. We analysed ERPs in response to thick (0.37 cycles per degree [c/deg]), medium (3 c/deg) and thin (12 c/deg) gratings, using mass univariate analysis, considering three factors in the general population: headache proneness, visual stress and discomfort. We found relationships between ERP features and the headache and discomfort factors. Stimulus main effects were driven by the medium stimulus regardless of participant characteristics. Participants with high discomfort ratings had larger P1 components for the initial presentation of medium stimuli, suggesting initial cortical hyperexcitability that is later suppressed. The participants with high headache ratings showed atypical N1‐P2 components for medium stripes relative to the other stimuli. This effect was present only after repeated stimulus presentation. These effects were also explored in the frequency domain, suggesting variations in intertrial theta band phase coherence. Our results suggest that discomfort and headache in response to striped stimuli are related to different neural processes; however, more exploration is needed to determine whether the results translate to a clinical migraine population.

Nice and slow: Measuring sensitivity and visual preference toward naturalistic stimuli varying in their amplitude spectra in space and time

The 1/f^α amplitude spectrum is a statistical property of natural scenes characterising a specific distribution of spatial and temporal frequencies and their associated luminance intensities. This property has been studied extensively in the spatial domain whereby sensitivity and visual preference overlap and peak for slopes within the natural range (α ≈ 1), but remains relatively less studied in the temporal domain. Here, we used a 4AFC task to measure sensitivity and a 2AFC task to measure visual preference and across a wide range of spatial (α = 0.25, 1.25, 2.25) and temporal (α = 0.25 to 2.50, step size: 0.25) slope conditions. Stimuli with a shallow temporal slope modulate rapidly (e.g. 0.25), whereas stimuli with a steep slope modulate slowly (e.g. 2.25). Interestingly, sensitivity and visual preference did not closely overlap. While the sensitivity of the visual system is highest for our stimulus with an intermediate modulation rate (1.25), which is most abundant in nature, the stimulus with the slowest modulation rate (2.25) was most preferred. It seems sensible for the visual system to be sensitive to spatiotemporal spectra that most commonly exist in nature (α ≈ 1). However, it is possible that preference might be related to what these properties signal in the natural world. Consider the cases of waves slowly vs. rapidly crashing on a beach or fast vs. slow animals. In both instances the slowest option is often the safest and preferential, suggesting that the temporal 1/f^α amplitude spectrum provides additional information that may indicate preferred environmental conditions.

Color Compensation in Anomalous Trichromats Assessed with fMRI

Anomalous trichromacy is a common form of congenital color deficiency resulting from a genetic alteration in the photopigments of the eye's light receptors. The changes reduce sensitivity to reddish and greenish hues, yet previous work suggests that these observers may experience the world to be more colorful than their altered receptor sensitivities would predict, potentially indicating an amplification of post-receptoral signals. However, past evidence suggesting such a gain adjustment rests on subjective measures of color appearance or salience. We directly tested for neural amplification by using fMRI to measure cortical responses in color-anomalous and normal control observers. Color contrast response functions were measured in two experiments with different tasks to control for attentional factors. Both experiments showed a predictable reduction in chromatic responses for anomalous trichromats in primary visual cortex. However, in later areas V2v and V3v, chromatic responses in the two groups were indistinguishable. Our results provide direct evidence for neural plasticity that compensates for the deficiency in the initial receptor color signals and suggest that the site of this compensation is in early visual cortex.

Symmetry Modulates the Amplitude Spectrum Slope Effect on Visual Preference

Within the spectrum of a natural image, the amplitude of modulation decreases with spatial frequency. The speed of such an amplitude decrease, or the amplitude spectrum slope, of an image affects the perceived aesthetic value. Additionally, a human observer would consider a symmetric image more appealing than they would an asymmetric one. We investigated how these two factors jointly affect aesthetic preferences by manipulating both the amplitude spectrum slope and the symmetric level of images to assess their effects on aesthetic preference on a 6-point Likert scale. Our results showed that the preference ratings increased with the symmetry level but had an inverted U-shaped relation to amplitude spectrum slope. In addition, a strong interaction existed between symmetry level and amplitude spectrum slope on preference rating, in that symmetry can amplify the amplitude spectrum slope’s effects. A quadratic function of the spectrum slope can describe such effects. That is, preference is an inverted U-shaped function of spectrum slope whose intercept is determined by the number of symmetry axes. The modulation depth of the quadratic function manifests the interaction between the two factors.

Development and Evaluation of a Visual Remediation Intervention for People with Schizophrenia

It is now well documented that schizophrenia is associated with impairments in visual processing at all levels of vision, and that these disturbances are related to deficits in multiple higher-level cognitive and social cognitive functions. Visual remediation methods have been slow to appear in the literature as a potential treatment strategy to target these impairments, however, in contrast to interventions that aim to improve auditory and higher cognitive functions in schizophrenia. In this report, we describe a National Institute of Mental Health (NIMH)-funded R61/R33 grant that uses a phased approach to optimize and evaluate a novel visual remediation intervention for people with schizophrenia. The goals of this project are: (1) in the R61 phase, to establish the optimal components and dose (number of sessions) of a visual remediation intervention from among two specific visual training strategies (and their combination) for improving low and mid-level visual functions in schizophrenia; and (2) in the R33 phase, to determine the extent to which the optimal intervention improves not only visual processing but also higher-level cognitive and role functions. Here we present the scientific background for and innovation of the study, along with our methods, hypotheses, and preliminary data. The results of this study will help determine the utility of this novel intervention approach for targeting visual perceptual, cognitive, and functional impairments in schizophrenia.

Visual discomfort from flicker: Effects of mean light level and contrast

Uncomfortable images generally have a particular spatial structure, which deviates from a reciprocal relationship between amplitude and spatial frequency (f) in the Fourier domain (1/f). Although flickering patterns with similar temporal structure also appear uncomfortable, the discomfort is affected by not only the amplitude spectrum but also the phase spectrum. Here we examined how discomfort from flicker with differing temporal profiles also varies as a function of the mean light level and luminance contrast of the stimulus. Participants were asked to rate discomfort for a 17° flickering uniform field at different light levels from scotopic to photopic. The flicker waveform was varied with a square wave or random phase spectrum and filtered by modulating the slope of the amplitude spectrum relative to 1/f. At photopic levels, the 1/f square wave flicker appeared most comfortable, whereas the discomfort from the random flicker increased monotonically as the slope of the amplitude spectrum decreased. This special status for the 1/f square wave condition was limited to photopic light levels. At the lower mesopic or scotopic levels, the effect of phase spectrum on the discomfort was diminished, with both phase spectra showing a monotonic change with the slope of the amplitude spectrum. We show that these changes cannot be accounted for by changes in the effective luminance contrast of the stimuli or by the responses from a linear model based on the temporal impulse responses under different light levels. However, discomfort from flicker is robustly correlated with judgments of the perceived naturalness of flicker across different contrasts and mean luminance levels.

Adaptation and visual discomfort from flicker

Spatial images with unnatural amplitude spectra tend to appear uncomfortable. Analogous effects are found in the temporal domain, yet discomfort in flickering patterns is also strongly dependent on the phase spectrum. Here we examined how discomfort in temporal flicker is affected by adaptation to different amplitude and phase spectra. Adapting and test flicker were square wave or random phase transitions in a uniform field filtered by increasing (blurred) or decreasing (sharpened) the slope of the amplitude spectrum. Participants rated the level of discomfort or sharpness/blur for the test flicker. Before adaptation, square wave transitions were rated as most comfortable when they had "focused" edges, which were defined as characterized by 1/f amplitude spectra, while random phase transitions instead appeared more comfortable the more blurred they were. After adapting to blurred or sharpened transitions, both square wave and random phase flicker appeared more sharpened or blurred, respectively, and these effects were consistent with renormalization of perceived temporal focus. In comparison, adaptation affected discomfort in the two waveforms in qualitatively different ways, and exposure to the adapting stimulus tended to increase rather than decreased its perceived discomfort. These results point to a dissociation between the perceived amplitude spectrum and perceived discomfort, suggesting they in part depend on distinct processes. The results further illustrate the importance of the phase spectrum in determining visual discomfort from flickering patterns.

Fragmented ambiguous objects: Stimuli with stable low-level features for object recognition tasks

Visual object recognition is a complex skill that relies on the interaction of many spatially distinct and specialized visual areas in the human brain. One tool that can help us better understand these specializations and interactions is a set of visual stimuli that do not differ along low-level dimensions (e.g., orientation, contrast) but do differ along high-level dimensions, such as whether a real-world object can be detected. The present work creates a set of line segment-based images that are matched for luminance, contrast, and orientation distribution (both for single elements and for pair-wise combinations) but result in a range of object and non-object percepts. Image generation started with images of isolated objects taken from publicly available databases and then progressed through 3-stages: a computer algorithm generating 718 candidate images, expert observers selecting 217 for further consideration, and naïve observers performing final ratings. This process identified a set of 100 images that all have the same low-level properties but cover a range of recognizability (proportion of naïve observers (N = 120) who indicated that the stimulus "contained a known object") and semantic stability (consistency across the categories of living, non-living/manipulable, and non-living/non-manipulable when the same observers named "known" objects). Stimuli are available at https://github.com/caolman/FAOT.git.

Inverted Encoding Models Reconstruct an Arbitrary Model Response, Not the Stimulus

Probing how large populations of neurons represent stimuli is key to understanding sensory representations as many stimulus characteristics can only be discerned from population activity and not from individual single-units. Recently, inverted encoding models have been used to produce channel response functions from large spatial-scale measurements of human brain activity that are reminiscent of single-unit tuning functions and have been proposed to assay "population-level stimulus representations" (Sprague et al., 2018a). However, these channel response functions do not assay population tuning. We show by derivation that the channel response function is only determined up to an invertible linear transform. Thus, these channel response functions are arbitrary, one of an infinite family and therefore not a unique description of population representation. Indeed, simulations demonstrate that bimodal, even random, channel basis functions can account perfectly well for population responses without any underlying neural response units that are so tuned. However, the approach can be salvaged by extending it to reconstruct the stimulus, not the assumed model. We show that when this is done, even using bimodal and random channel basis functions, a unimodal function peaking at the appropriate value of the stimulus is recovered which can be interpreted as a measure of population selectivity. More precisely, the recovered function signifies how likely any value of the stimulus is, given the observed population response. Whether an analysis is recovering the hypothetical responses of an arbitrary model rather than assessing the selectivity of population representations is not an issue unique to the inverted encoding model and human neuroscience, but a general problem that must be confronted as more complex analyses intervene between measurement of population activity and presentation of data.

Neural Mechanisms of Material Perception: Quest on Shitsukan

In recent years, a growing body of research has addressed the nature and mechanism of material perception. Material perception entails perceiving and recognizing a material, surface quality or internal state of an object based on sensory stimuli such as visual, tactile, and/or auditory sensations. This process is ongoing in every aspect of daily life. We can, for example, easily distinguish whether an object is made of wood or metal, or whether a surface is rough or smooth. Judging whether the ground is wet or dry or whether a fish is fresh also involves material perception. Information obtained through material perception can be used to govern actions toward objects and to make decisions about whether to approach an object or avoid it. Because the physical processes leading to sensory signals related to material perception is complicated, it has been difficult to manipulate experimental stimuli in a rigorous manner. However, that situation is now changing thanks to advances in technology and knowledge in related fields. In this article, we will review what is currently known about the neural mechanisms responsible for material perception. We will show that cortical areas in the ventral visual pathway are strongly involved in material perception. Our main focus is on vision, but every sensory modality is involved in material perception. Information obtained through different sensory modalities is closely linked in material perception. Such cross-modal processing is another important feature of material perception, and will also be covered in this review.

A quantitative framework for motion visibility in human cortex

Despite the central use of motion visibility to reveal the neural basis of perception, perceptual decision making, and sensory inference there exists no comprehensive quantitative framework establishing how motion visibility parameters modulate human cortical response. Random-dot motion stimuli can be made less visible by reducing image contrast or motion coherence, or by shortening the stimulus duration. Because each of these manipulations modulates the strength of sensory neural responses they have all been extensively used to reveal cognitive and other nonsensory phenomena such as the influence of priors, attention, and choice-history biases. However, each of these manipulations is thought to influence response in different ways across different cortical regions and a comprehensive study is required to interpret this literature. Here, human participants observed random-dot stimuli varying across a large range of contrast, coherence, and stimulus durations as we measured blood-oxygen-level dependent responses. We developed a framework for modeling these responses that quantifies their functional form and sensitivity across areas. Our framework demonstrates the sensitivity of all visual areas to each parameter, with early visual areas V1-V4 showing more parametric sensitivity to changes in contrast and V3A and the human middle temporal area to coherence. Our results suggest that while motion contrast, coherence, and duration share cortical representation, they are encoded with distinct functional forms and sensitivity. Thus, our quantitative framework serves as a reference for interpretation of the vast perceptual literature manipulating these parameters and shows that different manipulations of visibility will have different effects across human visual cortex and need to be interpreted accordingly. NEW & NOTEWORTHY Manipulations of motion visibility have served as a key tool for understanding the neural basis for visual perception. Here we measured human cortical response to changes in visibility across a comprehensive range of motion visibility parameters and modeled these with a quantitative framework. Our quantitative framework can be used as a reference for linking human cortical response to perception and underscores that different manipulations of motion visibility can have greatly different effects on cortical representation.

Computational Modeling of Contrast Sensitivity and Orientation Tuning in First-Episode and Chronic Schizophrenia

Computational modeling is a useful method for generating hypotheses about the contributions of impaired neurobiological mechanisms, and their interactions, to psychopathology. Modeling is being increasingly used to further our understanding of schizophrenia, but to date, it has not been applied to questions regarding the common perceptual disturbances in the disorder. In this article, we model aspects of low-level visual processing and demonstrate how this can lead to testable hypotheses about both the nature of visual abnormalities in schizophrenia and the relationships between the mechanisms underlying these disturbances and psychotic symptoms. Using a model that incorporates retinal, lateral geniculate nucleus (LGN), and V1 activity, as well as gain control in the LGN, homeostatic adaptation in V1, lateral excitation and inhibition in V1, and self-organization of synaptic weights based on Hebbian learning and divisive normalization, we show that (a) prior data indicating increased contrast sensitivity for low-spatial-frequency stimuli in first-episode schizophrenia can be successfully modeled as a function of reduced retinal and LGN efferent activity, leading to overamplification at the cortical level, and (b) prior data on reduced contrast sensitivity and broadened orientation tuning in chronic schizophrenia can be successfully modeled by a combination of reduced V1 lateral inhibition and an increase in the Hebbian learning rate at V1 synapses for LGN input. These models are consistent with many current findings, and they predict several relationships that have not yet been demonstrated. They also have implications for understanding changes in brain and visual function from the first psychotic episode to the chronic stage of illness.

Inverted Encoding Models of Human Population Response Conflate Noise and Neural Tuning Width

Channel-encoding models offer the ability to bridge different scales of neuronal measurement by interpreting population responses, typically measured with BOLD imaging in humans, as linear sums of groups of neurons (channels) tuned for visual stimulus properties. Inverting these models to form predicted channel responses from population measurements in humans seemingly offers the potential to infer neuronal tuning properties. Here, we test the ability to make inferences about neural tuning width from inverted encoding models. We examined contrast invariance of orientation selectivity in human V1 (both sexes) and found that inverting the encoding model resulted in channel response functions that became broader with lower contrast, thus apparently violating contrast invariance. Simulations showed that this broadening could be explained by contrast-invariant single-unit tuning with the measured decrease in response amplitude at lower contrast. The decrease in response lowers the signal-to-noise ratio of population responses that results in poorer population representation of orientation. Simulations further showed that increasing signal to noise makes channel response functions less sensitive to underlying neural tuning width, and in the limit of zero noise will reconstruct the channel function assumed by the model regardless of the bandwidth of single units. We conclude that our data are consistent with contrast-invariant orientation tuning in human V1. More generally, our results demonstrate that population selectivity measures obtained by encoding models can deviate substantially from the behavior of single units because they conflate neural tuning width and noise and are therefore better used to estimate the uncertainty of decoded stimulus properties.SIGNIFICANCE STATEMENT It is widely recognized that perceptual experience arises from large populations of neurons, rather than a few single units. Yet, much theory and experiment have examined links between single units and perception. Encoding models offer a way to bridge this gap by explicitly interpreting population activity as the aggregate response of many single neurons with known tuning properties. Here we use this approach to examine contrast-invariant orientation tuning of human V1. We show with experiment and modeling that due to lower signal to noise, contrast-invariant orientation tuning of single units manifests in population response functions that broaden at lower contrast, rather than remain contrast-invariant. These results highlight the need for explicit quantitative modeling when making a reverse inference from population response profiles to single-unit responses.

Change Blindness Is Influenced by Both Contrast Energy and Subjective Importance within Local Regions of the Image

Our visual system receives an enormous amount of information, but not all information is retained. This is exemplified by the fact that subjects fail to detect large changes in a visual scene, i.e., change-blindness. Current theories propose that our ability to detect these changes is influenced by the gist or interpretation of an image. On the other hand, stimulus-driven image features such as contrast energy dominate the representation in early visual cortex (De Valois and De Valois, 1988; Boynton et al., 1999; Olman et al., 2004; Mante and Carandini, 2005; Dumoulin et al., 2008). Here we investigated whether contrast energy contributes to our ability to detect changes within a visual scene. We compared the ability to detect changes in contrast energy together with changes to a measure of the interpretation of an image. We used subjective important aspects of the image as a measure of the interpretation of an image. We measured reaction times while manipulating the contrast energy and subjective important properties using the change blindness paradigm. Our results suggest that our ability to detect changes in a visual scene is not only influenced by the subjective importance, but also by contrast energy. Also, we find that contrast energy and subjective importance interact. We speculate that contrast energy and subjective important properties are not independently represented in the visual system. Thus, our results suggest that the information that is retained of a visual scene is both influenced by stimulus-driven information as well as the interpretation of a scene.

Image identification from brain activity using the population receptive field model

A goal of computational models is not only to explain experimental data but also to make new predictions. A current focus of computational neuroimaging is to predict features of the presented stimulus from measured brain signals. These computational neuroimaging approaches may be agnostic about the underlying neural processes or may be biologically inspired. Here, we use the biologically inspired population receptive field (pRF) approach to identify presented images from fMRI recordings of the visual cortex, using an explicit model of the underlying neural response selectivity. The advantage of the pRF-model is its simplicity: it is defined by a handful of parameters, which can be estimated from fMRI data that was collected within half an hour. Using 7T MRI, we measured responses elicited by different visual stimuli: (i) conventional pRF mapping stimuli, (ii) semi-random synthetic images and (iii) natural images. The pRF mapping stimuli were used to estimate the pRF-properties of each cortical location in early visual cortex. Next, we used these pRFs to identify which synthetic or natural images was presented to the subject from the fMRI responses. We show that image identification using V1 responses is far above chance, both for the synthetic and natural images. Thus, we can identify visual images, including natural images, using the most fundamental low-parameter pRF model estimated from conventional pRF mapping stimuli. This allows broader application of image identification.

The effects of orientation and attention during surround suppression of small image features: A 7 Tesla fMRI study

Although V1 responses are driven primarily by elements within a neuron's receptive field, which subtends about 1° visual angle in parafoveal regions, previous work has shown that localized fMRI responses to visual elements reflect not only local feature encoding but also long-range pattern attributes. However, separating the response to an image feature from the response to the surrounding stimulus and studying the interactions between these two responses demands both spatial precision and signal independence, which may be challenging to attain with fMRI. The present study used 7 Tesla fMRI with 1.2-mm resolution to measure the interactions between small sinusoidal grating patches (targets) at 3° eccentricity and surrounds of various sizes and orientations to test the conditions under which localized, context-dependent fMRI responses could be predicted from either psychophysical or electrophysiological data. Targets were presented at 8%, 16%, and 32% contrast while manipulating (a) spatial extent of parallel (strongly suppressive) or orthogonal (weakly suppressive) surrounds, (b) locus of attention, (c) stimulus onset asynchrony between target and surround, and (d) blocked versus event-related design. In all experiments, the V1 fMRI signal was lower when target stimuli were flanked by parallel versus orthogonal context. Attention amplified fMRI responses to all stimuli but did not show a selective effect on central target responses or a measurable effect on orientation-dependent surround suppression. Suppression of the V1 fMRI response by parallel surrounds was stronger than predicted from psychophysics but showed a better match to previous electrophysiological reports.

The tuning of human visual cortex to variations in the 1/fα amplitude spectra and fractal properties of synthetic noise images

Natural scenes share a consistent distribution of energy across spatial frequencies (SF) known as the 1/f^α amplitude spectrum (α≈0.8-1.5, mean 1.2). This distribution is scale-invariant, which is a fractal characteristic of natural scenes with statistically similar structure at different spatial scales. While the sensitivity of the visual system to the 1/f properties of natural scenes has been studied extensively using psychophysics, relatively little is known about the tuning of cortical responses to these properties. Here, we use fMRI and retinotopic mapping techniques to measure and analyze BOLD responses in early visual cortex (V1, V2, and V3) to synthetic noise images that vary in their 1/f^α amplitude spectra (α=0.25 to 2.25, step size: 0.50) and contrast levels (10% and 30%) (Experiment 1). To compare the dependence of the BOLD response between the photometric (intensity based) and geometric (fractal) properties of our stimuli, in Experiment 2 we compared grayscale noise images to their binary (thresholded) counterparts, which contain only black and white regions. In both experiments, early visual cortex responded maximally to stimuli generated to have an input 1/f slope corresponding to natural 1/f^α amplitude spectra, and lower BOLD responses were found for steeper or shallower 1/f slopes (peak modulation: 0.59% for 1.25 vs. 0.31% for 2.25). To control for changing receptive field sizes, responses were also analyzed across multiple eccentricity bands in cortical surface space. For most eccentricity bands, BOLD responses were maximal for natural 1/f^α amplitude spectra, but importantly there was no difference in the BOLD response to grayscale stimuli and their corresponding thresholded counterparts. Since the thresholding of an image changes its measured 1/f slope (α) but not its fractal characteristics, this suggests that neuronal responses in early visual cortex are not strictly driven by spectral slope values (photometric properties) but rather their embedded geometric, fractal-like scaling properties.

Particle Pollution Estimation Based on Image Analysis

Exposure to fine particles can cause various diseases, and an easily accessible method to monitor the particles can help raise public awareness and reduce harmful exposures. Here we report a method to estimate PM air pollution based on analysis of a large number of outdoor images available for Beijing, Shanghai (China) and Phoenix (US). Six image features were extracted from the images, which were used, together with other relevant data, such as the position of the sun, date, time, geographic information and weather conditions, to predict PM_2.5 index. The results demonstrate that the image analysis method provides good prediction of PM_2.5 indexes, and different features have different significance levels in the prediction.

The absolute CBF response to activation is preserved during elevated perfusion: Implications for neurovascular coupling measures

Functional magnetic resonance imaging (fMRI) techniques in which the blood oxygenation level dependent (BOLD) and cerebral blood flow (CBF) response to a neural stimulus are measured, can be used to estimate the fractional increase in the cerebral metabolic rate of oxygen consumption (CMRO2) that accompanies evoked neural activity. A measure of neurovascular coupling is obtained from the ratio of fractional CBF and CMRO2 responses, defined as n, with the implicit assumption that relative rather than absolute changes in CBF and CMRO2 adequately characterise the flow-metabolism response to neural activity. The coupling parameter n is important in terms of its effect on the BOLD response, and as potential insight into the flow-metabolism relationship in both normal and pathological brain function. In 10 healthy human subjects, BOLD and CBF responses were measured to test the effect of baseline perfusion (modulated by a hypercapnia challenge) on the coupling parameter n during graded visual stimulation. A dual-echo pulsed arterial spin labelling (PASL) sequence provided absolute quantification of CBF in baseline and active states as well as relative BOLD signal changes, which were used to estimate CMRO2 responses to the graded visual stimulus. The absolute CBF response to the visual stimuli were constant across different baseline CBF levels, meaning the fractional CBF responses were reduced at the hyperperfused baseline state. For the graded visual stimuli, values of n were significantly reduced during hypercapnia induced hyperperfusion. Assuming the evoked neural responses to the visual stimuli are the same for both baseline CBF states, this result has implications for fMRI studies that aim to measure neurovascular coupling using relative changes in CBF. The coupling parameter n is sensitive to baseline CBF, which would confound its interpretation in fMRI studies where there may be significant differences in baseline perfusion between groups. The absolute change in CBF, as opposed to the change relative to baseline, may more closely match the underlying increase in neural activity in response to a stimulus.

Using an achiasmic human visual system to quantify the relationship between the fMRI BOLD signal and neural response

Achiasma in humans causes gross mis-wiring of the retinal-fugal projection, resulting in overlapped cortical representations of left and right visual hemifields. We show that in areas V1-V3 this overlap is due to two co-located but non-interacting populations of neurons, each with a receptive field serving only one hemifield. Importantly, the two populations share the same local vascular control, resulting in a unique organization useful for quantifying the relationship between neural and fMRI BOLD responses without direct measurement of neural activity. Specifically, we can non-invasively double local neural responses by stimulating both neuronal populations with identical stimuli presented symmetrically across the vertical meridian to both visual hemifields, versus one population by stimulating in one hemifield. Measurements from a series of such doubling experiments show that the amplitude of BOLD response is proportional to approximately 0.5 power of the underlying neural response. Reanalyzing published data shows that this inferred relationship is general.

Cue Integration: A Common Framework for Social Cognition and Physical Perception

Scientists examining how people understand other minds have long thought that this task must be something like how people perceive the physical world. This comparison has proven to be deeply generative, as models of physical perception and social cognition have evolved in parallel. In this article, I propose extending this classic analogy in a new direction by proposing cue integration as a common feature of social cognition and physical perception. When encountering complex social cues-which happens often-perceivers use multiple processes for understanding others' minds. Like physical senses (e.g., vision or audition), social cognitive processes have often been studied as though they operate in relative isolation. In the domain of physical perception, this assumption has broken down, following evidence that perception is instead characterized by pervasive integration of multisensory information. Such integration is, in turn, elegantly described by Bayesian inferential models. By adopting a similar cue integration framework, researchers can similarly understand and formally model the ways that we perceive others' minds based on complex social information.

A case for human systems neuroscience

Can the human brain itself serve as a model for a systems neuroscience approach to understanding the human brain? After all, how the brain is able to create the richness and complexity of human behavior is still largely mysterious. What better choice to study that complexity than to study it in humans? However, measurements of brain activity typically need to be made non-invasively which puts severe constraints on what can be learned about the internal workings of the brain. Our approach has been to use a combination of psychophysics in which we can use human behavioral flexibility to make quantitative measurements of behavior and link those through computational models to measurements of cortical activity through magnetic resonance imaging. In particular, we have tested various computational hypotheses about what neural mechanisms could account for behavioral enhancement with spatial attention (Pestilli et al., 2011). Resting both on quantitative measurements and considerations of what is known through animal models, we concluded that weighting of sensory signals by the magnitude of their response is a neural mechanism for efficient selection of sensory signals and consequent improvements in behavioral performance with attention. While animal models have many technical advantages over studying the brain in humans, we believe that human systems neuroscience should endeavor to validate, replicate and extend basic knowledge learned from animal model systems and thus form a bridge to understanding how the brain creates the complex and rich cognitive capacities of humans.

Spatial frequency processing in scene-selective cortical regions

Visual analysis begins with the parallel extraction of different attributes at different spatial frequencies. Low spatial frequencies (LSF) convey coarse information and are characterized by high luminance contrast, while high spatial frequencies (HSF) convey fine details and are characterized by low luminance contrast. In the present fMRI study, we examined how scene-selective regions-the parahippocampal place area (PPA), the retrosplenial cortex (RSC) and the occipital place area (OPA)-responded to spatial frequencies when contrast was either equalized or not equalized across spatial frequencies. Participants performed a categorization task on LSF, HSF and non-filtered scenes belonging to two different categories (indoors and outdoors). We either left contrast across scenes untouched, or equalized it using a root-mean-square contrast normalization. We found that when contrast remained unmodified, LSF and NF scenes elicited greater activation than HSF scenes in the PPA. However, when contrast was equalized across spatial frequencies, the PPA was selective to HFS. This suggests that PPA activity relies on an interaction between spatial frequency and contrast in scenes. In the RSC, LSF and NF elicited greater response than HSF scenes when contrast was not modified, while no effect of spatial frequencies appeared when contrast was equalized across filtered scenes, suggesting that the RSC is sensitive to high-contrast information. Finally, we observed selective activation of the OPA in response to HSF, irrespective of contrast manipulation. These results provide new insights into how scene-selective areas operate during scene processing.

Contrast response functions with wide-view stimuli in the human visual cortex

In this manuscript, using a novel wide-view visual presentation system that we developed for vision research and functional magnetic resonance imaging (fMRI), we studied contrast response functions in regions of the brain that are central and peripheral to the entire set of visual areas (V1, V2, V3, V3A, MT+), regions that have not been all investigated in previous vision research. Under the stimulus conditions which were 0-20 deg, 20-40 deg, and 40-60 deg eccentricity black-and-white checkerboard patterns, we measured the blood oxygenation level-dependent fMRI contrast response at five contrast levels (6, 12, 24, 48, and 96%) in the visual areas. On the basis of these data, the central and pericentral visual areas had low-contrast gain, whereas the peripheral visual areas had high-contrast gain. In addition, our results showed that the signals fundamentally shift during visual processing through posterior visual cortical areas (V1, V2, and V3) to superior visual cortical areas (V3A and MT+).

Larger neural responses produce BOLD signals that begin earlier in time

Functional MRI analyses commonly rely on the assumption that the temporal dynamics of hemodynamic response functions (HRFs) are independent of the amplitude of the neural signals that give rise to them. The validity of this assumption is particularly important for techniques that use fMRI to resolve sub-second timing distinctions between responses, in order to make inferences about the ordering of neural processes. Whether or not the detailed shape of the HRF is independent of neural response amplitude remains an open question, however. We performed experiments in which we measured responses in primary visual cortex (V1) to large, contrast-reversing checkerboards at a range of contrast levels, which should produce varying amounts of neural activity. Ten subjects (ages 22-52) were studied in each of two experiments using 3 Tesla scanners. We used rapid, 250 ms, temporal sampling (repetition time, or TR) and both short and long inter-stimulus interval (ISI) stimulus presentations. We tested for a systematic relationship between the onset of the HRF and its amplitude across conditions, and found a strong negative correlation between the two measures when stimuli were separated in time (long- and medium-ISI experiments, but not the short-ISI experiment). Thus, stimuli that produce larger neural responses, as indexed by HRF amplitude, also produced HRFs with shorter onsets. The relationship between amplitude and latency was strongest in voxels with lowest mean-normalized variance (i.e., parenchymal voxels). The onset differences observed in the longer-ISI experiments are likely attributable to mechanisms of neurovascular coupling, since they are substantially larger than reported differences in the onset of action potentials in V1 as a function of response amplitude.

Human cortical areas involved in perception of surface glossiness

Glossiness is the visual appearance of an object's surface as defined by its surface reflectance properties. Despite its ecological importance, little is known about the neural substrates underlying its perception. In this study, we performed the first human neuroimaging experiments that directly investigated where the processing of glossiness resides in the visual cortex. First, we investigated the cortical regions that were more activated by observing high glossiness compared with low glossiness, where the effects of simple luminance and luminance contrast were dissociated by controlling the illumination conditions (Experiment 1). As cortical regions that may be related to the processing of glossiness, V2, V3, hV4, VO-1, VO-2, collateral sulcus (CoS), LO-1, and V3A/B were identified, which also showed significant correlation with the perceived level of glossiness. This result is consistent with the recent monkey studies that identified selective neural response to glossiness in the ventral visual pathway, except for V3A/B in the dorsal visual pathway, whose involvement in the processing of glossiness could be specific to the human visual system. Second, we investigated the cortical regions that were modulated by selective attention to glossiness (Experiment 2). The visual areas that showed higher activation to attention to glossiness than that to either form or orientation were identified as right hV4, right VO-2, and right V3A/B, which were commonly identified in Experiment 1. The results indicate that these commonly identified visual areas in the human visual cortex may play important roles in glossiness perception.

Minimizing biases in estimating the reorganization of human visual areas with BOLD retinotopic mapping

There is substantial interest in using functional magnetic resonance imaging (fMRI) retinotopic mapping techniques to examine reorganization of the occipital cortex after vision loss in humans and nonhuman primates. However, previous reports suggest that standard phase encoding and the more recent population Receptive Field (pRF) techniques give biased estimates of retinotopic maps near the boundaries of retinal or cortical scotomas. Here we examine the sources of this bias and show how it can be minimized with a simple modification of the pRF method. In normally sighted subjects, we measured fMRI responses to a stimulus simulating a foveal scotoma; we found that unbiased retinotopic map estimates can be obtained in early visual areas, as long as the pRF fitting algorithm takes the scotoma into account and a randomized "multifocal" stimulus sequence is used.

BOLD human responses to chromatic spatial features

Animal physiological and human psychophysical studies suggest that an early step in visual processing involves the detection and identification of features such as lines and edges, by neural mechanisms with even- and odd-symmetric receptive fields. Functional imaging studies also demonstrate mechanisms with even- and odd-receptive fields in early visual areas, in response to luminance-modulated stimuli. In this study we measured fMRI BOLD responses to 2-D stimuli composed of only even or only odd symmetric features, and to an amplitude-matched random noise control, modulated in red-green equiluminant colour contrast. All these stimuli had identical power but different phase spectra, either highly congruent (even or odd symmetry stimuli) or random (noise). At equiluminance, V1 BOLD activity showed no preference between congruent- and random-phase stimuli, as well as no preference between even and odd symmetric stimuli. Areas higher in the visual hierarchy, both along the dorsal pathway (caudal part of the intraparietal sulcus, dorsal LO and V3A) and the ventral pathway (V4), responded preferentially to odd symmetry over even symmetry stimuli, and to congruent over random phase stimuli. Interestingly, V1 showed an equal increase in BOLD activity at each alternation between stimuli of different symmetry, suggesting the existence of specialised mechanisms for the detection of edges and lines such as even- and odd-chromatic receptive fields. Overall the results indicate a high selectivity of colour-selective neurons to spatial phase along both the dorsal and the ventral pathways in humans.

Different spatial frequency bands selectively signal for natural image statistics in the early visual system

Early visual evoked potentials (VEPs) measured in humans have recently been observed to be modulated by the image statistics of natural scene imagery. Specifically, the early VEP is dominated by a strong positivity when participants view minimally complex natural scene imagery, with the magnitude of that component being modulated by luminance contrast differences across spatial frequency (i.e., the slope of the amplitude spectrum). For scenes high in structural complexity, the early VEP is dominated by a prominent negativity that exhibits little dependency on luminance contrast. However, since natural scene imagery is broad band in terms of spatial frequency, it is not known whether the above-mentioned modulation results from a complex interaction within or between the early neural processes tuned to different bands of spatial frequency. Here, we sought to address this question by measuring early VEPs (specifically, the C1, P1, and N1 components) while human participants viewed natural scene imagery that was filtered to contain specific bands of spatial frequency information. The results show that the C1 component is largely unmodulated by the luminance statistics of natural scene imagery (being only measurable when such stimuli were made to contain high spatial frequencies). The P1 and N1, on the other hand, were observed to exhibit strong spatial frequency-dependent modulation to the luminance statistics of natural scene imagery. The results therefore suggest that the dependency of early VEPs on natural image statistics results from an interaction between the early neural processes tuned to different bands of spatial frequency.

Hippocampal involvement in processing of indistinct visual motion stimuli

Perception of known patterns results from the interaction of current sensory input with existing internal representations. It is unclear how perceptual and mnemonic processes interact when visual input is dynamic and structured such that it does not allow immediate recognition of obvious objects and forms. In an fMRI experiment, meaningful visual motion stimuli depicting movement through a virtual tunnel and indistinct, meaningless visual motion stimuli, achieved through phase scrambling of the same stimuli, were presented while participants performed an optic flow task. We found that our indistinct visual motion stimuli evoked hippocampal activation, whereas the corresponding meaningful stimuli did not. Using independent component analysis, we were able to demonstrate a functional connectivity between the hippocampus and early visual areas, with increased activity for indistinct stimuli. In a second experiment, we used the same stimuli to test whether our results depended on the participants' task. We found task-independent bilateral hippocampal activation in response to indistinct motion stimuli. For both experiments, psychophysiological interaction analysis revealed a coupling from posterior hippocampus to dorsal visuospatial and ventral visual object processing areas when viewing indistinct stimuli. These results indicate a close functional link between stimulus-dependent perceptual and mnemonic processes. The observed pattern of hippocampal functional connectivity, in the absence of an explicit memory task, suggests that cortical-hippocampal networks are recruited when visual stimuli are temporally uncertain and do not immediately reveal a clear meaning.

ENCODING AND DECODING V1 FMRI RESPONSES TO NATURAL IMAGES WITH SPARSE NONPARAMETRIC MODELS

Functional MRI (fMRI) has become the most common method for investigating the human brain. However, fMRI data present some complications for statistical analysis and modeling. One recently developed approach to these data focuses on estimation of computational encoding models that describe how stimuli are transformed into brain activity measured in individual voxels. Here we aim at building encoding models for fMRI signals recorded in the primary visual cortex of the human brain. We use residual analyses to reveal systematic nonlinearity across voxels not taken into account by previous models. We then show how a sparse nonparametric method [bJ. Roy. Statist. Soc. Ser. B71 (2009b) 1009-1030] can be used together with correlation screening to estimate nonlinear encoding models effectively. Our approach produces encoding models that predict about 25% more accurately than models estimated using other methods [Nature452 (2008a) 352-355]. The estimated nonlinearity impacts the inferred properties of individual voxels, and it has a plausible biological interpretation. One benefit of quantitative encoding models is that estimated models can be used to decode brain activity, in order to identify which specific image was seen by an observer. Encoding models estimated by our approach also improve such image identification by about 12% when the correct image is one of 11,500 possible images.

Attention extracts signal in external noise: a BOLD fMRI study

On the basis of results from behavioral studies that spatial attention improves the exclusion of external noise in the target region, we predicted that attending to a spatial region would reduce the impact of external noise on the BOLD response in corresponding cortical areas, seen as reduced BOLD responses in conditions with large amounts of external noise but relatively low signal, and increased dynamic range of the BOLD response to variations in signal contrast. We found that, in the presence of external noise, covert attention reduced the trial-by-trial BOLD response by 15.5-18.9% in low signal contrast conditions in V1. It also increased the BOLD dynamic range in V1, V2, V3, V3A/B, and V4 by a factor of at least three. Overall, covert attention reduced the impact of external noise by about 73-85% in these early visual areas. It also increased the contrast gain by a factor of 2.6-3.8.

Dissociating the effect of noise on sensory processing and overall decision difficulty

It has been proposed that perceptual decision making involves a task-difficulty component, which detects perceptual uncertainty and guides allocation of attentional resources. It is thought to take place immediately after the early extraction of sensory information and is specifically reflected in a positive component of the event related potentials, peaking at ∼ 220 ms after stimulus onset. However, in the previous research, neural processes associated with the monitoring of overall task difficulty were confounded by those associated with the increased sensory processing demands as a result of adding noise to the stimuli. Here we dissociated the effect of phase noise on sensory processing and overall decision difficulty using a face gender categorization task. Task difficulty was manipulated either by adding noise to the stimuli or by adjusting the female/male characteristics of the face images. We found that it is the presence of noise and not the increased overall task difficulty that affects the electrophysiological responses in the first 300 ms following stimulus onset in humans. Furthermore, we also showed that processing of phase-randomized as compared to intact faces is associated with increased fMRI responses in the lateral occipital cortex. These results revealed that noise-induced modulation of the early electrophysiological responses reflects increased visual cortical processing demands and thus failed to provide support for a task-difficulty component taking place between the early sensory processing and the later sensory accumulation stages of perceptual decision making.

Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech

Humans are remarkably adept at understanding speech, even when it is contaminated by noise. Multisensory integration may explain some of this ability: combining independent information from the auditory modality (vocalizations) and the visual modality (mouth movements) reduces noise and increases accuracy. Converging evidence suggests that the superior temporal sulcus (STS) is a critical brain area for multisensory integration, but little is known about its role in the perception of noisy speech. Behavioral studies have shown that perceptual judgments are weighted by the reliability of the sensory modality: more reliable modalities are weighted more strongly, even if the reliability changes rapidly. We hypothesized that changes in the functional connectivity of STS with auditory and visual cortex could provide a neural mechanism for perceptual reliability weighting. To test this idea, we performed five blood oxygenation level-dependent functional magnetic resonance imaging and behavioral experiments in 34 healthy subjects. We found increased functional connectivity between the STS and auditory cortex when the auditory modality was more reliable (less noisy) and increased functional connectivity between the STS and visual cortex when the visual modality was more reliable, even when the reliability changed rapidly during presentation of successive words. This finding matched the results of a behavioral experiment in which the perception of incongruent audiovisual syllables was biased toward the more reliable modality, even with rapidly changing reliability. Changes in STS functional connectivity may be an important neural mechanism underlying the perception of noisy speech.

From spatial frequency contrast to edge preponderance: the differential modulation of early visual evoked potentials by natural scene stimuli

The contrast response function of early visual evoked potentials elicited by sinusoidal gratings is known to exhibit characteristic potentials closely associated with the processes of parvocellular and magnocellular pathways. Specifically, the N1 component has been linked with parvocellular processes, while the P1 component has been linked with magnocellular processes. However, little is known regarding the response properties of the N1 and P1 components during the processing and encoding of complex (i.e., broadband) stimuli such as natural scenes. Here, we examine how established physical characteristics of natural scene imagery modulate the N1 and P1 components in humans by providing a systematic investigation of component modulation as visual stimuli are gradually built up from simple sinusoidal gratings to highly complex natural scene imagery. The results suggest that the relative dominance in signal output of the N1 and P1 components is dependent on spatial frequency (SF) luminance contrast for simple stimuli up to natural scene imagery possessing few edges. However, such a dependency shifts to a dominant N1 signal for natural scenes possessing abundant edge content and operates independently of SF luminance contrast.

Lower-level stimulus features strongly influence responses in the fusiform face area

An intriguing region of human visual cortex (the fusiform face area; FFA) responds selectively to faces as a general higher-order stimulus category. However, the potential role of lower-order stimulus properties in FFA remains incompletely understood. To clarify those lower-level influences, we measured FFA responses to independent variation in 4 lower-level stimulus dimensions using standardized face stimuli and functional Magnetic Resonance Imaging (fMRI). These dimensions were size, position, contrast, and rotation in depth (viewpoint). We found that FFA responses were strongly influenced by variations in each of these image dimensions; that is, FFA responses were not "invariant" to any of them. Moreover, all FFA response functions were highly correlated with V1 responses (r = 0.95-0.99). As in V1, FFA responses could be accurately modeled as a combination of responses to 1) local contrast plus 2) the cortical magnification factor. In some measurements (e.g., face size or a combinations of multiple cues), the lower-level variations dominated the range of FFA responses. Manipulation of lower-level stimulus parameters could even change the category preference of FFA from "face selective" to "object selective." Altogether, these results emphasize that a significant portion of the FFA response reflects lower-level visual responses.

Spatiotemporal phase-scrambling increases visual cortex activity

The hemodynamic response of the visual cortex to continuously moving spatial stimuli of virtual tunnels and phase-scrambled versions thereof was examined using functional magnetic resonance imaging. Earlier functional magnetic resonance imaging studies found either no difference or less early visual cortex (VC) activation when presenting normal versus phase-manipulated static natural images. Here we describe an increase in VC activation while viewing phase-scrambled films compared with normal films, although basic image statistics and average local flow were the same. The normal films, in contrast, resulted in an increased lateral occipital and precuneus activity sparing VC. In summary, our results show that earlier findings for scrambling of static images no longer hold for spatiotemporal stimuli.

Inter-ocular contrast normalization in human visual cortex

The brain combines visual information from the two eyes and forms a coherent percept, even when inputs to the eyes are different. However, it is not clear how inputs from the two eyes are combined in visual cortex. We measured fMRI responses to single gratings presented monocularly, or pairs of gratings presented monocularly or dichoptically with several combinations of contrasts. Gratings had either the same orientation or orthogonal orientations (i.e., plaids). Observers performed a demanding task at fixation to minimize top-down modulation of the stimulus-evoked responses. Dichoptic presentation of compatible gratings (same orientation) evoked greater activity than monocular presentation of a single grating only when contrast was low (<10%). A model that assumes linear summation of activity from each eye failed to explain binocular responses at 10% contrast or higher. However, a model with binocular contrast normalization, such that activity from each eye reduced the gain for the other eye, fitted the results very well. Dichoptic presentation of orthogonal gratings evoked greater activity than monocular presentation of a single grating for all contrasts. However, activity evoked by dichoptic plaids was equal to that evoked by monocular plaids. Introducing an onset asynchrony (stimulating one eye 500 ms before the other, which under attentive vision results in flash suppression) had no impact on the results; the responses to dichoptic and monocular plaids were again equal. We conclude that when attention is diverted, inter-ocular suppression in V1 can be explained by a normalization model in which the mutual suppression between orthogonal orientations does not depend on the eye of origin, nor on the onset times, and cross-orientation suppression is weaker than inter-ocular (same orientation) suppression.

Adaptive changes in visual cortex following prolonged contrast reduction

How does prolonged reduction in retinal-image contrast affect visual-contrast coding? Recent evidence indicates that some forms of long-term visual deprivation result in compensatory perceptual and neural changes in the adult visual pathway. It has not been established whether changes due to contrast adaptation are best characterized as "contrast gain" or "response gain." We present a theoretical rationale for predicting that adaptation to long-term contrast reduction should result in response gain. To test this hypothesis, normally sighted subjects adapted for four hours by viewing their environment through contrast-reducing goggles. During the adaptation period, the subjects went about their usual daily activities. Subjects' contrast-discrimination thresholds and fMRI BOLD responses in cortical areas V1 and V2 were obtained before and after adaptation. Following adaptation, we observed a significant decrease in contrast-discrimination thresholds, and significant increase in BOLD responses in V1 and V2. The observed interocular transfer of the adaptation effect suggests that the adaptation has a cortical origin. These results reveal a new kind of adaptability of the adult visual cortex, an adjustment in the gain of the contrast-response in the presence of a reduced range of stimulus contrasts, which is consistent with a response-gain mechanism. The adaptation appears to be compensatory, such that the precision of contrast coding is improved for low retinal-image contrasts.