Laminar Differences in Responses to Naturalistic Texture in Macaque V1 and V2

Receptive-field nonlinearities in primary auditory cortex: a comparative perspective

Cortical processing of auditory information can be affected by interspecies differences as well as brain states. Here we compare multifeature spectro-temporal receptive fields (STRFs) and associated input/output functions or nonlinearities (NLs) of neurons in primary auditory cortex (AC) of four mammalian species. Single-unit recordings were performed in awake animals (female squirrel monkeys, female, and male mice) and anesthetized animals (female squirrel monkeys, rats, and cats). Neuronal responses were modeled as consisting of two STRFs and their associated NLs. The NLs for the STRF with the highest information content show a broad distribution between linear and quadratic forms. In awake animals, we find a higher percentage of quadratic-like NLs as opposed to more linear NLs in anesthetized animals. Moderate sex differences of the shape of NLs were observed between male and female unanesthetized mice. This indicates that the core AC possesses a rich variety of potential computations, particularly in awake animals, suggesting that multiple computational algorithms are at play to enable the auditory system's robust recognition of auditory events.

Neuronal and behavioral responses to naturalistic texture images in macaque monkeys

The visual world is richly adorned with texture, which can serve to delineate important elements of natural scenes. In anesthetized macaque monkeys, selectivity for the statistical features of natural texture is weak in V1, but substantial in V2, suggesting that neuronal activity in V2 might directly support texture perception. To test this, we investigated the relation between single cell activity in macaque V1 and V2 and simultaneously measured behavioral judgments of texture. We generated stimuli along a continuum between naturalistic texture and phase-randomized noise and trained two macaque monkeys to judge whether a sample texture more closely resembled one or the other extreme. Analysis of responses revealed that individual V1 and V2 neurons carried much less information about texture naturalness than behavioral reports. However, the sensitivity of V2 neurons, especially those preferring naturalistic textures, was significantly closer to that of behavior compared with V1. The firing of both V1 and V2 neurons predicted perceptual choices in response to repeated presentations of the same ambiguous stimulus in one monkey, despite low individual neural sensitivity. However, neither population predicted choice in the second monkey. We conclude that neural responses supporting texture perception likely continue to develop downstream of V2. Further, combined with neural data recorded while the same two monkeys performed an orientation discrimination task, our results demonstrate that choice-correlated neural activity in early sensory cortex is unstable across observers and tasks, untethered from neuronal sensitivity, and thus unlikely to reflect a critical aspect of the formation of perceptual decisions.

Significance statement

As visual signals propagate along the cortical hierarchy, they encode increasingly complex aspects of the sensory environment and likely have a more direct relationship with perceptual experience. We replicate and extend previous results from anesthetized monkeys differentiating the selectivity of neurons along the first step in cortical vision from area V1 to V2. However, our results further complicate efforts to establish neural signatures that reveal the relationship between perception and the neuronal activity of sensory populations. We find that choice-correlated activity in V1 and V2 is unstable across different observers and tasks, and also untethered from neuronal sensitivity and other features of nonsensory response modulation.

Top-down influence of areas 21a and 7 differently affects the surround suppression of V1 neurons in cats

Surround suppression (SS) is a phenomenon whereby a neuron's response to stimuli in its central receptive field (cRF) is suppressed by stimuli extending to its surround receptive field (sRF). Recent evidence show that top-down influence contributed to SS in the primary visual cortex (V1). However, how the top-down influence from different high-level cortical areas affects SS in V1 has not been comparatively observed. The present study applied transcranial direct current stimulation (tDCS) to modulate the neural activity in area 21a (A21a) and area 7 (A7) of cats and examined the changes in the cRF and sRF of V1 neurons. We found that anode-tDCS at A21a reduced V1 neurons' cRF size and increased their response to visual stimuli in cRF, causing an improved SS strength. By contrast, anode-tDCS at A7 increased V1 neurons' sRF size and response to stimuli in cRF, also enhancing the SS. Modeling analysis based on DoG function indicated that the increased SS of V1 neurons after anode-tDCS at A21a could be explained by a center-only mechanism, whereas the improved SS after anode-tDCS at A7 might be mediated through a combined center and surround mechanism. In conclusion, A21a and A7 may affect the SS of V1 neurons through different mechanisms.

Spatiotemporal cortical dynamics for visual scene processing as revealed by EEG decoding

The human visual system rapidly recognizes the categories and global properties of complex natural scenes. The present study investigated the spatiotemporal dynamics of neural signals involved in visual scene processing using electroencephalography (EEG) decoding. We recorded visual evoked potentials from 11 human observers for 232 natural scenes, each of which belonged to one of 13 natural scene categories (e.g., a bedroom or open country) and had three global properties (naturalness, openness, and roughness). We trained a deep convolutional classification model of the natural scene categories and global properties using EEGNet. Having confirmed that the model successfully classified natural scene categories and the three global properties, we applied Grad-CAM to the EEGNet model to visualize the EEG channels and time points that contributed to the classification. The analysis showed that EEG signals in the occipital electrodes at short latencies (approximately 80 ~ ms) contributed to the classifications, whereas those in the frontal electrodes at relatively long latencies (200 ~ ms) contributed to the classification of naturalness and the individual scene category. These results suggest that different global properties are encoded in different cortical areas and with different timings, and that the combination of the EEGNet model and Grad-CAM can be a tool to investigate both temporal and spatial distribution of natural scene processing in the human brain.

Photorealistic Reconstruction of Visual Texture From EEG Signals

Recent advances in brain decoding have made it possible to classify image categories based on neural activity. Increasing numbers of studies have further attempted to reconstruct the image itself. However, because images of objects and scenes inherently involve spatial layout information, the reconstruction usually requires retinotopically organized neural data with high spatial resolution, such as fMRI signals. In contrast, spatial layout does not matter in the perception of "texture," which is known to be represented as spatially global image statistics in the visual cortex. This property of "texture" enables us to reconstruct the perceived image from EEG signals, which have a low spatial resolution. Here, we propose an MVAE-based approach for reconstructing texture images from visual evoked potentials measured from observers viewing natural textures such as the textures of various surfaces and object ensembles. This approach allowed us to reconstruct images that perceptually resemble the original textures with a photographic appearance. The present approach can be used as a method for decoding the highly detailed "impression" of sensory stimuli from brain activity.

Conceptual representation of real-world surface material: Early integration with linguistic-labels indicated in the N400-component

Research in perception in the visual and auditory domains has traditionally focused on investigating highly controlled artificial stimulus material. However, a key feature of our perceptual system is the ease with which the input of a wide set of naturalistic co-occurring information is dealt with. This study investigated whether, during perception of real-world surface material, a conceptual representation is built that has the potential to interact with a linguistic description of the material directly. Short sentences were presented (e.g., This surface is smooth) followed by a matching or mismatching picture of a real-world surface material. The results showed early cross-modal integration effects during material surface perception in an N400-like potential, originating approximately 280 ms after stimulus presentation. Overall, these findings suggest a rather early influence of linguistic information on material perception, suggesting that in line with object representation, real-world materials are represented in the brain in a format that allows interaction with non-visual information.

Opposing effects of selectivity and invariance in peripheral vision

Sensory processing necessitates discarding some information in service of preserving and reformatting more behaviorally relevant information. Sensory neurons seem to achieve this by responding selectively to particular combinations of features in their inputs, while averaging over or ignoring irrelevant combinations. Here, we expose the perceptual implications of this tradeoff between selectivity and invariance, using stimuli and tasks that explicitly reveal their opposing effects on discrimination performance. We generate texture stimuli with statistics derived from natural photographs, and ask observers to perform two different tasks: Discrimination between images drawn from families with different statistics, and discrimination between image samples with identical statistics. For both tasks, the performance of an ideal observer improves with stimulus size. In contrast, humans become better at family discrimination but worse at sample discrimination. We demonstrate through simulations that these behaviors arise naturally in an observer model that relies on a common set of physiologically plausible local statistical measurements for both tasks.

Analysis and Synthesis of Natural Texture Perception From Visual Evoked Potentials

The primate visual system analyzes statistical information in natural images and uses it for the immediate perception of scenes, objects, and surface materials. To investigate the dynamical encoding of image statistics in the human brain, we measured visual evoked potentials (VEPs) for 166 natural textures and their synthetic versions, and performed a reverse-correlation analysis of the VEPs and representative texture statistics of the image. The analysis revealed occipital VEP components strongly correlated with particular texture statistics. VEPs correlated with low-level statistics, such as subband SDs, emerged rapidly from 100 to 250 ms in a spatial frequency dependent manner. VEPs correlated with higher-order statistics, such as subband kurtosis and cross-band correlations, were observed at slightly later times. Moreover, these robust correlations enabled us to inversely estimate texture statistics from VEP signals via linear regression and to reconstruct texture images that appear similar to those synthesized with the original statistics. Additionally, we found significant differences in VEPs at 200-300 ms between some natural textures and their Portilla-Simoncelli (PS) synthesized versions, even though they shared almost identical texture statistics. This differential VEP was related to the perceptual "unnaturalness" of PS-synthesized textures. These results suggest that the visual cortex rapidly encodes image statistics hidden in natural textures specifically enough to predict the visual appearance of a texture, while it also represents high-level information beyond image statistics, and that electroencephalography can be used to decode these cortical signals.

GABAergic and non-GABAergic subpopulations of Kv3.1b-expressing neurons in macaque V2 and MT: laminar distributions and proportion of total neuronal population

The Kv3.1b potassium channel subunit, which facilitates the fast-spiking phenotype characteristic of parvalbumin (PV)-expressing inhibitory interneurons, is also expressed by subpopulations of excitatory neurons in macaque cortex. We have previously shown that V1 neurons expressing Kv3.1b but not PV or GABA were largely concentrated within layers 4Cα and 4B of V1, suggesting laminar or pathway specificity. In the current study, the distribution and pattern of co-immunoreactivity of GABA, PV, and Kv3.1b across layers in extrastriate cortical areas V2 and MT of the macaque monkey were measured using the same triple immunofluorescence labeling, confocal microscopy, and partially automated cell-counting strategies used in V1. For comparison, densities of the overall cell and neuronal populations were also measured for each layer of V2 and MT using tissue sections immunofluorescence labeled for the pan-neuronal marker NeuN. GABAergic neurons accounted for 14% of the total neuronal population in V2 and 25% in MT. Neurons expressing Kv3.1b but neither GABA nor PV were present in both areas. This subpopulation was most prevalent in the lowest subcompartment of layer 3, comprising 5% of the total neuronal population in layer 3C of both areas, and 41% and 36% of all Kv3.1b+ neurons in this layer in V2 and MT, respectively. The prevalence and laminar distribution of this subpopulation were remarkably consistent between V2 and MT and showed a striking similarity to the patterns observed previously in V1, suggesting a common contribution to the cortical circuit across areas.