Organization of local horizontal functional interactions between neurons in the inferior temporal cortex of macaque monkeys

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

Correlation between neural responses and human perception in figure-ground segregation

Segmentation of a natural scene into objects (figures) and background (ground) is one of crucial functions for object recognition and scene understanding. Recent studies have investigated neural mechanisms underlying figure-ground (FG) segregation and reported neural modulation to FG in the intermediate-level visual area, V4, of macaque monkeys (FG neurons). However, whether FG neurons contribute to the perception of FG segregation has not been clarified. To examine the contribution of FG neurons, we examined the correlations between perceptual consistency (PC), which quantified perceptual ambiguity in FG determination, and the reliability of neural signals in response to FG. First, we evaluated PCs for the images that were used in the previous neural recording in V4; specifically, we measured how consistently FG can be determined across trials and participants for each stimulus. The PCs were widely distributed, so that we identified the ambiguity in FG segregation for each stimulus. Next, we analyzed the correlation between the PCs and the reliability of neural modulation to FG. We found that the stimuli with higher PCs evoked more consistent and greater modulation in the responses of single neurons than those with lower PCs. Since perception is expected to show a greater correlation with responses of neural population compared to those of single neurons, we examined the correlation between the PCs and the consistency of the population responses in FG determination. Stimuli with higher PCs evoked higher population consistency than those with lower PCs. Finally, we analyzed the correlation between the PCs and neural latencies in FG modulation. We found that the stimuli with higher PCs showed shorter reaction times in FG perception and evoked shorter modulation latencies in FG neurons. These results indicate that the responses of FG neurons recorded from macaque monkeys show significant correlations with human FG perception, suggesting that V4 neurons with FG-dependent responses contribute to the perception of FG segregation.

Correspondence between Monkey Visual Cortices and Layers of a Saliency Map Model Based on a Deep Convolutional Neural Network for Representations of Natural Images

Attentional selection is a function that allocates the brain’s computational resources to the most important part of a visual scene at a specific moment. Saliency map models have been proposed as computational models to predict attentional selection within a spatial location. Recent saliency map models based on deep convolutional neural networks (DCNNs) exhibit the highest performance for predicting the location of attentional selection and human gaze, which reflect overt attention. Trained DCNNs potentially provide insight into the perceptual mechanisms of biological visual systems. However, the relationship between artificial and neural representations used for determining attentional selection and gaze location remains unknown. To understand the mechanism underlying saliency map models based on DCNNs and the neural system of attentional selection, we investigated the correspondence between layers of a DCNN saliency map model and monkey visual areas for natural image representations. We compared the characteristics of the responses in each layer of the model with those of the neural representation in the primary visual (V1), intermediate visual (V4), and inferior temporal (IT) cortices. Regardless of the DCNN layer level, the characteristics of the responses were consistent with that of the neural representation in V1. We found marked peaks of correspondence between V1 and the early level and higher-intermediate-level layers of the model. These results provide insight into the mechanism of the trained DCNN saliency map model and suggest that the neural representations in V1 play an important role in computing the saliency that mediates attentional selection, which supports the V1 saliency hypothesis.

Population coding of figure and ground in natural image patches by V4 neurons

Segmentation of a natural scene into objects and background is a fundamental but challenging task for recognizing objects. Investigating intermediate-level visual cortical areas with a focus on local information is a crucial step towards understanding the formation of the cortical representations of figure and ground. We examined the activity of a population of macaque V4 neurons during the presentation of natural image patches and their respective variations. The natural image patches were optimized to exclude the influence of global context but included various characteristics of local stimulus. Around one fourth of the patch-responsive V4 neurons exhibited significant modulation of firing activity that was dependent on the positional relation between the figural region of the stimulus and the classical receptive field of the neuron. However, the individual neurons showed low consistency in figure-ground modulation across a variety of image patches (55–62%), indicating that individual neurons were capable of correctly signaling figure and ground only for a limited number of stimuli. We examined whether integration of the activity of multiple neurons enabled higher consistency across a variety of natural patches by training a support vector machine to classify figure and ground of the stimuli from the population firing activity. The integration of the activity of a few tens of neurons yielded discrimination accuracy much greater than that of single neurons (up to 85%), suggesting a crucial role of population coding for figure-ground discrimination in natural images.

Network Anisotropy Trumps Noise for Efficient Object Coding in Macaque Inferior Temporal Cortex

How neuronal ensembles compute information is actively studied in early visual cortex. Much less is known about how local ensembles function in inferior temporal (IT) cortex, the last stage of the ventral visual pathway that supports visual recognition. Previous reports suggested that nearby neurons carry information mostly independently, supporting efficient processing (Barlow, 1961). However, others postulate that noise covariation effects may depend on network anisotropy/homogeneity and on how the covariation relates to representation. Do slow trial-by-trial noise covariations increase or decrease IT's object coding capability, how does encoding capability relate to correlational structure (i.e., the spatial pattern of signal and noise redundancy/homogeneity across neurons), and does knowledge of correlational structure matter for decoding? We recorded simultaneously from ∼80 spiking neurons in ∼1 mm(3) of macaque IT under light neurolept anesthesia. Noise correlations were stronger for neurons with correlated tuning, and noise covariations reduced object encoding capability, including generalization across object pose and illumination. Knowledge of noise covariations did not lead to better decoding performance. However, knowledge of anisotropy/homogeneity improved encoding and decoding efficiency by reducing the number of neurons needed to reach a given performance level. Such correlated neurons were found mostly in supragranular and infragranular layers, supporting theories that link recurrent circuitry to manifold representation. These results suggest that redundancy benefits manifold learning of complex high-dimensional information and that subsets of neurons may be more immune to noise covariation than others.

SIGNIFICANCE STATEMENT

How noise affects neuronal population coding is poorly understood. By sampling densely from local populations supporting visual object recognition, we show that recurrent circuitry supports useful representations and that subsets of neurons may be more immune to noise covariation than others.

Correlated activity supports efficient cortical processing

Visual recognition is a computational challenge that is thought to occur via efficient coding. An important concept is sparseness, a measure of coding efficiency. The prevailing view is that sparseness supports efficiency by minimizing redundancy and correlations in spiking populations. Yet, we recently reported that "choristers", neurons that behave more similarly (have correlated stimulus preferences and spontaneous coincident spiking), carry more generalizable object information than uncorrelated neurons ("soloists") in macaque inferior temporal (IT) cortex. The rarity of choristers (as low as 6% of IT neurons) indicates that they were likely missed in previous studies. Here, we report that correlation strength is distinct from sparseness (choristers are not simply broadly tuned neurons), that choristers are located in non-granular output layers, and that correlated activity predicts human visual search efficiency. These counterintuitive results suggest that a redundant correlational structure supports efficient processing and behavior.