Coarse-to-fine processing drives the efficient coding of natural scenes in mouse visual cortex

Temporal dynamics of energy-efficient coding in mouse primary visual cortex

Sparse coding enables cortical populations to represent sensory inputs efficiently, yet its temporal dynamics remain poorly understood. Consistent with theoretical predictions, we show that stimulus onset triggers broad cortical activation, initially reducing sparseness and increasing mutual information. Subsequently, competitive interactions sustain mutual information as activity declines and sparseness increases. Notably, coding efficiency, defined as the ratio of mutual information to metabolic cost, progressively increases, demonstrating the dynamic optimization of sensory representations.

Functional organization and natural scene responses across mouse visual cortical areas revealed with encoding manifolds

A challenge in sensory neuroscience is understanding how populations of neurons operate in concert to represent diverse stimuli. To meet this challenge, we have created "encoding manifolds" that reveal the overall responses of brain areas to diverse stimuli with the resolution of individual neurons and their response dynamics. Here we use encoding manifold to compare the population-level encoding of primary visual cortex (VISp) with five higher visual areas (VISam, VISal, VISpm, VISlm, and VISrl). We used data from the Allen Institute Visual Coding-Neuropixels dataset from the mouse. We show that the encoding manifold topology computed only from responses to grating stimuli is continuous, for V1 and for higher visual areas, with smooth coordinates spanning it that include orientation selectivity and firing-rate magnitude. Surprisingly, the manifolds for each visual area revealed novel relationships between how natural scenes are encoded relative to static gratings-a relationship that was conserved across visual areas. Namely, neurons preferring natural scenes preferred either low or high spatial frequency gratings, but not intermediate ones. Analyzing responses by cortical layer reveals a preference for gratings concentrated in layer 6, whereas preferences for natural scenes tended to be higher in layers 2/3 and 4. The results reveal how machine learning approaches can be used to organize and visualize the structure of sensory coding, thereby revealing novel relationships within and across brain areas and sensory stimuli.

Natural visual behavior and active sensing in the mouse

In the natural world, animals use vision for a wide variety of behaviors not reflected in most laboratory paradigms. Although mice have low-acuity vision, they use their vision for many natural behaviors, including predator avoidance, prey capture, and navigation. They also perform active sensing, moving their head and eyes to achieve behavioral goals and acquire visual information. These aspects of natural vision result in visual inputs and corresponding behavioral outputs that are outside the range of conventional vision studies but are essential aspects of visual function. Here, we review recent studies in mice that have tapped into natural behavior and active sensing to reveal the computational logic of neural circuits for vision.

Nonuniform and pathway-specific laminar processing of spatial frequencies in the primary visual cortex of primates

The neocortex comprises six cortical layers that play a crucial role in information processing; however, it remains unclear whether laminar processing is consistent across all regions within a single cortex. In this study, we demonstrate diverse laminar response patterns in the primary visual cortex (V1) of three male macaque monkeys when exposed to visual stimuli at different spatial frequencies (SFs). These response patterns can be categorized into two groups. One group exhibit suppressed responses in the output layers for all SFs, while the other type shows amplified responses specifically at high SFs. Further analysis suggests that both magnocellular (M) and parvocellular (P) pathways contribute to the suppressive effect through feedforward mechanisms, whereas amplification is specific to local recurrent mechanisms within the parvocellular pathway. These findings highlight the non-uniform distribution of neural mechanisms involved in laminar processing and emphasize how pathway-specific amplification selectively enhances representations of high-SF information in primate V1.

A dynamic sequence of visual processing initiated by gaze shifts

Animals move their head and eyes as they explore the visual scene. Neural correlates of these movements have been found in rodent primary visual cortex (V1), but their sources and computational roles are unclear. We addressed this by combining head and eye movement measurements with neural recordings in freely moving mice. V1 neurons responded primarily to gaze shifts, where head movements are accompanied by saccadic eye movements, rather than to head movements where compensatory eye movements stabilize gaze. A variety of activity patterns followed gaze shifts and together these formed a temporal sequence that was absent in darkness. Gaze-shift responses resembled those evoked by sequentially flashed stimuli, suggesting a large component corresponds to onset of new visual input. Notably, neurons responded in a sequence that matches their spatial frequency bias, consistent with coarse-to-fine processing. Recordings in freely gazing marmosets revealed a similar sequence following saccades, also aligned to spatial frequency preference. Our results demonstrate that active vision in both mice and marmosets consists of a dynamic temporal sequence of neural activity associated with visual sampling.

A large-scale fMRI dataset for the visual processing of naturalistic scenes

One ultimate goal of visual neuroscience is to understand how the brain processes visual stimuli encountered in the natural environment. Achieving this goal requires records of brain responses under massive amounts of naturalistic stimuli. Although the scientific community has put a lot of effort into collecting large-scale functional magnetic resonance imaging (fMRI) data under naturalistic stimuli, more naturalistic fMRI datasets are still urgently needed. We present here the Natural Object Dataset (NOD), a large-scale fMRI dataset containing responses to 57,120 naturalistic images from 30 participants. NOD strives for a balance between sampling variation between individuals and sampling variation between stimuli. This enables NOD to be utilized not only for determining whether an observation is generalizable across many individuals, but also for testing whether a response pattern is generalized to a variety of naturalistic stimuli. We anticipate that the NOD together with existing naturalistic neuroimaging datasets will serve as a new impetus for our understanding of the visual processing of naturalistic stimuli.