Higher order visual processing in macaque extrastriate cortex

Higher-order thalamic input to cortex selectively conveys state information

Communication among neocortical areas is largely thought to be mediated by long-range synaptic interactions between cortical neurons, with the thalamus providing only an initial relay of information from the sensory periphery. Higher-order thalamic nuclei receive strong synaptic inputs from the cortex and send robust projections back to other cortical areas, providing a distinct and potentially critical route for corticocortical communication. However, the relative contributions of corticocortical and thalamocortical inputs to higher-order cortical function remain unclear. Using imaging of neurons and axon terminals in combination with optogenetic manipulations, we find that the higher-order visual thalamus of mice has a unique impact on the posterior medial visual cortex (PM). Whereas corticocortical projections from lower cortical areas convey robust visual information to PM, higher-order thalamocortical projections convey information about global arousal state. Together, these findings suggest a key role for the higher-order thalamus in providing contextual signals that may flexibly modulate cortical sensory processing.

A dual-mode organic memristor for coordinated visual perceptive computing

The hierarchically coordinated processing of visual information with the data degradation characteristic embodies the energy consumption minimization and signal transmission efficiency maximization of brain activities. This inspires machine vision to handle the explosively increased data in real-time. In this contribution, we demonstrate the possibility of constructing a coordinated perceptive computing paradigm with dual-mode organic memristors to emulate the visual processing capability of the brain systems. The 32-state modulation of the device photoresponsivity and conductance via photo-induced molecular reconfiguration and electrochemical redox activities enables the execution of computing-in-sensor and computing-in-memory tasks, respectively, which in turn allows the homogeneous hardware integration of a single-layer perceptron and a convolutional neural network for high-efficiency hierarchical visual processing. Compared to the sole optoelectronic CIS mode to recognize visual targets, the dual-mode organic memristor-based coordinated computing scheme demonstrates a 24.5% improvement in the recognition accuracy and 45.8% reduction in the network size.

Paying attention to natural scenes in area V1

Natural scene responses in the primary visual cortex are modulated simultaneously by attention and by contextual signals about scene statistics stored across the connectivity of the visual processing hierarchy. We hypothesized that attentional and contextual signals interact in V1 in a manner that primarily benefits the representation of natural stimuli, rich in high-order statistical structure. Recording from two macaques engaged in a spatial attention task, we found that attention enhanced the decodability of stimulus identity from population responses evoked by natural scenes, but not by synthetic stimuli lacking higher-order statistical regularities. Population analysis revealed that neuronal responses converged to a low-dimensional subspace only for natural stimuli. Critically, we determined that the attentional enhancement in stimulus decodability was captured by the natural-scene subspace, indicating an alignment between the attentional and natural stimulus variance. These results suggest that attentional and contextual signals interact in V1 in a manner optimized for natural vision.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Transthalamic input to higher-order cortex selectively conveys state information

Communication among different neocortical areas is largely thought to be mediated by long-range synaptic interactions between cortical neurons, with the thalamus providing only an initial relay of information from the sensory periphery. Higher-order thalamic nuclei receive strong synaptic inputs from the cortex and send robust projections back to other cortical areas, providing a distinct and potentially critical route for cortico-cortical communication. However, the relative contributions of corticocortical and thalamocortical inputs to higher-order cortical function remain unclear. Using imaging of cortical neurons and projection axon terminals in combination with optogenetic manipulations, we find that the higher-order visual thalamus of mice conveys a specialized stream of information to higher-order visual cortex. Whereas corticocortical projections from lower cortical areas convey robust visual information, higher-order thalamocortical projections convey strong behavioral state information. Together, these findings suggest a key role for higher-order thalamus in providing contextual signals that flexibly modulate sensory processing in higher-order cortex.

Phenotyping Superagers Using Resting-State fMRI

BACKGROUND AND PURPOSE

Superagers are defined as older adults with episodic memory performance similar or superior to that in middle-aged adults. This study aimed to investigate the key differences in discriminative networks and their main nodes between superagers and cognitively average elderly controls. In addition, we sought to explore differences in sensitivity in detecting these functional activities across the networks at 3T and 7T MR imaging fields.

MATERIALS AND METHODS

Fifty-five subjects 80 years of age or older were screened using a detailed neuropsychological protocol, and 31 participants, comprising 14 superagers and 17 cognitively average elderly controls, were included for analysis. Participants underwent resting-state-fMRI at 3T and 7T MR imaging. A prediction classification algorithm using a penalized regression model on the measurements of the network was used to calculate the probabilities of a healthy older adult being a superager. Additionally, ORs quantified the influence of each node across preselected networks.

RESULTS

The key networks that differentiated superagers and elderly controls were the default mode, salience, and language networks. The most discriminative nodes (ORs > 1) in superagers encompassed areas in the precuneus posterior cingulate cortex, prefrontal cortex, temporoparietal junction, temporal pole, extrastriate superior cortex, and insula. The prediction classification model for being a superager showed better performance using the 7T compared with 3T resting-state-fMRI data set.

CONCLUSIONS

Our findings suggest that the functional connectivity in the default mode, salience, and language networks can provide potential imaging biomarkers for predicting superagers. The 7T field holds promise for the most appropriate study setting to accurately detect the functional connectivity patterns in superagers.

Dissociation in neuronal encoding of object versus surface motion in the primate brain

A paradox exists in our understanding of motion processing in the primate visual system: neurons in the dorsal motion processing stream often strikingly fail to encode long-range and perceptually salient jumps of a moving stimulus. Psychophysical studies suggest that such long-range motion, which requires integration over more distant parts of the visual field, may be based on higher-order motion processing mechanisms that rely on feature or object tracking. Here, we demonstrate that ventral visual area V4, long recognized as critical for processing static scenes, includes neurons that maintain direction selectivity for long-range motion, even when conflicting local motion is present. These V4 neurons exhibit specific selectivity for the motion of objects, i.e., targets with defined boundaries, rather than the motion of surfaces behind apertures, and are selective for direction of motion over a broad range of spatial displacements and defined by a variety of features. Motion direction at a range of speeds can be accurately decoded on single trials from the activity of just a few V4 neurons. Thus, our results identify a novel motion computation in the ventral stream that is strikingly different from, and complementary to, the well-established system in the dorsal stream, and they support the hypothesis that the ventral stream system interacts with the dorsal stream to achieve the higher level of abstraction critical for tracking dynamic objects.

Sex-Specific Brain Activations during Single-Leg Exercise

Background

Females have an increased incidence of musculoskeletal injuries compared to males. Sex differences in neuromuscular control has been widely studied regarding the dynamics and muscle activity during preplanned movements. While muscle activation patterns and movement biomechanics are understood to differ between sexes, it is not well understood how sex influences brain activity for lower extremity movement. Since the brain plays a vital role for voluntary movement and joint stability, it is important to understand the sex differences in brain function in order to better understand neuromuscular control associated with increased musculoskeletal injury risk in female.

Hypothesis/Purpose

The purpose of this study is to understand the differences in brain activation patterns between sexes during a simple active knee extension-flexion movement. It was hypothesized that females would demonstrate higher cortical activation in the somatosensory areas compared to males as a compensatory strategy.

Study Design

Cross-Sectional Study.

Methods

Thirteen males and seventeen females who were healthy and physically active participated in this study (Male: 23.7±3.8 years, 74.5±13.5 kg, 172.3±6.4 cm; Female: 20.6±1.6 years, 65.4±12.8 kg, 163±6.1 cm). Functional magnetic resonance imaging data were obtained during a simple left knee extension-flexion exercise with their own leg weight while lying on the MRI table. The blood oxygen level dependent (BOLD) signals were compared between sexes.

Results

There was significantly greater activation in the visual cortices and premotor cortex in females compared to males during the studied movement. Males demonstrated significantly greater activation in the right cerebellum.

Conclusion

The results revealed sex differences in BOLD signal during simple knee extension-flexion movement. The results suggest that sex may be a biological factor in understanding brain activity associated with knee motor control.

Level of Evidence

Level 3.

Learning bio-inspired head-centric representations of 3D shapes in an active fixation setting

When exploring the surrounding environment with the eyes, humans and primates need to interpret three-dimensional (3D) shapes in a fast and invariant way, exploiting a highly variant and gaze-dependent visual information. Since they have front-facing eyes, binocular disparity is a prominent cue for depth perception. Specifically, it serves as computational substrate for two ground mechanisms of binocular active vision: stereopsis and binocular coordination. To this aim, disparity information, which is expressed in a retinotopic reference frame, is combined along the visual cortical pathways with gaze information and transformed in a head-centric reference frame. Despite the importance of this mechanism, the underlying neural substrates still remain widely unknown. In this work, we investigate the capabilities of the human visual system to interpret the 3D scene exploiting disparity and gaze information. In a psychophysical experiment, human subjects were asked to judge the depth orientation of a planar surface either while fixating a target point or while freely exploring the surface. Moreover, we used the same stimuli to train a recurrent neural network to exploit the responses of a modelled population of cortical (V1) cells to interpret the 3D scene layout. The results for both human performance and from the model network show that integrating disparity information across gaze directions is crucial for a reliable and invariant interpretation of the 3D geometry of the scene.

The human middle temporal cortex responds to both active leg movements and egomotion-compatible visual motion

The human middle-temporal region MT+ is highly specialized in processing visual motion. However, recent studies have shown that this region is modulated by extraretinal signals, suggesting a possible involvement in processing motion information also from non-visual modalities. Here, we used functional MRI data to investigate the influence of retinal and extraretinal signals on MT+ in a large sample of subjects. Moreover, we used resting-state functional MRI to assess how the subdivisions of MT+ (i.e., MST, FST, MT, and V4t) are functionally connected. We first compared responses in MST, FST, MT, and V4t to coherent vs. random visual motion. We found that only MST and FST were positively activated by coherent motion. Furthermore, regional analyses revealed that MST and FST were positively activated by leg, but not arm, movements, while MT and V4t were deactivated by arm, but not leg, movements. Taken together, regional analyses revealed a visuomotor role for the anterior areas MST and FST and a pure visual role for the anterior areas MT and V4t. These findings were mirrored by the pattern of functional connections between these areas and the rest of the brain. Visual and visuomotor regions showed distinct patterns of functional connectivity, with the latter preferentially connected with the somatosensory and motor areas representing leg and foot. Overall, these findings reveal a functional sensitivity for coherent visual motion and lower-limb movements in MST and FST, suggesting their possible involvement in integrating sensory and motor information to perform locomotion.

Egomotion-related visual areas respond to goal-directed movements

Integration of proprioceptive signals from the various effectors with visual feedback of self-motion from the retina is necessary for whole-body movement and locomotion. Here, we tested whether the human visual motion areas involved in processing optic flow signals simulating self-motion are also activated by goal-directed movements (as saccades or pointing) performed with different effectors (eye, hand, and foot), suggesting a role in visually guiding movements through the external environment. To achieve this aim, we used a combined approach of task-evoked activity and effective connectivity (PsychoPhysiological Interaction, PPI) by fMRI. We localized a set of six egomotion-responsive visual areas through the flow field stimulus and distinguished them into visual (pIPS/V3A, V6+ , IPSmot/VIP) and visuomotor (pCi, CSv, PIC) areas according to recent literature. We tested their response to a visuomotor task implying spatially directed delayed eye, hand, and foot movements. We observed a posterior-to-anterior gradient of preference for eye-to-foot movements, with posterior (visual) regions showing a preference for saccades, and anterior (visuomotor) regions showing a preference for foot pointing. No region showed a clear preference for hand pointing. Effective connectivity analysis showed that visual areas were more connected to each other with respect to the visuomotor areas, particularly during saccades. We suggest that visual and visuomotor egomotion regions can play different roles within a network that integrates sensory-motor signals with the aim of guiding movements in the external environment.

Diversity of spatiotemporal coding reveals specialized visual processing streams in the mouse cortex

The cerebral cortex contains diverse neural representations of the visual scene, each enabling distinct visual and spatial abilities. However, the extent to which representations are distributed or segregated across cortical areas remains poorly understood. By determining the spatial and temporal responses of >30,000 layer 2/3 pyramidal neurons, we characterize the functional organization of parallel visual streams across eight areas of the mouse cortex. While dorsal and ventral areas form complementary representations of spatiotemporal frequency, motion speed, and spatial patterns, the anterior and posterior dorsal areas show distinct specializations for fast and slow oriented contrasts. At the cellular level, while diverse spatiotemporal tuning lies along a continuum, oriented and non-oriented spatial patterns are encoded by distinct tuning types. The identified tuning types are present across dorsal and ventral streams. The data underscore the highly specific and highly distributed nature of visual cortical representations, which drives specialization of cortical areas and streams.

The cerebral cortex contains different neural representations of the visual scene. Here, the authors show diverse and stereotyped tuning composing specialized representations in the dorsal and ventral areas of the mouse visual cortex, suggesting parallel processing channels and streams.

Multi-regional module-based signal transmission in mouse visual cortex

The visual cortex is hierarchically organized, yet the presence of extensive recurrent and parallel pathways make it challenging to decipher how signals flow between neuronal populations. Here, we tracked the flow of spiking activity recorded from six interconnected levels of the mouse visual hierarchy. By analyzing leading and lagging spike-timing relationships among pairs of simultaneously recorded neurons, we created a cellular-scale directed network graph. Using a module-detection algorithm to cluster neurons based on shared connectivity patterns, we uncovered two multi-regional communication modules distributed across the hierarchy. The direction of signal flow both between and within these modules, differences in layer and area distributions, and distinct temporal dynamics suggest that one module transmits feedforward sensory signals, whereas the other integrates inputs for recurrent processing. These results suggest that multi-regional functional modules may be a fundamental feature of organization beyond cortical areas that supports signal propagation across hierarchical recurrent networks.

Rat sensitivity to multipoint statistics is predicted by efficient coding of natural scenes

Efficient processing of sensory data requires adapting the neuronal encoding strategy to the statistics of natural stimuli. Previously, in Hermundstad et al., 2014, we showed that local multipoint correlation patterns that are most variable in natural images are also the most perceptually salient for human observers, in a way that is compatible with the efficient coding principle. Understanding the neuronal mechanisms underlying such adaptation to image statistics will require performing invasive experiments that are impossible in humans. Therefore, it is important to understand whether a similar phenomenon can be detected in animal species that allow for powerful experimental manipulations, such as rodents. Here we selected four image statistics (from single- to four-point correlations) and trained four groups of rats to discriminate between white noise patterns and binary textures containing variable intensity levels of one of such statistics. We interpreted the resulting psychometric data with an ideal observer model, finding a sharp decrease in sensitivity from two- to four-point correlations and a further decrease from four- to three-point. This ranking fully reproduces the trend we previously observed in humans, thus extending a direct demonstration of efficient coding to a species where neuronal and developmental processes can be interrogated and causally manipulated.

Rats spontaneously perceive global motion direction of drifting plaids

Computing global motion direction of extended visual objects is a hallmark of primate high-level vision. Although neurons selective for global motion have also been found in mouse visual cortex, it remains unknown whether rodents can combine multiple motion signals into global, integrated percepts. To address this question, we trained two groups of rats to discriminate either gratings (G group) or plaids (i.e., superpositions of gratings with different orientations; P group) drifting horizontally along opposite directions. After the animals learned the task, we applied a visual priming paradigm, where presentation of the target stimulus was preceded by the brief presentation of either a grating or a plaid. The extent to which rat responses to the targets were biased by such prime stimuli provided a measure of the spontaneous, perceived similarity between primes and targets. We found that gratings and plaids, when used as primes, were equally effective at biasing the perception of plaid direction for the rats of the P group. Conversely, for the G group, only the gratings acted as effective prime stimuli, while the plaids failed to alter the perception of grating direction. To interpret these observations, we simulated a decision neuron reading out the representations of gratings and plaids, as conveyed by populations of either component or pattern cells (i.e., local or global motion detectors). We concluded that the findings for the P group are highly consistent with the existence of a population of pattern cells, playing a functional role similar to that demonstrated in primates. We also explored different scenarios that could explain the failure of the plaid stimuli to elicit a sizable priming magnitude for the G group. These simulations yielded testable predictions about the properties of motion representations in rodent visual cortex at the single-cell and circuitry level, thus paving the way to future neurophysiology experiments.

Visual Field Test With Gaze Check Tasks: Application in a Homonymous Hemianopic Patient Unaware of the Visual Defects

Gaze control is required for applying visual stimuli to a particular area of the visual field. We developed a visual field test with gaze check tasks to investigate hemianopia. In this test, participants must report the presence or absence of visual stimuli when a small object at the fixation point vibrates. Trials in the absence of visual stimuli were used as gaze check tasks, since the vibration could be observed only when the gaze was directed at the fixation point. We evaluated the efficacy of our test in four control participants and one patient with homonymous hemianopia who was unaware of the defects in the left visual field. This patient presented hemianopia in the test with gaze check tasks, but not when the gaze check tasks were omitted. The patient showed spontaneous gaze movements from the fixation point to the upper left direction, as well as scanning of the left visual field during the test without gaze check tasks. Thus, we concluded that the visual defects in this patient were compensated in daily life by spontaneous eye movements coordinated with visual information processing. The present results show the usefulness of the visual field test with gaze check tasks.

Cortical and Subcortical Circuits for Cross-Modal Plasticity Induced by Loss of Vision

Cortical areas are highly interconnected both via cortical and subcortical pathways, and primary sensory cortices are not isolated from this general structure. In primary sensory cortical areas, these pre-existing functional connections serve to provide contextual information for sensory processing and can mediate adaptation when a sensory modality is lost. Cross-modal plasticity in broad terms refers to widespread plasticity across the brain in response to losing a sensory modality, and largely involves two distinct changes: cross-modal recruitment and compensatory plasticity. The former involves recruitment of the deprived sensory area, which includes the deprived primary sensory cortex, for processing the remaining senses. Compensatory plasticity refers to plasticity in the remaining sensory areas, including the spared primary sensory cortices, to enhance the processing of its own sensory inputs. Here, we will summarize potential cellular plasticity mechanisms involved in cross-modal recruitment and compensatory plasticity, and review cortical and subcortical circuits to the primary sensory cortices which can mediate cross-modal plasticity upon loss of vision.

Development of visual motion integration involves coordination of multiple cortical stages

A central feature of cortical function is hierarchical processing of information. Little is currently known about how cortical processing cascades develop. Here, we investigate the joint development of two nodes of the ferret’s visual motion pathway, primary visual cortex (V1), and higher-level area PSS. In adult animals, motion processing transitions from local to global computations between these areas. We now show that PSS global motion signals emerge a week after the development of V1 and PSS direction selectivity. Crucially, V1 responses to more complex motion stimuli change in parallel, in a manner consistent with supporting increased PSS motion integration. At the same time, these V1 responses depend on feedback from PSS. Our findings suggest that development does not just proceed in parallel in different visual areas, it is coordinated across network nodes. This has important implications for understanding how visual experience and developmental disorders can influence the developing visual system.

Primate extrastriate cortical area MST: a gateway between sensation and cognition

Primate visual cortex consists of dozens of distinct brain areas, each providing a highly specialized component to the sophisticated task of encoding the incoming sensory information and creating a representation of our visual environment that underlies our perception and action. One such area is the medial superior temporal cortex (MST), a motion-sensitive, direction-selective part of the primate visual cortex. It receives most of its input from the middle temporal (MT) area, but MST cells have larger receptive fields and respond to more complex motion patterns. The finding that MST cells are tuned for optic flow patterns has led to the suggestion that the area plays an important role in the perception of self-motion. This hypothesis has received further support from studies showing that some MST cells also respond selectively to vestibular cues. Furthermore, the area is part of a network that controls the planning and execution of smooth pursuit eye movements and its activity is modulated by cognitive factors, such as attention and working memory. This review of more than 90 studies focuses on providing clarity of the heterogeneous findings on MST in the macaque cortex and its putative homolog in the human cortex. From this analysis of the unique anatomical and functional position in the hierarchy of areas and processing steps in primate visual cortex, MST emerges as a gateway between perception, cognition, and action planning. Given this pivotal role, this area represents an ideal model system for the transition from sensation to cognition.

Exploitation of image statistics with sparse coding in the case of stereo vision

The sparse coding algorithm has served as a model for early processing in mammalian vision. It has been assumed that the brain uses sparse coding to exploit statistical properties of the sensory stream. We hypothesize that sparse coding discovers patterns from the data set, which can be used to estimate a set of stimulus parameters by simple readout. In this study, we chose a model of stereo vision to test our hypothesis. We used the Locally Competitive Algorithm (LCA), followed by a naïve Bayes classifier, to infer stereo disparity. From the results we report three observations. First, disparity inference was successful with this naturalistic processing pipeline. Second, an expanded, highly redundant representation is required to robustly identify the input patterns. Third, the inference error can be predicted from the number of active coefficients in the LCA representation. We conclude that sparse coding can generate a suitable general representation for subsequent inference tasks.

Stable readout of observed actions from format-dependent activity of monkey's anterior intraparietal neurons

The anterior intraparietal area (AIP) is a crucial hub in the observed manipulative action (OMA) network of primates. While macaques observe manipulative action videos, their AIP neuronal activity robustly encodes first the viewpoint from which the action is observed, then the actor’s body posture, and finally the observed-action identity. Despite the lack of fully invariant OMA-selective single neurons, OMA exemplars could be decoded accurately from the activity of a set of units that maintain stable OMA selectivity despite rescaling their firing rate across formats. We propose that by integrating signals multiplicatively about others’ action and their visual format, the AIP can provide a stable readout of OMA identity at the population level.

Humans accurately identify observed actions despite large dynamic changes in their retinal images and a variety of visual presentation formats. A large network of brain regions in primates participates in the processing of others’ actions, with the anterior intraparietal area (AIP) playing a major role in routing information about observed manipulative actions (OMAs) to the other nodes of the network. This study investigated whether the AIP also contributes to invariant coding of OMAs across different visual formats. We recorded AIP neuronal activity from two macaques while they observed videos portraying seven manipulative actions (drag, drop, grasp, push, roll, rotate, squeeze) in four visual formats. Each format resulted from the combination of two actor’s body postures (standing, sitting) and two viewpoints (lateral, frontal). Out of 297 recorded units, 38% were OMA-selective in at least one format. Robust population code for viewpoint and actor’s body posture emerged shortly after stimulus presentation, followed by OMA selectivity. Although we found no fully invariant OMA-selective neuron, we discovered a population code that allowed us to classify action exemplars irrespective of the visual format. This code depends on a multiplicative mixing of signals about OMA identity and visual format, particularly evidenced by a set of units maintaining a relatively stable OMA selectivity across formats despite considerable rescaling of their firing rate depending on the visual specificities of each format. These findings suggest that the AIP integrates format-dependent information and the visual features of others’ actions, leading to a stable readout of observed manipulative action identity.

Task-dependent fMRI decoder with the power to extend Gabor patch results to Natural images

Scientists are often asked to what extent a simple finding in a laboratory can be generalized to complicated phenomena in our daily lives. The same is equally true of vision science; numerous critical discoveries about our visual system have been made using very simple visual images, such as Gabor patches, but to what extent can these findings be applied to more natural images? Here, we used the fMRI decoding technique and directly tested whether the findings obtained with primitive visual stimuli (Gabor patches) were applicable to natural images. In the fMRI experiments, participants performed depth and resolution tasks with both Gabor patches and natural images. We created a fMRI decoder made from the results of the Gabor patch experiments that classified a brain activity pattern into the depth or resolution task, and then examined how successful the task-dependent decoder could sort a brain activity pattern in the natural image experiment into the depth or resolution task. As a result, we found that the task-dependent decoder constructed from Gabor patch experiments could predict which task (depth or resolution task) a participant was engaged in the natural image experiments, especially in the V3 and middle temporal (MT+) areas of the brain. This is consistent with previous researches on the cortical activation relating to depth perception rather than perceptual processing of display resolution. These results provide firm evidence that fMRI decoding technique possesses the power to evaluate the application of Gabor patch results (laboratory findings) to the natural images (everyday affairs), representing a new approach for studying the mechanism of visual perception.

Face categorization and behavioral templates in rats

Rodents have become a popular model in vision science. It is still unclear how vision in rodents relates to primate vision when it comes to complex visual tasks. Here we report on the results of training rats in a face-categorization and generalization task. Additionally, the Bubbles paradigm is used to determine the behavioral templates of the animals. We found that rats are capable of face categorization and can generalize to previously unseen exemplars. Performance is affected-but remains above chance-by stimulus modifications such as upside-down and contrast-inverted stimuli. The behavioral templates of the rats overlap with a pixel-based template, with a bias toward the upper left parts of the stimuli. Together, these findings significantly expand the evidence about the extent to which rats learn complex visual-categorization tasks.

A model of how depth facilitates scene-relative object motion perception

Many everyday interactions with moving objects benefit from an accurate perception of their movement. Self-motion, however, complicates object motion perception because it generates a global pattern of motion on the observer’s retina and radically influences an object’s retinal motion. There is strong evidence that the brain compensates by suppressing the retinal motion due to self-motion, however, this requires estimates of depth relative to the object—otherwise the appropriate self-motion component to remove cannot be determined. The underlying neural mechanisms are unknown, but neurons in brain areas MT and MST may contribute given their sensitivity to motion parallax and depth through joint direction, speed, and disparity tuning. We developed a neural model to investigate whether cells in areas MT and MST with well-established neurophysiological properties can account for human object motion judgments during self-motion. We tested the model by comparing simulated object motion signals to human object motion judgments in environments with monocular, binocular, and ambiguous depth. Our simulations show how precise depth information, such as that from binocular disparity, may improve estimates of the retinal motion pattern due the self-motion through increased selectivity among units that respond to the global self-motion pattern. The enhanced self-motion estimates emerged from recurrent feedback connections in MST and allowed the model to better suppress the appropriate direction, speed, and disparity signals from the object’s retinal motion, improving the accuracy of the object’s movement direction represented by motion signals.

Research has shown that the accuracy with which humans perceive object motion during self-motion improves in the presence of stereo cues. Using a neural modelling approach, we explore whether this finding can be explained through improved estimation of the retinal motion induced by self-motion. Our results show that depth cues that provide information about scene structure may have a large effect on the specificity with which the neural mechanisms for motion perception represent the visual self-motion signal. This in turn enables effective removal of the retinal motion due to self-motion when the goal is to perceive object motion relative to the stationary world. These results reveal a hitherto unknown critical function of stereo tuning in the MT-MST complex, and shed important light on how the brain may recruit signals from upstream and downstream brain areas to simultaneously perceive self-motion and object motion.

Neural, functional, and aesthetic impacts of spatially heterogeneous flicker: A potential role of natural flicker

Spatially heterogeneous flicker, characterized by probabilistic and locally independent luminance modulations, abounds in nature. It is generated by flames, water surfaces, rustling leaves, and so on, and it is pleasant to the senses. It affords spatiotemporal multistability that allows sensory activation conforming to the biases of the visual system, thereby generating the perception of spontaneous motion and likely facilitating the calibration of motion detectors. One may thus hypothesize that spatially heterogeneous flicker might potentially provide restoring stimuli to the visual system that engage fluent (requiring minimal top-down control) and self-calibrating processes. Here, we present some converging behavioral and electrophysiological evidence consistent with this idea. Spatially heterogeneous (multistable) flicker (relative to controls matched in temporal statistics) reduced posterior EEG (electroencephalography) beta power implicated in long-range neural interactions that impose top-down influences on sensory processing. Further, the degree of spatiotemporal multistability, the amount of posterior beta-power reduction, and the aesthetic responses to flicker were closely associated. These results are consistent with the idea that the pleasantness of natural flicker may derive from its spatiotemporal multistability that affords fluent and self-calibrating visual processing.

A Biologically-Inspired Model to Predict Perceived Visual Speed as a Function of the Stimulated Portion of the Visual Field

Spatial orientation relies on a representation of the position and orientation of the body relative to the surrounding environment. When navigating in the environment, this representation must be constantly updated taking into account the direction, speed, and amplitude of body motion. Visual information plays an important role in this updating process, notably via optical flow. Here, we systematically investigated how the size and the simulated portion of the field of view (FoV) affect perceived visual speed of human observers. We propose a computational model to account for the patterns of human data. This model is composed of hierarchical cells' layers that model the neural processing stages of the dorsal visual pathway. Specifically, we consider that the activity of the MT area is processed by populations of modeled MST cells that are sensitive to the differential components of the optical flow, thus producing selectivity for specific patterns of optical flow. Our results indicate that the proposed computational model is able to describe the experimental evidence and it could be used to predict expected biases of speed perception for conditions in which only some portions of the visual field are visible.

Data-Driven Approaches to Understanding Visual Neuron Activity

With modern neurophysiological methods able to record neural activity throughout the visual pathway in the context of arbitrarily complex visual stimulation, our understanding of visual system function is becoming limited by the available models of visual neurons that can be directly related to such data. Different forms of statistical models are now being used to probe the cellular and circuit mechanisms shaping neural activity, understand how neural selectivity to complex visual features is computed, and derive the ways in which neurons contribute to systems-level visual processing. However, models that are able to more accurately reproduce observed neural activity often defy simple interpretations. As a result, rather than being used solely to connect with existing theories of visual processing, statistical modeling will increasingly drive the evolution of more sophisticated theories.

Representational untangling by the firing rate nonlinearity in V1 simple cells

An important computational goal of the visual system is ‘representational untangling’ (RU): representing increasingly complex features of visual scenes in an easily decodable format. RU is typically assumed to be achieved in high-level visual cortices via several stages of cortical processing. Here we show, using a canonical population coding model, that RU of low-level orientation information is already performed at the first cortical stage of visual processing, but not before that, by a fundamental cellular-level property: the thresholded firing rate nonlinearity of simple cells in the primary visual cortex (V1). We identified specific, experimentally measurable parameters that determined the optimal firing threshold for RU and found that the thresholds of V1 simple cells extracted from in vivo recordings in awake behaving mice were near optimal. These results suggest that information re-formatting, rather than maximisation, may already be a relevant computational goal for the early visual system.

Visual Motion and Form Integration in the Behaving Ferret

Ferrets have become a standard animal model for the development of early visual stages. Less is known about higher-level vision in ferrets, both during development and in adulthood. Here, as a step towards establishing higher-level vision research in ferrets, we used behavioral experiments to test the motion and form integration capacity of adult ferrets. Motion integration was assessed by training ferrets to discriminate random dot kinematograms (RDK) based on their direction. Task difficulty was varied systematically by changing RDK coherence levels, which allowed the measurement of motion integration thresholds. Form integration was measured analogously by training ferrets to discriminate linear Glass patterns of varying coherence levels based on their orientation. In all experiments, ferrets proved to be good psychophysical subjects that performed tasks reliably. Crucially, the behavioral data showed clear evidence of perceptual motion and form integration. In the monkey, motion and form integration are usually associated with processes occurring in higher-level visual areas. In a second set of experiments, we therefore tested whether PSS, a higher-level motion area in the ferret, could similarly support motion integration behavior in this species. To this end, we measured responses of PSS neurons to RDK of different coherence levels. Indeed, neurometric functions for PSS were in good agreement with the behaviorally derived psychometric functions. In conclusion, our experiments demonstrate that ferrets are well suited for higher-level vision research.

Action at a distance on object-related ventral temporal representations

The representation of objects in ventral temporal cortex is relatively resilient to transformations in the stimuli. There is emerging recognition that ventral temporal object representations are forged via interactions among a broader network of regions that receive independent inputs about a stimulus. Here we test whether ventral temporal representations are causally modulated by disrupting processing in distal associative areas. We used transcranial direct current stimulation (tDCS) to stimulate left parietal areas and functional Magnetic Resonance Imaging (fMRI) to measure object-related neural responses in the ventral stream. We find that representational geometries and category discriminability within ventral temporal cortex, as well as functional connectivity between ventral temporal and parietal areas, are enhanced by anodal compared to cathodal stimulation of left parietal associative cortex. These results demonstrate that ventral temporal representations can be causally modulated by processing distal to the ventral stream.

The effect of mild traumatic brain injury on the visual processing of global form and motion

Cortical visual processing involves the ventral stream (form perception) and the dorsal stream (motion perception). We assessed whether mild traumatic brain injury (TBI) differentially affects these two streams. Eleven adults with mild TBI (28 ± 9 yrs, 17 ± 5 months post injury) and 25 controls (25 ± 5 yrs) participated. Participants completed tests of global processing involving Glass patterns (form) and random dot kinematograms (motion), measurement of contrast thresholds for motion direction discrimination, a comprehensive vision screening and the Post-Concussion Symptom Inventory (PCSI). Our results showed that the mild TBI group had significantly higher (worse) global form (mean ± SD: TBI 25 ± 6%, control 21 ± 5%) and motion (TBI 14 ± 7%, control 11 ± 3%) coherence thresholds than controls. The magnitude of the mild TBI group deficit did not differ between the two tasks. Contrast thresholds for motion direction discrimination did not differ between the groups, but were positively correlated with PCSI score (r² = 0.51. p = 0.01) in the mild TBI group. The mild TBI group had worse outcomes than controls for all clinical measurements of vision except distance visual acuity. In conclusion, mild TBI affects processing in both the dorsal and ventral cortical processing streams equally. In addition, spatiotemporal contrast sensitivity may be related to the symptoms of mild TBI.

A lexical semantic hub for heteromodal naming in middle fusiform gyrus

Semantic memory underpins our understanding of objects, people, places, and ideas. Anomia, a disruption of semantic memory access, is the most common residual language disturbance and is seen in dementia and following injury to temporal cortex. While such anomia has been well characterized by lesion symptom mapping studies, its pathophysiology is not well understood. We hypothesize that inputs to the semantic memory system engage a specific heteromodal network hub that integrates lexical retrieval with the appropriate semantic content. Such a network hub has been proposed by others, but has thus far eluded precise spatiotemporal delineation. This limitation in our understanding of semantic memory has impeded progress in the treatment of anomia. We evaluated the cortical structure and dynamics of the lexical semantic network in driving speech production in a large cohort of patients with epilepsy using electrocorticography (n = 64), functional MRI (n = 36), and direct cortical stimulation (n = 30) during two generative language processes that rely on semantic knowledge: visual picture naming and auditory naming to definition. Each task also featured a non-semantic control condition: scrambled pictures and reversed speech, respectively. These large-scale data of the left, language-dominant hemisphere uniquely enable convergent, high-resolution analyses of neural mechanisms characterized by rapid, transient dynamics with strong interactions between distributed cortical substrates. We observed three stages of activity during both visual picture naming and auditory naming to definition that were serially organized: sensory processing, lexical semantic processing, and articulation. Critically, the second stage was absent in both the visual and auditory control conditions. Group activity maps from both electrocorticography and functional MRI identified heteromodal responses in middle fusiform gyrus, intraparietal sulcus, and inferior frontal gyrus; furthermore, the spectrotemporal profiles of these three regions revealed coincident activity preceding articulation. Only in the middle fusiform gyrus did direct cortical stimulation disrupt both naming tasks while still preserving the ability to repeat sentences. These convergent data strongly support a model in which a distinct neuroanatomical substrate in middle fusiform gyrus provides access to object semantic information. This under-appreciated locus of semantic processing is at risk in resections for temporal lobe epilepsy as well as in trauma and strokes that affect the inferior temporal cortex-it may explain the range of anomic states seen in these conditions. Further characterization of brain network behaviour engaging this region in both healthy and diseased states will expand our understanding of semantic memory and further development of therapies directed at anomia.

Neural correlates of sparse coding and dimensionality reduction

Supported by recent computational studies, there is increasing evidence that a wide range of neuronal responses can be understood as an emergent property of nonnegative sparse coding (NSC), an efficient population coding scheme based on dimensionality reduction and sparsity constraints. We review evidence that NSC might be employed by sensory areas to efficiently encode external stimulus spaces, by some associative areas to conjunctively represent multiple behaviorally relevant variables, and possibly by the basal ganglia to coordinate movement. In addition, NSC might provide a useful theoretical framework under which to understand the often complex and nonintuitive response properties of neurons in other brain areas. Although NSC might not apply to all brain areas (for example, motor or executive function areas) the success of NSC-based models, especially in sensory areas, warrants further investigation for neural correlates in other regions.

Adaptation reveals multi-stage coding of visual duration

In conflict with historically dominant models of time perception, recent evidence suggests that the encoding of our environment's temporal properties may not require a separate class of neurons whose raison d'être is the dedicated processing of temporal information. If true, it follows that temporal processing should be imbued with the known selectivity found within non-temporal neurons. In the current study, we tested this hypothesis for the processing of a poorly understood stimulus parameter: visual event duration. We used sensory adaptation techniques to generate duration aftereffects: bidirectional distortions of perceived duration. Presenting adapting and test durations to the same vs different eyes utilises the visual system's anatomical progression from monocular, pre-cortical neurons to their binocular, cortical counterparts. Duration aftereffects exhibited robust inter-ocular transfer alongside a small but significant contribution from monocular mechanisms. We then used novel stimuli which provided duration information that was invisible to monocular neurons. These stimuli generated robust duration aftereffects which showed partial selectivity for adapt-test changes in retinal disparity. Our findings reveal distinct duration encoding mechanisms at monocular, depth-selective and depth-invariant stages of the visual hierarchy.

A Model of Motion Processing in the Visual Cortex Using Neural Field With Asymmetric Hebbian Learning

Neurons in the dorsal pathway of the visual cortex are thought to be involved in motion processing. The first site of motion processing is the primary visual cortex (V1), encoding the direction of motion in local receptive fields, with higher order motion processing happening in the middle temporal area (MT). Complex motion properties like optic flow are processed in higher cortical areas of the Medial Superior Temporal area (MST). In this study, a hierarchical neural field network model of motion processing is presented. The model architecture has an input layer followed by either one or cascade of two neural fields (NF): the first of these, NF1, represents V1, while the second, NF2, represents MT. A special feature of the model is that lateral connections used in the neural fields are trained by asymmetric Hebbian learning, imparting to the neural field the ability to process sequential information in motion stimuli. The model was trained using various traditional moving patterns such as bars, squares, gratings, plaids, and random dot stimulus. In the case of bar stimuli, the model had only a single NF, the neurons of which developed a direction map of the moving bar stimuli. Training a network with two NFs on moving square and moving plaids stimuli, we show that, while the neurons in NF1 respond to the direction of the component (such as gratings and edges) motion, the neurons in NF2 (analogous to MT) responding to the direction of the pattern (plaids, square object) motion. In the third study, a network with 2 NFs was simulated using random dot stimuli (RDS) with translational motion, and show that the NF2 neurons can encode the direction of the concurrent dot motion (also called translational flow motion), independent of the dot configuration. This translational RDS flow motion is decoded by a simple perceptron network (a layer above NF2) with an accuracy of 100% on train set and 90% on the test set, thereby demonstrating that the proposed network can generalize to new dot configurations. Also, the response properties of the model on different input stimuli closely resembled many of the known features of the neurons found in electrophysiological studies.

Processing of object motion and self-motion in the lateral subdivision of the medial superior temporal area in macaques

Multiple areas of macaque cortex are involved in visual motion processing, but their relative functional roles remain unclear. The medial superior temporal (MST) area is typically divided into lateral (MSTl) and dorsal (MSTd) subdivisions that are thought to be involved in processing object motion and self-motion, respectively. Whereas MSTd has been studied extensively with regard to processing visual and nonvisual self-motion cues, little is known about self-motion signals in MSTl, especially nonvisual signals. Moreover, little is known about how self-motion and object motion signals interact in MSTl and how this differs from interactions in MSTd. We compared the visual and vestibular heading tuning of neurons in MSTl and MSTd using identical stimuli. Our findings reveal that both visual and vestibular heading signals are weaker in MSTl than in MSTd, suggesting that MSTl is less well suited to participate in self-motion perception than MSTd. We also tested neurons in both areas with a variety of combinations of object motion and self-motion. Our findings reveal that vestibular signals improve the separability of coding of heading and object direction in both areas, albeit more strongly in MSTd due to the greater strength of vestibular signals. Based on a marginalization technique, population decoding reveals that heading and object direction can be more effectively dissociated from MSTd responses than MSTl responses. Our findings help to clarify the respective contributions that MSTl and MSTd make to processing of object motion and self-motion, although our conclusions may be somewhat specific to the multipart moving objects that we employed. NEW & NOTEWORTHY Retinal image motion reflects contributions from both the observer's self-motion and the movement of objects in the environment. The neural mechanisms by which the brain dissociates self-motion and object motion remain unclear. This study provides the first systematic examination of how the lateral subdivision of area MST (MSTl) contributes to dissociating object motion and self-motion. We also examine, for the first time, how MSTl neurons represent translational self-motion based on both vestibular and visual cues.

Nonlinear Processing of Shape Information in Rat Lateral Extrastriate Cortex

In rodents, the progression of extrastriate areas located laterally to primary visual cortex (V1) has been assigned to a putative object-processing pathway (homologous to the primate ventral stream), based on anatomical considerations. Recently, we found functional support for such attribution (Tafazoli et al., 2017), by showing that this cortical progression is specialized for coding object identity despite view changes, the hallmark property of a ventral-like pathway. Here, we sought to clarify what computations are at the base of such specialization. To this aim, we performed multielectrode recordings from V1 and laterolateral area LL (at the apex of the putative ventral-like hierarchy) of male adult rats, during the presentation of drifting gratings and noise movies. We found that the extent to which neuronal responses were entrained to the phase of the gratings sharply dropped from V1 to LL, along with the quality of the receptive fields inferred through reverse correlation. Concomitantly, the tendency of neurons to respond to different oriented gratings increased, whereas the sharpness of orientation tuning declined. Critically, these trends are consistent with the nonlinear summation of visual inputs that is expected to take place along the ventral stream, according to the predictions of hierarchical models of ventral computations and a meta-analysis of the monkey literature. This suggests an intriguing homology between the mechanisms responsible for building up shape selectivity and transformation tolerance in the visual cortex of primates and rodents, reasserting the potential of the latter as models to investigate ventral stream functions at the circuitry level.SIGNIFICANCE STATEMENT Despite the growing popularity of rodents as models of visual functions, it remains unclear whether their visual cortex contains specialized modules for processing shape information. To addresses this question, we compared how neuronal tuning evolves from rat primary visual cortex (V1) to a downstream visual cortical region (area LL) that previous work has implicated in shape processing. In our experiments, LL neurons displayed a stronger tendency to respond to drifting gratings with different orientations while maintaining a sustained response across the whole duration of the drift cycle. These trends match the increased complexity of pattern selectivity and the augmented tolerance to stimulus translation found in monkey visual temporal cortex, thus revealing a homology between shape processing in rodents and primates.

Ferrets as a Model for Higher-Level Visual Motion Processing

Ferrets are a major developmental animal model due to their early parturition. Here we show for the first time that ferrets could be used to study development of higher-level visual processes previously identified in primates. In primates, complex motion processing involves primary visual cortex (V1), which generates local motion signals, and higher-level visual area MT, which integrates these signals over more global spatial regions. Our data show similar transformations in motion signals between ferret V1 and higher-level visual area PSS, located in the posterior bank of the suprasylvian sulcus. We found that PSS neurons, like MT neurons, were tuned for stimulus motion and showed strong suppression between opposing direction inputs. Most strikingly, PSS, like MT, exhibited robust global motion signals when tested with coherent plaids-the classic test for motion integration across multiple moving elements. These PSS responses were described well by computational models developed for MT. Our findings establish the ferret as a strong animal model for development of higher-level visual processing.

Auditory and Visual Motion Processing and Integration in the Primate Cerebral Cortex

The ability of animals to detect motion is critical for survival, and errors or even delays in motion perception may prove costly. In the natural world, moving objects in the visual field often produce concurrent sounds. Thus, it can highly advantageous to detect motion elicited from sensory signals of either modality, and to integrate them to produce more reliable motion perception. A great deal of progress has been made in understanding how visual motion perception is governed by the activity of single neurons in the primate cerebral cortex, but far less progress has been made in understanding both auditory motion and audiovisual motion integration. Here we, review the key cortical regions for motion processing, focussing on translational motion. We compare the representations of space and motion in the visual and auditory systems, and examine how single neurons in these two sensory systems encode the direction of motion. We also discuss the way in which humans integrate of audio and visual motion cues, and the regions of the cortex that may mediate this process.

Probing Sensory Readout via Combined Choice-Correlation Measures and Microstimulation Perturbation

It is controversial whether covariation between neuronal activity and perceptual choice (i.e., choice correlation) reflects the functional readout of sensory signals. Here, we combined choice-correlation measures and electrical microstimulation on a site-to-site basis in the medial superior temporal area (MST), middle temporal area (MT), and ventral intraparietal area (VIP) when macaques discriminated between motion directions in both fine and coarse tasks. Microstimulation generated comparable effects between tasks but heterogeneous effects across and within brain regions. Within the MST and MT, microstimulation significantly biased an animal's choice toward the sensory preference instead of choice-related signals of the stimulated units. This was particularly evident for sites with conflict preference of sensory and choice-related signals. In the VIP, microstimulation failed to produce significant effects in either task despite strong choice correlations presented in this area. Our results suggest that sensory readout may not be inferred from choice-related signals during perceptual decision-making tasks.

A Biologically Inspired Approach for Robot Depth Estimation

Aimed at building autonomous service robots, reasoning, perception, and action should be properly integrated. In this paper, the depth cue has been analysed as an early stage given its importance for robotic tasks. So, from neuroscience findings, a hierarchical four-level dorsal architecture has been designed and implemented. Mainly, from a stereo image pair, a set of complex Gabor filters is applied for estimating an egocentric quantitative disparity map. This map leads to a quantitative depth scene representation that provides the raw input for a qualitative approach. So, the reasoning method infers the data required to make the right decision at any time. As it will be shown, the experimental results highlight the robust performance of the biologically inspired approach presented in this paper.

Beyond Rehabilitation of Acuity, Ocular Alignment, and Binocularity in Infantile Strabismus

Infantile strabismus impairs the perception of all attributes of the visual scene. High spatial frequency components are no longer visible, leading to amblyopia. Binocularity is altered, leading to the loss of stereopsis. Spatial perception is impaired as well as detection of vertical orientation, the fastest movements, directions of movement, the highest contrasts and colors. Infantile strabismus also affects other vision-dependent processes such as control of postural stability. But presently, rehabilitative therapies for infantile strabismus by ophthalmologists, orthoptists and optometrists are restricted to preventing or curing amblyopia of the deviated eye, aligning the eyes and, whenever possible, preserving or restoring binocular vision during the critical period of development, i.e., before ~10 years of age. All the other impairments are thus ignored; whether they may recover after strabismus treatment even remains unknown. We argue here that medical and paramedical professionals may extend their present treatments of the perceptual losses associated with infantile strabismus. This hypothesis is based on findings from fundamental research on visual system organization of higher mammals in particular at the cortical level. In strabismic subjects (as in normal-seeing ones), information about all of the visual attributes converge, interact and are thus inter-dependent at multiple levels of encoding ranging from the single neuron to neuronal assemblies in visual cortex. Thus if the perception of one attribute is restored this may help to rehabilitate the perception of other attributes. Concomitantly, vision-dependent processes may also improve. This could occur spontaneously, but still should be assessed and validated. If not, medical and paramedical staff, in collaboration with neuroscientists, will have to break new ground in the field of therapies to help reorganize brain circuitry and promote more comprehensive functional recovery. Findings from fundamental research studies in both young and adult patients already support our hypothesis and are reviewed here. For example, presenting different contrasts to each eye of a strabismic patient during training sessions facilitates recovery of acuity in the amblyopic eye as well as of 3D perception. Recent data also demonstrate that visual recoveries in strabismic subjects improve postural stability. These findings form the basis for a roadmap for future research and clinical development to extend presently applied rehabilitative therapies for infantile strabismus.

Toward an Integrative Theory of Thalamic Function

The thalamus has long been suspected to have an important role in cognition, yet recent theories have favored a more corticocentric view. According to this view, the thalamus is an excitatory feedforward relay to or between cortical regions, and cognitively relevant computations are exclusively cortical. Here, we review anatomical, physiological, and behavioral studies along evolutionary and theoretical dimensions, arguing for essential and unique thalamic computations in cognition. Considering their architectural features as well as their ability to initiate, sustain, and switch cortical activity, thalamic circuits appear uniquely suited for computing contextual signals that rapidly reconfigure task-relevant cortical representations. We introduce a framework that formalizes this notion, show its consistency with several findings, and discuss its prediction of thalamic roles in perceptual inference and behavioral flexibility. Overall, our framework emphasizes an expanded view of the thalamus in cognitive computations and provides a roadmap to test several of its theoretical and experimental predictions.

Organization of Neural Population Code in Mouse Visual System

The mammalian visual system consists of several anatomically distinct areas, layers, and cell types. To understand the role of these subpopulations in visual information processing, we analyzed neural signals recorded from excitatory neurons from various anatomical and functional structures. For each of 186 mice, one of six genetically tagged cell types and one of six visual areas were targeted while the mouse was passively viewing various visual stimuli. We trained linear classifiers to decode one of six visual stimulus categories with distinct spatiotemporal structures from the population neural activity. We found that neurons in both the primary visual cortex and secondary visual areas show varying degrees of stimulus-specific decodability, and neurons in superficial layers tend to be more informative about the stimulus categories. Additional decoding analyses of directional motion were consistent with these findings. We observed synergy in the population code of direction in several visual areas suggesting area-specific organization of information representation across neurons. These differences in decoding capacities shed light on the specialized organization of neural information processing across anatomically distinct subpopulations, and further establish the mouse as a model for understanding visual perception.

No-reference stereoscopic image quality assessment guided by visual hierarchical structure and binocular effects

Stereoscopic image quality assessment (SIQA) is an essential technique for modern 3D image and video processing systems serving as performance evaluators and monitors. However, the study on SIQA remains immature due to the complexity of the human visual system (HVS) and binocular effects that binocular vision brings about. To overcome the difficulties, a novel method is proposed that extracts and quantifies image quality-aware features related to cortex areas in charge of visual quality perception, rather than attempting to rigorously simulate the biological processing in HVS, so that the predicting accuracy is preserved while the computational complexity remains moderate. Meanwhile, binocular effects including binocular rivalry and visual discomfort are taken into consideration. Moreover, the proposed method can be operated completely without the assistance of reference images, indicating its wide practical usages. Compared to state-of-the-art works, our method shows evident superiority in terms of effectiveness and robustness.