Attention Effects on Neural Population Representations for Shape and Location Are Stronger in the Ventral than Dorsal Stream

Two "What" Networks in the Human Brain

Ungerleider and Mishkin, in their influential work that relied on detailed anatomical and ablation studies, suggested that visual information is processed along two distinct pathways: the dorsal "where" pathway, primarily responsible for spatial vision, and the ventral "what" pathway, dedicated to object vision. This strict division of labor has faced challenges in light of compelling evidence revealing robust shape and object selectivity within the putative "where" pathway. This article reviews evidence that supports the presence of shape selectivity in the dorsal pathway. A comparative examination of dorsal and ventral object representations in terms of invariance, task dependency, and representational content reveals similarities and differences between the two pathways. Both exhibit some level of tolerance to image transformations and are influenced by tasks, but responses in the dorsal pathway show weaker tolerance and stronger task modulations than those in the ventral pathway. Furthermore, an examination of their representational content highlights a divergence between the responses in the two pathways, suggesting that they are sensitive to distinct features of objects. Collectively, these findings suggest that two networks exist in the human brain for processing object shapes, one in the dorsal and another in the ventral visual cortex. These studies lay the foundation for future research aimed at revealing the precise roles the two "what" networks play in our ability to understand and interact with objects.

A deep learning model of dorsal and ventral visual streams for DVSD

Artificial intelligence (AI) methods attempt to simulate the behavior and the neural activity of the brain. In particular, Convolutional Neural Networks (CNNs) offer state-of-the-art models of the ventral visual stream. Furthermore, no proposed model estimates the distance between objects as a function of the dorsal stream. In this paper, we present a quantitatively accurate model for the visual system. Specifically, we propose a VeDo-Net model that comprises both ventral and dorsal branches. As in the ventral visual stream, our model recognizes objects. The model also locates and estimates the distance between objects as a spatial relationship task performed by the dorsal stream. One application of the proposed model is in the simulation of visual impairments. In this study, however, we show how the proposed model can simulate the occurrence of dorsal stream impairments such as Autism Spectrum Disorder (ASD) and cerebral visual impairment (CVI). In the end, we explore the impacts of learning on the recovery of the synaptic disruptions of the dorsal visual stream. Results indicated a direct relationship between the positive and negative changes in the weights of the dorsal stream's last layers and the output of the dorsal stream under an allocentric situation. Our results also demonstrate that visual-spatial perception impairments in ASD may be caused by a disturbance in the last layers of the dorsal stream.

Adaptation of the inferior temporal neurons and efficient visual processing

Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.

The role of temporal cortex in the control of attention

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Task-specific modulation of PFC activity for matching-rule governed decision-making

Storing information from incoming stimuli in working memory (WM) is essential for decision-making. The prefrontal cortex (PFC) plays a key role to support this process. Previous studies have characterized different neuronal populations in the PFC for working memory judgements based on whether an originally presented stimulus matches a subsequently presented one (matching-rule decision-making). However, much remains to be understood about this mechanism at the population level of PFC neurons. Here, we hypothesized differences in processing of feature vs. spatial WM within the PFC during a matching-rule decision-making task. To test this hypothesis, the modulation of neural activity within the PFC during two types of decision-making tasks (spatial WM and feature WM) in comparison to a passive fixation task was determined. We discovered that neural population-level activity within the PFC is different for the match vs. non-match condition exclusively in the case of the feature-specific decision-making task. For this task, the non-match condition exhibited a greater firing rate and lower trial-to-trial variability in spike count compared to the feature-match condition. Furthermore, the feature-match condition exhibited lower variability compared to the spatial-match condition. This was accompanied by a faster behavioral response time for the feature-match compared to the spatial-match WM task. We attribute this lower across-trial spiking variability and behavioral response time to a higher task-relevant attentional level in the feature WM compared to the spatial WM task. The findings support our hypothesis for task-specific differences in the processing of feature vs. spatial WM within the PFC. This also confirms the general conclusion that PFC neurons play an important role during the process of matching-rule governed decision-making.

Face Representations via Tensorfaces of Various Complexities

Neurons selective for faces exist in humans and monkeys. However, characteristics of face cell receptive fields are poorly understood. In this theoretical study, we explore the effects of complexity, defined as algorithmic information (Kolmogorov complexity) and logical depth, on possible ways that face cells may be organized. We use tensor decompositions to decompose faces into a set of components, called tensorfaces, and their associated weights, which can be interpreted as model face cells and their firing rates. These tensorfaces form a high-dimensional representation space in which each tensorface forms an axis of the space. A distinctive feature of the decomposition algorithm is the ability to specify tensorface complexity. We found that low-complexity tensorfaces have blob-like appearances crudely approximating faces, while high-complexity tensorfaces appear clearly face-like. Low-complexity tensorfaces require a larger population to reach a criterion face reconstruction error than medium- or high-complexity tensorfaces, and thus are inefficient by that criterion. Low-complexity tensorfaces, however, generalize better when representing statistically novel faces, which are faces falling beyond the distribution of face description parameters found in the tensorface training set. The degree to which face representations are parts based or global forms a continuum as a function of tensorface complexity, with low and medium tensorfaces being more parts based. Given the computational load imposed in creating high-complexity face cells (in the form of algorithmic information and logical depth) and in the absence of a compelling advantage to using high-complexity cells, we suggest face representations consist of a mixture of low- and medium-complexity face cells.

Extrinsic and intrinsic representations

We extend the discussion in the target article about distinctions between extrinsic coding (external references to known things, as required by information theory) and the alternative we and the target article both favor, intrinsic coding (internal relationships within sensory and motor signals). Central to our thinking about intrinsic coding is population coding and the concept of high-dimensional neural response spaces.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.