Organization of the macaque extrastriate visual cortex re-examined using the principle of spatial continuity of function

Cortical topographic motifs emerge in a self-organized map of object space

The human ventral visual stream has a highly systematic organization of object information, but the causal pressures driving these topographic motifs are highly debated. Here, we use self-organizing principles to learn a topographic representation of the data manifold of a deep neural network representational space. We find that a smooth mapping of this representational space showed many brain-like motifs, with a large-scale organization by animacy and real-world object size, supported by mid-level feature tuning, with naturally emerging face- and scene-selective regions. While some theories of the object-selective cortex posit that these differently tuned regions of the brain reflect a collection of distinctly specified functional modules, the present work provides computational support for an alternate hypothesis that the tuning and topography of the object-selective cortex reflect a smooth mapping of a unified representational space.

Neurodevelopment of the association cortices: Patterns, mechanisms, and implications for psychopathology

The human brain undergoes a prolonged period of cortical development that spans multiple decades. During childhood and adolescence, cortical development progresses from lower-order, primary and unimodal cortices with sensory and motor functions to higher-order, transmodal association cortices subserving executive, socioemotional, and mentalizing functions. The spatiotemporal patterning of cortical maturation thus proceeds in a hierarchical manner, conforming to an evolutionarily rooted, sensorimotor-to-association axis of cortical organization. This developmental program has been characterized by data derived from multimodal human neuroimaging and is linked to the hierarchical unfolding of plasticity-related neurobiological events. Critically, this developmental program serves to enhance feature variation between lower-order and higher-order regions, thus endowing the brain's association cortices with unique functional properties. However, accumulating evidence suggests that protracted plasticity within late-maturing association cortices, which represents a defining feature of the human developmental program, also confers risk for diverse developmental psychopathologies.

Representation of shape, space, and attention in monkey cortex

Attentional deficits are core to numerous developmental, neurological, and psychiatric disorders. At the single-cell level, much knowledge has been garnered from studies of shape and spatial properties, as well as from numerous demonstrations of attentional modulation of those properties. Despite this wealth of knowledge of single-cell responses across many brain regions, little is known about how these cellular characteristics relate to population level representations and how such representations relate to behavior; in particular, how these cellular responses relate to the representation of shape, space, and attention, and how these representations differ across cortical areas and streams. Here we will emphasize the role of population coding as a missing link for connecting single-cell properties with behavior. Using a data-driven intrinsic approach to population decoding, we show that both 'what' and 'where' cortical visual streams encode shape, space, and attention, yet demonstrate striking differences in these representations. We suggest that both pathways fully process shape and space, but that differences in representation may arise due to their differing functions and input and output constraints. Moreover, differences in the effects of attention on shape and spatial population representations in the two visual streams suggest two distinct strategies: in a ventral area, attention or task demands modulate the population representations themselves (perhaps to expand or enhance one part at the expense of other parts) while in a dorsal area, at a population representation level, attention effects are weak and nearly non-existent, perhaps in order to maintain veridical representations needed for visuomotor control. We show that an intrinsic approach, as opposed to theory-driven and labeled approaches, is useful for understanding how representations develop and differ across brain regions. Most importantly, these approaches help link cellular properties more tightly with behavior, a much-needed step to better understand and interpret cellular findings and key to providing insights to improve interventions in human disorders.

How do the two visual streams interact with each other?

The current consensus divides primate cortical visual processing into two broad networks or "streams" composed of highly interconnected areas (Milner and Goodale 2006, 2008; Goodale 2014). The ventral stream, passing from primary visual cortex (V1) through to inferior parts of the temporal lobe, is considered to mediate the transformation of the contents of the visual signal into the mental furniture that guides memory, recognition and conscious perception. In contrast the dorsal stream, passing from V1 through to various areas in the posterior parietal lobe, is generally considered to mediate the visual guidance of action, primarily in real time. The brain, however, does not work through mutually insulated subsystems, and indeed there are well-documented interconnections between the two streams. Evidence for contributions from ventral stream systems to the dorsal stream comes from human neuropsychological and neuroimaging research, and indicates a crucial role in mediating complex and flexible visuomotor skills. Complementary evidence points to a role for posterior dorsal-stream visual analysis in certain aspects of 3-D perceptual function in the ventral stream. A series of studies of a patient with visual form agnosia has been instrumental in shaping our knowledge of what each stream can achieve in isolation; but it has also helped us to tease apart the relative dependence of parietal visuomotor systems on direct bottom-up visual inputs versus inputs redirected via perceptual systems within the ventral stream.

Topographic organization of areas V3 and V4 and its relation to supra-areal organization of the primate visual system

Areas V3 and V4 are commonly thought of as individual entities in the primate visual system, based on definition criteria such as their representation of visual space, connectivity, functional response properties, and relative anatomical location in cortex. Yet, large-scale functional and anatomical organization patterns not only emphasize distinctions within each area, but also links across visual cortex. Specifically, the visuotopic organization of V3 and V4 appears to be part of a larger, supra-areal organization, clustering these areas with early visual areas V1 and V2. In addition, connectivity patterns across visual cortex appear to vary within these areas as a function of their supra-areal eccentricity organization. This complicates the traditional view of these regions as individual functional "areas." Here, we will review the criteria for defining areas V3 and V4 and will discuss functional and anatomical studies in humans and monkeys that emphasize the integration of individual visual areas into broad, supra-areal clusters that work in concert for a common computational goal. Specifically, we propose that the visuotopic organization of V3 and V4, which provides the criteria for differentiating these areas, also unifies these areas into the supra-areal organization of early visual cortex. We propose that V3 and V4 play a critical role in this supra-areal organization by filtering information about the visual environment along parallel pathways across higher-order cortex.

Decoding neural representational spaces using multivariate pattern analysis

A major challenge for systems neuroscience is to break the neural code. Computational algorithms for encoding information into neural activity and extracting information from measured activity afford understanding of how percepts, memories, thought, and knowledge are represented in patterns of brain activity. The past decade and a half has seen significant advances in the development of methods for decoding human neural activity, such as multivariate pattern classification, representational similarity analysis, hyperalignment, and stimulus-model-based encoding and decoding. This article reviews these advances and integrates neural decoding methods into a common framework organized around the concept of high-dimensional representational spaces.

Hands in motion: an upper-limb-selective area in the occipitotemporal cortex shows sensitivity to viewed hand kinematics

Regions in the occipitotemporal cortex (OTC) show clear selectivity to static images of human body parts, and upper limbs in particular, with respect to other object categories. Such selectivity was previously attributed to shape aspects, which presumably vary across categories. Alternatively, it has been proposed that functional selectivity for upper limbs is driven by processing of their distinctive motion features. In the present study we show that selectivity to static upper-limb images and motion processing go hand in hand. Using resting-state and task-based functional MRI, we demonstrate that OTC voxels showing greater preference to static images of arms and hands also show stronger functional connectivity with motion coding regions within the human middle temporal complex (hMT+), but not with shape-selective midtier areas, such as hV4 or LO-1, suggesting a tight link between upper-limb selectivity and motion processing. To test this directly, we created a set of natural arm-movement videos where kinematic patterns were parametrically manipulated, while keeping shape information constant. Using multivariate pattern analysis, we show that the degree of (dis)similarity in arm-velocity profiles across the video set predicts, to a significant extent, the degree of (dis)similarity in multivoxel activation patterns in both upper-limb-selective OTC regions and the hMT+. Together, these results suggest that the functional specificity of upper-limb-selective regions may be partially determined by their involvement in the processing of upper-limb dynamics. We propose that the selectivity to static upper-limb images in the OTC may be a result of experience-dependent association between shape elements, which characterize upper limbs, and upper-limb-specific motion patterns.

A mouse model of higher visual cortical function

During sensory experience, the retina transmits a diverse array of signals to the brain, which must be parsed to generate meaningful percepts that can guide decisions and actions. Decades of anatomical and physiological studies in primates and carnivores have revealed a complex parallel and hierarchical organization by which distinct visual features are distributed to, and processed by, different brain regions. However, these studies have been limited in their ability to dissect the circuit mechanisms involved in the transformation of sensory inputs into complex cortical representations and action patterns. Multiple groups have therefore pushed to explore the organization and function of higher visual areas in the mouse. Here we review the anatomical and physiological findings of these recent explorations in mouse visual cortex. These studies find that sensory input is processed in a diverse set of higher areas that are each interconnected with specific limbic and motor systems. This hierarchical and parallel organization is consistent with the multiple streams that have been found in the higher visual areas of primates. We therefore propose that the mouse visual system is a useful model to explore the circuits underlying the transformation of sensory inputs into goal-directed perceptions and actions.

Whole person-evoked fMRI activity patterns in human fusiform gyrus are accurately modeled by a linear combination of face- and body-evoked activity patterns

Visual cues from the face and the body provide information about another's identity, emotional state, and intentions. Previous neuroimaging studies that investigated neural responses to (bodiless) faces and (headless) bodies have reported overlapping face- and body-selective brain regions in right fusiform gyrus (FG). In daily life, however, faces and bodies are typically perceived together and are effortlessly integrated into the percept of a whole person, raising the possibility that neural responses to whole persons are qualitatively different than responses to isolated faces and bodies. The present study used fMRI to examine how FG activity in response to a whole person relates to activity in response to the same face and body but presented in isolation. Using multivoxel pattern analysis, we modeled person-evoked response patterns in right FG through a linear combination of face- and body-evoked response patterns. We found that these synthetic patterns were able to accurately approximate the response patterns to whole persons, with face and body patterns each adding unique information to the response patterns evoked by whole person stimuli. These results suggest that whole person responses in FG primarily arise from the coactivation of independent face- and body-selective neural populations.

A predictive network model of cerebral cortical connectivity based on a distance rule

Recent advances in neuroscience have engendered interest in large-scale brain networks. Using a consistent database of cortico-cortical connectivity, generated from hemisphere-wide, retrograde tracing experiments in the macaque, we analyzed interareal weights and distances to reveal an important organizational principle of brain connectivity. Using appropriate graph theoretical measures, we show that although very dense (66%), the interareal network has strong structural specificity. Connection weights exhibit a heavy-tailed lognormal distribution spanning five orders of magnitude and conform to a distance rule reflecting exponential decay with interareal separation. A single-parameter random graph model based on this rule predicts numerous features of the cortical network: (1) the existence of a network core and the distribution of cliques, (2) global and local binary properties, (3) global and local weight-based communication efficiencies modeled as network conductance, and (4) overall wire-length minimization. These findings underscore the importance of distance and weight-based heterogeneity in cortical architecture and processing.

Tripartite organization of the ventral stream by animacy and object size

Occipito-temporal cortex is known to house visual object representations, but the organization of the neural activation patterns along this cortex is still being discovered. Here we found a systematic, large-scale structure in the neural responses related to the interaction between two major cognitive dimensions of object representation: animacy and real-world size. Neural responses were measured with functional magnetic resonance imaging while human observers viewed images of big and small animals and big and small objects. We found that real-world size drives differential responses only in the object domain, not the animate domain, yielding a tripartite distinction in the space of object representation. Specifically, cortical zones with distinct response preferences for big objects, all animals, and small objects, are arranged in a spoked organization around the occipital pole, along a single ventromedial, to lateral, to dorsomedial axis. The preference zones are duplicated on the ventral and lateral surface of the brain. Such a duplication indicates that a yet unknown higher-order division of labor separates object processing into two substreams of the ventral visual pathway. Broadly, we suggest that these large-scale neural divisions reflect the major joints in the representational structure of objects and thus place informative constraints on the nature of the underlying cognitive architecture.

The role of long-range connections on the specificity of the macaque interareal cortical network

We investigated the influence of interareal distance on connectivity patterns in a database obtained from the injection of retrograde tracers in 29 areas distributed over six regions (occipital, temporal, parietal, frontal, prefrontal, and limbic). One-third of the 1,615 pathways projecting to the 29 target areas were reported only recently and deemed new-found projections (NFPs). NFPs are predominantly long-range, low-weight connections. A minimum dominating set analysis (a graph theoretic measure) shows that NFPs play a major role in globalizing input to small groups of areas. Randomization tests show that (i) NFPs make important contributions to the specificity of the connectivity profile of individual cortical areas, and (ii) NFPs share key properties with known connections at the same distance. We developed a similarity index, which shows that intraregion similarity is high, whereas the interregion similarity declines with distance. For area pairs, there is a steep decline with distance in the similarity and probability of being connected. Nevertheless, the present findings reveal an unexpected binary specificity despite the high density (66%) of the cortical graph. This specificity is made possible because connections are largely concentrated over short distances. These findings emphasize the importance of long-distance connections in the connectivity profile of an area. We demonstrate that long-distance connections are particularly prevalent for prefrontal areas, where they may play a prominent role in large-scale communication and information integration.

A weighted and directed interareal connectivity matrix for macaque cerebral cortex

Retrograde tracer injections in 29 of the 91 areas of the macaque cerebral cortex revealed 1,615 interareal pathways, a third of which have not previously been reported. A weight index (extrinsic fraction of labeled neurons [FLNe]) was determined for each area-to-area pathway. Newly found projections were weaker on average compared with the known projections; nevertheless, the 2 sets of pathways had extensively overlapping weight distributions. Repeat injections across individuals revealed modest FLNe variability given the range of FLNe values (standard deviation <1 log unit, range 5 log units). The connectivity profile for each area conformed to a lognormal distribution, where a majority of projections are moderate or weak in strength. In the G_{29 × 29} interareal subgraph, two-thirds of the connections that can exist do exist. Analysis of the smallest set of areas that collects links from all 91 nodes of the G_{29 × 91} subgraph (dominating set analysis) confirms the dense (66%) structure of the cortical matrix. The G_{29 × 29} subgraph suggests an unexpectedly high incidence of unidirectional links. The directed and weighted G_{29 × 91} connectivity matrix for the macaque will be valuable for comparison with connectivity analyses in other species, including humans. It will also inform future modeling studies that explore the regularities of cortical networks.

Division of labor between lateral and ventral extrastriate representations of faces, bodies, and objects

The occipito-temporal cortex is strongly implicated in carrying out the high-level computations associated with vision. In human neuroimaging studies, focal regions are consistently found within this broad region that respond strongly and selectively to faces, bodies, or objects. A notable feature of these selective regions is that they are found in pairs. In the posterior-lateral occipito-temporal cortex, focal selectivity is found for faces (occipital face area), bodies (extrastriate body area), and objects (lateral occipital). These three areas are found bilaterally and at close quarters to each other. Likewise, in the ventro-medial occipito-temporal cortex, three similar category-selective regions are found, also in proximity to each other: for faces (fusiform face area), bodies (fusiform body area), and objects (posterior fusiform). Here we review some of the extensive evidence on the functional properties of these areas with two aims. First, we seek to identify principles that distinguish the posterior-lateral and ventro-medial clusters of selective regions but that apply generally within each cluster across the three stimulus kinds. Our review identifies and elaborates several principles by which these relationships hold. In brief, the posterior-lateral representations are more primitive, local, and stimulus-driven relative to the ventro-medial representations, which in contrast are more invariant to visual features, global, and linked to the subjective percept. Second, because the evidence base of studies that compare both posterior-lateral and ventro-medial representations of faces, bodies, and objects is still relatively small, we seek to provoke more cross-talk among the research strands that are traditionally separate. We identify several promising approaches for such future work.

Human consciousness and its relationship to social neuroscience: A novel hypothesis

A common modern view of consciousness is that it is an emergent property of the brain, perhaps caused by neuronal complexity, and perhaps with no adaptive value. Exactly what emerges, how it emerges, and from what specific neuronal process, is in debate. One possible explanation of consciousness, proposed here, is that it is a construct of the social perceptual machinery. Humans have specialized neuronal machinery that allows us to be socially intelligent. The primary role for this machinery is to construct models of other people's minds thereby gaining some ability to predict the behavior of other individuals. In the present hypothesis, awareness is a perceptual reconstruction of attentional state; and the machinery that computes information about other people's awareness is the same machinery that computes information about our own awareness. The present article brings together a variety of lines of evidence including experiments on the neural basis of social perception, on hemispatial neglect, on the out-of-body experience, on mirror neurons, and on the mechanisms of decision-making, to explore the possibility that awareness is a construct of the social machinery in the brain.

A new neural framework for visuospatial processing

The division of cortical visual processing into distinct dorsal and ventral streams is a key framework that has guided visual neuroscience. The characterization of the ventral stream as a 'What' pathway is relatively uncontroversial, but the nature of dorsal stream processing is less clear. Originally proposed as mediating spatial perception ('Where'), more recent accounts suggest it primarily serves non-conscious visually guided action ('How'). Here, we identify three pathways emerging from the dorsal stream that consist of projections to the prefrontal and premotor cortices, and a major projection to the medial temporal lobe that courses both directly and indirectly through the posterior cingulate and retrosplenial cortices. These three pathways support both conscious and non-conscious visuospatial processing, including spatial working memory, visually guided action and navigation, respectively.