A principle for the formation of the spatial structure of cortical feature maps

A unifying framework for functional organization in early and higher ventral visual cortex

A key feature of cortical systems is functional organization: the arrangement of functionally distinct neurons in characteristic spatial patterns. However, the principles underlying the emergence of functional organization in the cortex are poorly understood. Here, we develop the topographic deep artificial neural network (TDANN), the first model to predict several aspects of the functional organization of multiple cortical areas in the primate visual system. We analyze the factors driving the TDANN's success and find that it balances two objectives: learning a task-general sensory representation and maximizing the spatial smoothness of responses according to a metric that scales with cortical surface area. In turn, the representations learned by the TDANN are more brain-like than in spatially unconstrained models. Finally, we provide evidence that the TDANN's functional organization balances performance with between-area connection length. Our results offer a unified principle for understanding the functional organization of the primate ventral visual system.

Natural statistics support a rational account of confidence biases

Previous work has sought to understand decision confidence as a prediction of the probability that a decision will be correct, leading to debate over whether these predictions are optimal, and whether they rely on the same decision variable as decisions themselves. This work has generally relied on idealized, low-dimensional models, necessitating strong assumptions about the representations over which confidence is computed. To address this, we used deep neural networks to develop a model of decision confidence that operates directly over high-dimensional, naturalistic stimuli. The model accounts for a number of puzzling dissociations between decisions and confidence, reveals a rational explanation of these dissociations in terms of optimization for the statistics of sensory inputs, and makes the surprising prediction that, despite these dissociations, decisions and confidence depend on a common decision variable.

Cortical topographic motifs emerge in a self-organized map of object space

The human ventral visual stream has a highly systematic organization of object information, but the causal pressures driving these topographic motifs are highly debated. Here, we use self-organizing principles to learn a topographic representation of the data manifold of a deep neural network representational space. We find that a smooth mapping of this representational space showed many brain-like motifs, with a large-scale organization by animacy and real-world object size, supported by mid-level feature tuning, with naturally emerging face- and scene-selective regions. While some theories of the object-selective cortex posit that these differently tuned regions of the brain reflect a collection of distinctly specified functional modules, the present work provides computational support for an alternate hypothesis that the tuning and topography of the object-selective cortex reflect a smooth mapping of a unified representational space.

A Unifying Principle for the Functional Organization of Visual Cortex

A key feature of many cortical systems is functional organization: the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN's success and find that it strikes a balance between two specific objectives: achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN's functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

Unification of free energy minimization, spatiotemporal energy, and dimension reduction models of V1 organization: Postnatal learning on an antenatal scaffold

Developmental selection of neurons and synapses so as to maximize pulse synchrony has recently been used to explain antenatal cortical development. Consequences of the same selection process—an application of the Free Energy Principle—are here followed into the postnatal phase in V1, and the implications for cognitive function are considered. Structured inputs transformed via lag relay in superficial patch connections lead to the generation of circumferential synaptic connectivity superimposed upon the antenatal, radial, “like-to-like” connectivity surrounding each singularity. The spatiotemporal energy and dimension reduction models of cortical feature preferences are accounted for and unified within the expanded model, and relationships of orientation preference (OP), space frequency preference (SFP), and temporal frequency preference (TFP) are resolved. The emergent anatomy provides a basis for “active inference” that includes interpolative modification of synapses so as to anticipate future inputs, as well as learn directly from present stimuli. Neurodynamic properties are those of heteroclinic networks with coupled spatial eigenmodes.

A connectivity-constrained computational account of topographic organization in primate high-level visual cortex

Inferotemporal (IT) cortex in humans and other primates is topographically organized, containing multiple hierarchically organized areas selective for particular domains, such as faces and scenes. This organization is commonly viewed in terms of evolved domain-specific visual mechanisms. Here, we develop an alternative, domain-general and developmental account of IT cortical organization. The account is instantiated in interactive topographic networks (ITNs), a class of computational models in which a hierarchy of model IT areas, subject to biologically plausible connectivity-based constraints, learns high-level visual representations optimized for multiple domains. We find that minimizing a wiring cost on spatially organized feedforward and lateral connections, alongside realistic constraints on the sign of neuronal connectivity within model IT, results in a hierarchical, topographic organization. This organization replicates a number of key properties of primate IT cortex, including the presence of domain-selective spatial clusters preferentially involved in the representation of faces, objects, and scenes; columnar responses across separate excitatory and inhibitory units; and generic spatial organization whereby the response correlation of pairs of units falls off with their distance. We thus argue that topographic domain selectivity is an emergent property of a visual system optimized to maximize behavioral performance under generic connectivity-based constraints.

A connectivity-constrained computational account of topographic organization in primate high-level visual cortex

Inferotemporal (IT) cortex in humans and other primates is topographically organized, containing multiple hierarchically organized areas selective for particular domains, such as faces and scenes. This organization is commonly viewed in terms of evolved domain-specific visual mechanisms. Here, we develop an alternative, domain-general and developmental account of IT cortical organization. The account is instantiated in interactive topographic networks (ITNs), a class of computational models in which a hierarchy of model IT areas, subject to biologically plausible connectivity-based constraints, learns high-level visual representations optimized for multiple domains. We find that minimizing a wiring cost on spatially organized feedforward and lateral connections, alongside realistic constraints on the sign of neuronal connectivity within model IT, results in a hierarchical, topographic organization. This organization replicates a number of key properties of primate IT cortex, including the presence of domain-selective spatial clusters preferentially involved in the representation of faces, objects, and scenes; columnar responses across separate excitatory and inhibitory units; and generic spatial organization whereby the response correlation of pairs of units falls off with their distance. We thus argue that topographic domain selectivity is an emergent property of a visual system optimized to maximize behavioral performance under generic connectivity-based constraints.

Punctuated evolution of visual cortical circuits? Evidence from the large rodent Dasyprocta leporina, and the tiny primate Microcebus murinus

Recent reports of the lack of periodic orientation columns in a very large rodent species, the red-rumped agouti, and the existence of incompressible hypercolumns in the lineage of primates, as demonstrated in one of the smallest primates, the mouse lemur, strengthen the interpretation that salt-and-pepper and columns-and-pinwheel mosaics are two distinct functional layouts. These layouts do neither depend on lifestyle nor scale with body size, brain size, absolute neuron numbers, binocular overlap, or visual acuity, but are primarily distinguishable by phylogenetic traits. The predictive value of other biological signatures such as V1 neuronal surface density and the central-peripheral density ratio of retinal ganglion cells are reconsidered, and experiments elucidating the intracortical connectivity in rodents are proposed.

Learning Intuitive Physics and One-Shot Imitation Using State-Action-Prediction Self-Organizing Maps

Human learning and intelligence work differently from the supervised pattern recognition approach adopted in most deep learning architectures. Humans seem to learn rich representations by exploration and imitation, build causal models of the world, and use both to flexibly solve new tasks. We suggest a simple but effective unsupervised model which develops such characteristics. The agent learns to represent the dynamical physical properties of its environment by intrinsically motivated exploration and performs inference on this representation to reach goals. For this, a set of self-organizing maps which represent state-action pairs is combined with a causal model for sequence prediction. The proposed system is evaluated in the cartpole environment. After an initial phase of playful exploration, the agent can execute kinematic simulations of the environment's future and use those for action planning. We demonstrate its performance on a set of several related, but different one-shot imitation tasks, which the agent flexibly solves in an active inference style.

The Interplay of Synaptic Plasticity and Scaling Enables Self-Organized Formation and Allocation of Multiple Memory Representations

It is commonly assumed that memories about experienced stimuli are represented by groups of highly interconnected neurons called cell assemblies. This requires allocating and storing information in the neural circuitry, which happens through synaptic weight adaptations at different types of synapses. In general, memory allocation is associated with synaptic changes at feed-forward synapses while memory storage is linked with adaptation of recurrent connections. It remains, however, largely unknown how memory allocation and storage can be achieved and the adaption of the different synapses involved be coordinated to allow for a faithful representation of multiple memories without disruptive interference between them. In this theoretical study, by using network simulations and phase space analyses, we show that the interplay between long-term synaptic plasticity and homeostatic synaptic scaling organizes simultaneously the adaptations of feed-forward and recurrent synapses such that a new stimulus forms a new memory and where different stimuli are assigned to distinct cell assemblies. The resulting dynamics can reproduce experimental in-vivo data, focusing on how diverse factors, such as neuronal excitability and network connectivity, influence memory formation. Thus, the here presented model suggests that a few fundamental synaptic mechanisms may suffice to implement memory allocation and storage in neural circuitry.

Development and Reorganization of Orientation Representation in the Cat Visual Cortex: Experience-Dependent Synaptic Rewiring in Early Life

To date, numerous mathematical models have been proposed on the basis of some types of Hebbian synaptic learning to account for the activity-dependent development of orientation maps as well as neuronal orientation selectivity. These models successfully reproduced orientation map-like spatial patterns. Nevertheless, we still have questions: (1) How does synaptic rewiring occur in the visual cortex during the formation of orderly orientation maps in early life? (2) How does visual experience contribute to the maturation of orientation selectivity of visual cortical neurons and reorganize orientation maps? (3) How does the sensitive period for orientation plasticity end? In this study, we performed animal experiments and mathematical modeling to understand the mechanisms underlying synaptic rewiring for experience-dependent formation and reorganization of orientation maps. At first, we visualized orientation maps from the intrinsic signal optical imaging in area 17 of kittens reared under single-orientation exposure through cylindrical-lens-fitted goggles. The experiments revealed that the degree of expansion of cortical domains representing the experienced orientation depends on the age at which the single-orientation exposure starts. As a result, we obtained the sensitive period profile for orientation plasticity. Next, we refined our previously proposed mathematical model for the activity-dependent self-organization of thalamo-cortical inputs on the assumption that rewiring is caused by the competitive interactions among transient synaptic contacts on the same dendritic spine. Although various kinds of molecules have been reported to be involved in such interactions, we attempt to build a mathematical model to describe synaptic rewiring focusing on brain-derived neurotrophic factor (BDNF) and its related molecules. Performing computer simulations based on the refined model, we successfully reproduced orientation maps reorganized in kittens reared under single-orientation exposure as well as normal visual experience. We also reproduced the experimentally obtained sensitive period profile for orientation plasticity. The excellent agreement between experimental observations and theoretical reproductions suggests that the BDNF-induced competitive interaction among synaptic contacts from different axons on the same spine is an important factor for the experience-dependent formation and reorganization of orientation selectivity and orientation maps.

The growth of cognition: Free energy minimization and the embryogenesis of cortical computation

The assumption that during cortical embryogenesis neurons and synaptic connections are selected to form an ensemble maximising synchronous oscillation explains mesoscopic cortical development, and a mechanism for cortical information processing is implied by consistency with the Free Energy Principle and Dynamic Logic. A heteroclinic network emerges, with stable and unstable fixed points of oscillation corresponding to activity in symmetrically connected, versus asymmetrically connected, sets of neurons. Simulations of growth explain a wide range of anatomical observations for columnar and non-columnar cortex, superficial patch connections, and the organization and dynamic interactions of neurone response properties. An antenatal scaffold is created, upon which postnatal learning can establish continuously ordered neuronal representations, permitting matching of co-synchronous fields in multiple cortical areas to solve optimization problems as in Dynamic Logic. Fast synaptic competition partitions equilibria, minimizing "the curse of dimensionality", while perturbations between imperfectly partitioned synchronous fields, under internal reinforcement, enable the cortex to become adaptively self-directed. As learning progresses variational free energy is minimized and entropy bounded.

Computational Foundations of Natural Intelligence

New developments in AI and neuroscience are revitalizing the quest to understanding natural intelligence, offering insight about how to equip machines with human-like capabilities. This paper reviews some of the computational principles relevant for understanding natural intelligence and, ultimately, achieving strong AI. After reviewing basic principles, a variety of computational modeling approaches is discussed. Subsequently, I concentrate on the use of artificial neural networks as a framework for modeling cognitive processes. This paper ends by outlining some of the challenges that remain to fulfill the promise of machines that show human-like intelligence.

Topographic Independent Component Analysis reveals random scrambling of orientation in visual space

Neurons at primary visual cortex (V1) in humans and other species are edge filters organized in orientation maps. In these maps, neurons with similar orientation preference are clustered together in iso-orientation domains. These maps have two fundamental properties: (1) retinotopy, i.e. correspondence between displacements at the image space and displacements at the cortical surface, and (2) a trade-off between good coverage of the visual field with all orientations and continuity of iso-orientation domains in the cortical space. There is an active debate on the origin of these locally continuous maps. While most of the existing descriptions take purely geometric/mechanistic approaches which disregard the network function, a clear exception to this trend in the literature is the original approach of Hyvärinen and Hoyer based on infomax and Topographic Independent Component Analysis (TICA). Although TICA successfully addresses a number of other properties of V1 simple and complex cells, in this work we question the validity of the orientation maps obtained from TICA. We argue that the maps predicted by TICA can be analyzed in the retinal space, and when doing so, it is apparent that they lack the required continuity and retinotopy. Here we show that in the orientation maps reported in the TICA literature it is easy to find examples of violation of the continuity between similarly tuned mechanisms in the retinal space, which suggest a random scrambling incompatible with the maps in primates. The new experiments in the retinal space presented here confirm this guess: TICA basis vectors actually follow a random salt-and-pepper organization back in the image space. Therefore, the interesting clusters found in the TICA topology cannot be interpreted as the actual cortical orientation maps found in cats, primates or humans. In conclusion, Topographic ICA does not reproduce cortical orientation maps.

The temporal-spatial dynamics of feature maps during monocular deprivation revealed by chronic imaging and self-organization model simulation

Experiments on the adult visual cortex of cats, ferrets and monkeys have revealed organized spatial relationships between multiple feature maps which can also be reproduced by the Kohonen and elastic net self-organization models. However, attempts to apply these models to simulate the temporal kinetics of monocular deprivation (MD) during the critical period, and their effects on the spatial arrangement of feature maps, have led to conflicting results. In this study, we performed MD and chronic imaging in the ferret visual cortex during the critical period of ocular dominance (OD) plasticity. We also used the Kohonen model to simulate the effects of MD on OD and orientation map development. Both the experiments and simulations demonstrated two general parameter-insensitive findings. Specifically, our first finding demonstrated that the OD index shift resulting from MD, and its subsequent recovery during binocular vision (BV), were both nonlinear, with a significantly stronger shift occurring during the initial period. Meanwhile, spatial reorganization of feature maps led to globally unchanged but locally shifted map patterns. In detail, we found that the periodicity of OD and orientation maps remained unchanged during, and after, deprivation. Relationships between OD and orientation maps remained similar but were significantly weakened due to OD border shifts. These results indicate that orthogonal gradient relationships between maps may be preset and are only mildly modifiable during the critical period. The Kohonen model was able to reproduce these experimental results, hence its role is further extended to the description of cortical feature map dynamics during development.

Further Work on the Shaping of Cortical Development and Function by Synchrony and Metabolic Competition

This paper furthers our attempts to resolve two major controversies-whether gamma synchrony plays a role in cognition, and whether cortical columns are functionally important. We have previously argued that the configuration of cortical cells that emerges in development is that which maximizes the magnitude of synchronous oscillation and minimizes metabolic cost. Here we analyze the separate effects in development of minimization of axonal lengths, and of early Hebbian learning and selective distribution of resources to growing synapses, by showing in simulations that these effects are partially antagonistic, but their interaction during development produces accurate anatomical and functional properties for both columnar and non-columnar cortex. The resulting embryonic anatomical order can provide a cortex-wide scaffold for postnatal learning that is dimensionally consistent with the representation of moving sensory objects, and, as learning progressively overwrites the embryonic order, further associations also occur in a dimensionally consistent framework. The role ascribed to cortical synchrony does not demand specific frequency, amplitude or phase variation of pulses to mediate "feature linking." Instead, the concerted interactions of pulse synchrony with short-term synaptic dynamics, and synaptic resource competition can further explain cortical information processing in analogy to Hopfield networks and quantum computation.

Cortical Maps

In this article, we review functional organization in sensory cortical regions—how the cortex represents the world. We consider four interrelated aspects of cortical organization: (1) the set of receptive fields of individual cortical sensory neurons, (2) how lateral interaction between cortical neurons reflects the similarity of their receptive fields, (3) the spatial distribution of receptive-field properties across the horizontal extent of the cortical tissue, and (4) how the spatial distributions of different receptive-field properties interact with one another. We show how these data are generally well explained by the theory of input-driven self-organization, with a family of computational models of cortical maps offering a parsimonious account for a wide range of map-related phenomena. We then discuss important challenges to this explanation, with respect to the maps present at birth, maps present under activity blockade, the limits of adult plasticity, and the lack of some maps in rodents. Because there is not at present another credible general theory for cortical map development, we conclude by proposing key experiments to help uncover other mechanisms that might also be operating during map development.

Rethinking Cortical Organization

One way to understand the topography of the cerebral cortex is that “like attracts like.” The cortex is organized to maximize nearest neighbor similarity. This principle can explain the separation of the cortex into discrete areas that emphasize different information domains. It can also explain the maps that form within cortical areas. However, because the cortex is two-dimensional, when a parameter space of much higher dimensionality is reduced onto the cortical sheet while optimizing nearest neighbor relationships, the result may lack an obvious global ordering into separate areas. Instead, the topography may consist of partial gradients, fractures, swirls, regions that resemble separate areas in some ways but not others, and in not a lack of topographic maps but an excess of maps overlaid on each other, no one of which seems to be entirely correct. Like a canvas in a gallery of modern art that no two observers interpret the same way, this lack of obvious ordering of high-dimensional spaces onto the cortex might then result in some scientific controversy over the true organization. In this review, the authors suggest that at least some sectors of the cortex do not have a simple global ordering and are better understood as a result of a reduction of a high-dimensional space onto the cortical sheet. The cortical motor system may be an example of this phenomenon. The authors discuss a model of the lateral motor cortex in which a reduction of many parameters onto a simulated cortical sheet results in a complex topographic pattern that matches the actual monkey motor cortex in surprising detail. Some of the ambiguities of topography and areal boundaries that have plagued the attempt to systematize the lateral motor cortex are explained by the model. NEUROSCIENTIST the attempt to syste 13(2):138—147, 2007.

Image and geometry processing with Oriented and Scalable Map

We turn the Self-organizing Map (SOM) into an Oriented and Scalable Map (OS-Map) by generalizing the neighborhood function and the winner selection. The homogeneous Gaussian neighborhood function is replaced with the matrix exponential. Thus we can specify the orientation either in the map space or in the data space. Moreover, we associate the map's global scale with the locality of winner selection. Our model is suited for a number of graphical applications such as texture/image synthesis, surface parameterization, and solid texture synthesis. OS-Map is more generic and versatile than the task-specific algorithms for these applications. Our work reveals the overlooked strength of SOMs in processing images and geometries.

What, if anything, are topological maps for?

What, if anything, is the functional significance of spatial patterning in cortical feature maps? We ask this question of four major theories of cortical map formation: self-organizing maps, wiring optimization, place coding, and reaction-diffusion. We argue that (i) self-organizing maps yield spatial patterning only as a by-product of efficient mechanisms for developing environmentally appropriate distributions of feature preferences, (ii) wiring optimization assumes rather than explains a map-like organization, (iii) place-coding mechanisms can at best explain only a subset of maps in functional terms, and (iv) reaction-diffusion models suggest two factors in the evolution of maps, the first based on efficient development of feature distributions, and the second based on generating feature-specific long-range recurrent cortical circuitry. None of these explanations for the existence of topological maps requires spatial patterning in maps to be useful. Thus despite these useful frameworks for understanding how maps form and how they are wired, the possibility that patterns are merely epiphenomena in the evolution of mammalian neocortex cannot be rejected. The article is intended as a nontechnical introduction to the assumptions and predictions of these four important classes of models, along with other possible functional explanations for maps.

Multimap formation in visual cortex

An extrastriate visual area such as V2 or V4 contains neurons selective for a multitude of complex shapes, all sharing a common topographic organization. Simultaneously developing multiple interdigitated maps--hereafter a "multimap"--is challenging in that neurons must compete to generate a diversity of response types locally, while cooperating with their dispersed same-type neighbors to achieve uniform visual field coverage for their response type at all orientations, scales, etc. Previously proposed map development schemes have relied on smooth spatial interaction functions to establish both topography and columnar organization, but by locally homogenizing cells' response properties, local smoothing mechanisms effectively rule out multimap formation. We found in computer simulations that the key requirements for multimap development are that neurons are enabled for plasticity only within highly active regions of cortex designated "learning eligibility regions" (LERs), but within an LER, each cell's learning rate is determined only by its activity level with no dependence on location. We show that a hybrid developmental rule that combines spatial and activity-dependent learning criteria in this way successfully produces multimaps when the input stream contains multiple distinct feature types, or in the degenerate case of a single feature type, produces a V1-like map with "salt-and-pepper" structure. Our results support the hypothesis that cortical maps containing a fine mixture of different response types, whether in monkey extrastriate cortex, mouse V1 or elsewhere in the cortex, rather than signaling a breakdown of map formation mechanisms at the fine scale, are a product of a generic cortical developmental scheme designed to map cells with a diversity of response properties across a shared topographic space.

Interconnected growing self-organizing maps for auditory and semantic acquisition modeling

Based on the incremental nature of knowledge acquisition, in this study we propose a growing self-organizing neural network approach for modeling the acquisition of auditory and semantic categories. We introduce an Interconnected Growing Self-Organizing Maps (I-GSOM) algorithm, which takes associations between auditory information and semantic information into consideration, in this paper. Direct phonetic–semantic association is simulated in order to model the language acquisition in early phases, such as the babbling and imitation stages, in which no phonological representations exist. Based on the I-GSOM algorithm, we conducted experiments using paired acoustic and semantic training data. We use a cyclical reinforcing and reviewing training procedure to model the teaching and learning process between children and their communication partners. A reinforcing-by-link training procedure and a link-forgetting procedure are introduced to model the acquisition of associative relations between auditory and semantic information. Experimental results indicate that (1) I-GSOM has good ability to learn auditory and semantic categories presented within the training data; (2) clear auditory and semantic boundaries can be found in the network representation; (3) cyclical reinforcing and reviewing training leads to a detailed categorization as well as to a detailed clustering, while keeping the clusters that have already been learned and the network structure that has already been developed stable; and (4) reinforcing-by-link training leads to well-perceived auditory–semantic associations. Our I-GSOM model suggests that it is important to associate auditory information with semantic information during language acquisition. Despite its high level of abstraction, our I-GSOM approach can be interpreted as a biologically-inspired neurocomputational model.

Mechanisms for stable, robust, and adaptive development of orientation maps in the primary visual cortex

Development of orientation maps in ferret and cat primary visual cortex (V1) has been shown to be stable, in that the earliest measurable maps are similar in form to the eventual adult map, robust, in that similar maps develop in both dark rearing and in a variety of normal visual environments, and yet adaptive, in that the final map pattern reflects the statistics of the specific visual environment. How can these three properties be reconciled? Using mechanistic models of the development of neural connectivity in V1, we show for the first time that realistic stable, robust, and adaptive map development can be achieved by including two low-level mechanisms originally motivated from single-neuron results. Specifically, contrast-gain control in the retinal ganglion cells and the lateral geniculate nucleus reduces variation in the presynaptic drive due to differences in input patterns, while homeostatic plasticity of V1 neuron excitability reduces the postsynaptic variability in firing rates. Together these two mechanisms, thought to be applicable across sensory systems in general, lead to biological maps that develop stably and robustly, yet adapt to the visual environment. The modeling results suggest that topographic map stability is a natural outcome of low-level processes of adaptation and normalization. The resulting model is more realistic, simpler, and far more robust, and is thus a good starting point for future studies of cortical map development.

An adaptive neuromorphic model of ocular dominance map using floating gate 'synapse'

A novel analogue CMOS design of a cortical cell, that computes weighted sum of inputs, is presented. The cell's feedback regime exploits the adaptation dynamics of floating gate pFET 'synapse' to perform competitive learning amongst input weights as time-staggered winner take all. A learning rate parameter regulates adaptation time and a bias enforces resource limitation by restricting the number of input branches and winners in a competition. When learning ends, the cell's response favours one input pattern over others to exhibit feature selectivity. Embedded in a 2-D RC grid, these feature selective cells are capable of performing a symmetry breaking pattern formation, observed in some reaction-diffusion models of cortical feature map formation, e.g. ocular dominance. Close similarity with biological networks in terms of adaptability and long term memory indicates that the cell's design is ideally suited for analogue VLSI implementation of Self-Organizing Feature Map (SOFM) models of cortical feature maps.

Versatile networks of simulated spiking neurons displaying winner-take-all behavior

We describe simulations of large-scale networks of excitatory and inhibitory spiking neurons that can generate dynamically stable winner-take-all (WTA) behavior. The network connectivity is a variant of center-surround architecture that we call center-annular-surround (CAS). In this architecture each neuron is excited by nearby neighbors and inhibited by more distant neighbors in an annular-surround region. The neural units of these networks simulate conductance-based spiking neurons that interact via mechanisms susceptible to both short-term synaptic plasticity and STDP. We show that such CAS networks display robust WTA behavior unlike the center-surround networks and other control architectures that we have studied. We find that a large-scale network of spiking neurons with separate populations of excitatory and inhibitory neurons can give rise to smooth maps of sensory input. In addition, we show that a humanoid brain-based-device (BBD) under the control of a spiking WTA neural network can learn to reach to target positions in its visual field, thus demonstrating the acquisition of sensorimotor coordination.

On the dynamics of cortical development: synchrony and synaptic self-organization

We describe a model for cortical development that resolves long-standing difficulties of earlier models. It is proposed that, during embryonic development, synchronous firing of neurons and their competition for limited metabolic resources leads to selection of an array of neurons with ultra-small-world characteristics. Consequently, in the visual cortex, macrocolumns linked by superficial patchy connections emerge in anatomically realistic patterns, with an ante-natal arrangement which projects signals from the surrounding cortex onto each macrocolumn in a form analogous to the projection of a Euclidean plane onto a Möbius strip. This configuration reproduces typical cortical response maps, and simulations of signal flow explain cortical responses to moving lines as functions of stimulus velocity, length, and orientation. With the introduction of direct visual inputs, under the operation of Hebbian learning, development of mature selective response “tuning” to stimuli of given orientation, spatial frequency, and temporal frequency would then take place, overwriting the earlier ante-natal configuration. The model is provisionally extended to hierarchical interactions of the visual cortex with higher centers, and a general principle for cortical processing of spatio-temporal images is sketched.

Coordinated optimization of visual cortical maps (I) symmetry-based analysis

In the primary visual cortex of primates and carnivores, functional architecture can be characterized by maps of various stimulus features such as orientation preference (OP), ocular dominance (OD), and spatial frequency. It is a long-standing question in theoretical neuroscience whether the observed maps should be interpreted as optima of a specific energy functional that summarizes the design principles of cortical functional architecture. A rigorous evaluation of this optimization hypothesis is particularly demanded by recent evidence that the functional architecture of orientation columns precisely follows species invariant quantitative laws. Because it would be desirable to infer the form of such an optimization principle from the biological data, the optimization approach to explain cortical functional architecture raises the following questions: i) What are the genuine ground states of candidate energy functionals and how can they be calculated with precision and rigor? ii) How do differences in candidate optimization principles impact on the predicted map structure and conversely what can be learned about a hypothetical underlying optimization principle from observations on map structure? iii) Is there a way to analyze the coordinated organization of cortical maps predicted by optimization principles in general? To answer these questions we developed a general dynamical systems approach to the combined optimization of visual cortical maps of OP and another scalar feature such as OD or spatial frequency preference. From basic symmetry assumptions we obtain a comprehensive phenomenological classification of possible inter-map coupling energies and examine representative examples. We show that each individual coupling energy leads to a different class of OP solutions with different correlations among the maps such that inferences about the optimization principle from map layout appear viable. We systematically assess whether quantitative laws resembling experimental observations can result from the coordinated optimization of orientation columns with other feature maps.

Deletion of Ten-m3 induces the formation of eye dominance domains in mouse visual cortex

The visual system is characterized by precise retinotopic mapping of each eye, together with exquisitely matched binocular projections. In many species, the inputs that represent the eyes are segregated into ocular dominance columns in primary visual cortex (V1), whereas in rodents, this does not occur. Ten-m3, a member of the Ten-m/Odz/Teneurin family, regulates axonal guidance in the retinogeniculate pathway. Significantly, ipsilateral projections are expanded in the dorsal lateral geniculate nucleus and are not aligned with contralateral projections in Ten-m3 knockout (KO) mice. Here, we demonstrate the impact of altered retinogeniculate mapping on the organization and function of V1. Transneuronal tracing and c-fos immunohistochemistry demonstrate that the subcortical expansion of ipsilateral input is conveyed to V1 in Ten-m3 KOs: Ipsilateral inputs are widely distributed across V1 and are interdigitated with contralateral inputs into eye dominance domains. Segregation is confirmed by optical imaging of intrinsic signals. Single-unit recording shows ipsilateral, and contralateral inputs are mismatched at the level of single V1 neurons, and binocular stimulation leads to functional suppression of these cells. These findings indicate that the medial expansion of the binocular zone together with an interocular mismatch is sufficient to induce novel structural features, such as eye dominance domains in rodent visual cortex.

Online learning and stimulus-driven responses of neurons in visual cortex

In understanding how visual scene is processed in visual cortex, it has been an intriguing problem for theoretical and experimental neuroscientists to examine the relationship between visual stimuli and the induced responses of visual cortex. In particular, it is less explored whether and how the collective responses of visual neurons are patterned to reflect the geometrical regularities. In this paper, through a computation model and statistical analysis, we show that the orientation preference maps induced from correlated visual stimuli exhibit geometrical regularities similar as observed in natural images.

Building a mechanistic model of the development and function of the primary visual cortex

Researchers have used a very wide range of different experimental and theoretical approaches to help understand mammalian visual systems. These approaches tend to have quite different assumptions, strengths, and weaknesses. Computational models of the visual cortex, in particular, have typically implemented either a proposed circuit for part of the visual cortex of the adult, assuming a very specific wiring pattern based on findings from adults, or else attempted to explain the long-term development of a visual cortex region from an initially undifferentiated starting point. Previous models of adult V1 have been able to account for many of the measured properties of V1 neurons, while not explaining how these properties arise or why neurons have those properties in particular. Previous developmental models have been able to reproduce the overall organization of specific feature maps in V1, such as orientation maps, but are generally formulated at an abstract level that does not allow testing with real images or analysis of detailed neural properties relevant for visual function. In this review of results from a large set of new, integrative models developed from shared principles and a set of shared software components, I show how these models now represent a single, consistent explanation for a wide body of experimental evidence, and form a compact hypothesis for much of the development and behavior of neurons in the visual cortex. The models are the first developmental models with wiring consistent with V1, the first to have realistic behavior with respect to visual contrast, and the first to include all of the demonstrated visual feature dimensions. The goal is to have a comprehensive explanation for why V1 is wired as it is in the adult, and how that circuitry leads to the observed behavior of the neurons during visual tasks.

Coverage, continuity, and visual cortical architecture

BACKGROUND

The primary visual cortex of many mammals contains a continuous representation of visual space, with a roughly repetitive aperiodic map of orientation preferences superimposed. It was recently found that orientation preference maps (OPMs) obey statistical laws which are apparently invariant among species widely separated in eutherian evolution. Here, we examine whether one of the most prominent models for the optimization of cortical maps, the elastic net (EN) model, can reproduce this common design. The EN model generates representations which optimally trade of stimulus space coverage and map continuity. While this model has been used in numerous studies, no analytical results about the precise layout of the predicted OPMs have been obtained so far.

RESULTS

We present a mathematical approach to analytically calculate the cortical representations predicted by the EN model for the joint mapping of stimulus position and orientation. We find that in all the previously studied regimes, predicted OPM layouts are perfectly periodic. An unbiased search through the EN parameter space identifies a novel regime of aperiodic OPMs with pinwheel densities lower than found in experiments. In an extreme limit, aperiodic OPMs quantitatively resembling experimental observations emerge. Stabilization of these layouts results from strong nonlocal interactions rather than from a coverage-continuity-compromise.

CONCLUSIONS

Our results demonstrate that optimization models for stimulus representations dominated by nonlocal suppressive interactions are in principle capable of correctly predicting the common OPM design. They question that visual cortical feature representations can be explained by a coverage-continuity-compromise.

The role of long-range connectivity for the characterization of the functional-anatomical organization of the cortex

This review focuses on the role of long-range connectivity as one element of brain structure that is of key importance for the functional–anatomical organization of the cortex. In this context, we discuss the putative guiding principles for mapping brain function and structure onto the cortical surface. Such mappings reveal a high degree of functional–anatomical segregation. Given that brain regions frequently maintain characteristic connectivity profiles and the functional repertoire of a cortical area is closely related to its anatomical connections, long-range connectivity may be used to define segregated cortical areas. This methodology is called connectivity-based parcellation. Within this framework, we investigate different techniques to estimate connectivity profiles with emphasis given to non-invasive methods based on diffusion magnetic resonance imaging (dMRI) and diffusion tractography. Cortical parcellation is then defined based on similarity between diffusion tractograms, and different clustering approaches are discussed. We conclude that the use of non-invasively acquired connectivity estimates to characterize the functional–anatomical organization of the brain is a valid, relevant, and necessary endeavor. Current and future developments in dMRI technology, tractography algorithms, and models of the similarity structure hold great potential for a substantial improvement and enrichment of the results of the technique.

Three-dimensional visual feature representation in the primary visual cortex

In the cat primary visual cortex, it is accepted that neurons optimally responding to similar stimulus orientations are clustered in a column extending from the superficial to deep layers. The cerebral cortex is, however, folded inside a skull, which makes gyri and fundi. The primary visual area of cats, area 17, is located on the fold of the cortex called the lateral gyrus. These facts raise the question of how to reconcile the tangential arrangement of the orientation columns with the curvature of the gyrus. In the present study, we show a possible configuration of feature representation in the visual cortex using a three-dimensional (3D) self-organization model. We took into account preferred orientation, preferred direction, ocular dominance and retinotopy, assuming isotropic interaction. We performed computer simulation only in the middle layer at the beginning and expanded the range of simulation gradually to other layers, which was found to be a unique method in the present model for obtaining orientation columns spanning all the layers in the flat cortex. Vertical columns of preferred orientations were found in the flat parts of the model cortex. On the other hand, in the curved parts, preferred orientations were represented in wedge-like columns rather than straight columns, and preferred directions were frequently reversed in the deeper layers. Singularities associated with orientation representation appeared as warped lines in the 3D model cortex. Direction reversal appeared on the sheets that were delimited by orientation-singularity lines. These structures emerged from the balance between periodic arrangements of preferred orientations and vertical alignment of the same orientations. Our theoretical predictions about orientation representation were confirmed by multi-slice, high-resolution functional MRI in the cat visual cortex. We obtained a close agreement between theoretical predictions and experimental observations. The present study throws a doubt about the conventional columnar view of orientation representation, although more experimental data are needed.

Computational modeling of neuronal map development: insights into disease

The study of the formation of neuronal maps in the brain has greatly increased our understanding of how the brain develops and, in some cases, regenerates. Computational modeling of neuronal map development has been invaluable in integrating complex biological phenomena and synthesizing them into quantitative and predictive frameworks. These models allow us to investigate how neuronal map development is perturbed under conditions of altered development, disease and regeneration. In this article, we use examples of activity-dependent and activity-independent models of retinotopic map formation to illustrate how they can aid our understanding of developmental and acquired disease processes. We note that fully extending these models to specific clinically relevant problems is a largely unexplored domain and suggest future work in this direction. We argue that this type of modeling will be necessary in furthering our understanding of the pathophysiology of neurological diseases and in developing treatments for them. Furthermore, we discuss how the nature of computational and theoretical approaches uniquely places them to bridge the gap between the bench and the clinic.

Statistical comparison of spike responses to natural stimuli in monkey area V1 with simulated responses of a detailed laminar network model for a patch of V1

A major goal of computational neuroscience is the creation of computer models for cortical areas whose response to sensory stimuli resembles that of cortical areas in vivo in important aspects. It is seldom considered whether the simulated spiking activity is realistic (in a statistical sense) in response to natural stimuli. Because certain statistical properties of spike responses were suggested to facilitate computations in the cortex, acquiring a realistic firing regimen in cortical network models might be a prerequisite for analyzing their computational functions. We present a characterization and comparison of the statistical response properties of the primary visual cortex (V1) in vivo and in silico in response to natural stimuli. We recorded from multiple electrodes in area V1 of 4 macaque monkeys and developed a large state-of-the-art network model for a 5 × 5-mm patch of V1 composed of 35,000 neurons and 3.9 million synapses that integrates previously published anatomical and physiological details. By quantitative comparison of the model response to the "statistical fingerprint" of responses in vivo, we find that our model for a patch of V1 responds to the same movie in a way which matches the statistical structure of the recorded data surprisingly well. The deviation between the firing regimen of the model and the in vivo data are on the same level as deviations among monkeys and sessions. This suggests that, despite strong simplifications and abstractions of cortical network models, they are nevertheless capable of generating realistic spiking activity. To reach a realistic firing state, it was not only necessary to include both N-methyl-d-aspartate and GABA(B) synaptic conductances in our model, but also to markedly increase the strength of excitatory synapses onto inhibitory neurons (>2-fold) in comparison to literature values, hinting at the importance to carefully adjust the effect of inhibition for achieving realistic dynamics in current network models.

Mapping of contextual modulation in the population response of primary visual cortex

We review the evidence of long-range contextual modulation in V1. Populations of neurons in V1 are activated by a wide variety of stimuli outside of their classical receptive fields (RF), well beyond their surround region. These effects generally involve extra-RF features with an orientation component. The population mapping of orientation preferences to the upper layers of V1 is well understood, as far as the classical RF properties are concerned, and involves organization into pinwheel-like structures. We introduce a novel hypothesis regarding the organization of V1's contextual response. We show that RF and extra-RF orientation preferences are mapped in related ways. Orientation pinwheels are the foci of both types of features. The mapping of contextual features onto the orientation pinwheel has a form that recapitulates the organization of the visual field: an iso-orientation patch within the pinwheel also responds to extra-RF stimuli of the same orientation. We hypothesize that the same form of mapping applies to other stimulus properties that are mapped out in V1, such as colour and contrast selectivity. A specific consequence is that fovea-like properties will be mapped in a systematic way to orientation pinwheels. We review the evidence that cytochrome oxidase blobs comprise the foci of this contextual remapping for colour and low contrasts. Neurodynamics and motion in the visual field are argued to play an important role in the shaping and maintenance of this type of mapping in V1.

A model for learning topographically organized parts-based representations of objects in visual cortex: topographic nonnegative matrix factorization

Object representation in the inferior temporal cortex (IT), an area of visual cortex critical for object recognition in the primate, exhibits two prominent properties: (1) objects are represented by the combined activity of columnar clusters of neurons, with each cluster representing component features or parts of objects, and (2) closely related features are continuously represented along the tangential direction of individual columnar clusters. Here we propose a learning model that reflects these properties of parts-based representation and topographic organization in a unified framework. This model is based on a nonnegative matrix factorization (NMF) basis decomposition method. NMF alone provides a parts-based representation where nonnegative inputs are approximated by additive combinations of nonnegative basis functions. Our proposed model of topographic NMF (TNMF) incorporates neighborhood connections between NMF basis functions arranged on a topographic map and attains the topographic property without losing the parts-based property of the NMF. The TNMF represents an input by multiple activity peaks to describe diverse information, whereas conventional topographic models, such as the self-organizing map (SOM), represent an input by a single activity peak in a topographic map. We demonstrate the parts-based and topographic properties of the TNMF by constructing a hierarchical model for object recognition where the TNMF is at the top tier for learning high-level object features. The TNMF showed better generalization performance over NMF for a data set of continuous view change of an image and more robustly preserving the continuity of the view change in its object representation. Comparison of the outputs of our model with actual neural responses recorded in the IT indicates that the TNMF reconstructs the neuronal responses better than the SOM, giving plausibility to the parts-based learning of the model.

Theoretical models of neural circuit development

Proper wiring up of the nervous system is critical to the development of organisms capable of complex and adaptable behaviors. Besides the many experimental advances in determining the cellular and molecular machinery that carries out this remarkable task precisely and robustly, theoretical approaches have also proven to be useful tools in analyzing this machinery. A quantitative understanding of these processes can allow us to make predictions, test hypotheses, and appraise established concepts in a new light. Three areas that have been fruitful in this regard are axon guidance, retinotectal mapping, and activity-dependent development. This chapter reviews some of the contributions made by mathematical modeling in these areas, illustrated by important examples of models in each section. For axon guidance, we discuss models of how growth cones respond to their environment, and how this environment can place constraints on growth cone behavior. Retinotectal mapping looks at computational models for how topography can be generated in populations of neurons based on molecular gradients and other mechanisms such as competition. In activity-dependent development, we discuss theoretical approaches largely based on Hebbian synaptic plasticity rules, and how they can generate maps in the visual cortex very similar to those seen in vivo. We show how theoretical approaches have substantially contributed to the advancement of developmental neuroscience, and discuss future directions for mathematical modeling in the field.

Quantification of optical images of cortical responses for inferring functional maps

Optical imaging of cortical signals enables the mapping of functional organization across large patches of cortex with good spatial resolution. But techniques for the quantitative analysis and interpretation of these images are limited. Frequently the functional architecture of the cortex is inferred from the visible topography of cortical reflectance images averaged or differenced across stimulus conditions and scaled or color-coded for presentation. Such qualitative assessments have sometimes led to divergent conclusions particularly about the organization of spatial and temporal frequency preferences in the primary visual cortex. We applied quantitative methods derived from signal detection theory to objectively interpret optical images. The differential response to any two arbitrary stimuli was represented at each pixel as the probability of discriminating between the two stimuli given the reflectance values at that pixel. These probability maps reduced false alarms and provided better signal-to-noise ratio in fewer trials than difference maps. We applied these methods to optical images of primate primary visual area (V1) obtained in response to sinusoidal gratings of different orientations and spatiotemporal frequencies. Clustering by orientation preference was stronger than that for spatial frequency, whereas clustering by temporal frequency preference was the weakest, largely in agreement with a previous electrophysiological study that quantified the degree of clustering of neurons for various response properties using uniform, quantitative criterion. We suggest that probability maps can extend the applicability of optical imaging to investigations of finer aspects of cortical functional organization through better signal-to-noise ratio and uniform, quantitative criteria for interpretation.

Cooperation in self-organizing map networks enhances information transmission in the presence of input background activity

The self-organizing map (SOM) algorithm produces artificial neural maps by simulating competition and cooperation among neurons. We study the consequences of input background activity on simulated self-organization, using the SOM, of the retinotopic map in the superior colliculus. The colliculus not only represents its inputs but also uses them to localize saccadic targets. Using the colliculus as a test-bed enables us to quantify the results of self- organization both descriptively, in terms of input-output mutual information, and functionally, in terms of the probability of error (expected distortion) in localizing targets. We find that mutual information is low, and distortion is high, when the SOM operates in the presence of input background activity but without the cooperative component (no neighbor training). Cooperation (training neighbors) greatly increases mutual information and greatly decreases expected distortion. Our simulation results extend theoretical work suggesting that cooperative mechanisms are needed to increase the information content of neural representations. They also identify input background activity as a factor affecting the self-organization of information-transmitting channels in the nervous system.

A model for the thick, thin and pale stripe organization of primate V2

Models based on the idea of dimension reduction have been successful in describing the patterns of ocular dominance, spatial frequency and orientation preference found in primate V1. It is shown here that this approach can be extended to describe the organization of thick, thin and pale cytochrome oxidase stripes of primate V2 given an appropriately constructed stimulus space which includes a 3-valued variable which co-varies with color, orientation and disparity. The model successfully describes several aspects of V2 organization, including the fact that there are two pale stripes for each thick and thin stripe and the strong tendency for stripes to run perpendicular to the V1 border. In addition it predicts the presence of reversals in the direction of mapping of retinal eccentricity which should be more common in the pale stripes than elsewhere.

Alteration of visual input results in a coordinated reorganization of multiple visual cortex maps

In the adult visual cortex, multiple feature maps exist and have characteristic spatial relationships with one another. The relationships can be reproduced by "dimension-reduction" computational models, suggesting that the principles of continuity and coverage may underlie cortical map organization. However, the mechanisms responsible for establishing these relationships are unknown. We explored whether removing one feature map during development causes a coordinated reorganization of the remaining maps or whether the remaining maps are unaffected. We removed the ocular dominance map by monocular enucleation in newborn ferrets, so that single eye stimulation drove the cortex in a more spatially uniform manner in adult monocular animals compared with normal animals. Maps of orientation, spatial frequency, and retinotopy formed in monocular ferrets, but their structures and spatial relationships differed from those in normal ferrets. The wavelength of the orientation map increased, so that the average orientation gradient across the cortex decreased. The decrease in the orientation gradient in monocular animals was most prominent in the high gradient regions of the spatial frequency map, indicating a coordinated reorganization between these two maps. In monocular animals, the orthogonal relationship between the orientation and spatial frequency maps was preserved, and the orthogonal relationship between the orientation and retinotopic maps became more pronounced. These results were consistent with detailed predictions of a dimension-reduction model of cortical organization. Thus, the number of feature maps in a cortical area influences the relationships between them, and inputs to the cortex have a significant role in generating these relationships.

The effect of angioscotomas on map structure in primary visual cortex

When blood vessels occlude the photoreceptor layer in the retina, they cast shadows onto the photoreceptors, creating angioscotomas (regions of the visual field to which that eye is blind). Remarkably, Adams and Horton (2002) have recently shown that it is sometimes possible to observe representations of these angioscotomas anatomically in the primary visual cortices of squirrel monkeys. However, there is substantial variability in the degree and form of these representations. The source of this variability is difficult to determine experimentally, because experimental studies are unavoidably limited by small sample size. In addition, experimental studies cannot compare the map structure that would develop with and without an angioscotoma. Here, we investigate these phenomena computationally using feature-mapping models of visual cortical development, which are not subject to the same limitations. These models suggest that the primary source of variability in angioscotoma representation is the precise timing of the onset of visual experience relative to the time course of ocular dominance column segregation. Furthermore, the models predict that angioscotomas could compete for control of local column layout with other influences such as cortical shape but that they have a small effect on the structure of orientation preference maps.

Cortical reorganization consistent with spike timing–but not correlation-dependent plasticity

The receptive fields of neurons in primary visual cortex that are inactivated by retinal damage are known to 'shift' to nondamaged retinal locations, seemingly due to the plasticity of intracortical connections. We have observed in cats that these shifts occur in a pattern that is highly convergent, even among receptive fields that are separated by large distances before inactivation. Here we show, using a computational model of primary visual cortex, that the observed convergent shifts are inconsistent with the common assumption that the underlying intracortical connection plasticity is dependent on the temporal correlation of pre- and postsynaptic action potentials. The shifts are, however, consistent with the hypothesis that this plasticity is dependent on the temporal order of pre- and postsynaptic action potentials. This convergent reorganization seems to require increased neuronal gain, revealing a mechanism that networks may use to selectively facilitate the didactic transfer of neuronal response properties.