Building a mechanistic model of the development and function of the primary visual cortex

Extreme image transformations affect humans and machines differently

Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. Their success in object recognition is, however, dependent on exploiting low-level features for solving visual tasks in a way that humans do not. As a result, out-of-distribution or adversarial input is often challenging for ANNs. Humans instead learn abstract patterns and are mostly unaffected by many extreme image distortions. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans. We quantify the differences in accuracy for humans and machines and find a ranking of difficulty for our transforms for human data. We also suggest how certain characteristics of human visual processing can be adapted to improve the performance of ANNs for our difficult-for-machines transforms.

Efficacy of MP-3 microperimeter biofeedback fixation training for low vision rehabilitation in patients with maculopathy

BACKGROUND

To evaluate the efficacy of MP-3 microperimeter biofeedback fixation training (MBFT) in vision rehabilitation of low-vision patients affected by macular disease with central vision loss.

METHODS

Seventeen eyes (7 age-related macular degeneration, 10 myopic maculopathy) of 17 patients were included in this prospective, interventional study. The preferred retinal locus was determined by comprehensive ophthalmoscopic fundus evaluation including fundus photography, autofluorescence, optical coherence tomography, and microperimetry. The rehabilitation consisted of three 10-min sessions per eye to be performed twice per week for 20 consecutive weeks using the MP-3 microperimeter. Best corrected visual acuity (BCVA), reading speed, mean central sensitivity, the percentages of fixation points within specified regions, bivariate contour ellipse area (BCEA) and the 25-item National Eye Institute visual function questionnaire (NEI-VFQ-25) were recorded pre- and post-training.

RESULTS

The final BCVA, reading speed and mean central sensitivity all showed significant improvements after rehabilitation (P <  0.0001, P = 0.0013, and P = 0.0002, respectively). The percentages of fixation points located within 2° and 4° diameter circles both significantly increased after training (P = 0.0008 and P = 0.0007, respectively). The BCEA encompassing 68.2, 95.4, 99.6% of fixation points were all significantly decreased after training (P = 0.0038, P = 0.0022, and P = 0.0021, respectively). The NEI-VFQ-25 scores were significantly increased at the end of the rehabilitation training (P <  0.0001).

CONCLUSION

Rehabilitation with MP-3 MBFT is a user-friendly therapeutic option for improving visual function, fixation stability, and quality of life in advanced macular disease.

TRIAL REGISTRATION

The prospective study was registered with the Chinese Clinical Trial Registry ( http://www.chictr.org.cn/ ).

TRIAL REGISTRATION NUMBER

ChiCTR2000029586 . Date of registration: 05/02/2020.

A Network Architecture for Bidirectional Neurovascular Coupling in Rat Whisker Barrel Cortex

Neurovascular coupling is typically considered as a master-slave relationship between the neurons and the cerebral vessels: the neurons demand energy which the vessels supply in the form of glucose and oxygen. In the recent past, both theoretical and experimental studies have suggested that the neurovascular coupling is a bidirectional system, a loop that includes a feedback signal from the vessels influencing neural firing and plasticity. An integrated model of bidirectionally connected neural network and the vascular network is hence required to understand the relationship between the informational and metabolic aspects of neural dynamics. In this study, we present a computational model of the bidirectional neurovascular system in the whisker barrel cortex and study the effect of such coupling on neural activity and plasticity as manifest in the whisker barrel map formation. In this model, a biologically plausible self-organizing network model of rate coded, dynamic neurons is nourished by a network of vessels modeled using the biophysical properties of blood vessels. The neural layer which is designed to simulate the whisker barrel cortex of rat transmits vasodilatory signals to the vessels. The feedback from the vessels is in the form of available oxygen for oxidative metabolism whose end result is the adenosine triphosphate (ATP) necessary to fuel neural firing. The model captures the effect of the feedback from the vascular network on the neuronal map formation in the whisker barrel model under normal and pathological (Hypoxia and Hypoxia-Ischemia) conditions.

Deep Predictive Learning in Neocortex and Pulvinar

How do humans learn from raw sensory experience? Throughout life, but most obviously in infancy, we learn without explicit instruction. We propose a detailed biological mechanism for the widely embraced idea that learning is driven by the differences between predictions and actual outcomes (i.e., predictive error-driven learning). Specifically, numerous weak projections into the pulvinar nucleus of the thalamus generate top-down predictions, and sparse driver inputs from lower areas supply the actual outcome, originating in Layer 5 intrinsic bursting neurons. Thus, the outcome representation is only briefly activated, roughly every 100 msec (i.e., 10 Hz, alpha), resulting in a temporal difference error signal, which drives local synaptic changes throughout the neocortex. This results in a biologically plausible form of error backpropagation learning. We implemented these mechanisms in a large-scale model of the visual system and found that the simulated inferotemporal pathway learns to systematically categorize 3-D objects according to invariant shape properties, based solely on predictive learning from raw visual inputs. These categories match human judgments on the same stimuli and are consistent with neural representations in inferotemporal cortex in primates.

A model integrating multiple processes of synchronization and coherence for information instantiation within a cortical area

What is the form of dynamic, e.g., sensory, information in the mammalian cortex? Information in the cortex is modeled as a coherence map of a mixed chimera state of synchronous, phasic, and disordered minicolumns. The theoretical model is built on neurophysiological evidence. Complex spatiotemporal information is instantiated through a system of interacting biological processes that generate a synchronized cortical area, a coherent aperture. Minicolumn elements are grouped in macrocolumns in an array analogous to a phased-array radar, modeled as an aperture, a "hole through which radiant energy flows." Coherence maps in a cortical area transform inputs from multiple sources into outputs to multiple targets, while reducing complexity and entropy. Coherent apertures can assume extremely large numbers of different information states as coherence maps, which can be communicated among apertures with corresponding very large bandwidths. The coherent aperture model incorporates considerable reported research, integrating five conceptually and mathematically independent processes: 1) a damped Kuramoto network model, 2) a pumped area field potential, 3) the gating of nearly coincident spikes, 4) the coherence of activity across cortical lamina, and 5) complex information formed through functions in macrocolumns. Biological processes and their interactions are described in equations and a functional circuit such that the mathematical pieces can be assembled the same way the neurophysiological ones are. The model can be conceptually convolved over the specifics of local cortical areas within and across species. A coherent aperture becomes a node in a graph of cortical areas with a corresponding distribution of information.

A review of visual aftereffects in schizophrenia

Psychosis-a cardinal symptom of schizophrenia-has been associated with a failure to appropriately create or use stored regularities about past states of the world to guide the interpretation of incoming information, which leads to abnormal perceptions and beliefs. The visual system provides a test bed for investigating the role of prior experience and prediction, as accumulated knowledge of the world informs our current perception. More specifically, the strength of visual aftereffects, illusory percepts that arise after prolonged viewing of a visual stimulus, can serve as a valuable measure of the influence of prior experience on current visual processing. In this paper, we review findings from a largely older body of work on visual aftereffects in schizophrenia, attempt to reconcile discrepant findings, highlight the role of antipsychotic medication, consider mechanistic interpretations for behavioral effects, and propose directions for future research.

Convis: A Toolbox to Fit and Simulate Filter-Based Models of Early Visual Processing

We developed Convis, a Python simulation toolbox for large scale neural populations which offers arbitrary receptive fields by 3D convolutions executed on a graphics card. The resulting software proves to be flexible and easily extensible in Python, while building on the PyTorch library (The Pytorch Project, 2017), which was previously used successfully in deep learning applications, for just-in-time optimization and compilation of the model onto CPU or GPU architectures. An alternative implementation based on Theano (Theano Development Team, 2016) is also available, although not fully supported. Through automatic differentiation, any parameter of a specified model can be optimized to approach a desired output which is a significant improvement over e.g., Monte Carlo or particle optimizations without gradients. We show that a number of models including even complex non-linearities such as contrast gain control and spiking mechanisms can be implemented easily. We show in this paper that we can in particular recreate the simulation results of a popular retina simulation software VirtualRetina (Wohrer and Kornprobst, 2009), with the added benefit of providing (1) arbitrary linear filters instead of the product of Gaussian and exponential filters and (2) optimization routines utilizing the gradients of the model. We demonstrate the utility of 3d convolution filters with a simple direction selective filter. Also we show that it is possible to optimize the input for a certain goal, rather than the parameters, which can aid the design of experiments as well as closed-loop online stimulus generation. Yet, Convis is more than a retina simulator. For instance it can also predict the response of V1 orientation selective cells. Convis is open source under the GPL-3.0 license and available from https://github.com/jahuth/convis/ with documentation at https://jahuth.github.io/convis/.

Computational Modeling of Contrast Sensitivity and Orientation Tuning in First-Episode and Chronic Schizophrenia

Computational modeling is a useful method for generating hypotheses about the contributions of impaired neurobiological mechanisms, and their interactions, to psychopathology. Modeling is being increasingly used to further our understanding of schizophrenia, but to date, it has not been applied to questions regarding the common perceptual disturbances in the disorder. In this article, we model aspects of low-level visual processing and demonstrate how this can lead to testable hypotheses about both the nature of visual abnormalities in schizophrenia and the relationships between the mechanisms underlying these disturbances and psychotic symptoms. Using a model that incorporates retinal, lateral geniculate nucleus (LGN), and V1 activity, as well as gain control in the LGN, homeostatic adaptation in V1, lateral excitation and inhibition in V1, and self-organization of synaptic weights based on Hebbian learning and divisive normalization, we show that (a) prior data indicating increased contrast sensitivity for low-spatial-frequency stimuli in first-episode schizophrenia can be successfully modeled as a function of reduced retinal and LGN efferent activity, leading to overamplification at the cortical level, and (b) prior data on reduced contrast sensitivity and broadened orientation tuning in chronic schizophrenia can be successfully modeled by a combination of reduced V1 lateral inhibition and an increase in the Hebbian learning rate at V1 synapses for LGN input. These models are consistent with many current findings, and they predict several relationships that have not yet been demonstrated. They also have implications for understanding changes in brain and visual function from the first psychotic episode to the chronic stage of illness.

The influence of astrocytes on the width of orientation hypercolumns in visual cortex: A computational perspective

Orientation preference maps (OPMs) are present in carnivores (such as cats and ferrets) and primates but are absent in rodents. In this study we investigate the possible link between astrocyte arbors and presence of OPMs. We simulate the development of orientation maps with varying hypercolumn widths using a variant of the Laterally Interconnected Synergetically Self-Organizing Map (LISSOM) model, the Gain Control Adaptive Laterally connected (GCAL) model, with an additional layer simulating astrocytic activation. The synaptic activity of V1 neurons is given as input to the astrocyte layer. The activity of this astrocyte layer is now used to modulate bidirectional plasticity of lateral excitatory connections in the V1 layer. By simply varying the radius of the astrocytes, the extent of lateral excitatory neuronal connections can be manipulated. An increase in the radius of lateral excitatory connections subsequently increases the size of a single hypercolumn in the OPM. When these lateral excitatory connections become small enough the OPM disappears and a salt-and-pepper organization emerges.

Columns of neurons in the primary visual cortex (V1) are known to be tuned to visual stimuli containing edges of a particular orientation. The arrangement of these cortical columns varies across species. In many species such as in ferrets, cats, and monkeys a topology preserving map is observed, wherein similarly tuned columns are observed in close proximity to each other, resulting in the formation of Orientation Preference Maps (OPMs). The width of the hypercolumns, the fundamental unit of an OPM, also varies across species. However, such an arrangement is not observed in rodents, wherein a more random arrangement of these cortical columns is reported. We explore the role of astrocytes in the arrangement of these cortical columns using a self-organizing computational model. Invoking evidence that astrocytes could influence bidirectional plasticity among effective lateral excitatory connections in V1, we introduce a mechanism by which astrocytic inputs can influence map formation in the neuronal network. In the resulting model-generated OPMs the radius of the hypercolumns is found to be correlated with that of astrocytic arbors, a feature that finds support in experimental studies.

Topographic Independent Component Analysis reveals random scrambling of orientation in visual space

Neurons at primary visual cortex (V1) in humans and other species are edge filters organized in orientation maps. In these maps, neurons with similar orientation preference are clustered together in iso-orientation domains. These maps have two fundamental properties: (1) retinotopy, i.e. correspondence between displacements at the image space and displacements at the cortical surface, and (2) a trade-off between good coverage of the visual field with all orientations and continuity of iso-orientation domains in the cortical space. There is an active debate on the origin of these locally continuous maps. While most of the existing descriptions take purely geometric/mechanistic approaches which disregard the network function, a clear exception to this trend in the literature is the original approach of Hyvärinen and Hoyer based on infomax and Topographic Independent Component Analysis (TICA). Although TICA successfully addresses a number of other properties of V1 simple and complex cells, in this work we question the validity of the orientation maps obtained from TICA. We argue that the maps predicted by TICA can be analyzed in the retinal space, and when doing so, it is apparent that they lack the required continuity and retinotopy. Here we show that in the orientation maps reported in the TICA literature it is easy to find examples of violation of the continuity between similarly tuned mechanisms in the retinal space, which suggest a random scrambling incompatible with the maps in primates. The new experiments in the retinal space presented here confirm this guess: TICA basis vectors actually follow a random salt-and-pepper organization back in the image space. Therefore, the interesting clusters found in the TICA topology cannot be interpreted as the actual cortical orientation maps found in cats, primates or humans. In conclusion, Topographic ICA does not reproduce cortical orientation maps.

Familiarization: A theory of repetition suppression predicts interference between overlapping cortical representations

Repetition suppression refers to a reduction in the cortical response to a novel stimulus that results from repeated presentation of the stimulus. We demonstrate repetition suppression in a well established computational model of cortical plasticity, according to which the relative strengths of lateral inhibitory interactions are modified by Hebbian learning. We present the model as an extension to the traditional account of repetition suppression offered by sharpening theory, which emphasises the contribution of afferent plasticity, by instead attributing the effect primarily to plasticity of intra-cortical circuitry. In support, repetition suppression is shown to emerge in simulations with plasticity enabled only in intra-cortical connections. We show in simulation how an extended 'inhibitory sharpening theory' can explain the disruption of repetition suppression reported in studies that include an intermediate phase of exposure to additional novel stimuli composed of features similar to those of the original stimulus. The model suggests a re-interpretation of repetition suppression as a manifestation of the process by which an initially distributed representation of a novel object becomes a more localist representation. Thus, inhibitory sharpening may constitute a more general process by which representation emerges from cortical re-organisation.

Extending the Stabilized Supralinear Network model for binocular image processing

The visual cortex is both extensive and intricate. Computational models are needed to clarify the relationships between its local mechanisms and high-level functions. The Stabilized Supralinear Network (SSN) model was recently shown to account for many receptive field phenomena in V1, and also to predict subtle receptive field properties that were subsequently confirmed in vivo. In this study, we performed a preliminary exploration of whether the SSN is suitable for incorporation into large, functional models of the visual cortex, considering both its extensibility and computational tractability. First, whereas the SSN receives abstract orientation signals as input, we extended it to receive images (through a linear-nonlinear stage), and found that the extended version behaved similarly. Secondly, whereas the SSN had previously been studied in a monocular context, we found that it could also reproduce data on interocular transfer of surround suppression. Finally, we reformulated the SSN as a convolutional neural network, and found that it scaled well on parallel hardware. These results provide additional support for the plausibility of the SSN as a model of lateral interactions in V1, and suggest that the SSN is well suited as a component of complex vision models. Future work will use the SSN to explore relationships between local network interactions and sophisticated vision processes in large networks.

A Global Orientation Map in the Primary Visual Cortex (V1): Could a Self Organizing Model Reveal Its Hidden Bias?

A remarkable accomplishment of self organizing models is their ability to simulate the development of feature maps in the cortex. Additionally, these models have been trained to tease out the differential causes of multiple feature maps, mapped on to the same output space. Recently, a Laterally Interconnected Synergetically Self Organizing Map (LISSOM) model has been used to simulate the mapping of eccentricity and meridional angle onto orthogonal axes in the primary visual cortex (V1). This model is further probed to simulate the development of the radial bias in V1, using a training set that consists of both radial (rectangular bars of random size and orientation) as well as non-radial stimuli. The radial bias describes the preference of the visual system toward orientations that match the angular position (meridional angle) of that orientation with respect to the point of fixation. Recent fMRI results have shown that there exists a coarse scale orientation map in V1, which resembles the meridional angle map, thereby providing a plausible neural basis for the radial bias. The LISSOM model, trained for the development of the retinotopic map, on probing for orientation preference, exhibits a coarse scale orientation map, consistent with these experimental results, quantified using the circular cross correlation (r_c ). The r_c between the orientation map developed on probing with a thin annular ring containing sinusoidal gratings with a spatial frequency of 0.5 cycles per degree (cpd) and the corresponding meridional map for the same annular ring, has a value of 0.8894. The results also suggest that the radial bias goes beyond the current understanding of a node to node correlation between the two maps.

Cortical Maps

In this article, we review functional organization in sensory cortical regions—how the cortex represents the world. We consider four interrelated aspects of cortical organization: (1) the set of receptive fields of individual cortical sensory neurons, (2) how lateral interaction between cortical neurons reflects the similarity of their receptive fields, (3) the spatial distribution of receptive-field properties across the horizontal extent of the cortical tissue, and (4) how the spatial distributions of different receptive-field properties interact with one another. We show how these data are generally well explained by the theory of input-driven self-organization, with a family of computational models of cortical maps offering a parsimonious account for a wide range of map-related phenomena. We then discuss important challenges to this explanation, with respect to the maps present at birth, maps present under activity blockade, the limits of adult plasticity, and the lack of some maps in rodents. Because there is not at present another credible general theory for cortical map development, we conclude by proposing key experiments to help uncover other mechanisms that might also be operating during map development.

Edge co-occurrences can account for rapid categorization of natural versus animal images

Making a judgment about the semantic category of a visual scene, such as whether it contains an animal, is typically assumed to involve high-level associative brain areas. Previous explanations require progressively analyzing the scene hierarchically at increasing levels of abstraction, from edge extraction to mid-level object recognition and then object categorization. Here we show that the statistics of edge co-occurrences alone are sufficient to perform a rough yet robust (translation, scale, and rotation invariant) scene categorization. We first extracted the edges from images using a scale-space analysis coupled with a sparse coding algorithm. We then computed the “association field” for different categories (natural, man-made, or containing an animal) by computing the statistics of edge co-occurrences. These differed strongly, with animal images having more curved configurations. We show that this geometry alone is sufficient for categorization, and that the pattern of errors made by humans is consistent with this procedure. Because these statistics could be measured as early as the primary visual cortex, the results challenge widely held assumptions about the flow of computations in the visual system. The results also suggest new algorithms for image classification and signal processing that exploit correlations between low-level structure and the underlying semantic category.

What, if anything, are topological maps for?

What, if anything, is the functional significance of spatial patterning in cortical feature maps? We ask this question of four major theories of cortical map formation: self-organizing maps, wiring optimization, place coding, and reaction-diffusion. We argue that (i) self-organizing maps yield spatial patterning only as a by-product of efficient mechanisms for developing environmentally appropriate distributions of feature preferences, (ii) wiring optimization assumes rather than explains a map-like organization, (iii) place-coding mechanisms can at best explain only a subset of maps in functional terms, and (iv) reaction-diffusion models suggest two factors in the evolution of maps, the first based on efficient development of feature distributions, and the second based on generating feature-specific long-range recurrent cortical circuitry. None of these explanations for the existence of topological maps requires spatial patterning in maps to be useful. Thus despite these useful frameworks for understanding how maps form and how they are wired, the possibility that patterns are merely epiphenomena in the evolution of mammalian neocortex cannot be rejected. The article is intended as a nontechnical introduction to the assumptions and predictions of these four important classes of models, along with other possible functional explanations for maps.

Developmental self-construction and -configuration of functional neocortical neuronal networks

The prenatal development of neural circuits must provide sufficient configuration to support at least a set of core postnatal behaviors. Although knowledge of various genetic and cellular aspects of development is accumulating rapidly, there is less systematic understanding of how these various processes play together in order to construct such functional networks. Here we make some steps toward such understanding by demonstrating through detailed simulations how a competitive co-operative (‘winner-take-all’, WTA) network architecture can arise by development from a single precursor cell. This precursor is granted a simplified gene regulatory network that directs cell mitosis, differentiation, migration, neurite outgrowth and synaptogenesis. Once initial axonal connection patterns are established, their synaptic weights undergo homeostatic unsupervised learning that is shaped by wave-like input patterns. We demonstrate how this autonomous genetically directed developmental sequence can give rise to self-calibrated WTA networks, and compare our simulation results with biological data.

Models of learning in artificial neural networks generally assume that the neurons and approximate network are given, and then learning tunes the synaptic weights. By contrast, we address the question of how an entire functional neuronal network containing many differentiated neurons and connections can develop from only a single progenitor cell. We chose a winner-take-all network as the developmental target, because it is a computationally powerful circuit, and a candidate motif of neocortical networks. The key aspect of this challenge is that the developmental mechanisms must be locally autonomous as in Biology: They cannot depend on global knowledge or supervision. We have explored this developmental process by simulating in physical detail the fundamental biological behaviors, such as cell proliferation, neurite growth and synapse formation that give rise to the structural connectivity observed in the superficial layers of the neocortex. These differentiated, approximately connected neurons then adapt their synaptic weights homeostatically to obtain a uniform electrical signaling activity before going on to organize themselves according to the fundamental correlations embedded in a noisy wave-like input signal. In this way the precursor expands itself through development and unsupervised learning into winner-take-all functionality and orientation selectivity in a biologically plausible manner.

Parametric Anatomical Modeling: a method for modeling the anatomical layout of neurons and their projections

Computational models of neural networks can be based on a variety of different parameters. These parameters include, for example, the 3d shape of neuron layers, the neurons' spatial projection patterns, spiking dynamics and neurotransmitter systems. While many well-developed approaches are available to model, for example, the spiking dynamics, there is a lack of approaches for modeling the anatomical layout of neurons and their projections. We present a new method, called Parametric Anatomical Modeling (PAM), to fill this gap. PAM can be used to derive network connectivities and conduction delays from anatomical data, such as the position and shape of the neuronal layers and the dendritic and axonal projection patterns. Within the PAM framework, several mapping techniques between layers can account for a large variety of connection properties between pre- and post-synaptic neuron layers. PAM is implemented as a Python tool and integrated in the 3d modeling software Blender. We demonstrate on a 3d model of the hippocampal formation how PAM can help reveal complex properties of the synaptic connectivity and conduction delays, properties that might be relevant to uncover the function of the hippocampus. Based on these analyses, two experimentally testable predictions arose: (i) the number of neurons and the spread of connections is heterogeneously distributed across the main anatomical axes, (ii) the distribution of connection lengths in CA3-CA1 differ qualitatively from those between DG-CA3 and CA3-CA3. Models created by PAM can also serve as an educational tool to visualize the 3d connectivity of brain regions. The low-dimensional, but yet biologically plausible, parameter space renders PAM suitable to analyse allometric and evolutionary factors in networks and to model the complexity of real networks with comparatively little effort.

A proto-architecture for innate directionally selective visual maps

Self-organizing artificial neural networks are a popular tool for studying visual system development, in particular the cortical feature maps present in real systems that represent properties such as ocular dominance (OD), orientation-selectivity (OR) and direction selectivity (DS). They are also potentially useful in artificial systems, for example robotics, where the ability to extract and learn features from the environment in an unsupervised way is important. In this computational study we explore a DS map that is already latent in a simple artificial network. This latent selectivity arises purely from the cortical architecture without any explicit coding for DS and prior to any self-organising process facilitated by spontaneous activity or training. We find DS maps with local patchy regions that exhibit features similar to maps derived experimentally and from previous modeling studies. We explore the consequences of changes to the afferent and lateral connectivity to establish the key features of this proto-architecture that support DS.

A three-layer model of natural image statistics

An important property of visual systems is to be simultaneously both selective to specific patterns found in the sensory input and invariant to possible variations. Selectivity and invariance (tolerance) are opposing requirements. It has been suggested that they could be joined by iterating a sequence of elementary selectivity and tolerance computations. It is, however, unknown what should be selected or tolerated at each level of the hierarchy. We approach this issue by learning the computations from natural images. We propose and estimate a probabilistic model of natural images that consists of three processing layers. Two natural image data sets are considered: image patches, and complete visual scenes downsampled to the size of small patches. For both data sets, we find that in the first two layers, simple and complex cell-like computations are performed. In the third layer, we mainly find selectivity to longer contours; for patch data, we further find some selectivity to texture, while for the downsampled complete scenes, some selectivity to curvature is observed.