Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex

"Extended Descriptive Risk-Averse Bayesian Model" a More Comprehensive Approach in Simulating Complex Biological Motion Perception

The ability to perceive biological motion is crucial for human survival, social interactions, and communication. Over the years, researchers have studied the mechanisms and neurobiological substrates that enable this ability. In a previous study, we proposed a descriptive Bayesian simulation model to represent the dorsal pathway of the visual system, which processes motion information. The model was inspired by recent studies that questioned the impact of dynamic form cues in biological motion perception and was trained to distinguish the direction of a soccer ball from a set of complex biological motion soccer-kick stimuli. However, the model was unable to simulate the reaction times of the athletes in a credible manner, and a few subjects could not be simulated. In this current work, we implemented a novel disremembering strategy to incorporate neural adaptation at the decision-making level, which improved the model's ability to simulate the athletes' reaction times. We also introduced receptive fields to detect rotational optic flow patterns not considered in the previous model to simulate a new subject and improve the correlation between the simulation and experimental data. The findings suggest that rotational optic flow plays a critical role in the decision-making process and sheds light on how different individuals perform at different levels. The correlation analysis of human versus simulation data shows a significant, almost perfect correlation between experimental and simulated angular thresholds and slopes, respectively. The analysis also reveals a strong relation between the average reaction times of the athletes and the simulations.

An interactive cortical architecture for perceptual organization by accentuation

Accentuation has been proposed as a general principle of perceptual organization. Here, we have developed a neurodynamic architecture to explain how accentuation affects boundary segmentation and shape perception. The model consists of bottom-up and top-down pathways. Bottom-up processing involves a set of feature maps that compute bottom-up salience of surfaces, boundaries, boundary completions, and junctions. Then, a feature-based winner-take-all network selects the most salient locations. Top-down processing includes an object-based attention stage that allows enhanced neural activity to propagate from the most salient locations to all connected locations, and a visual segmentation stage that employs inhibitory connections to segregate boundaries into distinct maps. The model was tested on a series of computer simulations showing how the position of the accent affects boundary segregation in the square-diamond and the pointing illusion. The model was also tested on a variety of texture segregation tasks, showing that its performance was comparable to that of human observers. The model suggests that there is an intermediate stage of visual processing between perceptual grouping and object recognition that helps the visual system choose between competing percepts of the ambiguous stimulus.

"Descriptive Risk-Averse Bayesian Decision-Making," a Model for Complex Biological Motion Perception in the Human Dorsal Pathway

Biological motion perception is integral not only to survival but also to the social life of human beings. Identifying the underlying mechanisms and their associated neurobiological substrates has been a matter of investigation and debate for some time. Although, in general, it is believed that the integration of local motion and dynamic form cues in the brain empowers the visual system to perceive/recognize biological motion stimuli, some recent studies have indicated the importance of dynamic form cues in such a process. Inspired by the previous neurophysiologically plausible biological motion perception models, a new descriptive risk-averse Bayesian simulation model, capable of discerning a ball's direction from a set of complex biological motion soccer kick stimuli, is proposed. The model represents only the dorsal pathway as a motion information processing section of the visual system according to the two-stream theory. The stimuli used have been obtained from a previous psychophysical study on athletes in our lab. Furthermore, the acquired psychophysical data from that study have been used to re-enact human behavior using our simulation model. By adjusting the model parameters, the psychometric function of athlete subjects has been mimicked. A correlation analysis between human and simulation data shows a significant and robust correlation between angular thresholds and slopes of the psychometric functions of both groups. Although it is established that the visual system optimally integrates all available information in the decision-making process, the results conform to the speculations favoring motion cue importance over dynamic form by testing the limits in which biological motion perception only depends on motion information processing.

A simple model of the attentional blink and its modulation by mental training

The attentional blink (AB) effect is the reduced probability of reporting a second target (T2) that appears shortly after a first one (T1) within a rapidly presented sequence of distractors. The AB effect has been shown to be reduced following intensive mental training in the form of mindfulness meditation, with a corresponding reduction in T1-evoked P3b brain potentials. However, the mechanisms underlying these effects remain unknown. We propose a dynamical-systems model of the AB, in which attentional load is described as the response of a dynamical system to incoming impulse signals. Non-task related mental activity is represented by additive noise modulated by meditation. The model provides a parsimonious computational framework relating behavioral performance, evoked brain potentials and training through the concept of reduced mental noise.

Bio-Inspired Presentation Attack Detection for Face Biometrics

Today, face biometric systems are becoming widely accepted as a standard method for identity authentication in many security settings. For example, their deployment in automated border control gates plays a crucial role in accurate document authentication and reduced traveler flow rates in congested border zones. The proliferation of such systems is further spurred by the advent of portable devices. On the one hand, modern smartphone and tablet cameras have in-built user authentication applications while on the other hand, their displays are being consistently exploited for face spoofing. Similar to biometric systems of other physiological biometric identifiers, face biometric systems have their own unique set of potential vulnerabilities. In this work, these vulnerabilities (presentation attacks) are being explored via a biologically-inspired presentation attack detection model which is termed "BIOPAD." Our model employs Gabor features in a feedforward hierarchical structure of layers that progressively process and train from visual information of people's faces, along with their presentation attacks, in the visible and near-infrared spectral regions. BIOPAD's performance is directly compared with other popular biologically-inspired layered models such as the "Hierarchical Model And X" (HMAX) that applies similar handcrafted features, and Convolutional Neural Networks (CNN) that discover low-level features through stochastic descent training. BIOPAD shows superior performance to both HMAX and CNN in all of the three presentation attack databases examined and these results were consistent in two different classifiers (Support Vector Machine and k-nearest neighbor). In certain cases, our findings have shown that BIOPAD can produce authentication rates with 99% accuracy. Finally, we further introduce a new presentation attack database with visible and near-infrared information for direct comparisons. Overall, BIOPAD's operation, which is to fuse information from different spectral bands at both feature and score levels for the purpose of face presentation attack detection, has never been attempted before with a biologically-inspired algorithm. Obtained detection rates are promising and confirm that near-infrared visual information significantly assists in overcoming presentation attacks.

A Normalization Circuit Underlying Coding of Spatial Attention in Primate Lateral Prefrontal Cortex

Lateral prefrontal cortex (LPFC) neurons signal the allocation of voluntary attention; however, the neural computations underlying this function remain unknown. To investigate this, we recorded from neuronal ensembles in the LPFC of two Macaca fascicularis performing a visuospatial attention task. LPFC neural responses to a single stimulus were normalized when additional stimuli/distracters appeared across the visual field and were well-characterized by an averaging computation. Deploying attention toward an individual stimulus surrounded by distracters shifted neural activity from an averaging regime toward a regime similar to that when the attended stimulus was presented in isolation (winner-take-all; WTA). However, attentional modulation is both qualitatively and quantitatively dependent on a neuron's visuospatial tuning. Our results show that during attentive vision, LPFC neuronal ensemble activity can be robustly read out by downstream areas to generate motor commands, and/or fed back into sensory areas to filter out distracter signals in favor of target signals.

Unconscious Biases in Neural Populations Coding Multiple Stimuli

Throughout the nervous system, information is commonly coded in activity distributed over populations of neurons. In idealized situations where a single, continuous stimulus is encoded in a homogeneous population code, the value of the encoded stimulus can be read out without bias. However, in many situations, multiple stimuli are simultaneously present; for example, multiple motion patterns might overlap. Here we find that when multiple stimuli that overlap in their neural representation are simultaneously encoded in the population, biases in the read-out emerge. Although the bias disappears in the absence of noise, the bias is remarkably persistent at low noise levels. The bias can be reduced by competitive encoding schemes or by employing complex decoders. To study the origin of the bias, we develop a novel general framework based on gaussian processes that allows an accurate calculation of the estimate distributions of maximum likelihood decoders, and reveals that the distribution of estimates is bimodal for overlapping stimuli. The results have implications for neural coding and behavioral experiments on, for instance, overlapping motion patterns.

Colour Constancy Beyond the Classical Receptive Field

The problem of removing illuminant variations to preserve the colours of objects (colour constancy) has already been solved by the human brain using mechanisms that rely largely on centre-surround computations of local contrast. In this paper we adopt some of these biological solutions described by long known physiological findings into a simple, fully automatic, functional model (termed Adaptive Surround Modulation or ASM). In ASM, the size of a visual neuron's receptive field (RF) as well as the relationship with its surround varies according to the local contrast within the stimulus, which in turn determines the nature of the centre-surround normalisation of cortical neurons higher up in the processing chain. We modelled colour constancy by means of two overlapping asymmetric Gaussian kernels whose sizes are adapted based on the contrast of the surround pixels, resembling the change of RF size. We simulated the contrast-dependent surround modulation by weighting the contribution of each Gaussian according to the centre-surround contrast. In the end, we obtained an estimation of the illuminant from the set of the most activated RFs' outputs. Our results on three single-illuminant and one multi-illuminant benchmark datasets show that ASM is highly competitive against the state-of-the-art and it even outperforms learning-based algorithms in one case. Moreover, the robustness of our model is more tangible if we consider that our results were obtained using the same parameters for all datasets, that is, mimicking how the human visual system operates. These results suggest a dynamical adaptation mechanisms contribute to achieving higher accuracy in computational colour constancy.

A Neurodynamic Model of Feature-Based Spatial Selection

Huang and Pashler (2007) suggested that feature-based attention creates a special form of spatial representation, which is termed a Boolean map. It partitions the visual scene into two distinct and complementary regions: selected and not selected. Here, we developed a model of a recurrent competitive network that is capable of state-dependent computation. It selects multiple winning locations based on a joint top-down cue. We augmented a model of the WTA circuit that is based on linear-threshold units with two computational elements: dendritic non-linearity that acts on the excitatory units and activity-dependent modulation of synaptic transmission between excitatory and inhibitory units. Computer simulations showed that the proposed model could create a Boolean map in response to a featured cue and elaborate it using the logical operations of intersection and union. In addition, it was shown that in the absence of top-down guidance, the model is sensitive to bottom-up cues such as saliency and abrupt visual onset.

Hue opponency: chromatic valence functions, individual differences, cortical winner-take-all opponent modeling, and the relationship between spikes and sensitivity

Neural spike rate data are more restricted in range than related psychophysical data. For example, several studies suggest a compressive (roughly cube root) nonlinear relationship between wavelength-opponent spike rates in primate midbrain and color appearance in humans, two rather widely separated domains. This presents an opportunity to partially bridge a chasm between these two domains and to probe the putative nonlinearity with other psychophysical data. Here neural wavelength-opponent data are used to create cortical competition models for hue opponency. This effort led to creation of useful models of spiking neuron winner-take-all (WTA) competition and MAX selection. When fed with actual primate data, the spiking WTA models generate reasonable wavelength-opponent spike rate behaviors. An average psychophysical observer for red-green and blue-yellow opponency is curated from eight applicable studies in the refereed and dissertation literatures, with cancellation data roughly every 10 nm in 18 subjects for yellow-blue opponency and 15 subjects for red-green opponency. A direct mapping between spiking neurons with broadband wavelength sensitivity and human psychophysical luminance yields a power law exponent of 0.27, similar to the cube root nonlinearity. Similarly, direct mapping between the WTA model opponent spike rates and psychophysical opponent data suggests power law relationships with exponents between 0.24 and 0.41.

Mechanisms of Orientation Selectivity in the Primary Visual Cortex

The mechanisms underlying the emergence of orientation selectivity in the visual cortex have been, and continue to be, the subjects of intense scrutiny. Orientation selectivity reflects a dramatic change in the representation of the visual world: Whereas afferent thalamic neurons are generally orientation insensitive, neurons in the primary visual cortex (V1) are extremely sensitive to stimulus orientation. This profound change in the receptive field structure along the visual pathway has positioned V1 as a model system for studying the circuitry that underlies neural computations across the neocortex. The neocortex is characterized anatomically by the relative uniformity of its circuitry despite its role in processing distinct signals from region to region. A combination of physiological, anatomical, and theoretical studies has shed some light on the circuitry components necessary for generating orientation selectivity in V1. This targeted effort has led to critical insights, as well as controversies, concerning how neural circuits in the neocortex perform computations.

Comparative Aspects of Hearing in Vertebrates and Insects with Antennal Ears

The evolution of hearing in terrestrial animals has resulted in remarkable adaptations enabling exquisitely sensitive sound detection by the ear and sophisticated sound analysis by the brain. In this review, we examine several such characteristics, using examples from insects and vertebrates. We focus on two strong and interdependent forces that have been shaping the auditory systems across taxa: the physical environment of auditory transducers on the small, subcellular scale, and the sensory-ecological environment within which hearing happens, on a larger, evolutionary scale. We briefly discuss acoustical feature selectivity and invariance in the central auditory system, highlighting a major difference between insects and vertebrates as well as a major similarity. Through such comparisons within a sensory ecological framework, we aim to emphasize general principles underlying acute sensitivity to airborne sounds.

Local Order within Global Disorder: Synaptic Architecture of Visual Space

Substantial evidence at the subcellular level indicates that the spatial arrangement of synaptic inputs onto dendrites could play a significant role in cortical computations, but how synapses of functionally defined cortical networks are arranged within the dendrites of individual neurons remains unclear. Here we assessed one-dimensional spatial receptive fields of individual dendritic spines within individual layer 2/3 neuron dendrites. Spatial receptive field properties of dendritic spines were strikingly diverse, with no evidence of large-scale topographic organization. At a fine scale, organization was evident: neighboring spines separated by less than 10 μm shared similar spatial receptive field properties and exhibited a distance-dependent correlation in sensory-driven and spontaneous activity patterns. Fine-scale dendritic organization was supported by the fact that functional groups of spines defined by dimensionality reduction of receptive field properties exhibited non-random dendritic clustering. Our results demonstrate that functional synaptic clustering is a robust feature existing at a local spatial scale. VIDEO ABSTRACT.

Representation of tactile scenes in the rodent barrel cortex

After half a century of research, the sensory features coded by neurons of the rodent barrel cortex remain poorly understood. Still, views of the sensory representation of whisker information are increasingly shifting from a labeled line representation of single-whisker deflections to a selectivity for specific elements of the complex statistics of the multi-whisker deflection patterns that take place during spontaneous rodent behavior - so called natural tactile scenes. Here we review the current knowledge regarding the coding of patterns of whisker stimuli by barrel cortex neurons, from responses to single-whisker deflections to the representation of complex tactile scenes. A number of multi-whisker tunings have already been identified, including center-surround feature extraction, angular tuning during edge-like multi-whisker deflections, and even tuning to specific statistical properties of the tactile scene such as the level of correlation across whiskers. However, a more general model of the representation of multi-whisker information in the barrel cortex is still missing. This is in part because of the lack of a human intuition regarding the perception emerging from a whisker system, but also because in contrast to other primary sensory cortices such as the visual cortex, the spatial feature selectivity of barrel cortex neurons rests on highly nonlinear interactions that remained hidden to classical receptive field approaches.

Effects of generalized pooling on binocular disparity selectivity of neurons in the early visual cortex

The key problem of stereoscopic vision is traditionally defined as accurately finding the positional shifts of corresponding object features between left and right images. Here, we demonstrate that the problem must be considered in a four-dimensional parameter space; with respect not only to shifts in space (X, Y), but also spatial frequency (SF) and orientation (OR). The proposed model sums outputs of binocular energy units linearly over the multi-dimensional V1 parameter space (X, Y, SF, OR). Theoretical analyses and physiological experiments show that many binocular neurons achieve sharp binocular tuning properties by pooling the output of multiple neurons with relatively broad tuning. Pooling in the space domain sharpens disparity-selective responses in the SF domain so that the responses to combinations of unmatched left-right SFs are attenuated. Conversely, pooling in the SF domain sharpens disparity selectivity in the space domain, reducing the possibility of false matches. Analogous effects are observed for the OR domain in that the spatial pooling sharpens the binocular tuning in the OR domain. Such neurons become selective to relative OR disparity. Therefore, pooling allows the visual system to refine binocular information into a form more desirable for stereopsis.This article is part of the themed issue 'Vision in our three-dimensional world'.

Contrast and lustre: A model that accounts for eleven different forms of contrast discrimination in binocular vision

Our goal here is a more complete understanding of how information about luminance contrast is encoded and used by the binocular visual system. In two-interval forced-choice experiments we assessed observers' ability to discriminate changes in contrast that could be an increase or decrease of contrast in one or both eyes, or an increase in one eye coupled with a decrease in the other (termed IncDec). The base or pedestal contrasts were either in-phase or out-of-phase in the two eyes. The opposed changes in the IncDec condition did not cancel each other out, implying that along with binocular summation, information is also available from mechanisms that do not sum the two eyes' inputs. These might be monocular mechanisms. With a binocular pedestal, monocular increments of contrast were much easier to see than monocular decrements. These findings suggest that there are separate binocular (B) and monocular (L,R) channels, but only the largest of the three responses, max(L,B,R), is available to perception and decision. Results from contrast discrimination and contrast matching tasks were described very accurately by this model. Stimuli, data, and model responses can all be visualized in a common binocular contrast space, allowing a more direct comparison between models and data. Some results with out-of-phase pedestals were not accounted for by the max model of contrast coding, but were well explained by an extended model in which gratings of opposite polarity create the sensation of lustre. Observers can discriminate changes in lustre alongside changes in contrast.

DehazeNet: An End-to-End System for Single Image Haze Removal

Single image haze removal is a challenging ill-posed problem. Existing methods use various constraints/priors to get plausible dehazing solutions. The key to achieve haze removal is to estimate a medium transmission map for an input hazy image. In this paper, we propose a trainable end-to-end system called DehazeNet, for medium transmission estimation. DehazeNet takes a hazy image as input, and outputs its medium transmission map that is subsequently used to recover a haze-free image via atmospheric scattering model. DehazeNet adopts convolutional neural network-based deep architecture, whose layers are specially designed to embody the established assumptions/priors in image dehazing. Specifically, the layers of Maxout units are used for feature extraction, which can generate almost all haze-relevant features. We also propose a novel nonlinear activation function in DehazeNet, called bilateral rectified linear unit, which is able to improve the quality of recovered haze-free image. We establish connections between the components of the proposed DehazeNet and those used in existing methods. Experiments on benchmark images show that DehazeNet achieves superior performance over existing methods, yet keeps efficient and easy to use.

Hierarchical winner-take-all particle swarm optimization social network for neural model fitting

Particle swarm optimization (PSO) has gained widespread use as a general mathematical programming paradigm and seen use in a wide variety of optimization and machine learning problems. In this work, we introduce a new variant on the PSO social network and apply this method to the inverse problem of input parameter selection from recorded auditory neuron tuning curves. The topology of a PSO social network is a major contributor to optimization success. Here we propose a new social network which draws influence from winner-take-all coding found in visual cortical neurons. We show that the winner-take-all network performs exceptionally well on optimization problems with greater than 5 dimensions and runs at a lower iteration count as compared to other PSO topologies. Finally we show that this variant of PSO is able to recreate auditory frequency tuning curves and modulation transfer functions, making it a potentially useful tool for computational neuroscience models.

Distributed and Dynamic Neural Encoding of Multiple Motion Directions of Transparently Moving Stimuli in Cortical Area MT

Segmenting visual scenes into distinct objects and surfaces is a fundamental visual function. To better understand the underlying neural mechanism, we investigated how neurons in the middle temporal cortex (MT) of macaque monkeys represent overlapping random-dot stimuli moving transparently in slightly different directions. It has been shown that the neuronal response elicited by two stimuli approximately follows the average of the responses elicited by the constituent stimulus components presented alone. In this scheme of response pooling, the ability to segment two simultaneously presented motion directions is limited by the width of the tuning curve to motion in a single direction. We found that, although the population-averaged neuronal tuning showed response averaging, subgroups of neurons showed distinct patterns of response tuning and were capable of representing component directions that were separated by a small angle--less than the tuning width to unidirectional stimuli. One group of neurons preferentially represented the component direction at a specific side of the bidirectional stimuli, weighting one stimulus component more strongly than the other. Another group of neurons pooled the component responses nonlinearly and showed two separate peaks in their tuning curves even when the average of the component responses was unimodal. We also show for the first time that the direction tuning of MT neurons evolved from initially representing the vector-averaged direction of slightly different stimuli to gradually representing the component directions. Our results reveal important neural processes underlying image segmentation and suggest that information about slightly different stimulus components is computed dynamically and distributed across neurons.

SIGNIFICANCE STATEMENT

Natural scenes often contain multiple entities. The ability to segment visual scenes into distinct objects and surfaces is fundamental to sensory processing and is crucial for generating the perception of our environment. Because cortical neurons are broadly tuned to a given visual feature, segmenting two stimuli that differ only slightly is a challenge for the visual system. In this study, we discovered that many neurons in the visual cortex are capable of representing individual components of slightly different stimuli by selectively and nonlinearly pooling the responses elicited by the stimulus components. We also show for the first time that the neural representation of individual stimulus components developed over a period of ∼70-100 ms, revealing a dynamic process of image segmentation.

BIT: Biologically Inspired Tracker

Visual tracking is challenging due to image variations caused by various factors, such as object deformation, scale change, illumination change, and occlusion. Given the superior tracking performance of human visual system (HVS), an ideal design of biologically inspired model is expected to improve computer visual tracking. This is, however, a difficult task due to the incomplete understanding of neurons' working mechanism in the HVS. This paper aims to address this challenge based on the analysis of visual cognitive mechanism of the ventral stream in the visual cortex, which simulates shallow neurons (S1 units and C1 units) to extract low-level biologically inspired features for the target appearance and imitates an advanced learning mechanism (S2 units and C2 units) to combine generative and discriminative models for target location. In addition, fast Gabor approximation and fast Fourier transform are adopted for real-time learning and detection in this framework. Extensive experiments on large-scale benchmark data sets show that the proposed biologically inspired tracker performs favorably against the state-of-the-art methods in terms of efficiency, accuracy, and robustness. The acceleration technique in particular ensures that biologically inspired tracker maintains a speed of approximately 45 frames/s.

One hundred ways to process time, frequency, rate and scale in the central auditory system: a pattern-recognition meta-analysis

The mammalian auditory system extracts features from the acoustic environment based on the responses of spatially distributed sets of neurons in the subcortical and cortical auditory structures. The characteristic responses of these neurons (linearly approximated by their spectro-temporal receptive fields, or STRFs) suggest that auditory representations are formed, as early as in the inferior colliculi, on the basis of a time, frequency, rate (temporal modulations) and scale (spectral modulations) analysis of sound. However, how these four dimensions are integrated and processed in subsequent neural networks remains unclear. In this work, we present a new methodology to generate computational insights into the functional organization of such processes. We first propose a systematic framework to explore more than a hundred different computational strategies proposed in the literature to process the output of a generic STRF model. We then evaluate these strategies on their ability to compute perceptual distances between pairs of environmental sounds. Finally, we conduct a meta-analysis of the dataset of all these algorithms' accuracies to examine whether certain combinations of dimensions and certain ways to treat such dimensions are, on the whole, more computationally effective than others. We present an application of this methodology to a dataset of ten environmental sound categories, in which the analysis reveals that (1) models are most effective when they organize STRF data into frequency groupings—which is consistent with the known tonotopic organization of receptive fields in auditory structures -, and that (2) models that treat STRF data as time series are no more effective than models that rely only on summary statistics along time—which corroborates recent experimental evidence on texture discrimination by summary statistics.

The responses of V1 cortical neurons to flashed presentations of orthogonal single lines and edges

How cortical neurons process multiple inputs is a fundamental issue in modern neuroscience. Neurons in visual cortical area V1 have been shown to exhibit cross-orientation suppression, where the response to an optimally oriented visual stimulus is reduced by the simultaneous presence of an orthogonally oriented stimulus. This is consistent with the view that cortical neurons respond to multiple inputs with a weighted average (or normalization) of the responses to the inputs presented separately. However, most of these studies have used drifting or counterphase-modulated grating stimuli, potentially confounding orientation effects with non-orientation-specific gain control mechanisms. Additionally, primate vision depends to a great extent on transient stimulus presentations during fixations between saccades. Therefore this study examined the responses of primate V1 neurons to orthogonal flashed-onset single edges and lines, and to their combinations. Single edges or lines do not typically cause strong suppression of the responses to an orthogonal stimulus, even though a grating does. This appears to hold true regardless of the relative contrasts of the orthogonal single lines or edges. This is consistent with response suppression from an orthogonal grating being due to non-orientation-specific contrast gain control (Koeling M, Shapley R, Shelley M. J Comp Neurosci 25: 390-400, 2008; Priebe NJ, Ferster D. Nat Neurosci 9: 552-561, 2006; Walker GA, Ohzawa I, Freeman RD. J.Neurophysiol 79: 227-239, 1998). While normalization mechanisms are clearly important for the cerebral cortex, under many conditions the responses of V1 cortical neurons to an optimally oriented stimulus can be unaffected by the presence of orthogonal stimuli, which may be important to avoid confounding the interpretation of a neural response.

Novel neural circuit mechanism for visual edge detection

The primary visual cortex is organized in a way that assigns a specific collection of neurons the job of providing the rest of the brain with all of the information it needs about each small part of the image present on the retina: Neighboring patches of the visual cortex provide the information about neighboring patches of the visual world. Each one of these cortical patches--often identified as a "pinwheel"--contains thousands of neurons, and its corresponding image patch is centered on a particular location in the retina. For stimuli within their image patch, neurons respond selectively to lines or edges with a particular slope (orientation tuning) and to regions of the patch of different sizes (known as spatial frequency tuning). The same number of neurons is devoted to reporting each possible slope (orientation). For the cells that cover different-sized regions of their image patch, however, the number of neurons assigned depends strongly on their preferred region size. Only a few neurons report on large and small parts of the image patch, but many neurons report visual information from medium-sized areas. I show here that having different numbers of neurons responsible for image regions of different sizes actually carries out a computation: Edges in the image patch are extracted. I also explain how this edge-detection computation is done.

Development of biological movement recognition by interaction between active basis model and fuzzy optical flow division

Following the study on computational neuroscience through functional magnetic resonance imaging claimed that human action recognition in the brain of mammalian pursues two separated streams, that is, dorsal and ventral streams. It follows up by two pathways in the bioinspired model, which are specialized for motion and form information analysis (Giese and Poggio 2003). Active basis model is used to form information which is different from orientations and scales of Gabor wavelets to form a dictionary regarding object recognition (human). Also biologically movement optic-flow patterns utilized. As motion information guides share sketch algorithm in form pathway for adjustment plus it helps to prevent wrong recognition. A synergetic neural network is utilized to generate prototype templates, representing general characteristic form of every class. Having predefined templates, classifying performs based on multitemplate matching. As every human action has one action prototype, there are some overlapping and consistency among these templates. Using fuzzy optical flow division scoring can prevent motivation for misrecognition. We successfully apply proposed model on the human action video obtained from KTH human action database. Proposed approach follows the interaction between dorsal and ventral processing streams in the original model of the biological movement recognition. The attained results indicate promising outcome and improvement in robustness using proposed approach.

Central auditory neurons display flexible feature recombination functions

Recognition of natural stimuli requires a combination of selectivity and invariance. Classical neurobiological models achieve selectivity and invariance, respectively, by assigning to each cortical neuron either a computation equivalent to the logical "AND" or a computation equivalent to the logical "OR." One powerful OR-like operation is the MAX function, which computes the maximum over input activities. The MAX function is frequently employed in computer vision to achieve invariance and considered a key operation in visual cortex. Here we explore the computations for selectivity and invariance in the auditory system of a songbird, using natural stimuli. We ask two related questions: does the MAX operation exist in auditory system? Is it implemented by specialized "MAX" neurons, as assumed in vision? By analyzing responses of individual neurons to combinations of stimuli we systematically sample the space of implemented feature recombination functions. Although we frequently observe the MAX function, we show that the same neurons that implement it also readily implement other operations, including the AND-like response. We then show that sensory adaptation, a ubiquitous property of neural circuits, causes transitions between these operations in individual neurons, violating the fixed neuron-to-computation mapping posited in the state-of-the-art object-recognition models. These transitions, however, accord with predictions of neural-circuit models incorporating divisive normalization and variable polynomial nonlinearities at the spike threshold. Because these biophysical properties are not tied to a particular sensory modality but are generic, the flexible neuron-to-computation mapping demonstrated in this study in the auditory system is likely a general property.

Rapid feedforward computation by temporal encoding and learning with spiking neurons

Primates perform remarkably well in cognitive tasks such as pattern recognition. Motivated by recent findings in biological systems, a unified and consistent feedforward system network with a proper encoding scheme and supervised temporal rules is built for solving the pattern recognition task. The temporal rules used for processing precise spiking patterns have recently emerged as ways of emulating the brain's computation from its anatomy and physiology. Most of these rules could be used for recognizing different spatiotemporal patterns. However, there arises the question of whether these temporal rules could be used to recognize real-world stimuli such as images. Furthermore, how the information is represented in the brain still remains unclear. To tackle these problems, a proper encoding method and a unified computational model with consistent and efficient learning rule are proposed. Through encoding, external stimuli are converted into sparse representations, which also have properties of invariance. These temporal patterns are then learned through biologically derived algorithms in the learning layer, followed by the final decision presented through the readout layer. The performance of the model with images of digits from the MNIST database is presented. The results show that the proposed model is capable of recognizing images correctly with a performance comparable to that of current benchmark algorithms. The results also suggest a plausibility proof for a class of feedforward models of rapid and robust recognition in the brain.

Human Object-Similarity Judgments Reflect and Transcend the Primate-IT Object Representation

Primate inferior temporal (IT) cortex is thought to contain a high-level representation of objects at the interface between vision and semantics. This suggests that the perceived similarity of real-world objects might be predicted from the IT representation. Here we show that objects that elicit similar activity patterns in human IT (hIT) tend to be judged as similar by humans. The IT representation explained the human judgments better than early visual cortex, other ventral-stream regions, and a range of computational models. Human similarity judgments exhibited category clusters that reflected several categorical divisions that are prevalent in the IT representation of both human and monkey, including the animate/inanimate and the face/body division. Human judgments also reflected the within-category representation of IT. However, the judgments transcended the IT representation in that they introduced additional categorical divisions. In particular, human judgments emphasized human-related additional divisions between human and non-human animals and between man-made and natural objects. hIT was more similar to monkey IT than to human judgments. One interpretation is that IT has evolved visual-feature detectors that distinguish between animates and inanimates and between faces and bodies because these divisions are fundamental to survival and reproduction for all primate species, and that other brain systems serve to more flexibly introduce species-dependent and evolutionarily more recent divisions.

Tangent bundle curve completion with locally connected parallel networks

We propose a theory for cortical representation and computation of visually completed curves that are generated by the visual system to fill in missing visual information (e.g., due to occlusions). Recent computational theories and physiological evidence suggest that although such curves do not correspond to explicit image evidence along their length, their construction emerges from corresponding activation patterns of orientation-selective cells in the primary visual cortex. Previous theoretical work modeled these patterns as least energetic 3D curves in the mathematical continuous space R2 × S1, which abstracts the mammalian striate cortex. Here we discuss the biological plausibility of this theory and present a neural architecture that implements it with locally connected parallel networks. Part of this contribution is also a first attempt to bridge the physiological literature on curve completion with the shape problem and a shape theory. We present completion simulations of our model in natural and synthetic scenes and discuss various observations and predictions that emerge from this theory in the context of curve completion.

Real-time knowledge-based processing of images: application of the online NLPM method to perceptual visual analysis

Perceptual analysis is an interesting topic in the field of image processing, and can be considered a missing link between image processing and human vision. Of the various forms of perception, one of the most important and best known is shape perception. In this work, a framework based on the online non local patch means (NLPM) method is developed, which is designed to infer possible perceptual observations of an input image using the knowledge images provided. Thanks to the speed of online NLPM, the proposed method can simulate the transformation of the input image to the final perceptual image in real time. In order to improve the performance of the method, a hidden chain series is considered for the model that delivers faster convergence. The capability of the method is evaluated on several well-known perceptual examples.

Visual object categorization in birds and primates: integrating behavioral, neurobiological, and computational evidence within a "general process" framework

Previous comparative work has suggested that the mechanisms of object categorization differ importantly for birds and primates. However, behavioral and neurobiological differences do not preclude the possibility that at least some of those mechanisms are shared across these evolutionarily distant groups. The present study integrates behavioral, neurobiological, and computational evidence concerning the "general processes" that are involved in object recognition in vertebrates. We start by reviewing work implicating error-driven learning in object categorization by birds and primates, and also consider neurobiological evidence suggesting that the basal ganglia might implement this process. We then turn to work with a computational model showing that principles of visual processing discovered in the primate brain can account for key behavioral findings in object recognition by pigeons, including cases in which pigeons' behavior differs from that of people. These results provide a proof of concept that the basic principles of visual shape processing are similar across distantly related vertebrate species, thereby offering important insights into the evolution of visual cognition.

Phoneme and word recognition in the auditory ventral stream

Spoken word recognition requires complex, invariant representations. Using a meta-analytic approach incorporating more than 100 functional imaging experiments, we show that preference for complex sounds emerges in the human auditory ventral stream in a hierarchical fashion, consistent with nonhuman primate electrophysiology. Examining speech sounds, we show that activation associated with the processing of short-timescale patterns (i.e., phonemes) is consistently localized to left mid-superior temporal gyrus (STG), whereas activation associated with the integration of phonemes into temporally complex patterns (i.e., words) is consistently localized to left anterior STG. Further, we show left mid- to anterior STG is reliably implicated in the invariant representation of phonetic forms and that this area also responds preferentially to phonetic sounds, above artificial control sounds or environmental sounds. Together, this shows increasing encoding specificity and invariance along the auditory ventral stream for temporally complex speech sounds.

Short-time scale dynamics in the responses to multiple stimuli in visual cortex

Many previous studies have used the presentation of multiple stimuli in the receptive fields (RFs) of visual cortical neurons to explore how neurons might operate on multiple inputs. Most of these experiments have used two fixed stimulus locations within the RF of each neuron. Here the effects of using different positions within the RF of a neuron were explored. The stimuli were presented singly at one of six locations, and also at 15 pair-wise combinations, for 24 V2 cortical neurons in two macaque monkeys. There was considerable variability in how pairs of stimuli interacted within the receptive field of any given neuron: changing the position of the stimuli could result in enhancement, winner-take-all, or suppression relative to the strongest response to a stimulus presented by itself. Across the population of neurons there was no correlation between response strength and response latency. However, for many stimulus pairs the response latency was tightly locked to the shortest response latency of any single stimulus presented by itself independent of changes in response magnitude. In other words, a stimulus that by itself elicited a relatively long latency response, would often affect the magnitude of the response to a pair of stimuli, but not change the latency. These results may provide constraints on the development of models of cortical information processing.

A differential model of the complex cell

The receptive fields of simple cells in the visual cortex can be understood as linear filters. These filters can be modeled by Gabor functions or gaussian derivatives. Gabor functions can also be combined in an energy model of the complex cell response. This letter proposes an alternative model of the complex cell, based on gaussian derivatives. It is most important to account for the insensitivity of the complex response to small shifts of the image. The new model uses a linear combination of the first few derivative filters, at a single position, to approximate the first derivative filter, at a series of adjacent positions. The maximum response, over all positions, gives a signal that is insensitive to small shifts of the image. This model, unlike previous approaches, is based on the scale space theory of visual processing. In particular, the complex cell is built from filters that respond to the 2D differential structure of the image. The computational aspects of the new model are studied in one and two dimensions, using the steerability of the gaussian derivatives. The response of the model to basic images, such as edges and gratings, is derived formally. The response to natural images is also evaluated, using statistical measures of shift insensitivity. The neural implementation and predictions of the model are discussed.

A single functional model accounts for the distinct properties of suppression in cortical area V1

Cross-orientation suppression and surround suppression have been extensively studied in primary visual cortex (V1). These two forms of suppression have some distinct properties which has led to the suggestion that they are generated by different underlying mechanisms. Furthermore, it has been suggested that mechanisms other than intracortical inhibition may be central to both forms of suppression. A simple computational model (PC/BC), in which intracortical inhibition is fundamental, is shown to simulate the distinct properties of cross-orientation and surround suppression. The same model has previously been shown to account for a large range of V1 response properties including orientation-tuning, spatial and temporal frequency tuning, facilitation and inhibition by flankers and textured surrounds as well as a range of other experimental results on cross-orientation suppression and surround suppression. The current results thus provide additional support for the PC/BC model of V1 and for the proposal that the diverse range of response properties observed in V1 neurons have a single computational explanation. Furthermore, these results demonstrate that current neurophysiological evidence is insufficient to discount intracortical inhibition as a central mechanism underlying both forms of suppression.

Spatial summation in macaque parietal area 7a follows a winner-take-all rule

While neurons in posterior parietal cortex have been found to signal the presence of a salient stimulus among multiple items in a display, spatial summation within their receptive field in the absence of an attentional bias has never been investigated. This information, however, is indispensable when one investigates the mechanisms of spatial attention and competition between multiple visual objects. To examine the spatial summation rule in parietal area 7a neurons, we trained rhesus monkeys to fixate on a central cross while two identical stimuli were briefly displayed in a neuron's receptive field. The response to a pair of dots was compared with the responses to the same dots when they were presented individually. The scaling and power parameters of a generalized summation algorithm varied greatly, both across neurons and across combinations of stimulus locations. However, the averaged response of the recorded population of 7a neurons was consistent with a winner-take-all rule for spatial summation. A control experiment where a monkey covertly attended to both stimuli simultaneously suggests that attention introduces additional competition by facilitating the less optimal stimulus. Thus an averaging stage is introduced between ∼ 200 and 300 ms of the response to a pair of stimuli. In short, the summation algorithm over the population of area 7a neurons carries the signature of a winner-take-all operation, with spatial attention possibly influencing the temporal dynamics of stimulus competition, that is the moment that the "winner" takes "victory" over the "loser" stimulus.

The role of V1 surround suppression in MT motion integration

Neurons in the primate extrastriate cortex are highly selective for complex stimulus features such as faces, objects, and motion patterns. One explanation for this selectivity is that neurons in these areas carry out sophisticated computations on the outputs of lower-level areas such as primary visual cortex (V1), where neuronal selectivity is often modeled in terms of linear spatiotemporal filters. However, it has long been known that such simple V1 models are incomplete because they fail to capture important nonlinearities that can substantially alter neuronal selectivity for specific stimulus features. Thus a key step in understanding the function of higher cortical areas is the development of realistic models of their V1 inputs. We have addressed this issue by constructing a computational model of the V1 neurons that provide the strongest input to extrastriate cortical middle temporal (MT) area. We find that a modest elaboration to the standard model of V1 direction selectivity generates model neurons with strong end-stopping, a property that is also found in the V1 layers that provide input to MT. With this computational feature in place, the seemingly complex properties of MT neurons can be simulated by assuming that they perform a simple nonlinear summation of their inputs. The resulting model, which has a very small number of free parameters, can simulate many of the diverse properties of MT neurons. In particular, we simulate the invariance of MT tuning curves to the orientation and length of tilted bar stimuli, as well as the accompanying temporal dynamics. We also show how this property relates to the continuum from component to pattern selectivity observed when MT neurons are tested with plaids. Finally, we confirm several key predictions of the model by recording from MT neurons in the alert macaque monkey. Overall our results demonstrate that many of the seemingly complex computations carried out by high-level cortical neurons can in principle be understood by examining the properties of their inputs.

Cortical circuitry implementing graphical models

In this letter, we develop and simulate a large-scale network of spiking neurons that approximates the inference computations performed by graphical models. Unlike previous related schemes, which used sum and product operations in either the log or linear domains, the current model uses an inference scheme based on the sum and maximization operations in the log domain. Simulations show that using these operations, a large-scale circuit, which combines populations of spiking neurons as basic building blocks, is capable of finding close approximations to the full mathematical computations performed by graphical models within a few hundred milliseconds. The circuit is general in the sense that it can be wired for any graph structure, it supports multistate variables, and it uses standard leaky integrate-and-fire neuronal units. Following previous work, which proposed relations between graphical models and the large-scale cortical anatomy, we focus on the cortical microcircuitry and propose how anatomical and physiological aspects of the local circuitry may map onto elements of the graphical model implementation. We discuss in particular the roles of three major types of inhibitory neurons (small fast-spiking basket cells, large layer 2/3 basket cells, and double-bouquet neurons), subpopulations of strongly interconnected neurons with their unique connectivity patterns in different cortical layers, and the possible role of minicolumns in the realization of the population-based maximum operation.

Spatial and temporal features of synaptic to discharge receptive field transformation in cat area 17

The aim of the present study was to characterize the spatial and temporal features of synaptic and discharge receptive fields (RFs), and to quantify their relationships, in cat area 17. For this purpose, neurons were recorded intracellularly while high-frequency flashing bars were used to generate RFs maps for synaptic and spiking responses. Comparison of the maps shows that some features of the discharge RFs depended strongly on those of the synaptic RFs, whereas others were less dependent. Spiking RF duration depended poorly and spiking RF amplitude depended moderately on those of the underlying synaptic RFs. At the other extreme, the optimal spatial frequency and phase of the discharge RFs in simple cells were almost entirely inherited from those of the synaptic RFs. Subfield width, in both simple and complex cells, was less for spiking responses compared with synaptic responses, but synaptic to discharge width ratio was relatively variable from cell to cell. When considering the whole RF of simple cells, additional variability in width ratio resulted from the presence of additional synaptic subfields that remained subthreshold. Due to these additional, subthreshold subfields, spatial frequency tuning predicted from synaptic RFs appears sharper than that predicted from spiking RFs. Excitatory subfield overlap in spiking RFs was well predicted by subfield overlap at the synaptic level. When examined in different regions of the RF, latencies appeared to be quite variable, but this variability showed negligible dependence on distance from the RF center. Nevertheless, spiking response latency faithfully reflected synaptic response latency.

Task effects, performance levels, features, configurations, and holistic face processing: a reply to Rossion

A recent article in Acta Psychologica ("Picture-plane inversion leads to qualitative changes of face perception" by Rossion [Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica (Amst), 128(2), 274-289]) criticized several aspects of an earlier paper of ours [Riesenhuber, M., Jarudi, I., Gilad, S., & Sinha, P. (2004). Face processing in humans is compatible with a simple shape-based model of vision. Proceedings of the Royal Society of London B (Supplements), 271, S448-S450]. We here address Rossion's criticisms and correct some misunderstandings. To frame the discussion, we first review our previously presented computational model of face recognition in cortex [Jiang, X., Rosen, E., Zeffiro, T., Vanmeter, J., Blanz, V., & Riesenhuber, M. (2006). Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques. Neuron, 50(1), 159-172] that provides a concrete biologically plausible computational substrate for holistic coding, namely a neural representation learned for upright faces, in the spirit of the original simple-to-complex hierarchical model of vision by Hubel and Wiesel. We show that Rossion's and others' data support the model, and that there is actually a convergence of views on the mechanisms underlying face recognition, in particular regarding holistic processing.

Representational similarity analysis - connecting the branches of systems neuroscience

A fundamental challenge for systems neuroscience is to quantitatively relate its three major branches of research: brain-activity measurement, behavioral measurement, and computational modeling. Using measured brain-activity patterns to evaluate computational network models is complicated by the need to define the correspondency between the units of the model and the channels of the brain-activity data, e.g., single-cell recordings or voxels from functional magnetic resonance imaging (fMRI). Similar correspondency problems complicate relating activity patterns between different modalities of brain-activity measurement (e.g., fMRI and invasive or scalp electrophysiology), and between subjects and species. In order to bridge these divides, we suggest abstracting from the activity patterns themselves and computing representational dissimilarity matrices (RDMs), which characterize the information carried by a given representation in a brain or model. Building on a rich psychological and mathematical literature on similarity analysis, we propose a new experimental and data-analytical framework called representational similarity analysis (RSA), in which multi-channel measures of neural activity are quantitatively related to each other and to computational theory and behavior by comparing RDMs. We demonstrate RSA by relating representations of visual objects as measured with fMRI in early visual cortex and the fusiform face area to computational models spanning a wide range of complexities. The RDMs are simultaneously related via second-level application of multidimensional scaling and tested using randomization and bootstrap techniques. We discuss the broad potential of RSA, including novel approaches to experimental design, and argue that these ideas, which have deep roots in psychology and neuroscience, will allow the integrated quantitative analysis of data from all three branches, thus contributing to a more unified systems neuroscience.

Curvature-based perceptual singularities and texture saliency with early vision mechanisms

Recent work has shown that salient perceptual singularities occur in visual textures even in the absence of feature gradients. In smoothly varying orientation-defined textures, these striking non-smooth percepts can be predicted from two texture curvatures, one tangential and one normal [Proc. Natl. Acad. Sci. USA103, 15704 (2006)]. We address the issue of detecting these perceptual singularities in a biologically plausible manner and present three different models to compute the tangential and normal curvatures using early cortical mechanisms. The first model relies on the response summation of similarly scaled even-symmetric simple cells at different positions by utilizing intercolumnar interactions in the primary visual cortex (V1). The second model is based on intracolumnar interactions in a two-layer mechanism of simple cells having the same orientation tuning but significantly different scales. Our third model uses a three-layer circuit in which both even-symmetric and odd-symmetric receptive fields (RFs) are used to compute all possible directional derivatives of the dominant orientation, from which the tangential and normal curvatures at each spatial position are selected using nonlinear shunting inhibition. We show experimental results of all three models, we outline an extension to oriented textures with multiple dominant orientations at each point, and we discuss how our results may be relevant to the processing of general textures.