The influence of stereopsis on visual saliency in a proto-object based model of selective attention

Some animals including humans use stereoscopic vision which reconstructs spatial information about the environment from the disparity between images captured by eyes in two separate adjacent locations. Like other sensory information, such stereoscopic information is expected to influence attentional selection. We develop a biologically plausible model of binocular vision to study its effect on bottom-up visual attention, i.e., visual saliency. In our model, the scene is organized in terms of proto-objects on which attention acts, rather than on unbound sets of elementary features. We show that taking into account the stereoscopic information improves the performance of the model in the prediction of human eye movements with statistically significant differences.

The cellular architecture of memory modules in Drosophila supports stochastic input integration

The ability to associate neutral stimuli with valence information and to store these associations as memories forms the basis for decision making. To determine the underlying computational principles, we build a realistic computational model of a central decision module within the Drosophila mushroom body (MB), the fly's center for learning and memory. Our model combines the electron microscopy-based architecture of one MB output neuron (MBON-α3), the synaptic connectivity of its 948 presynaptic Kenyon cells (KCs), and its membrane properties obtained from patch-clamp recordings. We show that this neuron is electrotonically compact and that synaptic input corresponding to simulated odor input robustly drives its spiking behavior. Therefore, sparse innervation by KCs can efficiently control and modulate MBON activity in response to learning with minimal requirements on the specificity of synaptic localization. This architecture allows efficient storage of large numbers of memories using the flexible stochastic connectivity of the circuit.

Quantifying Phase-Amplitude Modulation in Neural Data

Phase-amplitude modulation (the modulation of the amplitude of higher frequency oscillations by the phase of lower frequency oscillations) is a specific type of cross-frequency coupling that has been observed in neural recordings from multiple species in a range of behavioral contexts. Given its potential importance, care must be taken with how it is measured and quantified. Previous studies have quantified phase-amplitude modulation by measuring the distance of the amplitude distribution from a uniform distribution. While this method is of general applicability, it is not targeted to the specific modulation pattern frequently observed with low-frequency oscillations. Here we develop a new method that has increased specificity to detect modulation in the sinusoidal shape commonly observed in neural data.

Decomposing Executive Function into Distinct Processes Underlying Human Decision Making

Executive function (EF) consists of higher level cognitive processes including working memory, cognitive flexibility, and inhibition which together enable goal-directed behaviors. Many neurological disorders are associated with EF dysfunctions which can lead to suboptimal behavior. To assess the roles of these processes, we introduce a novel behavioral task and modeling approach. The gamble-like task, with sub-tasks targeting different EF capabilities, allows for quantitative assessment of the main components of EF. We demonstrate that human participants exhibit dissociable variability in the component processes of EF. These results will allow us to map behavioral outcomes to EEG recordings in future work in order to map brain networks associated with EF deficits. Clinical relevance- This work will allow us to quantify EF deficits and corresponding brain activity in patient populations in future work.

Event-driven proto-object based saliency in 3D space to attract a robot's attention

To interact with its environment, a robot working in 3D space needs to organise its visual input in terms of objects or their perceptual precursors, proto-objects. Among other visual cues, depth is a submodality used to direct attention to visual features and objects. Current depth-based proto-object attention models have been implemented for standard RGB-D cameras that produce synchronous frames. In contrast, event cameras are neuromorphic sensors that loosely mimic the function of the human retina by asynchronously encoding per-pixel brightness changes at very high temporal resolution, thereby providing advantages like high dynamic range, efficiency (thanks to their high degree of signal compression), and low latency. We propose a bio-inspired bottom-up attention model that exploits event-driven sensing to generate depth-based saliency maps that allow a robot to interact with complex visual input. We use event-cameras mounted in the eyes of the iCub humanoid robot to directly extract edge, disparity and motion information. Real-world experiments demonstrate that our system robustly selects salient objects near the robot in the presence of clutter and dynamic scene changes, for the benefit of downstream applications like object segmentation, tracking and robot interaction with external objects.

A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model With a Hybrid FPGA Implementation

Computing and attending to salient regions of a visual scene is an innate and necessary preprocessing step for both biological and engineered systems performing high-level visual tasks including object detection, tracking, and classification. Computational bandwidth and speed are improved by preferentially devoting computational resources to salient regions of the visual field. The human brain computes saliency effortlessly, but modeling this task in engineered systems is challenging. We first present a neuromorphic dynamic saliency model, which is bottom-up, feed-forward, and based on the notion of proto-objects with neurophysiological spatio-temporal features requiring no training. Our neuromorphic model outperforms state-of-the-art dynamic visual saliency models in predicting human eye fixations (i.e., ground truth saliency). Secondly, we present a hybrid FPGA implementation of the model for real-time applications, capable of processing 112×84 resolution frames at 18.71 Hz running at a 100 MHz clock rate - a 23.77× speedup from the software implementation. Additionally, our fixed-point model of the FPGA implementation yields comparable results to the software implementation.

Naturalistic Spike Trains Drive State-Dependent Homeostatic Plasticity in Superficial Layers of Visual Cortex

The history of neural activity determines the synaptic plasticity mechanisms employed in the brain. Previous studies report a rapid reduction in the strength of excitatory synapses onto layer 2/3 (L2/3) pyramidal neurons of the primary visual cortex (V1) following two days of dark exposure and subsequent re-exposure to light. The abrupt increase in visually driven activity is predicted to drive homeostatic plasticity, however, the parameters of neural activity that trigger these changes are unknown. To determine this, we first recorded spike trains in vivo from V1 layer 4 (L4) of dark exposed (DE) mice of both sexes that were re-exposed to light through homogeneous or patterned visual stimulation. We found that delivering the spike patterns recorded in vivo to L4 of V1 slices was sufficient to reduce the amplitude of miniature excitatory postsynaptic currents (mEPSCs) of V1 L2/3 neurons in DE mice, but not in slices obtained from normal reared (NR) controls. Unexpectedly, the same stimulation pattern produced an up-regulation of mEPSC amplitudes in V1 L2/3 neurons from mice that received 2 h of light re-exposure (LE). A Poisson spike train exhibiting the same average frequency as the patterns recorded in vivo was equally effective at depressing mEPSC amplitudes in L2/3 neurons in V1 slices prepared from DE mice. Collectively, our results suggest that the history of visual experience modifies the responses of V1 neurons to stimulation and that rapid homeostatic depression of excitatory synapses can be driven by non-patterned input activity.

Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention

The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention.

Vision allows us to make sense out of a very complex signal, the patterns of light rays reaching our eyes. Two mechanisms are essential for this: perceptual organization which structures the input into meaningful visual objects, and attention which selects only the most important parts in the input. Prior work suggests that both of these mechanisms are implemented by neurons called grouping cells. These organize the object features into coherent entities (perceptual grouping) and access them as needed (selective attention). For technical reasons it is difficult to observe grouping cells but their effect can be seen in the influence they have on responses of other classes of cells. These responses have been measured experimentally and it was found that they depend in unexpected ways on where the subject was attending. Using a computational model, we here demonstrate that the responses can be understood in terms of the interaction between two kinds of selective attention, both of which are known to occur in primate perception. One is attention to a specific area in the environment, the other is to specific objects. A model including both of these attentional mechanisms generates neuronal responses in agreement with the observed patterns of neural activity.

Proto-Object Based Saliency Model With Texture Detection Channel

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

The formation of preference in risky choice

A key question in decision-making is how people integrate amounts and probabilities to form preferences between risky alternatives. Here we rely on the general principle of integration-to-boundary to develop several biologically plausible process models of risky-choice, which account for both choices and response-times. These models allowed us to contrast two influential competing theories: i) within-alternative evaluations, based on multiplicative interaction between amounts and probabilities, ii) within-attribute comparisons across alternatives. To constrain the preference formation process, we monitored eye-fixations during decisions between pairs of simple lotteries, designed to systematically span the decision-space. The behavioral results indicate that the participants' eye-scanning patterns were associated with risk-preferences and expected-value maximization. Crucially, model comparisons showed that within-alternative process models decisively outperformed within-attribute ones, in accounting for choices and response-times. These findings elucidate the psychological processes underlying preference formation when making risky-choices, and suggest that compensatory, within-alternative integration is an adaptive mechanism employed in human decision-making.

Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2

A crucial step in understanding visual input is its organization into meaningful components, in particular object contours and partially occluded background structures. This requires that all contours are assigned to either the foreground or the background (border ownership assignment). While earlier studies showed that neurons in primate extrastriate cortex signal border ownership for simple geometric shapes, recent studies show consistent border ownership coding also for complex natural scenes. In order to understand how the brain performs this task, we developed a biologically plausible recurrent neural network that is fully image computable. Our model uses local edge detector ( B ) cells and grouping ( G ) cells whose activity represents proto-objects based on the integration of local feature information. G cells send modulatory feedback connections to those B cells that caused their activation, making the B cells border ownership selective. We found close agreement between our model and neurophysiological results in terms of the timing of border ownership signals (BOSs) as well as the consistency of BOSs across scenes. We also benchmarked our model on the Berkeley Segmentation Dataset and achieved performance comparable to recent state-of-the-art computer vision approaches. Our proposed model provides insight into the cortical mechanisms of figure-ground organization.

Unique objects attract attention even when faint

Locally contrasting objects, e.g. a red apple surrounded by green apples, attract attention. Does this generalize to differences in feature space? That is, do unique objects-regardless of their location-stand out from a collection of objects that are similar to one another, even when the unique object has lower local contrast with the background than the other objects? Behavioral data show indeed a preference for unique items but previous experiments enabled viewers to anticipate what response they were "supposed" to give. We developed a new experimental paradigm that minimizes such top-down effects. Pitting local contrast against global uniqueness, we show that unique stimuli attract attention even in not-anticipated, never-seen images, and even when the unique stimuli are faint (low contrast). A computational model explains how competition between objects in feature space favors dissimilar objects over those with similar features. The model explains how humans select unique objects, without a loss of performance on natural scenes.

Analytically determining frequency and amplitude of spontaneous alpha oscillation in Jansen's neural mass model using the describing function method

Spontaneous alpha oscillations are a ubiquitous phenomenon in the brain and play a key role in neural information processing and various cognitive functions. Jansen's neural mass model (NMM) was initially proposed to study the origin of alpha oscillations. Most of previous studies of the spontaneous alpha oscillations in the NMM were conducted using numerical methods. In this study, we aim to propose an analytical approach using the describing function method to elucidate the spontaneous alpha oscillation mechanism in the NMM. First, the sigmoid nonlinear function in the NMM is approximated by its describing function, allowing us to reformulate the NMM and derive its standard form composed of one nonlinear part and one linear part. Second, by conducting a theoretical analysis, we can assess whether or not the spontaneous alpha oscillation would occur in the NMM and, furthermore, accurately determine its amplitude and frequency. The results reveal analytically that the interaction between linearity and nonlinearity of the NMM plays a key role in generating the spontaneous alpha oscillations. Furthermore, strong nonlinearity and large linear strength are required to generate the spontaneous alpha oscillations.

Short-term depression and transient memory in sensory cortex

Persistent neuronal activity is usually studied in the context of short-term memory localized in central cortical areas. Recent studies show that early sensory areas also can have persistent representations of stimuli which emerge quickly (over tens of milliseconds) and decay slowly (over seconds). Traditional positive feedback models cannot explain sensory persistence for at least two reasons: (i) They show attractor dynamics, with transient perturbations resulting in a quasi-permanent change of system state, whereas sensory systems return to the original state after a transient. (ii) As we show, those positive feedback models which decay to baseline lose their persistence when their recurrent connections are subject to short-term depression, a common property of excitatory connections in early sensory areas. Dual time constant network behavior has also been implemented by nonlinear afferents producing a large transient input followed by much smaller steady state input. We show that such networks require unphysiologically large onset transients to produce the rise and decay observed in sensory areas. Our study explores how memory and persistence can be implemented in another model class, derivative feedback networks. We show that these networks can operate with two vastly different time courses, changing their state quickly when new information is coming in but retaining it for a long time, and that these capabilities are robust to short-term depression. Specifically, derivative feedback networks with short-term depression that acts differentially on positive and negative feedback projections are capable of dynamically changing their time constant, thus allowing fast onset and slow decay of responses without requiring unrealistically large input transients.

A recurrent neural model for proto-object based contour integration and figure-ground segregation

Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects ("proto-objects") based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594-6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492-1499 2007; Chen et al. Neuron, 82(3), 682-694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.

Attentive pointing in natural scenes correlates with other measures of attention

Finger pointing is a natural human behavior frequently used to draw attention to specific parts of sensory input. Since this pointing behavior is likely preceded and/or accompanied by the deployment of attention by the pointing person, we hypothesize that pointing can be used as a natural means of providing self-reports of attention and, in the case of visual input, visual salience. We here introduce a new method for assessing attentional choice by asking subjects to point to and tap the first place they look at on an image appearing on an electronic tablet screen. Our findings show that the tap data are well-correlated with other measures of attention, including eye fixations and selections of interesting image points, as well as with predictions of a saliency map model. We also develop an analysis method for comparing attentional maps (including fixations, reported points of interest, finger pointing, and computed salience) that takes into account the error in estimating those maps from a finite number of data points. This analysis strengthens our original findings by showing that the measured correlation between attentional maps drawn from identical underlying processes is systematically underestimated. The underestimation is strongest when the number of samples is small but it is always present. Our analysis method is not limited to data from attentional paradigms but, instead, it is broadly applicable to measures of similarity made between counts of multinomial data or probability distributions.

The Edge of Stability: Response Times and Delta Oscillations in Balanced Networks

The standard architecture of neocortex is a network with excitation and inhibition in closely maintained balance. These networks respond fast and with high precision to their inputs and they allow selective amplification of patterned signals. The stability of such networks is known to depend on balancing the strengths of positive and negative feedback. We here show that a second condition is required for stability which depends on the relative strengths and time courses of fast (AMPA) and slow (NMDA) currents in the excitatory projections. This condition also determines the response time of the network. We show that networks which respond quickly to an input are necessarily close to an oscillatory instability which resonates in the delta range. This instability explains the existence of neocortical delta oscillations and the emergence of absence epilepsy. Although cortical delta oscillations are a network-level phenomenon, we show that in non-pathological networks, individual neurons receive sufficient information to keep the network in the fast-response regime without sliding into the instability.

Many networks in the brain are finely balanced, with equal contributions from excitation and inhibition. Deviations from this balance, if for instance the total amount of excitation exceeds that of inhibition, lead to potentially devastating instabilities. Unlike previous work we consider the interaction between fast and slow excitatory connections. We show that not only the amount of excitation needs to be controlled to achieve network stability but also the ratio of slow to fast excitation. Furthermore, optimally fast network performance requires that networks approach instability. However, networks very close to this instability develop oscillations in the delta range (1–4Hz) which potentially cause absence epilepsy. We show that a normal (non-pathological) network can auto-regulate its activity to avoid the instability.

Spike synchrony generated by modulatory common input through NMDA-type synapses

Common excitatory input to neurons increases their firing rates and the strength of the spike correlation (synchrony) between them. Little is known, however, about the synchronizing effects of modulatory common input. Here, we show that modulatory common input with the slow synaptic kinetics of N-methyl-d-aspartate (NMDA) receptors enhances firing rates and also produces synchrony. Tight synchrony (correlations on the order of milliseconds) always increases with modulatory strength. Unexpectedly, the relationship between strength of modulation and strength of loose synchrony (tens of milliseconds) is not monotonic: The strongest loose synchrony is obtained for intermediate modulatory amplitudes. This finding explains recent neurophysiological results showing that in cortical areas V1 and V2, presumed modulatory top-down input due to contour grouping increases (loose and tight) synchrony but that additional modulatory input due to top-down attention does not change tight synchrony and actually decreases loose synchrony. These neurophysiological findings are understood from our model of integrate-and-fire neurons under the assumption that contour grouping as well as attention lead to additive modulatory common input through NMDA-type synapses. In contrast, circuits with common projections through model α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptors did not exhibit the paradoxical decrease of synchrony with increased input. Our results suggest that NMDA receptors play a critical role in top-down response modulation in the visual cortex.

Neural mechanisms of selective attention in the somatosensory system

Selective attention allows organisms to extract behaviorally relevant information while ignoring distracting stimuli that compete for the limited resources of their central nervous systems. Attention is highly flexible, and it can be harnessed to select information based on sensory modality, within-modality feature(s), spatial location, object identity, and/or temporal properties. In this review, we discuss the body of work devoted to understanding mechanisms of selective attention in the somatosensory system. In particular, we describe the effects of attention on tactile behavior and corresponding neural activity in somatosensory cortex. Our focus is on neural mechanisms that select tactile stimuli based on their location on the body (somatotopic-based attention) or their sensory feature (feature-based attention). We highlight parallels between selection mechanisms in touch and other sensory systems and discuss several putative neural coding schemes employed by cortical populations to signal the behavioral relevance of sensory inputs. Specifically, we contrast the advantages and disadvantages of using a gain vs. spike-spike correlation code for representing attended sensory stimuli. We favor a neural network model of tactile attention that is composed of frontal, parietal, and subcortical areas that controls somatosensory cells encoding the relevant stimulus features to enable preferential processing throughout the somatosensory hierarchy. Our review is based on data from noninvasive electrophysiological and imaging data in humans as well as single-unit recordings in nonhuman primates.

Suppressing epileptic activity in a neural mass model using a closed-loop proportional-integral controller

Closed-loop control is a promising deep brain stimulation (DBS) strategy that could be used to suppress high-amplitude epileptic activity. However, there are currently no analytical approaches to determine the stimulation parameters for effective and safe treatment protocols. Proportional-integral (PI) control is the most extensively used closed-loop control scheme in the field of control engineering because of its simple implementation and perfect performance. In this study, we took Jansen's neural mass model (NMM) as a test bed to develop a PI-type closed-loop controller for suppressing epileptic activity. A graphical stability analysis method was employed to determine the stabilizing region of the PI controller in the control parameter space, which provided a theoretical guideline for the choice of the PI control parameters. Furthermore, we established the relationship between the parameters of the PI controller and the parameters of the NMM in the form of a stabilizing region, which provided insights into the mechanisms that may suppress epileptic activity in the NMM. The simulation results demonstrated the validity and effectiveness of the proposed closed-loop PI control scheme.

A proto-object based saliency model in three-dimensional space

Most models of visual saliency operate on two-dimensional images, using elementary image features such as intensity, color, or orientation. The human visual system, however, needs to function in complex three-dimensional environments, where depth information is often available and may be used to guide the bottom-up attentional selection process. In this report we extend a model of proto-object based saliency to include depth information and evaluate its performance on three separate three-dimensional eye tracking datasets. Our results show that the additional depth information provides a small, but statistically significant, improvement in the model's ability to predict perceptual saliency (eye fixations) in natural scenes. The computational mechanisms of our model have direct neural correlates, and our results provide further evidence that proto-objects help to establish perceptual organization of the scene.

Temporal correlation mechanisms and their role in feature selection: a single-unit study in primate somatosensory cortex

How neurons pay attention Top-down selective attention mediates feature selection by reducing the noise correlations in neural populations and enhancing the synchronized activity across subpopulations that encode the relevant features of sensory stimuli.

Studies in vision show that attention enhances the firing rates of cells when it is directed towards their preferred stimulus feature. However, it is unknown whether other sensory systems employ this mechanism to mediate feature selection within their modalities. Moreover, whether feature-based attention modulates the correlated activity of a population is unclear. Indeed, temporal correlation codes such as spike-synchrony and spike-count correlations (r_sc) are believed to play a role in stimulus selection by increasing the signal and reducing the noise in a population, respectively. Here, we investigate (1) whether feature-based attention biases the correlated activity between neurons when attention is directed towards their common preferred feature, (2) the interplay between spike-synchrony and r_sc during feature selection, and (3) whether feature attention effects are common across the visual and tactile systems. Single-unit recordings were made in secondary somatosensory cortex of three non-human primates while animals engaged in tactile feature (orientation and frequency) and visual discrimination tasks. We found that both firing rate and spike-synchrony between neurons with similar feature selectivity were enhanced when attention was directed towards their preferred feature. However, attention effects on spike-synchrony were twice as large as those on firing rate, and had a tighter relationship with behavioral performance. Further, we observed increased r_sc when attention was directed towards the visual modality (i.e., away from touch). These data suggest that similar feature selection mechanisms are employed in vision and touch, and that temporal correlation codes such as spike-synchrony play a role in mediating feature selection. We posit that feature-based selection operates by implementing multiple mechanisms that reduce the overall noise levels in the neural population and synchronize activity across subpopulations that encode the relevant features of sensory stimuli.

Attention can select stimuli in space based on the stimulus features most relevant for a task. Attention effects have been linked to several important phenomena such as modulations in neuronal spiking rate (i.e., the average number of spikes per unit time) and spike-spike synchrony between neurons. Attention has also been associated with spike count correlations, a measure that is thought to reflect correlated noise in the population of neurons. Here, we studied whether feature-based attention biases the correlated activity between neurons when attention is directed towards their common preferred feature. Simultaneous single-unit recordings were obtained from multiple neurons in secondary somatosensory cortex in non-human primates performing feature-attention tasks. Both firing rate and spike-synchrony were enhanced when attention was directed towards the preferred feature of cells. However, attention effects on spike-synchrony had a tighter relationship with behavior. Further, attention decreased spike-count correlations when it was directed towards the receptive field of cells. Our data indicate that temporal correlation codes play a role in mediating feature selection, and are consistent with a feature-based selection model that operates by reducing the overall noise in a population and synchronizing activity across subpopulations that encode the relevant features of sensory stimuli.

Local spectral anisotropy is a valid cue for figure-ground organization in natural scenes

An important step in the process of understanding visual scenes is its organization in different perceptual objects which requires figure-ground segregation. The determination of which side of an occlusion boundary is figure (closer to the observer) and which is ground (further away from the observer) is made through a combination of global cues, like convexity, and local cues, like T-junctions. We here focus on a novel set of local cues in the intensity patterns along occlusion boundaries which we show to differ between figure and ground. Image patches are extracted from natural scenes from two standard image sets along the boundaries of objects and spectral analysis is performed separately on figure and ground. On the figure side, oriented spectral power orthogonal to the occlusion boundary significantly exceeds that parallel to the boundary. This "spectral anisotropy" is present only for higher spatial frequencies, and absent on the ground side. The difference in spectral anisotropy between the two sides of an occlusion border predicts which is the figure and which the background with an accuracy exceeding 60% per patch. Spectral anisotropy of close-by locations along the boundary co-varies but is largely independent over larger distances which allows to combine results from different image regions. Given the low cost of this strictly local computation, we propose that spectral anisotropy along occlusion boundaries is a valuable cue for figure-ground segregation. A data base of images and extracted patches labeled for figure and ground is made freely available.

A model of proto-object based saliency

Organisms use the process of selective attention to optimally allocate their computational resources to the instantaneously most relevant subsets of a visual scene, ensuring that they can parse the scene in real time. Many models of bottom-up attentional selection assume that elementary image features, like intensity, color and orientation, attract attention. Gestalt psychologists, however, argue that humans perceive whole objects before they analyze individual features. This is supported by recent psychophysical studies that show that objects predict eye-fixations better than features. In this report we present a neurally inspired algorithm of object based, bottom-up attention. The model rivals the performance of state of the art non-biologically plausible feature based algorithms (and outperforms biologically plausible feature based algorithms) in its ability to predict perceptual saliency (eye fixations and subjective interest points) in natural scenes. The model achieves this by computing saliency as a function of proto-objects that establish the perceptual organization of the scene. All computational mechanisms of the algorithm have direct neural correlates, and our results provide evidence for the interface theory of attention.

Mechanisms underlying the influence of saliency on value-based decisions

Objects in the environment differ in their low-level perceptual properties (e.g., how easily a fruit can be recognized) as well as in their subjective value (how tasty it is). We studied the influence of visual salience on value-based decisions using a two alternative forced choice task, in which human subjects rapidly chose items from a visual display. All targets were equally easy to detect. Nevertheless, both value and salience strongly affected choices made and reaction times. We analyzed the neuronal mechanisms underlying these behavioral effects using stochastic accumulator models, allowing us to characterize not only the averages of reaction times but their full distributions. Independent models without interaction between the possible choices failed to reproduce the observed choice behavior, while models with mutual inhibition between alternative choices produced much better results. Mutual inhibition thus is an important feature of the decision mechanism. Value influenced the amount of accumulation in all models. In contrast, increased salience could either lead to an earlier start (onset model) or to a higher rate (speed model) of accumulation. Both models explained the data from the choice trials equally well. However, salience also affected reaction times in no-choice trials in which only one item was present, as well as error trials. Only the onset model could explain the observed reaction time distributions of error trials and no-choice trials. In contrast, the speed model could not, irrespective of whether the rate increase resulted from more frequent accumulated quanta or from larger quanta. Visual salience thus likely provides an advantage in the onset, not in the processing speed, of value-based decision making.

Locally Contractive Dynamics in Generalized Integrate-and-Fire Neurons

Integrate-and-fire models of biological neurons combine differential equations with discrete spike events. In the simplest case, the reset of the neuronal voltage to its resting value is the only spike event. The response of such a model to constant input injection is limited to tonic spiking. We here study a generalized model in which two simple spike-induced currents are added. We show that this neuron exhibits not only tonic spiking at various frequencies but also the commonly observed neuronal bursting. Using analytical and numerical approaches, we show that this model can be reduced to a one-dimensional map of the adaptation variable and that this map is locally contractive over a broad set of parameter values. We derive a sufficient analytical condition on the parameters for the map to be globally contractive, in which case all orbits tend to a tonic spiking state determined by the fixed point of the return map. We then show that bursting is caused by a discontinuity in the return map, in which case the map is piecewise contractive. We perform a detailed analysis of a class of piecewise contractive maps that we call bursting maps and show that they robustly generate stable bursting behavior. To the best of our knowledge, this work is the first to point out the intimate connection between bursting dynamics and piecewise contractive maps. Finally, we discuss bifurcations in this return map, which cause transitions between spiking patterns.

Visual Attention and Applications in Multimedia Technologies

Making technological advances in the field of human-machine interactions requires that the capabilities and limitations of the human perceptual system are taken into account. The focus of this report is an important mechanism of perception, visual selective attention, which is becoming more and more important for multimedia applications. We introduce the concept of visual attention and describe its underlying mechanisms. In particular, we introduce the concepts of overt and covert visual attention, and of bottom-up and top-down processing. Challenges related to modeling visual attention and their validation using ad hoc ground truth are also discussed. Examples of the usage of visual attention models in image and video processing are presented. We emphasize multimedia delivery, retargeting and quality assessment of image and video, medical imaging, and the field of stereoscopic 3D images applications.

A simple model of mechanotransduction in primate glabrous skin

Tactile stimulation of the hand evokes highly precise and repeatable patterns of activity in mechanoreceptive afferents; the strength (i.e., firing rate) and timing of these responses have been shown to convey stimulus information. To achieve an understanding of the mechanisms underlying the representation of tactile stimuli in the nerve, we developed a two-stage computational model consisting of a nonlinear mechanical transduction stage followed by a generalized integrate-and-fire mechanism. The model improves upon a recently published counterpart in two important ways. First, complexity is dramatically reduced (at least one order of magnitude fewer parameters). Second, the model comprises a saturating nonlinearity and therefore can be applied to a much wider range of stimuli. We show that both the rate and timing of afferent responses are predicted with remarkable precision and that observed adaptation patterns and threshold behavior are well captured. We conclude that the responses of mechanoreceptive afferents can be understood using a very parsimonious mechanistic model, which can then be used to accurately simulate the responses of afferent populations.

Parameter estimation of a spiking silicon neuron

Spiking neuron models are used in a multitude of tasks ranging from understanding neural behavior at its most basic level to neuroprosthetics. Parameter estimation of a single neuron model, such that the model's output matches that of a biological neuron is an extremely important task. Hand tuning of parameters to obtain such behaviors is a difficult and time consuming process. This is further complicated when the neuron is instantiated in silicon (an attractive medium in which to implement these models) as fabrication imperfections make the task of parameter configuration more complex. In this paper we show two methods to automate the configuration of a silicon (hardware) neuron's parameters. First, we show how a Maximum Likelihood method can be applied to a leaky integrate and fire silicon neuron with spike induced currents to fit the neuron's output to desired spike times. We then show how a distance based method which approximates the negative log likelihood of the lognormal distribution can also be used to tune the neuron's parameters. We conclude that the distance based method is better suited for parameter configuration of silicon neurons due to its superior optimization speed.

Estimating parameters of generalized integrate-and-fire neurons from the maximum likelihood of spike trains

When a neuronal spike train is observed, what can we deduce from it about the properties of the neuron that generated it? A natural way to answer this question is to make an assumption about the type of neuron, select an appropriate model for this type, and then choose the model parameters as those that are most likely to generate the observed spike train. This is the maximum likelihood method. If the neuron obeys simple integrate-and-fire dynamics, Paninski, Pillow, and Simoncelli (2004) showed that its negative log-likelihood function is convex and that, at least in principle, its unique global minimum can thus be found by gradient descent techniques. Many biological neurons are, however, known to generate a richer repertoire of spiking behaviors than can be explained in a simple integrate-and-fire model. For instance, such a model retains only an implicit (through spike-induced currents), not an explicit, memory of its input; an example of a physiological situation that cannot be explained is the absence of firing if the input current is increased very slowly. Therefore, we use an expanded model (Mihalas & Niebur, 2009 ), which is capable of generating a large number of complex firing patterns while still being linear. Linearity is important because it maintains the distribution of the random variables and still allows maximum likelihood methods to be used. In this study, we show that although convexity of the negative log-likelihood function is not guaranteed for this model, the minimum of this function yields a good estimate for the model parameters, in particular if the noise level is treated as a free parameter. Furthermore, we show that a nonlinear function minimization method (r-algorithm with space dilation) usually reaches the global minimum.

Optimization methods for spiking neurons and networks

Spiking neurons and spiking neural circuits are finding uses in a multitude of tasks such as robotic locomotion control, neuroprosthetics, visual sensory processing, and audition. The desired neural output is achieved through the use of complex neuron models, or by combining multiple simple neurons into a network. In either case, a means for configuring the neuron or neural circuit is required. Manual manipulation of parameters is both time consuming and non-intuitive due to the nonlinear relationship between parameters and the neuron's output. The complexity rises even further as the neurons are networked and the systems often become mathematically intractable. In large circuits, the desired behavior and timing of action potential trains may be known but the timing of the individual action potentials is unknown and unimportant, whereas in single neuron systems the timing of individual action potentials is critical. In this paper, we automate the process of finding parameters. To configure a single neuron we derive a maximum likelihood method for configuring a neuron model, specifically the Mihalas-Niebur Neuron. Similarly, to configure neural circuits, we show how we use genetic algorithms (GAs) to configure parameters for a network of simple integrate and fire with adaptation neurons. The GA approach is demonstrated both in software simulation and hardware implementation on a reconfigurable custom very large scale integration chip.

Improved integral equation solution for the first passage time of leaky integrate-and-fire neurons

An accurate calculation of the first passage time probability density (FPTPD) is essential for computing the likelihood of solutions of the stochastic leaky integrate-and-fire model. The previously proposed numerical calculation of the FPTPD based on the integral equation method discretizes the probability current of the voltage crossing the threshold. While the method is accurate for high noise levels, we show that it results in large numerical errors for small noise. The problem is solved by analytically computing, in each time bin, the mean probability current. Efficiency is further improved by identifying and ignoring time bins with negligible mean probability current.

Self-organized criticality occurs in non-conservative neuronal networks during Up states

During sleep, under anesthesia and in vitro, cortical neurons in sensory, motor, association and executive areas fluctuate between Up and Down states (UDS) characterized by distinct membrane potentials and spike rates [1, 2, 3, 4, 5]. Another phenomenon observed in preparations similar to those that exhibit UDS, such as anesthetized rats [6], brain slices and cultures devoid of sensory input [7], as well as awake monkey cortex [8] is self-organized criticality (SOC). This is characterized by activity "avalanches" whose size distributions obey a power law with critical exponent of about [Formula: see text] and branching parameter near unity. Recent work has demonstrated SOC in conservative neuronal network models [9, 10], however critical behavior breaks down when biologically realistic non-conservatism is introduced [9]. We here report robust SOC behavior in networks of non-conservative leaky integrate-and-fire neurons with short-term synaptic depression. We show analytically and numerically that these networks typically have 2 stable activity levels corresponding to Up and Down states, that the networks switch spontaneously between them, and that Up states are critical and Down states are subcritical.

Theory of the locomotion of nematodes: Dynamics of undulatory progression on a surface

We develop a model of the undulatory locomotion of nematodes, in particular that of Caenorhabditis elegans, based on mechanics. The model takes into account the most important forces acting on a moving worm and allows the computer simulation of a creeping nematode. These forces are produced by the interior pressure in the liquid-filled body cavity, the elasticity of the cuticle, the excitation of certain sets of muscles and the friction between the body and its support.We propose that muscle excitation patterns can be generated by stretch receptor control. By solving numerically the equations of motion of the model of the nematode, we demonstrate that these muscle excitation patterns are suitable for the propulsion of the animal.

Everyone knows what is interesting: salient locations which should be fixated

Most natural scenes are too complex to be perceived instantaneously in their entirety. Observers therefore have to select parts of them and process these parts sequentially. We study how this selection and prioritization process is performed by humans at two different levels. One is the overt attention mechanism of saccadic eye movements in a free-viewing paradigm. The second is a conscious decision process in which we asked observers which points in a scene they considered the most interesting. We find in a very large participant population (more than one thousand) that observers largely agree on which points they consider interesting. Their selections are also correlated with the eye movement pattern of different subjects. Both are correlated with predictions of a purely bottom-up saliency map model. Thus, bottom-up saliency influences cognitive processes as far removed from the sensory periphery as in the conscious choice of what an observer considers interesting.

A generalized linear integrate-and-fire neural model produces diverse spiking behaviors

For simulations of neural networks, there is a trade-off between the size of the network that can be simulated and the complexity of the model used for individual neurons. In this study, we describe a generalization of the leaky integrate-and-fire model that produces a wide variety of spiking behaviors while still being analytically solvable between firings. For different parameter values, the model produces spiking or bursting, tonic, phasic or adapting responses, depolarizing or hyperpolarizing after potentials and so forth. The model consists of a diagonalizable set of linear differential equations describing the time evolution of membrane potential, a variable threshold, and an arbitrary number of firing-induced currents. Each of these variables is modified by an update rule when the potential reaches threshold. The variables used are intuitive and have biological significance. The model's rich behavior does not come from the differential equations, which are linear, but rather from complex update rules. This single-neuron model can be implemented using algorithms similar to the standard integrate-and-fire model. It is a natural match with event-driven algorithms for which the firing times are obtained as a solution of a polynomial equation.

Synchrony and the binding problem in macaque visual cortex

We tested the binding-by-synchrony hypothesis which proposes that object representations are formed by synchronizing spike activity between neurons that code features of the same object. We studied responses of 32 pairs of neurons recorded with microelectrodes 3 mm apart in the visual cortex of macaques performing a fixation task. Upon mapping the receptive fields of the neurons, a quadrilateral was generated so that two of its sides were centered in the receptive fields at the optimal orientations. This one-figure condition was compared with a two-figure condition in which the neurons were stimulated by two separate figures, keeping the local edges in the receptive fields identical. For each neuron, we also determined its border ownership selectivity (H. Zhou, H. S. Friedman, & R. von der Heydt, 2000). We examined both synchronization and correlation at nonzero time lag. After correcting for effects of the firing rate, we found that synchrony did not depend on the binding condition. However, finding synchrony in a pair of neurons was correlated with finding border-ownership selectivity in both members of the pair. This suggests that the synchrony reflected the connectivity in the network that generates border ownership assignment. Thus, we have not found evidence to support the binding-by-synchrony hypothesis.

Exact solutions for rate and synchrony in recurrent networks of coincidence detectors

We provide analytical solutions for mean firing rates and cross-correlations of coincidence detector neurons in recurrent networks with excitatory or inhibitory connectivity, with rate-modulated steady-state spiking inputs. We use discrete-time finite-state Markov chains to represent network state transition probabilities, which are subsequently used to derive exact analytical solutions for mean firing rates and cross-correlations. As illustrated in several examples, the method can be used for modeling cortical microcircuits and clarifying single-neuron and population coding mechanisms. We also demonstrate that increasing firing rates do not necessarily translate into increasing cross-correlations, though our results do support the contention that firing rates and cross-correlations are likely to be coupled. Our analytical solutions underscore the complexity of the relationship between firing rates and cross-correlations.

A model for neuronal competition during development

We report that developmental competition between sympathetic neurons for survival is critically dependent on a sensitization process initiated by target innervation and mediated by a series of feedback loops. Target-derived nerve growth factor (NGF) promoted expression of its own receptor TrkA in mouse and rat neurons and prolonged TrkA-mediated signals. NGF also controlled expression of brain-derived neurotrophic factor and neurotrophin-4, which, through the receptor p75, can kill neighboring neurons with low retrograde NGF-TrkA signaling whereas neurons with high NGF-TrkA signaling are protected. Perturbation of any of these feedback loops disrupts the dynamics of competition. We suggest that three target-initiated events are essential for rapid and robust competition between neurons: sensitization, paracrine apoptotic signaling, and protection from such effects.

High-frequency gamma activity (80-150Hz) is increased in human cortex during selective attention

OBJECTIVE

To study the role of gamma oscillations (>30Hz) in selective attention using subdural electrocorticography (ECoG) in humans.

METHODS

We recorded ECoG in human subjects implanted with subdural electrodes for epilepsy surgery. Sequences of auditory tones and tactile vibrations of 800 ms duration were presented asynchronously, and subjects were asked to selectively attend to one of the two stimulus modalities in order to detect an amplitude increase at 400 ms in some of the stimuli.

RESULTS

Event-related ECoG gamma activity was greater over auditory cortex when subjects attended auditory stimuli and was greater over somatosensory cortex when subjects attended vibrotactile stimuli. Furthermore, gamma activity was also observed over prefrontal cortex when stimuli appeared in either modality, but only when they were attended. Attentional modulation of gamma power began approximately 400 ms after stimulus onset, consistent with the temporal demands on attention. The increase in gamma activity was greatest at frequencies between 80 and 150 Hz, in the so-called high-gamma frequency range.

CONCLUSIONS

There appears to be a strong link between activity in the high-gamma range (80-150 Hz) and selective attention.

SIGNIFICANCE

Selective attention is correlated with increased activity in a frequency range that is significantly higher than what has been reported previously using EEG recordings.

Generation of synthetic spike trains with defined pairwise correlations

Recent technological advances as well as progress in theoretical understanding of neural systems have created a need for synthetic spike trains with controlled mean rate and pairwise cross-correlation. This report introduces and analyzes a novel algorithm for the generation of discretized spike trains with arbitrary mean rates and controlled cross correlation. Pairs of spike trains with any pairwise correlation can be generated, and higher-order correlations are compatible with common synaptic input. Relations between allowable mean rates and correlations within a population are discussed. The algorithm is highly efficient, its complexity increasing linearly with the number of spike trains generated and therefore inversely with the number of cross-correlated pairs.

Synchrony: a neural correlate of somatosensory attention

We investigated whether synchrony between neuronal spike trains is affected by the animal's attentional state. Cross-correlation functions between pairs of spike trains in the second somatosensory cortex (SII) of three macaque monkeys trained to switch attention between a visual task and a tactile task were computed. We previously showed that the majority of recorded neuron pairs (66%) in SII cortex fire synchronously while the animals performed either task and that in a subset of neuron pairs (17%), the degree of synchrony was affected by the animal's attentional state. Of the neuron pairs that showed changes in synchrony with attention, about 80% showed increased synchrony when the animal attended to the tactile stimulus. Here, we show that peak correlation typically occurred at a delay <25 ms; most commonly the delay was close to zero. Half-widths of the correlation peaks were distributed between a few milliseconds and hundreds of milliseconds, with the majority lying <100 ms and the mode of the distribution around 20-30 ms. Maximal change in synchrony occurred mainly during the periods when the stimulus was present, and synchrony usually increased when attention was on the tactile stimulus. If periods of elevated firing rates around the motor response times were removed from the analysis, the percentage of pairs that changed the degree of synchrony with attention more than doubled (from 35 to 72%). The observed effects did not depend on details of the statistical criteria or of the time window used in the analysis.