Learning dynamic representations of the functional connectome in neurobiological networks

The static synaptic connectivity of neuronal circuits stands in direct contrast to the dynamics of their function. As in changing community interactions, different neurons can participate actively in various combinations to effect behaviors at different times. We introduce an unsupervised approach to learn the dynamic affinities between neurons in live, behaving animals, and to reveal which communities form among neurons at different times. The inference occurs in two major steps. First, pairwise non-linear affinities between neuronal traces from brain-wide calcium activity are organized by non-negative tensor factorization (NTF). Each factor specifies which groups of neurons are most likely interacting for an inferred interval in time, and for which animals. Finally, a generative model that allows for weighted community detection is applied to the functional motifs produced by NTF to reveal a dynamic functional connectome. Since time codes the different experimental variables (e.g., application of chemical stimuli), this provides an atlas of neural motifs active during separate stages of an experiment (e.g., stimulus application or spontaneous behaviors). Results from our analysis are experimentally validated, confirming that our method is able to robustly predict causal interactions between neurons to generate behavior.

Population encoding of stimulus features along the visual hierarchy

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to the mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

Population encoding of stimulus features along the visual hierarchy

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

IAN: Iterated Adaptive Neighborhoods for Manifold Learning and Dimensionality Estimation

Invoking the manifold assumption in machine learning requires knowledge of the manifold's geometry and dimension, and theory dictates how many samples are required. However, in most applications, the data are limited, sampling may not be uniform, and the manifold's properties are unknown; this implies that neighborhoods must adapt to the local structure. We introduce an algorithm for inferring adaptive neighborhoods for data given by a similarity kernel. Starting with a locally conservative neighborhood (Gabriel) graph, we sparsify it iteratively according to a weighted counterpart. In each step, a linear program yields minimal neighborhoods globally, and a volumetric statistic reveals neighbor outliers likely to violate manifold geometry. We apply our adaptive neighborhoods to nonlinear dimensionality reduction, geodesic computation, and dimension estimation. A comparison against standard algorithms using, for example, k-nearest neighbors, demonstrates the usefulness of our approach.

GOOD CONTINUATION IN 3D: THE NEUROGEOMETRY OF STEREO VISION

Classical good continuation for image curves is based on 2D position and orientation. It is supported by the columnar organization of cortex, by psychophysical experiments, and by rich models of (differential) geometry. Here we extend good continuation to stereo. We introduce a neurogeometric model, in which the parametrizations involve both spatial and orientation disparities. Our model provides insight into the neurobiology, suggesting an implicit organization for neural interactions and a well-defined 3D association field. Our model sheds light on the computations underlying the correspondence problem, and illustrates how good continuation in the world generalizes good continuation in the plane.

From boundaries to bumps: When closed (extremal) contours are critical

Invariants underlying shape inference are elusive: A variety of shapes can give rise to the same image, and a variety of images can be rendered from the same shape. The occluding contour is a rare exception: It has both image salience, in terms of isophotes, and surface meaning, in terms of surface normal. We relax the notion of occluding contour and, more accurately, the rim on the object that projects to it, to define closed extremal curves. This new shape descriptor is invariant over different renderings. It exists at the topological level, which guarantees an image-based counterpart. It surrounds bumps and dents, as well as common interior shape components, and formalizes the qualitative nature of bump perception. The invariants are biologically computable, unify shape inferences from shading and specular materials, and predict new phenomena in bump and dent perception. Most important, working at the topological level allows us to capture the elusive aspect of bump boundaries.

Colour, contours, shading and shape: flow interactions reveal anchor neighbourhoods

Two dilemmas arise in inferring shape information from shading. First, depending on the rendering physics, images can change significantly with (even) small changes in lighting or viewpoint, while the percept frequently does not. Second, brightness variations can be induced by material effects-such as pigmentation-as well as by shading effects. Improperly interpreted, material effects would confound shading effects. We show how these dilemmas are coupled by reviewing recent developments in shape inference together with a role for colour in separating material from shading effects. Aspects of both are represented in a common geometric (flow) framework, and novel displays of hue/shape interaction demonstrate a global effect with interactions limited to localized regions. Not all parts of an image are perceptually equal; shape percepts appear to be constructed from image anchor regions.

Spatial self-organization resolves conflicts between individuality and collective migration

Collective behavior can spontaneously emerge when individuals follow common rules of interaction. However, the behavior of each individual differs due to existing genetic and non-genetic variation within the population. It remains unclear how this individuality is managed to achieve collective behavior. We quantify individuality in bands of clonal Escherichia coli cells that migrate collectively along a channel by following a self-generated gradient of attractant. We discover that despite substantial differences in individual chemotactic abilities, the cells are able to migrate as a coherent group by spontaneously sorting themselves within the moving band. This sorting mechanism ensures that differences between individual chemotactic abilities are compensated by differences in the local steepness of the traveling gradient each individual must navigate, and determines the minimum performance required to travel with the band. By resolving conflicts between individuality and collective migration, this mechanism enables populations to maintain advantageous diversity while on the move.

How bacteria migrate collectively despite individual phenotypic variation is not understood. Here, the authors show that cells spontaneously sort themselves within moving bands such that variations in individual tumble bias, a determinant of gradient climbing speed, are compensated by the local gradient steepness experienced by individuals.

Feedback between motion and sensation provides nonlinear boost in run-and-tumble navigation

Many organisms navigate gradients by alternating straight motions (runs) with random reorientations (tumbles), transiently suppressing tumbles whenever attractant signal increases. This induces a functional coupling between movement and sensation, since tumbling probability is controlled by the internal state of the organism which, in turn, depends on previous signal levels. Although a negative feedback tends to maintain this internal state close to adapted levels, positive feedback can arise when motion up the gradient reduces tumbling probability, further boosting drift up the gradient. Importantly, such positive feedback can drive large fluctuations in the internal state, complicating analytical approaches. Previous studies focused on what happens when the negative feedback dominates the dynamics. By contrast, we show here that there is a large portion of physiologically-relevant parameter space where the positive feedback can dominate, even when gradients are relatively shallow. We demonstrate how large transients emerge because of non-normal dynamics (non-orthogonal eigenvectors near a stable fixed point) inherent in the positive feedback, and further identify a fundamental nonlinearity that strongly amplifies their effect. Most importantly, this amplification is asymmetric, elongating runs in favorable directions and abbreviating others. The result is a “ratchet-like” gradient climbing behavior with drift speeds that can approach half the maximum run speed of the organism. Our results thus show that the classical drawback of run-and-tumble navigation—wasteful runs in the wrong direction—can be mitigated by exploiting the non-normal dynamics implicit in the run-and-tumble strategy.

Countless bacteria, larvae and even larger organisms (and robots) navigate gradients by alternating periods of straight motion (runs) with random reorientation events (tumbles). Control of the tumble probability is based on previously-encountered signals. A drawback of this run-and-tumble strategy is that occasional runs in the wrong direction are wasteful. Here we show that there is an operating regime within the organism’s internal parameter space where run-and-tumble navigation can be extremely efficient. We characterize how the positive feedback between behavior and sensed signal results in a type of non-equilibrium dynamics, with the organism rapidly tumbling after moving in the wrong direction and extending motion in the right ones. For a distant source, then, the organism can find it fast.

Toward Efficient Trajectory Planning: The Path-Velocity Decomposition

We present a novel approach to solving the trajectory plan ning problem (TPP) in time-varying environments. The es sence of our approach lies in a heuristic but natural decom position of TPP into two subproblems: (1) planning a path to avoid collision with static obstacles and (2) planning the velocity along the path to avoid collision with moving obsta cles. We call thefirst subproblem the path planning problem (PPP) and the second the velocity planning problem (VPP). Thus, our decomposition is summarized by the equation TPP => PPP + VPP. The symbol => indicates that the de composition holds under certain assumptions, e.g., when obstacles are moving independently of ( i.e., not tracking ) the robot. Furthermore, we pose the VPP in path-time space, where time is explicitly represented as an extra dimension, and reduce it to a graph search in this space. In fact, VPP is transformed to a two-dimensional PPP in path-time space with some additional constraints. Algorithms are then pre sented to solve the VPP with different optimality criteria: minimum length in path-time space, and minimum time.

Local field potentials and border ownership: A conjecture about computation in visual cortex

Border ownership is an intermediate-level visual task: it must integrate (upward flowing) image information about edges with (downward flowing) shape information. This highlights the familiar local-to-global aspect of border formation (linking of edge elements to form contours) with the much less studied global-to-local aspect (which edge elements form part of the same shape). To address this task we show how to incorporate certain high-level notions of distance and geometric arrangement into a form that can influence image-based edge information. The center of the argument is a reaction-diffusion equation that reveals how (global) aspects of the distance map (that is, shape) can be "read out" locally, suggesting a solution to the border ownership problem. Since the reaction-diffusion equation defines a field, a possible information processing role for the local field potential can be defined. We argue that such fields also underlie the Gestalt notion of closure, especially when it is refined using modern experimental techniques. An important implication of this theoretical argument is that, if true, then network modeling must be extended to include the substrate surrounding spiking neurons, including glia.

Differential geometric inference in surface stereo

Many traditional two-view stereo algorithms explicitly or implicitly use the frontal parallel plane assumption when exploiting contextual information since, e.g., the smoothness prior biases toward constant disparity (depth) over a neighborhood. This introduces systematic errors to the matching process for slanted or curved surfaces. These errors are nonnegligible for detailed geometric modeling of natural objects such as a human face. We show how to use contextual information geometrically to avoid such errors. A differential geometric study of smooth surfaces allows contextual information to be encoded in Cartan's moving frame model over local quadratic approximations, providing a framework of geometric consistency for both depth and surface normals; the accuracy of our reconstructions argues for the sufficiency of the approximation. In effect, Cartan's model provides the additional constraint necessary to move beyond the frontal parallel plane assumption in stereo reconstruction. It also suggests how geometry can extend surfaces to account for unmatched points due to partial occlusion.

Attention, segregation, and textons: bridging the gap between object-based attention and texton-based segregation

Studies of object-based attention (OBA) have suggested that attentional selection is intimately associated with discrete objects. However, the relationship of this association to the basic visual features ('textons') which guide the segregation of visual scenes into 'objects' remains largely unexplored. Here we study this hypothesized relationship for one of the most conspicuous features of early vision: orientation. To do so we examine how attention spreads through uniform (one 'object') orientation-defined textures (ODTs), and across texture-defined boundaries in discontinuous (two 'objects') ODTs. Using the divided-attention paradigm we find that visual events that are known to trigger orientation-based texture segregation, namely perceptual boundaries defined by high orientation and/or curvature gradients, also induce a significant cost on attentional selection. At the same time we show that no effect is incurred by the absolute value of the textons, i.e., by the general direction (or, the 'grain') of the texture-in conflict with previous findings in the OBA literature. Collectively these experiments begin to reveal the link between object-based attention and texton-based segregation, a link which also offers important cross-disciplinary methodological advantages.

A constant production hypothesis guides leaf venation patterning

We propose a theoretical mechanism that enables the elaboration of veins to supply distant cells during leaf development. In contrast to the more standard view that a signal (e.g., auxin) is produced at isolated sites to stimulate growth, we determine the consequences of the hypothesis that auxin is produced at a constant rate in every cell. High concentration sites for auxin emerge naturally in a reaction-diffusion model, together with global information about leaf shape and existing venation. Because the global information is encoded as auxin concentration and its gradient, those signals provide individual cells with sufficient information to determine their own fate. Unlike other models, a single substance suffices for the reaction-diffusion at early, but not initial, stages of development. Neither complex interactions nor predetermination are necessary. We predict angiosperm areolation patterns in simulation, and our model further implies the Sachs Canalization Hypothesis and resolves a dilemma regarding the role of auxin in cell growth.

Geometric diffusions for the analysis of data from sensor networks

Harmonic analysis on manifolds and graphs has recently led to mathematical developments in the field of data analysis. The resulting new tools can be used to compress and analyze large and complex data sets, such as those derived from sensor networks or neuronal activity datasets, obtained in the laboratory or through computer modeling. The nature of the algorithms (based on diffusion maps and connectivity strengths on graphs) possesses a certain analogy with neural information processing, and has the potential to provide inspiration for modeling and understanding biological organization in perception and memory formation.

Indexing hierarchical structures using graph spectra

Hierarchical image structures are abundant in computer vision and have been used to encode part structure, scale spaces, and a variety of multiresolution features. In this paper, we describe a framework for indexing such representations that embeds the topological structure of a directed acyclic graph (DAG) into a low-dimensional vector space. Based on a novel spectral characterization of a DAG, this topological signature allows us to efficiently retrieve a promising set of candidates from a database of models using a simple nearest-neighbor search. We establish the insensitivity of the signature to minor perturbation of graph structure due to noise, occlusion, or node split/merge. To accommodate large-scale occlusion, the DAG rooted at each nonleaf node of the query "votes" for model objects that share that "part," effectively accumulating local evidence in a model DAG's topological subspaces. We demonstrate the approach with a series of indexing experiments in the domain of view-based 3D object recognition using shock graphs.

Geometric diffusions as a tool for harmonic analysis and structure definition of data: multiscale methods

In the companion article, a framework for structural multiscale geometric organization of subsets of R(n) and of graphs was introduced. Here, diffusion semigroups are used to generate multiscale analyses in order to organize and represent complex structures. We emphasize the multiscale nature of these problems and build scaling functions of Markov matrices (describing local transitions) that lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. This article deals with the construction of fast-order N algorithms for data representation and for homogenization of heterogeneous structures.

Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps

We provide a framework for structural multiscale geometric organization of graphs and subsets of R(n). We use diffusion semigroups to generate multiscale geometries in order to organize and represent complex structures. We show that appropriately selected eigenfunctions or scaling functions of Markov matrices, which describe local transitions, lead to macroscopic descriptions at different scales. The process of iterating or diffusing the Markov matrix is seen as a generalization of some aspects of the Newtonian paradigm, in which local infinitesimal transitions of a system lead to global macroscopic descriptions by integration. We provide a unified view of ideas from data analysis, machine learning, and numerical analysis.

Hue geometry and horizontal connections

Primate visual systems support an elaborate specialization for processing color information. Concentrating on the hue component, we observe that, contrary to Mondrian-like assumptions, hue varies in a smooth manner for ecologically important natural imagery. To represent these smooth variations, and to support those information processing tasks that utilize hue, a piecewise smooth hue field is postulated. The geometry of hue-patch interactions is developed analogously to orientation-patch interactions in texture. The result is a model for long-range (horizontal) interactions in the color domain, the power of which is demonstrated on a number of examples. Implications for computer image processing, computer vision, visual neurophysiology and psychophysics are discussed.

Cortical connections and early visual function: intra- and inter-columnar processing

While it is widely assumed that the long-range horizontal connections in V1 are present to support contour integration, there has been only limited consideration of other possible relationships between anatomy and physiology (the horizontal connections) and visual function beyond contour integration. We introduce the possibility of other relationships directly from the perspective of computation and differential geometry by identifying orientation columns in visual physiology with the (unit) tangent bundle in differential geometry. This suggests abstracting early vision in a space that incorporates both position and orientation, from which we show that the physiology is capable of supporting a number of functional computations beyond contour integration, including texture-flow and shading-flow integration, as well as certain relationships between them. The geometric abstraction emphasizes the role of curvature, which necessitates a coupled investigation into how it might be estimated. The result is an elaboration of layer-to-layer interactions within an orientation column, with non-linearities possibly implemented by shunting inhibition. Finally, we show how the same computational framework naturally lends itself to solving stereo correspondence, with binocular tangents abstracting curves in space.

Sensitivity to curvatures in orientation-based texture segmentation

Texture segregation has long been attributed to changes in the distribution of elementary features across the visual field [Nature 290 (12) (1981) 91; Biol. Cybernet. 54 (1986) 245]. The study of orientation, a conspicuous feature, has led to models of orientation-based texture segmentation (OBTS) that depend on the magnitude of one or two orientation gradients [Vis. Res. 31 (4) (1991) 679; Vis. Res. 31 (6) (1991) 1073] and influenced further by the relative configuration between the orientation textons and the global orientation edge [Percept. Psychophys. 52 (4) (1992) 255; Vis. Res. 35 (20) (1995) 2863]. Here we show that these models are at best partial and that the notion of orientation gradient has been incompletely used in the study of OBTS. To do so, we first study the behavior of orientation in orientation-defined texture patches. Geometrical analysis identifies two texture curvatures and reveals the incompleteness of previous stimuli. Psychophysical experimentation then demonstrates that segmentation is strongly affected by discontinuities in these curvatures. Importantly, we show that this sensitivity to curvature is independent of the orientation gradients and inconsistent with the simple configural considerations proposed in the past.

On the psychophysics of the shape triangle

We earlier introduced an approach to categorical shape description based on the singularities (shocks) of curve evolution equations. We now consider the simplest compositions of shocks, and show that they lead to three classes of parametrically ordered shape sequences, organized along the sides of a shape triangle. By conducting several psychophysical experiments we demonstrate that shock-based descriptions are predictive of performance in shape perception. Most significantly, the experiments reveal a fundamental difference between perceptual effects dominated by when shocks form with respect to one another, versus those dominated by where they form. The shock-based theory provides a foundation for unifying tasks as diverse as shape bisection, recognition, and categorization.

Computing with self-excitatory cliques: A model and an application to hyperacuity-scale computation in visual cortex

We present a model of visual computation based on tightly inter-connected cliques of pyramidal cells. It leads to a formal theory of cell assemblies, a specific relationship between correlated firing patterns and abstract functionality, and a direct calculation relating estimates of cortical cell counts to orientation hyperacuity. Our network architecture is unique in that (1) it supports a mode of computation that is both reliable and efficient; (2) the current-spike relations are modeled as an analog dynamical system in which the requisite computations can take place on the time scale required for an early stage of visual processing; and (3) the dynamics are triggered by the spatiotemporal response of cortical cells. This final point could explain why moving stimuli improve vernier sensitivity.

Evidence for boundary-specific grouping

A prerequisite for higher-level visual tasks such as object recognition is a segmentation of the image into distinct two-dimensional regions. While it has long been assumed that the human visual system jointly exploits region and boundary cues for image segmentation, we report the results of psychophysical experiments which suggest that the visual system relies on geometric properties of bounding contours such as closure and not on the texture of the two-dimensional regions they partition. These findings suggest that the visual system may code and links contours into coherent shapes before surface properties are conjoined.

Area and length minimizing flows for shape segmentation

A number of active contour models have been proposed that unify the curve evolution framework with classical energy minimization techniques for segmentation, such as snakes. The essential idea is to evolve a curve (in two dimensions) or a surface (in three dimensions) under constraints from image forces so that it clings to features of interest in an intensity image. The evolution equation has been derived from first principles as the gradient flow that minimizes a modified length functional, tailored to features such as edges. However, because the flow may be slow to converge in practice, a constant (hyperbolic) term is added to keep the curve/surface moving in the desired direction. We derive a modification of this term based on the gradient flow derived from a weighted area functional, with image dependent weighting factor. When combined with the earlier modified length gradient flow, we obtain a partial differential equation (PDE) that offers a number of advantages, as illustrated by several examples of shape segmentation on medical images. In many cases the weighted area flow may be used on its own, with significant computational savings.

Efficient Simplex-Like Methods for Equilibria of Nonsymmetric Analog Networks

What is the complexity of computing equilibria for physically implementable analog networks (Hopfield 1984; Sejnowski 1981) with arbitrary connectivity? We show that if the amplifiers are piecewise-linear, then such networks are instances of a game-theoretic model known as polymatrix games. In contrast with the usual gradient descent methods for symmetric networks, equilibria for polymatrix games may be computed by vertex pivoting algorithms similar to the simplex method for linear programming. Like the simplex method, these algorithms have characteristic low order polynomial behavior in virtually all practical cases, though not certain theoretical ones. While these algorithms cannot be applied to models requiring evolution from an initial point, they are applicable to “clamping” models whose input is expressed purely as a bias. Thus we have an a priori indication that such models are computationally tractable.

The discriminability of smooth stereoscopic surfaces

In this study the sensitivity of human vision to the smoothness of stereoscopic surface structure was investigated. In experiments 1 and 2 random-dot stereograms were used to evaluate the discrimination of smooth versus 'noisy' sinusoidal surfaces differing in the percentages of points on a single smooth surface. Fully coherent smooth surfaces were found to be much more discriminable than other less smooth randomly perturbed surfaces. In experiment 3 the discrimination between discontinuous triangle-wave surfaces and similarly shaped smoothly curved surfaces obtained from the addition of the fundamental and the third harmonic of the corresponding triangle-wave surface was evaluated. The triangle-wave surfaces were found to be more accurately discriminated from the smoothly curved surfaces than would be predicted from the detectability of the difference in their Fourier power spectra. This superior discriminability was attributed to differences between the curvature and/or discontinuity of the two surfaces. In experiment 3 the effects of incoherent 'noise' points on the discrimination between the two surface types were also evaluated. These randomly positioned noise points had a relatively small effect on the discrimination between the two surfaces. In general, the results of these experiments indicate that smooth surfaces are salient for stereopsis and that isolated local violations of smoothness are highly discriminable.

Coherent Compound Motion: Corners and Nonrigid Configurations

Consider two wire gratings, superimposed and moving across each other. Under certain conditions the two gratings will cohere into a single, compound pattern, which will appear to be moving in another direction. Such coherent motion patterns have been studied for sinusoidal component gratings, and give rise to percepts of rigid, planar motions. In this paper we show how to construct coherent motion displays that give rise to nonuniform, nonrigid, and nonplanar percepts. Most significantly, they also can define percepts with corners. Since these patterns are more consistent with the structure of natural scenes than rigid sinusoidal gratings, they stand as interesting stimuli for both computational and physiological studies. To illustrate, our display with sharp corners (tangent discontinuities or singularities) separating regions of coherent motion suggests that smoothing does not cross tangent discontinuities, a point that argues against existing (regularization) algorithms for computing motion. This leads us to consider how singularities can be confronted directly within optical flow computations, and we conclude with two hypotheses: (1) that singularities are represented within the motion system as multiple directions at the same retinotopic location; and (2) for component gratings to cohere, they must be at the same depth from the viewer. Both hypotheses have implications for the neural computation of coherent motion.

Two Stages of Curve Detection Suggest Two Styles of Visual Computation

The problem of detecting curves in visual images arises in both computer vision and biological visual systems. Our approach integrates constraints from these two sources and suggests that there are two different stages to curve detection, the first resulting in a local description, and the second in a global one. Each stage involves a different style of computation: in the first stage, hypotheses are represented explicitly and coarsely in a fixed, preconfigured architecture; in the second stage, hypotheses are represented implicitly and more finely in a dynamically constructed architecture. We also show how these stages could be related to physiology, specifying the earlier parts in a relatively fine-grained fashion and the later ones more coarsely.

Endstopping and curvature

Hypercomplex or endstopped visual cortical neurons are usually supposed to be concerned with length or end point analysis. However, recent evidence demonstrates that endstopped neurons are curvature-selective, a connection that we explore here in some detail. A model of endstopped simple cells is developed and a variety of computational simulations examine the connection of the model to the reported length and orientation responses of endstopped neurons. Even and odd versions of the model are described, both of which are shown to be curvature-selective. Even-symmetric instances of the model respond well to thin curves over a range of curve orientation and curvature, independent of sign of curvature. In contrast, odd-symmetric instances respond to both thin and thick curves while exhibiting a more complex curvature-sign dependence--responding in a sign-selective fashion to curved lines but not to curved edges. Finally, the response of the endstopped model to curve singularities is explored, and the possible role of nonendstopped and endstopped cells in building curve descriptions is discussed.

Texture fields and texture flows: sensitivity to differences

There are two large classes of textures, those with an overall orientation structure (texture flows) and those without (texture fields). We investigate human sensitivity to detecting a patch of texture field within a texture flow psychophysically by using random not Moiré patterns. The resultant sensitivity, as a function of patch-size and path-length, is then related to a computational model of orientation selection, which reveals a connection between texture structure and the estimation of curvature. Finally, the connection back to curvature is confirmed by demonstrating a similarity between the patch sensitivity data and previous data on sensitivity to corners in flow patterns.

Points and endpoints: a size/spacing constraint for dot grouping

One-dimensional arrangements of dots immediately group into contours. It is reported that, when these contours participate in certain larger arrangements, there is an abrupt point at which the percept changes as a function of dot spacing (or density along the contour). Closely spaced arrangements give rise to subjective effects involving apparent brightness and depth, whereas sparsely spaced ones do not. The effects are most clear in configurations that involve endpoints and possible occlusions. For these configurations, densely dotted contours are perceptually equivalent to solid ones, but sparse ones are not. This change in percept occurs abruptly and consistently at a dot to space ratio of 1:5, when the dot density is normalized by dot size, and this point is called the size/spacing constraint. It holds only for dots of the order of 1 min visual angle in diameter when small to modest contrast values are used. The subjective effects are not present for dotted contours (or even for solid ones) that are smaller (less than 0.5 min), and differ for contours that are larger (greater than 10 min). To demonstrate the significance of size/spacing constraints for early vision, a framework for grouping consisting of processes at many different levels is outlined, and the requirements for the earliest one (orientation selection) are sketched in greater detail. The size/spacing constraint follows directly from one of these requirements--receptive field structure--and seems to indicate a switch from early orientation-selection processes to later ones.

Endstopped neurons in the visual cortex as a substrate for calculating curvature

Neurons in the visual cortex typically respond selectively to the orientation, and velocity and direction of movement, of moving-bar stimuli. These responses are generally thought to provide information about the orientation and position of lines and edges in the visual field. Some cells are also endstopped, that is selective for bars of specific lengths. Hubel and Wiesel first observed that endstopped hypercomplex cells could respond to curved stimuli and suggested they might be involved in detection of curvature, but the exact relationship between endstopping and curvature has never been determined. We present here a mathematical model relating endstopping to curvature in which the difference in response of two simple cells gives rise to endstopping and varies in proportion to curvature. We also provide physiological evidence that endstopped cells in area 17 of the cat visual cortex are selective for curvature, whereas non-endstopped cells are not, and that some are selective for the sign of curvature. The prevailing view of edge and curve determination is that orientations are selected locally by the class of simple cortical cells and then integrated to form global curves. We have developed a computational theory of orientation selection which shows that measurements of orientation obtained by simple cells are not sufficient because there will be strong, incorrect responses from cells whose receptive fields (RFs) span distinct curves (Fig. 1). If estimates of curvature are available, however, these inappropriate responses can be eliminated. Curvature provides the key to structuring the network that underlies our theory and distinguishes it from previous lateral inhibition schemes.

The local structure of image discontinuities in one dimension

The detailed structure of intensities in the local neighborhood of an edge can often indicate the nature of the physical event givinig rise to that edge. We argue that the limit, as we approach arbitrarily close to either side of an edge, of such image parameters as type of texture, texture gradient, color, appropriate directional derivatives of intensity, etc., is a key aspect of this structure. However, the general problem of capturing this local structure is surprisingly complex. Thus, we restrict ourselves in this paper to a relatively simple domain¿one-dimensional cuts through idealized images modeled by piecewise smooth (C1) functions corrupted by Gaussian noise. Within this domain, we define local structure to be the limit of the uncorrupted intensity and of its derivatives as we approach arbitrarily close to either side of a discontinuity. We develop a technique that captures this local structure while simultaneously locating the discontinuities, and demonstrate that these tasks are in fact inseparable. The technique is an extension, using estimation theory, of the classical definition of discontinuity. It handles, in a consistent fashion, both jump discontinuities in the function and jump discontinuities in its first derivative (so-called step-edges are a special case of the former and roof-edges of the latter). It also integrates, again in a consistent fashion, information derived from a number of different neighborhood sizes.