The computational basis of an identified neuronal circuit for elementary motion detection in dipterous insects

Spatial Encoding of Translational Optic Flow in Planar Scenes by Elementary Motion Detector Arrays

Elementary Motion Detectors (EMD) are well-established models of visual motion estimation in insects. The response of EMDs are tuned to specific temporal and spatial frequencies of the input stimuli, which matches the behavioural response of insects to wide-field image rotation, called the optomotor response. However, other behaviours, such as speed and position control, cannot be fully accounted for by EMDs because these behaviours are largely unaffected by image properties and appear to be controlled by the ratio between the flight speed and the distance to an object, defined here as relative nearness. We present a method that resolves this inconsistency by extracting an unambiguous estimate of relative nearness from the output of an EMD array. Our method is suitable for estimation of relative nearness in planar scenes such as when flying above the ground or beside large flat objects. We demonstrate closed loop control of the lateral position and forward velocity of a simulated agent flying in a corridor. This finding may explain how insects can measure relative nearness and control their flight despite the frequency tuning of EMDs. Our method also provides engineers with a relative nearness estimation technique that benefits from the low computational cost of EMDs.

A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance

The most versatile and robust flying machines are still those produced by nature through evolution. The solutions to the 6 DOF control problem faced by these machines are implemented in extremely small neuronal structures comprising thousands of neurons. Hence, the biological principles of flight control are not only very effective but also efficient in terms of their implementation. An important question is to what extent these principles can be generalized to man-made flying platforms. Here, this question is investigated in relation to the computational and behavioral principles of the opto-motor system of the fly and locust. The aim is to provide a control infrastructure based only on biologically plausible and realistic neuronal models of the insect opto-motor system. It is shown that relying solely on vision, biologically constrained neuronal models of the fly visual system suffice for course stabilization and altitude control of a blimp-based UAV. Moreover, the system is augmented with a collision avoidance model based on the Lobula Giant Movement Detector neuron of the Locust. It is shown that the biologically constrained course stabilization model is highly robust and that the combined model is able to perform autonomous indoor flight.

Temporal order detection and coding in nervous systems

This letter discusses temporal order coding and detection in nervous systems. Detection of temporal order in the external world is an adaptive function of nervous systems. In addition, coding based on the temporal order of signals can be used as an internal code. Such temporal order coding is a subset of temporal coding. We discuss two examples of processing the temporal order of external events: the auditory location detection system in birds and the visual direction detection system in flies. We then discuss how somatosensory stimulus intensities are translated into a temporal order code in the human peripheral nervous system. We next turn our attention to input order coding in the mammalian cortex. We review work demonstrating the capabilities of cortical neurons for detecting input order. We then discuss research refuting and demonstrating the representation of stimulus features in the cortex by means of input order. After some general theoretical considerations on input order detection and coding, we conclude by discussing the existing and potential use of input order coding in neuromorphic engineering.

Neuronal representation of visual motion and orientation in the fly medulla

In insects, the first extraction of motion and direction clues from local brightness modulations is thought to take place in the medulla. However, whether and how these computations are represented in the medulla stills remain widely unknown, because electrical recording of the neurons in the medulla is difficult. As an effort to overcome this difficulty, we employed local electroporation in vivo in the medulla of the blowfly (Calliphora vicina) to stain small ensembles of neurons with a calcium-sensitive dye. We studied the responses of these neuronal ensembles to spatial and temporal brightness modulations and found selectivity for grating orientation. In contrast, the responses to the two opposite directions of motion of a grating with the same orientation were similar in magnitude, indicating that strong directional selectivity is either not present in the types of neurons covered by our data set, or that direction-selective signals are too closely spaced to be distinguished by our calcium imaging. The calcium responses also showed a bell-shaped dependency on the temporal frequency of drifting gratings, with an optimum higher than that observed in one of the subsequent processing stages, i.e., the lobula plate. Medulla responses were elicited by on- as well as off-stimuli with some spatial heterogeneity in the sensitivity for “on” and “off”, and in the polarity of the responses. Medulla neurons thus show similarities to some established principles of motion and edge detection in the vertebrate visual system.

Cholinergic circuits integrate neighboring visual signals in a Drosophila motion detection pathway

Detecting motion is a feature of all advanced visual systems [1], nowhere more so than in flying animals, like insects [2, 3]. In flies, an influential autocorrelation model for motion detection, the elementary motion detector circuit (EMD; [4, 5]), compares visual signals from neighboring photoreceptors to derive information on motion direction and velocity. This information is fed by two types of interneuron, L1 and L2, in the first optic neuropile, or lamina, to downstream local motion detectors in columns of the second neuropile, the medulla. Despite receiving carefully matched photoreceptor inputs, L1 and L2 drive distinct, separable pathways responding preferentially to moving "on" and "off" edges, respectively [6, 7]. Our serial electron microscopy (EM) identifies two types of transmedulla (Tm) target neurons, Tm1 and Tm2, that receive apparently matched synaptic inputs from L2. Tm2 neurons also receive inputs from two retinotopically posterior neighboring columns via L4, a third type of lamina neuron. Light microscopy reveals that the connections in these L2/L4/Tm2 circuits are highly determinate. Single-cell transcript profiling suggests that nicotinic acetylcholine receptors mediate transmission within the L2/L4/Tm2 circuits, whereas L1 is apparently glutamatergic. We propose that Tm2 integrates sign-conserving inputs from neighboring columns to mediate the detection of front-to-back motion generated during forward motion.

A neuronally based model of contrast gain adaptation in fly motion vision

Motion-sensitive neurons in the visual systems of many species, including humans, exhibit a depression of motion responses immediately after being exposed to rapidly moving images. This motion adaptation has been extensively studied in flies, but a neuronal mechanism that explains the most prominent component of adaptation, which occurs regardless of the direction of motion of the visual stimulus, has yet to be proposed. We identify a neuronal mechanism, namely frequency-dependent synaptic depression, which explains a number of the features of adaptation in mammalian motion-sensitive neurons and use it to model fly motion adaptation. While synaptic depression has been studied mainly in spiking cells, we use the same principles to develop a simple model for depression in a graded synapse. By incorporating this synaptic model into a neuronally based model for elementary motion detection, along with the implementation of a center-surround spatial band-pass filtering stage that mimics the interactions among a subset of visual neurons, we show that we can predict with remarkable success most of the qualitative features of adaptation observed in electrophysiological experiments. Our results support the idea that diverse species share common computational principles for processing visual motion and suggest that such principles could be neuronally implemented in very similar ways.

Neural correlates of illusory motion perception in Drosophila

When the contrast of an image flickers as it moves, humans perceive an illusory reversal in the direction of motion. This classic illusion, called reverse-phi motion, has been well-characterized using psychophysics, and several models have been proposed to account for its effects. Here, we show that Drosophila melanogaster also respond behaviorally to the reverse-phi illusion and that the illusion is present in dendritic calcium signals of motion-sensitive neurons in the fly lobula plate. These results closely match the predictions of the predominant model of fly motion detection. However, high flicker rates cause an inversion of the reverse-phi behavioral response that is also present in calcium signals of lobula plate tangential cell dendrites but not predicted by the model. The fly's behavioral and neural responses to the reverse-phi illusion reveal unexpected interactions between motion and flicker signals in the fly visual system and suggest that a similar correlation-based mechanism underlies visual motion detection across the animal kingdom.

Non-directional motion detectors can be used to mimic optic flow dependent behaviors

Insect navigational behaviors including obstacle avoidance, grazing landings, and visual odometry are dependent on the ability to estimate flight speed based only on visual cues. In honeybees, this visual estimate of speed is largely independent of both the direction of motion and the spatial frequency content of the image. Electrophysiological recordings from the motion-sensitive cells believed to underlie these behaviors have long supported spatio-temporally tuned correlation-type models of visual motion detection whose speed tuning changes as the spatial frequency of a stimulus is varied. The result is an apparent conflict between behavioral experiments and the electrophysiological and modeling data. In this article, we demonstrate that conventional correlation-type models are sufficient to reproduce some of the speed-dependent behaviors observed in honeybees when square wave gratings are used, contrary to the theoretical predictions. However, these models fail to match the behavioral observations for sinusoidal stimuli. Instead, we show that non-directional motion detectors, which underlie the correlation-based computation of directional motion, can be used to mimic these same behaviors even when narrowband gratings are used. The existence of such non-directional motion detectors is supported both anatomically and electrophysiologically, and they have been hypothesized to be critical in the Dipteran elementary motion detector (EMD) circuit.

Neural organization of first optic neuropils in the littoral crab Hemigrapsus oregonensis and the semiterrestrial species Chasmagnathus granulatus

Crustaceans are among the most extensively distributed arthropods, occupying many ecologies and manifesting a great variety of compound eye optics; but in comparison with insects, relatively little is known about the organization and neuronal morphologies of their underlying optic neuropils. Most studies, which have been limited to descriptions of the first neuropil--the lamina--suggest that different species have approximately comparable cell types. However, such studies have been limited with regard to the types of neurons they identify and most omit their topographic relationships. It is also uncertain whether similarities, such as they are, are independent of visual ecologies. The present account describes and compares the morphologies and dispositions of monopolar and other efferent neurons as well as the organization of tangential and smaller centrifugal neurons in two grapsoid crabs, one from the South Atlantic, the other from the North Pacific. Because these species occupy significantly disparate ecologies we ask whether this might be reflected in differences of cell arrangements within the most peripheral levels of the visual system. The present study identifies such differences with respect to the organization of centrifugal neurons to the lamina. We also identify in both species neurons in the lamina that have hitherto not been identified in crustaceans and we draw specific comparisons between the layered organization of the grapsoid lamina and layered laminas of insects.

Organization of local interneurons in optic glomeruli of the dipterous visual system and comparisons with the antennal lobes

The lateral protocerebrum of the fly's brain is composed of a system of optic glomeruli, the organization of which compares to that of antennal lobe glomeruli. Each optic glomerulus receives converging axon terminals from a unique ensemble of optic lobe output neurons. Glomeruli are interconnected by systems of spiking and nonspiking local interneurons that are morphologically similar to diffuse and polarized local interneurons in the antennal lobes. GABA-like immunoreactive processes richly supply optic glomeruli, which are also invaded by processes originating from the midbrain and subesophageal ganglia. These arrangements support the suggestion that circuits amongst optic glomeruli refine and elaborate visual information carried by optic lobe outputs, relaying data to long-axoned neurons that extend to other parts of the central nervous system including thoracic ganglia. The representation in optic glomeruli of other modalities suggests that gating of visual information by other sensory inputs, a phenomenon documented from the recordings of descending neurons, could occur before the descending neuron dendrites. The present results demonstrate that future studies must consider the roles of other senses in visual processing.

The spatial, temporal and contrast properties of expansion and rotation flight optomotor responses inDrosophila

Fruit flies respond to panoramic retinal patterns of visual expansion with robust steering maneuvers directed away from the focus of expansion to avoid collisions and maintain an upwind flight posture. Panoramic rotation elicits comparatively weak syndirectional steering maneuvers, which also maintain visual stability. Full-field optic flow patterns like expansion and rotation are elicited by distinct flight maneuvers such as body translation during straight flight or body rotation during hovering, respectively. Recent analyses suggest that under some experimental conditions the rotation optomotor response reflects the linear sum of different expansion response components. Are expansion and rotation-mediated visual stabilization responses part of a single optomotor response subserved by a neural circuit that is differentially stimulated by the two flow fields, or rather do the two behavioral responses reflect two distinct control systems? Guided by the principle that the properties of neural circuits are revealed in the behaviors they mediate, we systematically varied the spatial, temporal and contrast properties of expansion and rotation stimuli, and quantified the time course and amplitude of optomotor responses during tethered flight. Our results support the conclusion that expansion and rotation optomotor responses are indeed two separate reflexes, which draw from the same system of elementary motion detectors, but are likely mediated by separate pre-motor circuits having different spatial integration properties, low-pass characteristics and contrast sensitivity.

Visual system of calliphorid flies: organization of optic glomeruli and their lobula complex efferents

Reconstructions of silver-stained brains revealed 27 optic glomeruli that occupy a major volume of the lateral protocerebrum. Axons from different morphological types of columnar output neurons from the lobula complex sort out to specific glomeruli. Glomeruli are partially enwrapped by glial processes and are invaded by the dendrites and terminals of local interneurons that connect different glomeruli in a manner analogous to local interneurons in the antennal lobes. Each type of columnar neuron contributes to a palisade-like ensemble that extends across the whole or a circumscribed area of the retinotopic mosaic. A second class of outputs from the lobula comprises wide-field neurons, the dendrites of which interact with planar fields or column-like patches of retinotopic inputs from the medulla. These neurons also send their axons to optic glomeruli. Dye fills demonstrate that lobula complex neurons supplying glomeruli do not generally terminate directly on descending neurons. Local interneurons and projection neurons provide integrative circuitry within and among glomeruli. As exemplified by the anterior optic tubercle, optic glomeruli can also have elaborate internal architectures. The results are discussed with respect to the identification of motion- and orientation-selective neurons at the level of the lobula and lateral protocerebrum and with respect to the evolutionary implications raised by the existence of neural arrangements serving the compound eyes, which are organized like neuropils serving segmental ganglia equipped with appendages.

Visual system of calliphorid flies: motion- and orientation-sensitive visual interneurons supplying dorsal optic glomeruli

Intracellular recordings accompanied by dye fills were made from neurons associated with optic glomeruli in the lateral protocerebrum of the brain of the blowfly Phaenicia sericata. The present account describes the morphology of these cells and their electrophysiological responses to oriented bar motion. The most dorsal glomeruli are each supplied by retinotopic efferent neurons that have restricted dendritic fields in the lobula and lobula plate of the optic lobes. Each of these lobula complex cells represents a morphologically identified type of neuron arranged as an ensemble that subtends the entire monocular visual field. Of the four recorded and filled efferent types, three were broadly tuned to the orientation of bar stimuli. At the level of optic glomeruli a relay neuron extending centrally from optic foci and a local interneuron that arborizes among glomeruli showed narrow tuning to oriented bar motion. The present results are discussed with respect to the behavioral significance of oriented motion discrimination by flies and other insects, and with respect to neuroanatomical data demonstrating the organization of deep visual neuropils.

Diverse speed response properties of motion sensitive neurons in the fly's optic lobe

Speed and acceleration are fundamental components of visual motion that animals can use to interpret the world. Behavioral studies have established that insects discriminate speed largely independently of contrast and spatial frequency, and physiological recordings suggest that a subset of premotor descending neurons is in this sense speed-selective. Neural substrates and mechanisms of speed selectivity in insects, however, are unknown. Using blow flies Phaenicia sericata, intracellular recordings and dye-fills were obtained from medulla and lobula complex neurons which, though not necessarily speed-selective themselves, are positioned to participate in circuits that produce speed-selectivity in descending neurons. Stimulation with sinusoidally varied grating motion (0-200 degrees /s) provided a range of instantaneous velocities and accelerations. The resulting speed response profiles are indicative of four distinct speed ranges, supporting the hypothesis that the spatiotemporal tuning of mid-level neurons contains sufficient diversity to account for the emergence of speed selectivity at the descending neuron level. This type of mechanism has been proposed to explain speed discrimination in both insects and mammals, but has seemed less likely for insects due to possible constraints on small brains. Two additional recordings are suggestive of acceleration-selectivity, a potentially useful visual capability that is of uncertain functional significance for arthropods.

Neuronal algorithms that detect the temporal order of events

One of the basic operations in sensory processing is the computation of the temporal order of excitation of sensors. Motivated by the discrepancy between models and experiments at high signal contrast, we obtain families of algorithms by solutions of a general set of equations that define temporal order detection as an input-to-output relationship. Delays and nonlinear operations are the basis of all algorithms found, but different algorithmic structures exist when the operations are multiplications, OR gates, different types of AND-NOT logical gates, or concatenated AND-NOT gates. Among others, we obtain the Hassenstein-Reichardt model, a network using a multiplicative operation that has been proposed to explain fly optomotor behavior. We also find extensions of the Barlow-Levick model (based on an AND-NOT gate with delayed inhibition and nondelayed excitation as inputs), originally proposed to explain the bipolar cell response of the rabbit retina to motion stimuli. In the extended models, there are two more steps, another AND-NOT gate, and a subtraction or two subtractions that make the model responsive only to motion. In response to low-contrast inputs, the concatenated AND-NOT gates or the AND-NOT gate followed by a subtraction in these new models act as the multiplicative operation in the Hassenstein-Reichardt model. At high contrast, the new models behave like the Hassenstein-Reichardt model except that they are independent of contrast as observed experimentally.

Artificial neural network properties associated with wiring patterns in the visual projections of vertebrates and arthropods

We model the functioning of different wiring schemes in visual projections using artificial neural networks and so speculate on selective factors underlying taxonomic variation in neural architecture. We model the high connective overlap of vertebrates (where networks have a dense mesh of connections) and the less overlapping, more modular architecture of arthropods. We also consider natural variation in these basic wiring schemes. Generally, arthropod networks are as efficient or more efficient in functioning compared to vertebrate networks. They do not show the confusion effect (decreasing targeting accuracy with increasing input group size), and they train as well or better. Arthropod networks are, however, generally poorer at reconstructing novel inputs. The ability of vertebrate networks to effectively process novel stimuli could promote behavioral sophistication and drive the evolution of vertebrate wiring schemes. Vertebrate networks with less connective overlap have, surprisingly, similar or superior properties compared to those with high connective overlap. Thus, the partial connective overlap seen in real vertebrate visual projections may be an optimal, evolved solution. Arthropod networks with and without whole-cell neural connections within neural layers have similar properties. This indicates that neural connections mediated by offshoots of single cells (dendrites) may be fundamental to generating the confusion effect.

Comparison of octopamine-like immunoreactivity in the brains of the fruit fly and blow fly

A serum raised against conjugated octopamine reveals structurally comparable systems of perikarya and arborizations in protocerebral neuropils of two species of Diptera, Drosophila melanogaster and Phaenicia sericata; the latter is used extensively for electrophysiological studies of the optic lobes and their central projections. Clusters of cell bodies in the brain as well as midline perikarya provide octopamine-like immunoreactive processes to the optic lobes, circumscribed regions of the protocerebrum and the central complex, particularly the protocerebral bridge, fan-shaped body, and ellipsoid body. Ventral unpaired median somata provide immunoreactive processes within the subesophageal ganglion and tritocerebrum. Ascending neurites from these cells also supply the antennal lobe glomeruli, regions of the lateral protocerebrum, the mushroom body calyces, and the lobula complex. The mushroom body's gamma lobes contain immunoreactive processes that originate from processes that arborize in the protocerebrum. The present observations are discussed with respect to similarities and differences between two species of Diptera, one of which has neurons large enough for intracellular penetrations. The results are also discussed with respect to recent studies on octopamine-immunoreactive organization in honey bees and cockroaches and the suggested roles of octopamine in sensory processing, learning, and memory.

Sign-conserving amacrine neurons in the fly's external plexiform layer

Amacrine cells in the external plexiform layer of the fly's lamina have been intracellulary recorded and dye-filled for the first time. The recordings demonstrate that like the lamina's short photoreceptors R1-R6, type 1 lamina amacrine neurons exhibit nonspiking, "sign-conserving" sustained depolarizations in response to illumination. This contrasts with the sign-inverting responses that typify first-order retinotopic relay neurons: monopolar cells L1-L5 and the T1 efferent neuron. The contrast frequency tuning of amacrine neurons is similar to that of photoreceptors and large lamina monopolar cells. Initial observations indicate that lamina amacrine receptive fields are also photoreceptor-like, suggesting either that their inputs originate from a small number of neighboring visual sampling units (VSUs), or that locally generated potentials decay rapidly with displacement. Lamina amacrines also respond to motion, and in one recording these responses were selective for the orientation of moving edges. This functional organization corresponds to the anatomy of amacrine cells, in which postsynaptic inputs from several neighboring photoreceptor endings are linked by a network of very thin distal processes. In this way, each VSU can receive convergent inputs from a surround of amacrine processes. This arrangement is well suited for relaying responses to local intensity fluctuations from neighboring VSUs to a central VSU where amacrines are known to be presynaptic to the dendrites of the T1 efferent. The T1 terminal converges at a deeper level with that of the L2 monopolar cell relaying from the same optic cartridge. Thus, the localized spatial responses and receptor-like temporal response properties of amacrines are consistent with possible roles in lateral inhibition, motion processing, or orientation processing.

Velocity constancy and models for wide-field visual motion detection in insects

The tangential neurons in the lobula plate region of the flies are known to respond to visual motion across broad receptive fields in visual space. When intracellular recordings are made from tangential neurons while the intact animal is stimulated visually with moving natural imagery,we find that neural response depends upon speed of motion but is nearly invariant with respect to variations in natural scenery. We refer to this invariance as velocity constancy. It is remarkable because natural scenes, in spite of similarities in spatial structure, vary considerably in contrast, and contrast dependence is a feature of neurons in the early visual pathway as well as of most models for the elementary operations of visual motion detection. Thus, we expect that operations must be present in the processing pathway that reduce contrast dependence in order to approximate velocity constancy. We consider models for such operations, including spatial filtering, motion adaptation, saturating nonlinearities, and nonlinear spatial integration by the tangential neurons themselves, and evaluate their effects in simulations of a tangential neuron and precursor processing in response to animated natural imagery. We conclude that all such features reduce interscene variance in response, but that the model system does not approach velocity constancy as closely as the biological tangential cell.

Nondirectional motion may underlie insect behavioral dependence on image speed

Behavioral experiments suggest that insects make use of the apparent image speed on their compound eyes to navigate through obstacles, control flight speed, land smoothly, and measure the distance they have flown. However, the vast majority of electrophysiological recordings from motion-sensitive insect neurons show responses which are tuned in spatial and temporal frequency and are thus unable to unambiguously represent image speed. We suggest that this contradiction may be resolved at an early stage of visual motion processing using nondirectional motion sensors that respond proportionally to image speed until their peak response. We describe and characterize a computational model of these sensors and propose a model by which a spatial collation of such sensors could be used to generate speed-dependent behavior.