Neural action fields for optic flow based navigation: a simulation study of the fly lobula plate network

Long-timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied Drosophila melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such 'anti-directional turning' is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Long timescale anti-directional rotation in Drosophila optomotor behavior

Locomotor movements cause visual images to be displaced across the eye, a retinal slip that is counteracted by stabilizing reflexes in many animals. In insects, optomotor turning causes the animal to turn in the direction of rotating visual stimuli, thereby reducing retinal slip and stabilizing trajectories through the world. This behavior has formed the basis for extensive dissections of motion vision. Here, we report that under certain stimulus conditions, two Drosophila species, including the widely studied D. melanogaster, can suppress and even reverse the optomotor turning response over several seconds. Such "anti-directional turning" is most strongly evoked by long-lasting, high-contrast, slow-moving visual stimuli that are distinct from those that promote syn-directional optomotor turning. Anti-directional turning, like the syn-directional optomotor response, requires the local motion detecting neurons T4 and T5. A subset of lobula plate tangential cells, CH cells, show involvement in these responses. Imaging from a variety of direction-selective cells in the lobula plate shows no evidence of dynamics that match the behavior, suggesting that the observed inversion in turning direction emerges downstream of the lobula plate. Further, anti-directional turning declines with age and exposure to light. These results show that Drosophila optomotor turning behaviors contain rich, stimulus-dependent dynamics that are inconsistent with simple reflexive stabilization responses.

Contrast independent biologically inspired translational optic flow estimation

The visual systems of insects are relatively simple compared to humans. However, they enable navigation through complex environments where insects perform exceptional levels of obstacle avoidance. Biology uses two separable modes of optic flow to achieve this: rapid gaze fixation (rotational motion known as saccades); and the inter-saccadic translational motion. While the fundamental process of insect optic flow has been known since the 1950's, so too has its dependence on contrast. The surrounding visual pathways used to overcome environmental dependencies are less well known. Previous work has shown promise for low-speed rotational motion estimation, but a gap remained in the estimation of translational motion, in particular the estimation of the time to impact. To consistently estimate the time to impact during inter-saccadic translatory motion, the fundamental limitation of contrast dependence must be overcome. By adapting an elaborated rotational velocity estimator from literature to work for translational motion, this paper proposes a novel algorithm for overcoming the contrast dependence of time to impact estimation using nonlinear spatio-temporal feedforward filtering. By applying bioinspired processes, approximately 15 points per decade of statistical discrimination were achieved when estimating the time to impact to a target across 360 background, distance, and velocity combinations: a 17-fold increase over the fundamental process. These results show the contrast dependence of time to impact estimation can be overcome in a biologically plausible manner. This, combined with previous results for low-speed rotational motion estimation, allows for contrast invariant computational models designed on the principles found in the biological visual system, paving the way for future visually guided systems.

Maximally efficient prediction in the early fly visual system may support evasive flight maneuvers

The visual system must make predictions to compensate for inherent delays in its processing. Yet little is known, mechanistically, about how prediction aids natural behaviors. Here, we show that despite a 20-30ms intrinsic processing delay, the vertical motion sensitive (VS) network of the blowfly achieves maximally efficient prediction. This prediction enables the fly to fine-tune its complex, yet brief, evasive flight maneuvers according to its initial ego-rotation at the time of detection of the visual threat. Combining a rich database of behavioral recordings with detailed compartmental modeling of the VS network, we further show that the VS network has axonal gap junctions that are critical for optimal prediction. During evasive maneuvers, a VS subpopulation that directly innervates the neck motor center can convey predictive information about the fly’s future ego-rotation, potentially crucial for ongoing flight control. These results suggest a novel sensory-motor pathway that links sensory prediction to behavior.

Survival-critical behaviors shape neural circuits to translate sensory information into strikingly fast predictions, e.g. in escaping from a predator faster than the system’s processing delay. We show that the fly visual system implements fast and accurate prediction of its visual experience. This provides crucial information for directing fast evasive maneuvers that unfold over just 40ms. Our work shows how this fast prediction is implemented, mechanistically, and suggests the existence of a novel sensory-motor pathway from the fly visual system to a wing steering motor neuron. Echoing and amplifying previous work in the retina, our work hypothesizes that the efficient encoding of predictive information is a universal design principle supporting fast, natural behaviors.

A biohybrid fly-robot interface system that performs active collision avoidance

We have designed a bio-hybrid fly-robot interface (FRI) to study sensorimotor control in insects. The FRI consists of a miniaturized recording platform mounted on a two-wheeled robot and is controlled by the neuronal spiking activity of an identified visual interneuron, the blowfly H1-cell. For a given turning radius of the robot, we found a proportional relationship between the spike rate of the H1-cell and the relative distance of the FRI from the patterned wall of an experimental arena. Under closed-loop conditions during oscillatory forward movements biased towards the wall, collision avoidance manoeuvres were triggered whenever the H1-cell spike rate exceeded a certain threshold value. We also investigated the FRI behaviour in corners of the arena. The ultimate goal is to enable autonomous and energy-efficient manoeuvrings of the FRI within arbitrary visual environments.

Bi-directional Control of Walking Behavior by Horizontal Optic Flow Sensors

Moving animals experience constant sensory feedback, such as panoramic image shifts on the retina, termed optic flow. Underlying neuronal signals are thought to be important for exploratory behavior by signaling unintended course deviations and by providing spatial information about the environment [1, 2]. Particularly in insects, the encoding of self-motion-related optic flow is well understood [1-5]. However, a gap remains in understanding how the associated neuronal activity controls locomotor trajectories. In flies, visual projection neurons belonging to two groups encode panoramic horizontal motion: horizontal system (HS) cells respond with depolarization to front-to-back motion and hyperpolarization to the opposite direction [6, 7], and other neurons have the mirror-symmetrical response profile [6, 8, 9]. With primarily monocular sensitivity, the neurons' responses are ambiguous for different rotational and translational self-movement components. Such ambiguities can be greatly reduced by combining signals from both eyes [10-12] to determine turning and movement speed [13-16]. Here, we explore the underlying functional logic by optogenetic HS cell manipulation in tethered walking Drosophila. We show that de- and hyperpolarization evoke opposite turning behavior, indicating that both direction-selective signals are transmitted to descending pathways for course control. Further experiments reveal a negative effect of bilaterally symmetric de- and hyperpolarization on walking velocity. Our results are therefore consistent with a functional architecture in which the HS cells' membrane potential influences walking behavior bi-directionally via two decelerating pathways.

Efficient encoding of motion is mediated by gap junctions in the fly visual system

Understanding the computational implications of specific synaptic connectivity patterns is a fundamental goal in neuroscience. In particular, the computational role of ubiquitous electrical synapses operating via gap junctions remains elusive. In the fly visual system, the cells in the vertical-system network, which play a key role in visual processing, primarily connect to each other via axonal gap junctions. This network therefore provides a unique opportunity to explore the functional role of gap junctions in sensory information processing. Our information theoretical analysis of a realistic VS network model shows that within 10 ms following the onset of the visual input, the presence of axonal gap junctions enables the VS system to efficiently encode the axis of rotation, θ, of the fly’s ego motion. This encoding efficiency, measured in bits, is near-optimal with respect to the physical limits of performance determined by the statistical structure of the visual input itself. The VS network is known to be connected to downstream pathways via a subset of triplets of the vertical system cells; we found that because of the axonal gap junctions, the efficiency of this subpopulation in encoding θ is superior to that of the whole vertical system network and is robust to a wide range of signal to noise ratios. We further demonstrate that this efficient encoding of motion by this subpopulation is necessary for the fly's visually guided behavior, such as banked turns in evasive maneuvers. Because gap junctions are formed among the axons of the vertical system cells, they only impact the system’s readout, while maintaining the dendritic input intact, suggesting that the computational principles implemented by neural circuitries may be much richer than previously appreciated based on point neuron models. Our study provides new insights as to how specific network connectivity leads to efficient encoding of sensory stimuli.

Understanding sensory stimuli from the environment and deciding how best to respond to it behaviorally is essential for survival. What makes organisms efficient in encoding these sensory stimuli? This study provides a novel view on this unresolved issue using the visual system of the fly. We show that a specific synaptic connectivity manifested via gap junctions (GJs) among axons in the Vertical System (VS) network leads to particularly high encoding efficiency of the axis of rotation of the fly’s ego motion. Due to these GJs, triplets of VS neurons (the VS5-6-7 triplet), which connect to a downstream motor system, encode motion stimuli at an efficiency close to the physical limit; this efficient encoding is necessary for evasive maneuvers that are critical for the fly to escape predators. We then suggest why GJs in the VS network enable such high encoding efficiency.

Insect-Inspired Self-Motion Estimation with Dense Flow Fields--An Adaptive Matched Filter Approach

The control of self-motion is a basic, but complex task for both technical and biological systems. Various algorithms have been proposed that allow the estimation of self-motion from the optic flow on the eyes. We show that two apparently very different approaches to solve this task, one technically and one biologically inspired, can be transformed into each other under certain conditions. One estimator of self-motion is based on a matched filter approach; it has been developed to describe the function of motion sensitive cells in the fly brain. The other estimator, the Koenderink and van Doorn (KvD) algorithm, was derived analytically with a technical background. If the distances to the objects in the environment can be assumed to be known, the two estimators are linear and equivalent, but are expressed in different mathematical forms. However, for most situations it is unrealistic to assume that the distances are known. Therefore, the depth structure of the environment needs to be determined in parallel to the self-motion parameters and leads to a non-linear problem. It is shown that the standard least mean square approach that is used by the KvD algorithm leads to a biased estimator. We derive a modification of this algorithm in order to remove the bias and demonstrate its improved performance by means of numerical simulations. For self-motion estimation it is beneficial to have a spherical visual field, similar to many flying insects. We show that in this case the representation of the depth structure of the environment derived from the optic flow can be simplified. Based on this result, we develop an adaptive matched filter approach for systems with a nearly spherical visual field. Then only eight parameters about the environment have to be memorized and updated during self-motion.

A network of visual motion-sensitive neurons for computing object position in an arthropod

Highly active insects and crabs depend on visual motion information for detecting and tracking mates, prey, or predators, for which they require directional control systems containing internal maps of visual space. A neural map formed by large, motion-sensitive neurons implicated in processing panoramic flow is known to exist in an optic ganglion of the fly. However, an equivalent map for processing spatial positions of single objects has not been hitherto identified in any arthropod. Crabs can escape directly away from a visual threat wherever the stimulus is located in the 360° field of view. When tested in a walking simulator, the crab Neohelice granulata immediately adjusts its running direction after changes in the position of the visual danger stimulus smaller than 1°. Combining mass and single-cell staining with in vivo intracellular recording, we show that a particular class of motion-sensitive neurons of the crab's lobula that project to the midbrain, the monostratified lobula giants type 1 (MLG1), form a system of 16 retinotopically organized elements that map the 360° azimuthal space. The preference of these neurons for horizontally moving objects conforms the visual ecology of the crab's mudflat world. With a mean receptive field of 118°, MLG1s have a large superposition among neighboring elements. Our results suggest that the MLG1 system conveys information on object position as a population vector. Such computational code can enable the accurate directional control observed in the visually guided behaviors of crabs.

Complementary motion tuning in frontal nerve motor neurons of the blowfly

Flies actively turn their head during flight to stabilize their gaze and reduce motion blur. This optomotor response is triggered by wide-field motion indicating a deviation from a desired flight path. We focus on the neuronal circuit that underlies this behavior in the blowfly Calliphora, studying the integration of optic flow in neck motor neurons that innervate muscles controlling head rotations. Frontal nerve motor neurons (FNMNs) have been described anatomically and recorded from extracellularly before. Here, we assign for the first time to five anatomical classes of FNMNs their visual motion tuning. We measured their responses to optic flow, as produced by rotations around particular body axes, recording intracellularly from single axons. Simultaneous injection of Neurobiotin allowed for the anatomical characterization of the recorded cells and revealed coupling patterns with neighboring neurons. The five FNMN classes can be divided into two groups that complement each other, regarding their preferred axes of rotation. The tuning matches the pulling planes of their innervated neck muscles, serving to rotate the head around its longitudinal axis. Anatomical and physiological findings demonstrate a synaptic connection between one FNMN and a well-described descending neuron, elucidating one important step from visual motion integration to neck motor output.

Near-optimal decoding of transient stimuli from coupled neuronal subpopulations

Coupling between sensory neurons impacts their tuning properties and correlations in their responses. How such coupling affects sensory representations and ultimately behavior remains unclear. We investigated the role of neuronal coupling during visual processing using a realistic biophysical model of the vertical system (VS) cell network in the blow fly. These neurons are thought to encode the horizontal rotation axis during rapid free-flight maneuvers. Experimental findings suggest that neurons of the VS are strongly electrically coupled, and that several downstream neurons driving motor responses to ego-rotation receive inputs primarily from a small subset of VS cells. These downstream neurons must decode information about the axis of rotation from a partial readout of the VS population response. To investigate the role of coupling, we simulated the VS response to a variety of rotating visual scenes and computed optimal Bayesian estimates from the relevant subset of VS cells. Our analysis shows that coupling leads to near-optimal estimates from a subpopulation readout. In contrast, coupling between VS cells has no impact on the quality of encoding in the response of the full population. We conclude that coupling at one level of the fly visual system allows for near-optimal decoding from partial information at the subsequent, premotor level. Thus, electrical coupling may provide a general mechanism to achieve near-optimal information transfer from neuronal subpopulations across organisms and modalities.

Subcellular mapping of dendritic activity in optic flow processing neurons

Dendritic integration is a fundamental element of neuronal information processing. So far, few studies have provided a detailed spatial picture of this process, describing the properties of local dendritic activity and its subcellular organization. Here, we used 2-photon calcium imaging in optic flow processing neurons of the fly Calliphora vicina to determine the preferred location and direction of local motion cues for small branchlets throughout the entire dendrite. We found a pronounced retinotopic mapping on both the subcellular and the cell population level. In addition, dendritic branchlets residing in different layers of the neuropil were tuned to distinct directions of motion. Summing the local receptive fields of all dendritic branchlets reproduced the characteristic properties of these neurons' axonal output receptive fields. Our results corroborate the notion that the dendritic morphology of vertical system cells allows them to selectively collect local motion inputs with particular directional preferences from a spatially organized input repertoire, thus forming filters that match global patterns of optic flow. Furthermore, we suggest that the facet arrangement across the fly's eye shapes the subcellular direction tuning to local motion stimuli. These data illustrate a highly structured circuit organization as an efficient way to hard-wire a complex sensory task.

Cooperative integration and representation underlying bilateral network of fly motion-sensitive neurons

How is binocular motion information integrated in the bilateral network of wide-field motion-sensitive neurons, called lobula plate tangential cells (LPTCs), in the visual system of flies? It is possible to construct an accurate model of this network because a complete picture of synaptic interactions has been experimentally identified. We investigated the cooperative behavior of the network of horizontal LPTCs underlying the integration of binocular motion information and the information representation in the bilateral LPTC network through numerical simulations on the network model. First, we qualitatively reproduced rotational motion-sensitive response of the H2 cell previously reported in vivo experiments and ascertained that it could be accounted for by the cooperative behavior of the bilateral network mainly via interhemispheric electrical coupling. We demonstrated that the response properties of single H1 and Hu cells, unlike H2 cells, are not influenced by motion stimuli in the contralateral visual hemi-field, but that the correlations between these cell activities are enhanced by the rotational motion stimulus. We next examined the whole population activity by performing principal component analysis (PCA) on the population activities of simulated LPTCs. We showed that the two orthogonal patterns of correlated population activities given by the first two principal components represent the rotational and translational motions, respectively, and similar to the H2 cell, rotational motion produces a stronger response in the network than does translational motion. Furthermore, we found that these population-coding properties are strongly influenced by the interhemispheric electrical coupling. Finally, to test the generality of our conclusions, we used a more simplified model and verified that the numerical results are not specific to the network model we constructed.

Spatial vision in insects is facilitated by shaping the dynamics of visual input through behavioral action

Insects such as flies or bees, with their miniature brains, are able to control highly aerobatic flight maneuvres and to solve spatial vision tasks, such as avoiding collisions with obstacles, landing on objects, or even localizing a previously learnt inconspicuous goal on the basis of environmental cues. With regard to solving such spatial tasks, these insects still outperform man-made autonomous flying systems. To accomplish their extraordinary performance, flies and bees have been shown by their characteristic behavioral actions to actively shape the dynamics of the image flow on their eyes ("optic flow"). The neural processing of information about the spatial layout of the environment is greatly facilitated by segregating the rotational from the translational optic flow component through a saccadic flight and gaze strategy. This active vision strategy thus enables the nervous system to solve apparently complex spatial vision tasks in a particularly efficient and parsimonious way. The key idea of this review is that biological agents, such as flies or bees, acquire at least part of their strength as autonomous systems through active interactions with their environment and not by simply processing passively gained information about the world. These agent-environment interactions lead to adaptive behavior in surroundings of a wide range of complexity. Animals with even tiny brains, such as insects, are capable of performing extraordinarily well in their behavioral contexts by making optimal use of the closed action-perception loop. Model simulations and robotic implementations show that the smart biological mechanisms of motion computation and visually-guided flight control might be helpful to find technical solutions, for example, when designing micro air vehicles carrying a miniaturized, low-weight on-board processor.

Temporal and spatial adaptation of transient responses to local features

Interpreting visual motion within the natural environment is a challenging task, particularly considering that natural scenes vary enormously in brightness, contrast and spatial structure. The performance of current models for the detection of self-generated optic flow depends critically on these very parameters, but despite this, animals manage to successfully navigate within a broad range of scenes. Within global scenes local areas with more salient features are common. Recent work has highlighted the influence that local, salient features have on the encoding of optic flow, but it has been difficult to quantify how local transient responses affect responses to subsequent features and thus contribute to the global neural response. To investigate this in more detail we used experimenter-designed stimuli and recorded intracellularly from motion-sensitive neurons. We limited the stimulus to a small vertically elongated strip, to investigate local and global neural responses to pairs of local “doublet” features that were designed to interact with each other in the temporal and spatial domain. We show that the passage of a high-contrast doublet feature produces a complex transient response from local motion detectors consistent with predictions of a simple computational model. In the neuron, the passage of a high-contrast feature induces a local reduction in responses to subsequent low-contrast features. However, this neural contrast gain reduction appears to be recruited only when features stretch vertically (i.e., orthogonal to the direction of motion) across at least several aligned neighboring ommatidia. Horizontal displacement of the components of elongated features abolishes the local adaptation effect. It is thus likely that features in natural scenes with vertically aligned edges, such as tree trunks, recruit the greatest amount of response suppression. This property could emphasize the local responses to such features vs. those in nearby texture within the scene.

Insect-inspired high-speed motion vision system for robot control

The mechanism for motion detection in a fly's vision system, known as the Reichardt correlator, suffers from a main shortcoming as a velocity estimator: low accuracy. To enable accurate velocity estimation, responses of the Reichardt correlator to image sequences are analyzed in this paper. An elaborated model with additional preprocessing modules is proposed. The relative error of velocity estimation is significantly reduced by establishing a real-time response-velocity lookup table based on the power spectrum analysis of the input signal. By exploiting the improved velocity estimation accuracy and the simple structure of the Reichardt correlator, a high-speed vision system of 1 kHz is designed and applied for robot yaw-angle control in real-time experiments. The experimental results demonstrate the potential and feasibility of applying insect-inspired motion detection to robot control.

Integration of binocular optic flow in cervical neck motor neurons of the fly

Global visual motion elicits an optomotor response of the eye that stabilizes the visual input on the retina. Here, we analyzed the neck motor system of the blowfly to understand binocular integration of visual motion information underlying a head optomotor response. We identified and characterized two cervical nerve motor neurons (called CNMN6 and CNMN7) tuned precisely to an optic flow corresponding to pitch movements of the head. By means of double recordings and dye coupling, we determined that these neurons are connected ipsilaterally to two vertical system cells (VS2 and VS3), and contralaterally to one horizontal system cell (HSS). In addition, CNMN7 turned out to be connected to the ipsilateral CNMN6 and to its contralateral counterpart. To analyze a potential function of this circuit, we performed behavioral experiments and found that the optomotor pitch response of the fly head was only observable when both eyes were intact. Thus, this neural circuit performs two visuomotor transformations: first, by integrating binocular visual information it enhances the tuning to the optic flow resulting from pitch movements of the head, and second it could assure an even head declination by coordinating the activity of the CNMN7 neurons on both sides.

Disentangling the functional consequences of the connectivity between optic-flow processing neurons

Typically, neurons in sensory areas are highly interconnected. Coupling two neurons can synchronize their activity and affect a variety of single-cell properties, such as their stimulus tuning, firing rate or gain. All of these factors must be considered to understand how two neurons should be coupled to optimally process stimuli. We quantified the functional effect of an interaction between two optic-flow processing neurons (Vi and H1) in the fly (Lucilia sericata). Using a generative model, we estimated a uni-directional coupling from H1 to Vi. Especially at a low signal-to-noise ratio (SNR), the coupling strongly improved the information about optic-flow in Vi. We identified two constraints confining the strength of the interaction. First, for weak couplings, Vi benefited from inputs by H1 without a concomitant shift of its stimulus tuning. Second, at both low and high SNR, the coupling strength lay in a range in which the information carried by single spikes is optimal.

An olfactory circuit increases the fidelity of visual behavior

Multimodal integration allows neural circuits to be activated in a behaviorally context-specific manner. In the case of odor plume tracking by Drosophila, an attractive odorant increases the influence of yaw-optic flow on steering behavior in flight, which enhances visual stability reflexes, resulting in straighter flight trajectories within an odor plume. However, it is not well understood whether context-specific changes in optomotor behavior are the result of an increased sensitivity to motion inputs (e.g., through increased visual attention) or direct scaling of motor outputs (i.e., increased steering gain). We address this question by examining the optomotor behavior of Drosophila melanogaster in a tethered flight assay and demonstrate that whereas olfactory cues decrease the gain of the optomotor response to sideslip optic flow, they concomitantly increase the gain of the yaw optomotor response by enhancing the animal's ability to follow transient visual perturbations. Furthermore, ablating the mushroom bodies (MBs) of the fly brain via larval hydroxyurea (HU) treatment results in a loss of olfaction-dependent increase in yaw optomotor fidelity. By expressing either tetanus toxin light chain or diphtheria toxin in gal4-defined neural circuits, we were able to replicate the loss of function observed in the HU treatment within the lines expressing broadly in the mushroom bodies, but not within specific mushroom body lobes. Finally, we were able to genetically separate the yaw responses and sideslip responses in our behavioral assay. Together, our results implicate the MBs in a fast-acting, memory-independent olfactory modification of a visual reflex that is critical for flight control.