Responses of blowfly motion-sensitive neurons to reconstructed optic flow along outdoor flight paths

The Antarium: A Reconstructed Visual Reality Device for Ant Navigation Research

We constructed a large projection device (the Antarium) with 20,000 UV-Blue-Green LEDs that allows us to present tethered ants with views of their natural foraging environment. The ants walk on an air-cushioned trackball, their movements are registered and can be fed back to the visual panorama. Views are generated in a 3D model of the ants’ environment so that they experience the changing visual world in the same way as they do when foraging naturally. The Antarium is a biscribed pentakis dodecahedron with 55 facets of identical isosceles triangles. The length of the base of the triangles is 368 mm resulting in a device that is roughly 1 m in diameter. Each triangle contains 361 blue/green LEDs and nine UV LEDs. The 55 triangles of the Antarium have 19,855 Green and Blue pixels and 495 UV pixels, covering 360° azimuth and elevation from −50° below the horizon to +90° above the horizon. The angular resolution is 1.5° for Green and Blue LEDs and 6.7° for UV LEDs, offering 65,536 intensity levels at a flicker frequency of more than 9,000 Hz and a framerate of 190 fps. Also, the direction and degree of polarisation of the UV LEDs can be adjusted through polarisers mounted on the axles of rotary actuators. We build 3D models of the natural foraging environment of ants using purely camera-based methods. We reconstruct panoramic scenes at any point within these models, by projecting panoramic images onto six virtual cameras which capture a cube-map of images to be projected by the LEDs of the Antarium. The Antarium is a unique instrument to investigate visual navigation in ants. In an open loop, it allows us to provide ants with familiar and unfamiliar views, with completely featureless visual scenes, or with scenes that are altered in spatial or spectral composition. In closed-loop, we can study the behavior of ants that are virtually displaced within their natural foraging environment. In the future, the Antarium can also be used to investigate the dynamics of navigational guidance and the neurophysiological basis of ant navigation in natural visual environments.

A biohybrid fly-robot interface system that performs active collision avoidance

We have designed a bio-hybrid fly-robot interface (FRI) to study sensorimotor control in insects. The FRI consists of a miniaturized recording platform mounted on a two-wheeled robot and is controlled by the neuronal spiking activity of an identified visual interneuron, the blowfly H1-cell. For a given turning radius of the robot, we found a proportional relationship between the spike rate of the H1-cell and the relative distance of the FRI from the patterned wall of an experimental arena. Under closed-loop conditions during oscillatory forward movements biased towards the wall, collision avoidance manoeuvres were triggered whenever the H1-cell spike rate exceeded a certain threshold value. We also investigated the FRI behaviour in corners of the arena. The ultimate goal is to enable autonomous and energy-efficient manoeuvrings of the FRI within arbitrary visual environments.

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.

Image statistics and their processing in insect vision

Natural scenes may appear random, but are not only constrained in space and time, but also show strong spatial and temporal correlations. Spatial constraints and correlations can be described by quantifying image statistics, which include intuitive measures such as contrast, color and luminance, but also parameters that need some type of transformation of the image. In this review we will discuss some common tools used to quantify spatial and temporal parameters of naturalistic visual input, and how these tools have been used to inform us about visual processing in insects. In particular, we will review findings that would not have been possible using conventional, experimenter defined stimuli.

Spatio-temporal dynamics of impulse responses to figure motion in optic flow neurons

White noise techniques have been used widely to investigate sensory systems in both vertebrates and invertebrates. White noise stimuli are powerful in their ability to rapidly generate data that help the experimenter decipher the spatio-temporal dynamics of neural and behavioral responses. One type of white noise stimuli, maximal length shift register sequences (m-sequences), have recently become particularly popular for extracting response kernels in insect motion vision. We here use such m-sequences to extract the impulse responses to figure motion in hoverfly lobula plate tangential cells (LPTCs). Figure motion is behaviorally important and many visually guided animals orient towards salient features in the surround. We show that LPTCs respond robustly to figure motion in the receptive field. The impulse response is scaled down in amplitude when the figure size is reduced, but its time course remains unaltered. However, a low contrast stimulus generates a slower response with a significantly longer time-to-peak and half-width. Impulse responses in females have a slower time-to-peak than males, but are otherwise similar. Finally we show that the shapes of the impulse response to a figure and a widefield stimulus are very similar, suggesting that the figure response could be coded by the same input as the widefield response.

Encoding of yaw in the presence of distractor motion: studies in a fly motion sensitive neuron

Motion estimation is crucial for aerial animals such as the fly, which perform fast and complex maneuvers while flying through a 3-D environment. Motion-sensitive neurons in the lobula plate, a part of the visual brain, of the fly have been studied extensively for their specialized role in motion encoding. However, the visual stimuli used in such studies are typically highly simplified, often move in restricted ways, and do not represent the complexities of optic flow generated during actual flight. Here, we use combined rotations about different axes to study how H1, a wide-field motion-sensitive neuron, encodes preferred yaw motion in the presence of stimuli not aligned with its preferred direction. Our approach is an extension of "white noise" methods, providing a framework that is readily adaptable to quantitative studies into the coding of mixed dynamic stimuli in other systems. We find that the presence of a roll or pitch ("distractor") stimulus reduces information transmitted by H1 about yaw, with the amount of this reduction depending on the variance of the distractor. Spike generation is influenced by features of both yaw and the distractor, where the degree of influence is determined by their relative strengths. Certain distractor features may induce bidirectional responses, which are indicative of an imbalance between global excitation and inhibition resulting from complex optic flow. Further, the response is shaped by the dynamics of the combined stimulus. Our results provide intuition for plausible strategies involved in efficient coding of preferred motion from complex stimuli having multiple motion components.

Influence of environmental information in natural scenes and the effects of motion adaptation on a fly motion-sensitive neuron during simulated flight

Gaining information about the spatial layout of natural scenes is a challenging task that flies need to solve, especially when moving at high velocities. A group of motion sensitive cells in the lobula plate of flies is supposed to represent information about self-motion as well as the environment. Relevant environmental features might be the nearness of structures, influencing retinal velocity during translational self-motion, and the brightness contrast. We recorded the responses of the H1 cell, an individually identifiable lobula plate tangential cell, during stimulation with image sequences, simulating translational motion through natural sceneries with a variety of differing depth structures. A correlation was found between the average nearness of environmental structures within large parts of the cell's receptive field and its response across a variety of scenes, but no correlation was found between the brightness contrast of the stimuli and the cell response. As a consequence of motion adaptation resulting from repeated translation through the environment, the time-dependent response modulations induced by the spatial structure of the environment were increased relatively to the background activity of the cell. These results support the hypothesis that some lobula plate tangential cells do not only serve as sensors of self-motion, but also as a part of a neural system that processes information about the spatial layout of natural scenes.

Motion as a source of environmental information: a fresh view on biological motion computation by insect brains

Despite their miniature brains insects, such as flies, bees and wasps, are able to navigate by highly erobatic flight maneuvers in cluttered environments. They rely on spatial information that is contained in the retinal motion patterns induced on the eyes while moving around ("optic flow") to accomplish their extraordinary performance. Thereby, they employ an active flight and gaze strategy that separates rapid saccade-like turns from translatory flight phases where the gaze direction is kept largely constant. This behavioral strategy facilitates the processing of environmental information, because information about the distance of the animal to objects in the environment is only contained in the optic flow generated by translatory motion. However, motion detectors as are widespread in biological systems do not represent veridically the velocity of the optic flow vectors, but also reflect textural information about the environment. This characteristic has often been regarded as a limitation of a biological motion detection mechanism. In contrast, we conclude from analyses challenging insect movement detectors with image flow as generated during translatory locomotion through cluttered natural environments that this mechanism represents the contours of nearby objects. Contrast borders are a main carrier of functionally relevant object information in artificial and natural sceneries. The motion detection system thus segregates in a computationally parsimonious way the environment into behaviorally relevant nearby objects and-in many behavioral contexts-less relevant distant structures. Hence, by making use of an active flight and gaze strategy, insects are capable of performing extraordinarily well even with a computationally simple motion detection mechanism.

Temporal statistics of natural image sequences generated by movements with insect flight characteristics

Many flying insects, such as flies, wasps and bees, pursue a saccadic flight and gaze strategy. This behavioral strategy is thought to separate the translational and rotational components of self-motion and, thereby, to reduce the computational efforts to extract information about the environment from the retinal image flow. Because of the distinguishing dynamic features of this active flight and gaze strategy of insects, the present study analyzes systematically the spatiotemporal statistics of image sequences generated during saccades and intersaccadic intervals in cluttered natural environments. We show that, in general, rotational movements with saccade-like dynamics elicit fluctuations and overall changes in brightness, contrast and spatial frequency of up to two orders of magnitude larger than translational movements at velocities that are characteristic of insects. Distinct changes in image parameters during translations are only caused by nearby objects. Image analysis based on larger patches in the visual field reveals smaller fluctuations in brightness and spatial frequency composition compared to small patches. The temporal structure and extent of these changes in image parameters define the temporal constraints imposed on signal processing performed by the insect visual system under behavioral conditions in natural environments.

Visual motion-sensitive neurons in the bumblebee brain convey information about landmarks during a navigational task

Bees use visual memories to find the spatial location of previously learnt food sites. Characteristic learning flights help acquiring these memories at newly discovered foraging locations where landmarks—salient objects in the vicinity of the goal location—can play an important role in guiding the animal's homing behavior. Although behavioral experiments have shown that bees can use a variety of visual cues to distinguish objects as landmarks, the question of how landmark features are encoded by the visual system is still open. Recently, it could be shown that motion cues are sufficient to allow bees localizing their goal using landmarks that can hardly be discriminated from the background texture. Here, we tested the hypothesis that motion sensitive neurons in the bee's visual pathway provide information about such landmarks during a learning flight and might, thus, play a role for goal localization. We tracked learning flights of free-flying bumblebees (Bombus terrestris) in an arena with distinct visual landmarks, reconstructed the visual input during these flights, and replayed ego-perspective movies to tethered bumblebees while recording the activity of direction-selective wide-field neurons in their optic lobe. By comparing neuronal responses during a typical learning flight and targeted modifications of landmark properties in this movie we demonstrate that these objects are indeed represented in the bee's visual motion pathway. We find that object-induced responses vary little with object texture, which is in agreement with behavioral evidence. These neurons thus convey information about landmark properties that are useful for view-based homing.

Depth information in natural environments derived from optic flow by insect motion detection system: a model analysis

Knowing the depth structure of the environment is crucial for moving animals in many behavioral contexts, such as collision avoidance, targeting objects, or spatial navigation. An important source of depth information is motion parallax. This powerful cue is generated on the eyes during translatory self-motion with the retinal images of nearby objects moving faster than those of distant ones. To investigate how the visual motion pathway represents motion-based depth information we analyzed its responses to image sequences recorded in natural cluttered environments with a wide range of depth structures. The analysis was done on the basis of an experimentally validated model of the visual motion pathway of insects, with its core elements being correlation-type elementary motion detectors (EMDs). It is the key result of our analysis that the absolute EMD responses, i.e., the motion energy profile, represent the contrast-weighted nearness of environmental structures during translatory self-motion at a roughly constant velocity. In other words, the output of the EMD array highlights contours of nearby objects. This conclusion is largely independent of the scale over which EMDs are spatially pooled and was corroborated by scrutinizing the motion energy profile after eliminating the depth structure from the natural image sequences. Hence, the well-established dependence of correlation-type EMDs on both velocity and textural properties of motion stimuli appears to be advantageous for representing behaviorally relevant information about the environment in a computationally parsimonious way.

Texture-defined objects influence responses of blowfly motion-sensitive neurons under natural dynamical conditions

The responses of visual interneurons of flies involved in the processing of motion information do not only depend on the velocity, but also on other stimulus parameters, such as the contrast and the spatial frequency content of the stimulus pattern. These dependencies have been known for long, but it is still an open question how they affect the neurons' performance in extracting information about the structure of the environment under the specific dynamical conditions of natural flight. Free-flight of blowflies is characterized by sequences of phases of translational movements lasting for just 30-100 ms interspersed with even shorter and extremely rapid saccade-like rotational shifts in flight and gaze direction. Previous studies already analyzed how nearby objects, leading to relative motion on the retina with respect to a more distant background, influenced the response of a class of fly motion sensitive visual interneurons, the horizontal system (HS) cells. In the present study, we focused on objects that differed from their background by discontinuities either in their brightness contrast or in their spatial frequency content. We found strong object-induced effects on the membrane potential even during the short intersaccadic intervals, if the background contrast was small and the object contrast sufficiently high. The object evoked similar response increments provided that it contained higher spatial frequencies than the background, but not under reversed conditions. This asymmetry in the response behavior is partly a consequence of the depolarization level induced by the background. Thus, our results suggest that, under the specific dynamical conditions of natural flight, i.e., on a very short timescale, the responses of HS cells represent object information depending on the polarity of the difference between object and background contrast and spatial frequency content.

Optogenetic control of fly optomotor responses

When confronted with a large-field stimulus rotating around the vertical body axis, flies display a following behavior called "optomotor response." As neural control elements, the large tangential horizontal system (HS) cells of the lobula plate have been prime candidates for long. Here, we applied optogenetic stimulation of HS cells to evaluate their behavioral role in Drosophila. To minimize interference of the optical activation of channelrhodopsin-2 with the visual perception of the flies, we used a bistable variant called ChR2-C128S. By applying pulses of blue and yellow light, we first demonstrate electrophysiologically that lobula plate tangential cells can be activated and deactivated repeatedly with no evident change in depolarization strength over trials. We next show that selective optogenetic activation of HS cells elicits robust yaw head movements and yaw turning responses in fixed and tethered flying flies, respectively.

Mapping and cracking sensorimotor circuits in genetic model organisms

One central goal of systems neuroscience is to understand how neural circuits implement the computations that link sensory inputs to behavior. Work combining electrophysiological and imaging-based approaches to measure neural activity with pharmacological and electrophysiological manipulations has provided fundamental insights. More recently, genetic approaches have been used to monitor and manipulate neural activity, opening up new experimental opportunities and challenges. Here, we discuss issues associated with applying genetic approaches to circuit dissection in sensorimotor transformations, outlining important considerations for experimental design and considering how modeling can complement experimental approaches.

Spatial vision in insects is facilitated by shaping the dynamics of visual input through behavioral action

Insects such as flies or bees, with their miniature brains, are able to control highly aerobatic flight maneuvres and to solve spatial vision tasks, such as avoiding collisions with obstacles, landing on objects, or even localizing a previously learnt inconspicuous goal on the basis of environmental cues. With regard to solving such spatial tasks, these insects still outperform man-made autonomous flying systems. To accomplish their extraordinary performance, flies and bees have been shown by their characteristic behavioral actions to actively shape the dynamics of the image flow on their eyes ("optic flow"). The neural processing of information about the spatial layout of the environment is greatly facilitated by segregating the rotational from the translational optic flow component through a saccadic flight and gaze strategy. This active vision strategy thus enables the nervous system to solve apparently complex spatial vision tasks in a particularly efficient and parsimonious way. The key idea of this review is that biological agents, such as flies or bees, acquire at least part of their strength as autonomous systems through active interactions with their environment and not by simply processing passively gained information about the world. These agent-environment interactions lead to adaptive behavior in surroundings of a wide range of complexity. Animals with even tiny brains, such as insects, are capable of performing extraordinarily well in their behavioral contexts by making optimal use of the closed action-perception loop. Model simulations and robotic implementations show that the smart biological mechanisms of motion computation and visually-guided flight control might be helpful to find technical solutions, for example, when designing micro air vehicles carrying a miniaturized, low-weight on-board processor.

Blowfly flight characteristics are shaped by environmental features and controlled by optic flow information

Blowfly flight consists of two main components, saccadic turns and intervals of mostly straight gaze direction, although, as a consequence of inertia, flight trajectories usually change direction smoothly. We investigated how flight behavior changes depending on the surroundings and how saccadic turns and intersaccadic translational movements might be controlled in arenas of different width with and without obstacles. Blowflies do not fly in straight trajectories, even when traversing straight flight arenas; rather, they fly in meandering trajectories. Flight speed and the amplitude of meanders increase with arena width. Although saccade duration is largely constant, peak angular velocity and succession into either direction are variable and depend on the visual surroundings. Saccade rate and amplitude also vary with arena layout and are correlated with the 'time-to-contact' to the arena wall. We provide evidence that both saccade and velocity control rely to a large extent on the intersaccadic optic flow generated in eye regions looking well in front of the fly, rather than in the lateral visual field, where the optic flow at least during forward flight tends to be strongest.

Octopaminergic modulation of contrast sensitivity

Sensory systems adapt to prolonged stimulation by decreasing their response to continuous stimuli. Whereas visual motion adaptation has traditionally been studied in immobilized animals, recent work indicates that the animal's behavioral state influences the response properties of higher-order motion vision-sensitive neurons. During insect flight octopamine is released, and pharmacological octopaminergic activation can induce a fictive locomotor state. In the insect optic ganglia, lobula plate tangential cells (LPTCs) spatially pool input from local elementary motion detectors (EMDs) that correlate luminosity changes from two spatially discrete inputs after delaying the signal from one. The LPTC velocity optimum thereby depends on the spatial separation of the inputs and on the EMD's delay properties. Recently it was shown that behavioral activity increases the LPTC velocity optimum, with modeling suggesting this to originate in the EMD's temporal delay filters. However, behavior induces an additional post-EMD effect: the LPTC membrane conductance increases in flying flies. To physiologically investigate the degree to which activity causes presynaptic and postsynaptic effects, we conducted intracellular recordings of Eristalis horizontal system (HS) neurons. We constructed contrast response functions before and after adaptation at different temporal frequencies, with and without the octopamine receptor agonist chlordimeform (CDM). We extracted three motion adaptation components, where two are likely to be generated presynaptically of the LPTCs, and one within them. We found that CDM affected the early, EMD-associated contrast gain reduction, temporal frequency dependently. However, a CDM-induced change of the HS membrane conductance disappeared during and after visual stimulation. This suggests that physical activity mainly affects motion adaptation presynaptically of LPTCs, whereas post-EMD effects have a minimal effect.

Species-Specific Flight Styles of Flies are Reflected in the Response Dynamics of a Homolog Motion-Sensitive Neuron

Hoverflies and blowflies have distinctly different flight styles. Yet, both species have been shown to structure their flight behavior in a way that facilitates extraction of 3D information from the image flow on the retina (optic flow). Neuronal candidates to analyze the optic flow are the tangential cells in the third optical ganglion - the lobula complex. These neurons are directionally selective and integrate the optic flow over large parts of the visual field. Homolog tangential cells in hoverflies and blowflies have a similar morphology. Because blowflies and hoverflies have similar neuronal layout but distinctly different flight behaviors, they are an ideal substrate to pinpoint potential neuronal adaptations to the different flight styles. In this article we describe the relationship between locomotion behavior and motion vision on three different levels: (1) We compare the different flight styles based on the categorization of flight behavior into prototypical movements. (2) We measure the species-specific dynamics of the optic flow under naturalistic flight conditions. We found the translational optic flow of both species to be very different. (3) We describe possible adaptations of a homolog motion-sensitive neuron. We stimulate this cell in blowflies (Calliphora) and hoverflies (Eristalis) with naturalistic optic flow generated by both species during free flight. The characterized hoverfly tangential cell responds faster to transient changes in the optic flow than its blowfly homolog. It is discussed whether and how the different dynamical response properties aid optic flow analysis.

Object representation and distance encoding in three-dimensional environments by a neural circuit in the visual system of the blowfly

Three motion-sensitive key elements of a neural circuit, presumably involved in processing object and distance information, were analyzed with optic flow sequences as experienced by blowflies in a three-dimensional environment. This optic flow is largely shaped by the blowfly's saccadic flight and gaze strategy, which separates translational flight segments from fast saccadic rotations. By modifying this naturalistic optic flow, all three analyzed neurons could be shown to respond during the intersaccadic intervals not only to nearby objects but also to changes in the distance to background structures. In the presence of strong background motion, the three types of neuron differ in their sensitivity for object motion. Object-induced response increments are largest in FD1, a neuron long known to respond better to moving objects than to spatially extended motion patterns, but weakest in VCH, a neuron that integrates wide-field motion from both eyes and, by inhibiting the FD1 cell, is responsible for its object preference. Small but significant object-induced response increments are present in HS cells, which serve both as a major input neuron of VCH and as output neurons of the visual system. In both HS and FD1, intersaccadic background responses decrease with increasing distance to the animal, although much more prominently in FD1. This strong dependence of FD1 on background distance is concluded to be the consequence of the activity of VCH that dramatically increases its activity and, thus, its inhibitory strength with increasing distance.

Rapid contrast gain reduction following motion adaptation

Neural and sensory systems adapt to prolonged stimulation to allow signaling across broader input ranges than otherwise possible with the limited bandwidth of single neurons and receptors. In the visual system, adaptation takes place at every stage of processing, from the photoreceptors that adapt to prevailing luminance conditions, to higher-order motion-sensitive neurons that adapt to prolonged exposure to motion. Recent experiments using dynamic, fluctuating visual stimuli indicate that adaptation operates on a time scale similar to that of the response itself. Further work from our own laboratory has highlighted the role for rapid motion adaptation in reliable encoding of natural image motion. Physiologically, motion adaptation can be broken down into four separate components. It is not clear from the previous studies which of these motion adaptation components are involved in the fast and dynamic response changes. To investigate the adapted response in more detail, we therefore analyzed the effect of motion adaptation using a test-adapt-test protocol with adapting durations ranging from 20 ms to 20 s. Our results underscore the very rapid rate of motion adaptation, suggesting that under free flight, visual motion-sensitive neurons continuously adapt to the changing scenery. This might help explain recent observations of strong invariance in the response to natural scenes with highly variable contrast and image structure.

Natural visual cues eliciting predator avoidance in fiddler crabs

To efficiently provide an animal with relevant information, the design of its visual system should reflect the distribution of natural signals and the animal's tasks. In many behavioural contexts, however, we know comparatively little about the moment-to-moment information-processing challenges animals face in their daily lives. In predator avoidance, for instance, we lack an accurate description of the natural signal stream and its value for risk assessment throughout the prey's defensive behaviour. We characterized the visual signals generated by real, potentially predatory events by video-recording bird approaches towards an Uca vomeris colony. Using four synchronized cameras allowed us to simultaneously monitor predator avoidance responses of crabs. We reconstructed the signals generated by dangerous and non-dangerous flying animals, identified the cues that triggered escape responses and compared them with those triggering responses to dummy predators. Fiddler crabs responded to a combination of multiple visual cues (including retinal speed, elevation and visual flicker) that reflect the visual signatures of distinct bird and insect behaviours. This allowed crabs to discriminate between dangerous and non-dangerous events. The results demonstrate the importance of measuring natural sensory signatures of biologically relevant events in order to understand biological information processing and its effects on behavioural organization.

Impact of visual motion adaptation on neural responses to objects and its dependence on the temporal characteristics of optic flow

It is still unclear how sensory systems efficiently encode signals with statistics as experienced by animals in the real world and what role adaptation plays during normal behavior. Therefore, we studied the performance of visual motion-sensitive neurons of blowflies, the horizontal system neurons, with optic flow that was reconstructed from the head trajectories of semi-free-flying flies. To test how motion adaptation is affected by optic flow dynamics, we manipulated the seminatural optic flow by targeted modifications of the flight trajectories and assessed to what extent neuronal responses to an object located close to the flight trajectory depend on adaptation dynamics. For all types of adapting optic flow object-induced response increments were stronger in the adapted compared with the nonadapted state. Adaptation with optic flow characterized by the typical alternation between translational and rotational segments produced this effect but also adaptation with optic flow that lacked these distinguishing features and even pure rotation at a constant angular velocity. The enhancement of object-induced response increments had a direction-selective component because preferred-direction rotation and natural optic flow were more efficient adaptors than null-direction rotation. These results indicate that natural dynamics of optic flow is not a basic requirement to adapt neurons in a specific, presumably functionally beneficial way. Our findings are discussed in the light of adaptation mechanisms proposed on the basis of experiments previously done with conventional experimenter-defined stimuli.

A syntax of hoverfly flight prototypes

Hoverflies such as Eristalis tenax Linnaeus are known for their distinctive flight style. They can hover on the same spot for several seconds and then burst into movement in apparently any possible direction. In order to determine a quantitative and structured description of complex flight manoeuvres, we searched for a set of repeatedly occurring prototypical movements (PMs) and a set of rules for their ordering. PMs were identified by applying clustering algorithms to the translational and rotational velocities of the body of Eristalis during free-flight sequences. This approach led to nine stable and reliable PMs, and thus provided a tremendous reduction in the complexity of behavioural description. This set of PMs together with the probabilities of transition between them constitute a syntactical description of flight behaviour. The PMs themselves can be roughly segregated into fast rotational turns (saccades) and a variety of distinct translational movements (intersaccadic intervals). We interpret this segregation as reflecting an active sensing strategy which facilitates the extraction of spatial information from retinal image displacements. Detailed analysis of saccades shows that they are performed around all rotational axes individually and in all possible combinations. We found the probability of occurrence of a given saccade type to depend on parameters such as the angle between the long body axis and the direction of flight.

The fine structure of honeybee head and body yaw movements in a homing task

Honeybees turn their thorax and thus their flight motor to change direction or to fly sideways. If the bee's head were fixed to its thorax, such movements would have great impact on vision. Head movements independent of thorax orientation can stabilize gaze and thus play an important and active role in shaping the structure of the visual input the animal receives. Here, we investigate how gaze and flight control interact in a homing task. We use high-speed video equipment to record the head and body movements of honeybees approaching and departing from a food source that was located between three landmarks in an indoor flight arena. During these flights, the bees' trajectories consist of straight flight segments combined with rapid turns. These short and fast yaw turns ('saccades') are in most cases accompanied by even faster head yaw turns that start about 8 ms earlier than the body saccades. Between saccades, gaze stabilization leads to a behavioural elimination of rotational components from the optical flow pattern, which facilitates depth perception from motion parallax.

Motion adaptation enhances object-induced neural activity in three-dimensional virtual environment

Many response characteristics of neurons sensitive to visual motion depend on stimulus history and change during prolonged stimulation. Although the changes are usually regarded as adaptive, their functional significance is still not fully understood. With experimenter-defined stimuli, previous research on motion adaptation has mainly focused on enhancing the detection of changes in the stimulus domain, on preventing output saturation and on energy efficient coding. Here we will analyze in the blowfly visual system the functional significance of motion adaptation under the complex stimulus conditions encountered in the three-dimensional world. Identified motion sensitive neurons are confronted with seminatural optic flow as is seen by semi-free-flying animals as well as targeted modifications of it. Motion adaptation is shown to enhance object-induced neural responses in a three-dimensional environment although the overall neuronal response amplitude decreases during prolonged motion stimulation.

Rugged, obstruction-free, mirror-lens combination for panoramic imaging

We present a new combination of lenses and reflective surfaces for obstruction-free wide-angle imaging. The panoramic imaging system consists of a reflective surface machined into solid Perspex, which together with an embedded lens, can be attached to a video camera lens. Unlike vision sensors with a single mirror mounted in front of a camera, the view in the forward direction (i.e., the direction of the optical axis) is not obstructed. Light rays contributing to the central region of the image are refracted at a centrally positioned lens and at the Perspex enclosure. For the outer image region, rays are reflected at a mirror surface of constant angular gain machined into the Perspex and coated with silver. The design produces a field of view of approximately 260 degrees with only a small separation of viewpoints. The shape of the enclosing Perspex is specifically designed in order to minimize internal reflections.

Visual Perception and the Statistical Properties of Natural Scenes

The environments in which we live and the tasks we must perform to survive and reproduce have shaped the design of our perceptual systems through evolution and experience. Therefore, direct measurement of the statistical regularities in natural environments (scenes) has great potential value for advancing our understanding of visual perception. This review begins with a general discussion of the natural scene statistics approach, of the different kinds of statistics that can be measured, and of some existing measurement techniques. This is followed by a summary of the natural scene statistics measured over the past 20 years. Finally, there is a summary of the hypotheses, models, and experiments that have emerged from the analysis of natural scene statistics.

Coding with spike shapes and graded potentials in cortical networks

In cortical neurones, analogue dendritic potentials are thought to be encoded into patterns of digital spikes. According to this view, neuronal codes and computations are based on the temporal patterns of spikes: spike times, bursts or spike rates. Recently, we proposed an 'action potential waveform code' for cortical pyramidal neurones in which the spike shape carries information. Broader somatic action potentials are reliably produced in response to higher conductance input, allowing for four times more information transfer than spike times alone. This information is preserved during synaptic integration in a single neurone, as back-propagating action potentials of diverse shapes differentially shunt incoming postsynaptic potentials and so participate in the next round of spike generation. An open question has been whether the information in action potential waveforms can also survive axonal conduction and directly influence synaptic transmission to neighbouring neurones. Several new findings have now brought new light to this subject, showing cortical information processing that transcends the classical models.

A `bright zone' in male hoverfly (Eristalis tenax) eyes and associated faster motion detection and increased contrast sensitivity

Eyes of the hoverfly Eristalis tenax are sexually dimorphic such that males have a fronto-dorsal region of large facets. In contrast to other large flies in which large facets are associated with a decreased interommatidial angle to form a dorsal `acute zone' of increased spatial resolution, we show that a dorsal region of large facets in males appears to form a `bright zone' of increased light capture without substantially increased spatial resolution. Theoretically, more light allows for increased performance in tasks such as motion detection. To determine the effect of the bright zone on motion detection, local properties of wide field motion detecting neurons were investigated using localized sinusoidal gratings. The pattern of local preferred directions of one class of these cells, the HS cells, in Eristalis is similar to that reported for the blowfly Calliphora. The bright zone seems to contribute to local contrast sensitivity; high contrast sensitivity exists in portions of the receptive field served by large diameter facet lenses of males and is not observed in females. Finally, temporal frequency tuning is also significantly faster in this frontal portion of the world, particularly in males, where it overcompensates for the higher spatial-frequency tuning and shifts the predicted local velocity optimum to higher speeds. These results indicate that increased retinal illuminance due to the bright zone of males is used to enhance contrast sensitivity and speed motion detector responses. Additionally, local neural properties vary across the visual world in a way not expected if HS cells serve purely as matched filters to measure yaw-induced visual motion.

Encoding of Naturalistic Optic Flow by a Population of Blowfly Motion-Sensitive Neurons

In sensory systems information is encoded by the activity of populations of neurons. To analyze the coding properties of neuronal populations sensory stimuli have usually been used that were much simpler than those encountered in real life. It has been possible only recently to stimulate visual interneurons of the blowfly with naturalistic visual stimuli reconstructed from eye movements measured during free flight. Therefore we now investigate with naturalistic optic flow the coding properties of a small neuronal population of identified visual interneurons in the blowfly, the so-called VS and HS neurons. These neurons are motion sensitive and directionally selective and are assumed to extract information about the animal's self-motion from optic flow. We could show that neuronal responses of VS and HS neurons are mainly shaped by the characteristic dynamical properties of the fly's saccadic flight and gaze strategy. Individual neurons encode information about both the rotational and the translational components of the animal's self-motion. Thus the information carried by individual neurons is ambiguous. The ambiguities can be reduced by considering neuronal population activity. The joint responses of different subpopulations of VS and HS neurons can provide unambiguous information about the three rotational and the three translational components of the animal's self-motion and also, indirectly, about the three-dimensional layout of the environment.

Representation of behaviourally relevant information by blowfly motion-sensitive visual interneurons requires precise compensatory head movements

Flying blowflies shift their gaze by saccadic turns of body and head, keeping their gaze basically fixed between saccades. For the head, this results in almost pure translational optic flow between saccades, enabling visual interneurons in the fly motion pathway to extract information about translation of the animal and thereby about the spatial layout of the environment. There are noticeable differences between head and body movements during flight. Head saccades are faster and shorter than body saccades, and the head orientation is more stable between saccades than the body orientation. Here, we analyse the functional importance of these differences by probing visual interneurons of the blowfly motion pathway with optic flow based on either head movements or body movements, as recorded accurately with a magnetic search coil technique. We find that the precise head-body coordination is essential for the visual system to separate the translational from the rotational optic flow. If the head were tightly coupled to the body, the resulting optic flow would not contain the behaviourally important information on translation. Since it is difficult to resolve head orientation in many experimental paradigms, even when employing state-of-the-art digital video techniques, we introduce a 'headifying algorithm', which transforms the time-dependent body orientation in free flight into an estimate of head orientation. We show that application of this algorithm leads to an estimated head orientation between saccades that is sufficiently stable to enable recovering information on translation. The algorithm may therefore be of practical use when head orientation is needed but cannot be measured.

The visual ecology of fiddler crabs

With their eyes on long vertical stalks, their panoramic visual field and their pronounced equatorial acute zone for vertical resolving power, the visual system of fiddler crabs is exquisitely tuned to the geometry of vision in the flat world of inter-tidal mudflats. The crabs live as burrow-centred grazers in dense, mixed-sex, mixed-age and mixed-species colonies, with the active space of an individual rarely exceeding 1 m(2). The full behavioural repertoire of fiddler crabs can thus be monitored over extended periods of time on a moment to moment basis together with the visual information they have available to guide their actions. These attributes make the crabs superb subjects for analysing visual tasks and the design of visual processing mechanisms under natural conditions, a prerequisite for understanding the evolution of visual systems. In this review we show, on the one hand, how deeply embedded fiddler crab vision is in the behavioural and the physical ecology of these animals and, on the other hand, how their behavioural options are constrained by their perceptual limitations. Studying vision in fiddler crabs reminds us that vision has a topography, that it is context-dependent and pragmatic and that there are perceptual limits to what animals can know and therefore care about.

Movement-induced motion signal distributions in outdoor scenes

The movement of an observer generates a characteristic field of velocity vectors on the retina (Gibson 1950). Because such optic flow-fields are useful for navigation, many theoretical, psychophysical and physiological studies have addressed the question how ego-motion parameters such as direction of heading can be estimated from optic flow. Little is known, however, about the structure of optic flow under natural conditions. To address this issue, we recorded sequences of panoramic images along accurately defined paths in a variety of outdoor locations and used these sequences as input to a two-dimensional array of correlation-based motion detectors (2DMD). We find that (a) motion signal distributions are sparse and noisy with respect to local motion directions; (b) motion signal distributions contain patches (motion streaks) which are systematically oriented along the principal flow-field directions; (c) motion signal distributions show a distinct, dorso-ventral topography, reflecting the distance anisotropy of terrestrial environments; (d) the spatiotemporal tuning of the local motion detector we used has little influence on the structure of motion signal distributions, at least for the range of conditions we tested; and (e) environmental motion is locally noisy throughout the visual field, with little spatial or temporal correlation; it can therefore be removed by temporal averaging and is largely over-ridden by image motion caused by observer movement. Our results suggest that spatial or temporal integration is important to retrieve reliable information on the local direction and size of motion vectors, because the structure of optic flow is clearly detectable in the temporal average of motion signal distributions. Ego-motion parameters can be reliably retrieved from such averaged distributions under a range of environmental conditions. These observations raise a number of questions about the role of specific environmental and computational constraints in the processing of natural optic flow.