Texture dependence of motion sensing and free flight behavior in blowflies

Finding the gap: neuromorphic motion-vision in dense environments

Animals have evolved mechanisms to travel safely and efficiently within different habitats. On a journey in dense terrains animals avoid collisions and cross narrow passages while controlling an overall course. Multiple hypotheses target how animals solve challenges faced during such travel. Here we show that a single mechanism enables safe and efficient travel. We developed a robot inspired by insects. It has remarkable capabilities to travel in dense terrain, avoiding collisions, crossing gaps and selecting safe passages. These capabilities are accomplished by a neuromorphic network steering the robot toward regions of low apparent motion. Our system leverages knowledge about vision processing and obstacle avoidance in insects. Our results demonstrate how insects might safely travel through diverse habitats. We anticipate our system to be a working hypothesis to study insects' travels in dense terrains. Furthermore, it illustrates that we can design novel hardware systems by understanding the underlying mechanisms driving behaviour.

A biohybrid fly-robot interface system that performs active collision avoidance

We have designed a bio-hybrid fly-robot interface (FRI) to study sensorimotor control in insects. The FRI consists of a miniaturized recording platform mounted on a two-wheeled robot and is controlled by the neuronal spiking activity of an identified visual interneuron, the blowfly H1-cell. For a given turning radius of the robot, we found a proportional relationship between the spike rate of the H1-cell and the relative distance of the FRI from the patterned wall of an experimental arena. Under closed-loop conditions during oscillatory forward movements biased towards the wall, collision avoidance manoeuvres were triggered whenever the H1-cell spike rate exceeded a certain threshold value. We also investigated the FRI behaviour in corners of the arena. The ultimate goal is to enable autonomous and energy-efficient manoeuvrings of the FRI within arbitrary visual environments.

Spatial Encoding of Translational Optic Flow in Planar Scenes by Elementary Motion Detector Arrays

Elementary Motion Detectors (EMD) are well-established models of visual motion estimation in insects. The response of EMDs are tuned to specific temporal and spatial frequencies of the input stimuli, which matches the behavioural response of insects to wide-field image rotation, called the optomotor response. However, other behaviours, such as speed and position control, cannot be fully accounted for by EMDs because these behaviours are largely unaffected by image properties and appear to be controlled by the ratio between the flight speed and the distance to an object, defined here as relative nearness. We present a method that resolves this inconsistency by extracting an unambiguous estimate of relative nearness from the output of an EMD array. Our method is suitable for estimation of relative nearness in planar scenes such as when flying above the ground or beside large flat objects. We demonstrate closed loop control of the lateral position and forward velocity of a simulated agent flying in a corridor. This finding may explain how insects can measure relative nearness and control their flight despite the frequency tuning of EMDs. Our method also provides engineers with a relative nearness estimation technique that benefits from the low computational cost of EMDs.

Biological Agency: Its Subjective Foundations and a Large-Scale Taxonomy

We will outline a theory of agency cast in theoretical psychology, viewed as a branch of a non-eliminativist biology. Our proposal will be based on an evolutionary view of the nature and functioning of the mind(s), reconsidered in a radically subjectivist, radically constructivist framework. We will argue that the activities of control systems should be studied in terms of interaction. Specifically, what an agent does belongs to the coupling of its internal dynamics with the dynamics of the external world. The internal dynamics, rooted in the species' phylogenetic history as well as in the individual's ontogenetic path, (a) determine which external dynamics are relevant to the organism, that is, they create the subjective ontology that the organism senses in the external world, and (b) determine what types of activities and actions the agent is able to conceive of and to adopt in the current situation. The external dynamics that the organism senses thus constitute its subjective environment. This notion of coupling is basically suitable for whichever organism one may want to consider. However, remarkable differences exist between the ways in which coupling may be realized, that is, between different natures and ways of functioning of control systems. We will describe agency at different phylogenetic levels: at the very least, it is necessary to discriminate between non-Intentional species, Intentional species, and a subtype of the latter called meta-Intentional. We will claim that agency can only be understood in a radically subjectivist perspective, which in turn is best grounded in a view of the mind as consciousness and experience. We will thus advance a radically constructivist view of agency and of several correlate notions (like meaning and ontology).

A Bio-inspired Collision Avoidance Model Based on Spatial Information Derived from Motion Detectors Leads to Common Routes

Avoiding collisions is one of the most basic needs of any mobile agent, both biological and technical, when searching around or aiming toward a goal. We propose a model of collision avoidance inspired by behavioral experiments on insects and by properties of optic flow on a spherical eye experienced during translation, and test the interaction of this model with goal-driven behavior. Insects, such as flies and bees, actively separate the rotational and translational optic flow components via behavior, i.e. by employing a saccadic strategy of flight and gaze control. Optic flow experienced during translation, i.e. during intersaccadic phases, contains information on the depth-structure of the environment, but this information is entangled with that on self-motion. Here, we propose a simple model to extract the depth structure from translational optic flow by using local properties of a spherical eye. On this basis, a motion direction of the agent is computed that ensures collision avoidance. Flying insects are thought to measure optic flow by correlation-type elementary motion detectors. Their responses depend, in addition to velocity, on the texture and contrast of objects and, thus, do not measure the velocity of objects veridically. Therefore, we initially used geometrically determined optic flow as input to a collision avoidance algorithm to show that depth information inferred from optic flow is sufficient to account for collision avoidance under closed-loop conditions. Then, the collision avoidance algorithm was tested with bio-inspired correlation-type elementary motion detectors in its input. Even then, the algorithm led successfully to collision avoidance and, in addition, replicated the characteristics of collision avoidance behavior of insects. Finally, the collision avoidance algorithm was combined with a goal direction and tested in cluttered environments. The simulated agent then showed goal-directed behavior reminiscent of components of the navigation behavior of insects.

A network of visual motion-sensitive neurons for computing object position in an arthropod

Highly active insects and crabs depend on visual motion information for detecting and tracking mates, prey, or predators, for which they require directional control systems containing internal maps of visual space. A neural map formed by large, motion-sensitive neurons implicated in processing panoramic flow is known to exist in an optic ganglion of the fly. However, an equivalent map for processing spatial positions of single objects has not been hitherto identified in any arthropod. Crabs can escape directly away from a visual threat wherever the stimulus is located in the 360° field of view. When tested in a walking simulator, the crab Neohelice granulata immediately adjusts its running direction after changes in the position of the visual danger stimulus smaller than 1°. Combining mass and single-cell staining with in vivo intracellular recording, we show that a particular class of motion-sensitive neurons of the crab's lobula that project to the midbrain, the monostratified lobula giants type 1 (MLG1), form a system of 16 retinotopically organized elements that map the 360° azimuthal space. The preference of these neurons for horizontally moving objects conforms the visual ecology of the crab's mudflat world. With a mean receptive field of 118°, MLG1s have a large superposition among neighboring elements. Our results suggest that the MLG1 system conveys information on object position as a population vector. Such computational code can enable the accurate directional control observed in the visually guided behaviors of crabs.

Motion as a source of environmental information: a fresh view on biological motion computation by insect brains

Despite their miniature brains insects, such as flies, bees and wasps, are able to navigate by highly erobatic flight maneuvers in cluttered environments. They rely on spatial information that is contained in the retinal motion patterns induced on the eyes while moving around ("optic flow") to accomplish their extraordinary performance. Thereby, they employ an active flight and gaze strategy that separates rapid saccade-like turns from translatory flight phases where the gaze direction is kept largely constant. This behavioral strategy facilitates the processing of environmental information, because information about the distance of the animal to objects in the environment is only contained in the optic flow generated by translatory motion. However, motion detectors as are widespread in biological systems do not represent veridically the velocity of the optic flow vectors, but also reflect textural information about the environment. This characteristic has often been regarded as a limitation of a biological motion detection mechanism. In contrast, we conclude from analyses challenging insect movement detectors with image flow as generated during translatory locomotion through cluttered natural environments that this mechanism represents the contours of nearby objects. Contrast borders are a main carrier of functionally relevant object information in artificial and natural sceneries. The motion detection system thus segregates in a computationally parsimonious way the environment into behaviorally relevant nearby objects and-in many behavioral contexts-less relevant distant structures. Hence, by making use of an active flight and gaze strategy, insects are capable of performing extraordinarily well even with a computationally simple motion detection mechanism.

Temporal statistics of natural image sequences generated by movements with insect flight characteristics

Many flying insects, such as flies, wasps and bees, pursue a saccadic flight and gaze strategy. This behavioral strategy is thought to separate the translational and rotational components of self-motion and, thereby, to reduce the computational efforts to extract information about the environment from the retinal image flow. Because of the distinguishing dynamic features of this active flight and gaze strategy of insects, the present study analyzes systematically the spatiotemporal statistics of image sequences generated during saccades and intersaccadic intervals in cluttered natural environments. We show that, in general, rotational movements with saccade-like dynamics elicit fluctuations and overall changes in brightness, contrast and spatial frequency of up to two orders of magnitude larger than translational movements at velocities that are characteristic of insects. Distinct changes in image parameters during translations are only caused by nearby objects. Image analysis based on larger patches in the visual field reveals smaller fluctuations in brightness and spatial frequency composition compared to small patches. The temporal structure and extent of these changes in image parameters define the temporal constraints imposed on signal processing performed by the insect visual system under behavioral conditions in natural environments.

Depth information in natural environments derived from optic flow by insect motion detection system: a model analysis

Knowing the depth structure of the environment is crucial for moving animals in many behavioral contexts, such as collision avoidance, targeting objects, or spatial navigation. An important source of depth information is motion parallax. This powerful cue is generated on the eyes during translatory self-motion with the retinal images of nearby objects moving faster than those of distant ones. To investigate how the visual motion pathway represents motion-based depth information we analyzed its responses to image sequences recorded in natural cluttered environments with a wide range of depth structures. The analysis was done on the basis of an experimentally validated model of the visual motion pathway of insects, with its core elements being correlation-type elementary motion detectors (EMDs). It is the key result of our analysis that the absolute EMD responses, i.e., the motion energy profile, represent the contrast-weighted nearness of environmental structures during translatory self-motion at a roughly constant velocity. In other words, the output of the EMD array highlights contours of nearby objects. This conclusion is largely independent of the scale over which EMDs are spatially pooled and was corroborated by scrutinizing the motion energy profile after eliminating the depth structure from the natural image sequences. Hence, the well-established dependence of correlation-type EMDs on both velocity and textural properties of motion stimuli appears to be advantageous for representing behaviorally relevant information about the environment in a computationally parsimonious way.

Texture-defined objects influence responses of blowfly motion-sensitive neurons under natural dynamical conditions

The responses of visual interneurons of flies involved in the processing of motion information do not only depend on the velocity, but also on other stimulus parameters, such as the contrast and the spatial frequency content of the stimulus pattern. These dependencies have been known for long, but it is still an open question how they affect the neurons' performance in extracting information about the structure of the environment under the specific dynamical conditions of natural flight. Free-flight of blowflies is characterized by sequences of phases of translational movements lasting for just 30-100 ms interspersed with even shorter and extremely rapid saccade-like rotational shifts in flight and gaze direction. Previous studies already analyzed how nearby objects, leading to relative motion on the retina with respect to a more distant background, influenced the response of a class of fly motion sensitive visual interneurons, the horizontal system (HS) cells. In the present study, we focused on objects that differed from their background by discontinuities either in their brightness contrast or in their spatial frequency content. We found strong object-induced effects on the membrane potential even during the short intersaccadic intervals, if the background contrast was small and the object contrast sufficiently high. The object evoked similar response increments provided that it contained higher spatial frequencies than the background, but not under reversed conditions. This asymmetry in the response behavior is partly a consequence of the depolarization level induced by the background. Thus, our results suggest that, under the specific dynamical conditions of natural flight, i.e., on a very short timescale, the responses of HS cells represent object information depending on the polarity of the difference between object and background contrast and spatial frequency content.