Seeing what is coming: building collision-sensitive neurones

Freezing of movements and its correspondence with MLG1 neuron response to looming stimuli in the crab Neohelice

Upon visually detecting a moving predator, animals often freeze, i.e. stop moving, to minimize being uncovered and to gather detailed information of the object's movements and properties. In certain conditions, the freezing behavior can be enough to avoid a predatory menace but, when the risk is high or increases to a higher level, animals switch strategy and engage in an escape response. The neural bases underlying escape responses to visual stimuli have been extensively investigated both in vertebrates and arthropods. However, those involved in freezing behaviors are much less studied. Here, we investigated the freezing behavior displayed by the crab Neohelice granulata when confronted with a variety of looming stimuli simulating objects of distinct sizes approaching on a collision course at different speeds. The experiments were performed in a treadmill-like device. Animals engaged in exploratory walks responded to the looming stimulus with freezing followed by escaping. The analysis of the stimulus optical variables shows that regardless of the looming dynamic, the freezing decision is made when the angular size of the object increases by 1.4 deg. In vivo intracellular recording responses of monostratified lobula giant neurons (MLG1) to the same looming stimuli show that the freezing times correlate with the times predicted by a hypothetical spike counter of this neuron.

A Computationally Efficient Neuronal Model for Collision Detection with Contrast Polarity-Specific Feed-Forward Inhibition

Animals utilize their well-evolved dynamic vision systems to perceive and evade collision threats. Driven by biological research, bio-inspired models based on lobula giant movement detectors (LGMDs) address certain gaps in constructing artificial collision-detecting vision systems with robust selectivity, offering reliable, low-cost, and miniaturized collision sensors across various scenes. Recent progress in neuroscience has revealed the energetic advantages of dendritic arrangements presynaptic to the LGMDs, which receive contrast polarity-specific signals on separate dendritic fields. Specifically, feed-forward inhibitory inputs arise from parallel ON/OFF pathways interacting with excitation. However, none of the previous research has investigated the evolution of a computational LGMD model with feed-forward inhibition (FFI) separated by opposite polarity. This study fills this vacancy by presenting an optimized neuronal model where FFI is divided into ON/OFF channels, each with distinct synaptic connections. To align with the energy efficiency of biological systems, we introduce an activation function associated with neural computation of FFI and interactions between local excitation and lateral inhibition within ON/OFF channels, ignoring non-active signal processing. This approach significantly improves the time efficiency of the LGMD model, focusing only on substantial luminance changes in image streams. The proposed neuronal model not only accelerates visual processing in relatively stationary scenes but also maintains robust selectivity to ON/OFF-contrast looming stimuli. Additionally, it can suppress translational motion to a moderate extent. Comparative testing with state-of-the-art based on ON/OFF channels was conducted systematically using a range of visual stimuli, including indoor structured and complex outdoor scenes. The results demonstrated significant time savings in silico while retaining original collision selectivity. Furthermore, the optimized model was implemented in the embedded vision system of a micro-mobile robot, achieving the highest success ratio of collision avoidance at 97.51% while nearly halving the processing time compared with previous models. This highlights a robust and parsimonious collision-sensing mode that effectively addresses real-world challenges.

An Angular Acceleration Based Looming Detector for Moving UAVs

Visual perception equips unmanned aerial vehicles (UAVs) with increasingly comprehensive and instant environmental perception, rendering it a crucial technology in intelligent UAV obstacle avoidance. However, the rapid movements of UAVs cause significant changes in the field of view, affecting the algorithms' ability to extract the visual features of collisions accurately. As a result, algorithms suffer from a high rate of false alarms and a delay in warning time. During the study of visual field angle curves of different orders, it was found that the peak times of the curves of higher-order information on the angular size of looming objects are linearly related to the time to collision (TTC) and occur before collisions. This discovery implies that encoding higher-order information on the angular size could resolve the issue of response lag. Furthermore, the fact that the image of a looming object adjusts to meet several looming visual cues compared to the background interference implies that integrating various field-of-view characteristics will likely enhance the model's resistance to motion interference. Therefore, this paper presents a concise A-LGMD model for detecting looming objects. The model is based on image angular acceleration and addresses problems related to imprecise feature extraction and insufficient time series modeling to enhance the model's ability to rapidly and precisely detect looming objects during the rapid self-motion of UAVs. The model draws inspiration from the lobula giant movement detector (LGMD), which shows high sensitivity to acceleration information. In the proposed model, higher-order information on the angular size is abstracted by the network and fused with multiple visual field angle characteristics to promote the selective response to looming objects. Experiments carried out on synthetic and real-world datasets reveal that the model can efficiently detect the angular acceleration of an image, filter out insignificant background motion, and provide early warnings. These findings indicate that the model could have significant potential in embedded collision detection systems of micro or small UAVs.

A monocular wide-field speed sensor inspired by the crabs' visual system for traffic analysis

The development of visual sensors for traffic analysis can benefit from mimicking two fundamental aspects of the visual system of crabs: their panoramic vision and their visual processing strategy adapted to a flat world. First, the use of omnidirectional cameras in urban environments allows for analyzing the simultaneous movement of many objects of interest over broad areas. This would reduce the costs and complications associated with infrastructure: installation, synchronization, maintenance, and operation of traditional vision systems that use multiple cameras with a limited field of view. Second, in urban traffic analysis, the objects of interest (e.g. vehicles and pedestrians) move on the ground surface. This constraint allows the calculation of the 3D trajectory of the vehicles using a single camera without the need to use binocular vision techniques.The main contribution of this work is to show that the strategy used by crabs to visually analyze their habitat (monocular omnidirectional vision with the assumption of a flat world ) is useful for developing a simple and effective method to estimate the speed of vehicles on long trajectories in urban environments. It is shown that the proposed method estimates the speed with a root mean squared error of 2.7 km h^-1.

Animal behavior: Timing escape on angular size or angular velocity?

Animals avoid rapidly approaching objects. In many arthropods, angular size is a common visual cue used to decide the time of avoidance. A new study shows that fiddler crabs decide when to escape based on object angular expansion velocity.

A directionally selective collision-sensing visual neural network based on fractional-order differential operator

In this paper, we propose a directionally selective fractional-order lobular giant motion detector (LGMD) visual neural network. Unlike most collision-sensing network models based on LGMDs, our model can not only sense collision threats but also obtain the motion direction of the collision object. Firstly, this paper simulates the membrane potential response of neurons using the fractional-order differential operator to generate reliable collision response spikes. Then, a new correlation mechanism is proposed to obtain the motion direction of objects. Specifically, this paper performs correlation operation on the signals extracted from two pixels, utilizing the temporal delay of the signals to obtain their position relationship. In this way, the response characteristics of direction-selective neurons can be characterized. Finally, ON/OFF visual channels are introduced to encode increases and decreases in brightness, respectively, thereby modeling the bipolar response of special neurons. Extensive experimental results show that the proposed visual neural system conforms to the response characteristics of biological LGMD and direction-selective neurons, and that the performance of the system is stable and reliable.

A Looming Spatial Localization Neural Network Inspired by MLG1 Neurons in the Crab Neohelice

Similar to most visual animals, the crab Neohelice granulata relies predominantly on visual information to escape from predators, to track prey and for selecting mates. It, therefore, needs specialized neurons to process visual information and determine the spatial location of looming objects. In the crab Neohelice granulata, the Monostratified Lobula Giant type1 (MLG1) neurons have been found to manifest looming sensitivity with finely tuned capabilities of encoding spatial location information. MLG1s neuronal ensemble can not only perceive the location of a looming stimulus, but are also thought to be able to influence the direction of movement continuously, for example, escaping from a threatening, looming target in relation to its position. Such specific characteristics make the MLG1s unique compared to normal looming detection neurons in invertebrates which can not localize spatial looming. Modeling the MLG1s ensemble is not only critical for elucidating the mechanisms underlying the functionality of such neural circuits, but also important for developing new autonomous, efficient, directionally reactive collision avoidance systems for robots and vehicles. However, little computational modeling has been done for implementing looming spatial localization analogous to the specific functionality of MLG1s ensemble. To bridge this gap, we propose a model of MLG1s and their pre-synaptic visual neural network to detect the spatial location of looming objects. The model consists of 16 homogeneous sectors arranged in a circular field inspired by the natural arrangement of 16 MLG1s' receptive fields to encode and convey spatial information concerning looming objects with dynamic expanding edges in different locations of the visual field. Responses of the proposed model to systematic real-world visual stimuli match many of the biological characteristics of MLG1 neurons. The systematic experiments demonstrate that our proposed MLG1s model works effectively and robustly to perceive and localize looming information, which could be a promising candidate for intelligent machines interacting within dynamic environments free of collision. This study also sheds light upon a new type of neuromorphic visual sensor strategy that can extract looming objects with locational information in a quick and reliable manner.

Gravity and Known Size Calibrate Visual Information to Time Parabolic Trajectories

Catching a ball in a parabolic flight is a complex task in which the time and area of interception are strongly coupled, making interception possible for a short period. Although this makes the estimation of time-to-contact (TTC) from visual information in parabolic trajectories very useful, previous attempts to explain our precision in interceptive tasks circumvent the need to estimate TTC to guide our action. Obtaining TTC from optical variables alone in parabolic trajectories would imply very complex transformations from 2D retinal images to a 3D layout. We propose based on previous work and show by using simulations that exploiting prior distributions of gravity and known physical size makes these transformations much simpler, enabling predictive capacities from minimal early visual information. Optical information is inherently ambiguous, and therefore, it is necessary to explain how these prior distributions generate predictions. Here is where the role of prior information comes into play: it could help to interpret and calibrate visual information to yield meaningful predictions of the remaining TTC. The objective of this work is: (1) to describe the primary sources of information available to the observer in parabolic trajectories; (2) unveil how prior information can be used to disambiguate the sources of visual information within a Bayesian encoding-decoding framework; (3) show that such predictions might be robust against complex dynamic environments; and (4) indicate future lines of research to scrutinize the role of prior knowledge calibrating visual information and prediction for action control.

Robustness of Bio-Inspired Visual Systems for Collision Prediction in Critical Robot Traffic

Collision prevention sets a major research and development obstacle for intelligent robots and vehicles. This paper investigates the robustness of two state-of-the-art neural network models inspired by the locust’s LGMD-1 and LGMD-2 visual pathways as fast and low-energy collision alert systems in critical scenarios. Although both the neural circuits have been studied and modelled intensively, their capability and robustness against real-time critical traffic scenarios where real-physical crashes will happen have never been systematically investigated due to difficulty and high price in replicating risky traffic with many crash occurrences. To close this gap, we apply a recently published robotic platform to test the LGMDs inspired visual systems in physical implementation of critical traffic scenarios at low cost and high flexibility. The proposed visual systems are applied as the only collision sensing modality in each micro-mobile robot to conduct avoidance by abrupt braking. The simulated traffic resembles on-road sections including the intersection and highway scenes wherein the roadmaps are rendered by coloured, artificial pheromones upon a wide LCD screen acting as the ground of an arena. The robots with light sensors at bottom can recognise the lanes and signals, tightly follow paths. The emphasis herein is laid on corroborating the robustness of LGMDs neural systems model in different dynamic robot scenes to timely alert potential crashes. This study well complements previous experimentation on such bio-inspired computations for collision prediction in more critical physical scenarios, and for the first time demonstrates the robustness of LGMDs inspired visual systems in critical traffic towards a reliable collision alert system under constrained computation power. This paper also exhibits a novel, tractable, and affordable robotic approach to evaluate online visual systems in dynamic scenes.

Intersection of motor volumes predicts the outcome of ambush predation of larval zebrafish

Escape maneuvers are key determinants of animal survival and are under intense selection pressure. A number of escape maneuver parameters contribute to survival, including response latency, escape speed and direction. However, the relative importance of these parameters is context dependent, suggesting that interactions between parameters and predatory context determine the likelihood of escape success. To better understand how escape maneuver parameters interact and contribute to survival, we analyzed the responses of larval zebrafish (Danio rerio) to the attacks of dragonfly nymphs (Sympetrum vicinum). We found that no single parameter explains the outcome. Instead, the relative intersection of the swept volume of the nymph's grasping organs with the volume containing all possible escape trajectories of the fish is the strongest predictor of escape success. In cases where the prey's motor volume exceeds that of the predator, the prey survives. By analyzing the intersection of these volumes, we compute the survival benefit of recruiting the Mauthner cell, a neuron in anamniotes devoted to producing escapes. We discuss how the intersection of motor volume approach provides a framework that unifies the influence of many escape maneuver parameters on the likelihood of survival.

A Robust Collision Perception Visual Neural Network With Specific Selectivity to Darker Objects

Building an efficient and reliable collision perception visual system is a challenging problem for future robots and autonomous vehicles. The biological visual neural networks, which have evolved over millions of years in nature and are working perfectly in the real world, could be ideal models for designing artificial vision systems. In the locust's visual pathways, a lobula giant movement detector (LGMD), that is, the LGMD2, has been identified as a looming perception neuron that responds most strongly to darker approaching objects relative to their backgrounds; similar situations which many ground vehicles and robots are often faced with. However, little has been done on modeling the LGMD2 and investigating its potential in robotics and vehicles. In this article, we build an LGMD2 visual neural network which possesses the similar collision selectivity of an LGMD2 neuron in locust via the modeling of biased-ON and -OFF pathways splitting visual signals into parallel ON/OFF channels. With stronger inhibition (bias) in the ON pathway, this model responds selectively to darker looming objects. The proposed model has been tested systematically with a range of stimuli including real-world scenarios. It has also been implemented in a micro-mobile robot and tested with real-time experiments. The experimental results have verified the effectiveness and robustness of the proposed model for detecting darker looming objects against various dynamic and cluttered backgrounds.

Visual motion sensitivity in descending neurons in the hoverfly

Many animals use motion vision information to control dynamic behaviors. For example, flying insects must decide whether to pursue a prey or not, to avoid a predator, to maintain their current flight trajectory, or to land. The neural mechanisms underlying the computation of visual motion have been particularly well investigated in the fly optic lobes. However, the descending neurons, which connect the optic lobes with the motor command centers of the ventral nerve cord, remain less studied. To address this deficiency, we describe motion vision sensitive descending neurons in the hoverfly Eristalis tenax. We describe how the neurons can be identified based on their receptive field properties, and how they respond to moving targets, looming stimuli and to widefield optic flow. We discuss their similarities with previously published visual neurons, in the optic lobes and ventral nerve cord, and suggest that they can be classified as target-selective, looming sensitive and optic flow sensitive, based on these similarities. Our results highlight the importance of using several visual stimuli as the neurons can rarely be identified based on only one response characteristic. In addition, they provide an understanding of the neurophysiology of visual neurons that are likely to affect behavior.

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.

Characterization and modelling of looming-sensitive neurons in the crab Neohelice

Looming-sensitive neurons (LSNs) are motion-sensitive neurons tuned for detecting imminent collision. Their main characteristic is the selectivity to looming (a 2D representation of an object approach), rather than to receding stimuli. We studied a set of LSNs by performing surface extracellular recordings in the optic nerve of Neohelice granulata crabs, and characterized their response against computer-generated visual stimuli with different combinations of moving edges, highlighting different components of the optical flow. In addition to their selectivity to looming stimuli, we characterized other properties of these neurons, such as low directionality; reduced response to sustained excitement; and an inhibition phenomenon in response to visual stimuli with dense optical flow of expansion, contraction, and translation. To analyze the spatio-temporal processing of these LSNs, we proposed a biologically plausible computational model which was inspired by previous computational models of the locust lobula giant motion detector (LGMD) neuron. The videos seen by the animal during electrophysiological experiments were applied as an input to the model which produced a satisfactory fit to the measured responses, suggesting that the computation performed by LSNs in a decapod crustacean appears to be based on similar physiological processing previously described for the LGMD in insects.

Hermit crab response to a visual threat is sensitive to looming cues

Prior work in our lab has shown that an expanding image on a computer screen elicits a hiding response in the Caribbean terrestrial hermit crab (Coenobita clypeatus). We conducted two experiments to identify what properties of the expanding stimulus contribute to its effectiveness as a visual threat. First we found that an expanding geometric star evoked a strong hiding response while a contracting or full-sized stationary star did not. A second experiment revealed that the more quickly the stimulus expanded the shorter the latency to hide. These findings suggest that the anti-predator response to looming stimulus relies heavily on visual cues relating to the manner of approach. The simulated visual threat on a computer screen captures key features of a real looming object that elicits hiding behavior in crabs in the wild.

Action ability modulates time-to-collision judgments

Time-to-collision (TTC) underestimation has been interpreted as an adaptive response that allows observers to have more time to engage in a defensive behaviour. This bias seems, therefore, strongly linked to action preparation. There is evidence that the observer’s physical fitness modulates the underestimation effect so that people who need more time to react (i.e. those with less physical fitness) show a stronger underestimation effect. Here we investigated whether this bias is influenced by the momentary action capability of the observers. In the first experiment, participants estimated the time-to-collision of threatening or non-threatening stimuli while being mildly immobilized (with a chin rest) or while standing freely. Having reduced the possibility of movement led participants to show more underestimation of the approaching stimuli. However, this effect was not stronger for threatening relative to non-threatening stimuli. The effect of the action capability found in the first experiment could be interpreted as an expansion of peripersonal space (PPS). In the second experiment, we thus investigated the generality of this effect using an established paradigm to measure the size of peripersonal space. Participants bisected lines from different distances while in the chin rest or standing freely. The results replicated the classic left-to-right gradient in lateral spatial attention with increasing viewing distance, but no effect of immobilization was found. The manipulation of the momentary action capability of the observers influenced the participants’ performance in the TTC task but not in the line bisection task. These results are discussed in relation to the different functions of PPS.

Rat superior colliculus neurons respond to large visual stimuli flashed outside the classical receptive field

Spatial integration of visual stimuli is a crucial step in visual information processing yet it is often unclear where this integration takes place in the visual system. In the superficial layers of the superior colliculus that form an early stage in visual information processing, neurons are known to have relatively small visual receptive fields, suggesting limited spatial integration. Here it is shown that at least for rats this conclusion may be wrong. Extracellular recordings in urethane-anaesthetized young adult rats (1.5–2 months old) showed that large stimuli of over 10° could evoke detectable responses well outside the borders of ‘classical’ receptive fields determined by employing 2° – 3.5° stimuli. The presence of responses to large stimuli well outside these ‘classical’ receptive fields could not be explained neither by partial overlap between the visual stimulus and the receptive field, nor by reflections or light dispersion from the stimulation site. However, very low frequency (<0.1 Hz) residual responses to small stimuli presented outside the receptive field may explain the obtained results if we assume that the frequency of action potentials during a response to a stimulus outside RF is proportional to the stimulus area. Thus, responses to large stimuli outside RF may be predicted by scaling according to the stimulus area of the responses to small stimuli. These data demonstrate that neurons in the superficial layers of the superior colliculus are capable of integrating visual stimuli over much larger area than it can be deduced from the classical receptive field.

The COGs (context, object, and goals) in multisensory processing

Our understanding of how perception operates in real-world environments has been substantially advanced by studying both multisensory processes and "top-down" control processes influencing sensory processing via activity from higher-order brain areas, such as attention, memory, and expectations. As the two topics have been traditionally studied separately, the mechanisms orchestrating real-world multisensory processing remain unclear. Past work has revealed that the observer's goals gate the influence of many multisensory processes on brain and behavioural responses, whereas some other multisensory processes might occur independently of these goals. Consequently, other forms of top-down control beyond goal dependence are necessary to explain the full range of multisensory effects currently reported at the brain and the cognitive level. These forms of control include sensitivity to stimulus context as well as the detection of matches (or lack thereof) between a multisensory stimulus and categorical attributes of naturalistic objects (e.g. tools, animals). In this review we discuss and integrate the existing findings that demonstrate the importance of such goal-, object- and context-based top-down control over multisensory processing. We then put forward a few principles emerging from this literature review with respect to the mechanisms underlying multisensory processing and discuss their possible broader implications.

Dissociating movement from movement timing in the rat primary motor cortex

Neural encoding of the passage of time to produce temporally precise movements remains an open question. Neurons in several brain regions across different experimental contexts encode estimates of temporal intervals by scaling their activity in proportion to the interval duration. In motor cortex the degree to which this scaled activity relies upon afferent feedback and is guided by motor output remains unclear. Using a neural reward paradigm to dissociate neural activity from motor output before and after complete spinal transection, we show that temporally scaled activity occurs in the rat hindlimb motor cortex in the absence of motor output and after transection. Context-dependent changes in the encoding are plastic, reversible, and re-established following injury. Therefore, in the absence of motor output and despite a loss of afferent feedback, thought necessary for timed movements, the rat motor cortex displays scaled activity during a broad range of temporally demanding tasks similar to that identified in other brain regions.

Role of a looming-sensitive neuron in triggering the defense behavior of the praying mantis Tenodera aridifolia

In responses to looming objects, the praying mantis shows a defense behavior, which consists of retracting forelegs under the prothorax. The role of a looming-sensitive neuron in triggering this behavior was investigated by simultaneously recording the activity and behavioral responses of the neuron. The mantis initiated the defense behavior earlier in response to larger and slower looming stimuli. The time remaining to collision at defense initiation was linearly correlated with the ratio of the half-size of an approaching object to its speed ( l/| v|), suggesting that the defense behavior occurred a fixed delay after the stimuli had reached a fixed angular threshold. Furthermore, the results suggested that high-frequency spikes of the looming-sensitive neuron were involved in triggering the defense behavior: the distribution of maximum firing rate for trials with defense was shifted to larger rates compared with trials without defense; the firing rate of the neuron exceeded 150 Hz ∼100 ms before the defense initiation regardless of stimulus parameters; when a looming stimulus ceased approach prematurely, high-frequency spikes were removed, and the occurrence of defense was reduced.

Organization of columnar inputs in the third optic ganglion of a highly visual crab

Motion information provides essential cues for a wide variety of animal behaviors such as mate, prey, or predator detection. In decapod crustaceans and pterygote insects, visual codification of object motion is associated with visual processing in the third optic neuropile, the lobula. In this neuropile, tangential neurons collect motion information from small field columnar neurons and relay it to the midbrain where behavioral responses would be finally shaped. In highly ordered structures, detailed knowledge of the neuroanatomy can give insight into their function. In spite of the relevance of the lobula in processing motion information, studies on the neuroarchitecture of this neuropile are scant. Here, by applying dextran-conjugated dyes in the second optic neuropile (the medulla) of the crab Neohelice, we mass stained the columnar neurons that convey visual information into the lobula. We found that the arborizations of these afferent columnar neurons lie at four main lobula depths. A detailed examination of serial optical sections of the lobula revealed that these input strata are composed of different number of substrata and that the strata are thicker in the centre of the neuropile. Finally, by staining the different lobula layers composed of tangential processes we combined the present characterization of lobula input strata with the previous characterization of the neuroarchitecture of the crab's lobula based on reduced-silver preparations. We found that the third lobula input stratum overlaps with the dendrites of lobula giant tangential neurons. This suggests that columnar neurons projecting from the medulla can directly provide visual input to the crab's lobula giant neurons.

Computation of object approach by a system of visual motion-sensitive neurons in the crab Neohelice

Similar to most visual animals, crabs perform proper avoidance responses to objects directly approaching them. The monostratified lobula giant neurons of type 1 (MLG1) of crabs constitute an ensemble of 14-16 bilateral pairs of motion-detecting neurons projecting from the lobula (third optic neuropile) to the midbrain, with receptive fields that are distributed over the extensive visual field of the animal's eye. Considering the crab Neohelice (previously Chasmagnathus) granulata, here we describe the response of these neurons to looming stimuli that simulate objects approaching the animal on a collision course. We found that the peak firing time of MLG1 acts as an angular threshold detector signaling, with a delay of δ = 35 ms, the time at which an object reaches a fixed angular threshold of 49°. Using in vivo intracellular recordings, we detected the existence of excitatory and inhibitory synaptic currents that shape the neural response. Other functional features identified in the MLG1 neurons were phasic responses at the beginning of the approach, a relation between the stimulus angular velocity and the excitation delay, and a mapping between membrane potential and firing frequency. Using this information, we propose a biophysical model of the mechanisms that regulate the encoding of looming stimuli. Furthermore, we found that the parameter encoded by the MLG1 firing frequency during the approach is the stimulus angular velocity. The proposed model fits the experimental results and predicts the neural response to a qualitatively different stimulus. Based on these and previous results, we propose that the MLG1 neuron system acts as a directional coding system for collision avoidance.

Asynchronous visual event-based time-to-contact

Reliable and fast sensing of the environment is a fundamental requirement for autonomous mobile robotic platforms. Unfortunately, the frame-based acquisition paradigm at the basis of main stream artificial perceptive systems is limited by low temporal dynamics and redundant data flow, leading to high computational costs. Hence, conventional sensing and relative computation are obviously incompatible with the design of high speed sensor-based reactive control for mobile applications, that pose strict limits on energy consumption and computational load. This paper introduces a fast obstacle avoidance method based on the output of an asynchronous event-based time encoded imaging sensor. The proposed method relies on an event-based Time To Contact (TTC) computation based on visual event-based motion flows. The approach is event-based in the sense that every incoming event adds to the computation process thus allowing fast avoidance responses. The method is validated indoor on a mobile robot, comparing the event-based TTC with a laser range finder TTC, showing that event-based sensing offers new perspectives for mobile robotics sensing.

Luminance and contrast in visual perception of time to collision

Many animals avoid dark, approaching objects seen against a lighter background but show no or weaker reactions to stimuli with inverted contrast. We investigated whether human observers would respond differently to such stimuli in terms of estimated time-to-arrival. We varied luminances of an approaching, light or dark disk and a plain, grey background, and for several conditions, continuously adjusted calibrations so as to keep contrast and/or overall lightness constant. Since no effects were found, we conclude that humans are able to discard luminance and contrast for the task at hand. Generally, however, performance was affected by different, consecutive regimes of feedback: Initially, without feedback, observers responded inconsistently and much too late; they improved after correct feedback, and in a third block of trials with pseudo-random feedback, they responded increasingly early without reverting to the initial level of uncertainty. We discuss our findings with regard to implications for neural mechanisms, put them in the context of evolutionary considerations, and propose continuative animal behavioral studies.

Unifying time to contact estimation and collision avoidance across species

The -function and the -function are phenomenological models that are widely used in the context of timing interceptive actions and collision avoidance, respectively. Both models were previously considered to be unrelated to each other: is a decreasing function that provides an estimation of time-to-contact (ttc) in the early phase of an object approach; in contrast, has a maximum before ttc. Furthermore, it is not clear how both functions could be implemented at the neuronal level in a biophysically plausible fashion. Here we propose a new framework – the corrected modified Tau function – capable of predicting both -type (“”) and -type (“”) responses. The outstanding property of our new framework is its resilience to noise. We show that can be derived from a firing rate equation, and, as , serves to describe the response curves of collision sensitive neurons. Furthermore, we show that predicts the psychophysical performance of subjects determining ttc. Our new framework is thus validated successfully against published and novel experimental data. Within the framework, links between -type and -type neurons are established. Therefore, it could possibly serve as a model for explaining the co-occurrence of such neurons in the brain.

In 1957, Sir Fred Hoyle published a science fiction novel in which he described humanity's encounter with an extraterrestrial life form. It came in the shape of a huge black cloud which approached the earth. Hoyle proposed a formula (“”) for computing the remaining time until contact (“ttc”) of the cloud with the earth. Nowadays in real science, serves as a model for ttc -perception for animals and humans, although it is not entirely undisputed. For instance, seems to be incompatible with a collision-sensitive neuron in locusts (the Lobula Giant Movement Detector or LGMD neuron). LGMD neurons are instead better described by the -function, which differs from . Here we propose a generic model (“”) that contains and as special cases. The validity of the model was confirmed with a psychophysical experiment. Also, we fitted many published response curves of LGMD neurons with our new model and with the -function. Both models fit these response curves well, and we thus can conclude that and possibly result from a generic neuronal circuit template such as it is described by .

A looming-sensitive pathway responds to changes in the trajectory of object motion

Two identified locust neurons, the lobula giant movement detector (LGMD) and its postsynaptic partner, the descending contralateral movement detector (DCMD), constitute one motion-sensitive pathway in the visual system that responds preferentially to objects that approach on a direct collision course and are implicated in collision-avoidance behavior. Previously described responses to the approach of paired objects and approaches at different time intervals (Guest BB, Gray JR. J Neurophysiol 95: 1428–1441, 2006) suggest that this pathway may also be affected by more complicated movements in the locust's visual environment. To test this possibility we presented stationary locusts with disks traveling along combinations of colliding (looming), noncolliding (translatory), and near-miss trajectories. Distinctly different responses to different trajectories and trajectory changes demonstrate that DCMD responds to complex aspects of local visual motion. DCMD peak firing rates associated with the time of collision remained relatively invariant after a trajectory change from translation to looming. Translatory motion initiated in the frontal visual field generated a larger peak firing rate relative to object motion initiated in the posterior visual field, and the peak varied with simulated distance from the eye. Transition from translation to looming produced a transient decrease in the firing rate, whereas transition away from looming produced a transient increase. The change in firing rate at the time of transition was strongly correlated with unique expansion parameters described by the instantaneous angular acceleration of the leading edge and subtense angle of the disk. However, response time remained invariant. While these results may reflect low spatial resolution of the compound eye, they also suggest that this motion-sensitive pathway may be capable of monitoring dynamic expansion properties of objects that change the trajectory of motion.

Visuo-motor transformations involved in the escape response to looming stimuli in the crab Neohelice (=Chasmagnathus) granulata

Escape responses to directly approaching predators represent one instance of an animal's ability to avoid collision. Usually, such responses can be easily evoked in the laboratory using two-dimensional computer simulations of approaching objects, known as looming stimuli. Therefore, escape behaviors are considered useful models for the study of computations performed by the brain to efficiently transform visual information into organized motor patterns. The escape response of the crab Neohelice (previously Chasmagnathus) granulata offers an opportunity to investigate the processing of looming stimuli and its transformation into complex motor patterns. Here we studied the escape performance of this crab to a variety of different looming stimuli. The response always consisted of a vigorous run away from the stimulus. However, the moment at which it was initiated, as well as the developed speed, closely matched the expansion dynamics of each particular stimulus. Thus, we analyzed the response events as a function of several variables that could theoretically be used by the crab (angular size, angular velocity, etc.). Our main findings were that: (1) the decision to initiate the escape run is made when the stimulus angular size increases by 7 deg; (2) the escape run is not a ballistic kind of response, as its speed is adjusted concurrently with changes in the optical stimulus variables; and (3) the speed of the escape run can be faithfully described by a phenomenological input-output relationship based on the stimulus angular increment and the angular velocity of the stimulus.

Sensory basis of vigilance behavior in birds: synthesis and future prospects

Birds gather visual information through scanning behavior to make decisions relevant for survival (e.g., detecting predators and finding food). The goal of this study was (a) to review some visual properties involved in scanning behavior (retinal specialization for visual resolution and motion detection, visual acuity, and size of the blind area), and (b) hypothesize how the inter-specific variability in these properties may lead to different scanning strategies. The avian visual system has a high degree of heterogeneity in visual performance across the visual field, with some sectors providing higher levels of visual resolution and motion detection (e.g., retinal specializations) than others (e.g., peripheral retina and blind area). Thus, information quality will vary in different parts of the visual field, which contradicts some theoretical assumptions on information gathering. Birds need to move their eyes and heads to align the retinal specializations to different sectors of visual space. The rates of eye and head movements can then be used as proxies for scanning strategies. I propose specific predictions as to how each of the visual properties studied can affect scanning strategies in the context of predator detection in different habitat types and with different levels of predation risk. Establishing the degree of association between sensory specializations and scanning strategies can enhance our understanding of the evolution of anti-predator behavior.

Neuronal responses to looming objects in the superior colliculus of the cat

The superior colliculus (SC) in the mammalian mesencephalon is involved in avoidance or escape behaviors, but little is known about the response properties of collicular neurons to an object approaching on a collision course towards the animal. The present study identified two classes of looming-sensitive neurons, rho and eta cells, in the SC of the cat, but did not find any tau cell, which has been observed in the pigeon tectofugal pathway. The looming responses were characterized by distinct firing patterns, in which the neuronal discharge steadily increased as the object was approaching, and peaked approximately at the time of collision (rho cell) or some time earlier (eta cell). The response onset time of both rho and eta cells was linearly related to the square root of the diameter/velocity ratio of looming objects; whereas for eta cells, the response peak time was linearly related to the diameter/velocity ratio. The receptive fields of these collicular cells were composed of an excitatory center and a suppressive surround, but the occurrence and development of neuronal responses to looming stimuli were independent of the receptive-field organization. Although the cell number was relatively small in the deep layers of the SC, the proportion of looming-sensitive neurons was close to that in the superficial layers. These results suggest that a population of collicular cells is involved in signaling impending collision of a looming object with the animal and the neural mechanisms underlying the collision avoidance behaviors are to some extent conservative across species from insects to mammals.

Response properties and receptive field organization of collision-sensitive neurons in the optic tectum of bullfrog, Rana catesbeiana

OBJECTIVE

Many studies have reported that animals will display collision avoidance behavior when the size of retinal image of an object reaches a threshold. The present study aimed to investigate the neural correlates underlying the frog collision avoidance behavior.

METHODS

Different types of visual stimuli simulating the retinal image of an approaching or a recessing object were generated by a computer and presented to the right eye of frog. A multielectrode array was used to examine the activity of collision-sensitive neurons, and single electrode recordings were employed to quantify visual parameter(s) of the frog collision-sensitive neurons.

RESULTS

The multielectrode array revealed that 40 neurons in the optic tectum showed selective responsiveness to objects approaching on a direct collision course. The response profiles of these collision-sensitive neurons were similar to those of lobula giant movement detector (LGMD) in the locust or to those of neurons in the pigeon. However, the receptive field (RF) size of the frog neurons [(18.5+/-3.8) degrees, n=33)] was smaller than those of collision-sensitive neurons of the locust and the pigeon. Multielectrode recordings also showed that the collision-sensitive neurons were activated only when the focus of expansion of a looming retinal image was located within the center of its RF. There was a linear relationship between the parameter l/v (l denotes half-size of the object, v denotes approaching velocity) and time-to-collision (time difference between the peak of the neuronal activity and the predictive collision) in 16 collision-sensitive neurons. Theoretical consideration showed that the peak firing rate always occurred at a fixed delay of (60.1 +/- 39.5) ms (n=16) after the object had reached a constant angular size of (14.8 +/- 3.4) degrees (n=16) on the retina.

CONCLUSION

The results may help clarify the mechanisms underlying the collision avoidance behavior in bullfrog.

Collision-Sensitive Neurons in the Optic Tectum of the Bullfrog, Rana catesbeiana

In this study, we examined the neuronal correlates of frog collision avoidance behavior. Single unit recordings in the optic tectum showed that 11 neurons gave selective responses to objects approaching on a direct collision course. The collision-sensitive neurons exhibited extremely tight tuning for collision bound trajectories with mean half-width at half height values of 0.8 and 0.9° ( n = 4) for horizontal and vertical deviations, respectively. The response of frog collision-sensitive neurons can be fitted by a function that simply multiplies the size dependence of its response, e−αθ( t), by the image's instantaneous angular velocity θ′( t). Using fitting analysis, we showed that the peak firing rate always occurred after the approaching object had reached a constant visual angle of 24.2 ± 2.6° (mean ± SD; n = 8), regardless of the approaching velocity. Moreover, a linear relationship was demonstrated between parameters l/v ( l: object's half-size, v: approach velocity) and time-to-collision (time difference between peak neuronal activity and the predicted collision) in the 11 collision-sensitive neurons. In addition, linear regression analysis was used to show that peak firing rate always occurred after the object had reached a constant angular size of 21.1° on the retina. The angular thresholds revealed by both theoretical analyses were comparable and showed a good agreement with that revealed by our previous behavioral experiments. This strongly suggests that the collision-sensitive neurons of the frog comprise a threshold detector, which triggers collision avoidance behavior.

Escapes with and without preparation: the neuroethology of visual startle in locusts

Locusts respond to the images of approaching (looming) objects with responses that include gliding while in flight and jumping while standing. For both of these responses there is good evidence that the DCMD neuron (descending contralateral movement detector), which carries spike trains from the brain to the thoracic ganglia, is involved. Sudden glides during flight, which cause a rapid loss of height, are last-chance manoeuvres without prior preparation. Jumps from standing require preparation over several tens of milliseconds because of the need to store muscle-derived energy in a catapult-like mechanism. Locusts' DCMD neurons respond selectively to looming stimuli, and make connections with some motor neurons and interneurons known to be involved in flying and jumping. For glides, a burst of high-frequency DCMD spikes is a key trigger. For jumping, a similar burst can influence timing, but neither the DCMD nor any other single interneuron has been shown to be essential for triggering any stage in preparation or take-off. Responses by the DCMD to looming stimuli can alter in different behavioural contexts: in a flying locust, arousal ensures a high level of both DCMD responsiveness and glide occurrence; and there are significant differences in DCMD activity between locusts in the gregarious and the solitarious phase.

A Contribution of Area 5 of the Posterior Parietal Cortex to the Planning of Visually Guided Locomotion: Limb-Specific and Limb-Independent Effects

We tested the hypothesis that area 5 of the posterior parietal cortex (PPC) contributes to the planning of visually guided gait modifications. We recorded 121 neurons from the PPC of two cats during a task in which cats needed to process visual input to step over obstacles attached to a moving treadmill belt. During unobstructed locomotion, 64/121 (53%) of cells showed rhythmic activity. During steps over the obstacles, 102/121 (84%) of cells showed a significant change of their activity. Of these, 46/102 were unmodulated during the control task. We divided the 102 task-related cells into two groups on the basis of their discharge when the limb contralateral to the recording site was the first to pass over the obstacle. One group (41/102) was characterized by a brief, phasic discharge as the lead forelimb passed over the obstacle (Step-related cells). These cells were recorded primarily from area 5a. The other group (61/102) showed a progressive increase in activity prior to the onset of the swing phase in the modified limb and frequently diverged from control at least one step cycle before the gait modification (Step-advanced cells). Most of these cells were recorded in area 5b. In both groups, some cells maintained a fixed relationship to the activity of the contralateral forelimb regardless of which limb was the first to pass over the obstacle (limb-specific cells), whereas others changed their phase of activity so that they were always related to activity of the first limb to pass over the obstacle, either contralateral or ipsilateral (limb-independent cells). Limb-independent cells were more common among the Step-advanced cell population. We suggest that both populations of cells contribute to the gait modification and that the discharge characteristics of the Step-advanced cells are compatible with a contribution to the planning of the gait modification.

Spatiotemporal receptive field properties of a looming-sensitive neuron in solitarious and gregarious phases of the desert locust

Desert locusts (Schistocerca gregaria) can transform reversibly between the swarming gregarious phase and a solitarious phase, which avoids other locusts. This transformation entails dramatic changes in morphology, physiology, and behavior. We have used the lobula giant movement detector (LGMD) and its postsynaptic target, the descending contralateral movement detector (DCMD), which are visual interneurons that detect looming objects, to analyze how differences in the visual ecology of the two phases are served by altered neuronal function. Solitarious locusts had larger eyes and a greater degree of binocular overlap than those of gregarious locusts. The receptive field to looming stimuli had a large central region of nearly equal response spanning 120 degrees x 60 degrees in both phases. The DCMDs of gregarious locusts responded more strongly than solitarious locusts and had a small caudolateral focus of even further sensitivity. More peripherally, the response was reduced in both phases, particularly ventrally, with gregarious locusts showing greater proportional decrease. Gregarious locusts showed less habituation to repeated looming stimuli along the eye equator than did solitarious locusts. By contrast, in other parts of the receptive field the degree of habituation was similar in both phases. The receptive field organization to looming stimuli contrasts strongly with the receptive field organization of the same neurons to nonlooming local-motion stimuli, which show much more pronounced regional variation. The DCMDs of both gregarious and solitarious locusts are able to detect approaching objects from across a wide expanse of visual space, but phase-specific changes in the spatiotemporal receptive field are linked to lifestyle changes.

Neural basis of postural instability identified by VTC and EEG

In this study, we investigated the neural basis of virtual time to contact (VTC) and the hypothesis that VTC provides predictive information for future postural instability. A novel approach to differentiate stable pre-falling and transition-to-instability stages within a single postural trial while a subject was performing a challenging single leg stance with eyes closed was developed. Specifically, we utilized wavelet transform and stage segmentation algorithms using VTC time series data set as an input. The VTC time series was time-locked with multichannel (n = 64) EEG signals to examine its underlying neural substrates. To identify the focal sources of neural substrates of VTC, a two-step approach was designed combining the independent component analysis (ICA) and low-resolution tomography (LORETA) of multichannel EEG. There were two major findings: (1) a significant increase of VTC minimal values (along with enhanced variability of VTC) was observed during the transition-to-instability stage with progression to ultimate loss of balance and falling; and (2) this VTC dynamics was associated with pronounced modulation of EEG predominantly within theta, alpha and gamma frequency bands. The sources of this EEG modulation were identified at the cingulate cortex (ACC) and the junction of precuneus and parietal lobe, as well as at the occipital cortex. The findings support the hypothesis that the systematic increase of minimal values of VTC concomitant with modulation of EEG signals at the frontal-central and parietal-occipital areas serve collectively to predict the future instability in posture.

Contributions of the human temporoparietal junction and MT/V5+ to the timing of interception revealed by transcranial magnetic stimulation

To intercept a fast target at destination, hand movements must be centrally triggered ahead of target arrival to compensate for neuromechanical delays. The role of visual-motion cortical areas is unclear. They likely feed downstream parietofrontal networks with signals reflecting target motion, but do they also contribute internal timing signals to trigger the motor response? We disrupted the activity of human temporoparietal junction (TPJ) and middle temporal area (hMT/V5+) by means of transcranial magnetic stimulation (TMS) while subjects pressed a button to intercept targets accelerated or decelerated in the vertical or horizontal direction. Target speed was randomized, making arrival time unpredictable across trials. We used either repetitive TMS (rTMS) before task execution or double-pulse TMS (dpTMS) during target motion. We found that after rTMS and dpTMS at 100-200 ms from motion onset, but not after dpTMS at 300-400 ms, the button-press responses occurred earlier than in the control, with time shifts independent of target speed. This suggests that activity in TPJ and hMT/V5+ can feed downstream regions not only with visual-motion information, but also with internal timing signals used for interception at destination. Moreover, we found that TMS of hMT/V5+ affected interception of all tested motion types, whereas TMS of TPJ significantly affected only interception of motion coherent with natural gravity. TPJ might specifically gate visual-motion information according to an internal model of the effects of gravity.

Arousal facilitates collision avoidance mediated by a looming sensitive visual neuron in a flying locust

Locusts have two large collision-detecting neurons, the descending contralateral movement detectors (DCMDs) that signal object approach and trigger evasive glides during flight. We sought to investigate whether vision for action, when the locust is in an aroused state rather than a passive viewer, significantly alters visual processing in this collision-detecting pathway. To do this we used two different approaches to determine how the arousal state of a locust affects the prolonged periods of high-frequency spikes typical of the DCMD response to approaching objects that trigger evasive glides. First, we manipulated arousal state in the locust by applying a brief mechanical stimulation to the hind leg; this type of change of state occurs when gregarious locusts accumulate in high-density swarms. Second, we examined DCMD responses during flight because flight produces a heightened physiological state of arousal in locusts. When arousal was induced by either method we found that the DCMD response recovered from a previously habituated state; that it followed object motion throughout approach; and--most important--that it was significantly more likely to generate the maintained spike frequencies capable of evoking gliding dives even with extremely short intervals (1.8 s) between approaches. Overall, tethered flying locusts responded to 41% of simulated approaching objects (sets of 6 with 1.8 s ISI). When we injected epinastine, the neuronal octopamine receptor antagonist, into the hemolymph responsiveness declined to 12%, suggesting that octopamine plays a significant role in maintaining responsiveness of the DCMD and the locust to visual stimuli during flight.

Nonlinear integration of binocular optic flow by DNOVS2, a descending neuron of the fly

For visual orientation and course stabilization, flies rely heavily on the optic flow perceived by the animal during flight. The processing of optic flow is performed in motion-sensitive tangential cells of the lobula plate, which are well described with respect to their visual response properties and the connectivity among them. However, little is known about the postsynaptic descending neurons, which convey motion information to the motor circuits in the thoracic ganglion. Here we investigate the physiology and connectivity of an identified premotor descending neuron, called DNOVS2 (for descending neuron of the ocellar and vertical system). We find that DNOVS2 is tuned in a supralinear way to rotation around the longitudinal body axis. Experiments involving stimulation of the ipsilateral and the contralateral eye indicate that ipsilateral computation of motion information is modified nonlinearly by motion information from the contralateral eye. Performing double recordings of DNOVS2 and lobula plate tangential cells, we find that DNOVS2 is connected ipsilaterally to a subset of vertical-sensitive cells. From the contralateral eye, DNOVS2 receives input most likely from V2, a heterolateral spiking neuron. This specific neural circuit is sufficient for the tuning of DNOVS2, making it probably an important element in optomotor roll movements of the head and body around the fly's longitudinal axis.