Vision for navigation: What can we learn from ants?

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer representing olfactory stimuli and linking them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their function remains unclear. Here, we identify inputs to visual Kenyon cells using the FlyWire adult whole-brain connectome. Input repertoires are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual neurons presynaptic to Kenyon cells have large receptive fields, while interneuron inputs receive spatially restricted signals that may be tuned to specific visual features. Individual visual Kenyon cells randomly sample sparse inputs from combinations of visual channels, including multiple optic lobe neuropils. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the specific input repertoire to the smaller population of visual Kenyon cells suggests a constrained encoding of visual stimuli.

Diversity of visual inputs to Kenyon cells of the Drosophila mushroom body

The arthropod mushroom body is well-studied as an expansion layer that represents olfactory stimuli and links them to contingent events. However, 8% of mushroom body Kenyon cells in Drosophila melanogaster receive predominantly visual input, and their tuning and function are poorly understood. Here, we use the FlyWire adult whole-brain connectome to identify inputs to visual Kenyon cells. The types of visual neurons we identify are similar across hemispheres and connectomes with certain inputs highly overrepresented. Many visual projection neurons presynaptic to Kenyon cells receive input from large swathes of visual space, while local visual interneurons, providing smaller fractions of input, receive more spatially restricted signals that may be tuned to specific features of the visual scene. Like olfactory Kenyon cells, visual Kenyon cells receive sparse inputs from different combinations of visual channels, including inputs from multiple optic lobe neuropils. The sets of inputs to individual visual Kenyon cells are consistent with random sampling of available inputs. These connectivity patterns suggest that visual coding in the mushroom body, like olfactory coding, is sparse, distributed, and combinatorial. However, the expansion coding properties appear different, with a specific repertoire of visual inputs projecting onto a relatively small number of visual Kenyon cells.

Impact of central complex lesions on innate and learnt visual navigation in ants

Wood ants are excellent navigators, using a combination of innate and learnt navigational strategies to travel between their nest and feeding sites. Visual navigation in ants has been studied extensively, however, we have little direct evidence for the underlying neural mechanisms. Here, we perform lateralized mechanical lesions in the central complex (CX) of wood ants, a midline structure known to allow an insect to keep track of the direction of sensory cues relative to its own orientation and to control movement. We lesioned two groups of ants and observed their behaviour in an arena with a large visual landmark present. The first group of ants were naïve and when intact such ants show a clear innate attraction to the conspicuous landmark. The second group of ants were trained to aim to a food location to the side of the landmark. The general heading of naïve ants towards a visual cue was not altered by the lesions, but the heading of ants trained to a landmark adjacent food position was affected. Thus, CX lesions had a specific impact on learnt visual guidance. We also observed that lateralised lesions altered the fine details of turning with lesioned ants spending less time turning to the side ipsilateral of the lesion. The results confirm the role of the CX in turn control and highlight its important role in the implementation of learnt behaviours that rely on information from other brain regions.

Innate visual attraction in wood ants is a hardwired behavior seen across different motivational and ecological contexts

Ants are expert navigators combining innate and learnt navigational strategies. Whereas we know that the ants' feeding state segregates visual-navigational memories in ants navigating along a learnt route, it is an open question if the motivational state also affects the ants' innate visual preferences. Wood ant foragers show an innate attraction to conspicuous visual cues. These foragers inhabit cluttered woodland habitat and feed on honeydew from aphids on trees. Hence, the attraction to 'tree-like' objects might be an ecologically relevant behavior that is tailored to the wood ants' foraging ecology. Foragers from other ant species with different foraging ecologies show very different innate attractions. We investigated here the innate visual response of wood ant foragers with different motivational states, i.e., unfed or fed, as well as males that show no foraging activity. Our results show that ants from all three groups orient toward a prominent visual cue, i.e., this intrinsic visuomotor response is not context-dependent, but a hardwired behavior seen across different motivational and ecological contexts.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s00040-022-00867-3.

Insights into insect cognition through mirror-induced behaviour in paper wasps

Mirror studies have uncovered evidence for self-awareness in several non-human animals; however, this approach has not been extended to invertebrates. Here, we examined mirror responses of Polistes paper wasps, insects with high social cognitive abilities. We used a highly replicated design, exposing wasps to mirror self-recognition (MSR) test stimuli (mirror, facial marking) and several control stimuli (non-reflective surfaces, invisible marks, and sham marks). We found that nest-founding queen wasps respond to mirrors with increases in antennation and touching over non-reflective surfaces. Visible marks also resulted in more exploratory behaviours (antennating and touching the mirror), but decreases in self-directed behaviours typically used as indicators of MSR. While this experiment does not support the notion that wasps possess self-awareness, it shows that wasps respond to subtle alterations in their visual environment and appearance with exploratory behaviour. This work thus increases our understanding of insects’ cognitive abilities, suggesting high levels of perceptual richness.

Approach Direction Prior to Landing Explains Patterns of Colour Learning in Bees

Gaze direction is closely coupled with body movement in insects and other animals. If movement patterns interfere with the acquisition of visual information, insects can actively adjust them to seek relevant cues. Alternatively, where multiple visual cues are available, an insect's movements may influence how it perceives a scene. We show that the way a foraging bumblebee approaches a floral pattern could determine what it learns about the pattern. When trained to vertical bicoloured patterns, bumblebees consistently approached from below centre in order to land in the centre of the target where the reward was located. In subsequent tests, the bees preferred the colour of the lower half of the pattern that they predominantly faced during the approach and landing sequence. A predicted change of learning outcomes occurred when the contrast line was moved up or down off-centre: learned preferences again reflected relative frontal exposure to each colour during the approach, independent of the overall ratio of colours. This mechanism may underpin learning strategies in both simple and complex visual discriminations, highlighting that morphology and action patterns determines how animals solve sensory learning tasks. The deterministic effect of movement on visual learning may have substantially influenced the evolution of floral signals, particularly where plants depend on fine-scaled movements of pollinators on flowers.

Movements, embodiment and the emergence of decisions. Insights from insect navigation

We readily infer that animals make decisions, but what this implies is usually not clearly defined. The notion of 'decision-making' ultimately stems from human introspection, and is thus loaded with anthropomorphic assumptions. Notably, the decision is made internally, is based on information, and precedes the goal directed behaviour. Also, making a decision implies that 'something' did it, thus hints at the presence of a cognitive mind, whose existence is independent of the decision itself. This view may convey some truth, but here I take the opposite stance. Using examples from research in insect navigation, this essay highlights how apparent decisions can emerge without a brain, how actions can precede information or how sophisticated goal directed behaviours can be implemented without neural decisions. This perspective requires us to shake off the idea that behaviour is a consequence of the brain; and embrace the concept that movements arise from - as much as participate in - distributed interactions between various computational centres - including the body - that reverberate in closed-loop with the environment. From this perspective we may start to picture how a cognitive mind can be the consequence, rather than the cause, of such neural and body movements.

Towards a multi-level understanding in insect navigation

To understand the brain is to understand behaviour. However, understanding behaviour itself requires consideration of sensory information, body movements and the animal's ecology. Therefore, understanding the link between neurons and behaviour is a multi-level problem, which can be achieved when considering Marr's three levels of understanding: behaviour, computation, and neural implementation. Rather than establishing direct links between neurons and behaviour, the matter boils down to understanding two transitions: the link between neurons and brain computation on one hand, and the link between brain computations and behaviour on the other hand. The field of insect navigation illustrates well the power of such two-sided endeavour. We provide here examples revealing that each transition requires its own approach with its own intrinsic difficulties, and show how modelling can help us reach the desired multi-level understanding.

The Antarium: A Reconstructed Visual Reality Device for Ant Navigation Research

We constructed a large projection device (the Antarium) with 20,000 UV-Blue-Green LEDs that allows us to present tethered ants with views of their natural foraging environment. The ants walk on an air-cushioned trackball, their movements are registered and can be fed back to the visual panorama. Views are generated in a 3D model of the ants’ environment so that they experience the changing visual world in the same way as they do when foraging naturally. The Antarium is a biscribed pentakis dodecahedron with 55 facets of identical isosceles triangles. The length of the base of the triangles is 368 mm resulting in a device that is roughly 1 m in diameter. Each triangle contains 361 blue/green LEDs and nine UV LEDs. The 55 triangles of the Antarium have 19,855 Green and Blue pixels and 495 UV pixels, covering 360° azimuth and elevation from −50° below the horizon to +90° above the horizon. The angular resolution is 1.5° for Green and Blue LEDs and 6.7° for UV LEDs, offering 65,536 intensity levels at a flicker frequency of more than 9,000 Hz and a framerate of 190 fps. Also, the direction and degree of polarisation of the UV LEDs can be adjusted through polarisers mounted on the axles of rotary actuators. We build 3D models of the natural foraging environment of ants using purely camera-based methods. We reconstruct panoramic scenes at any point within these models, by projecting panoramic images onto six virtual cameras which capture a cube-map of images to be projected by the LEDs of the Antarium. The Antarium is a unique instrument to investigate visual navigation in ants. In an open loop, it allows us to provide ants with familiar and unfamiliar views, with completely featureless visual scenes, or with scenes that are altered in spatial or spectral composition. In closed-loop, we can study the behavior of ants that are virtually displaced within their natural foraging environment. In the future, the Antarium can also be used to investigate the dynamics of navigational guidance and the neurophysiological basis of ant navigation in natural visual environments.

Vision does not impact walking performance in Argentine ants

Many walking insects use vision for long-distance navigation, but the influence of vision on rapid walking performance that requires close-range obstacle detection and directing the limbs towards stable footholds remains largely untested. We compared Argentine ant (Linepithema humile) workers in light versus darkness while traversing flat and uneven terrain. In darkness, ants reduced flat-ground walking speeds by only 5%. Similarly, the approach speed and time to cross a step obstacle were not significantly affected by lack of lighting. To determine whether tactile sensing might compensate for vision loss, we tracked antennal motion and observed shifts in spatiotemporal activity as a result of terrain structure but not illumination. Together, these findings suggest that vision does not impact walking performance in Argentine ant workers. Our results help contextualize eye variation across ants, including subterranean, nocturnal and eyeless species that walk in complete darkness. More broadly, our findings highlight the importance of integrating vision, proprioception and tactile sensing for robust locomotion in unstructured environments.

The Neuroscience of Spatial Navigation and the Relationship to Artificial Intelligence

Recent advances in artificial intelligence (AI) and neuroscience are impressive. In AI, this includes the development of computer programs that can beat a grandmaster at GO or outperform human radiologists at cancer detection. A great deal of these technological developments are directly related to progress in artificial neural networks-initially inspired by our knowledge about how the brain carries out computation. In parallel, neuroscience has also experienced significant advances in understanding the brain. For example, in the field of spatial navigation, knowledge about the mechanisms and brain regions involved in neural computations of cognitive maps-an internal representation of space-recently received the Nobel Prize in medicine. Much of the recent progress in neuroscience has partly been due to the development of technology used to record from very large populations of neurons in multiple regions of the brain with exquisite temporal and spatial resolution in behaving animals. With the advent of the vast quantities of data that these techniques allow us to collect there has been an increased interest in the intersection between AI and neuroscience, many of these intersections involve using AI as a novel tool to explore and analyze these large data sets. However, given the common initial motivation point-to understand the brain-these disciplines could be more strongly linked. Currently much of this potential synergy is not being realized. We propose that spatial navigation is an excellent area in which these two disciplines can converge to help advance what we know about the brain. In this review, we first summarize progress in the neuroscience of spatial navigation and reinforcement learning. We then turn our attention to discuss how spatial navigation has been modeled using descriptive, mechanistic, and normative approaches and the use of AI in such models. Next, we discuss how AI can advance neuroscience, how neuroscience can advance AI, and the limitations of these approaches. We finally conclude by highlighting promising lines of research in which spatial navigation can be the point of intersection between neuroscience and AI and how this can contribute to the advancement of the understanding of intelligent behavior.

Dynamic multimodal interactions in navigating wood ants: what do path details tell us about cue integration?

Ants are expert navigators, using multimodal information to navigate successfully. Here, we present the results of systematic studies of multimodal cue use in navigating wood ants, Formica rufa Ants learnt to navigate to a feeder that was defined by an olfactory cue (O), visual cue (V) and airflow (A) presented together. When the feeder, olfactory cue and airflow were all placed at the centre of the visual cue (VOACentre), ants did not directly approach the learnt feeder when either the olfactory or visual cue was removed. This confirms that some form of cue binding has taken place. However, in a visually simpler task with the feeder located at the edge of the visual cue (VOAEdge), ants still approached the feeder directly when individual cue components were removed. Hence, cue binding is flexible and depends on the navigational context. In general, cues act additively in determining the ants' path accuracy, i.e. the use of multiple cues increased navigation performance. Moreover, across different training conditions, we saw different motor patterns in response to different sensory cues. For instance, ants had more sinuous paths with more turns when they followed an odour plume but did not have any visual cues. Having visual information together with the odour enhanced performance and therefore positively impacted on plume following. Interestingly, path characteristics of ants from the different multimodal groups (VOACentre versus VOAEdge) were different, suggesting that the observed flexibility in cue binding may be a result of ants' movement characteristics.

The role of attractive and repellent scene memories in ant homing (Myrmecia croslandi)

Solitary foraging ants rely on vision when travelling along routes and when pinpointing their nest. We tethered foragers of Myrmecia croslandi on a trackball and recorded their intended movements when the trackball was located on their normal foraging corridor (on-route), above their nest and at a location several meters away where they have never been before (off-route). We find that at on- and off-route locations, most ants walk in the nest or foraging direction and continue to do so for tens of metres in a straight line. In contrast, above the nest, ants walk in random directions and change walking direction frequently. In addition, the walking direction of ants above the nest oscillates at a fine scale, reflecting search movements that are absent from the paths of ants at the other locations. An agent-based simulation shows that the behaviour of ants at all three locations can be explained by the integration of attractive and repellent views directed towards or away from the nest, respectively. Ants are likely to acquire such views via systematic scanning movements during their learning walks. The model predicts that ants placed in a completely unfamiliar environment should behave as if at the nest, which our subsequent experiments confirmed. We conclude first, that the ants’ behaviour at release sites is exclusively driven by what they currently see and not by information on expected outcomes of their behaviour. Second, that navigating ants might continuously integrate attractive and repellent visual memories. We discuss the benefits of such a procedure.

Carpenter ants use diverse antennae sampling strategies to track odor trails

Directed and meaningful animal behavior depends on the ability to sense key features in the environment. Among the different environmental signals, olfactory cues are critically important for foraging, navigation and social communication in many species, including ants. Ants use their two antennae to explore the olfactory world, but how they do so remains largely unknown. In this study, we used high-resolution videography to characterize the antennae dynamics of carpenter ants (Camponotus pennsylvanicus). Antennae are highly active during both odor tracking and exploratory behavior. When tracking, ants used several distinct behavioral strategies with stereotyped antennae sampling patterns (which we call 'sinusoidal', 'probing' and 'trail following'). In all behaviors, left and right antennae movements were anti-correlated, and tracking ants exhibited biases in the use of left versus right antenna to sample the odor trail. These results suggest non-redundant roles for the two antennae. In one of the behavioral modules (trail following), ants used both antennae to detect trail edges and direct subsequent turns, suggesting a specialized form of tropotaxis. Lastly, removal of an antenna resulted not only in less accurate tracking but also in changes in the sampling pattern of the remaining antenna. Our quantitative characterization of odor trail tracking lays a foundation to build better models of olfactory sensory processing and sensorimotor behavior in terrestrial insects.

How to Navigate in Different Environments and Situations: Lessons From Ants

Ants are a globally distributed insect family whose members have adapted to live in a wide range of different environments and ecological niches. Foraging ants everywhere face the recurring challenge of navigating to find food and to bring it back to the nest. More than a century of research has led to the identification of some key navigational strategies, such as compass navigation, path integration, and route following. Ants have been shown to rely on visual, olfactory, and idiothetic cues for navigational guidance. Here, we summarize recent behavioral work, focusing on how these cues are learned and stored as well as how different navigational cues are integrated, often between strategies and even across sensory modalities. Information can also be communicated between different navigational routines. In this way, a shared toolkit of fundamental navigational strategies can lead to substantial flexibility in behavioral outcomes. This allows individual ants to tune their behavioral repertoire to different tasks (e.g., foraging and homing), lifestyles (e.g., diurnal and nocturnal), or environments, depending on the availability and reliability of different guidance cues. We also review recent anatomical and physiological studies in ants and other insects that have started to reveal neural correlates for specific navigational strategies, and which may provide the beginnings of a truly mechanistic understanding of navigation behavior.

Fractal dimension and the navigational information provided by natural scenes

Recent work on virtual reality navigation in humans has suggested that navigational success is inversely correlated with the fractal dimension (FD) of artificial scenes. Here we investigate the generality of this claim by analysing the relationship between the fractal dimension of natural insect navigation environments and a quantitative measure of the navigational information content of natural scenes. We show that the fractal dimension of natural scenes is in general inversely proportional to the information they provide to navigating agents on heading direction as measured by the rotational image difference function (rotIDF). The rotIDF determines the precision and accuracy with which the orientation of a reference image can be recovered or maintained and the range over which a gradient descent in image differences will find the minimum of the rotIDF, that is the reference orientation. However, scenes with similar fractal dimension can differ significantly in the depth of the rotIDF, because FD does not discriminate between the orientations of edges, while the rotIDF is mainly affected by edge orientation parallel to the axis of rotation. We present a new equation for the rotIDF relating navigational information to quantifiable image properties such as contrast to show (1) that for any given scene the maximum value of the rotIDF (its depth) is proportional to pixel variance and (2) that FD is inversely proportional to pixel variance. This contrast dependence, together with scene differences in orientation statistics, explains why there is no strict relationship between FD and navigational information. Our experimental data and their numerical analysis corroborate these results.

Early foraging life: spatial and temporal aspects of landmark learning in the ant Cataglyphis noda

Within the powerful navigational toolkit implemented in desert ants, path integration and landmark guidance are the key routines. Here, we use cue-conflict experiments to investigate the interplay between these two routines in ants, Cataglyphis noda, which start their foraging careers (novices) with learning walks and are then tested at different stages of experience. During their learning walks, the novices take nest-centered views from various directions around the nest. In the present experiments, these learning walks are spatially restricted by arranging differently sized water moats around the nest entrance and thus, limiting the space available around the nest and the nest-feeder route. First, we show that the ants are able to return to the nest by landmark guidance only when the novices have had enough space around the nest entrance for properly performing their learning walks. Second, in 180° cue-conflict situations between path integration and landmark guidance, path integration dominates in the beginning of foraging life (after completion of the learning walks), but with increasing numbers of visits to a familiar feeder landmark guidance comes increasingly into play.