From static electric images to electric flow: towards dynamic perceptual cues in active electroreception

Mormyrid fish as models for investigating sensory-motor integration: A behavioural perspective

Animals possess senses which gather information from their environment. They can tune into important aspects of this information and decide on the most appropriate response, requiring coordination of their sensory and motor systems. This interaction is bidirectional. Animals can actively shape their perception with self-driven motion, altering sensory flow to maximise the environmental information they are able to extract. Mormyrid fish are excellent candidates for studying sensory-motor interactions, because they possess a unique sensory system (the active electric sense) and exhibit notable behaviours that seem to be associated with electrosensing. This review will take a behavioural approach to unpicking this relationship, using active electrolocation as an example where body movements and sensing capabilities are highly related and can be assessed in tandem. Active electrolocation is the process where individuals will generate and detect low-voltage electric fields to locate and recognise nearby objects. We will focus on research in the mormyrid Gnathonemus petersii (G. petersii), given the extensive study of this species, particularly its object recognition abilities. By studying object detection and recognition, we can assess the potential benefits of self-driven movements to enhance selection of biologically relevant information. Finally, these findings are highly relevant to understanding the involvement of movement in shaping the sensory experience of animals that use other sensory modalities. Understanding the overlap between sensory and motor systems will give insight into how different species have become adapted to their environments.

The Use of Supervised Learning Models in Studying Agonistic Behavior and Communication in Weakly Electric Fish

Despite considerable advances, studying electrocommunication of weakly electric fish, particularly in pulse-type species, is challenging as very short signal epochs at variable intervals from a few hertz up to more than 100 Hz need to be assigned to individuals. In this study, we show that supervised learning approaches offer a promising tool to automate or semiautomate the workflow, and thereby allowing the analysis of much longer episodes of behavior in a reasonable amount of time. We provide a detailed workflow mainly based on open resource software. We demonstrate the usefulness by applying the approach to the analysis of dyadic interactions of Gnathonemus petersii. Coupling of the proposed methods with a boundary element modeling approach, we are thereby able to model the information gained and provided during agonistic encounters. The data indicate that the passive electrosensory input, in particular, provides sufficient information to localize a contender during the pre-contest phase, fish did not use or rely on the theoretically also available sensory information of the contest outcome-determining size difference between contenders before engaging in agonistic behavior.

Task-Related Sensorimotor Adjustments Increase the Sensory Range in Electrolocation

Perception and motor control traditionally are studied separately. However, motor activity can serve as a scaffold to shape the sensory flow. This tight link between motor actions and sensing is particularly evident in active sensory systems. Here, we investigate how the weakly electric mormyrid fish Gnathonemus petersii of undetermined sex structure their sensing and motor behavior while learning a perceptual task. We find systematic adjustments of the motor behavior that correlate with an increased performance. Using a model to compute the electrosensory input, we show that these behavioral adjustments improve the sensory input. As we find low neuronal detection thresholds at the level of medullary electrosensory neurons, it seems that the behavior-driven improvements of the sensory input are highly suitable to overcome the sensory limitations, thereby increasing the sensory range. Our results show that motor control is an active component of sensory learning, demonstrating that a detailed understanding of contribution of motor actions to sensing is needed to understand even seemingly simple behaviors.SIGNIFICANCE STATEMENT Motor-guided sensation and perception are intertwined, with motor behavior serving as a scaffold to shape the sensory input. We characterized how the weakly electric mormyrid fish Gnathonemus petersii, as it learns a perceptual task, restructures its sensorimotor behavior. We find that systematic adjustments of the motor behavior correlate with increased performance and a shift of the sensory attention of the animal. Analyzing the afferent electrosensory input shows that a significant gain in information results from these sensorimotor adjustments. Our results show that motor control can be an active component of sensory learning. Researching the sensory corollaries of motor control thus can be crucial to understand sensory sensation and perception under naturalistic conditions.

Sensory Flow as a Basis for a Novel Distance Cue in Freely Behaving Electric Fish

The sensory input that an animal receives is directly linked to its motor activity. Behavior thus enables animals to influence their sensory input, a concept referred to as active sensing. How such behavior can serve as a scaffold for generating sensory information is of general scientific interest. In this article, we investigate how behavior can shape sensory information by using some unique features of the sensorimotor system of the weakly electric fish. Based on quantitative behavioral characterizations and computational reconstruction of sensory input, we show how electrosensory flow is actively created during highly patterned, spontaneous behavior in Gnathonemus petersii. The spatiotemporal structure of the sensory input provides information for the computation of a novel distance cue, which allows for a continuous estimation of distance. This has significant advantages over previously known nondynamic distance estimators as determined from electric image blur. Our investigation of the sensorimotor interactions in pulsatile electrolocation shows, for the first time, that the electrosensory flow contains behaviorally relevant information accessible only through active behavior. As patterned sensory behaviors are a shared feature of (active) sensory systems, our results have general implications for the understanding of (active) sensing, with the proposed sensory flow-based measure being potentially pertinent to a broad range of sensory modalities.

SIGNIFICANCE STATEMENT

Acquisition of sensory information depends on motion, as either an animal or its sensors move. Behavior can thus actively influence the sensory flow; and in this way, behavior can be seen as a manifestation of the brain's integrative functions. The properties of the active pulsatile electrolocation system in Gnathonemus petersii allow for the sensory input to be computationally reconstructed, enabling us to link the informational content of spatiotemporal sensory dynamics to behavior. Our study reveals a novel sensory cue for estimating depth that is actively generated by the fishes' behavior. The physical and behavioral similarities between electrolocation and other active sensory systems suggest that this may be a mechanism shared by (active) sensory systems.

Model based estimation of ellipsoidal object using artificial electric sense

In this article we address the issue of shape estimation using electric sense inspired by the active electric fish. These fish can perceive their environment by measuring the perturbations in a self-generated electric field caused by nearby objects. The approach proceeded in three stages. Firstly, the object was detected and its electric properties (insulator or conductor) identified. Secondly, the object was localized using the multiple signal classification algorithm, which was originally developed to localize a radio wave emitter using a network of antennas. Thirdly, the shape estimation relied on the concept of generalized polarization tensor, which enabled us to model the electric response of an object polarized by an ambient electric field. We describe the implementation of the approach through numerous experiments. The system was able to estimate shape with an average error of 16%, and opened the way toward further improvements. In particular, self-aligning the sensor with the ellipsoid through a reactive feedback makes the shape estimation errors drop to 10%.

The bioinspiring potential of weakly electric fish

Electric fish are privileged animals for bio-inspiring man-built autonomous systems since they have a multimodal sense that allows underwater navigation, object classification and intraspecific communication. Although there are taxon dependent variations adapted to different environments, this multimodal system can be schematically described as having four main components: active electroreception, passive electroreception, lateral line sense and, proprioception. Amongst these sensory modalities, proprioception and electroreception show 'active' systems that extrct information carried by self generated forms of energy. This ensemble of four sensory modalities is present in African mormyriformes and American gymnotiformes. The convergent evolution of similar imaging, peripheral encoding, and central processing mechanisms suggests that these mechanisms may be the most suitable for dealing with electric images in the context of the other and self generated actions. This review deals with the way in which biological organisms address three of the problems that are faced when designing a bioinspired electroreceptive agent: (a) body shape, material and mobility, (b) peripheral encoding of electric images, and (c) early processing of electrosensory signals. Taking into account biological solutions I propose that the new generation of underwater agents should have electroreceptive arms, use complex peripheral sensors for encoding the images and cerebellum like architecture for image feature extraction and implementing sensory-motor transformations.

An active electrolocation catheter system for imaging and analysis of coronary plaques

Coronary artery disease-currently one of the most frequent causes of death-is characterized by atherosclerotic plaques grown in the wall of blood vessels and inhibiting blood flow. Preventive assessment focusses on critical sizes of structural plaque parameters like relative lipid core area and cap thickness to identify high-risk plaques called thin cap fibroatheromas. Although state-of-the-art catheter systems were successfully applied in invasive plaque diagnostics, the high costs induced by these devices inhibit usage in daily clinical practice. To overcome this shortcoming, we follow a biomimetic approach to construct a prospective low-cost catheter system that adapts the active electrolocation principles of weakly electric fish Gnathonemus petersii. Only a few and simple parameters relevant for plaque detection and characterization are estimated from plaque-evoked electric images which are projected on the surface of the catheter. Two prototypical electrolocation catheter systems were tested. The first catheter system featured a ring electrode catheter and was used to obtain dynamic 1D electric images of synthetic plaques in an agarose atherosclerosis model. Our proof of concept showed that synthetic plaques could be reliably detected from 1D electric images. Based on a cluster analysis of selected key image features, synthetic plaques could be categorized into four plaque conditions, predefined from thresholds for critical structural parameters, representing high to low risk plaques. In the second recording approach, plaque-evoked dynamic and static spatial electric images were obtained by a multi-electrode catheter system. Based on these recordings, a synthetic plaque with a critical cap thickness could be detected and localized in a pig coronary artery.

Object discrimination through active electrolocation: Shape recognition and the influence of electrical noise

The weakly electric fish Gnathonemus petersii can recognise objects using active electrolocation. Here, we tested two aspects of object recognition; first whether shape recognition might be influenced by movement of the fish, and second whether object discrimination is affected by the presence of electrical noise from conspecifics. (i) Unlike other object features, such as size or volume, no parameter within a single electrical image has been found that encodes object shape. We investigated whether shape recognition might be facilitated by movement-induced modulations (MIM) of the set of electrical images that are created as a fish swims past an object. Fish were trained to discriminate between pairs of objects that either created similar or dissimilar levels of MIM of the electrical images. As predicted, the fish were able to discriminate between objects up to a longer distance if there was a large difference in MIM between the objects than if there was a small difference. This supports an involvement of MIMs in shape recognition but the use of other cues cannot be excluded. (ii) Electrical noise might impair object recognition if the noise signals overlap with the EODs of an electrolocating fish. To avoid jamming, we predicted that fish might employ pulsing strategies to prevent overlaps. To investigate the influence of electrical noise on discrimination performance, two fish were tested either in the presence of a conspecific or of playback signals and the electric signals were recorded during the experiments. The fish were surprisingly immune to jamming by conspecifics: While the discrimination performance of one fish dropped to chance level when more than 22% of its EODs overlapped with the noise signals, the performance of the other fish was not impaired even when all its EODs overlapped. Neither of the fish changed their pulsing behaviour, suggesting that they did not use any kind of jamming avoidance strategy.

Passive and active electroreception during agonistic encounters in the weakly electric fish Gymnotus omarorum

Agonistic behaviour related to territorial defence is likely to be costly in terms of energy loss and risk of injury. Hence information about the fighting ability of a potential opponent could influence the outcome of the contest. We here study electric images of the territorial and aggressive weakly electric fish Gymnotus omarorum in the context of agonistic behaviour. We show that passive and active electric images may drive the approach towards an opponent. The likelihood of first attacks can be predicted in these fish based on electric image information, suggesting that aggressive interactions may in fact be triggered through the passive electrosensory information.

Reactive underwater object inspection based on artificial electric sense

Weakly electric fish can perform complex cognitive tasks based on extracting information from blurry electric images projected from their immediate environment onto their electro-sensitive skin. In particular they can be trained to recognize the intrinsic properties of objects such as their shape, size and electric nature. They do this by means of novel perceptual strategies that exploit the relations between the physics of a self-generated electric field, their body morphology and the ability to perform specific movement termed probing motor acts (PMAs). In this article we artificially reproduce and combine these PMAs to build an autonomous control strategy that allows an artificial electric sensor to find electrically contrasted objects, and to orbit around them based on a minimum set of measurements and simple reactive feedback control laws of the probe's motion. The approach does not require any simulation models and could be implemented on an autonomous underwater vehicle (AUV) equipped with artificial electric sense. The AUV has only to satisfy certain simple geometric properties, such as bi-laterally (left/right) symmetrical electrodes and possess a reasonably high aspect (length/width) ratio.

Computational modeling of electric imaging in weakly electric fish: insights for physiology, behavior and evolution

Weakly electric fish can sense electric signals produced by other animals whether they are conspecifics, preys or predators. These signals, sensed by passive electroreception, sustain electrocommunication, mating and agonistic behavior. Weakly electric fish can also generate a weak electrical discharge with which they can actively sense the animate and inanimate objects in their surroundings. Understanding both sensory modalities depends on our knowledge of how pre-receptorial electric images are formed and how movements modify them during behavior. The inability of effectively measuring pre-receptorial fields at the level of the skin contrasts with the amount of knowledge on electric fields and the availability of computational methods for estimating them. In this work we review past work on modeling of electric organ discharge and electric images, showing the usefulness of these methods to calculate the field and providing a brief explanation of their principles. In addition, we focus on recent work demonstrating the potential of electric image modeling and what the method has to offer for experimentalists studying sensory physiology, behavior and evolution.

Electric imaging through evolution, a modeling study of commonalities and differences

Modeling the electric field and images in electric fish contributes to a better understanding of the pre-receptor conditioning of electric images. Although the boundary element method has been very successful for calculating images and fields, complex electric organ discharges pose a challenge for active electroreception modeling. We have previously developed a direct method for calculating electric images which takes into account the structure and physiology of the electric organ as well as the geometry and resistivity of fish tissues. The present article reports a general application of our simulator for studying electric images in electric fish with heterogeneous, extended electric organs. We studied three species of Gymnotiformes, including both wave-type (Apteronotus albifrons) and pulse-type (Gymnotus obscurus and Gymnotus coropinae) fish, with electric organs of different complexity. The results are compared with the African (Gnathonemus petersii) and American (Gymnotus omarorum) electric fish studied previously. We address the following issues: 1) how to calculate equivalent source distributions based on experimental measurements, 2) how the complexity of the electric organ discharge determines the features of the electric field and 3) how the basal field determines the characteristics of electric images. Our findings allow us to generalize the hypothesis (previously posed for G. omarorum) in which the perioral region and the rest of the body play different sensory roles. While the "electrosensory fovea" appears suitable for exploring objects in detail, the rest of the body is likened to a "peripheral retina" for detecting the presence and movement of surrounding objects. We discuss the commonalities and differences between species. Compared to African species, American electric fish show a weaker field. This feature, derived from the complexity of distributed electric organs, may endow Gymnotiformes with the ability to emit site-specific signals to be detected in the short range by a conspecific and the possibility to evolve predator avoidance strategies.

Motor patterns during active electrosensory acquisition

Motor patterns displayed during active electrosensory acquisition of information seem to be an essential part of a sensory strategy by which weakly electric fish actively generate and shape sensory flow. These active sensing strategies are expected to adaptively optimize ongoing behavior with respect to either motor efficiency or sensory information gained. The tight link between the motor domain and sensory perception in active electrolocation make weakly electric fish like Gnathonemus petersii an ideal system for studying sensory-motor interactions in the form of active sensing strategies. Analyzing the movements and electric signals of solitary fish during unrestrained exploration of objects in the dark, we here present the first formal quantification of motor patterns used by fish during electrolocation. Based on a cluster analysis of the kinematic values we categorized the basic units of motion. These were then analyzed for their associative grouping to identify and extract short coherent chains of behavior. This enabled the description of sensory behavior on different levels of complexity: from single movements, over short behaviors to more complex behavioral sequences during which the kinematics alter between different behaviors. We present detailed data for three classified patterns and provide evidence that these can be considered as motor components of active sensing strategies. In accordance with the idea of active sensing strategies, we found categorical motor patterns to be modified by the sensory context. In addition these motor patterns were linked with changes in the temporal sampling in form of differing electric organ discharge frequencies and differing spatial distributions. The ability to detect such strategies quantitatively will allow future research to investigate the impact of such behaviors on sensing.

Long-term behavioral tracking of freely swimming weakly electric fish

Long-term behavioral tracking can capture and quantify natural animal behaviors, including those occurring infrequently. Behaviors such as exploration and social interactions can be best studied by observing unrestrained, freely behaving animals. Weakly electric fish (WEF) display readily observable exploratory and social behaviors by emitting electric organ discharge (EOD). Here, we describe three effective techniques to synchronously measure the EOD, body position, and posture of a free-swimming WEF for an extended period of time. First, we describe the construction of an experimental tank inside of an isolation chamber designed to block external sources of sensory stimuli such as light, sound, and vibration. The aquarium was partitioned to accommodate four test specimens, and automated gates remotely control the animals' access to the central arena. Second, we describe a precise and reliable real-time EOD timing measurement method from freely swimming WEF. Signal distortions caused by the animal's body movements are corrected by spatial averaging and temporal processing stages. Third, we describe an underwater near-infrared imaging setup to observe unperturbed nocturnal animal behaviors. Infrared light pulses were used to synchronize the timing between the video and the physiological signal over a long recording duration. Our automated tracking software measures the animal's body position and posture reliably in an aquatic scene. In combination, these techniques enable long term observation of spontaneous behavior of freely swimming weakly electric fish in a reliable and precise manner. We believe our method can be similarly applied to the study of other aquatic animals by relating their physiological signals with exploratory or social behaviors.

Sensory flow shaped by active sensing: sensorimotor strategies in electric fish

Goal-directed behavior in most cases is composed of a sequential order of elementary motor patterns shaped by sensorimotor contingencies. The sensory information acquired thus is structured in both space and time. Here we review the role of motion during the generation of sensory flow focusing on how animals actively shape information by behavioral strategies. We use the well-studied examples of vision in insects and echolocation in bats to describe commonalities of sensory-related behavioral strategies across sensory systems, and evaluate what is currently known about comparable active sensing strategies in electroreception of electric fish. In this sensory system the sensors are dispersed across the animal's body and the carrier source emitting energy used for sensing, the electric organ, is moved while the animal moves. Thus ego-motions strongly influence sensory dynamics. We present, for the first time, data of electric flow during natural probing behavior in Gnathonemus petersii (Mormyridae), which provide evidence for this influence. These data reveal a complex interdependency between the physical input to the receptors and the animal's movements, posture and objects in its environment. Although research on spatiotemporal dynamics in electrolocation is still in its infancy, the emerging field of dynamical sensory systems analysis in electric fish is a promising approach to the study of the link between movement and acquisition of sensory information.

Real-Time Localization of Moving Dipole Sources for Tracking Multiple Free-Swimming Weakly Electric Fish

In order to survive, animals must quickly and accurately locate prey, predators, and conspecifics using the signals they generate. The signal source location can be estimated using multiple detectors and the inverse relationship between the received signal intensity (RSI) and the distance, but difficulty of the source localization increases if there is an additional dependence on the orientation of a signal source. In such cases, the signal source could be approximated as an ideal dipole for simplification. Based on a theoretical model, the RSI can be directly predicted from a known dipole location; but estimating a dipole location from RSIs has no direct analytical solution. Here, we propose an efficient solution to the dipole localization problem by using a lookup table (LUT) to store RSIs predicted by our theoretically derived dipole model at many possible dipole positions and orientations. For a given set of RSIs measured at multiple detectors, our algorithm found a dipole location having the closest matching normalized RSIs from the LUT, and further refined the location at higher resolution. Studying the natural behavior of weakly electric fish (WEF) requires efficiently computing their location and the temporal pattern of their electric signals over extended periods. Our dipole localization method was successfully applied to track single or multiple freely swimming WEF in shallow water in real-time, as each fish could be closely approximated by an ideal current dipole in two dimensions. Our optimized search algorithm found the animal’s positions, orientations, and tail-bending angles quickly and accurately under various conditions, without the need for calibrating individual-specific parameters. Our dipole localization method is directly applicable to studying the role of active sensing during spatial navigation, or social interactions between multiple WEF. Furthermore, our method could be extended to other application areas involving dipole source localization.

On the haptic nature of the active electric sense of fish

Electroreception is a sensory modality present in chondrichthyes, actinopterygii, amphibians, and mammalian monotremes. The study of this non-intuitive sensory modality has provided insights for better understanding of sensory systems in general and inspired the development of innovative artificial devices. Here we review evidence obtained from the analysis of electrosensory images, neurophysiological data from the recording of unitary activity in the electrosensory lobe, and psychophysical data from analysis of novelty responses provoked in well-defined stimulus conditions, which all confirm that active electroreception has a short range, and that the influence of exploratory movements on object identification is strong. In active electric images two components can be identified: a "global" image profile depending on the volume, shape and global impedance of an object and a "texture" component depending on its surface attributes. There is a short range of the active electric sense and the progressive "blurring" of object image with distance. Consequently, the lack of precision regarding object location, considered together, challenge the current view of this sense as serving long range electrolocation and the commonly used metaphor of "electric vision". In fact, the active electric sense shares more commonalities with human active touch than with teleceptive senses as vision or audition. Taking into account that other skin exteroceptors and proprioception may be congruently stimulated during fish exploratory movements we propose that electric, mechanoceptive and proprioceptive sensory modalities found in electric fish could be considered together as a single haptic sensory system. This article is part of a Special Issue entitled Neural Coding 2012.