Cross-modal object recognition and dynamic weighting of sensory inputs in a fish

Crossmodal Correspondence Mediates Crossmodal Transfer from Visual to Auditory Stimuli in Category Learning

This article investigated whether crossmodal correspondence, as a sensory translation phenomenon, can mediate crossmodal transfer from visual to auditory stimuli in category learning and whether multimodal category learning can influence the crossmodal correspondence between auditory and visual stimuli. Experiment 1 showed that the category knowledge acquired from elevation stimuli affected the categorization of pitch stimuli when there were robust crossmodal correspondence effects between elevation and size, indicating that crossmodal transfer occurred between elevation and pitch stimuli. Experiments 2 and 3 revealed that the size category knowledge could not be transferred to the categorization of pitches, but interestingly, size and pitch category learning determined the direction of the pitch-size correspondence, suggesting that the pitch-size correspondence was not stable and could be determined using multimodal category learning. Experiment 4 provided further evidence that there was no crossmodal transfer between size and pitch, due to the absence of a robust pitch-size correspondence. These results demonstrated that crossmodal transfer can occur between audio-visual stimuli with crossmodal correspondence, and multisensory category learning can change the corresponding relationship between audio-visual stimuli. These findings suggest that crossmodal transfer and crossmodal correspondence share similar abstract representations, which can be mediated by semantic content such as category labels.

Daubenton's bats maintain stereotypical echolocation behaviour and a lombard response during target interception in light

Most bats hunt insects on the wing at night using echolocation as their primary sensory modality, but nevertheless maintain complex eye anatomy and functional vision. This raises the question of how and when insectivorous bats use vision during their largely nocturnal lifestyle. Here, we test the hypothesis that the small insectivorous bat, Myotis daubentonii, relies less on echolocation, or dispenses with it entirely, as visual cues become available during challenging acoustic noise conditions. We trained five wild-caught bats to land on a spherical target in both silence and when exposed to broad-band noise to decrease echo detectability, while light conditions were manipulated in both spectrum and intensity. We show that during noise exposure, the bats were almost three times more likely to use multiple attempts to solve the task compared to in silent controls. Furthermore, the bats exhibited a Lombard response of 0.18 dB/dBnoise and decreased call intervals earlier in their flight during masking noise exposures compared to in silent controls. Importantly, however, these adjustments in movement and echolocation behaviour did not differ between light and dark control treatments showing that small insectivorous bats maintain the same echolocation behaviour when provided with visual cues under challenging conditions for echolocation. We therefore conclude that bat echolocation is a hard-wired sensory system with stereotyped compensation strategies to both target range and masking noise (i.e. Lombard response) irrespective of light conditions. In contrast, the adjustments of call intervals and movement strategies during noise exposure varied substantially between individuals indicating a degree of flexibility that likely requires higher order processing and perhaps vocal learning.

First-sight recognition of touched objects shows that chicks can solve Molyneux's problem

If a congenitally blind person learns to distinguish between a cube and a sphere by touch, would they immediately recognize these objects by sight once their vision is restored? This question, posed by Molyneux in 1688, has puzzled philosophers and scientists since then. To overcome ethical and practical difficulties in the investigation of cross-modal recognition, we studied inexperienced poultry chicks, which can be reared in darkness until the moment of a visual test with no detrimental consequences. After hatching chicks in darkness, we exposed them to either tactile smooth or tactile bumpy stimuli for 24 h. Immediately after the tactile exposure, chicks were tested in a visual recognition task, during their first experience with light. At first sight, chicks that had been exposed in the tactile modality to smooth stimuli approached the visual smooth stimulus significantly more than those exposed to the tactile bumpy stimuli. These results show that visually inexperienced chicks can solve Molyneux's problem, indicating cross-modal recognition does not require previous multimodal experience. At least in this precocial species, supra-modal brain areas appear functional already at birth. This discovery paves the way for the investigation of predisposed cross-modal cognition that does not depend on visual experience.

A perspective on neuroethology: what the past teaches us about the future of neuroethology

For 100 years, the Journal of Comparative Physiology-A has significantly supported research in the field of neuroethology. The celebration of the journal's centennial is a great time point to appreciate the recent progress in neuroethology and to discuss possible avenues of the field. Animal behavior is the main source of inspiration for neuroethologists. This is illustrated by the huge diversity of investigated behaviors and species. To explain behavior at a mechanistic level, neuroethologists combine neuroscientific approaches with sophisticated behavioral analysis. The rapid technological progress in neuroscience makes neuroethology a highly dynamic and exciting field of research. To summarize the recent scientific progress in neuroethology, I went through all abstracts of the last six International Congresses for Neuroethology (ICNs 2010-2022) and categorized them based on the sensory modalities, experimental model species, and research topics. This highlights the diversity of neuroethology and gives us a perspective on the field's scientific future. At the end, I highlight three research topics that may, among others, influence the future of neuroethology. I hope that sharing my roots may inspire other scientists to follow neuroethological approaches.

Mormyrid fish as models for investigating sensory-motor integration: A behavioural perspective

Animals possess senses which gather information from their environment. They can tune into important aspects of this information and decide on the most appropriate response, requiring coordination of their sensory and motor systems. This interaction is bidirectional. Animals can actively shape their perception with self-driven motion, altering sensory flow to maximise the environmental information they are able to extract. Mormyrid fish are excellent candidates for studying sensory-motor interactions, because they possess a unique sensory system (the active electric sense) and exhibit notable behaviours that seem to be associated with electrosensing. This review will take a behavioural approach to unpicking this relationship, using active electrolocation as an example where body movements and sensing capabilities are highly related and can be assessed in tandem. Active electrolocation is the process where individuals will generate and detect low-voltage electric fields to locate and recognise nearby objects. We will focus on research in the mormyrid Gnathonemus petersii (G. petersii), given the extensive study of this species, particularly its object recognition abilities. By studying object detection and recognition, we can assess the potential benefits of self-driven movements to enhance selection of biologically relevant information. Finally, these findings are highly relevant to understanding the involvement of movement in shaping the sensory experience of animals that use other sensory modalities. Understanding the overlap between sensory and motor systems will give insight into how different species have become adapted to their environments.

Ketamine disrupts locomotion and electrolocation in a novel model of schizophrenia, Gnathonemus petersii fish

The present study aimed to examine a weakly electric fish Gnathonemus petersii (G. petersii) as a candidate model organism of glutamatergic theory of schizophrenia. The idea of G. petersii elevating the modeling of schizophrenia symptoms is based on the fish's electrolocation and electrocommunication abilities. Fish were exposed to the NMDA antagonist ketamine in two distinct series differing in the dose of ketamine. The main finding revealed ketamine-induced disruption of the relationship between electric signaling and behavior indicating impairment of fish navigation. Moreover, lower doses of ketamine significantly increased locomotion and erratic movement and higher doses of ketamine reduced the number of electric organ discharges indicating successful induction of positive schizophrenia-like symptoms and disruption of fish navigation. Additionally, a low dose of haloperidol was used to test the normalization of the positive symptoms to suggest a predictive validity of the model. However, although successfully induced, positive symptoms were not normalized using the low dose of haloperidol; hence, more doses of the typical antipsychotic haloperidol and probably also of a representative of atypical antipsychotic drugs need to be examined to confirm the predictive validity of the model.

Cross-modal perception of identity by sound and taste in bottlenose dolphins

While studies have demonstrated concept formation in animals, only humans are known to label concepts to use them in mental simulations or predictions. To investigate whether other animals use labels comparably, we studied cross-modal, individual recognition in bottlenose dolphins (Tursiops truncatus) that use signature whistles as labels for conspecifics in their own communication. First, we tested whether dolphins could use gustatory stimuli and found that they could distinguish between water and urine samples, as well as between urine from familiar and unfamiliar individuals. Then, we paired playbacks of signature whistles of known animals with urine samples from either the same dolphin or a different, familiar animal. Dolphins investigated the presentation area longer when the acoustic and gustatory sample matched than when they mismatched. This demonstrates that dolphins recognize other individuals by gustation alone and can integrate information from acoustic and taste inputs indicating a modality independent, labeled concept for known conspecifics.

Evaluation of Visual and Tactile Perception by Plain-Body Octopus (Callistoctopus aspilosomatis) of Prey-Like Objects

We investigated the characteristic features of perception in octopuses by examining multisensory information from an object simulating prey, which provided different visual and tactile stimuli. In experiments, we presented plain-body octopus with four kinds of models, namely, the Lifelike crab, the Embedded crab, the Translucent crab, and the Black cuboid. These models contain different amounts of visual and tactile information that a crab originally contains: the Lifelike crab resembles a crab both visually and tactilely, the Embedded crab resembles a crab visually but provides different tactile information, the Translucent crab provides tactile information of a crab but contains less visual information, and the Black cuboid lacks both visual and tactile information of a crab. Among these four models, octopuses contacted most with the Lifelike crab, which was similar to their behavior with a crab. Indeed, octopuses were fastest to contact the Lifelike crab and had the longest duration of contacting it among the four models. Octopuses contacted the Embedded crab more than the Translucent crab, both of which had contrasting visuo-tactile information compared to that of a crab. Quickness of octopuses to contact and duration of contact with the Embedded crab were more similar to those with the Lifelike crab than to those with the Translucent crab. Furthermore, octopuses contacted the Black cuboid least among the models. These results suggest that octopuses compositely detect both visual and tactile information in order to perceive an object. Furthermore, octopuses possess the potential priority either for visual or tactile information, by which they process the target object.

Five Breakthroughs: A First Approximation of Brain Evolution From Early Bilaterians to Humans

Retracing the evolutionary steps by which human brains evolved can offer insights into the underlying mechanisms of human brain function as well as the phylogenetic origin of various features of human behavior. To this end, this article presents a model for interpreting the physical and behavioral modifications throughout major milestones in human brain evolution. This model introduces the concept of a "breakthrough" as a useful tool for interpreting suites of brain modifications and the various adaptive behaviors these modifications enabled. This offers a unique view into the ordered steps by which human brains evolved and suggests several unique hypotheses on the mechanisms of human brain function.

Bumble bees display cross-modal object recognition between visual and tactile senses

Many animals can associate object shapes with incentives. However, such behavior is possible without storing images of shapes in memory that are accessible to more than one sensory modality. One way to explore whether there are modality-independent internal representations of object shapes is to investigate cross-modal recognition-experiencing an object in one sensory modality and later recognizing it in another. We show that bumble bees trained to discriminate two differently shaped objects (cubes and spheres) using only touch (in darkness) or vision (in light, but barred from touching the objects) could subsequently discriminate those same objects using only the other sensory information. Our experiments demonstrate that bumble bees possess the ability to integrate sensory information in a way that requires modality-independent internal representations.

Cross-modal sensory transfer: Bumble bees do it

Stored sensory input permits two sensory channels to exchange and compare information

Touch improves visual discrimination of object features in capuchin monkeys (Sapajus spp.)

Primates perceive many object features through vision and touch. To date, little is known on how the synergy of these two sensory modalities contributes to enhance object recognition. Here, we investigated in capuchin monkeys (N = 12) whether manipulating objects and retaining tactile information enhanced visual recognition of geometrical object properties on different scales. Capuchins were trained to visually select the rewarded one of two objects differing in size, shape (larger-scale) or surface structure (smaller-scale). Objects were explored in two experimental conditions: the Sight condition prevented capuchins from touching the chosen object; the Sight and Touch condition allowed them to touch the selected object. Our results indicated that tactile information increased the capuchins' learning speed for visual discrimination of object features. Moreover, the capuchins' learning speed was higher in both size and shape discrimination compared to surface discrimination regardless of the availability of tactile input. Overall, our data demonstrated that the acquisition of tactile information about object features was advantageous for the capuchins and allowed them to achieve high levels of visual accuracy faster. This suggests that information from touch potentiated object recognition in the visual modality.

Assessment of fighting ability in the vocal cichlid Metriaclima zebra in face of incongruent audiovisual information

Information transfer between individuals typically depends on multiple sensory channels. Yet, how multi-sensory inputs shape adaptive behavioural decisions remains largely unexplored. We tested the relative importance of audio and visual sensory modalities in opponent size assessment in the vocal cichlid fish, Metriaclima zebra, by playing back mismatched agonistic sounds mimicking larger or smaller opponents during fights of size-matched males. Trials consisted in three 5-min periods: PRE (visual), PBK (acoustic+visual) and POST (visual). During PBK agonistic sounds of smaller (high frequency or low amplitude) or larger (low frequency or high amplitude) males were played back interactively. As a control, we used white noise and silence. We show that sound frequency but not amplitude affects aggression, indicating that spectral cues reliably signal fighting ability. In addition, males reacted to the contrasting audio-visual information by giving prevalence to the sensory channel signalling a larger opponent. Our results suggest that fish can compare the relevance of information provided by different sensory inputs to make behavioural decisions during fights, which ultimately contributes to their individual fitness. These findings have implications for our understanding of the role of multi-sensory inputs in shaping behavioural output during conflicts in vertebrates.

Summary: Cichlid fish rely on the sensory channel indicating higher risk when facing incongruent visual and acoustic information during opponent assessment.

Central connections of the trigeminal motor command system in the weakly electric Elephantnose fish (Gnathonemus petersii)

The highly mobile chin appendage of Gnathonemus petersii, the Schnauzenorgan, is used to actively probe the environment and is known to be a fovea of the electrosensory system. It receives an important innervation from both the trigeminal sensory and motor systems. However, little is known about the premotor control pathways that coordinate the movements of the Schnauzenorgan, or about central pathways originating from the trigeminal motor nucleus. The present study focuses on the central connections of the trigeminal motor system to elucidate premotor centers controlling Schnauzenorgan movements, with particular interest in the possible connections between the electrosensory and trigeminal systems. Neurotracer injections into the trigeminal motor nucleus revealed bilateral, reciprocal connections between the two trigeminal motor nuclei and between the trigeminal sensory and motor nuclei by bilateral labeling of cells and terminals. Prominent afferent input to the trigeminal motor nucleus originates from the nucleus lateralis valvulae, the nucleus dorsalis mesencephali, the cerebellar corpus C1, the reticular formation, and the Raphe nuclei. Retrogradely labeled cells were also observed in the central pretectal nucleus, the dorsal anterior pretectal nucleus, the tectum, the ventroposterior nucleus of the torus semicircularis, the gustatory sensory and motor nuclei, and in the hypothalamus. Labeled terminals, but not cell bodies, were observed in the nucleus lateralis valvulae and the reticular formation. No direct connections were found between the electrosensory system and the V motor nucleus but the central connections identified would provide several multisynaptic pathways linking these two systems, including possible efference copy and corollary discharge mechanisms.

Integrating vision and echolocation for navigation and perception in bats

How animals integrate information from various senses to navigate and generate perceptions is a fundamental question. Bats are ideal animal models to study multisensory integration due to their reliance on vision and echolocation, two modalities that allow distal sensing with high spatial resolution. Using three behavioral paradigms, we studied different aspects of multisensory integration in Egyptian fruit bats. We show that bats learn the three-dimensional shape of an object using vision only, even when using both vision and echolocation. Nevertheless, we demonstrate that they can classify objects using echolocation and even translate echoic information into a visual representation. Last, we show that in navigation, bats dynamically switch between the modalities: Vision was given more weight when deciding where to fly, while echolocation was more dominant when approaching an obstacle. We conclude that sensory integration is task dependent and that bimodal information is weighed in a more complex manner than previously suggested.

The Mormyrid Optic Tectum Is a Topographic Interface for Active Electrolocation and Visual Sensing

The African weakly electric fish Gnathonemus petersii is capable of cross-modal object recognition using its electric sense or vision. Thus, object features stored in the brain are accessible by multiple senses, either through connections between unisensory brain regions or because of multimodal representations in multisensory areas. Primary electrosensory information is processed in the medullary electrosensory lateral line lobe, which projects topographically to the lateral nucleus of the torus semicircularis (NL). Visual information reaches the optic tectum (TeO), which projects to various other brain regions. We investigated the neuroanatomical connections of these two major midbrain visual and electrosensory brain areas, focusing on the topographical relationship of interconnections between the two structures. Thus, the neural tracer DiI was injected systematically into different tectal quadrants, as well as into the NL. Tectal tracer injections revealed topographically organized retrograde and anterograde label in the NL. Rostral and caudal tectal regions were interconnected with rostral and caudal areas of the NL, respectively. However, dorsal and ventral tectal regions were represented in a roughly inverted fashion in NL, as dorsal tectal injections labeled ventral areas in NL and vice versa. In addition, tracer injections into TeO or NL revealed extensive inputs to both structures from ipsilateral (NL also contralateral) efferent basal cells in the valvula cerebelli; the NL furthermore projected back to the valvula. Additional tectal and NL connections were largely confirmatory to earlier studies. For example, the TeO received ipsilateral inputs from the central zone of the dorsal telencephalon, torus longitudinalis, nucleus isthmi, various tegmental, thalamic and pretectal nuclei, as well as other nuclei of the torus semicircularis. Also, the TeO projected to the dorsal preglomerular and dorsal posterior thalamic nuclei as well as to nuclei in the torus semicircularis and nucleus isthmi. Beyond the clear topographical relationship of NL and TeO interconnections established here, the known neurosensory upstream circuitry was used to suggest a model of how a defined spot in the peripheral sensory world comes to be represented in a common associated neural locus both in the NL and the TeO, thereby providing the neural substrate for cross-modal object recognition.

Electric pulse characteristics can enable species recognition in African weakly electric fish species

Communication is key to a wide variety of animal behaviours and multiple modalities are often involved in this exchange of information from sender to receiver. The communication of African weakly electric fish, however, is thought to be predominantly unimodal and is mediated by their electric sense, in which species-specific electric organ discharges (EODs) are generated in a context-dependent and thus variable sequence of pulse intervals (SPI). While the primary function of the electric sense is considered to be electrolocation, both of its components likely carry information regarding identity of the sender. However, a clear understanding of their contribution to species recognition is incomplete. We therefore analysed these two electrocommunication components (EOD waveform and SPI statistics) in two sympatric mormyrid Campylomormyrus species. In a set of five playback conditions, we further investigated which components may drive interspecific recognition and discrimination. While we found that both electrocommunication components are species-specific, the cues necessary for species recognition differ between the two species studied. While the EOD waveform and SPI were both necessary and sufficient for species recognition in C. compressirostris males, C. tamandua males apparently utilize other, non-electric modalities. Mapped onto a recent phylogeny, our results suggest that discrimination by electric cues alone may be an apomorphic trait evolved during a recent radiation in this taxon.

Current-Efficient Preamplifier Architecture for CMRR Sensitive Neural Recording Applications

There are neural recording applications in which the amplitude of common-mode interfering signals is several orders of magnitude higher than the amplitude of the signals of interest. This challenging situation for neural amplifiers occurs, among other applications, in neural recordings of weakly electric fish or nerve activity recordings made with cuff electrodes. This paper reports an integrated neural amplifier architecture targeting in-vivo recording of local field potentials and unitary signals from the brain stem of a weakly electric fish Gymnotus omarorum. The proposed architecture offers low noise, high common-mode rejection ratio (CMRR), current-efficiency, and a high-pass frequency fixed without MOS pseudoresistors. The main contributions of this work are the overall architecture coupled with an efficient and simple single-stage circuit for the amplifier main transconductor, and the ability of the amplifier to acquire biopotential signals from high-amplitude common-mode interference in an unshielded environment. A fully-integrated neural preamplifier, which performs well in line with the state-of-the-art of the field while providing enhanced CMRR performance, was fabricated in a 0.5  m CMOS process. Results from measurements show that the gain is 49.5 dB, the bandwidth ranges from 13 Hz to 9.8 kHz, the equivalent input noise is 1.88  V, the CMRR is 87 dB and the Noise Efficiency Factor is 2.1. In addition, in-vivo recordings of weakly electric fish neural activity performed by the proposed amplifier are introduced and favorably compared with those of a commercial laboratory instrumentation system.

Disembodying the invisible: electrocommunication and social interactions by passive reception of a moving playback signal

Mormyrid weakly electric fish have a special electrosensory modality that allows them to actively sense their environment and to communicate with conspecifics by emitting sequences of electric signals. Electroreception is mediated by different types of dermal electroreceptor organs for active electrolocation and electrocommunication, respectively. During electrocommunication, mormyrids exhibit stereotyped discharge sequences and locomotor patterns, which can be induced by playback of electric signals. This raises the question: what sensory information is required to initiate and sustain social interactions, and which electrosensory pathway mediates such interactions? By experimentally excluding stimuli from vision and the lateral line system, we show that Mormyrus rume proboscirostris can rely exclusively on its electrosensory system to track a mobile source of electric communication signals. Detection of electric playback signals induced discharge cessations, followed by double-pulse patterns. The animals tried to interact with the moving signal source and synchronized their discharge activity to the playback. These behaviors were absent in control trials without playback. Silencing the electric organ in some fish did not impair their ability to track the signal source. Silenced fish followed on trajectories similar to those obtained from intact animals, indicating that active electrolocation is no precondition for close-range interactions based on electrocommunication. However, some silenced animals changed their strategy when searching for the stationary playback source, which indicates passive sensing. Social interactions among mormyrids can therefore be induced and mediated by passive reception of electric communication signals without the need for perception of the location of the signal source through other senses.

Electrosensory capture during multisensory discrimination of nearby objects in the weakly electric fish Gnathonemus petersii

Animal multisensory systems are able to cope with discrepancies in information provided by individual senses by integrating information using a weighted average of the sensory inputs. Such sensory weighting often leads to a dominance of a certain sense during particular tasks and conditions, also called sensory capture. Here we investigated the interaction of vision and active electrolocation during object discrimination in the weakly electric fish Gnathonemus petersii. Fish were trained to discriminate between two objects using both senses and were subsequently tested using either only vision or only the active electric sense. We found that at short range the electric sense dominates over vision, leading to a decreased ability to discriminate between objects visually when vision and electrolocation provide conflicting information. In line with visual capture in humans, we call this dominance of the electric sense electrosensory capture. Further, our results suggest that the fish are able to exploit the advantages of multiple senses using vision and electrolocation redundantly, synergistically and complementarily. Together our results show that by providing similar information about the environment on different spatial scales, vision and the electric sense of G. petersii are well attuned to each other producing a robust and flexible percept.

Waveform sensitivity of electroreceptors in the pulse-type weakly electric fish Gymnotus omarorum

As in most sensory systems, electrosensory images in weakly electric fish are encoded in two parallel pathways, fast and slow. From work on wave-type electric fish, these fast and slow pathways are thought to encode the time and amplitude of electrosensory signals, respectively. The present study focuses on the primary afferents giving origin to the slow path of the pulse-type weakly electric fish Gymnotus omarorum We found that burst duration coders respond with a high-frequency train of spikes to each electric organ discharge. They also show high sensitivity to phase-frequency distortions of the self-generated local electric field. We explored this sensitivity by manipulating the longitudinal impedance of a probe cylinder to modulate the stimulus waveform, while extracellularly recording isolated primary afferents. Resistive loads only affect the amplitude of the re-afferent signals without distorting the waveform. Capacitive loads cause large waveform distortions aside from amplitude changes. Stepping from a resistive to a capacitive load in such a way that the stimulus waveform was distorted, without changing its total energy, caused strong changes in latency, inter-spike interval and number of spikes of primary afferent responses. These burst parameters are well correlated suggesting that they may contribute synergistically in driving downstream neurons. This correlation also suggests that each receptor encodes a single parameter in the stimulus waveform. The finding of waveform distortion sensitivity is relevant because it may contribute to: (a) enhance electroreceptive range in the peripheral 'electrosensory field', (b) a better identification of living prey at the 'foveal electrosensory field' and (c) detect the presence and orientation of conspecifics. Our results also suggest a revision of the classical view of amplitude and time encoding by fast and slow pathways in pulse-type electric fish.

The bioinspiring potential of weakly electric fish

Electric fish are privileged animals for bio-inspiring man-built autonomous systems since they have a multimodal sense that allows underwater navigation, object classification and intraspecific communication. Although there are taxon dependent variations adapted to different environments, this multimodal system can be schematically described as having four main components: active electroreception, passive electroreception, lateral line sense and, proprioception. Amongst these sensory modalities, proprioception and electroreception show 'active' systems that extrct information carried by self generated forms of energy. This ensemble of four sensory modalities is present in African mormyriformes and American gymnotiformes. The convergent evolution of similar imaging, peripheral encoding, and central processing mechanisms suggests that these mechanisms may be the most suitable for dealing with electric images in the context of the other and self generated actions. This review deals with the way in which biological organisms address three of the problems that are faced when designing a bioinspired electroreceptive agent: (a) body shape, material and mobility, (b) peripheral encoding of electric images, and (c) early processing of electrosensory signals. Taking into account biological solutions I propose that the new generation of underwater agents should have electroreceptive arms, use complex peripheral sensors for encoding the images and cerebellum like architecture for image feature extraction and implementing sensory-motor transformations.

The Transition to Minimal Consciousness through the Evolution of Associative Learning

The minimal state of consciousness is sentience. This includes any phenomenal sensory experience - exteroceptive, such as vision and olfaction; interoceptive, such as pain and hunger; or proprioceptive, such as the sense of bodily position and movement. We propose unlimited associative learning (UAL) as the marker of the evolutionary transition to minimal consciousness (or sentience), its phylogenetically earliest sustainable manifestation and the driver of its evolution. We define and describe UAL at the behavioral and functional level and argue that the structural-anatomical implementations of this mode of learning in different taxa entail subjective feelings (sentience). We end with a discussion of the implications of our proposal for the distribution of consciousness in the animal kingdom, suggesting testable predictions, and revisiting the ongoing debate about the function of minimal consciousness in light of our approach.

Computational principles and models of multisensory integration

Combining information from multiple senses creates robust percepts, speeds up responses, enhances learning, and improves detection, discrimination, and recognition. In this review, I discuss computational models and principles that provide insight into how this process of multisensory integration occurs at the behavioral and neural level. My initial focus is on drift-diffusion and Bayesian models that can predict behavior in multisensory contexts. I then highlight how recent neurophysiological and perturbation experiments provide evidence for a distributed redundant network for multisensory integration. I also emphasize studies which show that task-relevant variables in multisensory contexts are distributed in heterogeneous neural populations. Finally, I describe dimensionality reduction methods and recurrent neural network models that may help decipher heterogeneous neural populations involved in multisensory integration.

Weakly electric fish learn both visual and electrosensory cues in a multisensory object discrimination task

Weakly electric fish use electrosensory, visual, olfactory and lateral line information to guide foraging and navigation behaviors. In many cases they preferentially rely on electrosensory cues. Do fish also memorize non-electrosensory cues? Here, we trained individuals of gymnotiform weakly electric fish Apteronotus albifrons in an object discrimination task. Objects were combinations of differently conductive materials covered with differently colored cotton hoods. By setting visual and electrosensory cues in conflict we analyzed the sensory hierarchy among the electrosensory and the visual sense in object discrimination. Our experiments show that: (i) black ghost knifefish can be trained to solve discrimination tasks similarly to the mormyrid fish; (ii) fish preferentially rely on electrosensory cues for object discrimination; (iii) despite the dominance of the electrosense they still learn the visual cue and use it when electrosensory information is not available; (iv) fish prefer the trained combination of rewarded cues over combinations that match only in a single feature and also memorize the non-rewarded combination.