A novel behavioral paradigm to assess multisensory processing in mice

Recent advances in neurotechnology-based biohybrid robots

Biohybrid robots retain the innate biological characteristics and behavioral traits of animals, making them valuable in applications such as disaster relief, exploration of unknown terrains, and medical care. This review aims to comprehensively discuss the evolution of biohybrid robots, their key technologies and applications, and the challenges they face. By analyzing studies conducted on terrestrial, aquatic, and aerial biohybrid robots, we gain a deeper understanding of how these technologies have made significant progress in simulating natural organisms, improving mechanical performance, and intelligent control. Additionally, we address challenges associated with the application of electrical stimulation technology, the precision of neural signal monitoring, and the ethical considerations for biohybrid robots. We highlight the importance of future research focusing on developing more sophisticated and biocompatible control methods while prioritizing animal welfare. We believe that exploring multimodal monitoring and stimulation technologies holds the potential to enhance the performance of biohybrid robots. These efforts are expected to pave the way for biohybrid robotics technology to introduce greater innovation and well-being to human society in the future.

Profiles of animal consciousness: A species-sensitive, two-tier account to quality and distribution

The science of animal consciousness investigates (i) which animal species are conscious (the distribution question) and (ii) how conscious experience differs in detail between species (the quality question). We propose a framework which clearly distinguishes both questions and tackles both of them. This two-tier account distinguishes consciousness along ten dimensions and suggests cognitive capacities which serve as distinct operationalizations for each dimension. The two-tier account achieves three valuable aims: First, it separates strong and weak indicators of the presence of consciousness. Second, these indicators include not only different specific contents but also differences in the way particular contents are processed (by processes of learning, reasoning or abstraction). Third, evidence of consciousness from each dimension can be combined to derive the distinctive multi-dimensional consciousness profile of various species. Thus, the two-tier account shows how the kind of conscious experience of different species can be systematically compared.

Cupid's quiver: Integrating sensory cues in rodent mating systems

In many animal species, males and females exploit different mating strategies, display sex-typical behaviors, and use distinct systems to recognize ethologically relevant cues. Mate selection thus requires mutual recognition across diverse social interactions based on distinct sensory signals. These sex differences in courtship and mating behaviors correspond to differences in sensory systems and downstream neural substrates engaged to recognize and respond to courtship signals. In many rodents, males tend to rely heavily on volatile olfactory and pheromone cues, while females appear to be guided more by a combination of these chemosensory signals with acoustic cues in the form of ultrasonic vocalizations. The mechanisms by which chemical and acoustic cues are integrated to control behavior are understudied in mating but are known to be important in the control of maternal behaviors. Socially monogamous species constitute a behaviorally distinct group of rodents. In these species, anatomic differences between males and females outside the nervous system are less prominent than in species with non-monogamous mating systems, and both sexes engage in more symmetric social behaviors and form attachments. Nevertheless, despite the apparent similarities in behaviors displayed by monogamous males and females, the circuitry supporting social, mating, and attachment behaviors in these species is increasingly thought to differ between the sexes. Sex differences in sensory modalities most important for mate recognition in across species are of particular interest and present a wealth of questions yet to be answered. Here, we discuss how distinct sensory cues may be integrated to drive social and attachment behaviors in rodents, and the differing roles of specific sensory systems in eliciting displays of behavior by females or males.

Auditory input enhances somatosensory encoding and tactile goal-directed behavior

The capacity of the brain to encode multiple types of sensory input is key to survival. Yet, how neurons integrate information from multiple sensory pathways and to what extent this influences behavior is largely unknown. Using two-photon Ca2+ imaging, optogenetics and electrophysiology in vivo and in vitro, we report the influence of auditory input on sensory encoding in the somatosensory cortex and show its impact on goal-directed behavior. Monosynaptic input from the auditory cortex enhanced dendritic and somatic encoding of tactile stimulation in layer 2/3 (L2/3), but not layer 5 (L5), pyramidal neurons in forepaw somatosensory cortex (S1). During a tactile-based goal-directed task, auditory input increased dendritic activity and reduced reaction time, which was abolished by photoinhibition of auditory cortex projections to forepaw S1. Taken together, these results indicate that dendrites of L2/3 pyramidal neurons encode multisensory information, leading to enhanced neuronal output and reduced response latency during goal-directed behavior.

Behavioral Approaches to Study Top-Down Influences on Active Listening

The massive network of descending corticofugal projections has been long-recognized by anatomists, but their functional contributions to sound processing and auditory-guided behaviors remain a mystery. Most efforts to characterize the auditory corticofugal system have been inductive; wherein function is inferred from a few studies employing a wide range of methods to manipulate varying limbs of the descending system in a variety of species and preparations. An alternative approach, which we focus on here, is to first establish auditory-guided behaviors that reflect the contribution of top-down influences on auditory perception. To this end, we postulate that auditory corticofugal systems may contribute to active listening behaviors in which the timing of bottom-up sound cues can be predicted from top-down signals arising from cross-modal cues, temporal integration, or self-initiated movements. Here, we describe a behavioral framework for investigating how auditory perceptual performance is enhanced when subjects can anticipate the timing of upcoming target sounds. Our first paradigm, studied both in human subjects and mice, reports species-specific differences in visually cued expectation of sound onset in a signal-in-noise detection task. A second paradigm performed in mice reveals the benefits of temporal regularity as a perceptual grouping cue when detecting repeating target tones in complex background noise. A final behavioral approach demonstrates significant improvements in frequency discrimination threshold and perceptual sensitivity when auditory targets are presented at a predictable temporal interval following motor self-initiation of the trial. Collectively, these three behavioral approaches identify paradigms to study top-down influences on sound perception that are amenable to head-fixed preparations in genetically tractable animals, where it is possible to monitor and manipulate particular nodes of the descending auditory pathway with unparalleled precision.

Expanding evolutionary neuroscience: insights from comparing variation in behavior

Neuroscientists have long studied species with convenient biological features to discover how behavior emerges from conserved molecular, neural, and circuit level processes. With the advent of new tools, from viral vectors and gene editing to automated behavioral analyses, there has been a recent wave of interest in developing new, "nontraditional" model species. Here, we advocate for a complementary approach to model species development, that is, model clade development, as a way to integrate an evolutionary comparative approach with neurobiological and behavioral experiments. Capitalizing on natural behavioral variation in and investing in experimental tools for model clades will be a valuable strategy for the next generation of neuroscience discovery.

Approaches to Understanding Multisensory Dysfunction in Autism Spectrum Disorder

Abnormal sensory responses are a DSM-5 symptom of autism spectrum disorder (ASD), and research findings demonstrate altered sensory processing in ASD. Beyond difficulties with processing information within single sensory domains, including both hypersensitivity and hyposensitivity, difficulties in multisensory processing are becoming a core issue of focus in ASD. These difficulties may be targeted by treatment approaches such as "sensory integration," which is frequently applied in autism treatment but not yet based on clear evidence. Recently, psychophysical data have emerged to demonstrate multisensory deficits in some children with ASD. Unlike deficits in social communication, which are best understood in humans, sensory and multisensory changes offer a tractable marker of circuit dysfunction that is more easily translated into animal model systems to probe the underlying neurobiological mechanisms. Paralleling experimental paradigms that were previously applied in humans and larger mammals, we and others have demonstrated that multisensory function can also be examined behaviorally in rodents. Here, we review the sensory and multisensory difficulties commonly found in ASD, examining laboratory findings that relate these findings across species. Next, we discuss the known neurobiology of multisensory integration, drawing largely on experimental work in larger mammals, and extensions of these paradigms into rodents. Finally, we describe emerging investigations into multisensory processing in genetic mouse models related to autism risk. By detailing findings from humans to mice, we highlight the advantage of multisensory paradigms that can be easily translated across species, as well as the potential for rodent experimental systems to reveal opportunities for novel treatments. LAY SUMMARY: Sensory and multisensory deficits are commonly found in ASD and may result in cascading effects that impact social communication. By using similar experiments to those in humans, we discuss how studies in animal models may allow an understanding of the brain mechanisms that underlie difficulties in multisensory integration, with the ultimate goal of developing new treatments. Autism Res 2020, 13: 1430-1449. © 2020 International Society for Autism Research, Wiley Periodicals, Inc.

Cross-Talk of Low-Level Sensory and High-Level Cognitive Processing: Development, Mechanisms, and Relevance for Cross-Modal Abilities of the Brain

The emergence of cross-modal learning capabilities requires the interaction of neural areas accounting for sensory and cognitive processing. Convergence of multiple sensory inputs is observed in low-level sensory cortices including primary somatosensory (S1), visual (V1), and auditory cortex (A1), as well as in high-level areas such as prefrontal cortex (PFC). Evidence shows that local neural activity and functional connectivity between sensory cortices participate in cross-modal processing. However, little is known about the functional interplay between neural areas underlying sensory and cognitive processing required for cross-modal learning capabilities across life. Here we review our current knowledge on the interdependence of low- and high-level cortices for the emergence of cross-modal processing in rodents. First, we summarize the mechanisms underlying the integration of multiple senses and how cross-modal processing in primary sensory cortices might be modified by top-down modulation of the PFC. Second, we examine the critical factors and developmental mechanisms that account for the interaction between neuronal networks involved in sensory and cognitive processing. Finally, we discuss the applicability and relevance of cross-modal processing for brain-inspired intelligent robotics. An in-depth understanding of the factors and mechanisms controlling cross-modal processing might inspire the refinement of robotic systems by better mimicking neural computations.

Monocular and binocular opto-locomotor reflex biases for random dot motion in mice

We investigated the relationship between eyes receiving visual input of large field translating random dot motion and subsequent reflexive changes in running direction in mice. The animals were head-fixed running on a Styrofoam ball and the opto-locomotor reflex (OLR) was measured in response to 2 s of dots patterns moving horizontally to the left or right. We measured the OLR in conditions with both eyes open (binocular) and one eye closed (monocular). When we covered the right or left eye in the monocular condition, we found reflexive behavior to be delayed for a few hundred milliseconds to leftward or rightward motion, respectively. After this delay, the bias disappeared and reflexive behavior was similar to responses to motion under binocular conditions. These results might be explained by different contributions of subcortical and cortical visual motion processing pathways to the OLR. Furthermore, we found no evidence for nonlinear interactions between the two eyes, because the sum of the OLR of the two monocular conditions was equal in amplitude and temporal characteristics to the OLR under binocular conditions.

Multisensory Integration as a Window into Orderly and Disrupted Cognition and Communication

During our everyday lives, we are confronted with a vast amount of information from several sensory modalities. This multisensory information needs to be appropriately integrated for us to effectively engage with and learn from our world. Research carried out over the last half century has provided new insights into the way such multisensory processing improves human performance and perception; the neurophysiological foundations of multisensory function; the time course for its development; how multisensory abilities differ in clinical populations; and, most recently, the links between multisensory processing and cognitive abilities. This review summarizes the extant literature on multisensory function in typical and atypical circumstances, discusses the implications of the work carried out to date for theory and research, and points toward next steps for advancing the field.

Auditory-induced visual illusions in rodents measured by spontaneous behavioural response

When two brief sounds are presented with a short flash of light, we often perceive that the flash blinks twice. This phenomenon, called the “sound-induced flash illusion”, has been investigated as an example of how humans finely integrate multisensory information, more specifically, the temporal content of perception. However, it is unclear whether nonhuman animals experience the illusion. Therefore, we investigated whether the Mongolian gerbil, a rodent with relatively good eyesight, experiences this illusion. The novel object recognition (NOR) paradigm was used to evaluate the gerbil’s natural (i.e., untrained) capacity for multimodal integration. A light-emitting diode embedded within an object presented time-varying visual stimuli (different flashing patterns). The animals were first familiarised with repetitive single flashes. Then, various sound stimuli were introduced during test trials. An increase in exploration suggested that the animals perceived a flashing pattern differently only when the contradicting sound (double beeps) was presented simultaneously with a single flash. This result shows that the gerbil may experience the sound-induced flash illusion and indicates for the first time that rodents may have the capacity to integrate temporal content of perception in a sophisticated manner as do humans.

An insular view of the social decision-making network

Social animals must detect, evaluate and respond to the emotional states of other individuals in their group. A constellation of gestures, vocalizations, and chemosignals enable animals to convey affect and arousal to others in nuanced, multisensory ways. Observers integrate social information with environmental and internal factors to select behavioral responses to others via a process call social decision-making. The Social Decision Making Network (SDMN) is a system of brain structures and neurochemicals that are conserved across species (mammals, reptiles, amphibians, birds) that are the proximal mediators of most social behaviors. However, how sensory information reaches the SDMN to shape behavioral responses during a social encounter is not well known. Here we review the empirical data that demonstrate the necessity of sensory systems in detecting social stimuli, as well as the anatomical connectivity of sensory systems with each node of the SDMN. We conclude that the insular cortex is positioned to link integrated social sensory cues to this network to produce flexible and appropriate behavioral responses to socioemotional cues.

Adult-Onset Hearing Impairment Induces Layer-Specific Cortical Reorganization: Evidence of Crossmodal Plasticity and Central Gain Enhancement

Adult-onset hearing impairment can lead to hyperactivity in the auditory pathway (i.e., central gain enhancement) as well as increased cortical responsiveness to nonauditory stimuli (i.e., crossmodal plasticity). However, it remained unclear to what extent hearing loss-induced hyperactivity is relayed beyond the auditory cortex, and thus, whether central gain enhancement competes or coexists with crossmodal plasticity throughout the distinct layers of the audiovisual cortex. To that end, we investigated the effects of partial hearing loss on laminar processing in the auditory, visual and audiovisual cortices of adult rats using extracellular electrophysiological recordings performed 2 weeks after loud noise exposure. Current-source density analyses revealed that central gain enhancement was not relayed to the audiovisual cortex (V2L), and was instead restricted to the granular layer of the higher order auditory area, AuD. In contrast, crossmodal plasticity was evident across multiple cortical layers within V2L, and also manifested in AuD. Surprisingly, despite this coexistence of central gain enhancement and crossmodal plasticity, noise exposure did not disrupt the responsiveness of these neighboring cortical regions to combined audiovisual stimuli. Overall, we have shown for the first time that adult-onset hearing impairment causes a complex assortment of intramodal and crossmodal changes across the layers of higher order sensory cortices.

The circuit architecture of cortical multisensory processing: Distinct functions jointly operating within a common anatomical network

Our perceptual systems continuously process sensory inputs from different modalities and organize these streams of information such that our subjective representation of the outside world is a unified experience. By doing so, they also enable further cognitive processing and behavioral action. While cortical multisensory processing has been extensively investigated in terms of psychophysics and mesoscale neural correlates, an in depth understanding of the underlying circuit-level mechanisms is lacking. Previous studies on circuit-level mechanisms of multisensory processing have predominantly focused on cue integration, i.e. the mechanism by which sensory features from different modalities are combined to yield more reliable stimulus estimates than those obtained by using single sensory modalities. In this review, we expand the framework on the circuit-level mechanisms of cortical multisensory processing by highlighting that multisensory processing is a family of functions - rather than a single operation - which involves not only the integration but also the segregation of modalities. In addition, multisensory processing not only depends on stimulus features, but also on cognitive resources, such as attention and memory, as well as behavioral context, to determine the behavioral outcome. We focus on rodent models as a powerful instrument to study the circuit-level bases of multisensory processes, because they enable combining cell-type-specific recording and interventional techniques with complex behavioral paradigms. We conclude that distinct multisensory processes share overlapping anatomical substrates, are implemented by diverse neuronal micro-circuitries that operate in parallel, and are flexibly recruited based on factors such as stimulus features and behavioral constraints.

Behavioral Plasticity of Audiovisual Perception: Rapid Recalibration of Temporal Sensitivity but Not Perceptual Binding Following Adult-Onset Hearing Loss

The ability to accurately integrate or bind stimuli from more than one sensory modality is highly dependent on the features of the stimuli, such as their intensity and relative timing. Previous studies have demonstrated that the ability to perceptually bind stimuli is impaired in various clinical conditions such as autism, dyslexia, schizophrenia, as well as aging. However, it remains unknown if adult-onset hearing loss, separate from aging, influences audiovisual temporal acuity. In the present study, rats were trained using appetitive operant conditioning to perform an audiovisual temporal order judgment (TOJ) task or synchrony judgment (SJ) task in order to investigate the nature and extent that audiovisual temporal acuity is affected by adult-onset hearing loss, with a specific focus on the time-course of perceptual changes following loud noise exposure. In our first series of experiments, we found that audiovisual temporal acuity in normal-hearing rats was influenced by sound intensity, such that when a quieter sound was presented, the rats were biased to perceive the audiovisual stimuli as asynchronous (SJ task), or as though the visual stimulus was presented first (TOJ task). Psychophysical testing demonstrated that noise-induced hearing loss did not alter the rats' temporal sensitivity 2-3 weeks post-noise exposure, despite rats showing an initial difficulty in differentiating the temporal order of audiovisual stimuli. Furthermore, consistent with normal-hearing rats, the timing at which the stimuli were perceived as simultaneous (i.e., the point of subjective simultaneity, PSS) remained sensitive to sound intensity following hearing loss. Contrary to the TOJ task, hearing loss resulted in persistent impairments in asynchrony detection during the SJ task, such that a greater proportion of trials were now perceived as synchronous. Moreover, psychophysical testing found that noise-exposed rats had altered audiovisual synchrony perception, consistent with impaired audiovisual perceptual binding (e.g., an increase in the temporal window of integration on the right side of simultaneity; right temporal binding window (TBW)). Ultimately, our collective results show for the first time that adult-onset hearing loss leads to behavioral plasticity of audiovisual perception, characterized by a rapid recalibration of temporal sensitivity but a persistent impairment in the perceptual binding of audiovisual stimuli.

Women with Anorexia Nervosa do not show altered tactile localization compared to healthy controls

Body image disturbance is a key symptom of Anorexia Nervosa (AN). Previous studies found that women with AN overestimate their body size in comparison with healthy controls (HC), at least for unimodal measures involving either only visual input (e.g. distorted photographs technique) or only tactile input (e.g. tactile distance tasks). Distorted body representations are hypothesized to cause this misperception in AN. We here tested whether this overestimation remains present in a novel one-point-localization (OPL) task involving the mapping of a tactile stimulus onto a visual image. Two experiments compared the ability of 27 women with AN and 40 HC to accurately localize a tactile stimulus on a live image of their body. Women with AN and HC did not differ in their performance. Instead, participants in both groups showed systematic distortions in their localization performance. This study suggests that the mapping of a tactile stimulus does not involve a distorted body representation in women with AN compared to HC.

The opto-locomotor reflex as a tool to measure sensitivity to moving random dot patterns in mice

We designed a method to quantify mice visual function by measuring reflexive opto-locomotor responses. Mice were placed on a Styrofoam ball at the center of a large dome on the inside of which we projected moving random dot patterns. Because we fixed the heads of the mice in space and the ball was floating on pressurized air, locomotion of the mice was translated to rotation of the ball, which we registered. Sudden onsets of rightward or leftward moving patterns caused the mice to reflexively change their running direction. We quantified the opto-locomotor responses to different pattern speeds, luminance contrasts, and dot sizes. We show that the method is fast and reliable and the magnitude of the reflex is stable within sessions. We conclude that this opto-locomotor reflex method is suitable to quantify visual function in mice.

Audio-Visual Integration in a Redundant Target Paradigm: A Comparison between Rhesus Macaque and Man

The mechanisms underlying multi-sensory interactions are still poorly understood despite considerable progress made since the first neurophysiological recordings of multi-sensory neurons. While the majority of single-cell neurophysiology has been performed in anesthetized or passive-awake laboratory animals, the vast majority of behavioral data stems from studies with human subjects. Interpretation of neurophysiological data implicitly assumes that laboratory animals exhibit perceptual phenomena comparable or identical to those observed in human subjects. To explicitly test this underlying assumption, we here characterized how two rhesus macaques and four humans detect changes in intensity of auditory, visual, and audio-visual stimuli. These intensity changes consisted of a gradual envelope modulation for the sound, and a luminance step for the LED. Subjects had to detect any perceived intensity change as fast as possible. By comparing the monkeys' results with those obtained from the human subjects we found that (1) unimodal reaction times differed across modality, acoustic modulation frequency, and species, (2) the largest facilitation of reaction times with the audio-visual stimuli was observed when stimulus onset asynchronies were such that the unimodal reactions would occur at the same time (response, rather than physical synchrony), and (3) the largest audio-visual reaction-time facilitation was observed when unimodal auditory stimuli were difficult to detect, i.e., at slow unimodal reaction times. We conclude that despite marked unimodal heterogeneity, similar multisensory rules applied to both species. Single-cell neurophysiology in the rhesus macaque may therefore yield valuable insights into the mechanisms governing audio-visual integration that may be informative of the processes taking place in the human brain.

An autism-associated serotonin transporter variant disrupts multisensory processing

Altered sensory processing is observed in many children with autism spectrum disorder (ASD), with growing evidence that these impairments extend to the integration of information across the different senses (that is, multisensory function). The serotonin system has an important role in sensory development and function, and alterations of serotonergic signaling have been suggested to have a role in ASD. A gain-of-function coding variant in the serotonin transporter (SERT) associates with sensory aversion in humans, and when expressed in mice produces traits associated with ASD, including disruptions in social and communicative function and repetitive behaviors. The current study set out to test whether these mice also exhibit changes in multisensory function when compared with wild-type (WT) animals on the same genetic background. Mice were trained to respond to auditory and visual stimuli independently before being tested under visual, auditory and paired audiovisual (multisensory) conditions. WT mice exhibited significant gains in response accuracy under audiovisual conditions. In contrast, although the SERT mutant animals learned the auditory and visual tasks comparably to WT littermates, they failed to show behavioral gains under multisensory conditions. We believe these results provide the first behavioral evidence of multisensory deficits in a genetic mouse model related to ASD and implicate the serotonin system in multisensory processing and in the multisensory changes seen in ASD.

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Extensive research on humans has improved our understanding of how the brain integrates information from our different senses, and has begun to uncover the brain regions and large-scale neural activity that contributes to an observer’s ability to perceive the relative timing of auditory and visual stimuli. In the present study, we developed the first behavioral tasks to assess the perception of audiovisual temporal synchrony in rats. Modeled after the parameters used in human studies, separate groups of rats were trained to perform: (1) a simultaneity judgment task in which they reported whether audiovisual stimuli at various stimulus onset asynchronies (SOAs) were presented simultaneously or not; and (2) a temporal order judgment task in which they reported whether they perceived the auditory or visual stimulus to have been presented first. Furthermore, using in vivo electrophysiological recordings in the lateral extrastriate visual (V2L) cortex of anesthetized rats, we performed the first investigation of how neurons in the rat multisensory cortex integrate audiovisual stimuli presented at different SOAs. As predicted, rats (n = 7) trained to perform the simultaneity judgment task could accurately (~80%) identify synchronous vs. asynchronous (200 ms SOA) trials. Moreover, the rats judged trials at 10 ms SOA to be synchronous, whereas the majority (~70%) of trials at 100 ms SOA were perceived to be asynchronous. During the temporal order judgment task, rats (n = 7) perceived the synchronous audiovisual stimuli to be “visual first” for ~52% of the trials, and calculation of the smallest timing interval between the auditory and visual stimuli that could be detected in each rat (i.e., the just noticeable difference (JND)) ranged from 77 ms to 122 ms. Neurons in the rat V2L cortex were sensitive to the timing of audiovisual stimuli, such that spiking activity was greatest during trials when the visual stimulus preceded the auditory by 20–40 ms. Ultimately, given that our behavioral and electrophysiological results were consistent with studies conducted on human participants and previous recordings made in multisensory brain regions of different species, we suggest that the rat represents an effective model for studying audiovisual temporal synchrony at both the neuronal and perceptual level.

Behavioral, perceptual, and neural alterations in sensory and multisensory function in autism spectrum disorder

Although sensory processing challenges have been noted since the first clinical descriptions of autism, it has taken until the release of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) in 2013 for sensory problems to be included as part of the core symptoms of autism spectrum disorder (ASD) in the diagnostic profile. Because sensory information forms the building blocks for higher-order social and cognitive functions, we argue that sensory processing is not only an additional piece of the puzzle, but rather a critical cornerstone for characterizing and understanding ASD. In this review we discuss what is currently known about sensory processing in ASD, how sensory function fits within contemporary models of ASD, and what is understood about the differences in the underlying neural processing of sensory and social communication observed between individuals with and without ASD. In addition to highlighting the sensory features associated with ASD, we also emphasize the importance of multisensory processing in building perceptual and cognitive representations, and how deficits in multisensory integration may also be a core characteristic of ASD.