Sound-driven synaptic inhibition in primary visual cortex

Mendelian randomization analyses support causal relationships between tinnitus of different stages and severity and structural characteristics of specific brain regions

This study aims to delineate the causal relationships between idiopathic tinnitus in different stages and severity and the morphological properties in specific brain regions. We utilized a two-sample bidirectional Mendelian randomization (MR) analysis to ascertain the causal effects of brain structural attributes on varying severities and stages of tinnitus. Our approach involved harnessing genetic variables derived from extensive genome-wide association studies as instrumental variables, centered mainly on pertinent single-nucleotide polymorphisms associated with tinnitus. Subsequently, we integrated this data with brain structural imaging inputs to facilitate the MR analysis. We also applied reverse MR analysis to pinpoint the critical brain regions implicated in the onset of tinnitus. Our analysis revealed a demonstrable causal relationship between tinnitus and brain structural alterations, including changes primarily within the auditory cortex and hub regions of the limbic system, as well as portions of the frontal-temporal-occipital circuit. We found that individuals exhibiting cortical thickness alterations in the bilateral peri-calcarine and right superior occipital gyrus might have previously experienced tinnitus. Changes in the cortical areas of the right rectus, left inferior frontal gyrus, and right pars-orbitalis appeared unrelated to tinnitus. Furthermore, moderate tinnitus patients showed more pronounced structural alterations. This study substantiates that tinnitus could instigate substantial structural alterations mainly within the auditory-limbic-frontal-visual system, while the reciprocal causality was not supported. Moreover, the data underscores that moderate, rather than severe, tinnitus precipitates the most significant structural changes. Morphological alterations in several specific brain areas either indicate a history of tinnitus or bear no relation to it.

Differences in brain functional connectivity between tinnitus with or without hearing loss

To explore the differences in brain imaging in tinnitus with or without hearing loss (HL). We acquired functional MRI scans from 26 tinnitus patients with HL (tinnitus-HL), 24 tinnitus patients with no HL (tinnitus-NHL), and 26 healthy controls (HCs) matched by age and sex. The left and right thalamus were selected as seeds to study the endogenous functional connectivity (FC) of the whole brain, and its correlation with clinical indices was analyzed. Brain regions showing FC differences among the three groups included the Heschl gyrus (HES), right Hippocampus (HIP), right Amygdala (AMYG), left Calcarine fissure and surrounding cortex (CAL). Post hoc analysis showed that the thalamus-HIP connection and thalamus-lingual gyrus (LING) connection were enhanced in the tinnitus-NHL group, as compared to tinnitus-HL. Compared with HCs, the tinnitus-NHL group showed an enhanced connection between the thalamus and the left Inferior occipital gyrus, left CAL and LING. While in the tinnitus-HL group, the connection between the thalamus and several brain regions (right HES, right AMYG, etc) was weakened. In the tinnitus-HL group, the tinnitus handicap inventory scores were positively correlated with the FC of the left thalamus and right HES, right thalamus and right Rolandic operculum. The duration of tinnitus was negatively correlated with the FC of the right thalamus and right HIP. Abnormal FC in the thalamus may play an important role in the pathogenesis of tinnitus. Tinnitus-NHL and tinnitus-HL show different connection patterns, indicating that there are some differences in their pathogenesis.

Behavior-related visual activations in the auditory cortex of nonhuman primates

While it is well established that sensory cortical regions traditionally thought to be unimodal can be activated by stimuli from modalities other than the dominant one, functions of such foreign-modal activations are still not clear. Here we show that visual activations in early auditory cortex can be related to whether or not the monkeys engaged in audio-visual tasks, to the time when the monkeys reacted to the visual component of such tasks, and to the correctness of the monkeys' response to the auditory component of such tasks. These relationships between visual activations and behavior suggest that auditory cortex can be recruited for visually-guided behavior and that visual activations can prime auditory cortex such that it is prepared for processing future sounds. Our study thus provides evidence that foreign-modal activations in sensory cortex can contribute to a subject's ability to perform tasks on stimuli from foreign and dominant modalities.

Multisensory Integration Underlies the Distinct Representation of Odor-Taste Mixtures in the Gustatory Cortex of Behaving Rats

The perception of food relies on the integration of olfactory and gustatory signals originating from the mouth. This multisensory process generates robust associations between odors and tastes, significantly influencing the perceptual judgment of flavors. However, the specific neural substrates underlying this integrative process remain unclear. Previous electrophysiological studies identified the gustatory cortex as a site of convergent olfactory and gustatory signals, but whether neurons represent multimodal odor-taste mixtures as distinct from their unimodal odor and taste components is unknown. To investigate this, we recorded single-unit activity in the gustatory cortex of behaving female rats during the intraoral delivery of individual odors, individual tastes, and odor-taste mixtures. Our results demonstrate that chemoselective neurons in the gustatory cortex are broadly responsive to intraoral chemosensory stimuli, exhibiting time-varying multiphasic changes in activity. In a subset of these chemoselective neurons, odor-taste mixtures elicit nonlinear cross-modal responses that distinguish them from their olfactory and gustatory components. These findings provide novel insights into multimodal chemosensory processing by the gustatory cortex, highlighting the distinct representation of unimodal and multimodal intraoral chemosensory signals. Overall, our findings suggest that olfactory and gustatory signals interact nonlinearly in the gustatory cortex to enhance the identity coding of both unimodal and multimodal chemosensory stimuli.

A nasal chemosensation-dependent critical window for somatosensory development

Nasal chemosensation is considered the evolutionarily oldest mammalian sense and, together with somatosensation, is crucial for neonatal well-being before auditory and visual pathways start engaging the brain. Using anatomical and functional approaches in mice, we reveal that odor-driven activity propagates to a large part of the cortex during the first postnatal week and enhances whisker-evoked activation of primary whisker somatosensory cortex (wS1). This effect disappears in adult animals, in line with the loss of excitatory connectivity from olfactory cortex to wS1. By performing neonatal odor deprivation, followed by electrophysiological and behavioral work in adult animals, we identify a key transient regulation of nasal chemosensory information necessary for the development of wS1 sensory-driven dynamics and somatosensation. Our work uncovers a cross-modal critical window for nasal chemosensation-dependent somatosensory functional maturation.

Tactile adaptation to orientation produces a robust tilt aftereffect and exhibits crossmodal transfer when tested in vision

Orientation processing is one of the most fundamental functions in both visual and somatosensory perception. Converging findings suggest that orientation processing in both modalities is closely linked: somatosensory neurons share a similar orientation organisation as visual neurons, and the visual cortex has been found to be heavily involved in tactile orientation perception. Hence, we hypothesized that somatosensation would exhibit a similar orientation adaptation effect, and this adaptation effect would be transferable between the two modalities, considering the above-mentioned connection. The tilt aftereffect (TAE) is a demonstration of orientation adaptation and is used widely in behavioural experiments to investigate orientation mechanisms in vision. By testing the classic TAE paradigm in both tactile and crossmodal orientation tasks between vision and touch, we were able to show that tactile perception of orientation shows a very robust TAE, similar to its visual counterpart. We further show that orientation adaptation in touch transfers to produce a TAE when tested in vision, but not vice versa. Additionally, when examining the test sequence following adaptation for serial effects, we observed another asymmetry between the two conditions where the visual test sequence displayed a repulsive intramodal serial dependence effect while the tactile test sequence exhibited an attractive serial dependence. These findings provide concrete evidence that vision and touch engage a similar orientation processing mechanism. However, the asymmetry in the crossmodal transfer of TAE and serial dependence points to a non-reciprocal connection between the two modalities, providing further insights into the underlying processing mechanism.

Cortical cellular encoding of thermotactile integration

Recent evidence suggests that primary sensory cortical regions play a role in the integration of information from multiple sensory modalities. How primary cortical neurons integrate different sources of sensory information is unclear, partly because non-primary sensory input to a cortical sensory region is often weak or modulatory. To address this question, we take advantage of the robust representation of thermal (cooling) and tactile stimuli in mouse forelimb primary somatosensory cortex (fS1). Using a thermotactile detection task, we show that the perception of threshold-level cool or tactile information is enhanced when they are presented simultaneously, compared with presentation alone. To investigate the cortical cellular correlates of thermotactile integration, we performed in vivo extracellular recordings from fS1 in awake resting and anesthetized mice during unimodal and bimodal stimulation of the forepaw. Unimodal stimulation evoked thermal- or tactile- specific excitatory and inhibitory responses of fS1 neurons. The most prominent features of combined thermotactile stimulation are the recruitment of unimodally silent fS1 neurons, non-linear integration features, and response dynamics that favor longer response durations with additional spikes. Together, we identify quantitative and qualitative changes in cortical encoding that may underlie the improvement in perception of thermotactile surfaces during haptic exploration.

Dynamic modulation of mouse thalamocortical visual activity by salient sounds

Visual responses of the primary visual cortex (V1) are altered by sound. Sound-driven behavioral arousal suggests that, in addition to direct inputs from the primary auditory cortex (A1), multiple other sources may shape V1 responses to sound. Here, we show in anesthetized mice that sound (white noise, ≥70dB) drives a biphasic modulation of V1 visually driven gamma-band activity, comprising fast-transient inhibitory and slow, prolonged excitatory (A1-independent) arousal-driven components. An analogous yet quicker modulation of the visual response also occurred earlier in the visual pathway, at the level of the dorsolateral geniculate nucleus (dLGN), where sound transiently inhibited the early phasic visual response and subsequently induced a prolonged increase in tonic spiking activity and gamma rhythmicity. Our results demonstrate that sound-driven modulations of visual activity are not exclusive to V1 and suggest that thalamocortical inputs from the dLGN to V1 contribute to shaping V1 visual response to sound.

A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration

Tactile sensation and vision are often both utilized for the exploration of objects that are within reach though it is not known whether or how these two distinct sensory systems combine such information. Here in mice, we used a combination of stereo photogrammetry for 3D reconstruction of the whisker array, brain-wide anatomical tracing and functional connectivity analysis to explore the possibility of tacto-visual convergence in sensory space and within the circuitry of the primary visual cortex (VISp). Strikingly, we find that stimulation of the contralateral whisker array suppresses visually evoked activity in a tacto-visual sub-region of VISp whose visual space representation closely overlaps with the whisker search space. This suppression is mediated by local fast-spiking interneurons that receive a direct cortico-cortical input predominantly from layer 6 neurons located in the posterior primary somatosensory barrel cortex (SSp-bfd). These data demonstrate functional convergence within and between two primary sensory cortical areas for multisensory object detection and recognition.

Auditory cortex conveys non-topographic sound localization signals to visual cortex

Spatiotemporally congruent sensory stimuli are fused into a unified percept. The auditory cortex (AC) sends projections to the primary visual cortex (V1), which could provide signals for binding spatially corresponding audio-visual stimuli. However, whether AC inputs in V1 encode sound location remains unknown. Using two-photon axonal calcium imaging and a speaker array, we measured the auditory spatial information transmitted from AC to layer 1 of V1. AC conveys information about the location of ipsilateral and contralateral sound sources to V1. Sound location could be accurately decoded by sampling AC axons in V1, providing a substrate for making location-specific audiovisual associations. However, AC inputs were not retinotopically arranged in V1, and audio-visual modulations of V1 neurons did not depend on the spatial congruency of the sound and light stimuli. The non-topographic sound localization signals provided by AC might allow the association of specific audiovisual spatial patterns in V1 neurons.

Triple dissociation of visual, auditory and motor processing in mouse primary visual cortex

Primary sensory cortices respond to crossmodal stimuli-for example, auditory responses are found in primary visual cortex (V1). However, it remains unclear whether these responses reflect sensory inputs or behavioral modulation through sound-evoked body movement. We address this controversy by showing that sound-evoked activity in V1 of awake mice can be dissociated into auditory and behavioral components with distinct spatiotemporal profiles. The auditory component began at approximately 27 ms, was found in superficial and deep layers and originated from auditory cortex. Sound-evoked orofacial movements correlated with V1 neural activity starting at approximately 80-100 ms and explained auditory frequency tuning. Visual, auditory and motor activity were expressed by different laminar profiles and largely segregated subsets of neuronal populations. During simultaneous audiovisual stimulation, visual representations remained dissociable from auditory-related and motor-related activity. This three-fold dissociability of auditory, motor and visual processing is central to understanding how distinct inputs to visual cortex interact to support vision.

Rodents' visual gamma as a biomarker of pathological neural conditions

Neural gamma oscillations (indicatively 30-100 Hz) are ubiquitous: they are associated with a broad range of functions in multiple cortical areas and across many animal species. Experimental and computational works established gamma rhythms as a global emergent property of neuronal networks generated by the balanced and coordinated interaction of excitation and inhibition. Coherently, gamma activity is strongly influenced by the alterations of synaptic dynamics which are often associated with pathological neural dysfunctions. We argue therefore that these oscillations are an optimal biomarker for probing the mechanism of cortical dysfunctions. Gamma oscillations are also highly sensitive to external stimuli in sensory cortices, especially the primary visual cortex (V1), where the stimulus dependence of gamma oscillations has been thoroughly investigated. Gamma manipulation by visual stimuli tuning is particularly easy in rodents, which have become a standard animal model for investigating the effects of network alterations on gamma oscillations. Overall, gamma in the rodents' visual cortex offers an accessible probe on dysfunctional information processing in pathological conditions. Beyond vision-related dysfunctions, alterations of gamma oscillations in rodents were indeed also reported in neural deficits such as migraine, epilepsy and neurodegenerative or neuropsychiatric conditions such as Alzheimer's, schizophrenia and autism spectrum disorders. Altogether, the connections between visual cortical gamma activity and physio-pathological conditions in rodent models underscore the potential of gamma oscillations as markers of neuronal (dys)functioning.

Older adults preserve audiovisual integration through enhanced cortical activations, not by recruiting new regions

Effective interactions with the environment rely on the integration of multisensory signals: Our brains must efficiently combine signals that share a common source, and segregate those that do not. Healthy ageing can change or impair this process. This functional magnetic resonance imaging study assessed the neural mechanisms underlying age differences in the integration of auditory and visual spatial cues. Participants were presented with synchronous audiovisual signals at various degrees of spatial disparity and indicated their perceived sound location. Behaviourally, older adults were able to maintain localisation accuracy. At the neural level, they integrated auditory and visual cues into spatial representations along dorsal auditory and visual processing pathways similarly to their younger counterparts but showed greater activations in a widespread system of frontal, temporal, and parietal areas. According to multivariate Bayesian decoding, these areas encoded critical stimulus information beyond that which was encoded in the brain areas commonly activated by both groups. Surprisingly, however, the boost in information provided by these areas with age-related activation increases was comparable across the 2 age groups. This dissociation-between comparable information encoded in brain activation patterns across the 2 age groups, but age-related increases in regional blood-oxygen-level-dependent responses-contradicts the widespread notion that older adults recruit new regions as a compensatory mechanism to encode task-relevant information. Instead, our findings suggest that activation increases in older adults reflect nonspecific or modulatory mechanisms related to less efficient or slower processing, or greater demands on attentional resources.

Sound suppresses earliest visual cortical processing after sight recovery in congenitally blind humans

Neuroscientific research has consistently shown more extensive non-visual activity in the visual cortex of congenitally blind humans compared to sighted controls; a phenomenon known as crossmodal plasticity. Whether or not crossmodal activation of the visual cortex retracts if sight can be restored is still unknown. The present study, involving a rare group of sight-recovery individuals who were born pattern vision blind, employed visual event-related potentials to investigate persisting crossmodal modulation of the initial visual cortical processing stages. Here we report that the earliest, stimulus-driven retinotopic visual cortical activity (<100 ms) was suppressed in a spatially specific manner in sight-recovery individuals when concomitant sounds accompanied visual stimulation. In contrast, sounds did not modulate the earliest visual cortical response in two groups of typically sighted controls, nor in a third control group of sight-recovery individuals who had suffered a transient phase of later (rather than congenital) visual impairment. These results provide strong evidence for persisting crossmodal activity in the visual cortex after sight recovery following a period of congenital visual deprivation. Based on the time course of this modulation, we speculate on a role of exuberant crossmodal thalamic input which may arise during a sensitive phase of brain development.

Layer 1 neocortex: Gating and integrating multidimensional signals

Layer 1 (L1) of the neocortex acts as a nexus for the collection and processing of widespread information. By integrating ascending inputs with extensive top-down activity, this layer likely provides critical information regulating how the perception of sensory inputs is reconciled with expectation. This is accomplished by sorting, directing, and integrating the complex network of excitatory inputs that converge onto L1. These signals are combined with neuromodulatory afferents and gated by the wealth of inhibitory interneurons that either are embedded within L1 or send axons from other cortical layers. Together, these interactions dynamically calibrate information flow throughout the neocortex. This review will primarily focus on L1 within the primary sensory cortex and will use these insights to understand L1 in other cortical areas.

Multimodal measures of spontaneous brain activity reveal both common and divergent patterns of cortical functional organization

Large-scale functional networks have been characterized in both rodent and human brains, typically by analyzing fMRI-BOLD signals. However, the relationship between fMRI-BOLD and underlying neural activity is complex and incompletely understood, which poses challenges to interpreting network organization obtained using this technique. Additionally, most work has assumed a disjoint functional network organization (i.e., brain regions belong to one and only one network). Here, we employ wide-field Ca2+ imaging simultaneously with fMRI-BOLD in mice expressing GCaMP6f in excitatory neurons. We determine cortical networks discovered by each modality using a mixed-membership algorithm to test the hypothesis that functional networks exhibit overlapping organization. We find that there is considerable network overlap (both modalities) in addition to disjoint organization. Our results show that multiple BOLD networks are detected via Ca2+ signals, and networks determined by low-frequency Ca2+ signals are only modestly more similar to BOLD networks. In addition, the principal gradient of functional connectivity is nearly identical for BOLD and Ca2+ signals. Despite similarities, important differences are also detected across modalities, such as in measures of functional connectivity strength and diversity. In conclusion, Ca2+ imaging uncovers overlapping functional cortical organization in the mouse that reflects several, but not all, properties observed with fMRI-BOLD signals.

The Development of Multisensory Integration at the Neuronal Level

Multisensory integration is a fundamental function of the brain. In the typical adult, multisensory neurons' response to paired multisensory (e.g., audiovisual) cues is significantly more robust than the corresponding best unisensory response in many brain regions. Synthesizing sensory signals from multiple modalities can speed up sensory processing and improve the salience of outside events or objects. Despite its significance, multisensory integration is testified to be not a neonatal feature of the brain. Neurons' ability to effectively combine multisensory information does not occur rapidly but develops gradually during early postnatal life (for cats, 4-12 weeks required). Multisensory experience is critical for this developing process. If animals were restricted from sensing normal visual scenes or sounds (deprived of the relevant multisensory experience), the development of the corresponding integrative ability could be blocked until the appropriate multisensory experience is obtained. This section summarizes the extant literature on the development of multisensory integration (mainly using cat superior colliculus as a model), sensory-deprivation-induced cross-modal plasticity, and how sensory experience (sensory exposure and perceptual learning) leads to the plastic change and modification of neural circuits in cortical and subcortical areas.

More Than the Sum of Its Parts: Visual-Tactile Integration in the Behaving Rat

We experience the world by constantly integrating cues from multiple modalities to form unified sensory percepts. Once familiar with multimodal properties of an object, we can recognize it regardless of the modality involved. In this chapter we will examine the case of a visual-tactile orientation categorization experiment in rats. We will explore the involvement of the cerebral cortex in recognizing objects through multiple sensory modalities. In the orientation categorization task, rats learned to examine and judge the orientation of a raised, black and white grating using touch, vision, or both. Their multisensory performance was better than the predictions of linear models for cue combination, indicating synergy between the two sensory channels. Neural recordings made from a candidate associative cortical area, the posterior parietal cortex (PPC), reflected the principal neuronal correlates of the behavioral results: PPC neurons encoded both graded information about the object and categorical information about the animal's decision. Intriguingly single neurons showed identical responses under each of the three modality conditions providing a substrate for a neural circuit in the cortex that is involved in modality-invariant processing of objects.

Hypersensitivity to Distractors in Fragile X Syndrome from Loss of Modulation of Cortical VIP Interneurons

Attention deficit is one of the most prominent and disabling symptoms in Fragile X syndrome (FXS). Hypersensitivity to sensory stimuli contributes to attention difficulties by overwhelming and/or distracting affected individuals, which disrupts activities of daily living at home and learning at school. We find that auditory or visual distractors selectively impair visual discrimination performance in humans and mice with FXS but not in typically developing controls. In both species, males and females were examined. Vasoactive intestinal polypeptide (VIP) neurons were significantly modulated by incorrect responses in the poststimulus period during early distractor trials in WT mice, consistent with their known role as error signals. Strikingly, however, VIP cells from Fmr1 -/- mice showed little modulation in error trials, and this correlated with their poor performance on the distractor task. Thus, VIP interneurons and their reduced modulatory influence on pyramidal cells could be a potential therapeutic target for attentional difficulties in FXS.SIGNIFICANCE STATEMENT Sensory hypersensitivity, impulsivity, and persistent inattention are among the most consistent clinical features of FXS, all of which impede daily functioning and create barriers to learning. However, the neural mechanisms underlying sensory over-reactivity remain elusive. To overcome a significant challenge in translational FXS research we demonstrate a compelling alignment of sensory over-reactivity in both humans with FXS and Fmr1 -/- mice (the principal animal model of FXS) using a novel analogous distractor task. Two-photon microscopy in mice revealed that lack of modulation by VIP cells contributes to susceptibility to distractors. Implementing research efforts we describe here can help identify dysfunctional neural mechanisms associated not only with sensory issues but broader impairments, including those in learning and cognition.

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Visuo-vestibular heading perception: a model system to study multi-sensory decision making

Integrating noisy signals across time as well as sensory modalities, a process named multi-sensory decision making (MSDM), is an essential strategy for making more accurate and sensitive decisions in complex environments. Although this field is just emerging, recent extraordinary works from different perspectives, including computational theory, psychophysical behaviour and neurophysiology, begin to shed new light onto MSDM. In the current review, we focus on MSDM by using a model system of visuo-vestibular heading. Combining well-controlled behavioural paradigms on virtual-reality systems, single-unit recordings, causal manipulations and computational theory based on spiking activity, recent progress reveals that vestibular signals contain complex temporal dynamics in many brain regions, including unisensory, multi-sensory and sensory-motor association areas. This challenges the brain for cue integration across time and sensory modality such as optic flow which mainly contains a motion velocity signal. In addition, new evidence from the higher-level decision-related areas, mostly in the posterior and frontal/prefrontal regions, helps revise our conventional thought on how signals from different sensory modalities may be processed, converged, and moment-by-moment accumulated through neural circuits for forming a unified, optimal perceptual decision. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

How 'visual' is the visual cortex? The interactions between the visual cortex and other sensory, motivational and motor systems as enabling factors for visual perception

The definition of the visual cortex is primarily based on the evidence that lesions of this area impair visual perception. However, this does not exclude that the visual cortex may process more information than of retinal origin alone, or that other brain structures contribute to vision. Indeed, research across the past decades has shown that non-visual information, such as neural activity related to reward expectation and value, locomotion, working memory and other sensory modalities, can modulate primary visual cortical responses to retinal inputs. Nevertheless, the function of this non-visual information is poorly understood. Here we review recent evidence, coming primarily from studies in rodents, arguing that non-visual and motor effects in visual cortex play a role in visual processing itself, for instance disentangling direct auditory effects on visual cortex from effects of sound-evoked orofacial movement. These findings are placed in a broader framework casting vision in terms of predictive processing under control of frontal, reward- and motor-related systems. In contrast to the prevalent notion that vision is exclusively constructed by the visual cortical system, we propose that visual percepts are generated by a larger network-the extended visual system-spanning other sensory cortices, supramodal areas and frontal systems. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Mouse frontal cortex mediates additive multisensory decisions

The brain can combine auditory and visual information to localize objects. However, the cortical substrates underlying audiovisual integration remain uncertain. Here, we show that mouse frontal cortex combines auditory and visual evidence; that this combination is additive, mirroring behavior; and that it evolves with learning. We trained mice in an audiovisual localization task. Inactivating frontal cortex impaired responses to either sensory modality, while inactivating visual or parietal cortex affected only visual stimuli. Recordings from >14,000 neurons indicated that after task learning, activity in the anterior part of frontal area MOs (secondary motor cortex) additively encodes visual and auditory signals, consistent with the mice's behavioral strategy. An accumulator model applied to these sensory representations reproduced the observed choices and reaction times. These results suggest that frontal cortex adapts through learning to combine evidence across sensory cortices, providing a signal that is transformed into a binary decision by a downstream accumulator.

Spike-based coupling between single neurons and populations across rat sensory cortices, perirhinal cortex, and hippocampus

Cortical computations require coordination of neuronal activity within and across multiple areas. We characterized spiking relationships within and between areas by quantifying coupling of single neurons to population firing patterns. Single-neuron population coupling (SNPC) was investigated using ensemble recordings from hippocampal CA1 region and somatosensory, visual, and perirhinal cortices. Within-area coupling was heterogeneous across structures, with area CA1 showing higher levels than neocortical regions. In contrast to known anatomical connectivity, between-area coupling showed strong firing coherence of sensory neocortices with CA1, but less with perirhinal cortex. Cells in sensory neocortices and CA1 showed positive correlations between within- and between-area coupling; these were weaker for perirhinal cortex. All four areas harbored broadcasting cells, connecting to multiple external areas, which was uncorrelated to within-area coupling strength. When examining correlations between SNPC and spatial coding, we found that, if such correlations were significant, they were negative. This result was consistent with an overall preservation of SNPC across different brain states, suggesting a strong dependence on intrinsic network connectivity. Overall, SNPC offers an important window on cell-to-population synchronization in multi-area networks. Instead of pointing to specific information-coding functions, our results indicate a primary function of SNPC in dynamically organizing communication in systems composed of multiple, interconnected areas.

Microglia enable cross-modal plasticity by removing inhibitory synapses

Cross-modal plasticity is the repurposing of brain regions associated with deprived sensory inputs to improve the capacity of other sensory modalities. The functional mechanisms of cross-modal plasticity can indicate how the brain recovers from various forms of injury and how different sensory modalities are integrated. Here, we demonstrate that rewiring of the microglia-mediated local circuit synapse is crucial for cross-modal plasticity induced by visual deprivation (monocular deprivation [MD]). MD relieves the usual inhibition of functional connectivity between the somatosensory cortex and secondary lateral visual cortex (V2L). This results in enhanced excitatory responses in V2L neurons during whisker stimulation and a greater capacity for vibrissae sensory discrimination. The enhanced cross-modal response is mediated by selective removal of inhibitory synapse terminals on pyramidal neurons by the microglia in the V2L via matrix metalloproteinase 9 signaling. Our results provide insights into how cortical circuits integrate different inputs to functionally compensate for neuronal damage.

Sound Improves Neuronal Encoding of Visual Stimuli in Mouse Primary Visual Cortex

In everyday life, we integrate visual and auditory information in routine tasks such as navigation and communication. While concurrent sound can improve visual perception, the neuronal correlates of audiovisual integration are not fully understood. Specifically, it remains unclear whether neuronal firing patters in the primary visual cortex (V1) of awake animals demonstrate similar sound-induced improvement in visual discriminability. Furthermore, presentation of sound is associated with movement in the subjects, but little is understood about whether and how sound-associated movement affects audiovisual integration in V1. Here, we investigated how sound and movement interact to modulate V1 visual responses in awake, head-fixed mice and whether this interaction improves neuronal encoding of the visual stimulus. We presented visual drifting gratings with and without simultaneous auditory white noise to awake mice while recording mouse movement and V1 neuronal activity. Sound modulated activity of 80% of light-responsive neurons, with 95% of neurons increasing activity when the auditory stimulus was present. A generalized linear model (GLM) revealed that sound and movement had distinct and complementary effects of the neuronal visual responses. Furthermore, decoding of the visual stimulus from the neuronal activity was improved with sound, an effect that persisted even when controlling for movement. These results demonstrate that sound and movement modulate visual responses in complementary ways, improving neuronal representation of the visual stimulus. This study clarifies the role of movement as a potential confound in neuronal audiovisual responses and expands our knowledge of how multimodal processing is mediated at a neuronal level in the awake brain.SIGNIFICANCE STATEMENT Sound and movement are both known to modulate visual responses in the primary visual cortex; however, sound-induced movement has largely remained unaccounted for as a potential confound in audiovisual studies in awake animals. Here, authors found that sound and movement both modulate visual responses in an important visual brain area, the primary visual cortex, in distinct, yet complementary ways. Furthermore, sound improved encoding of the visual stimulus even when accounting for movement. This study reconciles contrasting theories on the mechanism underlying audiovisual integration and asserts the primary visual cortex as a key brain region participating in tripartite sensory interactions.

Effects of developmental alcohol exposure on cortical multisensory integration

Fetal alcohol spectrum disorder (FASD) is one of the most common causes of mental disabilities in the world with a prevalence of 1%-6% of all births. Sensory processing deficits and cognitive problems are a major feature in this condition. Because developmental alcohol exposure can impair neuronal plasticity, and neuronal plasticity is crucial for the establishment of neuronal circuits in sensory areas, we predicted that exposure to alcohol during the third trimester equivalent of human gestation would disrupt the development of multisensory integration (MSI) in the rostral portion of the posterior parietal cortex (PPr), an integrative visual-tactile area. We conducted in vivo electrophysiology in 17 ferrets from four groups (saline/alcohol; infancy/adolescence). A total of 1157 neurons were recorded after visual, tactile and combined visual-tactile stimulation. A multisensory (MS) enhancement or suppression is characterized by a significantly increased or decreased number of elicited spikes after combined visual-tactile stimulation compared to the strongest unimodal (visual or tactile) response. At the neuronal level, those in infant animals were more prone to show MS suppression whereas adolescents were more prone to show MS enhancement. Although alcohol-treated animals showed similar developmental changes between infancy and adolescence, they always 'lagged behind' controls showing more MS suppression and less enhancement. Our findings suggest that alcohol exposure during the last months of human gestation would stunt the development of MSI, which could underlie sensory problems seen in FASD.

Behavioral origin of sound-evoked activity in mouse visual cortex

Sensory cortices can be affected by stimuli of multiple modalities and are thus increasingly thought to be multisensory. For instance, primary visual cortex (V1) is influenced not only by images but also by sounds. Here we show that the activity evoked by sounds in V1, measured with Neuropixels probes, is stereotyped across neurons and even across mice. It is independent of projections from auditory cortex and resembles activity evoked in the hippocampal formation, which receives little direct auditory input. Its low-dimensional nature starkly contrasts the high-dimensional code that V1 uses to represent images. Furthermore, this sound-evoked activity can be precisely predicted by small body movements that are elicited by each sound and are stereotyped across trials and mice. Thus, neural activity that is apparently multisensory may simply arise from low-dimensional signals associated with internal state and behavior.

Metaplasticity framework for cross-modal synaptic plasticity in adults

Sensory loss leads to widespread adaptation of neural circuits to mediate cross-modal plasticity, which allows the organism to better utilize the remaining senses to guide behavior. While cross-modal interactions are often thought to engage multisensory areas, cross-modal plasticity is often prominently observed at the level of the primary sensory cortices. One dramatic example is from functional imaging studies in humans where cross-modal recruitment of the deprived primary sensory cortex has been observed during the processing of the spared senses. In addition, loss of a sensory modality can lead to enhancement and refinement of the spared senses, some of which have been attributed to compensatory plasticity of the spared sensory cortices. Cross-modal plasticity is not restricted to early sensory loss but is also observed in adults, which suggests that it engages or enables plasticity mechanisms available in the adult cortical circuit. Because adult cross-modal plasticity is observed without gross anatomical connectivity changes, it is thought to occur mainly through functional plasticity of pre-existing circuits. The underlying cellular and molecular mechanisms involve activity-dependent homeostatic and Hebbian mechanisms. A particularly attractive mechanism is the sliding threshold metaplasticity model because it innately allows neurons to dynamically optimize their feature selectivity. In this mini review, I will summarize the cellular and molecular mechanisms that mediate cross-modal plasticity in the adult primary sensory cortices and evaluate the metaplasticity model as an effective framework to understand the underlying mechanisms.

Hypersensitivity to distractors in Fragile X syndrome from loss of modulation of cortical VIP interneurons

Attention deficit is one of the most prominent and disabling symptoms in Fragile X Syndrome (FXS). Hypersensitivity to sensory stimuli contributes to attention difficulties by overwhelming and/or distracting affected individuals, which disrupts activities of daily living at home and learning at school. We find that auditory or visual distractors selectively impair visual discrimination performance in both humans and mice with FXS, but not their typically developing controls. Vasoactive intestinal polypeptide (VIP) neurons were significantly modulated by incorrect responses in the post-stimulus period during early distractor trials in WT mice, consistent with their known role as 'error' signals. Strikingly, however, VIP cells from Fmr1-/- mice showed little modulation in error trials, and this correlated with their poor performance on the distractor task. Thus, VIP interneurons and their reduced modulatory influence on pyramidal cells, could be a potential therapeutic target for attentional difficulties in FXS.

Multisensory integration of orally-sourced gustatory and olfactory inputs to the posterior piriform cortex in awake rats

Flavour refers to the sensory experience of food, which is a combination of sensory inputs sourced from multiple modalities during consumption, including taste and odour. Previous work has demonstrated that orally-sourced taste and odour cues interact to determine perceptual judgements of flavour stimuli, although the underlying cellular- and circuit-level neural mechanisms remain unknown. We recently identified a region of the piriform olfactory cortex in rats that responds to both taste and odour stimuli. Here, we investigated how converging taste and odour inputs to this area interact to affect single neuron responsiveness ensemble coding of flavour identity. To accomplish this, we recorded spiking activity from ensembles of single neurons in the posterior piriform cortex (pPC) in awake, tasting rats while delivering taste solutions, odour solutions and taste + odour mixtures directly into the oral cavity. Our results show that taste and odour inputs evoke highly selective, temporally-overlapping responses in multisensory pPC neurons. Comparing responses to mixtures and their unisensory components revealed that taste and odour inputs interact in a non-linear manner to produce unique response patterns. Taste input enhances trial-by-trial decoding of odour identity from small ensembles of simultaneously recorded neurons. Together, these results demonstrate that taste and odour inputs to pPC interact in complex, non-linear ways to form amodal flavour representations that enhance identity coding. KEY POINTS: Experience of food involves taste and smell, although how information from these different senses is combined by the brain to create our sense of flavour remains unknown. We recorded from small groups of neurons in the olfactory cortex of awake rats while they consumed taste solutions, odour solutions and taste + odour mixtures. Taste and smell solutions evoke highly selective responses. When presented in a mixture, taste and smell inputs interacted to alter responses, resulting in activation of unique sets of neurons that could not be predicted by the component responses. Synergistic interactions increase discriminability of odour representations. The olfactory cortex uses taste and smell to create new information representing multisensory flavour identity.

Probing top-down information in neocortical layer 1

Accurate perception of the environment is a constructive process that requires integration of external bottom-up sensory signals with internally generated top-down information. Decades of work have elucidated how sensory neocortex processes physical stimulus features. By contrast, examining how top-down information is encoded and integrated with bottom-up signals has been challenging using traditional neuroscience methods. Recent technological advances in functional imaging of brain-wide afferents in behaving mice have enabled the direct measurement of top-down information. Here, we review the emerging literature on encoding of these internally generated signals by different projection systems enriched in neocortical layer 1 during defined brain functions, including memory, attention, and predictive coding. Moreover, we identify gaps in current knowledge and highlight future directions for this rapidly advancing field.

Hierarchical unimodal processing within the primary somatosensory cortex during a bimodal detection task

Do sensory cortices process more than one sensory modality? To answer these questions, scientists have generated a wide variety of studies at distinct space-time scales in different animal models, and often shown contradictory conclusions. Some conclude that this process occurs in early sensory cortices, but others that this occurs in areas central to sensory cortices. Here, we sought to determine whether sensory neurons process and encode physical stimulus properties of different modalities (tactile and acoustic). For this, we designed a bimodal detection task where the senses of touch and hearing compete from trial to trial. Two Rhesus monkeys performed this novel task, while neural activity was recorded in areas 3b and 1 of the primary somatosensory cortex (S1). We analyzed neurons' coding properties and variability, organizing them by their receptive field's position relative to the stimulation zone. Our results indicate that neurons of areas 3b and 1 are unimodal, encoding only the tactile modality in both the firing rate and variability. Moreover, we found that neurons in area 3b carried more information about the periodic stimulus structure than those in area 1, possessed lower response and coding latencies, and had a lower intrinsic time scale. In sum, these differences reveal a hidden processing-based hierarchy. Finally, using a powerful nonlinear dimensionality reduction algorithm, we show that the activity from areas 3b and 1 can be separated, establishing a clear division in the functionality of these two subareas of S1.

Gamma oscillations and application of 40-Hz audiovisual stimulation to improve brain function

BACKGROUND

Audiovisual stimulation, such as auditory stimulation, light stimulation, and audiovisual combined stimulation, as a non-invasive stimulation, which can induce gamma oscillation, has received increased attention in recent years, and it has been preliminarily applied in the clinical rehabilitation of brain dysfunctions, such as cognitive, language, motor, mood, and sleep dysfunctions. However, the exact mechanism underlying the therapeutic effect of 40-Hz audiovisual stimulation remains unclear; the clinical applications of 40-Hz audiovisual stimulation in brain dysfunctions rehabilitation still need further research.

OBJECTIVE

In order to provide new insights into brain dysfunction rehabilitation, this review begins with a discussion of the mechanism underlying 40-Hz audiovisual stimulation, followed by a brief evaluation of its clinical application in the rehabilitation of brain dysfunctions.

RESULTS

Currently, 40-Hz audiovisual stimulation was demonstrated to affect synaptic plasticity and modify the connection status of related brain networks in animal experiments and clinical trials. Although its promising efficacy has been shown in the treatment of cognitive, mood, and sleep impairment, research studies into its application in language and motor dysfunctions are still ongoing.

CONCLUSIONS

Although 40-Hz audiovisual stimulation seems to be effective in treating cognitive, mood, and sleep disorders, its role in language and motor dysfunctions has yet to be determined.

A stable sensory map emerges from a dynamic equilibrium of neurons with unstable tuning properties

Recent long-term measurements of neuronal activity have revealed that, despite stability in large-scale topographic maps, the tuning properties of individual cortical neurons can undergo substantial reformatting over days. To shed light on this apparent contradiction, we captured the sound response dynamics of auditory cortical neurons using repeated 2-photon calcium imaging in awake mice. We measured sound-evoked responses to a set of pure tone and complex sound stimuli in more than 20,000 auditory cortex neurons over several days. We found that a substantial fraction of neurons dropped in and out of the population response. We modeled these dynamics as a simple discrete-time Markov chain, capturing the continuous changes in responsiveness observed during stable behavioral and environmental conditions. Although only a minority of neurons were driven by the sound stimuli at a given time point, the model predicts that most cells would at least transiently become responsive within 100 days. We observe that, despite single-neuron volatility, the population-level representation of sound frequency was stably maintained, demonstrating the dynamic equilibrium underlying the tonotopic map. Our results show that sensory maps are maintained by shifting subpopulations of neurons “sharing” the job of creating a sensory representation.

Synaptic Remodeling of the Auditory Cortex Following Bilateral Blindness: Evidence of Cross-modal Plasticity

We aimed to evaluate structural dynamic changes of neurons in the auditory cortex after visual deprivation. We longitudinally tracked dendritic spines for 3 weeks after visual deprivation in vivo using a two-photon microscope. GFP-labeled dendritic spines in the auditory cortex were serially followed after bilateral enucleation. The turnover rate, density, and size of the spines in the dendrites were evaluated 1, 2, and 3 weeks after visual deprivation. The turnover rate of the dendritic spines in the auditory cortex increased at 1 week (20.1±7.3%) after bilateral enucleation compared to baseline (12.5±7.9%); the increase persisted for up to 3 weeks (20.9±11.0%). The spine loss rate was slightly higher than the spine gain rate. The average spine density (number of spines per 1 μm of dendrite) was significantly lower at 2 weeks (2W; 0.22±0.06 1/μm) and 3 W (0.22±0.08 1/μm) post-nucleation compared to baseline (0.026±0.09 1/μm). We evaluated the change of synaptic strength in the stable spines at each time point. The normalized spine size in the auditory cortex was significantly increased after bilateral blindness at 1 W postoperatively (1.36±0.92), 2 W postoperatively (1.40±1.18), and 3 W postoperatively (1.36±0.88) compared to baseline. Sensory deprivation resulted in remodeling of the neural circuitry in the spared cortex, via cross-modal plasticity in the direction of partial breakdown of synapses, and enhanced strength of the remaining synapses.

Integration of somatosensory and motor-related information in the auditory system

An ability to integrate information provided by different sensory modalities is a fundamental feature of neurons in many brain areas. Because visual and auditory inputs often originate from the same external object, which may be located some distance away from the observer, the synthesis of these cues can improve localization accuracy and speed up behavioral responses. By contrast, multisensory interactions occurring close to the body typically involve a combination of tactile stimuli with other sensory modalities. Moreover, most activities involving active touch generate sound, indicating that stimuli in these modalities are frequently experienced together. In this review, we examine the basis for determining sound-source distance and the contribution of auditory inputs to the neural encoding of space around the body. We then consider the perceptual consequences of combining auditory and tactile inputs in humans and discuss recent evidence from animal studies demonstrating how cortical and subcortical areas work together to mediate communication between these senses. This research has shown that somatosensory inputs interface with and modulate sound processing at multiple levels of the auditory pathway, from the cochlear nucleus in the brainstem to the cortex. Circuits involving inputs from the primary somatosensory cortex to the auditory midbrain have been identified that mediate suppressive effects of whisker stimulation on auditory thalamocortical processing, providing a possible basis for prioritizing the processing of tactile cues from nearby objects. Close links also exist between audition and movement, and auditory responses are typically suppressed by locomotion and other actions. These movement-related signals are thought to cancel out self-generated sounds, but they may also affect auditory responses via the associated somatosensory stimulation or as a result of changes in brain state. Together, these studies highlight the importance of considering both multisensory context and movement-related activity in order to understand how the auditory cortex operates during natural behaviors, paving the way for future work to investigate auditory-somatosensory interactions in more ecological situations.

Excitatory Crossmodal Input to a Widespread Population of Primary Sensory Cortical Neurons

Crossmodal information processing in sensory cortices has been reported in sparsely distributed neurons under normal conditions and can undergo experience- or activity-induced plasticity. Given the potential role in brain function as indicated by previous reports, crossmodal connectivity in the sensory cortex needs to be further explored. Using perforated whole-cell recording in anesthetized adult rats, we found that almost all neurons recorded in the primary somatosensory, auditory, and visual cortices exhibited significant membrane-potential responses to crossmodal stimulation, as recorded when brain activity states were pharmacologically down-regulated in light anesthesia. These crossmodal cortical responses were excitatory and subthreshold, and further seemed to be relayed primarily by the sensory thalamus, but not the sensory cortex, of the stimulated modality. Our experiments indicate a sensory cortical presence of widespread excitatory crossmodal inputs, which might play roles in brain functions involving crossmodal information processing or plasticity.

Cross-Modal Plasticity in Brains Deprived of Visual Input Before Vision

Unimodal sensory loss leads to structural and functional changes in both deprived and nondeprived brain circuits. This process is broadly known as cross-modal plasticity. The evidence available indicates that cross-modal changes underlie the enhanced performances of the spared sensory modalities in deprived subjects. Sensory experience is a fundamental driver of cross-modal plasticity, yet there is evidence from early-visually deprived models supporting an additional role for experience-independent factors. These experience-independent factors are expected to act early in development and constrain neuronal plasticity at later stages. Here we review the cross-modal adaptations elicited by congenital or induced visual deprivation prior to vision. In most of these studies, cross-modal adaptations have been addressed at the structural and functional levels. Here, we also appraise recent data regarding behavioral performance in early-visually deprived models. However, further research is needed to explore how circuit reorganization affects their function and what brings about enhanced behavioral performance.

Olfactory modulation of barrel cortex activity during active whisking and passive whisker stimulation

Rodents depend on olfaction and touch to meet many of their fundamental needs. However, the impact of simultaneous olfactory and tactile inputs on sensory representations in the cortex remains elusive. To study these interactions, we recorded large populations of barrel cortex neurons using 2-photon calcium imaging in head-fixed mice during olfactory and tactile stimulation. Here we show that odors bidirectionally alter activity in a small but significant population of barrel cortex neurons through at least two mechanisms, first by enhancing whisking, and second by a central mechanism that persists after whisking is abolished by facial nerve sectioning. Odor responses have little impact on tactile information, and they are sufficient for decoding odor identity, while behavioral parameters like whisking, sniffing, and facial movements are not odor identity-specific. Thus, barrel cortex activity encodes specific olfactory information that is not linked with odor-induced changes in behavior.

Rodents use both touch and smell to get around. This work describes how olfactory information is combined with touch perception in the cortex to guide behavior.

Existing function in primary visual cortex is not perturbed by new skill acquisition of a non-matched sensory task

Acquisition of new skills has the potential to disturb existing network function. To directly assess whether previously acquired cortical function is altered during learning, mice were trained in an abstract task in which selected activity patterns were rewarded using an optical brain-computer interface device coupled to primary visual cortex (V1) neurons. Excitatory neurons were longitudinally recorded using 2-photon calcium imaging. Despite significant changes in local neural activity during task performance, tuning properties and stimulus encoding assessed outside of the trained context were not perturbed. Similarly, stimulus tuning was stable in neurons that remained responsive following a different, visual discrimination training task. However, visual discrimination training increased the rate of representational drift. Our results indicate that while some forms of perceptual learning may modify the contribution of individual neurons to stimulus encoding, new skill learning is not inherently disruptive to the quality of stimulus representation in adult V1.

Response of the somatosensory cortex following thermal stimuli to dental implants

In the current study, we investigated the cortical response of the somatosensory cortex following thermal stimuli to dental implants. Five implants were inserted at the site of the left upper canine with immediate implant placement protocols in five cats. Intrinsic signal optical imaging was applied to measure the cortical responses evoked by thermal sensing via dental implants. The cortical response evoked by 60 g of tactile stimulus to implants was also examined. The response strength and activated location were compared between implants and natural teeth. Thermal stimuli via the implant could evoke reliable cortical responses in the tooth-related region. However, the response amplitude evoked by the cold stimuli applied to the implants was significantly lower than that evoked by the cold stimuli applied to the natural teeth, indicating that the implants were less sensitive to thermal change than the natural tooth. The response evoked by tactile stimuli to implants was significantly stronger than that evoked by cold stimuli. Thermal and tactile stimuli activated the same location of the tooth-related somatosensory cortex in both the implants and natural teeth. Therefore, the thermal change in implants could be detected at the cortical response level. Multimodal sensory integration of thermal and tactile functions existed for implants.

Influence of Auditory Cues on the Neuronal Response to Naturalistic Visual Stimuli in a Virtual Reality Setting

Virtual reality environments offer great opportunities to study the performance of brain-computer interfaces (BCIs) in real-world contexts. As real-world stimuli are typically multimodal, their neuronal integration elicits complex response patterns. To investigate the effect of additional auditory cues on the processing of visual information, we used virtual reality to mimic safety-related events in an industrial environment while we concomitantly recorded electroencephalography (EEG) signals. We simulated a box traveling on a conveyor belt system where two types of stimuli – an exploding and a burning box – interrupt regular operation. The recordings from 16 subjects were divided into two subsets, a visual-only and an audio-visual experiment. In the visual-only experiment, the response patterns for both stimuli elicited a similar pattern – a visual evoked potential (VEP) followed by an event-related potential (ERP) over the occipital-parietal lobe. Moreover, we found the perceived severity of the event to be reflected in the signal amplitude. Interestingly, the additional auditory cues had a twofold effect on the previous findings: The P1 component was significantly suppressed in the case of the exploding box stimulus, whereas the N2c showed an enhancement for the burning box stimulus. This result highlights the impact of multisensory integration on the performance of realistic BCI applications. Indeed, we observed alterations in the offline classification accuracy for a detection task based on a mixed feature extraction (variance, power spectral density, and discrete wavelet transform) and a support vector machine classifier. In the case of the explosion, the accuracy slightly decreased by –1.64% p. in an audio-visual experiment compared to the visual-only. Contrarily, the classification accuracy for the burning box increased by 5.58% p. when additional auditory cues were present. Hence, we conclude, that especially in challenging detection tasks, it is favorable to consider the potential of multisensory integration when BCIs are supposed to operate under (multimodal) real-world conditions.

Estimating How Sounds Modulate Orientation Representation in the Primary Visual Cortex Using Shallow Neural Networks

Audiovisual perception results from the interaction between visual and auditory processing. Hence, presenting auditory and visual inputs simultaneously usually improves the accuracy of the unimodal percepts, but can also lead to audiovisual illusions. Cross-talks between visual and auditory inputs during sensory processing were recently shown to occur as early as in the primary visual cortex (V1). In a previous study, we demonstrated that sounds improve the representation of the orientation of visual stimuli in the naïve mouse V1 by promoting the recruitment of neurons better tuned to the orientation and direction of the visual stimulus. However, we did not test if this type of modulation was still present when the auditory and visual stimuli were both behaviorally relevant. To determine the effect of sounds on active visual processing, we performed calcium imaging in V1 while mice were performing an audiovisual task. We then compared the representations of the task stimuli orientations in the unimodal visual and audiovisual context using shallow neural networks (SNNs). SNNs were chosen because of the biological plausibility of their computational structure and the possibility of identifying post hoc the biological neurons having the strongest influence on the classification decision. We first showed that SNNs can categorize the activity of V1 neurons evoked by drifting gratings of 12 different orientations. Then, we demonstrated using the connection weight approach that SNN training assigns the largest computational weight to the V1 neurons having the best orientation and direction selectivity. Finally, we showed that it is possible to use SNNs to determine how V1 neurons represent the orientations of stimuli that do not belong to the set of orientations used for SNN training. Once the SNN approach was established, we replicated the previous finding that sounds improve orientation representation in the V1 of naïve mice. Then, we showed that, in mice performing an audiovisual detection task, task tones improve the representation of the visual cues associated with the reward while deteriorating the representation of non-rewarded cues. Altogether, our results suggest that the direction of sound modulation in V1 depends on the behavioral relevance of the visual cue.

Unique spatiotemporal fMRI dynamics in the awake mouse brain

Human imaging studies have shown that spontaneous brain activity exhibits stereotypic spatiotemporal reorganization in awake, conscious conditions with respect to minimally conscious states. However, whether and how this phenomenon can be generalized to lower mammalian species remains unclear. Leveraging a robust protocol for resting-state fMRI (rsfMRI) mapping in non-anesthetized, head-fixed mice, we investigated functional network topography and dynamic structure of spontaneous brain activity in wakeful animals. We found that rsfMRI networks in the awake state, while anatomically comparable to those observed under anesthesia, are topologically configured to maximize interregional communication, departing from the underlying community structure of the mouse axonal connectome. We further report that rsfMRI activity in wakeful animals exhibits unique spatiotemporal dynamics characterized by a state-dependent, dominant occurrence of coactivation patterns encompassing a prominent participation of arousal-related forebrain nuclei and functional anti-coordination between visual-auditory and polymodal cortical areas. We finally show that rsfMRI dynamics in awake mice exhibits a stereotypical temporal structure, in which state-dominant coactivation patterns are configured as network attractors. These findings suggest that spontaneous brain activity in awake mice is critically shaped by state-specific involvement of basal forebrain arousal systems and document that its dynamic structure recapitulates distinctive, evolutionarily relevant principles that are predictive of conscious states in higher mammalian species.

•

fMRI networks in awake mice depart from underlying anatomical structure

•

fMRI dynamics in wakeful mice is critically shaped by arousal-related nuclei

•

Occurrence and topography of rsfMRI coactivation patterns define conscious states

•

fMRI coactivation dynamics defines a signature of consciousness in the mouse brain

Context-dependent persistency as a coding mechanism for robust and widely distributed value coding

Task-related information is widely distributed across the brain with different coding properties, such as persistency. We found in mice that coding persistency of action history and value was variable across areas, learning phases, and task context, with the highest persistency in the retrosplenial cortex of expert mice performing value-based decisions where history needs to be maintained across trials. Persistent coding also emerged in artificial networks trained to perform mouse-like reinforcement learning. Persistency allows temporally untangled value representations in neuronal manifolds where population activity exhibits cyclic trajectories that transition along the value axis after action outcomes, collectively forming cylindrical dynamics. Simulations indicated that untangled persistency facilitates robust value retrieval by downstream networks. Even leakage of persistently maintained value through non-specific connectivity could contribute to the brain-wide distributed value coding with different levels of persistency. These results reveal that context-dependent, untangled persistency facilitates reliable signal coding and its distribution across the brain.

Multisensory Integration: Is Medial Prefrontal Cortex Signaling Relevant for the Treatment of Higher-Order Visual Dysfunctions?

In the mammalian brain, information processing in sensory modalities and global mechanisms of multisensory integration facilitate perception. Emerging experimental evidence suggests that the contribution of multisensory integration to sensory perception is far more complex than previously expected. Here we revise how associative areas such as the prefrontal cortex, which receive and integrate inputs from diverse sensory modalities, can affect information processing in unisensory systems via processes of down-stream signaling. We focus our attention on the influence of the medial prefrontal cortex on the processing of information in the visual system and whether this phenomenon can be clinically used to treat higher-order visual dysfunctions. We propose that non-invasive and multisensory stimulation strategies such as environmental enrichment and/or attention-related tasks could be of clinical relevance to fight cerebral visual impairment.

Homeostatic regulation through strengthening of neuronal network-correlated synaptic inputs

Homeostatic regulation is essential for stable neuronal function. Several synaptic mechanisms of homeostatic plasticity have been described, but the functional properties of synapses involved in homeostasis are unknown. We used longitudinal two-photon functional imaging of dendritic spine calcium signals in visual and retrosplenial cortices of awake adult mice to quantify the sensory deprivation-induced changes in the responses of functionally identified spines. We found that spines whose activity selectively correlated with intrinsic network activity underwent tumor necrosis factor alpha (TNF-α)-dependent homeostatic increases in their response amplitudes, but spines identified as responsive to sensory stimulation did not. We observed an increase in the global sensory-evoked responses following sensory deprivation, despite the fact that the identified sensory inputs did not strengthen. Instead, global sensory-evoked responses correlated with the strength of network-correlated inputs. Our results suggest that homeostatic regulation of global responses is mediated through changes to intrinsic network-correlated inputs rather than changes to identified sensory inputs thought to drive sensory processing.