Development of the Mechanisms Governing Midbrain Multisensory Integration

Specific rules for time and space of multisensory plasticity in the superior colliculus

Cat superior colliculus (SC) neurons commonly combine information from different senses, which facilitates event detection and localization. Integration in SC multisensory neurons depends on the spatial and temporal relationships between cross-modal cues. Here, we revealed the parallel process of short-term plasticity in the temporal/spatial integration process during adulthood that adapts multisensory integration to reliable changes in environmental conditions. Short-term experience alters the temporal preferences of SC multisensory neurons, and this short-term plasticity in the temporal/spatial integration process is limited to changes in cross-modal timing (a factor commonly induced by events at different distances from the receiver). However, this plasticity was not evident in response to changes in the cross-modal spatial configuration.

Cross-modal exposure restores multisensory enhancement after hemianopia

Hemianopia is a common consequence of unilateral damage to visual cortex that manifests as a profound blindness in contralesional space. A noninvasive cross-modal (visual-auditory) exposure paradigm has been developed in an animal model to ameliorate this disorder. Repeated stimulation of a visual-auditory stimulus restores overt responses to visual stimuli in the blinded hemifield. It is believed to accomplish this by enhancing the visual sensitivity of circuits remaining after a lesion of visual cortex; in particular, circuits involving the multisensory neurons of the superior colliculus. Neurons in this midbrain structure are known to integrate spatiotemporally congruent visual and auditory signals to amplify their responses, which, in turn, enhances behavioral performance. Here we evaluated the relationship between the rehabilitation of hemianopia and this process of multisensory integration. Induction of hemianopia also eliminated multisensory enhancement in the blinded hemifield. Both vision and multisensory enhancement rapidly recovered with the rehabilitative cross-modal exposures. However, although both reached pre-lesion levels at similar rates, they did so with different spatial patterns. The results suggest that the capability for multisensory integration and enhancement is not a pre-requisite for visual recovery in hemianopia, and that the underlying mechanisms for recovery may be more complex than currently appreciated.

The self and the Bayesian brain: Testing probabilistic models of body ownership through a self-localization task

Simple multisensory manipulations can induce the illusory misattribution of external objects to one's own body, allowing to experimentally investigate body ownership. In this context, body ownership has been conceptualized as the result of the online Bayesian optimal estimation of the probability that one object belongs to the body from the congruence of multisensory inputs. This idea has been highly influential, as it provided a quantitative basis to bottom-up accounts of self-consciousness. However, empirical evidence fully supporting this view is scarce, as the optimality of the putative inference process has not been assessed rigorously. This pre-registered study aimed at filling this gap by testing a Bayesian model of hand ownership based on spatial and temporal visuo-proprioceptive congruences. Model predictions were compared to data from a virtual-reality reaching task, whereby reaching errors induced by a spatio-temporally mismatching virtual hand have been used as an implicit proxy of hand ownership. To rigorously test optimality, we compared the Bayesian model versus alternative non-Bayesian models of multisensory integration, and independently assess unisensory components and compare them to model estimates. We found that individually measured values of proprioceptive precision correlated with those fitted from our reaching task, providing compelling evidence that the underlying visuo-proprioceptive integration process approximates Bayesian optimality. Furthermore, reaching errors correlated with explicit ownership ratings at the single individual and trial level. Taken together, these results provide novel evidence that body ownership, a key component of self-consciousness, can be truly described as the bottom-up, behaviourally optimal processing of multisensory inputs.

Noise-rearing precludes the behavioral benefits of multisensory integration

Concordant visual-auditory stimuli enhance the responses of individual superior colliculus (SC) neurons. This neuronal capacity for "multisensory integration" is not innate: it is acquired only after substantial cross-modal (e.g. auditory-visual) experience. Masking transient auditory cues by raising animals in omnidirectional sound ("noise-rearing") precludes their ability to obtain this experience and the ability of the SC to construct a normal multisensory (auditory-visual) transform. SC responses to combinations of concordant visual-auditory stimuli are depressed, rather than enhanced. The present experiments examined the behavioral consequence of this rearing condition in a simple detection/localization task. In the first experiment, the auditory component of the concordant cross-modal pair was novel, and only the visual stimulus was a target. In the second experiment, both component stimuli were targets. Noise-reared animals failed to show multisensory performance benefits in either experiment. These results reveal a close parallel between behavior and single neuron physiology in the multisensory deficits that are induced when noise disrupts early visual-auditory experience.

Association between different sensory modalities based on concurrent time series data obtained by a collaborative reservoir computing model

Humans perceive the external world by integrating information from different modalities, obtained through the sensory organs. However, the aforementioned mechanism is still unclear and has been a subject of widespread interest in the fields of psychology and brain science. A model using two reservoir computing systems, i.e., a type of recurrent neural network trained to mimic each other's output, can detect stimulus patterns that repeatedly appear in a time series signal. We applied this model for identifying specific patterns that co-occur between information from different modalities. The model was self-organized by specific fluctuation patterns that co-occurred between different modalities, and could detect each fluctuation pattern. Additionally, similarly to the case where perception is influenced by synchronous/asynchronous presentation of multimodal stimuli, the model failed to work correctly for signals that did not co-occur with corresponding fluctuation patterns. Recent experimental studies have suggested that direct interaction between different sensory systems is important for multisensory integration, in addition to top-down control from higher brain regions such as the association cortex. Because several patterns of interaction between sensory modules can be incorporated into the employed model, we were able to compare the performance between them; the original version of the employed model incorporated such an interaction as the teaching signals for learning. The performance of the original and alternative models was evaluated, and the original model was found to perform the best. Thus, we demonstrated that feedback of the outputs of appropriately learned sensory modules performed the best when compared to the other examined patterns of interaction. The proposed model incorporated information encoded by the dynamic state of the neural population and the interactions between different sensory modules, both of which were based on recent experimental observations; this allowed us to study the influence of the temporal relationship and frequency of occurrence of multisensory signals on sensory integration, as well as the nature of interaction between different sensory signals.

The brain-body disconnect: A somatic sensory basis for trauma-related disorders

Although the manifestation of trauma in the body is a phenomenon well-endorsed by clinicians and traumatized individuals, the neurobiological underpinnings of this manifestation remain unclear. The notion of somatic sensory processing, which encompasses vestibular and somatosensory processing and relates to the sensory systems concerned with how the physical body exists in and relates to physical space, is introduced as a major contributor to overall regulatory, social-emotional, and self-referential functioning. From a phylogenetically and ontogenetically informed perspective, trauma-related symptomology is conceptualized to be grounded in brainstem-level somatic sensory processing dysfunction and its cascading influences on physiological arousal modulation, affect regulation, and higher-order capacities. Lastly, we introduce a novel hierarchical model bridging somatic sensory processes with limbic and neocortical mechanisms regulating an individual's emotional experience and sense of a relational, agentive self. This model provides a working framework for the neurobiologically informed assessment and treatment of trauma-related conditions from a somatic sensory processing perspective.

Learning multisensory cue integration: A computational model of crossmodal synaptic plasticity enables reliability-based cue weighting by capturing stimulus statistics

The brain forms unified, coherent, and accurate percepts of events occurring in the environment by integrating information from multiple senses through the process of multisensory integration. The neural mechanisms underlying this process, its development and its maturation in a multisensory environment are yet to be properly understood. Numerous psychophysical studies suggest that the multisensory cue integration process follows the principle of Bayesian estimation, where the contributions of individual sensory modalities are proportional to the relative reliabilities of the different sensory stimuli. In this article I hypothesize that experience dependent crossmodal synaptic plasticity may be a plausible mechanism underlying development of multisensory cue integration. I test this hypothesis via a computational model that implements Bayesian multisensory cue integration using reliability-based cue weighting. The model uses crossmodal synaptic plasticity to capture stimulus statistics within synaptic weights that are adapted to reflect the relative reliabilities of the participating stimuli. The model is embodied in a simulated robotic agent that learns to localize an audio-visual target by integrating spatial location cues extracted from of auditory and visual sensory modalities. Results of multiple randomized target localization trials in simulation indicate that the model is able to learn modality-specific synaptic weights proportional to the relative reliabilities of the auditory and visual stimuli. The proposed model with learned synaptic weights is also compared with a maximum-likelihood estimation model for cue integration via regression analysis. Results indicate that the proposed model reflects maximum-likelihood estimation.

Resolution of impaired multisensory processing in autism and the cost of switching sensory modality

Children with autism spectrum disorders (ASD) exhibit alterations in multisensory processing, which may contribute to the prevalence of social and communicative deficits in this population. Resolution of multisensory deficits has been observed in teenagers with ASD for complex, social speech stimuli; however, whether this resolution extends to more basic multisensory processing deficits remains unclear. Here, in a cohort of 364 participants we show using simple, non-social audiovisual stimuli that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Computational modelling indicated that multisensory processing transitions from a default state of competition to one of facilitation, and that this transition is delayed in ASD. Further analysis revealed group differences in how sensory channels are weighted, and how this is impacted by preceding cross-sensory inputs. Our findings indicate that there is a complex and dynamic interplay among the sensory systems that differs considerably in individuals with ASD.

Crosse et al. study a cohort of 364 participants with autism spectrum disorders (ASD) and matched controls, and show that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Using computational modelling they go on to demonstrate that there is a delayed transition of multisensory processing from a default state of competition to one of facilitation in ASD, as well as differences in sensory weighting and the ability to switch between sensory modalities, which sheds light on the interplay among sensory systems that differ in ASD individuals.

Wired to Connect: The Autonomic Socioemotional Reflex Arc

We have previously proposed that mothers and infants co-regulate one another’s autonomic state through an autonomic conditioning mechanism, which starts during gestation and results in the formation of autonomic socioemotional reflexes (ASRs) following birth. Theoretically, autonomic physiology associated with the ASR should correlate concomitantly with behaviors of mother and infant, although the neuronal pathway by which this phenomenon occurs has not been elucidated. In this paper, we consider the neuronal pathway by which sensory stimuli between a mother and her baby/child affect the physiology and emotional behavior of each. We divide our paper into two parts. In the first part, to gain perspective on current theories on the subject, we conduct a 500-year narrative history of scientific investigations into the human nervous system and theories that describe the neuronal pathway between sensory stimulus and emotional behavior. We then review inconsistencies between several currently accepted theories and recent data. In the second part, we lay out a new theory of emotions that describes how sensory stimuli between mother and baby unconsciously control the behavior and physiology of both. We present a theory of mother/infant emotion based on a set of assumptions fundamentally different from current theories. Briefly, we propose that mother/infant sensory stimuli trigger conditional autonomic socioemotional reflexes (ASRs), which drive cardiac function and behavior without the benefit of the thalamus, amygdala or cortex. We hold that the ASR is shaped by an evolutionarily conserved autonomic learning mechanism (i.e., functional Pavlovian conditioning) that forms between mother and fetus during gestation and continues following birth. We highlight our own and others research findings over the past 15 years that support our contention that mother/infant socioemotional behavior is driven by mutual autonomic state plasticity, as opposed to cortical trait plasticity. We review a novel assessment tool designed to measure the behaviors associated with the ASR phenomenon. Finally, we discuss the significance of our theory for the treatment of mothers and infants with socioemotional disorders.

Orienting our view of the superior colliculus: specializations and general functions

The mammalian superior colliculus (SC) and its non-mammalian homolog, the optic tectum are implicated in sensorimotor transformations. Historically, emphasis on visuomotor functions of the SC has led to a popular view that it operates as an oculomotor structure rather than a more general orienting structure. In this review, we consider comparative work on the SC/optic tectum, with a particular focus on non-visual sensing and orienting, which reveals a broader perspective on SC functions and their role in species-specific behaviors. We highlight several recent studies that consider ethological context and natural behaviors to advance knowledge of the SC as a site of multi-sensory integration and motor initiation in diverse species.

Neuroplasticity and Crossmodal Connectivity in the Normal, Healthy Brain

Objective

Neuroplasticity enables the brain to establish new crossmodal connections or reorganize old connections which are essential to perceiving a multisensorial world. The intent of this review is to identify and summarize the current developments in neuroplasticity and crossmodal connectivity, and deepen understanding of how crossmodal connectivity develops in the normal, healthy brain, highlighting novel perspectives about the principles that guide this connectivity.

Methods

To the above end, a narrative review is carried out. The data documented in prior relevant studies in neuroscience, psychology and other related fields available in a wide range of prominent electronic databases are critically assessed, synthesized, interpreted with qualitative rather than quantitative elements, and linked together to form new propositions and hypotheses about neuroplasticity and crossmodal connectivity.

Results

Three major themes are identified. First, it appears that neuroplasticity operates by following eight fundamental principles and crossmodal integration operates by following three principles. Second, two different forms of crossmodal connectivity, namely direct crossmodal connectivity and indirect crossmodal connectivity, are suggested to operate in both unisensory and multisensory perception. Third, three principles possibly guide the development of crossmodal connectivity into adulthood. These are labeled as the principle of innate crossmodality, the principle of evolution-driven 'neuromodular' reorganization and the principle of multimodal experience. These principles are combined to develop a three-factor interaction model of crossmodal connectivity.

Conclusions

The hypothesized principles and the proposed model together advance understanding of neuroplasticity, the nature of crossmodal connectivity, and how such connectivity develops in the normal, healthy brain.

Using the Principles of Multisensory Integration to Reverse Hemianopia

Hemianopia can be rehabilitated by an auditory-visual "training" procedure, which restores visual responsiveness in midbrain neurons indirectly compromised by the cortical lesion and reinstates vision in contralesional space. Presumably, these rehabilitative changes are induced via mechanisms of multisensory integration/plasticity. If so, the paradigm should fail if the stimulus configurations violate the spatiotemporal principles that govern these midbrain processes. To test this possibility, hemianopic cats were provided spatially or temporally noncongruent auditory-visual training. Rehabilitation failed in all cases even after approximately twice the number of training trials normally required for recovery, and even after animals learned to approach the location of the undetected visual stimulus. When training was repeated with these stimuli in spatiotemporal concordance, hemianopia was resolved. The results identify the conditions needed to engage changes in remaining neural circuits required to support vision in the absence of visual cortex, and have implications for rehabilitative strategies in human patients.

Experience Creates the Multisensory Transform in the Superior Colliculus

Although the ability to integrate information across the senses is compromised in some individuals for unknown reasons, similar defects have been observed when animals are reared without multisensory experience. The experience-dependent development of multisensory integration has been studied most extensively using the visual-auditory neuron of the cat superior colliculus (SC) as a neural model. In the normally-developed adult, SC neurons react to concordant visual-auditory stimuli by integrating their inputs in real-time to produce non-linearly amplified multisensory responses. However, when prevented from gathering visual-auditory experience, their multisensory responses are no more robust than their responses to the individual component stimuli. The mechanisms operating in this defective state are poorly understood. Here we examined the responses of SC neurons in “naïve” (i.e., dark-reared) and “neurotypic” (i.e., normally-reared) animals on a millisecond-by-millisecond basis to determine whether multisensory experience changes the operation by which unisensory signals are converted into multisensory outputs (the “multisensory transform”), or whether it changes the dynamics of the unisensory inputs to that transform (e.g., their synchronization and/or alignment). The results reveal that the major impact of experience was on the multisensory transform itself. Whereas neurotypic multisensory responses exhibited non-linear amplification near their onset followed by linear amplification thereafter, the naive responses showed no integration in the initial phase of the response and a computation consistent with competition in its later phases. The results suggest that multisensory experience creates an entirely new computation by which convergent unisensory inputs are used cooperatively to enhance the physiological salience of cross-modal events and thereby facilitate normal perception and behavior.

An innate brainstem self-other system involving orienting, affective responding, and polyvalent relational seeking: Some clinical implications for a "Deep Brain Reorienting" trauma psychotherapy approach

Underlying any complex relational intersubjectivity there is an inherent urge to connect, to have proximity, to engage in an experience of interpersonal contact. The hypothesis set out here is that this most basic urge to connect is dependent on circuits based in three main components: the midbrain superior colliculi (SC), the midbrain periaqueductal gray (PAG), and the mesolimbic and mesocortical dopamine systems originating in the midbrain ventral tegmental area. Firstly, there is orienting towards or away from interpersonal contact, dependent on approach and/or defensive/withdrawal areas of the SC. Secondly, there is an affective response to the contact, mediated by the PAG. Thirdly, there is an associated, affectively-loaded, seeking drive based in the mesolimbic and mesocortical dopamine systems. The neurochemical milieu of these dopaminergic systems is responsive to environmental factors, creating the possibility of multiple states of functioning with different affective valences, a polyvalent range of subjectively positive and negative experiences. The recognition of subtle tension changes in skeletal muscles when orienting to an affectively significant experience or event has clinical implications for processing of traumatic memories, including those of a relational/interpersonal nature. Sequences established at the brainstem level can underlie patterns of attachment responding that repeat over many years in different contexts. The interaction of the innate system for connection with that for alarm, through circuits based in the locus coeruleus, and that for defence, based in circuits through the PAG, can lay down deep patterns of emotional and energetic responses to relational stimuli. There may be simultaneous sequences for attachment approach and defensive aggression underlying relational styles that are so deep as to be seen as personality characteristics, for example, of borderline type. A clinical approach derived from these hypotheses, Deep Brain Reorienting, is briefly outlined as it provides a way to address the somatic residues of adverse interpersonal interactions underlying relational patterns and also the residual shock and horror of traumatic experiences. We suggest that the innate alarm system involving the SC and the locus coeruleus can generate a pre-affective shock while an affective shock can arise from excessive stimulation of the PAG. Clinically significant residues can be accessed through careful, mindful, attention to orienting-tension-affect-seeking sequences when the therapist and the client collaborate on eliciting and describing them.

Reorganization of the somatosensory pathway after subacute incomplete cervical cord injury

Objective

The main purpose of the present study was to investigate the possible somatosensory-related brain functional reorganization after traumatic spinal cord injury (SCI).

Methods

Thirteen patients with subacute incomplete cervical cord injury (ICCI) and thirteen age- and sex-matched healthy controls (HCs) were recruited. Eleven patients and all the HCs underwent both sensory task-related brain functional scanning and whole brain structural scanning on a 3.0 Tesla MRI system, and two patients underwent only structural scanning; the process of structural scanning was completed on thirteen patients, while functional scanning was only applied to eleven patients. We performed sensory task-related functional MRI (fMRI) to investigate the functional changes in the brain. In addition, voxel-based morphometry (VBM) was applied to explore whether any sensory-related brain structural changes occur in the whole brain after SCI.

Results

Compared with HCs, ICCI patients exhibited decreased activation in the left postcentral gyrus (postCG), the brainstem (midbrain and right pons) and the right cerebellar lobules IV-VI. Moreover, a significant positive association was found between the activation in the left PostCG and the activation in both the brainstem and the right cerebellar lobules IV-VI. Additionally, the decrease in gray matter volume (GMV) was detected in the left superior parietal lobule (SPL). The decrease of white matter volume (WMV) was observed in the right temporal lobe, the right occipital lobe, and the right calcarine gyrus. No structural change in the primary sensory cortex (S1), the secondary somatosensory cortex (S2) or the thalamus was detected.

Conclusion

These functional and structural findings may demonstrate the existence of an alternative pathway in the impairment of somatosensory function after SCI, which consists of the ipsilateral cerebellum, the brainstem and the contralateral postCG. It provides a new theoretical basis for the mechanism of sensory-related brain alteration in SCI patients and the rehabilitation therapy based on this pathway in the future.

•

We found that sensory-related brain reorganization may not occur in the thalamus in patients with ICCI.

•

We found that brain structural reorganization did not occur in the S1 or the S2 in patients with ICCI.

•

We observed that SCI can cause brain structural reorganization in non-sensory-related areas.

•

We observed that an alternative pathway may exist in the impairment of somatosensory function after SCI.

Cross-Modal Competition: The Default Computation for Multisensory Processing

Mature multisensory superior colliculus (SC) neurons integrate information across the senses to enhance their responses to spatiotemporally congruent cross-modal stimuli. The development of this neurotypic feature of SC neurons requires experience with cross-modal cues. In the absence of such experience the response of an SC neuron to congruent cross-modal cues is no more robust than its response to the most effective component cue. This "default" or "naive" state is believed to be one in which cross-modal signals do not interact. The present results challenge this characterization by identifying interactions between visual-auditory signals in male and female cats reared without visual-auditory experience. By manipulating the relative effectiveness of the visual and auditory cross-modal cues that were presented to each of these naive neurons, an active competition between cross-modal signals was revealed. Although contrary to current expectations, this result is explained by a neuro-computational model in which the default interaction is mutual inhibition. These findings suggest that multisensory neurons at all maturational stages are capable of some form of multisensory integration, and use experience with cross-modal stimuli to transition from their initial state of competition to their mature state of cooperation. By doing so, they develop the ability to enhance the physiological salience of cross-modal events thereby increasing their impact on the sensorimotor circuitry of the SC, and the likelihood that biologically significant events will elicit SC-mediated overt behaviors.SIGNIFICANCE STATEMENT The present results demonstrate that the default mode of multisensory processing in the superior colliculus is competition, not non-integration as previously characterized. A neuro-computational model explains how these competitive dynamics can be implemented via mutual inhibition, and how this default mode is superseded by the emergence of cooperative interactions during development.

Integration of visual and whisker signals in rat superior colliculus

Multisensory integration is a process by which signals from different sensory modalities are combined to facilitate detection and localization of external events. One substrate for multisensory integration is the midbrain superior colliculus (SC) which plays an important role in orienting behavior. In rodent SC, visual and somatosensory (whisker) representations are in approximate registration, but whether and how these signals interact is unclear. We measured spiking activity in SC of anesthetized hooded rats, during presentation of visual- and whisker stimuli that were tested simultaneously or in isolation. Visual responses were found in all layers, but were primarily located in superficial layers. Whisker responsive sites were primarily found in intermediate layers. In single- and multi-unit recording sites, spiking activity was usually only sensitive to one modality, when stimuli were presented in isolation. By contrast, we observed robust and primarily suppressive interactions when stimuli were presented simultaneously to both modalities. We conclude that while visual and whisker representations in SC of rat are partially overlapping, there is limited excitatory convergence onto individual sites. Multimodal integration may instead rely on suppressive interactions between modalities.

Evidence for Brainstem Contributions to Autism Spectrum Disorders

Autism spectrum disorder (ASD) is a neurodevelopmental condition that affects one in 59 children in the United States. Although there is a mounting body of knowledge of cortical and cerebellar contributions to ASD, our knowledge about the early developing brainstem in ASD is only beginning to accumulate. Understanding how brainstem neurotransmission is implicated in ASD is important because many of this condition’s sensory and motor symptoms are consistent with brainstem pathology. Therefore, the purpose of this review was to integrate epidemiological, behavioral, histological, neuroimaging, and animal evidence of brainstem contributions to ASD. Because ASD is a neurodevelopmental condition, we examined the available data through a lens of hierarchical brain development. The review of the literature suggests that developmental alterations of the brainstem could have potential cascading effects on cortical and cerebellar formation, ultimately leading to ASD symptoms. This view is supported by human epidemiology findings and data from animal models of ASD, showing that perturbed development of the brainstem substructures, particularly during the peak formation of the brainstem’s monoaminergic centers, may relate to ASD or ASD-like behaviors. Furthermore, we review evidence from human histology, psychophysiology, and neuroimaging suggesting that brainstem development and maturation may be atypical in ASD and may be related to key ASD symptoms, such as atypical sensorimotor features and social responsiveness. From this review there emerges the need of future research to validate early detection of the brainstem-based somatosensory and psychophysiological behaviors that emerge in infancy, and to examine the brainstem across the life span, while accounting for age. In all, there is preliminary evidence for brainstem involvement in ASD, but a better understanding of the brainstem’s role would likely pave the way for earlier diagnosis and treatment of ASD.