Organization, maturation, and plasticity of multisensory integration: insights from computational modeling studies

Changing the Tendency to Integrate the Senses

Integration of sensory signals that emanate from the same source, such as the visual of lip articulations and the sound of the voice of a speaking individual, can improve perception of the source signal (e.g., speech). Because momentary sensory inputs are typically corrupted with internal and external noise, there is almost always a discrepancy between the inputs, facing the perceptual system with the problem of determining whether the two signals were caused by the same source or different sources. Thus, whether or not multisensory stimuli are integrated and the degree to which they are bound is influenced by factors such as the prior expectation of a common source. We refer to this factor as the tendency to bind stimuli, or for short, binding tendency. In theory, the tendency to bind sensory stimuli can be learned by experience through the acquisition of the probabilities of the co-occurrence of the stimuli. It can also be influenced by cognitive knowledge of the environment. The binding tendency varies across individuals and can also vary within an individual over time. Here, we review the studies that have investigated the plasticity of binding tendency. We discuss the protocols that have been reported to produce changes in binding tendency, the candidate learning mechanisms involved in this process, the possible neural correlates of binding tendency, and outstanding questions pertaining to binding tendency and its plasticity. We conclude by proposing directions for future research and argue that understanding mechanisms and recipes for increasing binding tendency can have important clinical and translational applications for populations or individuals with a deficiency in multisensory integration.

A Computational Analysis of Neural Mechanisms Underlying the Maturation of Multisensory Speech Integration in Neurotypical Children and Those on the Autism Spectrum

Failure to appropriately develop multisensory integration (MSI) of audiovisual speech may affect a child's ability to attain optimal communication. Studies have shown protracted development of MSI into late-childhood and identified deficits in MSI in children with an autism spectrum disorder (ASD). Currently, the neural basis of acquisition of this ability is not well understood. Here, we developed a computational model informed by neurophysiology to analyze possible mechanisms underlying MSI maturation, and its delayed development in ASD. The model posits that strengthening of feedforward and cross-sensory connections, responsible for the alignment of auditory and visual speech sound representations in posterior superior temporal gyrus/sulcus, can explain behavioral data on the acquisition of MSI. This was simulated by a training phase during which the network was exposed to unisensory and multisensory stimuli, and projections were crafted by Hebbian rules of potentiation and depression. In its mature architecture, the network also reproduced the well-known multisensory McGurk speech effect. Deficits in audiovisual speech perception in ASD were well accounted for by fewer multisensory exposures, compatible with a lack of attention, but not by reduced synaptic connectivity or synaptic plasticity.

Unisensory and multisensory Self-referential stimulation of the lower limb: An exploratory fMRI study on healthy subjects

BACKGROUND

The holistic view of the person is the essence of the physiotherapy. Knowledge of approaches that develop the whole person promotes better patient outcomes. Multisensory Self-referential stimulation, more than a unisensory one, seems to produce a holistic experience of the Self ("Core-Self").

OBJECTIVES

(1) To analyze the somatotopic brain activation during unisensory and multisensorial Self-referential stimulus; and (2) to understand if the areas activated by multisensorial Self-referential stimulation are the ones responsible for the "Core-Self."

METHODS

An exploratory functional magnetic resonance imaging (fMRI) study was performed with 10 healthy subjects, under the stimulation of the lower limbs with three Self-referential stimuli: unisensory auditory-verbal, unisensory tactile-manual, and multisensory, applying the unisensory stimuli simultaneously.

RESULTS

Unisensory stimulation elicits bilateral activations of the temporoparietal junction (TPJ), of the primary somatosensory cortex (S1), of the primary motor cortex (BA4), of the premotor cortex (BA6) and of BA44; multisensory stimulation also elicits activity in TPJ, BA4, and BA6, and when compared with unisensory stimuli, activations were found in: (1) Cortical and subcortical midline structures-BA7 (precuneus), BA9 (medial prefrontal cortex), BA30 (posterior cingulated), superior colliculum and posterior cerebellum; and (2) Posterior lateral cortex-TPJ, posterior BA13 (insula), BA19, and BA37. Bilateral TPJ is the one that showed the biggest activation volume.

CONCLUSION

This specific multisensory stimulation produces a brain activation map in regions that are responsible for multisensory Self-processing and may represent the Core-Self. We recommend the use of this specific multisensory stimulation as a physiotherapy intervention strategy that might promote the Self-reorganization.

Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception

Which is my body and how do I distinguish it from the bodies of others, or from objects in the surrounding environment? The perception of our own body and more particularly our sense of body ownership is taken for granted. Nevertheless, experimental findings from body ownership illusions (BOIs), show that under specific multisensory conditions, we can experience artificial body parts or fake bodies as our own body parts or body, respectively. The aim of the present paper is to discuss how and why BOIs are induced. We review several experimental findings concerning the spatial, temporal, and semantic principles of crossmodal stimuli that have been applied to induce BOIs. On the basis of these principles, we discuss theoretical approaches concerning the underlying mechanism of BOIs. We propose a conceptualization based on Bayesian causal inference for addressing how our nervous system could infer whether an object belongs to our own body, using multisensory, sensorimotor, and semantic information, and we discuss how this can account for several experimental findings. Finally, we point to neural network models as an implementational framework within which the computational problem behind BOIs could be addressed in the future.

To blink or not to blink: fine cognitive tuning of the defensive peripersonal space

The blink reflex elicited by the electrical stimulation of the median nerve at the wrist [hand blink reflex (HBR)] is a subcortical, defensive response that is enhanced when the stimulated hand is inside the peripersonal space of the face. Such enhancement results from a tonic, top-down modulation of the excitability of the brainstem interneurons mediating the HBR. Here we aim to (1) characterize the somatotopical specificity of this top-down modulation and investigate its dependence on (2) cognitive expectations and (3) the presence of objects protecting the face, in healthy humans. Experiment 1 showed that the somatotopical specificity of the HBR enhancement is partially homosegmental, i.e., it is greater for the HBR elicited by the stimulation of the hand near the face compared with the other hand, always kept far from the face. Experiment 2 showed that the HBR is enhanced only when participants expect to receive stimuli on the hand close to the face and is thus strongly dependent on cognitive expectations. Experiment 3 showed that the HBR enhancement by hand-face proximity is suppressed when a thin wooden screen is placed between the participants' face and their hand. Thus, the screen reduces the extension of the defensive peripersonal space, so that the hand is never inside the peripersonal space of the face, even in the "near" condition. Together, these findings indicate a fine somatotopical and cognitive tuning of the excitability of brainstem circuits subserving the HBR, whose strength is adjusted depending on the context in a purposeful manner.