Organization and plasticity in multisensory integration: early and late experience affects its governing principles

The Sensory Processing Measure - Second Edition: A Critical Review and Appraisal

This paper aims to provide an overview and critique of the newly updated Sensory Processing Measure - second edition (SPM-2) to assist clinicians in understanding its strengths and limitations and in selecting an appropriate sensory processing scale that best meets their clients' needs. Using four established research methodological quality critique tools, the SPM-2's family of age-related versions was evaluated by administering the tools separately and discussed their scoring and findings to reach a consensus on all ratings. The tools identified several strengths and weaknesses of the SPM-2. The SPM-2's items on its age-related versions scored well in the criteria of internal consistency, test-retest reliability, explanation of the instrument design, construct validity, content validity, criterion validity and ease of administration, and scoring. However, a lack of normative data for participant groups outside of the United States, methodological limitations, and a lack of investigation into some important psychometric properties, particularly responsiveness, were identified as notable weaknesses of the SPM-2's items on its age-related versions based on the critique criteria. Overall, despite the areas needing further development and investigation, the SPM-2 is considered a psychometrically sound tool that provides a reliable and valid approach to measuring aspects of the sensory processing construct across the lifespan.

Pathways for Naturalistic Looking Behavior in Primate II. Superior Colliculus Integrates Parallel Top-down and Bottom-up Inputs

Volitional signals for gaze control are provided by multiple parallel pathways converging on the midbrain superior colliculus (SC), whose deeper layers output to the brainstem gaze circuits. In the first of two papers (Takahashi and Veale, 2023), we described the properties of gaze behavior of several species under both laboratory and natural conditions, as well as the current understanding of the brainstem and spinal cord circuits implementing gaze control in primate. In this paper, we review the parallel pathways by which sensory and task information reaches SC and how these sensory and task signals interact within SC's multilayered structure. This includes both bottom-up (world statistics) signals mediated by sensory cortex, association cortex, and subcortical structures, as well as top-down (goal and task) influences which arrive via either direct excitatory pathways from cerebral cortex, or via indirect basal ganglia relays resulting in inhibition or dis-inhibition as appropriate for alternative behaviors. Models of attention such as saliency maps serve as convenient frameworks to organize our understanding of both the separate computations of each neural pathway, as well as the interaction between the multiple parallel pathways influencing gaze. While the spatial interactions between gaze's neural pathways are relatively well understood, the temporal interactions between and within pathways will be an important area of future study, requiring both improved technical methods for measurement and improvement of our understanding of how temporal dynamics results in the observed spatiotemporal allocation of gaze.

Supervised Multisensory Calibration Signals Are Evident in VIP But Not MSTd

Multisensory plasticity enables our senses to dynamically adapt to each other and the external environment, a fundamental operation that our brain performs continuously. We searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) in 2 male rhesus macaques using a paradigm of supervised calibration. We report little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. In contrast, neural correlates of plasticity are found in higher-level multisensory VIP, an area with strong decision-related activity. Accordingly, we observed systematic shifts of VIP tuning curves, which were reflected in the choice-related component of the population response. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions. These results lay the foundation for understanding multisensory neural plasticity, applicable broadly to maintaining accuracy for sensorimotor tasks.SIGNIFICANCE STATEMENT Multisensory plasticity is a fundamental and continual function of the brain that enables our senses to adapt dynamically to each other and to the external environment. Yet, very little is known about the neuronal mechanisms of multisensory plasticity. In this study, we searched for neural correlates of adult multisensory plasticity in the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP) using a paradigm of supervised calibration. We found little plasticity in neural responses in the relatively low-level multisensory cortical area MSTd. By contrast, neural correlates of plasticity were found in VIP, a higher-level multisensory area with strong decision-related activity. This is the first demonstration of neuronal calibration, together with behavioral calibration, in single sessions.

Neuroplasticity and Crossmodal Connectivity in the Normal, Healthy Brain

Objective

Neuroplasticity enables the brain to establish new crossmodal connections or reorganize old connections which are essential to perceiving a multisensorial world. The intent of this review is to identify and summarize the current developments in neuroplasticity and crossmodal connectivity, and deepen understanding of how crossmodal connectivity develops in the normal, healthy brain, highlighting novel perspectives about the principles that guide this connectivity.

Methods

To the above end, a narrative review is carried out. The data documented in prior relevant studies in neuroscience, psychology and other related fields available in a wide range of prominent electronic databases are critically assessed, synthesized, interpreted with qualitative rather than quantitative elements, and linked together to form new propositions and hypotheses about neuroplasticity and crossmodal connectivity.

Results

Three major themes are identified. First, it appears that neuroplasticity operates by following eight fundamental principles and crossmodal integration operates by following three principles. Second, two different forms of crossmodal connectivity, namely direct crossmodal connectivity and indirect crossmodal connectivity, are suggested to operate in both unisensory and multisensory perception. Third, three principles possibly guide the development of crossmodal connectivity into adulthood. These are labeled as the principle of innate crossmodality, the principle of evolution-driven 'neuromodular' reorganization and the principle of multimodal experience. These principles are combined to develop a three-factor interaction model of crossmodal connectivity.

Conclusions

The hypothesized principles and the proposed model together advance understanding of neuroplasticity, the nature of crossmodal connectivity, and how such connectivity develops in the normal, healthy brain.

Reduced Functional Connectivity in Children With Congenital Cataracts Using Resting-State Electroencephalography Measurement

Objectives

Numerous task-based functional magnetic resonance imaging studies indicate the presence of compensatory functional improvement in patients with congenital cataracts. However, there is neuroimaging evidence that shows decreased sensory perception or cognition information processing related to visual dysfunction, which favors a general loss hypothesis. This study explored the functional connectivity between visual and other networks in children with congenital cataracts using resting state electroencephalography.

Methods

Twenty-one children with congenital cataracts (age: 8.02 ± 2.03 years) and thirty-five sex- and age-matched normal sighted controls were enrolled to investigate functional connectivity between the visual cortex and the default mode network, the salience network, and the cerebellum network during resting state electroencephalography (eyes closed) recordings.

Result

The congenital cataract group was less active, than the control group, in the occipital, temporal, frontal and limbic lobes in the theta, alpha, beta1 and beta2 frequency bands. Additionally, there was reduced alpha-band connectivity between the visual and somatosensory cortices and between regions of the frontal and parietal cortices associated with cognitive and attentive control.

Conclusion

The results indicate abnormalities in sensory, cognition, motion and execution functional connectivity across the developing brains of children with congenital cataracts when compared with normal controls. Reduced frontal alpha activity and alpha-band connectivity between the visual cortex and salience network might reflect attenuated inhibitory information flow, leading to higher attentional states, which could contribute to adaptation of environmental change in this group of patients.

Layer 4 organization and respiration locking in the rodent nose somatosensory cortex

Rodents and other mammals acquire sensory information by precisely orchestrated head, whisker, and respiratory movements. We have, however, only limited information about integration of these signals. In the somatosensory domain, the integration of somatosensory information with other modalities is particularly pertinent for body parts such as eyes, ears, and nose, which serve another modality. Here we analyzed the nose/nostril representation in the rodent somatosensory cortex. We identified the representation of the nose/nostril in the rat somatosensory cortex by receptive field mapping and subsequent histological reconstruction. In tangential somatosensory cortical sections, the rat nostril cortex was evident as a prominent stripe-like recess of layer 4 revealed by cytochrome-c oxidase reactivity or by antibodies against the vesicular glutamate-transporter-2 (identifying thalamic afferents). We compared flattened somatosensory cortices of various rodents including rats, mice, gerbils, chinchillas, and chipmunks. We found that such a nose/nostril module was evident as a region with thinned or absent layer 4 at the expected somatotopic position of the nostril. Extracellular spike activity was strongly modulated by respiration in the rat somatosensory cortex, and field potential recordings revealed a stronger locking of nostril recording sites to respiration than for whisker/barrel cortex recoding sites. We conclude that the rodent nose/nostril representation has a conserved architecture and specifically interfaces with respiration signals.NEW & NOTEWORTHY We characterized the rodent nose somatosensory cortex. The nostril representation appeared as a kind of "hole" (i.e., as a stripe-like recess of layer 4) in tangential cortical sections. Neural activity in nose somatosensory cortex was locked to respiration, and simultaneous field recordings indicate that this locking was specific to this region. Our results reveal previously unknown cytoarchitectonic and physiological properties of the rodent nose somatosensory cortex, potentially enabling it to integrate multiple sensory modalities.

Development and binocular matching of orientation selectivity in visual cortex: a computational model

In mouse visual cortex, right after eye opening binocular cells have different preferred orientations for input from the two eyes. With normal visual experience during a critical period, these preferred orientations evolve and eventually become well matched. To gain insight into the matching process, we developed a computational model of a cortical cell receiving orientation selective inputs via plastic synapses. The model captures the experimentally observed matching of the preferred orientations, the dependence of matching on ocular dominance of the cell, and the relationship between the degree of matching and the resulting monocular orientation selectivity. Moreover, our model puts forward testable predictions: 1) The matching speed increases with initial ocular dominance. 2) While the matching improves more slowly for cells that are more orientation selective, the selectivity increases faster for better matched cells during the matching process. This suggests that matching drives orientation selectivity but not vice versa. 3) There are two main routes to matching: the preferred orientations either drift toward each other or one of the orientations switches suddenly. The latter occurs for cells with large initial mismatch and can render the cells monocular. We expect that these results provide insight more generally into the development of neuronal systems that integrate inputs from multiple sources, including different sensory modalities.NEW & NOTEWORTHY Animals gather information through multiple modalities (vision, audition, touch, etc.). These information streams have to be merged coherently to provide a meaningful representation of the world. Thus, for neurons in visual cortex V1, the orientation selectivities for inputs from the two eyes have to match to enable binocular vision. We analyze the postnatal process underlying this matching using computational modeling. It captures recent experimental results and reveals interdependence between matching, ocular dominance, and orientation selectivity.

A Theoretical Framework of Haptic Processing in Automotive User Interfaces and Its Implications on Design and Engineering

Driving a car is a highly visual task. Despite the trend towards increased driver assistance and autonomous vehicles, drivers still need to interact with the car for both driving and non-driving relevant tasks, at times simultaneously. The often-resulting high cognitive load is a safety issue, which can be addressed by providing the driver with alternative feedback modalities, such as haptics. Recent trends in the automotive industry are moving towards the seamless integration of control elements through touch-sensitive surfaces. Psychological knowledge on optimally utilizing haptic technologies remains limited. The literature on automotive haptic feedback consists mainly of singular findings without putting them into a broader user context with respect to haptic design of interfaces. Moreover, haptic feedback has primarily been limited to the confirmation of control actions rather than the searching or finding of control elements, the latter of which becomes particularly important considering the current trends. This paper presents an integrated framework on haptic processing in automotive user interfaces and provides guidelines for haptic design of user interfaces in car interiors.

Multisensory processing in event-based prospective memory

Failures in prospective memory (PM) - that is, the failure to remember intended future actions - can have adverse consequences. It is therefore important to study those processes that may help to minimize such cognitive failures. Although multisensory integration has been shown to enhance a wide variety of behaviors, including perception, learning, and memory, its effect on prospective memory, in particular, is largely unknown. In the present study, we investigated the effects of multisensory processing on two simultaneously-performed memory tasks: An ongoing 2- or 3-back working memory (WM) task (20% target ratio), and a PM task in which the participants had to respond to a rare predefined letter (8% target ratio). For PM trials, multisensory enhancement was observed for congruent multisensory signals; however, this effect did not generalize to the ongoing WM task. Participants were less likely to make errors for PM than for WM trials, thus suggesting that they may have biased their attention toward the PM task. Multisensory advantages on memory tasks, such as PM and WM, may be dependent on how attention resources are allocated across dual tasks.

Development of the Mechanisms Governing Midbrain Multisensory Integration

The ability to integrate information across multiple senses enhances the brain's ability to detect, localize, and identify external events. This process has been well documented in single neurons in the superior colliculus (SC), which synthesize concordant combinations of visual, auditory, and/or somatosensory signals to enhance the vigor of their responses. This increases the physiological salience of crossmodal events and, in turn, the speed and accuracy of SC-mediated behavioral responses to them. However, this capability is not an innate feature of the circuit and only develops postnatally after the animal acquires sufficient experience with covariant crossmodal events to form links between their modality-specific components. Of critical importance in this process are tectopetal influences from association cortex. Recent findings suggest that, despite its intuitive appeal, a simple generic associative rule cannot explain how this circuit develops its ability to integrate those crossmodal inputs to produce enhanced multisensory responses. The present neurocomputational model explains how this development can be understood as a transition from a default state in which crossmodal SC inputs interact competitively to one in which they interact cooperatively. Crucial to this transition is the operation of a learning rule requiring coactivation among tectopetal afferents for engagement. The model successfully replicates findings of multisensory development in normal cats and cats of either sex reared with special experience. In doing so, it explains how the cortico-SC projections can use crossmodal experience to craft the multisensory integration capabilities of the SC and adapt them to the environment in which they will be used.SIGNIFICANCE STATEMENT The brain's remarkable ability to integrate information across the senses is not present at birth, but typically develops in early life as experience with crossmodal cues is acquired. Recent empirical findings suggest that the mechanisms supporting this development must be more complex than previously believed. The present work integrates these data with what is already known about the underlying circuit in the midbrain to create and test a mechanistic model of multisensory development. This model represents a novel and comprehensive framework that explains how midbrain circuits acquire multisensory experience and reveals how disruptions in this neurotypic developmental trajectory yield divergent outcomes that will affect the multisensory processing capabilities of the mature brain.

Multisensory emotion perception in congenitally, early, and late deaf CI users

Emotions are commonly recognized by combining auditory and visual signals (i.e., vocal and facial expressions). Yet it is unknown whether the ability to link emotional signals across modalities depends on early experience with audio-visual stimuli. In the present study, we investigated the role of auditory experience at different stages of development for auditory, visual, and multisensory emotion recognition abilities in three groups of adolescent and adult cochlear implant (CI) users. CI users had a different deafness onset and were compared to three groups of age- and gender-matched hearing control participants. We hypothesized that congenitally deaf (CD) but not early deaf (ED) and late deaf (LD) CI users would show reduced multisensory interactions and a higher visual dominance in emotion perception than their hearing controls. The CD (n = 7), ED (deafness onset: <3 years of age; n = 7), and LD (deafness onset: >3 years; n = 13) CI users and the control participants performed an emotion recognition task with auditory, visual, and audio-visual emotionally congruent and incongruent nonsense speech stimuli. In different blocks, participants judged either the vocal (Voice task) or the facial expressions (Face task). In the Voice task, all three CI groups performed overall less efficiently than their respective controls and experienced higher interference from incongruent facial information. Furthermore, the ED CI users benefitted more than their controls from congruent faces and the CD CI users showed an analogous trend. In the Face task, recognition efficiency of the CI users and controls did not differ. Our results suggest that CI users acquire multisensory interactions to some degree, even after congenital deafness. When judging affective prosody they appear impaired and more strongly biased by concurrent facial information than typically hearing individuals. We speculate that limitations inherent to the CI contribute to these group differences.

From Cortical Blindness to Conscious Visual Perception: Theories on Neuronal Networks and Visual Training Strategies

Homonymous hemianopia (HH) is the most common cortical visual impairment leading to blindness in the contralateral hemifield. It is associated with many inconveniences and daily restrictions such as exploration and visual orientation difficulties. However, patients with HH can preserve the remarkable ability to unconsciously perceive visual stimuli presented in their blindfield, a phenomenon known as blindsight. Unfortunately, the nature of this captivating residual ability is still misunderstood and the rehabilitation strategies in terms of visual training have been insufficiently exploited. This article discusses type I and type II blindsight in a neuronal framework of altered global workspace, resulting from inefficient perception, attention and conscious networks. To enhance synchronization and create global availability for residual abilities to reach visual consciousness, rehabilitation tools need to stimulate subcortical extrastriate pathways through V5/MT. Multisensory bottom-up compensation combined with top-down restitution training could target pre-existing and new neuronal mechanisms to recreate a framework for potential functionality.

A Matched Comparison Across Three Different Sensory Pairs of Cross-Modal Temporal Recalibration From Sustained and Transient Adaptation

Sustained exposure to an asynchronous multisensory signal causes perceived simultaneity to shift in the direction of the leading component of the adapting stimulus. This is known as temporal recalibration, and recent evidence suggests that it can occur very rapidly, even after a single asynchronous audiovisual (AV) stimulus. However, this form of rapid recalibration appears to be unique to AV stimuli, in contrast to recalibration following sustained asynchronies which occurs with audiotactile (AT) and visuotactile (VT) stimuli. This study examines temporal recalibration to AV, VT and AT asynchrony with spatially collocated stimuli using a design that produces both sustained and inter-trial recalibration by combining the traditional sustained adaptation approach with an inter-trial analysis of sequential dependencies in an extended test period. Thus, we compare temporal recalibration to both sustained and transient asynchrony in three crossmodal combinations using the same design, stimuli and observers. The results reveal that prolonged exposure to asynchrony produced equivalent temporal recalibration for all combinations: AV, AT and VT. The pattern for rapid, inter-trial recalibration was very different. Rapid recalibration occurred strongly for AV stimuli, weakly for AT and did not occur at all for VT. For all sensory pairings, recalibration from sustained asynchrony decayed to baseline during the test phase while inter-trial recalibration was present and stable throughout testing, suggesting different mechanisms may underlie adaptation at long and short timescales.

Experience-dependent plasticity of excitatory and inhibitory intertectal inputs in Xenopus tadpoles

Communication between optic tecta/superior colliculi is thought to be required for sensorimotor behaviors by comparing inputs across the midline, however the development of and the role of visual experience in the function and plasticity of intertectal connections are unclear. We combined neuronal tracing, in vivo time-lapse imaging, and electrophysiology to characterize the structural and functional development of intertectal axons and synapses in Xenopus tadpole optic tectum. We find that intertectal connections are established early during optic tectal circuit development. We determined the neurotransmitter identity of intertectal neurons using both rabies virus-mediated tracing combined with post-hoc immunohistochemistry, and electrophysiology. Excitatory and inhibitory intertectal neuronal somata are similarly distributed throughout the tectum. Excitatory and inhibitory intertectal axons are structurally similar and elaborate broadly in the contralateral tectum. We demonstrate that intertectal and retinotectal axons converge onto tectal neurons by recording postsynaptic currents after stimulating intertectal and retinotectal inputs. Cutting the intertectal commissure removes synaptic responses to contralateral tectal stimulation. In vivo time-lapse imaging demonstrated that visual experience drives plasticity in intertectal bouton size and dynamics. Finally, visual experience coordinately drives the maturation of excitatory and inhibitory intertectal inputs by increasing AMPA- and GABA-receptor mediated currents, comparable to experience-dependent maturation of retinotectal inputs. These data indicate that visual experience regulates plasticity of excitatory and inhibitory intertectal inputs, maintaining the excitatory: inhibitory ratio of intertectal input. These studies place intertectal inputs as key players in tectal circuit development and suggest that they may play a role in sensory information processing critical to sensorimotor behaviors.

Crossmodal interactions during non-linguistic auditory processing in cochlear-implanted deaf patients

Due to signal distortion, speech comprehension in cochlear-implanted (CI) patients relies strongly on visual information, a compensatory strategy supported by important cortical crossmodal reorganisations. Though crossmodal interactions are evident for speech processing, it is unclear whether a visual influence is observed in CI patients during non-linguistic visual-auditory processing, such as face-voice interactions, which are important in social communication. We analyse and compare visual-auditory interactions in CI patients and normal-hearing subjects (NHS) at equivalent auditory performance levels. Proficient CI patients and NHS performed a voice-gender categorisation in the visual-auditory modality from a morphing-generated voice continuum between male and female speakers, while ignoring the presentation of a male or female visual face. Our data show that during the face-voice interaction, CI deaf patients are strongly influenced by visual information when performing an auditory gender categorisation task, in spite of maximum recovery of auditory speech. No such effect is observed in NHS, even in situations of CI simulation. Our hypothesis is that the functional crossmodal reorganisation that occurs in deafness could influence nonverbal processing, such as face-voice interaction; this is important for patient internal supramodal representation.

Dissecting neural circuits for multisensory integration and crossmodal processing

We rely on rich and complex sensory information to perceive and understand our environment. Our multisensory experience of the world depends on the brain's remarkable ability to combine signals across sensory systems. Behavioural, neurophysiological and neuroimaging experiments have established principles of multisensory integration and candidate neural mechanisms. Here we review how targeted manipulation of neural activity using invasive and non-invasive neuromodulation techniques have advanced our understanding of multisensory processing. Neuromodulation studies have provided detailed characterizations of brain networks causally involved in multisensory integration. Despite substantial progress, important questions regarding multisensory networks remain unanswered. Critically, experimental approaches will need to be combined with theory in order to understand how distributed activity across multisensory networks collectively supports perception.

The Neurobiological Basis of the Distress Thermometer: A PET Study in Cancer Patients

The objective of this study was to investigate the possible associations between the Distress Thermometer (DT) scores and the brain metabolism of structures involved in stress response. Twenty-one cancer patients were assessed using the DT, Problem Checklist and Hospital Anxiety and Depression Scale (HADS). The psychological measures were correlated with [18 F]PET-FDG brain glucose metabolism. Multiple and linear regression and binary logistic regression were run to analyse data. The DT and HADS scores illustrated that 48% of patients were distressed, 19% were depressed and 48% were anxious. Results showed that some subcortical areas activity, such as part of midbrain and of hypothalamus, was correlated with the DT scores. The Problem Checklist scores correlated with the activity of the same areas and included more regions in the limbic forebrain and brainstem. Compared with the DT and Problem Checklist, HADS-Depression scores showed a more extensive pattern of correlation with brain activity, including limbic and cortical areas. The results highlighted that the DT scores correlated with the activity of brain areas typically involved in stress response. Indeed, hypothalamus metabolism was found to be the best predictor of distressed patients.

An integrative role for the superior colliculus in selecting targets for movements

A fundamental goal of systems neuroscience is to understand the neural mechanisms underlying decision making. The midbrain superior colliculus (SC) is known to be central to the selection of one among many potential spatial targets for movements, which represents an important form of decision making that is tractable to rigorous experimental investigation. In this review, we first discuss data from mammalian models-including primates, cats, and rodents-that inform our understanding of how neural activity in the SC underlies the selection of targets for movements. We then examine the anatomy and physiology of inputs to the SC from three key regions that are themselves implicated in motor decisions-the basal ganglia, parabrachial region, and neocortex-and discuss how they may influence SC activity related to target selection. Finally, we discuss the potential for methodological advances to further our understanding of the neural bases of target selection. Our overarching goal is to synthesize what is known about how the SC and its inputs act together to mediate the selection of targets for movements, to highlight open questions about this process, and to spur future studies addressing these questions.

Multisensory interactions between vestibular, visual and somatosensory signals

Vestibular inputs are constantly processed and integrated with signals from other sensory modalities, such as vision and touch. The multiply-connected nature of vestibular cortical anatomy led us to investigate whether vestibular signals could participate in a multi-way interaction with visual and somatosensory perception. We used signal detection methods to identify whether vestibular stimulation might interact with both visual and somatosensory events in a detection task. Participants were instructed to detect near-threshold somatosensory stimuli that were delivered to the left index finger in one half of experimental trials. A visual signal occurred close to the finger in half of the trials, independent of somatosensory stimuli. A novel Near infrared caloric vestibular stimulus (NirCVS) was used to artificially activate the vestibular organs. Sham stimulations were used to control for non-specific effects of NirCVS. We found that both visual and vestibular events increased somatosensory sensitivity. Critically, we found no evidence for supra-additive multisensory enhancement when both visual and vestibular signals were administered together: in fact, we found a trend towards sub-additive interaction. The results are compatible with a vestibular role in somatosensory gain regulation.

Preference for concentric orientations in the mouse superior colliculus

The superior colliculus is a layered structure important for body- and gaze-orienting responses. Its superficial layer is, next to the lateral geniculate nucleus, the second major target of retinal ganglion axons and is retinotopically organized. Here we show that in the mouse there is also a precise organization of orientation preference. In columns perpendicular to the tectal surface, neurons respond to the same visual location and prefer gratings of the same orientation. Calcium imaging and extracellular recording revealed that the preferred grating varies with retinotopic location, and is oriented parallel to the concentric circle around the centre of vision through the receptive field. This implies that not all orientations are equally represented across the visual field. This makes the superior colliculus different from visual cortex and unsuitable for translation-invariant object recognition and suggests that visual stimuli might have different behavioural consequences depending on their retinotopic location.

The mammalian superior colliculus (SC) processes visual stimuli but little is known about the spatial organization of the response preferences for specific visual features. Here the authors show that the mouse SC contains a map for orientation preference such that preferred grating orientation is aligned to concentric circles around the centre of the visual field.

Predictive coding and multisensory integration: an attentional account of the multisensory mind

Multisensory integration involves a host of different cognitive processes, occurring at different stages of sensory processing. Here I argue that, despite recent insights suggesting that multisensory interactions can occur at very early latencies, the actual integration of individual sensory traces into an internally consistent mental representation is dependent on both top–down and bottom–up processes. Moreover, I argue that this integration is not limited to just sensory inputs, but that internal cognitive processes also shape the resulting mental representation. Studies showing that memory recall is affected by the initial multisensory context in which the stimuli were presented will be discussed, as well as several studies showing that mental imagery can affect multisensory illusions. This empirical evidence will be discussed from a predictive coding perspective, in which a central top–down attentional process is proposed to play a central role in coordinating the integration of all these inputs into a coherent mental representation.

The effect of early visual deprivation on the neural bases of multisensory processing

Developmental vision is deemed to be necessary for the maturation of multisensory cortical circuits. Thus far, this has only been investigated in animal studies, which have shown that congenital visual deprivation markedly reduces the capability of neurons to integrate cross-modal inputs. The present study investigated the effect of transient congenital visual deprivation on the neural mechanisms of multisensory processing in humans. We used functional magnetic resonance imaging to compare responses of visual and auditory cortical areas to visual, auditory and audio-visual stimulation in cataract-reversal patients and normally sighted controls. The results showed that cataract-reversal patients, unlike normally sighted controls, did not exhibit multisensory integration in auditory areas. Furthermore, cataract-reversal patients, but not normally sighted controls, exhibited lower visual cortical processing within visual cortex during audio-visual stimulation than during visual stimulation. These results indicate that congenital visual deprivation affects the capability of cortical areas to integrate cross-modal inputs in humans, possibly because visual processing is suppressed during cross-modal stimulation. Arguably, the lack of vision in the first months after birth may result in a reorganization of visual cortex, including the suppression of noisy visual input from the deprived retina in order to reduce interference during auditory processing.

Retinofugal projections in the mouse

The laboratory mouse is increasingly a subject for visual system investigation, but there has been no comprehensive evaluation of this species' visual projections. Here, projections were visualized and mapped following intraocular injection of cholera toxin B subunit. Tissue was processed using standard procedures applied to 30 μm free-floating sections with diaminobenzidine as the chromogen. The mouse retina projects to ~46 brain regions, including 14 not previously described in this species. These include two amygdaloid nuclei, the horizontal limb of the diagonal band, the paraventricular hypothalamic nucleus, several visual thalamic nuclei, the paranigral nucleus, several pretectal nuclei, and the dorsal cortex of the inferior colliculus. Dense retinal patches were also observed in a narrow portion of the ipsilateral intermediate layer of the superior colliculus. The superior fasciculus of the accessory optic tract, which innervates the medial terminal nucleus, was also determined to be a terminal zone throughout its length. The results are compared with previous descriptions of projections from mouse intrinsically photoreceptive retinal ganglion cells, and with data from the hamster, Nile grass rat, and laboratory rat. The retinal projection patterns are similar in all four species, although there are many differences with respect to the details. The specific visual functions of most retinorecipient areas are unknown, but there is substantial convergence of retinal projections onto regions concerned with olfaction and audition.

Atypical multisensory integration in Niemann-Pick type C disease - towards potential biomarkers

Background

Niemann-Pick type C (NPC) is an autosomal recessive disease in which cholesterol and glycosphingolipids accumulate in lysosomes due to aberrant cell-transport mechanisms. It is characterized by progressive and ultimately terminal neurological disease, but both pre-clinical studies and direct human trials are underway to test the safety and efficacy of cholesterol clearing compounds, with good success already observed in animal models. Key to assessing the effectiveness of interventions in patients, however, is the development of objective neurobiological outcome measures. Multisensory integration mechanisms present as an excellent candidate since they necessarily rely on the fidelity of long-range neural connections between the respective sensory cortices (e.g. the auditory and visual systems).

Methods

A simple way to test integrity of the multisensory system is to ask whether individuals respond faster to the occurrence of a bisensory event than they do to the occurrence of either of the unisensory constituents alone. Here, we presented simple auditory, visual, and audio-visual stimuli in random sequence. Participants responded as fast as possible with a button push. One 11-year-old and two 14-year-old boys with NPC participated in the experiment and their results were compared to those of 35 age-matched neurotypical boys.

Results

Reaction times (RTs) to the stimuli when presented simultaneously were significantly faster than when they were presented alone in the neurotypical children, a facilitation that could not be accounted for by probability summation, as evidenced by violation of the so-called ‘race’ model. In stark contrast, the NPC boys showed no such speeding, despite the fact that their unisensory RTs fell within the distribution of RTs observed in the neurotypicals.

Conclusions

These results uncover a previously undescribed deficit in multisensory integrative abilities in NPC, with implications for ongoing treatment of the clinical symptoms of these children. They also suggest that multisensory processes may represent a good candidate biomarker against which to test the efficacy of therapeutic interventions.

Electronic supplementary material

The online version of this article (doi:10.1186/s13023-014-0149-x) contains supplementary material, which is available to authorized users.

Brief cortical deactivation early in life has long-lasting effects on multisensory behavior

Detecting and locating environmental events are markedly enhanced by the midbrain's ability to integrate visual and auditory cues. Its capacity for multisensory integration develops in cats 1-4 months after birth but only after acquiring extensive visual-auditory experience. However, briefly deactivating specific regions of association cortex during this period induced long-term disruption of this maturational process, such that even 1 year later animals were unable to integrate visual and auditory cues to enhance their behavioral performance. The data from this animal model reveal a window of sensitivity within which association cortex mediates the encoding of cross-modal experience in the midbrain. Surprisingly, however, 3 years later, and without any additional intervention, the capacity appeared fully developed. This suggests that, although sensitivity degrades with age, the potential for acquiring or modifying multisensory integration capabilities extends well into adulthood.

The development of multisensory integration in high-functioning autism: high-density electrical mapping and psychophysical measures reveal impairments in the processing of audiovisual inputs

Successful integration of auditory and visual inputs is crucial for both basic perceptual functions and for higher-order processes related to social cognition. Autism spectrum disorders (ASD) are characterized by impairments in social cognition and are associated with abnormalities in sensory and perceptual processes. Several groups have reported that individuals with ASD are impaired in their ability to integrate socially relevant audiovisual (AV) information, and it has been suggested that this contributes to the higher-order social and cognitive deficits observed in ASD. However, successful integration of auditory and visual inputs also influences detection and perception of nonsocial stimuli, and integration deficits may impair earlier stages of information processing, with cascading downstream effects. To assess the integrity of basic AV integration, we recorded high-density electrophysiology from a cohort of high-functioning children with ASD (7-16 years) while they performed a simple AV reaction time task. Children with ASD showed considerably less behavioral facilitation to multisensory inputs, deficits that were paralleled by less effective neural integration. Evidence for processing differences relative to typically developing children was seen as early as 100 ms poststimulation, and topographic analysis suggested that children with ASD relied on different cortical networks during this early multisensory processing stage.

Oscillatory entrainment of primary somatosensory cortex encodes visual control of tactile processing

Optimal behavior relies on the successful integration of complementary information from multiple senses. The neural mechanisms underlying multisensory interactions are still poorly understood. Here, we demonstrate the critical role of neural network oscillations and direct connectivity between primary sensory cortices in visual-somatosensory interactions. Extracellular recordings from all layers of the barrel field in Brown Norway rats in vivo showed that bimodal stimulation (simultaneous light flash and whisker deflection) augmented the somatosensory-evoked response and changed the power of induced network oscillations by resetting their phase. Anatomical tracing revealed sparse direct connectivity between primary visual (V1) and somatosensory (S1) cortices. Pharmacological silencing of V1 diminished but did not abolish cross-modal effects on S1 oscillatory activity, while leaving the early enhancement of the evoked response unaffected. Thus, visual stimuli seem to impact tactile processing by modulating network oscillations in S1 via corticocortical projections and subcortical feedforward interactions.

Multisensory integration and neuroplasticity in the human cerebral cortex

There is a strong interaction between multisensory processing and the neuroplasticity of the human brain. On one hand, recent research demonstrates that experience and training in various domains modifies how information from the different senses is integrated; and, on the other hand multisensory training paradigms seem to be particularly effective in driving functional and structural plasticity. Multisensory training affects early sensory processing within separate sensory domains, as well as the functional and structural connectivity between uni- and multisensory brain regions. In this review, we discuss the evidence for interactions of multisensory processes and brain plasticity and give an outlook on promising clinical applications and open questions.

Eye opening and PSD95 are required for long-term potentiation in developing superior colliculus

The only major glutamate receptor membrane-associated guanylate kinase scaffolds expressed in the young superficial superior colliculus (SC) are synapse-associated protein 102 (SAP102) and postsynaptic density protein 95 (PSD95). In this, as in all visual brain regions examined, synaptic PSD95 increases rapidly following simultaneous eyelid opening (EO). We show that EO and PSD95 are necessary for SC NMDA receptor (NMDAR)-dependent long-term potentiation (LTP) and this LTP is eliminated or reinstated by manipulating EO. PSD95 knockdown (KD) in vivo blocks this LTP, but not long-term depression, and reduces frequencies of miniature AMPA receptor and NMDAR currents with no change in presynaptic release. Furthermore, miniature NMDAR currents after PSD95 KD show an activity-triggered calcineurin sensitivity that is normally only found in the pre-EO period when SAP102 binds mixed GluN2A/GluN2B NMDARs. These data indicate that young SC LTP arises from PSD95 unsilencing of silent synapses, that unsilencing is labile in young brain, and that even though SAP102 and PSD95 can bind the same NMDARs, only PSD95 enables SC synaptic maturation.

Relationship between temporomandibular disorders, widespread palpation tenderness, and multiple pain conditions: a case-control study

The multiple bodily pain conditions in temporomandibular disorders (TMD) have been associated with generalized alterations in pain processing. The purpose of this study was to examine the relationship between the presence of widespread body palpation tenderness (WPT) and the likelihood of multiple comorbid pain conditions in TMD patients and controls. This case-control study was conducted in 76 TMD subjects with WPT, 83 TMD subjects without WPT, and 181 non-TMD matched control subjects. The study population was also characterized for clinical pain, experimental pain sensitivity, and related psychological phenotypes. Results showed that: 1) TMD subjects reported an average of 1.7 comorbid pain conditions compared to .3 reported by the control subjects (P < .001); 2) Compared to control subjects, the odds ratio (OR) for multiple comorbid pain conditions is higher for TMD subjects with WPT [OR 8.4 (95% CI 3.1-22.8) for TMD with WPT versus OR 3.3 (95% CI 1.3-8.4) for TMD without WPT]; 3) TMD subjects with WPT presented with reduced pressure pain thresholds (PPTs) in both cranial and extracranial regions compared to TMD subjects without WPT; and 4) TMD subjects with WPT reported increased somatic symptoms. These findings suggest that pain assessment outside of the orofacial region may prove valuable for the classification, diagnosis, and management of TMD patients.

PERSPECTIVE

TMD subjects with WPT experience a greater level of multiple comorbid pain conditions, compared to TMD subjects without WPT and non-TMD controls. Integration of bodily pain assessments can be informative for evaluation, diagnosis, and management of TMD.