Visuohaptic convergence in a corticocerebellar network

Subcortical contributions to the sense of body ownership

The sense of body ownership (i.e. the feeling that our body or its parts belong to us) plays a key role in bodily self-consciousness and is believed to stem from multisensory integration. Experimental paradigms such as the rubber hand illusion have been developed to allow the controlled manipulation of body ownership in laboratory settings, providing effective tools for investigating malleability in the sense of body ownership and the boundaries that distinguish self from other. Neuroimaging studies of body ownership converge on the involvement of several cortical regions, including the premotor cortex and posterior parietal cortex. However, relatively less attention has been paid to subcortical structures that may also contribute to body ownership perception, such as the cerebellum and putamen. Here, on the basis of neuroimaging and neuropsychological observations, we provide an overview of relevant subcortical regions and consider their potential role in generating and maintaining a sense of ownership over the body. We also suggest novel avenues for future research targeting the role of subcortical regions in making sense of the body as our own.

A transdiagnostic network for psychiatric illness derived from atrophy and lesions

Psychiatric disorders share neurobiology and frequently co-occur. This neurobiological and clinical overlap highlights opportunities for transdiagnostic treatments. In this study, we used coordinate and lesion network mapping to test for a shared brain network across psychiatric disorders. In our meta-analysis of 193 studies, atrophy coordinates across six psychiatric disorders mapped to a common brain network defined by positive connectivity to anterior cingulate and insula, and by negative connectivity to posterior parietal and lateral occipital cortex. This network was robust to leave-one-diagnosis-out cross-validation and specific to atrophy coordinates from psychiatric versus neurodegenerative disorders (72 studies). In 194 patients with penetrating head trauma, lesion damage to this network correlated with the number of post-lesion psychiatric diagnoses. Neurosurgical ablation targets for psychiatric illness (four targets) also aligned with the network. This convergent brain network for psychiatric illness may partially explain high rates of psychiatric comorbidity and could highlight neuromodulation targets for patients with more than one psychiatric disorder.

MULTISENSORY RESPONSES IN A BELT REGION OF THE DORSAL AUDITORY CORTICAL PATHWAY

A basic function of the cerebral cortex is to receive and integrate information from different sensory modalities into a comprehensive percept of the environment. Neurons that demonstrate multisensory convergence occur across the necortex, but are especially prevalent in higher-order, association areas. However, a recent study of a cat higher-order auditory area, the dorsal zone (DZ) of auditory cortex, did not observe any multisensory features. Therefore, the goal of the present investigation was to address this conflict using recording and testing methodologies that are established for exposing and studying multisensory neuronal processing. Among the 482 neurons studied, we found that 76.6% were influenced by non-auditory stimuli. Of these neurons, 99% were affected by visual stimulation, but only 11% by somatosensory. Furthermore, a large proportion of the multisensory neurons showed integrated responses to multisensory stimulation, constituted a majority of the excitatory and inhibitory neurons encountered (as identified by the duration of their waveshape), and exhibited a distinct spatial distribution within DZ. These findings demonstrate that the dorsal zone of auditory cortex robustly exhibits multisensory properties and that the proportions of multisensory neurons encountered are consistent with those identified in other higher-order cortices.

Hippocampus plays a role in speech feedback processing

There is increasing evidence that the hippocampus is involved in language production and verbal communication, although little is known about its possible role. According to one view, hippocampus contributes semantic memory to spoken language. Alternatively, hippocampus is involved in the processing the (mis)match between expected sensory consequences of speaking and the perceived speech feedback. In the current study, we re-analysed functional magnetic resonance (fMRI) data of two overt picture-naming studies to test whether hippocampus is involved in speech production and, if so, whether the results can distinguish between a "pure memory" versus a "prediction" account of hippocampal involvement. In both studies, participants overtly named pictures during scanning while hearing their own speech feedback unimpededly or impaired by a superimposed noise mask. Results showed decreased hippocampal activity when speech feedback was impaired, compared to when feedback was unimpeded. Further, we found increased functional coupling between auditory cortex and hippocampus during unimpeded speech feedback, compared to impaired feedback. Finally, we found significant functional coupling between a hippocampal/supplementary motor area (SMA) interaction term and auditory cortex, anterior cingulate cortex and cerebellum during overt picture naming, but not during listening to one's own pre-recorded voice. These findings indicate that hippocampus plays a role in speech production that is in accordance with a "prediction" view of hippocampal functioning.

Central neurogenetic signatures of the visuomotor integration system

Visuomotor impairments characterize numerous neurological disorders and neurogenetic syndromes, such as autism spectrum disorder (ASD) and Dravet, Fragile X, Prader-Willi, Turner, and Williams syndromes. Despite recent advances in systems neuroscience, the biological basis underlying visuomotor functional impairments associated with these clinical conditions is poorly understood. In this study, we used neuroimaging connectomic approaches to map the visuomotor integration (VMI) system in the human brain and investigated the topology approximation of the VMI network to the Allen Human Brain Atlas, a whole-brain transcriptome-wide atlas of cortical genetic expression. We found the genetic expression of four genes-TBR1, SCN1A, MAGEL2, and CACNB4-to be prominently associated with visuomotor integrators in the human cortex. TBR1 gene transcripts, an ASD gene whose expression is related to neural development of the cortex and the hippocampus, showed a central spatial allocation within the VMI system. Our findings delineate gene expression traits underlying the VMI system in the human cortex, where specific genes, such as TBR1, are likely to play a central role in its neuronal organization, as well as on specific phenotypes of neurogenetic syndromes.

Multisensory Part-based Representations of Objects in Human Lateral Occipital Cortex

The format of high-level object representations in temporal-occipital cortex is a fundamental and as yet unresolved issue. Here we use fMRI to show that human lateral occipital cortex (LOC) encodes novel 3-D objects in a multisensory and part-based format. We show that visual and haptic exploration of objects leads to similar patterns of neural activity in human LOC and that the shared variance between visually and haptically induced patterns of BOLD contrast in LOC reflects the part structure of the objects. We also show that linear classifiers trained on neural data from LOC on a subset of the objects successfully predict a novel object based on its component part structure. These data demonstrate a multisensory code for object representations in LOC that specifies the part structure of objects.

Preserved Haptic Shape Processing after Bilateral LOC Lesions

The visual and haptic perceptual systems are understood to share a common neural representation of object shape. A region thought to be critical for recognizing visual and haptic shape information is the lateral occipital complex (LOC). We investigated whether LOC is essential for haptic shape recognition in humans by studying behavioral responses and brain activation for haptically explored objects in a patient (M.C.) with bilateral lesions of the occipitotemporal cortex, including LOC. Despite severe deficits in recognizing objects using vision, M.C. was able to accurately recognize objects via touch. M.C.'s psychophysical response profile to haptically explored shapes was also indistinguishable from controls. Using fMRI, M.C. showed no object-selective visual or haptic responses in LOC, but her pattern of haptic activation in other brain regions was remarkably similar to healthy controls. Although LOC is routinely active during visual and haptic shape recognition tasks, it is not essential for haptic recognition of object shape.

SIGNIFICANCE STATEMENT

The lateral occipital complex (LOC) is a brain region regarded to be critical for recognizing object shape, both in vision and in touch. However, causal evidence linking LOC with haptic shape processing is lacking. We studied recognition performance, psychophysical sensitivity, and brain response to touched objects, in a patient (M.C.) with extensive lesions involving LOC bilaterally. Despite being severely impaired in visual shape recognition, M.C. was able to identify objects via touch and she showed normal sensitivity to a haptic shape illusion. M.C.'s brain response to touched objects in areas of undamaged cortex was also very similar to that observed in neurologically healthy controls. These results demonstrate that LOC is not necessary for recognizing objects via touch.

Enhanced visuo-haptic integration for the non-dominant hand

Visuo-haptic integration contributes essentially to object shape recognition. Although there has been a considerable advance in elucidating the neural underpinnings of multisensory perception, it is still unclear whether seeing an object and exploring it with the dominant hand elicits the same brain response as compared to the non-dominant hand. Using fMRI to measure brain activation in right-handed participants, we found that for both left- and right-hand stimulation the left lateral occipital complex (LOC) and anterior cerebellum (aCER) were involved in visuo-haptic integration of familiar objects. These two brain regions were then further investigated in another study, where unfamiliar, novel objects were presented to a different group of right-handers. Here the left LOC and aCER were more strongly activated by bimodal than unimodal stimuli only when the left but not the right hand was used. A direct comparison indicated that the multisensory gain of the fMRI activation was significantly higher for the left than the right hand. These findings are in line with the principle of "inverse effectiveness", implying that processing of bimodally presented stimuli is particularly enhanced when the unimodal stimuli are weak. This applies also when right-handed subjects see and simultaneously touch unfamiliar objects with their non-dominant left hand. Thus, the fMRI signal in the left LOC and aCER induced by visuo-haptic stimulation is dependent on which hand was employed for haptic exploration.

Haptic shape processing in visual cortex

Humans typically rely upon vision to identify object shape, but we can also recognize shape via touch (haptics). Our haptic shape recognition ability raises an intriguing question: To what extent do visual cortical shape recognition mechanisms support haptic object recognition? We addressed this question using a haptic fMRI repetition design, which allowed us to identify neuronal populations sensitive to the shape of objects that were touched but not seen. In addition to the expected shape-selective fMRI responses in dorsal frontoparietal areas, we observed widespread shape-selective responses in the ventral visual cortical pathway, including primary visual cortex. Our results indicate that shape processing via touch engages many of the same neural mechanisms as visual object recognition. The shape-specific repetition effects we observed in primary visual cortex show that visual sensory areas are engaged during the haptic exploration of object shape, even in the absence of concurrent shape-related visual input. Our results complement related findings in visually deprived individuals and highlight the fundamental role of the visual system in the processing of object shape.

Disintegration of multisensory signals from the real hand reduces default limb self-attribution: an fMRI study

The perception of our limbs in space is built upon the integration of visual, tactile, and proprioceptive signals. Accumulating evidence suggests that these signals are combined in areas of premotor, parietal, and cerebellar cortices. However, it remains to be determined whether neuronal populations in these areas integrate hand signals according to basic temporal and spatial congruence principles of multisensory integration. Here, we developed a setup based on advanced 3D video technology that allowed us to manipulate the spatiotemporal relationships of visuotactile (VT) stimuli delivered on a healthy human participant's real hand during fMRI and investigate the ensuing neural and perceptual correlates. Our experiments revealed two novel findings. First, we found responses in premotor, parietal, and cerebellar regions that were dependent upon the spatial and temporal congruence of VT stimuli. This multisensory integration effect required a simultaneous match between the seen and felt postures of the hand, which suggests that congruent visuoproprioceptive signals from the upper limb are essential for successful VT integration. Second, we observed that multisensory conflicts significantly disrupted the default feeling of ownership of the seen real limb, as indexed by complementary subjective, psychophysiological, and BOLD measures. The degree to which self-attribution was impaired could be predicted from the attenuation of neural responses in key multisensory areas. These results elucidate the neural bases of the integration of multisensory hand signals according to basic spatiotemporal principles and demonstrate that the disintegration of these signals leads to "disownership" of the seen real hand.

Multisensory interactions between auditory and haptic object recognition

Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.

Resting-state functional connectivity of the vermal and hemispheric subregions of the cerebellum with both the cerebral cortical networks and subcortical structures

The human cerebellum is a heterogeneous structure, and the pattern of resting-state functional connectivity (rsFC) of each subregion has not yet been fully characterized. We aimed to systematically investigate rsFC pattern of each cerebellar subregion in 228 healthy young adults. Voxel-based analysis revealed that several subregions showed similar rsFC patterns, reflecting functional integration; however, different subregions displayed distinct rsFC patterns, representing functional segregation. The same vermal and hemispheric subregions showed either different patterns or different strengths of rsFCs with the cerebrum, and different subregions of lobules VII and VIII displayed different rsFC patterns. Region of interest (ROI)-based analyses also confirmed these findings. Specifically, strong rsFCs were found: between lobules I-VI and vermal VIIb-IX and the visual network; between hemispheric VI, VIIb, VIIIa and the auditory network; between lobules I-VI, VIII and the sensorimotor network; between lobule IX, vermal VIIIb and the default-mode network; between lobule Crus I, hemispheric Crus II and the fronto-parietal network; between hemispheric VIIb, VIII and the task-positive network; between hemispheric VI, VIIb, VIII and the salience network; between most cerebellar subregions and the thalamus; between lobules V, VIIb and the midbrain red nucleus; between hemispheric Crus I, Crus II, vermal VIIIb, IX and the caudate nucleus; between lobules V, VI, VIIb, VIIIa and the pallidum and putamen; and between lobules I-V, hemispheric VIII, IX and the hippocampus and amygdala. These results confirm the existence of both functional integration and segregation among cerebellar subregions and largely improve our understanding of the functional organization of the human cerebellum.

Visuo-haptic neuronal convergence demonstrated with an inversely effective pattern of BOLD activation

We investigated the neural substrates involved in visuo-haptic neuronal convergence using an additive-factors design in combination with fMRI. Stimuli were explored under three sensory modality conditions: viewing the object through a mirror without touching (V), touching the object with eyes closed (H), or simultaneously viewing and touching the object (VH). This modality factor was crossed with a task difficulty factor, which had two levels. On the basis of an idea similar to the principle of inverse effectiveness, we predicted that increasing difficulty would increase the relative level of multisensory gain in brain regions where visual and haptic sensory inputs converged. An ROI analysis focused on the lateral occipital tactile-visual area found evidence of inverse effectiveness in the left lateral occipital tactile-visual area, but not in the right. A whole-brain analysis also found evidence for the same pattern in the anterior aspect of the intraparietal sulcus, the premotor cortex, and the posterior insula, all in the left hemisphere. In conclusion, this study is the first to demonstrate visuo-haptic neuronal convergence based on an inversely effective pattern of brain activation.

Smoking experience modulates the cortical integration of vision and haptics

Human neuroplasticity of multisensory integration has been studied mainly in the context of natural or artificial training situations in healthy subjects. However, regular smokers also offer the opportunity to assess the impact of intensive daily multisensory interactions with smoking-related objects on the neural correlates of crossmodal object processing. The present functional magnetic resonance imaging study revealed that smokers show a comparable visuo-haptic integration pattern for both smoking paraphernalia and control objects in the left lateral occipital complex, a region playing a crucial role in crossmodal object recognition. Moreover, the degree of nicotine dependence correlated positively with the magnitude of visuo-haptic integration in the left lateral occipital complex (LOC) for smoking-associated but not for control objects. In contrast, in the left LOC non-smokers displayed a visuo-haptic integration pattern for control objects, but not for smoking paraphernalia. This suggests that prolonged smoking-related multisensory experiences in smokers facilitate the merging of visual and haptic inputs in the lateral occipital complex for the respective stimuli. Studying clinical populations who engage in compulsive activities may represent an ecologically valid approach to investigating the neuroplasticity of multisensory integration.

Integration of visual and tactile signals from the hand in the human brain: an FMRI study

In the non-human primate brain, a number of multisensory areas have been described where individual neurons respond to visual, tactile and bimodal visuotactile stimulation of the upper limb. It has been shown that such bimodal neurons can integrate sensory inputs in a linear or nonlinear fashion. In humans, activity in a similar set of brain regions has been associated with visuotactile stimulation of the hand. However, little is known about how these areas integrate visual and tactile information. In this functional magnetic resonance imaging experiment, we employed tactile, visual, and visuotactile stimulation of the right hand in an ecologically valid setup where participants were looking directly at their upper limb. We identified brain regions that were activated by both visual and tactile stimuli as well as areas exhibiting greater activity in the visuotactile condition than in both unisensory ones. The posterior and inferior parietal, dorsal, and ventral premotor cortices, as well as the cerebellum, all showed evidence of multisensory linear (additive) responses. Nonlinear, superadditive responses were observed in the cortex lining the left anterior intraparietal sulcus, the insula, dorsal premotor cortex, and, subcortically, the putamen. These results identify a set of candidate frontal, parietal and subcortical regions that integrate visual and tactile information for the multisensory perception of one's own hand.