Multimodal convergence within the intraparietal sulcus of the macaque monkey

Multisensory integration in the mammalian brain: diversity and flexibility in health and disease

Multisensory integration (MSI) occurs in a variety of brain areas, spanning cortical and subcortical regions. In traditional studies on sensory processing, the sensory cortices have been considered for processing sensory information in a modality-specific manner. The sensory cortices, however, send the information to other cortical and subcortical areas, including the higher association cortices and the other sensory cortices, where the multiple modality inputs converge and integrate to generate a meaningful percept. This integration process is neither simple nor fixed because these brain areas interact with each other via complicated circuits, which can be modulated by numerous internal and external conditions. As a result, dynamic MSI makes multisensory decisions flexible and adaptive in behaving animals. Impairments in MSI occur in many psychiatric disorders, which may result in an altered perception of the multisensory stimuli and an abnormal reaction to them. This review discusses the diversity and flexibility of MSI in mammals, including humans, primates and rodents, as well as the brain areas involved. It further explains how such flexibility influences perceptual experiences in behaving animals in both health and disease. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Singing training predicts increased insula connectivity with speech and respiratory sensorimotor areas at rest

The insula contributes to the detection of salient events during goal-directed behavior and participates in the coordination of motor, multisensory, and cognitive systems. Recent task-fMRI studies with trained singers suggest that singing experience can enhance the access to these resources. However, the long-term effects of vocal training on insula-based networks are still unknown. In this study, we employed resting-state fMRI to assess experience-dependent differences in insula co-activation patterns between conservatory-trained singers and non-singers. Results indicate enhanced bilateral anterior insula connectivity in singers relative to non-singers with constituents of the speech sensorimotor network. Specifically, with the cerebellum (lobule V-VI) and the superior parietal lobes. The reversed comparison showed no effects. The amount of accumulated singing training predicted enhanced bilateral insula co-activation with primary sensorimotor areas representing the diaphragm and the larynx/phonation area-crucial regions for cortico-motor control of complex vocalizations-as well as the bilateral thalamus and the left putamen. Together, these findings highlight the neuroplastic effect of expert singing training on insula-based networks, as evidenced by the association between enhanced insula co-activation profiles in singers and the brain's speech motor system components.

Effects of developmental alcohol exposure on cortical multisensory integration

Fetal alcohol spectrum disorder (FASD) is one of the most common causes of mental disabilities in the world with a prevalence of 1%-6% of all births. Sensory processing deficits and cognitive problems are a major feature in this condition. Because developmental alcohol exposure can impair neuronal plasticity, and neuronal plasticity is crucial for the establishment of neuronal circuits in sensory areas, we predicted that exposure to alcohol during the third trimester equivalent of human gestation would disrupt the development of multisensory integration (MSI) in the rostral portion of the posterior parietal cortex (PPr), an integrative visual-tactile area. We conducted in vivo electrophysiology in 17 ferrets from four groups (saline/alcohol; infancy/adolescence). A total of 1157 neurons were recorded after visual, tactile and combined visual-tactile stimulation. A multisensory (MS) enhancement or suppression is characterized by a significantly increased or decreased number of elicited spikes after combined visual-tactile stimulation compared to the strongest unimodal (visual or tactile) response. At the neuronal level, those in infant animals were more prone to show MS suppression whereas adolescents were more prone to show MS enhancement. Although alcohol-treated animals showed similar developmental changes between infancy and adolescence, they always 'lagged behind' controls showing more MS suppression and less enhancement. Our findings suggest that alcohol exposure during the last months of human gestation would stunt the development of MSI, which could underlie sensory problems seen in FASD.

The current status and trend of the functional magnetic resonance combined with stimulation in animals

As a non-radiative, non-invasive imaging technique, functional magnetic resonance imaging (fMRI) has excellent effects on studying the activation of blood oxygen levels and functional connectivity of the brain in human and animal models. Compared with resting-state fMRI, fMRI combined with stimulation could be used to assess the activation of specific brain regions and the connectivity of specific pathways and achieve better signal capture with a clear purpose and more significant results. Various fMRI methods and specific stimulation paradigms have been proposed to investigate brain activation in a specific state, such as electrical, mechanical, visual, olfactory, and direct brain stimulation. In this review, the studies on animal brain activation using fMRI combined with different stimulation methods were retrieved. The instruments, experimental parameters, anesthesia, and animal models in different stimulation conditions were summarized. The findings would provide a reference for studies on estimating specific brain activation using fMRI combined with stimulation.

The Role of the Interaction between the Inferior Parietal Lobule and Superior Temporal Gyrus in the Multisensory Go/No-go Task

Information from multiple sensory modalities interacts. Using functional magnetic resonance imaging (fMRI), we aimed to identify the neural structures correlated with how cooccurring sound modulates the visual motor response execution. The reaction time (RT) to audiovisual stimuli was significantly faster than the RT to visual stimuli. Signal detection analyses showed no significant difference in the perceptual sensitivity (d') between audiovisual and visual stimuli, while the response criteria (β or c) of the audiovisual stimuli was decreased compared to the visual stimuli. The functional connectivity between the left inferior parietal lobule (IPL) and bilateral superior temporal gyrus (STG) was enhanced in Go processing compared with No-go processing of audiovisual stimuli. Furthermore, the left precentral gyrus (PreCG) showed enhanced functional connectivity with the bilateral STG and other areas of the ventral stream in Go processing compared with No-go processing of audiovisual stimuli. These results revealed that the neuronal network correlated with modulations of the motor response execution after the presentation of both visual stimuli along with cooccurring sound in a multisensory Go/Nogo task, including the left IPL, left PreCG, bilateral STG and some areas of the ventral stream. The role of the interaction between the IPL and STG in transforming audiovisual information into motor behavior is discussed. The current study provides a new perspective for exploring potential brain mechanisms underlying how humans execute appropriate behaviors on the basis of multisensory information.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

Parietal maps of visual signals for bodily action planning

The posterior parietal cortex (PPC) has long been understood as a high-level integrative station for computing motor commands for the body based on sensory (i.e., mostly tactile and visual) input from the outside world. In the last decade, accumulating evidence has shown that the parietal areas not only extract the pragmatic features of manipulable objects, but also subserve sensorimotor processing of others’ actions. A paradigmatic case is that of the anterior intraparietal area (AIP), which encodes the identity of observed manipulative actions that afford potential motor actions the observer could perform in response to them. On these bases, we propose an AIP manipulative action-based template of the general planning functions of the PPC and review existing evidence supporting the extension of this model to other PPC regions and to a wider set of actions: defensive and locomotor actions. In our model, a hallmark of PPC functioning is the processing of information about the physical and social world to encode potential bodily actions appropriate for the current context. We further extend the model to actions performed with man-made objects (e.g., tools) and artifacts, because they become integral parts of the subject’s body schema and motor repertoire. Finally, we conclude that existing evidence supports a generally conserved neural circuitry that transforms integrated sensory signals into the variety of bodily actions that primates are capable of preparing and performing to interact with their physical and social world.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Topographical and laminar distribution of audiovisual processing within human planum temporale

The brain is capable of integrating signals from multiple sensory modalities. Such multisensory integration can occur in areas that are commonly considered unisensory, such as planum temporale (PT) representing the auditory association cortex. However, the roles of different afferents (feedforward vs. feedback) to PT in multisensory processing are not well understood. Our study aims to understand that by examining laminar activity patterns in different topographical subfields of human PT under unimodal and multisensory stimuli. To this end, we adopted an advanced mesoscopic (sub-millimeter) fMRI methodology at 7 T by acquiring BOLD (blood-oxygen-level-dependent contrast, which has higher sensitivity) and VAPER (integrated blood volume and perfusion contrast, which has superior laminar specificity) signal concurrently, and performed all analyses in native fMRI space benefiting from an identical acquisition between functional and anatomical images. We found a division of function between visual and auditory processing in PT and distinct feedback mechanisms in different subareas. Specifically, anterior PT was activated more by auditory inputs and received feedback modulation in superficial layers. This feedback depended on task performance and likely arose from top-down influences from higher-order multimodal areas. In contrast, posterior PT was preferentially activated by visual inputs and received visual feedback in both superficial and deep layers, which is likely projected directly from the early visual cortex. Together, these findings provide novel insights into the mechanism of multisensory interaction in human PT at the mesoscopic spatial scale.

Peripersonal space in the front, rear, left and right directions for audio-tactile multisensory integration

Peripersonal space (PPS) is important for humans to perform body–environment interactions. However, many previous studies only focused on the specific direction of the PPS, such as the front space, despite suggesting that there were PPSs in all directions. We aimed to measure and compare the peri-trunk PPS in four directions (front, rear, left, and right). To measure the PPS, we used a tactile and an audio stimulus because auditory information is available at any time in all directions. We used the approaching and receding task-irrelevant sounds in the experiment. Observers were asked to respond as quickly as possible when a tactile stimulus was applied to a vibrator on their chest. We found that peri-trunk PPS representations exist with an approaching sound, irrespective of the direction.

Action-dependent processing of self-motion in parietal cortex of macaque monkeys

Successful interaction with the environment requires the dissociation of self-induced from externally induced sensory stimulation. Temporal proximity of action and effect is hereby often used as an indicator of whether an observed event should be interpreted as a result of own actions or not. We tested how the delay between an action (press of a touch bar) and an effect (onset of simulated self-motion) influences the processing of visually simulated self-motion in the ventral intraparietal area (VIP) of macaque monkeys. We found that a delay between the action and the start of the self-motion stimulus led to a rise of activity above the baseline activity before motion onset in a subpopulation of 21% of the investigated neurons. In the responses to the stimulus, we found a significantly lower sustained activity when the press of a touch bar and the motion onset were contiguous compared to the condition when the motion onset was delayed. We speculate that this weak inhibitory effect might be part of a mechanism that sharpens the tuning of VIP neurons during self-induced motion and thus has the potential to increase the precision of heading information that is required to adjust the orientation of self-motion in everyday navigational tasks.NEW & NOTEWORTHY Neurons in macaque ventral intraparietal area (VIP) are responding to sensory stimulation related to self-motion, e.g. visual optic flow. Here, we found that self-motion induced activation depends on the sense of agency, i.e., it differed when optic flow was perceived as self- or externally induced. This demonstrates that area VIP is well suited for study of the interplay between active behavior and sensory processing during self-motion.

Approach of visual stimuli facilitates the prediction of tactile events and suppresses beta band oscillations around the primary somatosensory area

The purpose of the present study was to investigate whether the approach of visual stimuli influences prediction of subsequent tactile events. For this purpose, we examined electroencephalograms (EEGs) during the prediction of tactile events when visual stimuli did or did not approach. Tactile stimuli were presented with a high probability (80%) of being applied to the left (or right) index finger and a low probability (20%) of being applied to the opposite index finger. In the approach condition, visual stimuli were presented towards the hand to which the high-probability tactile stimuli were presented; in the neutral condition, visual stimuli did not approach. The result of time-frequency analysis for the EEGs showed that beta band event-related spectral perturbation at the electrodes around the primary somatosensory area (C3 and C4) was suppressed about 300 ms before the presentation of a tactile stimulus and that event-related desynchronization (ERD) occurred in all conditions. Moreover, the beta band ERD of the approach condition was larger than that of the neutral condition. These results provide evidence that the approach of visual stimuli facilitates prediction itself for subsequent tactile events.

Fearful faces modulate spatial processing in peripersonal space: An ERP study

Peripersonal space (PPS) represents the region of space surrounding the body. A pivotal function of PPS is to coordinate defensive responses to threat. We have previously shown that a centrally-presented, looming fearful face, signalling a potential threat in one's surroundings, modulates spatial processing by promoting a redirection of sensory resources away from the face towards the periphery, where the threat may be expected - but only when the face is presented in near, rather than far space. Here, we use electrophysiological measures to investigate the neural mechanism underlying this effect. Participants made simple responses to tactile stimuli delivered on the cheeks, while watching task-irrelevant neutral or fearful avatar faces, looming towards them either in near or far space. Simultaneously with the tactile stimulation, a ball with a checkerboard pattern (probe) appeared to the left or right of the avatar face. Crucially, this probe could either be close to the avatar face, and thus more central in the participant's vision, or further away from the avatar face, and thus more peripheral in the participant's vision. Electroencephalography was continuously recorded. Behavioural results confirmed that in near space only, and for fearful relative to neutral faces, tactile processing was facilitated by the peripheral compared to the central probe. This behavioural effect was accompanied by a reduction of the N1 mean amplitude elicited by the peripheral probe for fearful relative to neutral faces. Moreover, the faster the participants responded to tactile stimuli with the peripheral probe, relative to the central, the smaller was their N1. Together these results, suggest that fearful faces intruding into PPS may increase expectation of a visual event occurring in the periphery. This fear-induced effect would enhance the defensive function of PPS when it is most needed, i.e., when the source of threat is nearby, but its location remains unknown.

A multisensory perspective onto primate pulvinar functions

Perception in ambiguous environments relies on the combination of sensory information from various sources. Most associative and primary sensory cortical areas are involved in this multisensory active integration process. As a result, the entire cortex appears as heavily multisensory. In this review, we focus on the contribution of the pulvinar to multisensory integration. This subcortical thalamic nucleus plays a central role in visual detection and selection at a fast time scale, as well as in the regulation of visual processes, at a much slower time scale. However, the pulvinar is also densely connected to cortical areas involved in multisensory integration. In spite of this, little is known about its multisensory properties and its contribution to multisensory perception. Here, we review the anatomical and functional organization of multisensory input to the pulvinar. We describe how visual, auditory, somatosensory, pain, proprioceptive and olfactory projections are differentially organized across the main subdivisions of the pulvinar and we show that topography is central to the organization of this complex nucleus. We propose that the pulvinar combines multiple sources of sensory information to enhance fast responses to the environment, while also playing the role of a general regulation hub for adaptive and flexible cognition.

Receptor-driven, multimodal mapping of cortical areas in the macaque monkey intraparietal sulcus

The intraparietal sulcus (IPS) is structurally and functionally heterogeneous. We performed a quantitative cyto-/myelo- and receptor architectonical analysis to provide a multimodal map of the macaque IPS. We identified 17 cortical areas, including novel areas PEipe, PEipi (external and internal subdivisions of PEip), and MIPd. Multivariate analyses of receptor densities resulted in a grouping of areas based on the degree of (dis)similarity of their receptor architecture: a cluster encompassing areas located in the posterior portion of the IPS and associated mainly with the processing of visual information, a cluster including areas found in the anterior portion of the IPS and involved in sensorimotor processing, and an ‘intermediate’ cluster of multimodal association areas. Thus, differences in cyto-/myelo- and receptor architecture segregate the cortical ribbon within the IPS, and receptor fingerprints provide novel insights into the relationship between the structural and functional segregation of this brain region in the macaque monkey.

Cortical Regions Encoding Hardness Perception Modulated by Visual Information Identified by Functional Magnetic Resonance Imaging With Multivoxel Pattern Analysis

Recent studies have revealed that hardness perception is determined by visual information along with the haptic input. This study investigated the cortical regions involved in hardness perception modulated by visual information using functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis (MVPA). Twenty-two healthy participants were enrolled. They were required to place their left and right hands at the front and back, respectively, of a mirror attached to a platform placed above them while lying in a magnetic resonance scanner. In conditions SFT, MED, and HRD, one of three polyurethane foam pads of varying hardness (soft, medium, and hard, respectively) was presented to the left hand in a given trial, while only the medium pad was presented to the right hand in all trials. MED was defined as the control condition, because the visual and haptic information was congruent. During the scan, the participants were required to push the pad with the both hands while observing the reflection of the left hand and estimate the hardness of the pad perceived by the right (hidden) hand based on magnitude estimation. Behavioral results showed that the perceived hardness was significantly biased toward softer or harder in >73% of the trials in conditions SFT and HRD; we designated these trials as visually modulated (SFTvm and HRDvm, respectively). The accuracy map was calculated individually for each of the pair-wise comparisons of (SFTvm vs. MED), (HRDvm vs. MED), and (SFTvm vs. HRDvm) by a searchlight MVPA, and the cortical regions encoding the perceived hardness with visual modulation were identified by conjunction of the three accuracy maps in group analysis. The cluster was observed in the right sensory motor cortex, left anterior intraparietal sulcus (aIPS), bilateral parietal operculum (PO), and occipito-temporal cortex (OTC). Together with previous findings on such cortical regions, we conclude that the visual information of finger movements processed in the OTC may be integrated with haptic input in the left aIPS, and the subjective hardness perceived by the right hand with visual modulation may be processed in the cortical network between the left PO and aIPS.

Frontier of Self and Impact Prediction

The construction of a coherent representation of our body and the mapping of the space immediately surrounding it are of the highest ecological importance. This space has at least three specificities: it is a space where actions are planned in order to interact with our environment; it is a space that contributes to the experience of self and self-boundaries, through tactile processing and multisensory interactions; last, it is a space that contributes to the experience of body integrity against external events. In the last decades, numerous studies have been interested in peripersonal space (PPS), defined as the space directly surrounding us and which we can interact with (for reviews, see Cléry et al., 2015b; de Vignemont and Iannetti, 2015; di Pellegrino and Làdavas, 2015). These studies have contributed to the understanding of how this space is constructed, encoded and modulated. The majority of these studies focused on subparts of PPS (the hand, the face or the trunk) and very few of them investigated the interaction between PPS subparts. In the present review, we summarize the latest advances in this research and we discuss the new perspectives that are set forth for futures investigations on this topic. We describe the most recent methods used to estimate PPS boundaries by the means of dynamic stimuli. We then highlight how impact prediction and approaching stimuli modulate this space by social, emotional and action-related components involving principally a parieto-frontal network. In a next step, we review evidence that there is not a unique representation of PPS but at least three sub-sections (hand, face and trunk PPS). Last, we discuss how these subspaces interact, and we question whether and how bodily self-consciousness (BSC) is functionally and behaviorally linked to PPS.

Reward activations and face fields in monkey cingulate motor areas

Several premotor areas have been identified within primate cingulate cortex; however their function is yet to be uncovered. Recent brain imaging work in humans revealed a topographic anatomofunctional overlap between feedback processing during exploratory behaviors and the corresponding body fields in the rostral cingulate motor area (RCZa), suggesting an embodied representation of feedback. In particular, a face field in RCZa processes juice feedback. Here we tested an extension of the embodied principle in which unexpected or relevant information obtained through the eye or the face would be processed by face fields in cingulate motor areas, and whether this applied to monkey cingulate cortex. We show that activations for juice reward, eye movement, eye blink, and tactile stimulation on the face overlap over two subfields within the cingulate sulcus likely corresponding to the rostral and caudal cingulate motor areas. This suggests that in monkeys as is the case in humans, behaviorally relevant information is processed through multiple cingulate body/effector maps.

NEW & NOTEWORTHY What is the role of cingulate motor areas? In this study we observed in monkeys that, as in humans, neural responses to face-related events, juice reward, eye movement, eye blink, and tactile stimulations, clustered redundantly in two separate cingulate subfields. This suggests that behaviorally relevant information is processed by multiple cingulate effector maps. Importantly, this overlap supports the principle that the cingulate cortex processes feedback based on where it is experienced on the body.

The mirror mechanism in the parietal lobe

The mirror mechanism is a basic mechanism that transforms sensory representations of others' actions into motor representations of the same actions in the brain of the observer. The mirror mechanism plays an important role in understanding actions of others. In the present chapter we discuss first the basic organization of the posterior parietal lobe in the monkey, stressing that it is best characterized as a motor scaffold, on the top of which sensory information is organized. We then describe the location of the mirror mechanism in the posterior parietal cortex of the monkey, and its functional role in areas PFG, and anterior, ventral, and lateral intraparietal areas. We will then present evidence that a similar functional organization is present in humans. We will conclude by discussing the role of the mirror mechanism in the recognition of action performed with tools.

The Prediction of Impact of a Looming Stimulus onto the Body Is Subserved by Multisensory Integration Mechanisms

In the jungle, survival is highly correlated with the ability to detect and distinguish between an approaching predator and a putative prey. From an ecological perspective, a predator rapidly approaching its prey is a stronger cue for flight than a slowly moving predator. In the present study, we use functional magnetic resonance imaging in the nonhuman primate, to investigate the neural bases of the prediction of an impact to the body by a looming stimulus, i.e., the neural bases of the interaction between a dynamic visual stimulus approaching the body and its expected consequences onto an independent sensory modality, namely, touch. We identify a core cortical network of occipital, parietal, premotor, and prefrontal areas maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus on the faces compared with the activations observed for spatially or temporally incongruent tactile and dynamic visual cues. These activations reflect both an active integration of visual and tactile information and of spatial and temporal prediction information. The identified cortical network coincides with a well described multisensory visuotactile convergence and integration network suggested to play a key role in the definition of peripersonal space. These observations are discussed in the context of multisensory integration and spatial, temporal prediction and Bayesian causal inference.SIGNIFICANCE STATEMENT Looming stimuli have a particular ecological relevance as they are expected to come into contact with the body, evoking touch or pain sensations and possibly triggering an approach or escape behavior depending on their identity. Here, we identify the nonhuman primate functional network that is maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus. Our findings suggest that the integration of spatial and temporal predictive cues possibly rely on the same neural mechanisms that are involved in multisensory integration.

Mapping the complex topological organization of the human parietal face area

The macaque monkey ventral intraparietal area (VIP) contains neurons with aligned visual-tactile receptive fields anchored to the face and upper body. Our previous fMRI studies using standard head coils found a human parietal face area (VIP+ complex; putative macaque VIP homologue) containing superimposed topological maps of the face and near-face visual space. Here, we construct high signal-to-noise surface coils and used phase-encoded air puffs and looming stimuli to map topological organization of the parietal face area at higher resolution. This area is consistently identified as a region extending between the superior postcentral sulcus and the upper bank of the anterior intraparietal sulcus (IPS), avoiding the fundus of IPS. Using smaller voxel sizes, our surface coils picked up strong fMRI signals in response to tactile and visual stimuli. By analyzing tactile and visual maps in our current and previous studies, we constructed a set of topological models illustrating commonalities and differences in map organization across subjects. The most consistent topological feature of the VIP+ complex is a central-anterior upper face (and upper visual field) representation adjoined by lower face (and lower visual field) representations ventrally (laterally) and/or dorsally (medially), potentially forming two subdivisions VIPv (ventral) and VIPd (dorsal). The lower visual field representations typically extend laterally into the anterior IPS to adjoin human area AIP, and medially to overlap with the parietal body areas at the superior parietal ridge. Significant individual variations are then illustrated to provide an accurate and comprehensive view of the topological organization of the parietal face area.

Connectome-scale functional intrinsic connectivity networks in macaques

There have been extensive studies of intrinsic connectivity networks (ICNs) in the human brains using resting-state functional magnetic resonance imaging (fMRI) in the literature. However, the functional organization of ICNs in macaque brains has been less explored so far, despite growing interests in the field. In this work, we propose a computational framework to identify connectome-scale group-wise consistent ICNs in macaques via sparse representation of whole-brain resting-state fMRI data. Experimental results demonstrate that 70 group-wise consistent ICNs are successfully identified in macaque brains via the proposed framework. These 70 ICNs are interpreted based on two publicly available parcellation maps of macaque brains and our work significantly expand currently known macaque ICNs already reported in the literature. In general, this set of connectome-scale group-wise consistent ICNs can potentially benefit a variety of studies in the neuroscience and brain-mapping fields, and they provide a foundation to better understand brain evolution in the future.

Visual Receptive Field Heterogeneity and Functional Connectivity of Adjacent Neurons in Primate Frontoparietal Association Cortices

The basic organization principles of the primary visual cortex (V1) are commonly assumed to also hold in the association cortex such that neurons within a cortical column share functional connectivity patterns and represent the same region of the visual field. We mapped the visual receptive fields (RFs) of neurons recorded at the same electrode in the ventral intraparietal area (VIP) and the lateral prefrontal cortex (PFC) of rhesus monkeys. We report that the spatial characteristics of visual RFs between adjacent neurons differed considerably, with increasing heterogeneity from VIP to PFC. In addition to RF incongruences, we found differential functional connectivity between putative inhibitory interneurons and pyramidal cells in PFC and VIP. These findings suggest that local RF topography vanishes with hierarchical distance from visual cortical input and argue for increasingly modified functional microcircuits in noncanonical association cortices that contrast V1.SIGNIFICANCE STATEMENT Our visual field is thought to be represented faithfully by the early visual brain areas; all the information from a certain region of the visual field is conveyed to neurons situated close together within a functionally defined cortical column. We examined this principle in the association areas, PFC, and ventral intraparietal area of rhesus monkeys and found that adjacent neurons represent markedly different areas of the visual field. This is the first demonstration of such noncanonical organization of these brain areas.

Integration of visual and tactile information in reproduction of traveled distance

In the natural world, self-motion always stimulates several different sensory modalities. Here we investigated the interplay between a visual optic flow stimulus simulating self-motion and a tactile stimulus (air flow resulting from self-motion) while human observers were engaged in a distance reproduction task. We found that adding congruent tactile information (i.e., speed of the air flow and speed of visual motion are directly proportional) to the visual information significantly improves the precision of the actively reproduced distances. This improvement, however, was smaller than predicted for an optimal integration of visual and tactile information. In contrast, incongruent tactile information (i.e., speed of the air flow and speed of visual motion are inversely proportional) did not improve subjects' precision indicating that incongruent tactile information and visual information were not integrated. One possible interpretation of the results is a link to properties of neurons in the ventral intraparietal area that have been shown to have spatially and action-congruent receptive fields for visual and tactile stimuli.NEW & NOTEWORTHY This study shows that tactile and visual information can be integrated to improve the estimates of the parameters of self-motion. This, however, happens only if the two sources of information are congruent-as they are in a natural environment. In contrast, an incongruent tactile stimulus is still used as a source of information about self-motion but it is not integrated with visual information.

Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations

Sensory discriminations, such as judgements about visual motion, often benefit from multisensory evidence. Despite many reports of enhanced brain activity during multisensory conditions, it remains unclear which dynamic processes implement the multisensory benefit for an upcoming decision in the human brain. Specifically, it remains difficult to attribute perceptual benefits to specific processes, such as early sensory encoding, the transformation of sensory representations into a motor response, or to more unspecific processes such as attention. We combined an audio-visual motion discrimination task with the single-trial mapping of dynamic sensory representations in EEG activity to localize when and where multisensory congruency facilitates perceptual accuracy. Our results show that a congruent sound facilitates the encoding of motion direction in occipital sensory - as opposed to parieto-frontal - cortices, and facilitates later - as opposed to early (i.e. below 100 ms) - sensory activations. This multisensory enhancement was visible as an earlier rise of motion-sensitive activity in middle-occipital regions about 350 ms from stimulus onset, which reflected the better discriminability of motion direction from brain activity and correlated with the perceptual benefit provided by congruent multisensory information. This supports a hierarchical model of multisensory integration in which the enhancement of relevant sensory cortical representations is transformed into a more accurate choice.

•

Feature specific multisensory integration occurs in sensory not amodal cortex.

•

Feature specific integration occurs late, i.e. around 350 ms post stimulus onset.

•

Acoustic and visual representations interact in occipital motion regions.

Differential amygdala activation during simulated personal space intrusion by men and women

Responses to personal space (PS) violations are variable and depend (besides many other factors) on the sex of the person who enters this space. The neuronal basis of this effect is still largely unknown. A previous neuroimaging investigation had shown that male participants responded with increased amygdala activation to PS violation, but only when the intruder was male. Gender-specific responses by females have not been studied yet. In the present study we recorded affective as well as hemodynamic responses of 30 women (mean age: M=27.3years; SD=8.1). The participants were exposed to images of neutral facial expressions from men and women. All stimuli were once shown as photos (static), and once were zoomed in (picture enlargement by the factor 2.75) in order to simulate PS intrusion. In both conditions ('static' and 'approaching' faces) the eyes and mouth region of the depicted persons were always completely visible. Approaching faces generally provoked activation of a parietal network (e.g., intraparietal sulcus, superior/inferior parietal cortex). When the approaching person was male additional amygdala activation was detected. Because the amygdala is a central structure for the initiation of defense responses, the heightened activation might reflect that male intrusion was decoded as potential threat. Hence, we observed a similar gender bias to simulated space intrusion in women as previously in men.

Multisensory effects on somatosensation: a trimodal visuo-vestibular-tactile interaction

Vestibular information about self-motion is combined with other sensory signals. Previous research described both visuo-vestibular and vestibular-tactile bilateral interactions, but the simultaneous interaction between all three sensory modalities has not been explored. Here we exploit a previously reported visuo-vestibular integration to investigate multisensory effects on tactile sensitivity in humans. Tactile sensitivity was measured during passive whole body rotations alone or in conjunction with optic flow, creating either purely vestibular or visuo-vestibular sensations of self-motion. Our results demonstrate that tactile sensitivity is modulated by perceived self-motion, as provided by a combined visuo-vestibular percept, and not by the visual and vestibular cues independently. We propose a hierarchical multisensory interaction that underpins somatosensory modulation: visual and vestibular cues are first combined to produce a multisensory self-motion percept. Somatosensory processing is then enhanced according to the degree of perceived self-motion.

Structural and functional correlates of motor imagery BCI performance: Insights from the patterns of fronto-parietal attention network

Motor imagery (MI)-based brain-computer interfaces (BCIs) have been widely used for rehabilitation of motor abilities and prosthesis control for patients with motor impairments. However, MI-BCI performance exhibits a wide variability across subjects, and the underlying neural mechanism remains unclear. Several studies have demonstrated that both the fronto-parietal attention network (FPAN) and MI are involved in high-level cognitive processes that are crucial for the control of BCIs. Therefore, we hypothesized that the FPAN may play an important role in MI-BCI performance. In our study, we recorded multi-modal datasets consisting of MI electroencephalography (EEG) signals, T1-weighted structural and resting-state functional MRI data for each subject. MI-BCI performance was evaluated using the common spatial pattern to extract the MI features from EEG signals. One cortical structural feature (cortical thickness (CT)) and two measurements (degree centrality (DC) and eigenvector centrality (EC)) of node centrality were derived from the structural and functional MRI data, respectively. Based on the information extracted from the EEG and MRI, a correlation analysis was used to elucidate the relationships between the FPAN and MI-BCI performance. Our results show that the DC of the right ventral intraparietal sulcus, the EC and CT of the left inferior parietal lobe, and the CT of the right dorsolateral prefrontal cortex were significantly associated with MI-BCI performance. Moreover, the receiver operating characteristic analysis and machine learning classification revealed that the EC and CT of the left IPL could effectively predict the low-aptitude BCI users from the high-aptitude BCI users with 83.3% accuracy. Those findings consistently reveal that the individuals who have efficient FPAN would perform better on MI-BCI. Our findings may deepen the understanding of individual variability in MI-BCI performance, and also may provide a new biomarker to predict individual MI-BCI performance.

Intertrial Variability in the Premotor Cortex Accounts for Individual Differences in Peripersonal Space

We live in a dynamic environment, constantly confronted with approaching objects that we may either avoid or be forced to address. A multisensory and sensorimotor interface, the peripersonal space (PPS), mediates every physical interaction between our body and the environment. Behavioral investigations show high variability in the extension of PPS across individuals, but there is a lack of evidence on the neural underpinnings of these large individual differences. Here, we used approaching auditory stimuli and fMRI to capture the individual boundary of PPS and examine its neural underpinnings. Precisely, we tested the hypothesis that intertrial variability (ITV) in brain regions coding PPS predicts individual differences of its boundary at the behavioral level. Selectively in the premotor cortex, we found that ITV, rather than trial-averaged amplitude, of BOLD responses to far rather than near dynamic stimuli predicts the individual extension of PPS. Our results provide the first empirical support for the relevance of ITV of brain responses for individual differences in human behavior.

SIGNIFICANCE STATEMENT

Peripersonal space (PPS) is a multisensory and sensorimotor interface mediating every physical interaction between the body and the environment. A major characteristic of the boundary of PPS in humans is the extremely high variability of its location across individuals. We show that interindividual differences in the extension of the PPS are predicted by variability of BOLD responses in the premotor cortex to far stimuli approaching our body. Our results provide the first empirical support to the relevance of variability of evoked responses for human behavior and its variance across individuals.

Looming sensitive cortical regions without V1 input: evidence from a patient with bilateral cortical blindness

Fast and automatic behavioral responses are required to avoid collision with an approaching stimulus. Accordingly, looming stimuli have been found to be highly salient and efficient attractors of attention due to the implication of potential collision and potential threat. Here, we address the question of whether looming motion is processed in the absence of any functional primary visual cortex and consequently without awareness. For this, we investigated a patient (TN) suffering from complete, bilateral damage to his primary visual cortex. Using an fMRI paradigm, we measured TN's brain activation during the presentation of looming, receding, rotating, and static point lights, of which he was unaware. When contrasted with other conditions, looming was found to produce bilateral activation of the middle temporal areas, as well as the superior temporal sulcus and inferior parietal lobe (IPL). The latter are generally thought to be involved in multisensory processing of motion in extrapersonal space, as well as attentional capture and saliency. No activity was found close to the lesioned V1 area. This demonstrates that looming motion is processed in the absence of awareness through direct subcortical projections to areas involved in multisensory processing of motion and saliency that bypass V1.

Tactile representation of the head and shoulders assessed by fMRI in the nonhuman primate

In nonhuman primates, tactile representation at the cortical level has mostly been studied using single-cell recordings targeted to specific cortical areas. In this study, we explored the representation of tactile information delivered to the face or the shoulders at the whole brain level, using functional magnetic resonance imaging (fMRI) in the nonhuman primate. We used air puffs delivered to the center of the face, the periphery of the face, or the shoulders. These stimulations elicited activations in numerous cortical areas, encompassing the primary and secondary somatosensory areas, prefrontal and premotor areas, and parietal, temporal, and cingulate areas as well as low-level visual cortex. Importantly, a specific parieto-temporo-prefrontal network responded to the three stimulations but presented a marked preference for air puffs directed to the center of the face. This network corresponds to areas that are also involved in near-space representation, as well as in the multisensory integration of information at the interface between this near space and the skin of the face, and is probably involved in the construction of a peripersonal space representation around the head.

Coordinates of Human Visual and Inertial Heading Perception

Heading estimation involves both inertial and visual cues. Inertial motion is sensed by the labyrinth, somatic sensation by the body, and optic flow by the retina. Because the eye and head are mobile these stimuli are sensed relative to different reference frames and it remains unclear if a perception occurs in a common reference frame. Recent neurophysiologic evidence has suggested the reference frames remain separate even at higher levels of processing but has not addressed the resulting perception. Seven human subjects experienced a 2s, 16 cm/s translation and/or a visual stimulus corresponding with this translation. For each condition 72 stimuli (360° in 5° increments) were delivered in random order. After each stimulus the subject identified the perceived heading using a mechanical dial. Some trial blocks included interleaved conditions in which the influence of ±28° of gaze and/or head position were examined. The observations were fit using a two degree-of-freedom population vector decoder (PVD) model which considered the relative sensitivity to lateral motion and coordinate system offset. For visual stimuli gaze shifts caused shifts in perceived head estimates in the direction opposite the gaze shift in all subjects. These perceptual shifts averaged 13 ± 2° for eye only gaze shifts and 17 ± 2° for eye-head gaze shifts. This finding indicates visual headings are biased towards retina coordinates. Similar gaze and head direction shifts prior to inertial headings had no significant influence on heading direction. Thus inertial headings are perceived in body-centered coordinates. Combined visual and inertial stimuli yielded intermediate results.

The organization of the posterior parietal cortex devoted to upper limb actions: An fMRI study

The present fMRI study examined whether upper-limb action classes differing in their motor goal are encoded by different PPC sectors. Action observation was used as a proxy for action execution. Subjects viewed actors performing object-related (e.g., grasping), skin-displacing (e.g., rubbing the skin), and interpersonal upper limb actions (e.g., pushing someone). Observation of the three action classes activated a three-level network including occipito-temporal, parietal, and premotor cortex. The parietal region common to observing all three action classes was located dorsally to the left intraparietal sulcus (DIPSM/DIPSA border). Regions specific for observing an action class were obtained by combining the interaction between observing action classes and stimulus types with exclusive masking for observing the other classes, while for regions considered preferentially active for a class the interaction was exclusively masked with the regions common to all observed actions. Left putative human anterior intraparietal was specific for observing manipulative actions, and left parietal operculum including putative human SII region, specific for observing skin-displacing actions. Control experiments demonstrated that this latter activation depended on seeing the skin being moved and not simply on seeing touch. Psychophysiological interactions showed that the two specific parietal regions had similar connectivities. Finally, observing interpersonal actions preferentially activated a dorsal sector of left DIPSA, possibly the homologue of ventral intraparietal coding the impingement of the target person's body into the peripersonal space of the actor. These results support the importance of segregation according to the action class as principle of posterior parietal cortex organization for action observation and by implication for action execution.

Auditory distance coding in rabbit midbrain neurons and human perception: monaural amplitude modulation depth as a cue

Mechanisms underlying sound source distance localization are not well understood. Here we tested the hypothesis that a novel mechanism can create monaural distance sensitivity: a combination of auditory midbrain neurons' sensitivity to amplitude modulation (AM) depth and distance-dependent loss of AM in reverberation. We used virtual auditory space (VAS) methods for sounds at various distances in anechoic and reverberant environments. Stimulus level was constant across distance. With increasing modulation depth, some rabbit inferior colliculus neurons increased firing rates whereas others decreased. These neurons exhibited monotonic relationships between firing rates and distance for monaurally presented noise when two conditions were met: (1) the sound had AM, and (2) the environment was reverberant. The firing rates as a function of distance remained approximately constant without AM in either environment and, in an anechoic condition, even with AM. We corroborated this finding by reproducing the distance sensitivity using a neural model. We also conducted a human psychophysical study using similar methods. Normal-hearing listeners reported perceived distance in response to monaural 1 octave 4 kHz noise source sounds presented at distances of 35-200 cm. We found parallels between the rabbit neural and human responses. In both, sound distance could be discriminated only if the monaural sound in reverberation had AM. These observations support the hypothesis. When other cues are available (e.g., in binaural hearing), how much the auditory system actually uses the AM as a distance cue remains to be determined.

Impact prediction by looming visual stimuli enhances tactile detection

From an ecological point of view, approaching objects are potentially more harmful than receding objects. A predator, a dominant conspecific, or a mere branch coming up at high speed can all be dangerous if one does not detect them and produce the appropriate escape behavior fast enough. And indeed, looming stimuli trigger stereotyped defensive responses in both monkeys and human infants. However, while the heteromodal somatosensory consequences of visual looming stimuli can be fully predicted by their spatiotemporal dynamics, few studies if any have explored whether visual stimuli looming toward the face predictively enhance heteromodal tactile sensitivity around the expected time of impact and at its expected location on the body. In the present study, we report that, in addition to triggering a defensive motor repertoire, looming stimuli toward the face provide the nervous system with predictive cues that enhance tactile sensitivity on the face. Specifically, we describe an enhancement of tactile processes at the expected time and location of impact of the stimulus on the face. We additionally show that a looming stimulus that brushes past the face also enhances tactile sensitivity on the nearby cheek, suggesting that the space close to the face is incorporated into the subjects' body schema. We propose that this cross-modal predictive facilitation involves multisensory convergence areas subserving the representation of a peripersonal space and a safety boundary of self.

Monkey cortex through fMRI glasses

In 1998 several groups reported the feasibility of fMRI experiments in monkeys, with the goal to bridge the gap between invasive nonhuman primate studies and human functional imaging. These studies yielded critical insights in the neuronal underpinnings of the BOLD signal. Furthermore, the technology has been successful in guiding electrophysiological recordings and identifying focal perturbation targets. Finally, invaluable information was obtained concerning human brain evolution. We here provide a comprehensive overview of awake monkey fMRI studies mainly confined to the visual system. We review the latest insights about the topographic organization of monkey visual cortex and discuss the spatial relationships between retinotopy and category- and feature-selective clusters. We briefly discuss the functional layout of parietal and frontal cortex and continue with a summary of some fascinating functional and effective connectivity studies. Finally, we review recent comparative fMRI experiments and speculate about the future of nonhuman primate imaging.

Predictive coding accounts of shared representations in parieto-insular networks

The discovery of mirror neurons in the ventral premotor cortex (area F5) and inferior parietal cortex (area PFG) in the macaque monkey brain has provided the physiological evidence for direct matching of the intrinsic motor representations of the self and the visual image of the actions of others. The existence of mirror neurons implies that the brain has mechanisms reflecting shared self and other action representations. This may further imply that the neural basis self-body representations may also incorporate components that are shared with other-body representations. It is likely that such a mechanism is also involved in predicting other's touch sensations and emotions. However, the neural basis of shared body representations has remained unclear. Here, we propose a neural basis of body representation of the self and of others in both human and non-human primates. We review a series of behavioral and physiological findings which together paint a picture that the systems underlying such shared representations require integration of conscious exteroception and interoception subserved by a cortical sensory-motor network involving parieto-inner perisylvian circuits (the ventral intraparietal area [VIP]/inferior parietal area [PFG]-secondary somatosensory cortex [SII]/posterior insular cortex [pIC]/anterior insular cortex [aIC]). Based on these findings, we propose a computational mechanism of the shared body representation in the predictive coding (PC) framework. Our mechanism proposes that processes emerging from generative models embedded in these specific neuronal circuits play a pivotal role in distinguishing a self-specific body representation from a shared one. The model successfully accounts for normal and abnormal shared body phenomena such as mirror-touch synesthesia and somatoparaphrenia. In addition, it generates a set of testable experimental predictions.

Eye-centered visual receptive fields in the ventral intraparietal area

The ventral intraparietal area (VIP) processes multisensory visual, vestibular, tactile, and auditory signals in diverse reference frames. We recently reported that visual heading signals in VIP are represented in an approximately eye-centered reference frame when measured using large-field optic flow stimuli. No VIP neuron was found to have head-centered visual heading tuning, and only a small proportion of cells had reference frames that were intermediate between eye- and head-centered. In contrast, previous studies using moving bar stimuli have reported that visual receptive fields (RFs) in VIP are head-centered for a substantial proportion of neurons. To examine whether these differences in previous findings might be due to the neuronal property examined (heading tuning vs. RF measurements) or the type of visual stimulus used (full-field optic flow vs. a single moving bar), we have quantitatively mapped visual RFs of VIP neurons using a large-field, multipatch, random-dot motion stimulus. By varying eye position relative to the head, we tested whether visual RFs in VIP are represented in head- or eye-centered reference frames. We found that the vast majority of VIP neurons have eye-centered RFs with only a single neuron classified as head-centered and a small minority classified as intermediate between eye- and head-centered. Our findings suggest that the spatial reference frames of visual responses in VIP may depend on the visual stimulation conditions used to measure RFs and might also be influenced by how attention is allocated during stimulus presentation.

fMRI Cortical Correlates of Spontaneous Eye Blinks in the Nonhuman Primate

Eyeblinks are defined as a rapid closing and opening of the eyelid. Three types of blinks are defined: spontaneous, reflexive, and voluntary. Here, we focus on the cortical correlates of spontaneous blinks, using functional magnetic resonance imaging (fMRI) in the nonhuman primate. Our observations reveal an ensemble of cortical regions processing the somatosensory, proprioceptive, peripheral visual, and possibly nociceptive consequences of blinks. These observations indicate that spontaneous blinks have consequences on the brain beyond the visual cortex, possibly contaminating fMRI protocols that generate in the participants heterogeneous blink behaviors. This is especially the case when these protocols induce (nonunusual) eye fatigue and corneal dryness due to demanding fixation requirements, as is the case here. Importantly, no blink related activations were observed in the prefrontal and parietal blinks motor command areas nor in the prefrontal, parietal, and medial temporal blink suppression areas. This indicates that the absence of activation in these areas is not a signature of the absence of blink contamination in the data. While these observations increase our understanding of the neural bases of spontaneous blinks, they also strongly call for new criteria to identify whether fMRI recordings are contaminated by a heterogeneous blink behavior or not.

Multisensory maps in parietal cortex

•

A new parietal multisensory area integrates lower body and lower visual field.

•

Rearrangement of parietal areas in human and non-human primates is rationalized.

•

In vivo myelin mapping outlines some parietal multisensory areas.

•

Multisensory parietal areas transform visual maps into non-retinocentric coordinates.

Parietal cortex has long been known to be a site of sensorimotor integration. Recent findings in humans have shown that it is divided up into a number of small areas somewhat specialized for eye movements, reaching, and hand movements, but also face-related movements (avoidance, eating), lower body movements, and movements coordinating multiple body parts. The majority of these areas contain rough sensory (receptotopic) maps, including a substantial multisensory representation of the lower body and lower visual field immediately medial to face VIP. There is strong evidence for retinotopic remapping in LIP and face-centered remapping in VIP, and weaker evidence for hand-centered remapping. The larger size of the functionally distinct inferior parietal default mode network in humans compared to monkeys results in a superior and medial displacement of middle parietal areas (e.g., the saccade-related LIP's). Multisensory superior parietal areas located anterior to the angular gyrus such as AIP and VIP are less medially displaced relative to macaque monkeys, so that human LIP paradoxically ends up medial to human VIP.