Tactile and visual motion direction processing in hMT+/V5

Localizing 3D motion through the fingertips: Following in the footsteps of elephants

Each sense serves a different specific function in spatial perception, and they all form a joint multisensory spatial representation. For instance, hearing enables localization in the entire 3D external space, while touch traditionally only allows localization of objects on the body (i.e., within the peripersonal space alone). We use an in-house touch-motion algorithm (TMA) to evaluate individuals' capability to understand externalized 3D information through touch, a skill that was not acquired during an individual's development or in evolution. Four experiments demonstrate quick learning and high accuracy in localization of motion using vibrotactile inputs on fingertips and successful audio-tactile integration in background noise. Subjective responses in some participants imply spatial experiences through visualization and perception of tactile "moving" sources beyond reach. We discuss our findings with respect to developing new skills in an adult brain, including combining a newly acquired "sense" with an existing one and computation-based brain organization.

Discrimination of the moving direction is improved depending on the pattern of the mechanical tactile stimulation intervention

BACKGROUND

The mechanical tactile stimulation, such as plastic pins and airflow-driven membrane, induces cortical activity. The cortical activity depends on the mechanical tactile stimulation pattern. Therefore, the stimulation pattern of mechanical tactile stimuli intervention may influence its effect on the somatosensory function. However, the effect of the mechanical tactile stimulation input pattern on the somatosensory function has not yet been investigated at the behavioral level. The present study aimed to clarify the effects of mechanical tactile stimuli intervention with different stimulation patterns on the ability to discriminate moving directions.

RESULTS

Twenty healthy adults participated in the experiment. Three conditions were used for mechanical tactile stimuli intervention: (1) the whole stimulus surface was stimulated, (2) the stimulus moved within the stimulus surface, and (3) a no-stimulus condition. The effects of mechanical tactile stimuli intervention on tactile discrimination were evaluated using a simple reaction task and a choice reaction task to discriminate the movement direction. Reaction time, correct rate, and rate correct score were calculated to measure task performance. We examined the effects of mechanical tactile stimuli intervention on the ability to discriminate the moving direction for a certain period under three intervention conditions. The results showed that the mean reaction time during the simple reaction task did not differ significantly before and after the intervention under all intervention conditions. Similarly, we compared the data obtained before and after the intervention during the choice reaction task. Our results revealed that the mean reaction time and correct rate did not differ significantly under vertical and horizontal conditions. However, the rate correct score showed a significant improvement after the horizontal moving tactile stimulation intervention under both vertical and horizontal conditions.

CONCLUSIONS

Our results showed that the effect of mechanical tactile stimuli intervention on mechanical tactile stimulation moving direction discrimination function depended on the input pattern of mechanical tactile stimuli intervention. Our results suggest the potential therapeutic benefits of sustained tactile stimulation intervention. This study revealed that it is possible to change behavioral levels via mechanical tactile stimuli intervention as well as the potential of mechanical tactile stimuli intervention in the field of rehabilitation.

Twelve weeks of physical exercise breaks with coordinative exercises at the workplace increase the sulcal depth and decrease gray matter volume in brain structures related to visuomotor processes

Physical exercise can evoke changes in the brain structure. Consequently, these can lead to positive impacts on brain health. However, physical exercise studies including coordinative exercises are rare. Therefore, in this study, we investigated how 12 weeks of physical exercise breaks (PEBs) with coordinative exercises, focusing mainly on juggling tasks, affected the brain structure. The participants were randomly allocated to an intervention group (IG, n = 16; 42.8 ± 10.2 years) and a control group (CG, n = 9; 44.2 ± 12.3 years). The IG performed the PEBs with coordinative exercises twice per week for 15-20 min per session. Before the intervention, after 6 weeks of the intervention, and after 12 weeks of the intervention, participants underwent a high-resolution 3T T1-weighted magnetic resonance imagining scan. Juggling performance was assessed by measuring the time taken to perform a three-ball cascade. A surface-based analysis revealed an increase in vertex-wise cortical depth in a cluster including the inferior parietal lobe after 6 and 12 weeks of training in the IG. After 12 weeks, the IG showed a decrease in gray matter (GM) volume in a cluster primarily involving the right insula and the right operculum. The changes in the GM volume were related to improvements in juggling performance. No significant changes were found for the CG. To conclude, the present study showed that regular engagement in PEBs with coordinative exercises led to changes in brain structures strongly implicated in visuomotor processes involving hand and arm movements.

Dynamic spatial coding in parietal cortex mediates tactile-motor transformation

Movements towards touch on the body require integrating tactile location and body posture information. Tactile processing and movement planning both rely on posterior parietal cortex (PPC) but their interplay is not understood. Here, human participants received tactile stimuli on their crossed and uncrossed feet, dissociating stimulus location relative to anatomy versus external space. Participants pointed to the touch or the equivalent location on the other foot, which dissociates sensory and motor locations. Multi-voxel pattern analysis of concurrently recorded fMRI signals revealed that tactile location was coded anatomically in anterior PPC but spatially in posterior PPC during sensory processing. After movement instructions were specified, PPC exclusively represented the movement goal in space, in regions associated with visuo-motor planning and with regional overlap for sensory, rule-related, and movement coding. Thus, PPC flexibly updates its spatial codes to accommodate rule-based transformation of sensory input to generate movement to environment and own body alike.

Brain structural changes in blindness: a systematic review and an anatomical likelihood estimation (ALE) meta-analysis

In recent decades, numerous structural brain imaging studies investigated purported morphometric changes in early (EB) and late onset blindness (LB). The results of these studies have not yielded very consistent results, neither with respect to the type, nor to the anatomical locations of the brain morphometric alterations. To better characterize the effects of blindness on brain morphometry, we performed a systematic review and an Anatomical-Likelihood-Estimation (ALE) coordinate-based-meta-analysis of 65 eligible studies on brain structural changes in EB and LB, including 890 EB, 466 LB and 1257 sighted controls. Results revealed atrophic changes throughout the whole extent of the retino-geniculo-striate system in both EB and LB, whereas changes in areas beyond the occipital lobe occurred in EB only. We discuss the nature of some of the contradictory findings with respect to the used brain imaging methodologies and characteristics of the blind populations such as the onset, duration and cause of blindness. Future studies should aim for much larger sample sizes, eventually by merging data from different brain imaging centers using the same imaging sequences, opt for multimodal structural brain imaging, and go beyond a purely structural approach by combining functional with structural connectivity network analyses.

Mechanisms of speed encoding in the human middle temporal cortex measured by 7T fMRI

Perception of dynamic scenes in our environment results from the evaluation of visual features such as the fundamental spatial and temporal frequency components of a moving object. The ratio between these two components represents the object's speed of motion. The human middle temporal cortex hMT+ has a crucial biological role in the direct encoding of object speed. However, the link between hMT+ speed encoding and the spatiotemporal frequency components of a moving object is still under explored. Here, we recorded high resolution 7T blood oxygen level-dependent BOLD responses to different visual motion stimuli as a function of their fundamental spatial and temporal frequency components. We fitted each hMT+ BOLD response with a 2D Gaussian model allowing for two different speed encoding mechanisms: (1) distinct and independent selectivity for the spatial and temporal frequencies of the visual motion stimuli; (2) pure tuning for the speed of motion. We show that both mechanisms occur but in different neuronal groups within hMT+, with the largest subregion of the complex showing separable tuning for the spatial and temporal frequency of the visual stimuli. Both mechanisms were highly reproducible within participants, reconciling single cell recordings from MT in animals that have showed both encoding mechanisms. Our findings confirm that a more complex process is involved in the perception of speed than initially thought and suggest that hMT+ plays a primary role in the evaluation of the spatial features of the moving visual input.

Association between somatosensory sensitivity and regional gray matter volume in healthy young volunteers: a voxel-based morphometry study

Two-point discrimination (2PD) test reflects somatosensory spatial discrimination ability, but evidence on the relationship between 2PD and cortical gray matter (GM) volume is limited. This study aimed to analyze the relationship between cortical GM volume and 2PD threshold in young healthy individuals and to clarify the characteristics of brain structure reflecting the individual differences in somatosensory function. 2PD was measured in 42 healthy (20 females) volunteers aged 20-32 years using a custom-made test system that can be controlled by a personal computer. The 2PD of the right index finger measured with this device has been confirmed to show good reproducibility. T1-weighted images were acquired using a 3-T magnetic resonance imaging scanner for voxel-based morphometry analysis. The mean 2PD threshold was 2.58 ± 0.54 mm. Whole-brain multiple regression analysis of the relationship between 2PD and GM volume showed that a lower 2PD threshold (i.e. better somatosensory function) significantly correlated with decreased GM volume from the middle temporal gyrus to the inferior parietal lobule (IPL) in the contralateral hemisphere. In conclusion, a lower GM volume in the middle temporal gyrus and IPL correlates with better somatosensory function. Thus, cortical GM volume may be a biomarker of somatosensory function.

Tactile expectancy modulates occipital alpha oscillations in early blindness

Alpha oscillatory activity is thought to contribute to visual expectancy through the engagement of task-relevant occipital regions. In early blindness, occipital alpha oscillations are systematically reduced, suggesting that occipital alpha depends on visual experience. However, it remains possible that alpha activity could serve expectancy in non-visual modalities in blind people, especially considering that previous research has shown the recruitment of the occipital cortex for non-visual processing. To test this idea, we used electroencephalography to examine whether alpha oscillations reflected a differential recruitment of task-relevant regions between expected and unexpected conditions in two haptic tasks (texture and shape discrimination). As expected, sensor-level analyses showed that alpha suppression in parieto-occipital sites was significantly reduced in early blind individuals compared with sighted participants. The source reconstruction analysis revealed that group differences originated in the middle occipital cortex. In that region, expected trials evoked higher alpha desynchronization than unexpected trials in the early blind group only. Our results support the role of alpha rhythms in the recruitment of occipital areas in early blind participants, and for the first time we show that although posterior alpha activity is reduced in blindness, it remains sensitive to expectancy factors. Our findings therefore suggest that occipital alpha activity is involved in tactile expectancy in blind individuals, serving a similar function to visual anticipation in sighted populations but switched to the tactile modality. Altogether, our results indicate that expectancy-dependent modulation of alpha oscillatory activity does not depend on visual experience. SIGNIFICANCE STATEMENT: Are posterior alpha oscillations and their role in expectancy and anticipation dependent on visual experience? Our results show that tactile expectancy can modulate posterior alpha activity in blind (but not sighted) individuals through the engagement of occipital regions, suggesting that in early blindness, alpha oscillations maintain their proposed role in visual anticipation but subserve tactile processing. Our findings bring a new understanding of the role that alpha oscillatory activity plays in blindness, contrasting with the view that alpha activity is task unspecific in blind populations.

Task-based functional MRI challenges in clinical neuroscience: Choice of the best head motion correction approach in multiple sclerosis

Introduction

Functional MRI (fMRI) is commonly used for understanding brain organization and connectivity abnormalities in neurological conditions, and in particular in multiple sclerosis (MS). However, head motion degrades fMRI data quality and influences all image-derived metrics. Persistent controversies regarding the best correction strategy motivates a systematic comparison, including methods such as scrubbing and volume interpolation, to find optimal correction models, particularly in studies with clinical populations prone to characterize by high motion. Moreover, strategies for correction of motion effects gain more relevance in task-based designs, which are less explored compared to resting-state, have usually lower sample sizes, and may have a crucial role in describing the functioning of the brain and highlighting specific connectivity changes.

Methods

We acquired fMRI data from 17 early MS patients and 14 matched healthy controls (HC) during performance of a visual task, characterized motion in both groups, and quantitatively compared the most used and easy to implement methods for correction of motion effects. We compared task-activation metrics obtained from: (i) models containing 6 or 24 motion parameters (MPs) as nuisance regressors; (ii) models containing nuisance regressors for 6 or 24 MPs and motion outliers (scrubbing) detected with Framewise Displacement or Derivative or root mean square VARiance over voxelS; and (iii) models with 6 or 24 MPs and motion outliers corrected through volume interpolation. To our knowledge, volume interpolation has not been systematically compared with scrubbing, nor investigated in task fMRI clinical studies in MS.

Results

No differences in motion were found between groups, suggesting that recently diagnosed MS patients may not present problematic motion. In general, models with 6 MPs perform better than models with 24 MPs, suggesting the 6 MPs as the best trade-off between correction of motion effects and preservation of valuable information. Parsimonious models with 6 MPs and volume interpolation were the best combination for correcting motion in both groups, surpassing the scrubbing methods. A joint analysis regardless of the group further highlighted the value of volume interpolation.

Discussion

Volume interpolation of motion outliers is an easy to implement technique, which may be an alternative to other methods and may improve the accuracy of fMRI analyses, crucially in clinical studies in MS and other neurological populations.

The human middle temporal cortex responds to both active leg movements and egomotion-compatible visual motion

The human middle-temporal region MT+ is highly specialized in processing visual motion. However, recent studies have shown that this region is modulated by extraretinal signals, suggesting a possible involvement in processing motion information also from non-visual modalities. Here, we used functional MRI data to investigate the influence of retinal and extraretinal signals on MT+ in a large sample of subjects. Moreover, we used resting-state functional MRI to assess how the subdivisions of MT+ (i.e., MST, FST, MT, and V4t) are functionally connected. We first compared responses in MST, FST, MT, and V4t to coherent vs. random visual motion. We found that only MST and FST were positively activated by coherent motion. Furthermore, regional analyses revealed that MST and FST were positively activated by leg, but not arm, movements, while MT and V4t were deactivated by arm, but not leg, movements. Taken together, regional analyses revealed a visuomotor role for the anterior areas MST and FST and a pure visual role for the anterior areas MT and V4t. These findings were mirrored by the pattern of functional connections between these areas and the rest of the brain. Visual and visuomotor regions showed distinct patterns of functional connectivity, with the latter preferentially connected with the somatosensory and motor areas representing leg and foot. Overall, these findings reveal a functional sensitivity for coherent visual motion and lower-limb movements in MST and FST, suggesting their possible involvement in integrating sensory and motor information to perform locomotion.

Enhanced processing of aversive stimuli on embodied artificial limbs by the human amygdala

Body perception has been extensively investigated, with one particular focus being the integration of vision and touch within a neuronal body representation. Previous studies have implicated a distributed network comprising the extrastriate body area (EBA), posterior parietal cortex (PPC) and ventral premotor cortex (PMv) during illusory self-attribution of a rubber hand. Here, we set up an fMRI paradigm in virtual reality (VR) to study whether and how the self-attribution of (artificial) body parts is altered if these body parts are somehow threatened. Participants (N = 30) saw a spider (aversive stimulus) or a toy-car (neutral stimulus) moving along a 3D-rendered virtual forearm positioned like their real forearm, while tactile stimulation was applied on the real arm in the same (congruent) or opposite (incongruent) direction. We found that the PPC was more activated during congruent stimulation; higher visual areas and the anterior insula (aIns) showed increased activation during aversive stimulus presentation; and the amygdala was more strongly activated for aversive stimuli when there was stronger multisensory integration of body-related information (interaction of aversiveness and congruency). Together, these findings suggest an enhanced processing of aversive stimuli within the amygdala when they represent a bodily threat.

Direction-selective modulation of visual motion rivalry by collocated tactile motion

Early models of multisensory integration posited that cross-modal signals only converged in higher-order association cortices and that vision automatically dominates. However, recent studies have challenged this view. In this study, the significance of the alignment of motion axes and spatial alignment across visual and tactile stimuli, as well as the effect of hand visibility on visuo-tactile interactions were examined. Using binocular rivalry, opposed motions were presented to each eye and participants were required to track the perceived visual direction. A tactile motion that was either a leftward or rightward sweep across the fingerpad was intermittently presented. Results showed that tactile effects on visual percepts were dependent on the alignment of motion axes: rivalry between up/down visual motions was not modulated at all by left/right tactile motion. On the other hand, visual percepts could be altered by tactile motion signals when both modalities shared a common axis of motion: a tactile stimulus could maintain the dominance duration of a congruent visual stimulus and shorten its suppression period. The effects were also conditional on the spatial alignment of the visual and tactile stimuli, being eliminated when the tactile device was displaced 15 cm away to the right of the visual stimulus. In contrast, visibility of the hand touching the tactile stimulus facilitated congruent switches relative to a visual-only baseline but did not present a significant advantage overall. In sum, these results show a low-level sensory interaction that is conditional on visual and tactile stimuli sharing a common motion axis and location in space.

A new patterned air-flow device to reveal the network for tactile motion coding using fMRI

BACKGROUND

Studying brain processes underlying tactile perception induced by natural-like stimulation is challenging yet crucial to closely match real-world situations.

NEW METHOD

We developed a computer-controlled pneumatic device that allows the delivery of complex airflow patterns on subject's body, through a MR-compatible system fixed on an independent clippable mounting device. The intensity of stimulation as well as the timing of each of the four air channels are completely programmable and independent, allowing the precise control and modularity of the airflow delivery.

RESULTS

An analysis of signal-to-noise ratio (SNR) measurements did not show any impact of the PAF device on anatomical or functional scan acquisitions. A psychophysical experiment was also performed on 24 volunteers to evaluate the perception of different airflow patterns delivered over the lower part of their face. It revealed that all participants were able to finely discriminate the direction of these leftward to rightward flow motions. The fMRI experiment, which consisted in presenting to 20 participants four different airflow patterns, shed light on the brain network associated with tactile motion perception. A multivariate analysis further showed a specific coding of the different patterns inside this tactile brain network including the primary and secondary somatosensory cortex COMPARISON WITH EXISTING METHOD(S): The Patterned Air-Flow (PAF) is an easy-to-set-up, portable, adaptable device, which can be spatially and temporally modulated to provide complex tactile stimuli.

CONCLUSIONS

This device will be useful to further explore complex dynamic touch exerted over various body parts and can also be combined with visual or auditory stimulation to study multisensory mechanisms.

Cross-modal motion aftereffects transfer between vision and touch in early deaf adults

Previous research on early deafness has primarily focused on the behavioral and neural changes in the intact visual and tactile modalities. However, how early deafness changes the interplay of these two modalities is not well understood. In the current study, we investigated the effect of auditory deprivation on visuo-tactile interaction by measuring the cross-modal motion aftereffect. Consistent with previous findings, motion aftereffect transferred between vision and touch in a bidirectional manner in hearing participants. However, for deaf participants, the cross-modal transfer occurred only in the tactile-to-visual direction but not in the visual-to-tactile direction. This unidirectional cross-modal motion aftereffect found in the deaf participants could not be explained by unisensory motion aftereffect or discrimination threshold. The results suggest a reduced visual influence on tactile motion perception in early deaf individuals.

Neuronal Encoding of Multisensory Motion Features in the Rat Associative Parietal Cortex

Motion perception is facilitated by the interplay of various sensory channels. In rodents, the cortical areas involved in multisensory motion coding remain to be identified. Using voltage-sensitive-dye imaging, we revealed a visuo-tactile convergent region that anatomically corresponds to the associative parietal cortex (APC). Single unit responses to moving visual gratings or whiskers deflections revealed a specific coding of motion characteristics strikingly found in both sensory modalities. The heteromodality of this region was further supported by a large proportion of bimodal neurons and by a classification procedure revealing that APC carries information about motion features, sensory origin and multisensory direction-congruency. Altogether, the results point to a central role of APC in multisensory integration for motion perception.

Electrocorticography Evidence of Tactile Responses in Visual Cortices

There is ongoing debate regarding the extent to which human cortices are specialized for processing a given sensory input versus a given type of information, independently of the sensory source. Many neuroimaging and electrophysiological studies have reported that primary and extrastriate visual cortices respond to tactile and auditory stimulation, in addition to visual inputs, suggesting these cortices are intrinsically multisensory. In particular for tactile responses, few studies have proven neuronal processes in visual cortex in humans. Here, we assessed tactile responses in both low-level and extrastriate visual cortices using electrocorticography recordings in a human participant. Specifically, we observed significant spectral power increases in the high frequency band (30-100 Hz) in response to tactile stimuli, reportedly associated with spiking neuronal activity, in both low-level visual cortex (i.e. V2) and in the anterior part of the lateral occipital-temporal cortex. These sites were both involved in processing tactile information and responsive to visual stimulation. More generally, the present results add to a mounting literature in support of task-sensitive and sensory-independent mechanisms underlying functions like spatial, motion, and self-processing in the brain and extending from higher-level as well as to low-level cortices.

Neuronal signatures of a random-dot motion comparison task

The study of perceptual decision making has made significant progress owing to major contributions from two experimental paradigms: the sequential vibrotactile frequency comparison task for the somatosensory domain requiring working memory, and the random-dot motion task in the visual domain requiring evidence accumulation over time. On the one hand, electrophysiological recordings in nonhuman primates and humans have identified changes in firing rates and power modulations of beta band oscillations with the vibrotactile frequencies held in working memory, as well as with the mental operation required for decision making. On the other hand, firing rates and centro-parietal potentials were found to increase to a fixed level at the time of responding during the random-dot motion task, possibly reflecting an underlying evidence accumulation mechanism until a decision threshold is met. Here, to bridge these two paradigms, we presented two visual random-dot motion stimuli in a sequential comparison task while recording EEG from human volunteers. We identified a modulation of prefrontal beta band power that scaled with the level of dot motion coherence of the first stimulus during a short retention interval. Furthermore, beta power in premotor areas was modulated by participants' choices approximately 700 ms before responses were given via button press. At the same time, dot motion patches of the second stimulus evoked a pattern of broadband centro-parietal signal build-up till responses were made, whose peak varied with trial difficulty. Hence, we show that known modulations of beta power during working memory and decision making extend from the vibrotactile to the visual domain and provide support for the notion of evidence accumulation as an unconfined decision-making mechanism generalizing over distinct decision types.

fNIRS Responses in Professional Violinists While Playing Duets: Evidence for Distinct Leader and Follower Roles at the Brain Level

Music played in ensembles is a naturalistic model to study joint action and leader-follower relationships. Recently, the investigation of the brain underpinnings of joint musical actions has gained attention; however, the cerebral correlates underlying the roles of leader and follower in music performance remain elusive. The present study addressed this question by simultaneously measuring the hemodynamic correlates of functional neural activity elicited during naturalistic violin duet performance using fNIRS. Findings revealed distinct patterns of functional brain activation when musicians played the Violin 2 (follower) than the Violin 1 part (leader) in duets, both compared to solo performance. More specifically, results indicated that musicians playing the Violin 2 part had greater oxy-Hb activation in temporo-parietal (p = 0.02) and somatomotor (p = 0.04) regions during the duo condition in relation to the solo. On the other hand, there were no significant differences in the activation of these areas between duo/solo conditions during the execution of the Violin 1 part (p's > 0.05). These findings suggest that ensemble cohesion during a musical performance may impose particular demands when musicians play the follower position, especially in brain areas associated with the processing of dynamic social information and motor simulation. This study is the first to use fNIRS hyperscanning technology to simultaneously measure the brain activity of two musicians during naturalistic music ensemble performance, opening new avenues for the investigation of brain correlates underlying joint musical actions with multiple subjects in a naturalistic environment.

The circuit architecture of cortical multisensory processing: Distinct functions jointly operating within a common anatomical network

Our perceptual systems continuously process sensory inputs from different modalities and organize these streams of information such that our subjective representation of the outside world is a unified experience. By doing so, they also enable further cognitive processing and behavioral action. While cortical multisensory processing has been extensively investigated in terms of psychophysics and mesoscale neural correlates, an in depth understanding of the underlying circuit-level mechanisms is lacking. Previous studies on circuit-level mechanisms of multisensory processing have predominantly focused on cue integration, i.e. the mechanism by which sensory features from different modalities are combined to yield more reliable stimulus estimates than those obtained by using single sensory modalities. In this review, we expand the framework on the circuit-level mechanisms of cortical multisensory processing by highlighting that multisensory processing is a family of functions - rather than a single operation - which involves not only the integration but also the segregation of modalities. In addition, multisensory processing not only depends on stimulus features, but also on cognitive resources, such as attention and memory, as well as behavioral context, to determine the behavioral outcome. We focus on rodent models as a powerful instrument to study the circuit-level bases of multisensory processes, because they enable combining cell-type-specific recording and interventional techniques with complex behavioral paradigms. We conclude that distinct multisensory processes share overlapping anatomical substrates, are implemented by diverse neuronal micro-circuitries that operate in parallel, and are flexibly recruited based on factors such as stimulus features and behavioral constraints.

Cross-decoding supramodal information in the human brain

Perceptual decision making is the cognitive process wherein the brain classifies stimuli into abstract categories for more efficient downstream processing. A system that, during categorization, can process information regardless of the information's original sensory modality (i.e., a supramodal system) would have a substantial advantage over a system with dedicated processes for specific sensory modalities. While many studies have probed decision processes through the lens of one sensory modality, it remains unclear whether there are such supramodal brain areas that can flexibly process task-relevant information regardless of the original "format" of the information. To investigate supramodality, one must ensure that supramodal information exists somewhere within the functional architecture by rendering information from multiple sensory systems necessary but insufficient for categorization. To this aim, we tasked participants with categorizing auditory and tactile frequency-modulated sweeps according to learned, supramodal categories in a delayed match-to-category paradigm while we measured their blood-oxygen-level dependent signal with functional MRI. To detect supramodal information, we implemented a set of cross-modality pattern classification analyses, which demonstrated that the left caudate nucleus encodes category-level information but not stimulus-specific information (such as spatial directions and stimulus modalities), while the right inferior frontal gyrus, showing the opposite pattern, encodes stimulus-specific information but not category-level information. Given our paradigm, these results reveal abstract representations in the brain that are independent of motor, semantic, and sensory-specific processing, instead reflecting supramodal, categorical information, which points to the caudate nucleus as a locus of cognitive processes involved in complex behavior.

Connectivity of the ventral visual cortex is necessary for object recognition in patients

The functional profiles of regions in the ventral occipital-temporal cortex (VTC), a critical region for object visual recognition, are associated with the VTC connectivity patterns to nonvisual regions relevant to the corresponding object domain. However, whether and how whole-brain connections affect recognition behavior remains untested. We directly examined the necessity of VTC connectivity in object recognition behavior by testing 82 patients whose lesion spared relevant VTC regions but affected various white matter (WM) tracts and other regions. In these patients, we extracted the whole-brain anatomical connections of two VTC domain-selective (large manmade objects and animals) clusters with probabilistic tractography, and examined whether such connectivity pattern can predict recognition performance of the corresponding domains with support vector regression (SVR) analysis. We found that the whole-brain anatomical connectivity of large manmade object-specific cluster successfully predicted patients' large object recognition performance but not animal recognition or control tasks, even after we excluded connections with early visual regions. The contributing connections to large object recognition included tracts between VTC-large object cluster and distributed regions both within and beyond the visual cortex (e.g., putamen, superior, and middle temporal gyrus). These results provide causal evidence that the VTC whole-brain anatomical connectivity is necessary for at least certain domains of object recognition behavior.

Structural changes in hand related cortical areas after median nerve injury and repair

Transection of the median nerve typically causes lifelong restriction of fine sensory and motor skills of the affected hand despite the best available surgical treatment. Inspired by recent findings on activity-dependent structural plasticity of the adult brain, we used voxel-based morphometry to analyze the brains of 16 right-handed adults who more than two years earlier had suffered injury to the left or right median nerve followed by microsurgical repair. Healthy individuals served as matched controls. Irrespective of side of injury, we observed gray matter reductions in left ventral and right dorsal premotor cortex, and white matter reductions in commissural pathways interconnecting those motor areas. Only left-side injured participants showed gray matter reduction in the hand area of the contralesional primary motor cortex. We interpret these effects as structural manifestations of reduced neural processing linked to restrictions in the diversity of the natural manual dexterity repertoire. Furthermore, irrespective of side of injury, we observed gray matter increases bilaterally in a motion-processing visual area. We interpret this finding as a consequence of increased neural processing linked to greater dependence on vision for control of manual dexterity after median nerve injury because of a compromised somatosensory innervation of the affected hand.

Trial by trial dependencies in multisensory perception and their correlates in dynamic brain activity

A well-known effect in multisensory perception is that congruent information received by different senses usually leads to faster and more accurate responses. Less well understood are trial-by-trial interactions, whereby the multisensory composition of stimuli experienced during previous trials shapes performance during a subsequent trial. We here exploit the analogy of multisensory paradigms with classical flanker tasks to investigate the neural correlates underlying trial-by-trial interactions of multisensory congruency. Studying an audio-visual motion task, we demonstrate that congruency benefits for accuracy and reaction times are reduced following an audio-visual incongruent compared to a congruent preceding trial. Using single trial analysis of motion-sensitive EEG components we then localize current-trial and serial interaction effects within distinct brain regions: while the multisensory congruency experienced during the current trial influences the encoding of task-relevant information in sensory-specific brain regions, the serial interaction arises from task-relevant processes within the inferior frontal lobe. These results highlight parallels between multisensory paradigms and classical flanker tasks and demonstrate a role of amodal association cortices in shaping perception based on the history of multisensory congruency.

Taking two to tango: fMRI analysis of improvised joint action with physical contact

Many forms of joint action involve physical coupling between the participants, such as when moving a sofa together or dancing a tango. We report the results of a novel two-person functional MRI study in which trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, and in which the two alternated between being the leader and the follower of joint improvised movements. Leading showed a general pattern of self-orientation, being associated with brain areas involved in motor planning, navigation, sequencing, action monitoring, and error correction. In contrast, following showed a far more sensory, externally-oriented pattern, revealing areas involved in somatosensation, proprioception, motion tracking, social cognition, and outcome monitoring. We also had participants perform a "mutual" condition in which the movement patterns were pre-learned and the roles were symmetric, thereby minimizing any tendency toward either leading or following. The mutual condition showed greater activity in brain areas involved in mentalizing and social reward than did leading or following. Finally, the analysis of improvisation revealed the dual importance of motor-planning and working-memory areas. We discuss these results in terms of theories of both joint action and improvisation.

Following during physically-coupled joint action engages motion area MT+/V5

Interpersonal coordination during joint action depends on the perception of the partner's movements. In many such situations - for example, while moving furniture together or dancing a tango - there are kinesthetic interactions between the partners due to the forces shared between them that allow them to directly perceive one another's movements. Joint action of this type often involves a contrast between the roles of leader and follower, where the leader imparts forces onto the follower, and the follower has to be responsive to these force-cues during movement. We carried out a novel 2-person functional MRI study with trained couple dancers engaged in bimanual contact with an experimenter standing next to the bore of the magnet, where the two alternated between being the leader and follower of joint improvised movements, all with the eyes closed. One brain area that was unexpectedly more active during following than leading was the region of MT+/V5. While classically described as an area for processing visual motion, it has more recently been shown to be responsive to tactile motion as well. We suggest that MT+/V5 responds to motion based on force-cues during joint haptic interaction, most especially when a follower responds to force-cues coming from a leader's movements.

Integration of visual and tactile information in reproduction of traveled distance

In the natural world, self-motion always stimulates several different sensory modalities. Here we investigated the interplay between a visual optic flow stimulus simulating self-motion and a tactile stimulus (air flow resulting from self-motion) while human observers were engaged in a distance reproduction task. We found that adding congruent tactile information (i.e., speed of the air flow and speed of visual motion are directly proportional) to the visual information significantly improves the precision of the actively reproduced distances. This improvement, however, was smaller than predicted for an optimal integration of visual and tactile information. In contrast, incongruent tactile information (i.e., speed of the air flow and speed of visual motion are inversely proportional) did not improve subjects' precision indicating that incongruent tactile information and visual information were not integrated. One possible interpretation of the results is a link to properties of neurons in the ventral intraparietal area that have been shown to have spatially and action-congruent receptive fields for visual and tactile stimuli.NEW & NOTEWORTHY This study shows that tactile and visual information can be integrated to improve the estimates of the parameters of self-motion. This, however, happens only if the two sources of information are congruent-as they are in a natural environment. In contrast, an incongruent tactile stimulus is still used as a source of information about self-motion but it is not integrated with visual information.

Brain networks involved in tactile speed classification of moving dot patterns: the effects of speed and dot periodicity

Humans are able to judge the speed of an object’s motion by touch. Research has suggested that tactile judgment of speed is influenced by physical properties of the moving object, though the neural mechanisms underlying this process remain poorly understood. In the present study, functional magnetic resonance imaging was used to investigate brain networks that may be involved in tactile speed classification and how such networks may be affected by an object’s texture. Participants were asked to classify the speed of 2-D raised dot patterns passing under their right middle finger. Activity in the parietal operculum, insula, and inferior and superior frontal gyri was positively related to the motion speed of dot patterns. Activity in the postcentral gyrus and superior parietal lobule was sensitive to dot periodicity. Psycho-physiological interaction (PPI) analysis revealed that dot periodicity modulated functional connectivity between the parietal operculum (related to speed) and postcentral gyrus (related to dot periodicity). These results suggest that texture-sensitive activity in the primary somatosensory cortex and superior parietal lobule influences brain networks associated with tactually-extracted motion speed. Such effects may be related to the influence of surface texture on tactile speed judgment.

Visual area V5/hMT+ contributes to perception of tactile motion direction: a TMS study

Human imaging studies have reported activations associated with tactile motion perception in visual motion area V5/hMT+, primary somatosensory cortex (SI) and posterior parietal cortex (PPC; Brodmann areas 7/40). However, such studies cannot establish whether these areas are causally involved in tactile motion perception. We delivered double-pulse transcranial magnetic stimulation (TMS) while moving a single tactile point across the fingertip, and used signal detection theory to quantify perceptual sensitivity to motion direction. TMS over both SI and V5/hMT+, but not the PPC site, significantly reduced tactile direction discrimination. Our results show that V5/hMT+ plays a causal role in tactile direction processing, and strengthen the case for V5/hMT+ serving multimodal motion perception. Further, our findings are consistent with a serial model of cortical tactile processing, in which higher-order perceptual processing depends upon information received from SI. By contrast, our results do not provide clear evidence that the PPC site we targeted (Brodmann areas 7/40) contributes to tactile direction perception.

Sounds facilitate visual motion discrimination via the enhancement of late occipital visual representations

Sensory discriminations, such as judgements about visual motion, often benefit from multisensory evidence. Despite many reports of enhanced brain activity during multisensory conditions, it remains unclear which dynamic processes implement the multisensory benefit for an upcoming decision in the human brain. Specifically, it remains difficult to attribute perceptual benefits to specific processes, such as early sensory encoding, the transformation of sensory representations into a motor response, or to more unspecific processes such as attention. We combined an audio-visual motion discrimination task with the single-trial mapping of dynamic sensory representations in EEG activity to localize when and where multisensory congruency facilitates perceptual accuracy. Our results show that a congruent sound facilitates the encoding of motion direction in occipital sensory - as opposed to parieto-frontal - cortices, and facilitates later - as opposed to early (i.e. below 100 ms) - sensory activations. This multisensory enhancement was visible as an earlier rise of motion-sensitive activity in middle-occipital regions about 350 ms from stimulus onset, which reflected the better discriminability of motion direction from brain activity and correlated with the perceptual benefit provided by congruent multisensory information. This supports a hierarchical model of multisensory integration in which the enhancement of relevant sensory cortical representations is transformed into a more accurate choice.

•

Feature specific multisensory integration occurs in sensory not amodal cortex.

•

Feature specific integration occurs late, i.e. around 350 ms post stimulus onset.

•

Acoustic and visual representations interact in occipital motion regions.

Tactile Apparent Motion on the Torso Modulates Perceived Forward Self-Motion Velocity

The present study investigated whether a tactile flow created by a matrix of vibrators in a seat pan simultaneously presented with an optical flow in peripheral vision enhances the perceived forward velocity of self-motion. A brief tactile motion stimulus consisted of four successive rows of vibration, and the interstimulus onset between the tactile rows was varied to change the velocity of the tactile motion. The results show that the forward velocity of self-motion is significantly overestimated for rapid tactile flows and underestimated for slow ones, compared with optical flow alone or non-motion vibrotactile stimulation conditions. In addition, the effect with a temporal tactile rhythm without changing the stimulus location was smaller than that with spatiotemporal tactile motion, with the interstimulus onset interval to elicit a clear sensation of tactile apparent motion. These findings suggest that spatiotemporal tactile motion is effective in inducing a change in the perceived forward velocity of self-motion.

Analysis of haptic information in the cerebral cortex

Haptic sensing of objects acquires information about a number of properties. This review summarizes current understanding about how these properties are processed in the cerebral cortex of macaques and humans. Nonnoxious somatosensory inputs, after initial processing in primary somatosensory cortex, are partially segregated into different pathways. A ventrally directed pathway carries information about surface texture into parietal opercular cortex and thence to medial occipital cortex. A dorsally directed pathway transmits information regarding the location of features on objects to the intraparietal sulcus and frontal eye fields. Shape processing occurs mainly in the intraparietal sulcus and lateral occipital complex, while orientation processing is distributed across primary somatosensory cortex, the parietal operculum, the anterior intraparietal sulcus, and a parieto-occipital region. For each of these properties, the respective areas outside primary somatosensory cortex also process corresponding visual information and are thus multisensory. Consistent with the distributed neural processing of haptic object properties, tactile spatial acuity depends on interaction between bottom-up tactile inputs and top-down attentional signals in a distributed neural network. Future work should clarify the roles of the various brain regions and how they interact at the network level.

Early Blindness Results in Developmental Plasticity for Auditory Motion Processing within Auditory and Occipital Cortex

Early blind subjects exhibit superior abilities for processing auditory motion, which are accompanied by enhanced BOLD responses to auditory motion within hMT+ and reduced responses within right planum temporale (rPT). Here, by comparing BOLD responses to auditory motion in hMT+ and rPT within sighted controls, early blind, late blind, and sight-recovery individuals, we were able to separately examine the effects of developmental and adult visual deprivation on cortical plasticity within these two areas. We find that both the enhanced auditory motion responses in hMT+ and the reduced functionality in rPT are driven by the absence of visual experience early in life; neither loss nor recovery of vision later in life had a discernable influence on plasticity within these areas. Cortical plasticity as a result of blindness has generally be presumed to be mediated by competition across modalities within a given cortical region. The reduced functionality within rPT as a result of early visual loss implicates an additional mechanism for cross modal plasticity as a result of early blindness—competition across different cortical areas for functional role.

Dissecting neural circuits for multisensory integration and crossmodal processing

We rely on rich and complex sensory information to perceive and understand our environment. Our multisensory experience of the world depends on the brain's remarkable ability to combine signals across sensory systems. Behavioural, neurophysiological and neuroimaging experiments have established principles of multisensory integration and candidate neural mechanisms. Here we review how targeted manipulation of neural activity using invasive and non-invasive neuromodulation techniques have advanced our understanding of multisensory processing. Neuromodulation studies have provided detailed characterizations of brain networks causally involved in multisensory integration. Despite substantial progress, important questions regarding multisensory networks remain unanswered. Critically, experimental approaches will need to be combined with theory in order to understand how distributed activity across multisensory networks collectively supports perception.

Diminished Quality of Life and Increased Brain Functional Connectivity in Patients with Hypothyroidism After Total Thyroidectomy

BACKGROUND

Acute hypothyroidism induced by thyroid hormone withdrawal (THW) in patients with thyroid cancer after total thyroidectomy can affect mood and quality of life (QoL). While loss or dysregulation of thyroid hormone (TH) has these well-known behavioral consequences, the effects of TH alterations on brain function are not well understood. Resting state functional connectivity (FC) measured by functional magnetic resonance imaging (fMRI) allows non-invasive evaluation of human brain function. This study therefore examined whether THW affects resting state FC and whether changes in FC correlate with the mood or QoL of the patients with THW status.

METHODS

Twenty-one patients who had undergone total thyroidectomy for thyroid cancer were recruited. Resting state fMRI scanning of the brain, thyroid function tests, and administration of the 12-Item Short Form Health Survey (SF-12) and the Patient Health Questionnaire-9 (PHQ-9) were performed before and after two weeks of THW. Regional homogeneity (ReHo), one of the measures of resting state FC, was calculated, and each voxel was compared between before and after THW in 19 patients. The ReHo values were extracted from the regions of interest showing within-group differences in ReHo values after THW, and correlations of ReHo values with thyrotropin (TSH) levels, total score of the PHQ-9, and composite scores of the SF-12 were statistically evaluated.

RESULTS

Higher ReHo was observed after THW in the brain cortical regions across primary motor and sensory, visual, and association cortices. Among the regions, the ReHo values in the bilateral pre- and postcentral gyri, bilateral middle occipito-temporal cortices, the left precuneus, and the left lingual gyrus showed positive correlations with serum TSH levels after THW. Higher ReHo values in the bilateral pre- and postcentral gyri, the left middle temporo-occipital cortices, and the left ligual gyrus correlated with the lower mental component summary score from the SF-12, while higher ReHo values in the bilateral pre- and postcentral gyri correlated with higher total scores in the PHQ-9.

CONCLUSIONS

Local brain FC is increased in the acute hypothyroid state. Higher FC correlates with a poorer mental QoL and increased depression in the hypothyroid state.

Auditory motion in the sighted and blind: Early visual deprivation triggers a large-scale imbalance between auditory and "visual" brain regions

How early blindness reorganizes the brain circuitry that supports auditory motion processing remains controversial. We used fMRI to characterize brain responses to in-depth, laterally moving, and static sounds in early blind and sighted individuals. Whole-brain univariate analyses revealed that the right posterior middle temporal gyrus and superior occipital gyrus selectively responded to both in-depth and laterally moving sounds only in the blind. These regions overlapped with regions selective for visual motion (hMT+/V5 and V3A) that were independently localized in the sighted. In the early blind, the right planum temporale showed enhanced functional connectivity with right occipito-temporal regions during auditory motion processing and a concomitant reduced functional connectivity with parietal and frontal regions. Whole-brain searchlight multivariate analyses demonstrated higher auditory motion decoding in the right posterior middle temporal gyrus in the blind compared to the sighted, while decoding accuracy was enhanced in the auditory cortex bilaterally in the sighted compared to the blind. Analyses targeting individually defined visual area hMT+/V5 however indicated that auditory motion information could be reliably decoded within this area even in the sighted group. Taken together, the present findings demonstrate that early visual deprivation triggers a large-scale imbalance between auditory and "visual" brain regions that typically support the processing of motion information.

Four-dimensional maps of the human somatosensory system

A fine-grained description of the spatiotemporal dynamics of human brain activity is a major goal of neuroscientific research. Limitations in spatial and temporal resolution of available noninvasive recording and imaging techniques have hindered so far the acquisition of precise, comprehensive four-dimensional maps of human neural activity. The present study combines anatomical and functional data from intracerebral recordings of nearly 100 patients, to generate highly resolved four-dimensional maps of human cortical processing of nonpainful somatosensory stimuli. These maps indicate that the human somatosensory system devoted to the hand encompasses a widespread network covering more than 10% of the cortical surface of both hemispheres. This network includes phasic components, centered on primary somatosensory cortex and neighboring motor, premotor, and inferior parietal regions, and tonic components, centered on opercular and insular areas, and involving human parietal rostroventral area and ventral medial-superior-temporal area. The technique described opens new avenues for investigating the neural basis of all levels of cortical processing in humans.

Progressive Thinning of Visual Motion Area in Lower Limb Amputees

Accumulating evidence has indicated that amputation or deafferentation of a limb induces functional or structural reorganization in the visual areas. However, the extent of the visual areas involved after lower limb amputation remains uncertain. In this investigation, we studied 48 adult patients with unilateral lower limb amputation and 48 matched healthy controls using T1-weighted magnetic resonance imaging. Template-based regions of interest analysis was implemented to detect the changes of cortical thickness in the specific visual areas. Compared with normal controls, amputees exhibited significantly lower thickness in the V5/middle temporal (V5/MT+) visual area, as well as a trend of cortical thinning in the V3d. There was no significant difference in the other visual areas between the two groups. In addition, no significant difference of cortical thickness was found between patients with amputation at different levels. Across all amputees, correlation analyses revealed that the cortical thickness of the V5/MT+ was negatively correlated to the time since amputation. In conclusion, our findings indicate that the amputation of unilateral lower limb could induce changes in the motor-related visual cortex and provide an update on the plasticity of the human brain after limb injury.

Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review

Research regarding crossmodal interactions has garnered much interest in the last few decades. A variety of studies have demonstrated that multisensory information (vision, audition, tactile sensation, and so on) can perceptually interact with each other in the spatial and temporal domains. Findings regarding crossmodal interactions in the spatiotemporal domain (i.e., motion processing) have also been reported, with updates in the last few years. In this review, we summarize past and recent findings on spatiotemporal processing in crossmodal interactions regarding perception of the external world. A traditional view regarding crossmodal interactions holds that vision is superior to audition in spatial processing, but audition is dominant over vision in temporal processing. Similarly, vision is considered to have dominant effects over the other sensory modalities (i.e., visual capture) in spatiotemporal processing. However, recent findings demonstrate that sound could have a driving effect on visual motion perception. Moreover, studies regarding perceptual associative learning reported that, after association is established between a sound sequence without spatial information and visual motion information, the sound sequence could trigger visual motion perception. Other sensory information, such as motor action or smell, has also exhibited similar driving effects on visual motion perception. Additionally, recent brain imaging studies demonstrate that similar activation patterns could be observed in several brain areas, including the motion processing areas, between spatiotemporal information from different sensory modalities. Based on these findings, we suggest that multimodal information could mutually interact in spatiotemporal processing in the percept of the external world and that common perceptual and neural underlying mechanisms would exist for spatiotemporal processing.

Re-examining overlap between tactile and visual motion responses within hMT+ and STS

Here, we examine overlap between tactile and visual motion BOLD responses within the human MT+ complex. Although several studies have reported tactile responses overlapping with hMT+, many used group average analyses, leaving it unclear whether these responses were restricted to subregions of hMT+. Moreover, previous studies either employed a tactile task or passive stimulation, leaving it unclear whether or not tactile responses in hMT+ are simply the consequence of visual imagery. Here, we carried out a replication of one of the classic papers finding tactile responses in hMT+. We mapped MT and MST in individual subjects using visual field localizers. We then examined responses to tactile motion on the arm, either presented passively or in the presence of a visual task performed at fixation designed to minimize visualization of the concurrent tactile stimulation. To our surprise, without a visual task, we found only weak tactile motion responses in MT (6% of voxels showing tactile responses) and MST (2% of voxels). With an unrelated visual task designed to withdraw attention from the tactile modality, responses in MST were reduced to almost nothing (<1% regions). Consistent with previous results, we did observe tactile responses in STS regions superior and anterior to hMT+. Despite the lack of individual overlap, group-averaged responses produced strong spurious overlap between tactile and visual motion responses within hMT+ that resembled those observed in previous studies. The weak nature of tactile responses in hMT+ (and their abolition by withdrawal of attention) suggests that hMT+ may not serve as a supramodal motion processing module.

The lateral occipitotemporal cortex in action

Understanding and responding to other people's actions is fundamental for social interactions. Whereas many studies emphasize the importance of parietal and frontal regions for these abilities, several lines of recent research show that the human lateral occipitotemporal cortex (LOTC) represents varied aspects of action, ranging from perception of tools and bodies and the way they typically move, to understanding the meaning of actions, to performing overt actions. Here, we highlight common themes across these lines of work, which have informed theories related to high-level vision, concepts, social cognition, and apraxia. We propose that patterns of activity in LOTC form representational spaces, the dimensions of which capture perceptual, semantic, and motor knowledge of how actions change the state of the world.

The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data

The multivariate analysis of brain signals has recently sparked a great amount of interest, yet accessible and versatile tools to carry out decoding analyses are scarce. Here we introduce The Decoding Toolbox (TDT) which represents a user-friendly, powerful and flexible package for multivariate analysis of functional brain imaging data. TDT is written in Matlab and equipped with an interface to the widely used brain data analysis package SPM. The toolbox allows running fast whole-brain analyses, region-of-interest analyses and searchlight analyses, using machine learning classifiers, pattern correlation analysis, or representational similarity analysis. It offers automatic creation and visualization of diverse cross-validation schemes, feature scaling, nested parameter selection, a variety of feature selection methods, multiclass capabilities, and pattern reconstruction from classifier weights. While basic users can implement a generic analysis in one line of code, advanced users can extend the toolbox to their needs or exploit the structure to combine it with external high-performance classification toolboxes. The toolbox comes with an example data set which can be used to try out the various analysis methods. Taken together, TDT offers a promising option for researchers who want to employ multivariate analyses of brain activity patterns.

Auditory motion processing after early blindness

Studies showing that occipital cortex responds to auditory and tactile stimuli after early blindness are often interpreted as demonstrating that early blind subjects "see" auditory and tactile stimuli. However, it is not clear whether these occipital responses directly mediate the perception of auditory/tactile stimuli, or simply modulate or augment responses within other sensory areas. We used fMRI pattern classification to categorize the perceived direction of motion for both coherent and ambiguous auditory motion stimuli. In sighted individuals, perceived motion direction was accurately categorized based on neural responses within the planum temporale (PT) and right lateral occipital cortex (LOC). Within early blind individuals, auditory motion decisions for both stimuli were successfully categorized from responses within the human middle temporal complex (hMT+), but not the PT or right LOC. These findings suggest that early blind responses within hMT+ are associated with the perception of auditory motion, and that these responses in hMT+ may usurp some of the functions of nondeprived PT. Thus, our results provide further evidence that blind individuals do indeed "see" auditory motion.

Decoding pattern motion information in V1

When two gratings drifting in different directions are superimposed, the resulting stimulus can be perceived as two overlapping component gratings moving in different directions or as a single pattern moving in one direction. Whilst the motion direction of component gratings is already represented in visual area V1, the majority of previous studies have found processing of pattern motion direction only from visual area V2 onwards. Here, we question these findings using multi-voxel pattern analysis (MVPA). In Experiment 1, we presented superimposed sinusoidal gratings with varying angles between the two component motions. These stimuli were perceived as patterns moving in one of two possible directions. We found that linear support vector machines (SVMs) could generalise across stimuli composed of different component motions to successfully discriminate pattern motion direction from brain activity in V1, V3A and hMT+/V5. This demonstrates the representation of pattern motion information present in these visual areas. This conclusion was verified in Experiment 2, where we manipulated similar plaid stimuli to induce the perception of either a single moving pattern or two separate component gratings. While a classifier could again generalise across stimuli composed of different component motions when they were perceived as a single moving pattern, its performance dropped substantially in the case where components were perceived. Our results indicate that pattern motion direction information is present in V1.