Crossmodal influences in somatosensory cortex: Interaction of vision and touch

Neural Encoding of Active Multi-Sensing Enhances Perceptual Decision-Making via a Synergistic Cross-Modal Interaction

Most perceptual decisions rely on the active acquisition of evidence from the environment involving stimulation from multiple senses. However, our understanding of the neural mechanisms underlying this process is limited. Crucially, it remains elusive how different sensory representations interact in the formation of perceptual decisions. To answer these questions, we used an active sensing paradigm coupled with neuroimaging, multivariate analysis, and computational modeling to probe how the human brain processes multisensory information to make perceptual judgments. Participants of both sexes actively sensed to discriminate two texture stimuli using visual (V) or haptic (H) information or the two sensory cues together (VH). Crucially, information acquisition was under the participants' control, who could choose where to sample information from and for how long on each trial. To understand the neural underpinnings of this process, we first characterized where and when active sensory experience (movement patterns) is encoded in human brain activity (EEG) in the three sensory conditions. Then, to offer a neurocomputational account of active multisensory decision formation, we used these neural representations of active sensing to inform a drift diffusion model of decision-making behavior. This revealed a multisensory enhancement of the neural representation of active sensing, which led to faster and more accurate multisensory decisions. We then dissected the interactions between the V, H, and VH representations using a novel information-theoretic methodology. Ultimately, we identified a synergistic neural interaction between the two unisensory (V, H) representations over contralateral somatosensory and motor locations that predicted multisensory (VH) decision-making performance.SIGNIFICANCE STATEMENT In real-world settings, perceptual decisions are made during active behaviors, such as crossing the road on a rainy night, and include information from different senses (e.g., car lights, slippery ground). Critically, it remains largely unknown how sensory evidence is combined and translated into perceptual decisions in such active scenarios. Here we address this knowledge gap. First, we show that the simultaneous exploration of information across senses (multi-sensing) enhances the neural encoding of active sensing movements. Second, the neural representation of active sensing modulates the evidence available for decision; and importantly, multi-sensing yields faster evidence accumulation. Finally, we identify a cross-modal interaction in the human brain that correlates with multisensory performance, constituting a putative neural mechanism for forging active multisensory perception.

Neuroplasticity and Crossmodal Connectivity in the Normal, Healthy Brain

Objective

Neuroplasticity enables the brain to establish new crossmodal connections or reorganize old connections which are essential to perceiving a multisensorial world. The intent of this review is to identify and summarize the current developments in neuroplasticity and crossmodal connectivity, and deepen understanding of how crossmodal connectivity develops in the normal, healthy brain, highlighting novel perspectives about the principles that guide this connectivity.

Methods

To the above end, a narrative review is carried out. The data documented in prior relevant studies in neuroscience, psychology and other related fields available in a wide range of prominent electronic databases are critically assessed, synthesized, interpreted with qualitative rather than quantitative elements, and linked together to form new propositions and hypotheses about neuroplasticity and crossmodal connectivity.

Results

Three major themes are identified. First, it appears that neuroplasticity operates by following eight fundamental principles and crossmodal integration operates by following three principles. Second, two different forms of crossmodal connectivity, namely direct crossmodal connectivity and indirect crossmodal connectivity, are suggested to operate in both unisensory and multisensory perception. Third, three principles possibly guide the development of crossmodal connectivity into adulthood. These are labeled as the principle of innate crossmodality, the principle of evolution-driven 'neuromodular' reorganization and the principle of multimodal experience. These principles are combined to develop a three-factor interaction model of crossmodal connectivity.

Conclusions

The hypothesized principles and the proposed model together advance understanding of neuroplasticity, the nature of crossmodal connectivity, and how such connectivity develops in the normal, healthy brain.

Neural Basis of Anticipatory Multisensory Integration

The brain is able to gather different sensory information to enhance salient event perception, thus yielding a unified perceptual experience of multisensory events. Multisensory integration has been widely studied, and the literature supports the hypothesis that it can occur across various stages of stimulus processing, including both bottom-up and top-down control. However, evidence on anticipatory multisensory integration occurring in the fore period preceding the presentation of the expected stimulus in passive tasks, is missing. By means of event-related potentials (ERPs), it has been recently proposed that visual and auditory unimodal stimulations are preceded by sensory-specific readiness activities. Accordingly, in the present study, we tested the occurrence of multisensory integration in the endogenous anticipatory phase of sensory processing, combining visual and auditory stimuli during unimodal and multimodal passive ERP paradigms. Results showed that the modality-specific pre-stimulus ERP components (i.e., the auditory positivity -aP- and the visual negativity -vN-) started earlier and were larger in the multimodal stimulation compared with the sum of the ERPs elicited by the unimodal stimulations. The same amplitude effect was also present for the early auditory N1 and visual P1 components. This anticipatory multisensory effect seems to influence stimulus processing, boosting the magnitude of early stimulus processing. This paves the way for new perspectives on the neural basis of multisensory integration.

Lower visual field preference for the visuomotor control of limb movements in the human dorsomedial parietal cortex

Visual cues coming from the lower visual field (VF) play an important role in the visual guidance of upper and lower limb movements. A recently described region situated in the dorsomedial parietal cortex, area hPEc (Pitzalis et al. in NeuroImage 202:116092, 2019), might have a role in integrating visually derived information with somatomotor signals to guide limb interaction with the environment. In macaque, it has been demonstrated that PEc receives visual information mostly from the lower visual field but, to date, there has been no systematic investigation of VF preference in the newly defined human homologue of macaque area PEc (hPEc). Here we examined the VF preferences of hPEc while participants performed a visuomotor task implying spatially directed delayed eye-, hand- and foot-movements towards different spatial locations within the VF. By analyzing data as a function of the different target locations towards which upcoming movements were planned (and then executed), we observed the presence of asymmetry in the vertical dimension of VF in area hPEc, being this area more strongly activated by limb movements directed towards visual targets located in the lower compared to the upper VF. This result confirms the view, first advanced in macaque monkey, that PEc is involved in processing visual information to guide body interaction with the external environment, including locomotion. We also observed a contralateral dominance for the lower VF preference in the foot selective somatomotor cortex anterior to hPEc. This result might reflect the role of this cortex (which includes areas PE and S-I) in providing highly topographically organized signals, likely useful to achieve an appropriate foot posture during locomotion.

Interactions between cardiac activity and conscious somatosensory perception

Fluctuations in the heart's activity can modulate the access of external stimuli to consciousness. The link between perceptual awareness and cardiac signals has been investigated mainly in the visual and auditory domain. Here, we investigated whether the phase of the cardiac cycle and the prestimulus heart rate influence conscious somatosensory perception. We also tested how conscious detection of somatosensory stimuli affects the heart rate. Electrocardiograms (ECG) of 33 healthy volunteers were recorded while applying near-threshold electrical pulses at a fixed intensity to the left index finger. Conscious detection was not uniformly distributed across the cardiac cycle but significantly higher in diastole than in systole. We found no evidence that the heart rate before a stimulus influenced its detection, but hits (correctly detected somatosensory stimuli) led to a more pronounced cardiac deceleration than misses. Our findings demonstrate interactions between cardiac activity and conscious somatosensory perception, which highlights the importance of internal bodily states for sensory processing beyond the auditory and visual domain.

Visually induced analgesia during face or limb stimulation in healthy and migraine subjects

Background

Visually induced analgesia (VIA) defines a phenomenon in which viewing one’s own body part during its painful stimulation decreases the perception of pain. VIA occurs during direct vision of the stimulated body part and also when seeing it reflected in a mirror. To the best of our knowledge, VIA has not been studied in the trigeminal area, where it could be relevant for the control of headache.

Subjects and methods

We used heat stimuli (53°C) to induce pain in the right forehead or wrist in 11 healthy subjects (HSs) and 14 female migraine without aura (MO) patients between attacks. The subjects rated pain on a visual analog scale (VAS) and underwent contact heat-evoked potential (CHEP) recordings (five sequential blocks of four responses) with or without observation of their face/wrist in a mirror.

Results

During wrist stimulation, amplitude of the first block of P1–P2 components of CHEPs decreased compared to that in the control recording when HSs were seeing their wrist reflected in the mirror (p = 0.036; Z = 2.08); however, this was not found in MO patients. In the latter, the VAS pain score increased viewing the reflected wrist (p = 0.049; Z = 1.96). Seeing their forehead reflected in the mirror induced a significant increase in N2 latency of CHEPs in HSs, as well as an amplitude reduction in the first block of P1–P2 components of CHEPs both in HSs (p = 0.007; Z = 2.69) and MO patients (p = 0.035; Z = 2.10). Visualizing the body part did not modify habituation of CHEP amplitudes over the five blocks of averaged responses, neither during wrist nor during forehead stimulation.

Conclusion

This study adds to the available knowledge on VIA and demonstrates this phenomenon for painful stimuli in the trigeminal area, as long as CHEPs are used as indices of central pain processing. In migraine patients during interictal periods, VIA assessed with CHEPs is within normal limits in the face but absent at the wrist, possibly reflecting dysfunctioning of extracephalic pain control.

Phantom Acupuncture Induces Placebo Credibility and Vicarious Sensations: A Parallel fMRI Study of Low Back Pain Patients

Although acupuncture is an effective therapeutic intervention for pain reduction, the exact difference between real and sham acupuncture has not been clearly understood because a somatosensory tactile component is commonly included in the existing sham acupuncture protocols. In an event-related fMRI experiment, we implemented a novel form of sham acupuncture, phantom acupuncture, that reproduces the acupuncture needling procedure without somatosensory tactile stimulation while maintaining the credibility of the acupuncture treatment context. Fifty-six non-specific low back pain patients received either real (REAL) or phantom (PHNT) acupuncture stimulation in a parallel group study. The REAL group exhibited greater activation in the posterior insula and anterior cingulate cortex, reflecting the needling-specific components of acupuncture. We demonstrated that PHNT could be delivered credibly. Interestingly, the PHNT-credible group exhibited bilateral activation in SI/SII and also reported vicarious acupuncture sensations without needling stimulation. The PHNT group showed greater activation in the bilateral dorsolateral/ventrolateral prefrontal cortex (dlPFC/vlPFC). Moreover, the PHNT group exhibited significant pain reduction, with a significant correlation between the subjective fMRI signal in the right dlPFC/vlPFC and a score assessing belief in acupuncture effectiveness. These results support an expectation-related placebo analgesic effect on subjective pain intensity ratings, possibly mediated by right prefrontal cortex activity.

Alice in Wonderland Syndrome: A Clinical and Pathophysiological Review

Alice in Wonderland Syndrome (AIWS) is a perceptual disorder, principally involving visual and somesthetic integration, firstly reported by Todd, on the literary suggestion of the strange experiences described by Lewis Carroll in Alice in Wonderland books. Symptoms may comprise among others aschematia and dysmetropsia. This syndrome has many different etiologies; however EBV infection is the most common cause in children, while migraine affects more commonly adults. Many data support a strict relationship between migraine and AIWS, which could be considered in many patients as an aura or a migraine equivalent, particularly in children. Nevertheless, AIWS seems to have anatomical correlates. According to neuroimaging, temporoparietal-occipital carrefour (TPO-C) is a key region for developing many of AIWS symptoms. The final part of this review aims to find the relationship between AIWS symptoms, presenting a pathophysiological model. In brief, AIWS symptoms depend on an alteration of TPO-C where visual-spatial and somatosensory information are integrated. Alterations in these brain regions may cause the cooccurrence of dysmetropsia and disorders of body schema. In our opinion, the association of other symptoms reported in literature could vary depending on different etiologies and the lack of clear diagnostic criteria.

A multisensory perspective of working memory

Although our sensory experience is mostly multisensory in nature, research on working memory representations has focused mainly on examining the senses in isolation. Results from the multisensory processing literature make it clear that the senses interact on a more intimate manner than previously assumed. These interactions raise questions regarding the manner in which multisensory information is maintained in working memory. We discuss the current status of research on multisensory processing and the implications of these findings on our theoretical understanding of working memory. To do so, we focus on reviewing working memory research conducted from a multisensory perspective, and discuss the relation between working memory, attention, and multisensory processing in the context of the predictive coding framework. We argue that a multisensory approach to the study of working memory is indispensable to achieve a realistic understanding of how working memory processes maintain and manipulate information.

Early modality-specific somatosensory cortical regions are modulated by attended visual stimuli: interaction of vision, touch and behavioral intent

Bimodal interactions between relevant visual and tactile inputs can facilitate attentional modulation at early stages in somatosensory cortices to achieve goal-oriented behaviors. However, the specific contribution of each sensory system during attentional processing and, importantly, how these interact with the required behavioral motor goals remains unclear. Here we used electroencephalography and event-related potentials (ERPs) to test the hypothesis that activity from modality-specific somatosensory cortical regions would be enhanced with task-relevant bimodal (visual-tactile) stimuli and that the degree of modulation would depend on the difficulty of the associated sensory-motor task demands. Tactile stimuli were discrete vibrations to the index finger and visual stimuli were horizontal bars on a computer screen, both with random amplitudes. Streams of unimodal (tactile) and crossmodal (visual and tactile) stimuli were randomly presented and participants were instructed to attend to one type of stimulus (unimodal or crossmodal) and responses involved either an indication of the presence of an attended stimulus (detect), or the integration and summation of two stimulus amplitudes using a pressure-sensitive ball (grade). Force-amplitude associations were learned in a training session, and no feedback was provided during the task. ERPs were time-locked to tactile stimuli and extracted for early modality-specific components (P50, P100, N140). The P50 was enhanced with bimodal (visual-tactile) stimuli that were attended to. This was maximal when the motor requirements involved integration of the two stimuli in the grade task and when the visual stimulus occurred before (100 ms) the tactile stimulus. These results suggest that visual information relevant for movement modulates somatosensory processing as early as the primary somatosensory cortex (S1) and that the motor behavioral context influences this likely through interaction of top-down attentional and motor preparatory systems with more bottom-up crossmodal influences.

The attentional-relevance and temporal dynamics of visual-tactile crossmodal interactions differentially influence early stages of somatosensory processing

BACKGROUND

Crossmodal interactions between relevant visual and tactile inputs can enhance attentional modulation at early stages in somatosensory cortices to achieve goal-oriented behaviors. However, the specific contribution of each sensory system during attentional processing remains unclear. We used EEG to investigate the effects of visual priming and attentional relevance in modulating somatosensory cortical responses.

METHODS

Healthy adults performed a sensory integration task that required scaled motor responses dependent on the amplitudes of tactile and visual stimuli. Participants completed an attentional paradigm comprised of 5 conditions that presented sequential or concurrent pairs of discrete stimuli with random amplitude variations: 1) tactile-tactile (TT), 2) visual-visual (VV), 3) visual-tactile simultaneous (SIM), 4) tactile-visual delay (TVd), and 5) visual-tactile delay (VTd), each with a 100 ms temporal delay between stimulus onsets. Attention was directed to crossmodal conditions and graded motor responses representing the summation of the 2 stimulus amplitudes were made.

RESULTS

Results of somatosensory ERPs showed that the modality-specific components (P50, P100) were sensitive to i) the temporal dynamics of crossmodal interactions, and ii) the relevance of these sensory signals for behaviour.

CONCLUSION

Notably, the P50 amplitude was greatest in the VTd condition, suggesting that presentation of relevant visual information for upcoming movement modulates somatosensory processing in modality-specific cortical regions, as early as the primary somatosensory cortex (SI).

Crossmodal influences on early somatosensory processing: interaction of vision, touch, and task-relevance

Previous research suggests that somatosensory cortex is subject to modulation based on the relevancy of incoming somatosensory stimuli to behavioural goals. Recent fMRI findings provide evidence for modulation of primary somatosensory cortex when simultaneous visual and tactile stimuli were relevant to the performance of a motor task. The present study aimed to (1) determine the temporal characteristics of this modulation using event-related potentials (ERPs) and (2) investigate the role of task-relevance in mediating such a modulation. Electroencephalography was collected from healthy subjects during visual, vibrotactile or bimodal stimulation as they performed a sensory-guided motor task. Experiment 1 tested the hypothesis that simultaneous bimodal stimuli would be associated with modulation of somatosensory ERPs, and Experiment 2 tested the hypothesis that such effects would only be seen when both modalities are relevant. ERPs were time-locked to stimulus onset, and mean ERP amplitudes and latencies were extracted for the P50, P100, and N140. The bimodal condition in the first experiment was associated with larger amplitudes at both early and mid-latency components. The manipulation of task-relevance under bimodal conditions produced more complex results for the mid-latency components. For the P50, this enhancement was observed only when both stimuli were relevant, whereas the P100 was smallest when the tactile stimuli were not relevant to the response. These results provide evidence that crossmodal stimuli can modulate early somatosensory event-related potentials and that these effects are mediated by stimulus relevance.

The functional architecture of S1 during touch observation described with 7 T fMRI

Recent studies indicate that the primary somatosensory cortex (S1) is active not only when touch is physically perceived but also when it is merely observed to be experienced by another person. This social responsivity of S1 has important implications for our understanding of S1 functioning. However, S1 activity during touch observation has not been characterized in great detail to date. We focused on two features of the S1 functional architecture during touch observation, namely the topographical arrangement of index and middle finger receptive fields (RFs), and their dynamic shrinkage during concurrent activation. Both features have important implications for human behavior. We conducted two fMRI studies at 7 T, one where touch was physically perceived, and one where touch was observed. In the two experiments, participants either had their index finger and/or middle finger stimulated using paintbrushes, or just observed similar touch events on video. Our data show that observing and physically experiencing touch elicits overlapping activity changes in S1. In addition, observing touch to the index finger or the middle finger alone evoked topographically arranged activation foci in S1. Importantly, when co-activated, the index and middle finger RFs not only shrank during physical touch perception, but also during touch observation. Our data, therefore, indicate a similarity between the functional architecture of S1 during touch observation and physical touch perception with respect to single-digit topography and RF shrinkage. These results may allow the tentative conclusion that even primary somatosensory experiences, such as physical touch perception, can be shared amongst individuals.

Effects of sensory behavioral tasks on pain threshold and cortical excitability

Background/Objective

Transcutaneous electrical stimulation has been proven to modulate nervous system activity, leading to changes in pain perception, via the peripheral sensory system, in a bottom up approach. We tested whether different sensory behavioral tasks induce significant effects in pain processing and whether these changes correlate with cortical plasticity.

Methodology/Principal Findings

This randomized parallel designed experiment included forty healthy right-handed males. Three different somatosensory tasks, including learning tasks with and without visual feedback and simple somatosensory input, were tested on pressure pain threshold and motor cortex excitability using transcranial magnetic stimulation (TMS). Sensory tasks induced hand-specific pain modulation effects. They increased pain thresholds of the left hand (which was the target to the sensory tasks) and decreased them in the right hand. TMS showed that somatosensory input decreased cortical excitability, as indexed by reduced MEP amplitudes and increased SICI. Although somatosensory tasks similarly altered pain thresholds and cortical excitability, there was no significant correlation between these variables and only the visual feedback task showed significant somatosensory learning.

Conclusions/Significance

Lack of correlation between cortical excitability and pain thresholds and lack of differential effects across tasks, but significant changes in pain thresholds suggest that analgesic effects of somatosensory tasks are not primarily associated with motor cortical neural mechanisms, thus, suggesting that subcortical neural circuits and/or spinal cord are involved with the observed effects. Identifying the neural mechanisms of somatosensory stimulation on pain may open novel possibilities for combining different targeted therapies for pain control.

Linking pain and the body: neural correlates of visually induced analgesia

The visual context of seeing the body can reduce the experience of acute pain, producing a multisensory analgesia. Here we investigated the neural correlates of this "visually induced analgesia" using fMRI. We induced acute pain with an infrared laser while human participants looked either at their stimulated right hand or at another object. Behavioral results confirmed the expected analgesic effect of seeing the body, while fMRI results revealed an associated reduction of laser-induced activity in ipsilateral primary somatosensory cortex (SI) and contralateral operculoinsular cortex during the visual context of seeing the body. We further identified two known cortical networks activated by sensory stimulation: (1) a set of brain areas consistently activated by painful stimuli (the so-called "pain matrix"), and (2) an extensive set of posterior brain areas activated by the visual perception of the body ("visual body network"). Connectivity analyses via psychophysiological interactions revealed that the visual context of seeing the body increased effective connectivity (i.e., functional coupling) between posterior parietal nodes of the visual body network and the purported pain matrix. Increased connectivity with these posterior parietal nodes was seen for several pain-related regions, including somatosensory area SII, anterior and posterior insula, and anterior cingulate cortex. These findings suggest that visually induced analgesia does not involve an overall reduction of the cortical response elicited by laser stimulation, but is consequent to the interplay between the brain's pain network and a posterior network for body perception, resulting in modulation of the experience of pain.

The interplay of cue modality and response latency in brain areas supporting crossmodal motor preparation: an event-related fMRI study

Crossmodal (auditory, visual) motor facilitation can be defined as a cue in one sensory modality eliciting speeded responses to targets in a different sensory modality. We used event-related functional magnetic resonance imaging (fMRI) to isolate brain activity underlying crossmodal motor preparation. Our predictions were that interactions between input modality and processes underlying response selection would be indexed by distinct spatiotemporal brain dynamics. A crossmodal response selection task was designed in which a central, nonspatial cue indicated the response rule (compatible or incompatible) to a lateralized target. Cues and targets appeared in auditory and visual modalities and were separated by a lengthy delay period in which cue-related brain activity could be dissociated. We found faster reaction times to auditory compared with visual cues. Next, we correlated brain activity with behavioural performance using multivariate spatiotemporal partial least squares. We identified a distinct, significant brain-behaviour pattern in which faster reaction times to auditory cues were correlated with higher blood oxygenation level-dependent percent signal change in medial visual, frontoparietal (inferior parietal lobule, superior frontal gyrus and premotor cortex) and subcortical (thalamus and cerebellum) areas. For visual cues, quicker responses were linked to greater activity in the same frontoparietal and subcortical but not medial visual areas. Our results show that both modality-dependent and modality-independent brain areas with different brain-behaviour relationships are implicated in crossmodal motor preparation.

Modulation of the response to a somatosensory stimulation of the hand during the observation of manual actions

Observation of hand movements has been repeatedly demonstrated to increase the excitability of the motor cortical representation of the hand. Little attention, however, has been devoted to its effect on somatosensory processing. Movement execution is well known to decrease somatosensory cortical excitability, a phenomenon termed 'gating'. As executed and observed actions share common cortical representations, we hypothesized that action observation (hand movements) should also modulate the cortical response to sensory stimulation of the hand. Seventeen healthy subjects participated in these experiments in which electroencephalographic (EEG) recordings of the somatosensory steady-state response (SSSR) were obtained. The SSSR provides a continuous measure of somatosensory processing. Recordings were made during a baseline condition and five observation conditions in which videos showed either a: (1) hand action; (2) passive stimulation of a hand; (3) static hand; (4) foot action; or (5) static object. The method employed consisted of applying a continuous 25 Hz vibratory stimulation to the index finger during the six conditions and measuring potential gating effects in the SSSR within the 25 Hz band (corresponding to the stimulation frequency). A significant effect of condition was found over the contralateral parietal cortex. Observation of hand actions resulted in a significant gating effect when compared to baseline (average gating of 22%). Observation of passive touch of the hand also gated the response (17% decrease). In conclusion, the results show that viewing a hand performing an action or being touched interferes with the processing of somatosensory information arising from the hand.