Predictions in the light of your own action repertoire as a general computational principle

EEG-based texture roughness classification in active tactile exploration with invariant representation learning networks

During daily activities, humans use their hands to grasp surrounding objects and perceive sensory information which are also employed for perceptual and motor goals. Multiple cortical brain regions are known to be responsible for sensory recognition, perception and motor execution during sensorimotor processing. While various research studies particularly focus on the domain of human sensorimotor control, the relation and processing between motor execution and sensory processing is not yet fully understood. Main goal of our work is to discriminate textured surfaces varying in their roughness levels during active tactile exploration using simultaneously recorded electroencephalogram (EEG) data, while minimizing the variance of distinct motor exploration movement patterns. We perform an experimental study with eight healthy participants who were instructed to use the tip of their dominant hand index finger while rubbing or tapping three different textured surfaces with varying levels of roughness. We use an adversarial invariant representation learning neural network architecture that performs EEG-based classification of different textured surfaces, while simultaneously minimizing the discriminability of motor movement conditions (i.e., rub or tap). Results show that the proposed approach can discriminate between three different textured surfaces with accuracies up to 70%, while suppressing movement related variability from learned representations.

Hierarchical dynamics of informational patterns and decision-making

Traditional studies on the interaction of cognitive functions in healthy and disordered brains have used the analyses of the connectivity of several specialized brain networks-the functional connectome. However, emerging evidence suggests that both brain networks and functional spontaneous brain-wide network communication are intrinsically dynamic. In the light of studies investigating the cooperation between different cognitive functions, we consider here the dynamics of hierarchical networks in cognitive space. We show, using an example of behavioural decision-making based on sequential episodic memory, how the description of metastable pattern dynamics underlying basic cognitive processes helps to understand and predict complex processes like sequential episodic memory recall and competition among decision strategies. The mathematical images of the discussed phenomena in the phase space of the corresponding cognitive model are hierarchical heteroclinic networks. One of the most important features of such networks is the robustness of their dynamics. Different kinds of instabilities of these dynamics can be related to 'dynamical signatures' of creativity and different psychiatric disorders. The suggested approach can also be useful for the understanding of the dynamical processes that are the basis of consciousness.

EEG correlates of sensorimotor processing: independent components involved in sensory and motor processing

Sensorimotor processing is a critical function of the human brain with multiple cortical areas specialised for sensory recognition or motor execution. Although there has been considerable research into sensorimotor control in humans, the steps between sensory recognition and motor execution are not fully understood. To provide insight into brain areas responsible for sensorimotor computation, we used complex categorization-response tasks (variations of a Stroop task requiring recognition, decision-making, and motor responses) to test the hypothesis that some functional modules are participating in both sensory as well as motor processing. We operationalize functional modules as independent components (ICs) yielded by an independent component analysis (ICA) of EEG data and measured event-related responses by means of inter-trial coherence (ITC). Our results consistently found ICs with event-related ITC responses related to both sensory stimulation and motor response onsets (on average 5.8 ICs per session). These findings reveal EEG correlates of tightly coupled sensorimotor processing in the human brain, and support frameworks like embodied cognition, common coding, and sensorimotor contingency that do not sequentially separate sensory and motor brain processes.

Connectivity patterns during music listening: Evidence for action-based processing in musicians

Musical expertise is visible both in the morphology and functionality of the brain. Recent research indicates that functional integration between multi-sensory, somato-motor, default-mode (DMN), and salience (SN) networks of the brain differentiates musicians from non-musicians during resting state. Here, we aimed at determining whether brain networks differentially exchange information in musicians as opposed to non-musicians during naturalistic music listening. Whole-brain graph-theory analyses were performed on participants' fMRI responses. Group-level differences revealed that musicians' primary hubs comprised cerebral and cerebellar sensorimotor regions whereas non-musicians' dominant hubs encompassed DMN-related regions. Community structure analyses of the key hubs revealed greater integration of motor and somatosensory homunculi representing the upper limbs and torso in musicians. Furthermore, musicians who started training at an earlier age exhibited greater centrality in the auditory cortex, and areas related to top-down processes, attention, emotion, somatosensory processing, and non-verbal processing of speech. We here reveal how brain networks organize themselves in a naturalistic music listening situation wherein musicians automatically engage neural networks that are action-based while non-musicians use those that are perception-based to process an incoming auditory stream. Hum Brain Mapp 38:2955-2970, 2017. © 2017 Wiley Periodicals, Inc.

Learning New Sensorimotor Contingencies: Effects of Long-Term Use of Sensory Augmentation on the Brain and Conscious Perception

Theories of embodied cognition propose that perception is shaped by sensory stimuli and by the actions of the organism. Following sensorimotor contingency theory, the mastery of lawful relations between own behavior and resulting changes in sensory signals, called sensorimotor contingencies, is constitutive of conscious perception. Sensorimotor contingency theory predicts that, after training, knowledge relating to new sensorimotor contingencies develops, leading to changes in the activation of sensorimotor systems, and concomitant changes in perception. In the present study, we spell out this hypothesis in detail and investigate whether it is possible to learn new sensorimotor contingencies by sensory augmentation. Specifically, we designed an fMRI compatible sensory augmentation device, the feelSpace belt, which gives orientation information about the direction of magnetic north via vibrotactile stimulation on the waist of participants. In a longitudinal study, participants trained with this belt for seven weeks in natural environment. Our EEG results indicate that training with the belt leads to changes in sleep architecture early in the training phase, compatible with the consolidation of procedural learning as well as increased sensorimotor processing and motor programming. The fMRI results suggest that training entails activity in sensory as well as higher motor centers and brain areas known to be involved in navigation. These neural changes are accompanied with changes in how space and the belt signal are perceived, as well as with increased trust in navigational ability. Thus, our data on physiological processes and subjective experiences are compatible with the hypothesis that new sensorimotor contingencies can be acquired using sensory augmentation.

The experience of new sensorimotor contingencies by sensory augmentation

•

We investigate learning of sensorimotor contingencies by sensory augmentation.

•

The sensory device maps information of magnetic north to vibrotactile stimulation.

•

Active training with the device leads to marked changes in perception of space.

•

The device facilitates navigation and alters navigational strategies.

•

The device gives subjects a strong feeling of security and of “never get lost”.

Embedded in the paradigm of embodied cognition, the theory of sensorimotor contingencies (SMCs) proposes that motor actions and associated sensory stimulations are tied together by lawful relations termed SMCs. We aimed to investigate whether SMCs can be learned by means of sensory augmentation. Therefore we focused on related perceptual changes. Subjects trained for 7 weeks with the feelSpace belt mapping information of the magnetic north to vibrotactile stimulation around the waist. They experienced substantial changes in their space perception. The belt facilitated navigation and stimulated the usage of new navigation strategies. The belt’s vibrating signal changed to a kind of spatial information over time while the belt’s appeal and perceived usability increased. The belt also induced certain emotional states. Overall, the results show that learning new SMCs with this relatively small and usable device leads to profound perceptual and emotional changes, which are fully compatible with embodied theories of cognition.