The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery

Radial bias alters high-level motion perception

The visual system involves various orientation and visual field anisotropies, one of which is a preference for radial orientations and motion directions. By radial, we mean those directions coursing symmetrically outward from the fovea into the periphery. This bias stems from anatomical and physiological substrates in the early visual system. We recently reported that this low-level visual anisotropy can alter perceived object orientation. Here, we report that radial bias can also alter another higher-level system, the perceived direction of apparent motion. We presented a bistable apparent motion quartet in the center of the screen while participants fixated on various locations around the quartet. Participants (N = 22) were strongly biased to see the motion direction that was radial with respect to their fixation, controlling for any biases with center fixation. This was observed using a vertical-horizontal quartet as well as an oblique quartet (45° rotated quartet). The latter allowed us to rule out the contribution of the hemisphere effect where motion across the midline is perceived less often. These results extend our earlier findings on perceived object orientation, showing that low-level structural aspects of the visual system alter yet another higher-level visual process, that of apparent motion perception.

The speed and temporal frequency of visual apparent motion modulate auditory duration perception

In the present study, we investigated how the perception of auditory duration could be modulated by a task-irrelevant, concurrent visual apparent motion, induced by visual bars alternating between left and right sides. Moreover, we examined the influence of the speed and temporal frequency of visual apparent motion on the perception of auditory duration. In each trial, the standard visual stimuli (two vertical bars) were presented sequentially, except that visual apparent motion was included in the fourth stimulus. A tone was presented simultaneously with each visual stimulus, while the fourth tone was presented with varied duration. Participants judged whether the fourth tone lasted longer than the other four tones. In Experiment 1, the speed of visual apparent motion (Fast vs. Slow) was manipulated by changing the interval between two bars. The mean point of subjective equality (PSE) in the Slow apparent motion condition was larger than that in the Static condition. Moreover, participants tended to overestimate the duration only in the Static condition, i.e., time dilation effect, which disappeared under apparent motion conditions. In Experiment 2, in addition to speed, we controlled the temporal frequency of apparent motion by manipulating the number of bars, generating four conditions of visual apparent motion (Physical-fast, Perceived-fast, Perceived-slow, vs. Static). The mean PSE was significantly smaller in the Physical-fast condition than in the Static and Perceived-slow conditions. Moreover, we found a time compression effect in both the Perceived-slow and Static conditions but not in the Perceived-fast and Physical-fast conditions. These results suggest that the auditory duration could be modulated by the concurrent, contextual visual apparent motion, and both the speed and temporal frequency of the task-irrelevant visual apparent motion contribute to the bias in perceiving the auditory duration.

Imaging the columnar functional organization of human area MT+ to axis-of-motion stimuli using VASO at 7 Tesla

Cortical columns of direction-selective neurons in the motion sensitive area (MT) have been successfully established as a microscopic feature of the neocortex in animals. The same property has been investigated at mesoscale (<1 mm) in the homologous brain area (hMT+, V5) in living humans by using ultra-high field functional magnetic resonance imaging (fMRI). Despite the reproducibility of the selective response to axis-of-motion stimuli, clear quantitative evidence for the columnar organization of hMT+ is still lacking. Using cerebral blood volume (CBV)-sensitive fMRI at 7 Tesla with submillimeter resolution and high spatial specificity to microvasculature, we investigate the columnar functional organization of hMT+ in 5 participants perceiving axis-of-motion stimuli for both blood oxygenation level dependent (BOLD) and vascular space occupancy (VASO) contrast mechanisms provided by the used slice-selective slab-inversion (SS-SI)-VASO sequence. With the development of a new searchlight algorithm for column detection, we provide the first quantitative columnarity map that characterizes the entire 3D hMT+ volume. Using voxel-wise measures of sensitivity and specificity, we demonstrate the advantage of using CBV-sensitive fMRI to detect mesoscopic cortical features by revealing higher specificity of axis-of-motion cortical columns for VASO as compared to BOLD contrast. These voxel-wise metrics also provide further insights on how to mitigate the highly debated draining veins effect. We conclude that using CBV-VASO fMRI together with voxel-wise measurements of sensitivity, specificity and columnarity offers a promising avenue to quantify the mesoscopic organization of hMT+ with respect to axis-of-motion stimuli. Furthermore, our approach and methodological developments are generalizable and applicable to other human brain areas where similar mesoscopic research questions are addressed.

Categorical encoding of moving colors during location tracking

Categorical perception (CP) describes our tendency to perceive the visual world in a categorical manner, suggesting that high-level cognition may affect perception. While most studies are conducted in static visual scenes, Sun and colleagues found CP effects of color in multiple object tracking (MOT). This study used functional magnetic resonance imaging to investigate the neural mechanism behind the categorical effects of color in MOT. Categorical effects were associated with activities in a broad range of brain regions, including both the ventral (V4, middle temporal gyrus) and dorsal pathways (MT + /V5, inferior parietal lobule) of feature processing, as well as frontal regions (middle frontal gyrus, medial superior frontal gyrus). We proposed that these regions are hierarchically organized and responsible for distinct functions. The color-selective V4 encodes color categories, making cross-category colors more discriminable than within-category colors. Meanwhile, the language and/or semantic regions encode the verbal information of the colors. Both visual and nonvisual codes of color categories then modulate the activities of motion-sensitive MT + areas and frontal areas responsible for attentional processes.

Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans

We investigated neural representations for visual perception of 10 handwritten digits and six visual objects from a convolutional neural network (CNN) and humans using functional magnetic resonance imaging (fMRI). Once our CNN model was fine-tuned using a pre-trained VGG16 model to recognize the visual stimuli from the digit and object categories, representational similarity analysis (RSA) was conducted using neural activations from fMRI and feature representations from the CNN model across all 16 classes. The encoded neural representation of the CNN model exhibited the hierarchical topography mapping of the human visual system. The feature representations in the lower convolutional (Conv) layers showed greater similarity with the neural representations in the early visual areas and parietal cortices, including the posterior cingulate cortex. The feature representations in the higher Conv layers were encoded in the higher-order visual areas, including the ventral/medial/dorsal stream and middle temporal complex. The neural representations in the classification layers were observed mainly in the ventral stream visual cortex (including the inferior temporal cortex), superior parietal cortex, and prefrontal cortex. There was a surprising similarity between the neural representations from the CNN model and the neural representations for human visual perception in the context of the perception of digits versus objects, particularly in the primary visual and associated areas. This study also illustrates the uniqueness of human visual perception. Unlike the CNN model, the neural representation of digits and objects for humans is more widely distributed across the whole brain, including the frontal and temporal areas.

Use of connectotyping on task functional MRI data reveals dynamic network level cross talking during task performance

Task-based functional MRI (fMRI) has greatly improved understanding of brain functioning, enabling the identification of brain areas associated with specific cognitive operations. Traditional analyses are limited to associating activation patterns in particular regions with specific cognitive operation, largely ignoring regional cross-talk or dynamic connectivity, which we propose is crucial for characterization of brain function in the context of task fMRI. We use connectotyping, which efficiently models functional brain connectivity to reveal the progression of temporal brain connectivity patterns in task fMRI. Connectotyping was employed on data from twenty-four participants (12 male, mean age 24.8 years, 2.57 std. dev) who performed a widely spaced event-related fMRI word vs. pseudoword decision task, where stimuli were presented every 20 s. After filtering for movement, we ended up with 15 participants that completed each trial and had enough usable data for our analyses. Connectivity matrices were calculated per participant across time for each stimuli type. A Repeated Measures ANOVA applied on the connectotypes was used to characterize differences across time for words and pseudowords. Our group level analyses found significantly different dynamic connectivity patterns during word vs. pseudoword processing between the Fronto-Parietal and Cingulo-Parietal Systems, areas involved in cognitive task control, memory retrieval, and semantic processing. Our findings support the presence of dynamic changes in functional connectivity during task execution and that such changes can be characterized using connectotyping but not with traditional Pearson’s correlations.

Visual mental imagery: Inside the mind's eyes

What are mental images needed for? A variety of everyday situations calls for us to plan ahead; one of the clever ways our mind prepares and strategizes our next move is through mental simulation. A powerful tool in running these simulations is visual mental imagery, which can be conceived as a way to activate and maintain an internal representation of the to-be-imagined object, giving rise to predictions. Therefore, under normal conditions imagination is primarily an endogenous process, and only more rarely can mental images be activated exogenously, for example, by means of intracerebral stimulation. A large debate is still ongoing regarding the neural substrates supporting mental imagery, with the neuropsychological and neuroimaging literature agreeing in some cases, but not others. This chapter reviews the neuroscientific literature on mental imagery, and attempts to reappraise the neuropsychological and neuroimaging evidence by drawing a model of mental imagery informed by both structural and functional brain data. Overall, the role of regions in the ventral temporal cortex, especially of the left hemisphere, stands out unequivocally as a key substrate in mental imagery.

Decoding Color Visual Working Memory from EEG Signals Using Graph Convolutional Neural Networks

Color has an important role in object recognition and visual working memory (VWM). Decoding color VWM in the human brain is helpful to understand the mechanism of visual cognitive process and evaluate memory ability. Recently, several studies showed that color could be decoded from scalp electroencephalogram (EEG) signals during the encoding stage of VWM, which process visible information with strong neural coding. Whether color could be decoded from other VWM processing stages, especially the maintaining stage which processes invisible information, is still unknown. Here, we constructed an EEG color graph convolutional network model (ECo-GCN) to decode colors during different VWM stages. Based on graph convolutional networks, ECo-GCN considers the graph structure of EEG signals and may be more efficient in color decoding. We found that (1) decoding accuracies for colors during the encoding, early, and late maintaining stages were 81.58%, 79.36%, and 77.06%, respectively, exceeding those during the pre-stimuli stage (67.34%), and (2) the decoding accuracy during maintaining stage could predict participants' memory performance. The results suggest that EEG signals during the maintaining stage may be more sensitive than behavioral measurement to predict the VWM performance of human, and ECo-GCN provides an effective approach to explore human cognitive function.

V5/MT+ modulates spatio-temporal integration differently across and within hemifields: Causal evidence from TMS

It is unclear how the brain reaches the correct balance between temporal and spatial processing necessary to perceive motion across space. Here, we tested whether visual motion area V5/MT + plays a causal role in Ternus illusion. Ternus displays can be perceived as showing either group motion or element motion and are empirically useful for dissociating temporal and spatial grouping across visual fields. Online single-pulse TMS was applied to observers during the presentation of Ternus displays, either within or across hemifields, over left V5/MT + or, respectively, a control site in the left somatosensory cortex, or an additional 'Sham' control condition. In the cross-hemifields condition, observers perceived more element motion with TMS over left V5/MT + than in either control condition. By contrast, in the within-hemifield condition, observers reported more group motion after left V5/MT + TMS. Our findings demonstrate a causal role of left V5/MT+ in the spatio-temporal grouping of Ternus apparent motion, and in maintaining the balance of spatio-temporal processing both within and across individual hemifields.

Pupil responses to implied motion in figurative and abstract paintings

Motion can be perceived in static images, such as photos and figurative paintings, representing realistic subjects in motion, with or without directional information (e.g., motion blur or speed lines). Motion impression can be achieved even in non-realistic static images such as motion illusions and abstract paintings. It has been shown that visual motion processing affects the diameter of the pupil, responding differently to real, illusory, and implied motion in photographs (IM). It has been suggested that these different effects might be due to top-down modulations from different cortical areas underlying their processing. It is worthwhile to investigate pupillary response to figurative paintings, since they require an even higher level of interpretation than photos representing the same kind of subjects, given the complexity of cognitive processes involved in the aesthetic experience. Also, pupil responses to abstract paintings allows to study the effect of IM perception in representations devoid of real-life motion cues. We measured pupil responses to IM in figurative and abstract artworks depicting static and dynamic scenes, as rated by a large group of individuals not participating in the following experiment. Since the pupillary response is modulated by the subjective image interpretation, a motion rating test has been used to correct individual pupil data according to whether participants actually perceived the presence of motion in the paintings. Pupil responses to movies showing figurative and abstract subjects, and to motion illusions were also measured, to compare real and illusory motion with painted IM. Movies, both figurative and abstract, elicit the largest pupillary dilation of all static stimuli, whereas motion illusions cause the smallest pupil size, as previously shown. Interestingly, pupil responses to IM depend on the paintings' style. Figurative paintings depicting moving subjects cause more dilation than those representing static figures, and pupil size increases with the strength of IM, as already found with realistic photos. The opposite effect is obtained with abstract artworks. Abstract paintings depicting motion produce less dilation than those depicting stillness. In any case, these results reflect the individual subjective perception of dynamism, as the very same paintings can induce opposite responses in observer which interpreted it as static or dynamic. Overall, our data show that pupil size depends on high-level interpretation of motion in paintings, even when they do not represent real-world scenes. Our findings further suggest that the pupil is modulated by multiple top-down cortical mechanisms, involving the processing of motion, attention, memory, imagination, and other cognitive functions necessary for enjoying a complete aesthetic experience.

Resolving visual motion through perceptual gaps

Perceptual gaps can be caused by objects in the foreground temporarily occluding objects in the background or by eyeblinks, which briefly but frequently interrupt visual information. Resolving visual motion across perceptual gaps is particularly challenging, as object position changes during the gap. We examine how visual motion is maintained and updated through externally driven (occlusion) and internally driven (eyeblinks) perceptual gaps. Focusing on both phenomenology and potential mechanisms such as suppression, extrapolation, and integration, we present a framework for how perceptual gaps are resolved over space and time. We finish by highlighting critical questions and directions for future work.

Pupillary response to real, illusory, and implied motion

The perception of moving objects (real motion) is a critical function for interacting with a dynamic environment. Motion perception can be also induced by particular structural features of static images (illusory motion) or by photographic images of subjects in motion (implied motion, IM). Many cortical areas are involved in motion processing, particularly the medial temporal cortical area (MT), dedicated to the processing of real, illusory, and implied motion. Recently, there has been a growing interest in the influence of high-level visual processes on pupillary responses. However, just a few studies have measured the effect of motion processing on the pupil, and not always with consistent results. Here we systematically investigate the effects of real, illusory, and implied motion on the pupil diameter for the first time, by showing different types of stimuli (movies, illusions, and photos) with the same average luminance to the same observers. We find different pupillary responses depending on the nature of motion. Real motion elicits a larger pupillary dilation than IM, which in turn induces more dilation than control photos representing static subjects (No-IM). The pupil response is sensitive even to the strength of IM, as photos with enhanced IM (blur, motion streaks, speed lines) induce larger dilation than simple freezed IM (subjects captured in the instant they are moving). Also, the subject represented in the stimulus matters: human figures are interpreted as more dynamic and induce larger dilation than objects/animals. Interestingly, illusory motion induces much less dilation than all the other motion categories, despite being seen as moving. Overall, pupil responses depend on the individual perception of dynamicity, confirming that the pupil is modulated by the subjective interpretation of complex stimuli. We argue that the different pupillary responses to real, illusory, and implied motion reflect the top-down modulations of different cortical areas involved in their processing.

Hemispheric asymmetries in visual mental imagery

Visual mental imagery is the faculty whereby we can "visualize" objects that are not in our line of sight. Longstanding evidence dating back over thirty years has shown that unilateral brain lesions, especially in the left temporal lobe, can impair aspects of this ability. Yet, there is currently no attempt to identify analogies between these neuropsychological findings of hemispheric asymmetry and those from other neuroscientific approaches. Here, we present a critical review of the available literature on the hemispheric laterality of visual mental imagery, by looking at cross-method patterns of evidence in the domains of lesion neuropsychology, neuroimaging, and direct cortical stimulation. Results can be summarized under three main axes. First, frontoparietal networks in both hemispheres appear to be associated with visual mental imagery. Second, lateralization patterns emerge in the temporal lobes, with the left inferior temporal lobe being the most common finding in the literature for endogenously generated images, especially, but not exclusively, when orthographic material is used to ignite imagery. Third, an opposite pattern of hemispheric laterality emerges when visual mental images are induced by exogenous stimulation; direct cortical electrical stimulation tends to produce visual imagery experiences predominantly when applied to the right temporal lobe. These patterns of hemispheric asymmetry are difficult to reconcile with the dominant model of visual mental imagery, which emphasizes the implication of early sensory cortices. They suggest instead that visual mental imagery relies on large-scale brain networks, with a crucial participation of high-level visual regions in the temporal lobes.

Effects of ketamine on brain function during metacognition of episodic memory

Only little research has been conducted on the pharmacological underpinnings of metacognition. Here, we tested the modulatory effects of a single intravenous dose (100 ng/ml) of the N-methyl-D-aspartate-glutamate-receptor antagonist ketamine, a compound known to induce altered states of consciousness, on metacognition and its neural correlates. Fifty-three young, healthy adults completed two study phases of an episodic memory task involving both encoding and retrieval in a double-blind, placebo-controlled fMRI study. Trial-by-trial confidence ratings were collected during retrieval. Effects on the subjective state of consciousness were assessed using the 5D-ASC questionnaire. Confirming that the drug elicited a psychedelic state, there were effects of ketamine on all 5D-ASC scales. Acute ketamine administration during retrieval had deleterious effects on metacognitive sensitivity (meta-d') and led to larger metacognitive bias, with retrieval performance (d') and reaction times remaining unaffected. However, there was no ketamine effect on metacognitive efficiency (meta-d'/d'). Measures of the BOLD signal revealed that ketamine compared to placebo elicited higher activation of posterior cortical brain areas, including superior and inferior parietal lobe, calcarine gyrus, and lingual gyrus, albeit not specific to metacognitive confidence ratings. Ketamine administered during encoding did not significantly affect performance or brain activation. Overall, our findings suggest that ketamine impacts metacognition, leading to significantly larger metacognitive bias and deterioration of metacognitive sensitivity as well as unspecific activation increases in posterior hot zone areas of the neural correlates of consciousness.

Burke's "sensitive" sublime rooted in brain and body versus Kant's transcendental concept in the German Romantic discourse

During the 18th century, debates about what constituted the sublime flourished in Europe, and particularly in Germany. These debates were nourished by two different visions: The Kantian concept supposed that the sublime is supra-sensible and rooted in reason (Logos) rather than in the object, thus provoking a mental state of tension between nature and art; Edmund Burke's concept, on the other hand, conceived of the sublime as a bodily immersive experience, which we here define as "sensitive" sublime. In summary, Burke's view of the sublime is rooted in the senses and not in the power of reason, unlike Kant's. This was to disrupt the mainstream ideas of that time, unconsciously anticipating some of the recent neuroaesthetic acquisitions regarding the central role played by the sensory apparatus in the experience of beauty and of the sublime.

Neural Selectivity for Visual Motion in Macaque Area V3A

The processing of visual motion is conducted by dedicated pathways in the primate brain. These pathways originate with populations of direction-selective neurons in the primary visual cortex, which projects to dorsal structures like the middle temporal (MT) and medial superior temporal (MST) areas. Anatomical and imaging studies have suggested that area V3A might also be specialized for motion processing, but there have been very few studies of single-neuron direction selectivity in this area. We have therefore performed electrophysiological recordings from V3A neurons in two macaque monkeys (one male and one female) and measured responses to a large battery of motion stimuli that includes translation motion, as well as more complex optic flow patterns. For comparison, we simultaneously recorded the responses of MT neurons to the same stimuli. Surprisingly, we find that overall levels of direction selectivity are similar in V3A and MT and moreover that the population of V3A neurons exhibits somewhat greater selectivity for optic flow patterns. These results suggest that V3A should be considered as part of the motion processing machinery of the visual cortex, in both human and non-human primates.

Aphantasia: The science of visual imagery extremes

Visual imagery allows us to revisit the appearance of things in their absence and to test out virtual combinations of sensory experience. Visual imagery has been linked to many cognitive processes, such as autobiographical and visual working memory. Imagery also plays symptomatic and mechanistic roles in neurologic and mental disorders and is utilized in treatment. A large network of brain activity spanning frontal, parietal, temporal, and visual cortex is involved in generating and maintain images in mind. The ability to visualize has extreme variations, ranging from completely absent (aphantasia) to photo-like (hyperphantasia). The anatomy and functionality of visual cortex, including primary visual cortex, have been associated with individual differences in visual imagery ability, pointing to a potential correlate for both aphantasia and hyperphantasia. Preliminary evidence suggests that lifelong aphantasia is associated with prosopagnosia and reduction in autobiographical memory; hyperphantasia is associated with synesthesia. Aphantasic individuals can also be highly imaginative and are able to complete many tasks that were previously thought to rely on visual imagery, demonstrating that visualization is only one of many ways of representing things in their absence. The study of extreme imagination reminds us how easily invisible differences can escape detection.

Occipital and Cerebellar Theta Burst Stimulation for Mal De Debarquement Syndrome

BACKGROUND

Individuals with Mal de Debarquement syndrome (MdDS) experience persistent oscillating vertigo lasting for months or years. Transcranial magnetic stimulation (TMS) can modulate the motion perception of MdDS.

MATERIALS AND METHODS

Twenty-six TMS naive individuals received single administrations of continuous theta burst stimulation (cTBS) over the occipital cortex, cerebellar vermis, and lateral cerebellar hemisphere, in randomized order. A 0-100 point Visual Analogue Scale was used to assess acute changes in oscillating vertigo severity after each session. Repeated treatments were given over the target that led to the most acute reduction in symptoms. All treatments were performed with neuronavigation using the participant's own brain MRI. The Dizziness Handicap Inventory (DHI), MdDS Balance Rating Scale (MBRS), and Hospital Anxiety and Depression Scale (HADS) were assessed weekly at four pretreatment and six posttreatment time points.

RESULTS

Twenty participants chose either the occipital cortex (11) or cerebellar vermis (9) targets as most effective in reducing the oscillating vertigo; one chose lateral cerebellar hemisphere; five chose none. After 10 to 12 sessions of 1,200 pulses over the target of choice, 19 of 25 treatment completers noted ≥ 25% reduction, 12 of 25 ≥50% reduction, and 8 of 25 ≥75% reduction in oscillating vertigo intensity. A one-way repeated measures ANOVA of DHI, MBRS, and HADS scores before and after treatment showed significant reductions in DHI, MBRS, and the HADS Anxiety subscore immediately after treatment with most improvement lasting through posttreatment week 6. There were no significant Depression subscore changes. Participants who had chosen vermis stimulation had comparatively worse balance at baseline than those who had chosen occipital cortex stimulation.

CONCLUSION

cTBS over either the occipital cortex or cerebellar vermis is effective in reducing the oscillating vertigo of MdDS acutely and may confer long-term benefits. Sustained improvement requires more frequent treatments.

Apparent Motion Induces Activity Suppression in Early Visual Cortex and Impairs Visual Detection

Apparent motion (AM) is induced when two stationary visual stimuli are presented in alternating sequence. Intriguingly, AM leads to an impaired detectability of stimuli along the AM path (i.e., AM-induced masking). It has been hypothesized that AM triggers an internal representation of a moving object in early visual cortex, which competes with stimulus-evoked representations of visual stimuli on the motion path in early visual cortex of 25 human adults (16 female). We tested this hypothesis by measuring BOLD responses in early visual cortex during the process of AM-induced masking, using fMRI and population receptive field methods. Surprisingly, and counter to our hypothesis, we showed that AM suppressed, rather than increased, BOLD responses along early visual (V1 and V2) representations of the AM path, including regions that were not directly activated by the AM inducer stimuli. This activity suppression of the visual response predicted the subsequent reduction in detectability of the target that appeared in the middle of the AM path. Our data thereby provide direct empirical evidence for suppressive neural mechanisms underlying AM and suggest that illusory motion can render us blind to objects on the motion path by suppressing neural activity at the earliest cortical stages of visual perception.SIGNIFICANCE STATEMENT When two spatially distinct visual objects are presented in alternating sequence, apparent motion (AM) occurs and impairs detectability of stimuli along its path. The underlying mechanism is thought to be that increased activation in human early visual cortex evoked by AM interferes with the representation of the stimulus. Strikingly, however, we show that AM suppresses neural activity along the motion path, and the strength of activity suppression predicts the subsequent behavioral performance decrement in terms of detecting a stimulus along the AM path. Our findings provide empirical evidence for a suppressive, rather than faciliatory, mechanism underlying AM.

Neural responses to apparent motion can be predicted by responses to non-moving stimuli

When two objects are presented in alternation at two locations, they are seen as a single object moving from one location to the other. This apparent motion (AM) percept is experienced for objects located at short and also at long distances. However, current models cannot explain how the brain integrates information over large distances to create such long-range AM. This study investigates the neural markers of AM by parcelling out the contribution of spatial and temporal interactions not specific to motion. In two experiments, participants’ EEG was recorded while they viewed two stimuli inducing AM. Different combinations of these stimuli were also shown in a static context to predict an AM neural response where no motion is perceived. We compared the goodness of fit between these different predictions and found consistent results in both experiments. At short-range, the addition of the inhibitory spatial and temporal interactions not specific to motion improved the AM prediction. However, there was no indication that spatial or temporal non-linear interactions were present at long-range. This suggests that short- and long-range AM rely on different neural mechanisms. Importantly, our results also show that at both short- and long-range, responses generated by a moving stimulus could be well predicted from conditions in which no motion is perceived. That is, the EEG response to a moving stimulus is simply a combination of individual responses to non-moving stimuli. This demonstrates a dissociation between the brain response and the subjective percept of motion.

•

EEG responses are inhibited by spatial and temporal stimulus interactions.

•

These interactions are important for motion at short but not at long distances.

•

We find no trace of a specific neural signature of motion perception.

•

Neural responses to motion are well predicted by responses to non-moving stimuli.

Temporal variability of brain networks predicts individual differences in bistable perception

When ambiguous visual stimuli are presented to the eyes, conscious perception can spontaneously alternate across the competing interpretations - which was known as bistable perception. The spontaneous alternation of perception might indicate a connection between bistable perception and the dynamic interaction of brain networks. Here, we hypothesized that individual differences in perceptual dynamics may be reflected in dynamics of spontaneous neural activities. To test this idea, we investigated the relationship between the percept duration and the reconfiguration patterns of dynamic brain networks as measured by the functional connectivity (FC) during the resting state. Firstly, we found that individual difference of percept duration is associated with the temporal variability of the brain regions which were previously reported in studies of bistable perception, including anterior cingulate cortex (ACC), dorsal medial prefrontal cortex (DMPFC), dorsal lateral prefrontal cortex (DLPFC), superior parietal lobule (SPL), inferior parietal lobule (IPL), precuneus, insula, and V5. Secondly, there is a positive relationship between the temporal variability within the frontal-parietal network (FPN) and the percept duration. Thirdly, our results indicated that individual difference of bistable perception was related to the dynamic interaction between large-scale functional networks including default mode network (DMN), FPN, cingulo-opercular network (CON), dorsal attention network (DAN), salience network (SN), memory retrieval network (MRN). Altogether, our results demonstrated that inter-individual variability in bistable perception was associated with dynamic coupling of brain regions and networks involved in primary visual processing, spatial attention, and cognitive control.

Mental Imagery and Brain Regulation-New Links Between Psychotherapy and Neuroscience

Mental imagery is a promising tool and mechanism of psychological interventions, particularly for mood and anxiety disorders. In parallel developments, neuromodulation techniques have shown promise as add-on therapies in psychiatry, particularly non-invasive brain stimulation for depression. However, these techniques have not yet been combined in a systematic manner. One novel technology that may be able to achieve this is neurofeedback, which entails the self-regulation of activation in specific brain areas or networks (or the self-modulation of distributed activation patterns) by the patients themselves, through real-time feedback of brain activation (for example, from functional magnetic resonance imaging). One of the key mechanisms by which patients learn such self-regulation is mental imagery. Here, we will first review the main mental imagery approaches in psychotherapy and the implicated brain networks. We will then discuss how these networks can be targeted with neuromodulation (neurofeedback or non-invasive or invasive brain stimulation). We will review the clinical evidence for neurofeedback and discuss possible ways of enhancing it through systematic combination with psychological interventions, with a focus on depression, anxiety disorders, and addiction. The overarching aim of this perspective paper will be to open a debate on new ways of developing neuropsychotherapies.

Behavioral and oculomotor evidence for visual simulation of object movement

We regularly interact with moving objects in our environment. Yet, little is known about how we extrapolate the future movements of visually perceived objects. One possibility is that movements are experienced by a mental visual simulation, allowing one to internally picture an object's upcoming motion trajectory, even as the object itself remains stationary. Here we examined this possibility by asking human participants to make judgments about the future position of a falling ball on an obstacle-filled display. We found that properties of the ball's trajectory were highly predictive of subjects' reaction times and accuracy on the task. We also found that the eye movements subjects made while attempting to ascertain where the ball might fall had significant spatiotemporal overlap with those made while actually perceiving the ball fall. These findings suggest that subjects simulated the ball's trajectory to inform their responses. Finally, we trained a convolutional neural network to see whether this problem could be solved by simple image analysis as opposed to the more intricate simulation strategy we propose. We found that while the network was able to solve our task, the model's output did not effectively or consistently predict human behavior. This implies that subjects employed a different strategy for solving our task, and bolsters the conclusion that they were engaging in visual simulation. The current study thus provides support for visual simulation of motion as a means of understanding complex visual scenes and paves the way for future investigations of this phenomenon at a neural level.

Interactions of flicker and motion

We present a series of novel observations about interactions between flicker and motion that lead to three distinct perceptual effects. We use the term flicker to describe alternating changes in a stimulus' luminance or color (i.e. a circle that flickers from black to white and visa-versa). When objects flicker, three distinct phenomena can be observed: (1) Flicker Induced Motion (FLIM) in which a single, stationary object, appears to move when it flickers at certain rates; (2) Flicker Induced Motion Suppression (FLIMS) in which a moving object appears to be stationary when it flickers at certain rates, and (3) Flicker-Induced Induced-Motion (FLIIM) in which moving objects that are flickering induce another flickering stationary object to appear to move. Across four psychophysical experiments, we characterize key stimulus parameters underlying these flicker-motion interactions. Interactions were strongest in the periphery and at flicker frequencies above 10 Hz. Induced motion occurred not just for luminance flicker, but for isoluminant color changes as well. We also found that the more physically moving objects there were, the more motion induction to stationary objects occurred. We present demonstrations that the effects reported here cannot be fully accounted for by eye movements: we show that the perceived motion of multiple stationary objects that are induced to move via flicker can appear to move independently and in random directions, whereas eye movements would have caused all of the objects to appear to move coherently. These effects highlight the fundamental role of spatiotemporal dynamics in the representation of motion and the intimate relationship between flicker and motion.

Action perception and motor imagery: Mental practice of action

Motor cognition is related to the planning and generation of actions as well as to the recognition and imagination of motor acts. Recently, there is evidence that the motor system participates not only in overt actions but also in mental processes supporting covert actions. Within this framework, we have investigated the cortical areas engaged in execution, observation, and imagination of the same action, by the use of the high resolution quantitative ¹⁴C-deoxyglucose method in monkeys and by fMRI in humans, throughout the entire primate brain. Our data demonstrated that observing or imagining an action excites virtually the same sensory-motor cortical network which supports execution of that same action. In general agreement with the results of five relevant meta-analyses that we discuss extensively, our results imply mental practice, i.e. internal rehearsal of the action including movements and their sensory effects. We suggest that we actively perceive and imagine actions by selecting and running off-line restored sensory-motor memories, by mentally simulating the actions. We provide empirical evidence that mental simulation of actions underlies motor cognition, and conceptual representations are grounded in sensory-motor codes. Motor cognition may, therefore, be embodied and modal. Finally, we consider questions regarding agency attribution and the possible causal or epiphenomenal role the involved sensory-motor network could play in motor cognition.

Reconstructing imagined letters from early visual cortex reveals tight topographic correspondence between visual mental imagery and perception

Visual mental imagery is the quasi-perceptual experience of “seeing in the mind’s eye”. While a tight correspondence between imagery and perception in terms of subjective experience is well established, their correspondence in terms of neural representations remains insufficiently understood. In the present study, we exploit the high spatial resolution of functional magnetic resonance imaging (fMRI) at 7T, the retinotopic organization of early visual cortex, and machine-learning techniques to investigate whether visual imagery of letter shapes preserves the topographic organization of perceived shapes. Sub-millimeter resolution fMRI images were obtained from early visual cortex in six subjects performing visual imagery of four different letter shapes. Predictions of imagery voxel activation patterns based on a population receptive field-encoding model and physical letter stimuli provided first evidence in favor of detailed topographic organization. Subsequent visual field reconstructions of imagery data based on the inversion of the encoding model further showed that visual imagery preserves the geometric profile of letter shapes. These results open new avenues for decoding, as we show that a denoising autoencoder can be used to pretrain a classifier purely based on perceptual data before fine-tuning it on imagery data. Finally, we show that the autoencoder can project imagery-related voxel activations onto their perceptual counterpart allowing for visually recognizable reconstructions even at the single-trial level. The latter may eventually be utilized for the development of content-based BCI letter-speller systems.

Melody Processing Characterizes Functional Neuroanatomy in the Aging Brain

The functional neuroanatomical mechanisms underpinning cognition in the normal older brain remain poorly defined, but have important implications for understanding the neurobiology of aging and the impact of neurodegenerative diseases. Auditory processing is an attractive model system for addressing these issues. Here, we used fMRI of melody processing to investigate auditory pattern processing in normal older individuals. We manipulated the temporal (rhythmic) structure and familiarity of melodies in a passive listening, 'sparse' fMRI protocol. A distributed cortico-subcortical network was activated by auditory stimulation compared with silence; and within this network, we identified separable signatures of anisochrony processing in bilateral posterior superior temporal lobes; melodic familiarity in bilateral anterior temporal and inferior frontal cortices; and melodic novelty in bilateral temporal and left parietal cortices. Left planum temporale emerged as a 'hub' region functionally partitioned for processing different melody dimensions. Activation of Heschl's gyrus by auditory stimulation correlated with the integrity of underlying cortical tissue architecture, measured using multi-parameter mapping. Our findings delineate neural substrates for analyzing perceptual and semantic properties of melodies in normal aging. Melody (auditory pattern) processing may be a useful candidate paradigm for assessing cerebral networks in the older brain and potentially, in neurodegenerative diseases of later life.

Eye movements during path integration

Self-motion induces spontaneous eye movements which serve the purpose of stabilizing the visual image on the retina. Previous studies have mainly focused on their reflexive nature and how the perceptual system disentangles visual flow components caused by eye movements and self-motion. Here, we investigated the role of eye movements in distance reproduction (path integration). We used bimodal (visual-auditory)-simulated self-motion: visual optic flow was paired with an auditory stimulus whose frequency was scaled with simulated speed. The task of the subjects in each trial was, first, to observe the simulated self-motion over a certain distance (Encoding phase) and, second, to actively reproduce the observed distance using only visual, only auditory, or bimodal feedback (Reproduction phase). We found that eye positions and eye speeds were strongly correlated between the Encoding and the Reproduction phases. This was the case even when reproduction relied solely on auditory information and thus no visual stimulus was presented. We believe that these correlations are indicative of a contribution of eye movements to path integration.

Visual imagery of faces and cars in face-selective visual areas

Neuroimaging provides a unique tool to investigate otherwise difficult-to-access mental processes like visual imagery. Prior studies support the idea that visual imagery is a top-down reinstatement of visual perception, and it is likely that this extends to object processing. Here we use functional MRI and multi-voxel pattern analysis to ask if mental imagery of cars engages the fusiform face area, similar to what is found during perception. We test only individuals who we assumed could imagine individual car models based on their above-average perceptual abilities with cars. Our results provide evidence that cars are represented differently from common objects in face-selective visual areas, at least in those with above-average car recognition ability. Moreover, pattern classifiers trained on data acquired during imagery can decode the neural response pattern acquired during perception, suggesting that the tested object categories are represented similarly during perception and visual imagery. The results suggest that, even at high-levels of visual processing, visual imagery mirrors perception to some extent, and that face-selective areas may in part support non-face object imagery.

Endogenously generated gamma-band oscillations in early visual cortex: A neurofeedback study

Human subjects were trained with neurofeedback (NFB) to enhance the power of narrow-band gamma oscillations in circumscribed regions of early visual cortex. To select the region and the oscillation frequency for NFB training, gamma oscillations were induced with locally presented drifting gratings. The source and frequency of these induced oscillations were determined using beamforming methods. During NFB training the power of narrow band gamma oscillations was continuously extracted from this source with online beamforming and converted into the pitch of a tone signal. We found that seven out of ten subjects were able to selectively increase the amplitude of gamma oscillations in the absence of visual stimulation. One subject however failed completely and two subjects succeeded to manipulate the feedback signal by contraction of muscles. In all subjects the attempts to enhance visual gamma oscillations were associated with an increase of beta oscillations over precentral/frontal regions. Only successful subjects exhibited an additional marked increase of theta oscillations over precentral/prefrontal and temporal regions whereas unsuccessful subjects showed an increase of alpha band oscillations over occipital regions. We argue that spatially confined networks in early visual cortex can be entrained to engage in narrow band gamma oscillations not only by visual stimuli but also by top down signals. We interpret the concomitant increase in beta oscillations as indication for an engagement of the fronto-parietal attention network and the increase of theta oscillations as a correlate of imagery. Our finding support the application of NFB in disease conditions associated with impaired gamma synchronization.

Brain networks underlying conscious tactile perception of textures as revealed using the velvet hand illusion

Humans are adept at perceiving textures through touch. Previous neuroimaging studies have identified a distributed network of brain regions involved in the tactile perception of texture. However, it remains unclear how nodes in this network contribute to the tactile awareness of texture. To examine the hypothesis that such awareness involves the interaction of the primary somatosensory cortex with higher order cortices, we conducted a functional magnetic resonance imaging (fMRI) study utilizing the velvet hand illusion, in which an illusory velvet-like surface is perceived between the hands. Healthy participants were subjected to a strong illusion, a weak illusion, and tactile perception of real velvet. The strong illusion induced greater activation in the primary somatosensory cortex (S1) than the weak illusion, and increases in such activation were positively correlated with the strength of the illusion. Furthermore, both actual and illusory perception of velvet induced common activation in S1. Psychophysiological interaction (PPI) analysis revealed that the strength of the illusion modulated the functional connectivity of S1 with each of the following regions: the parietal operculum, superior parietal lobule, precentral gyrus, insula, and cerebellum. The present results indicate that S1 is associated with the conscious tactile perception of textures, which may be achieved via interactions with higher order somatosensory areas.

Real-motion signals in human early visual cortex

Eye movements induce visual motion that can complicate the stable perception of the world. The visual system compensates for such self-induced visual motion by integrating visual input with efference copies of eye movement commands. This mechanism is central as it does not only support perceptual stability but also mediates reliable perception of world-centered objective motion. In humans, it remains elusive whether visual motion responses in early retinotopic cortex are driven by objective motion or by retinal motion associated with it. To address this question, we used fMRI to examine functional responses of sixteen visual areas to combinations of planar objective motion and pursuit eye movements. Observers were exposed to objective motion that was faster, matched or slower relative to pursuit, allowing us to compare conditions that differed in objective motion velocity while retinal motion and eye movement signals were matched. Our results show that not only higher level motion regions such as V3A and V6, but also early visual areas signaled the velocity of objective motion, hence the product of integrating retinal with non-retinal signals. These results shed new light on mechanisms that mediate perceptual stability and real-motion perception, and show that extra-retinal signals related to pursuit eye movements influence processing in human early visual cortex.

Domain-specific reports of visual imagery vividness are not related to perceptual expertise

Just as people vary in their perceptual expertise with a given domain, they also vary in their abilities to imagine objects. Visual imagery and perception share common mechanisms. However, it is unclear whether domain-specific expertise is relevant to visual imagery. Although the vividness of visual imagery is typically measured as a domain-general construct, a component of this vividness may be domain-specific. For example, individuals who have gained perceptual expertise with a specific domain might experience clearer mental images within this domain. Here we investigated whether perceptual expertise for cars relates to visual imagery vividness in the same domain, by assessing the correlations between a widely used domain-general measure of visual imagery vividness (the Vividness of Visual Imagery Questionnaire; Marks in British Journal of Psychology, 64, 17-24, 1973), a new measure of visual imagery vividness specific to cars, and behavioral tests of car expertise. We found that domain-specific imagery relates most strongly to general imagery vividness and less strongly to self-reported expertise, while it does not relate to perceptual or semantic expertise.

Predictive feedback to V1 dynamically updates with sensory input

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

Changes in fMRI BOLD dynamics reflect anticipation to moving objects

The human brain is thought to respond differently to novel versus predictable neural input. In human visual cortex, neural response amplitude to visual input might be determined by the degree of predictability. We investigated how fMRI BOLD responses in human early visual cortex reflect the anticipation of a single moving bar's trajectory. We found that BOLD signals decreased linearly from onset to offset of the stimulus trajectory. Moreover, decreased amplitudes of BOLD responses coincided with an increased initial dip as the stimulus moved along its trajectory. Importantly, motion anticipation effects were absent, when motion coherence was disrupted by means of stimulus contrast reversals. These results show that human early visual cortex anticipates the trajectory of a coherently moving object at the initial stages of visual motion processing. The results can be explained by suppression of predictable input, plausibly underlying the formation of stable visual percepts.

The role of regional heterogeneity in age-related differences in functional hemispheric asymmetry: an fMRI study

Neuroimaging literature has documented age-related hemispheric asymmetry reduction in frontal regions during task performances. As most studies employed working memory paradigms, it is therefore less clear if this pattern of neural reorganization is constrained by working memory processes or it would also emerge in other cognitive domains which are predominantly lateralized. Using blocked functional magnetic resonance imaging (fMRI), the present study used a homophone judgment task and a line judgment task to investigate age-related differences in functional hemispheric asymmetry in language and visuospatial processing respectively. Young and older adults achieved similar task accuracy although older adults required a significantly longer time. Age-related functional hemispheric asymmetry reduction was found only in dorsal inferior frontal gyrus and was associated with better performance when the homophone condition was contrasted against fixation, and not line condition. Our data thus highlights the importance of considering regional heterogeneity of aging effects together with general age-related cognitive processes.

Heteromodal Cortical Areas Encode Sensory-Motor Features of Word Meaning

The capacity to process information in conceptual form is a fundamental aspect of human cognition, yet little is known about how this type of information is encoded in the brain. Although the role of sensory and motor cortical areas has been a focus of recent debate, neuroimaging studies of concept representation consistently implicate a network of heteromodal areas that seem to support concept retrieval in general rather than knowledge related to any particular sensory-motor content. We used predictive machine learning on fMRI data to investigate the hypothesis that cortical areas in this "general semantic network" (GSN) encode multimodal information derived from basic sensory-motor processes, possibly functioning as convergence-divergence zones for distributed concept representation. An encoding model based on five conceptual attributes directly related to sensory-motor experience (sound, color, shape, manipulability, and visual motion) was used to predict brain activation patterns associated with individual lexical concepts in a semantic decision task. When the analysis was restricted to voxels in the GSN, the model was able to identify the activation patterns corresponding to individual concrete concepts significantly above chance. In contrast, a model based on five perceptual attributes of the word form performed at chance level. This pattern was reversed when the analysis was restricted to areas involved in the perceptual analysis of written word forms. These results indicate that heteromodal areas involved in semantic processing encode information about the relative importance of different sensory-motor attributes of concepts, possibly by storing particular combinations of sensory and motor features.

SIGNIFICANCE STATEMENT

The present study used a predictive encoding model of word semantics to decode conceptual information from neural activity in heteromodal cortical areas. The model is based on five sensory-motor attributes of word meaning (color, shape, sound, visual motion, and manipulability) and encodes the relative importance of each attribute to the meaning of a word. This is the first demonstration that heteromodal areas involved in semantic processing can discriminate between different concepts based on sensory-motor information alone. This finding indicates that the brain represents concepts as multimodal combinations of sensory and motor representations.

Veridical stimulus localization is linked to human area V5/MT+ activity

How the brain represents visual space is an unsolved mystery. Spatial localization becomes particularly challenging when visual information processing is briefly disrupted, as in the case of saccadic eye movements, blinks, or visual masks. As we have recently reported, a compression of visual space, illustrated by displacements of shortly flashed stimuli, can be observed in the temporal vicinity of masking stimuli during ocular fixation (Zimmermann et al., 2013). We here aimed at investigating the neural mechanisms underlying these displacements using functional magnetic resonance imaging. On the behavioral level, we detected significant stimulus displacement when visual masks were simultaneously presented. At the neural level, we observed decreased human motion complex V5/MT+ activation associated with these displacements: When comparing trials with a perceived stimulus shift in space to trials of veridical perception of stimulus localization, human V5/MT+ was significantly less activated although no differences in perceived motion can account for this. Data suggest an important role of human V5/MT+ in the process of spatial localization of briefly presented objects and thus extend current concepts of the functions of human V5/MT+.

Oscillatory Correlates of Visual Consciousness

Conscious experiences are linked to activity in our brain: the neural correlates of consciousness (NCC). Empirical research on these NCCs covers a wide range of brain activity signals, measures, and methodologies. In this paper, we focus on spontaneous brain oscillations; rhythmic fluctuations of neuronal (population) activity which can be characterized by a range of parameters, such as frequency, amplitude (power), and phase. We provide an overview of oscillatory measures that appear to correlate with conscious perception. We also discuss how increasingly sophisticated techniques allow us to study the causal role of oscillatory activity in conscious perception (i.e., ‘entrainment’). This review of oscillatory correlates of consciousness suggests that, for example, activity in the alpha-band (7–13 Hz) may index, or even causally support, conscious perception. But such results also showcase an increasingly acknowledged difficulty in NCC research; the challenge of separating neural activity necessary for conscious experience to arise (prerequisites) from neural activity underlying the conscious experience itself (substrates) or its results (consequences).

Neural mechanisms underlying sound-induced visual motion perception: An fMRI study

Studies of crossmodal interactions in motion perception have reported activation in several brain areas, including those related to motion processing and/or sensory association, in response to multimodal (e.g., visual and auditory) stimuli that were both in motion. Recent studies have demonstrated that sounds can trigger illusory visual apparent motion to static visual stimuli (sound-induced visual motion: SIVM): A visual stimulus blinking at a fixed location is perceived to be moving laterally when an alternating left-right sound is also present. Here, we investigated brain activity related to the perception of SIVM using a 7T functional magnetic resonance imaging technique. Specifically, we focused on the patterns of neural activities in SIVM and visually induced visual apparent motion (VIVM). We observed shared activations in the middle occipital area (V5/hMT), which is thought to be involved in visual motion processing, for SIVM and VIVM. Moreover, as compared to VIVM, SIVM resulted in greater activation in the superior temporal area and dominant functional connectivity between the V5/hMT area and the areas related to auditory and crossmodal motion processing. These findings indicate that similar but partially different neural mechanisms could be involved in auditory-induced and visually-induced motion perception, and neural signals in auditory, visual, and, crossmodal motion processing areas closely and directly interact in the perception of SIVM.

Stationary Objects Flashed Periodically Appear to Move During Smooth Pursuit Eye Movement

We discovered that a white disc flashed twice at the same location appears to move during smooth pursuit eye tracking in the direction opposite to that of the eye movement. We called this novel phenomenon movement-induced apparent motion (MIAM). Using the method of constant stimuli, we measured the required displacement of the second appearance of the disc in the pursuit direction to null the effect during the closed-loop stage of smooth pursuit eye tracking. We observed a strong linear relationship between the points of subjective stationarity and the inter-stimuli intervals for four smooth pursuit eye movement speeds. The slopes and y-intercepts of these linear fits were well predicted by the hypothesis according to which subjects saw illusory motion from the first to the second retinal projections of the flashed disc during smooth pursuit eye movement, with no extra-retinal signal compensation.

Integration of visual and tactile information in reproduction of traveled distance

In the natural world, self-motion always stimulates several different sensory modalities. Here we investigated the interplay between a visual optic flow stimulus simulating self-motion and a tactile stimulus (air flow resulting from self-motion) while human observers were engaged in a distance reproduction task. We found that adding congruent tactile information (i.e., speed of the air flow and speed of visual motion are directly proportional) to the visual information significantly improves the precision of the actively reproduced distances. This improvement, however, was smaller than predicted for an optimal integration of visual and tactile information. In contrast, incongruent tactile information (i.e., speed of the air flow and speed of visual motion are inversely proportional) did not improve subjects' precision indicating that incongruent tactile information and visual information were not integrated. One possible interpretation of the results is a link to properties of neurons in the ventral intraparietal area that have been shown to have spatially and action-congruent receptive fields for visual and tactile stimuli.NEW & NOTEWORTHY This study shows that tactile and visual information can be integrated to improve the estimates of the parameters of self-motion. This, however, happens only if the two sources of information are congruent-as they are in a natural environment. In contrast, an incongruent tactile stimulus is still used as a source of information about self-motion but it is not integrated with visual information.

High resolution data analysis strategies for mesoscale human functional MRI at 7 and 9.4T

The advent of ultra-high field functional magnetic resonance imaging (fMRI) has greatly facilitated submillimeter resolution acquisitions (voxel volume below (1 mm³)), allowing the investigation of cortical columns and cortical depth dependent (i.e. laminar) structures in the human brain. Advanced data analysis techniques are essential to exploit the information in high resolution functional measures. In this article, we use recent, exemplary 9.4 T human functional and anatomical data to review the advantages and disadvantages of (1) pooling high resolution data across regions of interest for cortical depth profile analysis, (2) pooling across cortical depths for mapping patches of cortex while discarding depth-dependent (i.e. columnar) effects, and (3) isotropic sampling without pooling to assess individual voxel’s responses. A set of cortical depth meshes may be a solution to sampling information tangentially while keeping correspondence across depths. For quantitative analysis of the spatial organization in fine-grained structures, a cortical grid approach is advantageous. We further extend this general framework by combining it with a previously introduced cortical layer volume-preserving (equi-volume) approach. This framework can readily accommodate the research questions which allow for spatial smoothing within or across layers. We demonstrate and discuss that equi-volume sampling yields a slight advantage over equidistant sampling given the current limitations of fMRI voxel size, participant motion, coregistration and segmentation. Our 9.4 T human anatomical and functional data indicate the advantage over lower fields including 7 T and demonstrate the practical applicability of T₂^* and T₂-weighted fMRI acquisitions.

•

High resolution regular cortical grids are advantageous for local applications.

•

Equi-volume sampling is slightly advantageous over equidistant sampling in-vivo.

•

Isotropic submillimeter cortical sampling without spatial pooling requires high SNR.

•

9.4 T human T2 and T2* BOLD fMRI are practically feasible and provide high SNR.

•

9.4 T T2*-weighted 0.35 mm iso. res. anatomical images for laminar contrast in vivo.

High resolution anatomical and quantitative MRI of the entire human occipital lobe ex vivo at 9.4T

Several magnetic resonance imaging (MRI) contrasts are sensitive to myelin content in gray matter in vivo which has ignited ambitions of MRI-based in vivo cortical histology. Ultra-high field (UHF) MRI, at fields of 7 T and beyond, is crucial to provide the resolution and contrast needed to sample contrasts over the depth of the cortex and get closer to layer resolved imaging. Ex vivo MRI of human post mortem samples is an important stepping stone to investigate MRI contrast in the cortex, validate it against histology techniques applied in situ to the same tissue, and investigate the resolutions needed to translate ex vivo findings to in vivo UHF MRI. Here, we investigate key technology to extend such UHF studies to large human brain samples while maintaining high resolution, which allows investigation of the layered architecture of several cortical areas over their entire 3D extent and their complete borders where architecture changes. A 16 channel cylindrical phased array radiofrequency (RF) receive coil was constructed to image a large post mortem occipital lobe sample (~80×80×80 mm³) in a wide-bore 9.4 T human scanner with the aim of achieving high-resolution anatomical and quantitative MR images. Compared with a human head coil at 9.4 T, the maximum Signal-to-Noise ratio (SNR) was increased by a factor of about five in the peripheral cortex. Although the transmit profile with a circularly polarized transmit mode at 9.4 T is relatively inhomogeneous over the large sample, this challenge was successfully resolved with parallel transmit using the kT-points method. Using this setup, we achieved 60μm anatomical images for the entire occipital lobe showing increased spatial definition of cortical details compared to lower resolutions. In addition, we were able to achieve sufficient control over SNR, B₀ and B₁ homogeneity and multi-contrast sampling to perform quantitative T₂* mapping over the same volume at 200 μm. Markov Chain Monte Carlo sampling provided maximum posterior estimates of quantitative T₂* and their uncertainty, allowing delineation of the stria of Gennari over the entire length and width of the calcarine sulcus. We discuss how custom RF receive coil arrays built to specific large post mortem sample sizes can provide a platform for UHF cortical layer-specific quantitative MRI over large fields of view.

•

Custom-built 16 channel 9.4 T RF-coil to image large post mortem samples at high resolution.

•

Parallel transmit techniques allow homogenization of B₁⁺ for 3D GRE imaging at UHF.

•

60 μm anatomical MRI of the entire human occipital lobe.

•

200 μm isotropic quantitative T₂^* mapping of the entire human occipital lobe.

•

A platform for future UHF cortical layer specific qMRI over large FoVs.

Neural Synergy Between Kinetic Vision and Touch

Ambiguous visual information often produces unstable visual perception. In four psychophysical experiments, we found that unambiguous tactile information about the direction of rotation of a globe whose three-dimensional structure is ambiguous significantly influences visual perception of the globe. This disambiguation of vision by touch occurs only when the two modalities are stimulated concurrently, however. Using functional magnetic resonance imaging, we discovered that touching the rotating globe, even when not looking at it, reliably activates the middle temporal visual area (MT+), a brain region commonly thought to be crucially involved in registering structure from motion. Considered together, our results show that the brain draws on somatosensory information to resolve visual conflict.

Apparent Motion Suppresses Responses in Early Visual Cortex: A Population Code Model

Two stimuli alternately presented at different locations can evoke a percept of a stimulus continuously moving between the two locations. The neural mechanism underlying this apparent motion (AM) is thought to be increased activation of primary visual cortex (V1) neurons tuned to locations along the AM path, although evidence remains inconclusive. AM masking, which refers to the reduced detectability of stimuli along the AM path, has been taken as evidence for AM-related V1 activation. AM-induced neural responses are thought to interfere with responses to physical stimuli along the path and as such impair the perception of these stimuli. However, AM masking can also be explained by predictive coding models, predicting that responses to stimuli presented on the AM path are suppressed when they match the spatio-temporal prediction of a stimulus moving along the path. In the present study, we find that AM has a distinct effect on the detection of target gratings, limiting the maximum performance at high contrast levels. This masking is strongest when the target orientation is identical to the orientation of the inducers. We developed a V1-like population code model of early visual processing, based on a standard contrast normalization model. We find that AM-related activation in early visual cortex is too small to either cause masking or to be perceived as motion. Our model instead predicts strong suppression of early sensory responses during AM, consistent with the theoretical framework of predictive coding.

Two spatially separate stimuli presented in rapid succession often induce the illusory perception of a moving stimulus (apparent motion or AM). Its underlying mechanism is thought to be increased activation in primary visual cortex representing the motion path. Indirect evidence for this account comes from the reduced detectability of stimuli presented along the motion path (AM masking). Here, we developed a computational model of AM-related effects on visual processing in early visual cortex, which predicted a neural activation that is too small to either account for the observed masking or the perception of motion. Instead, our model predicts strong suppression of neural responses to stimuli presented along the motion path, especially when they match the spatio-temporal prediction of a stimulus moving along the path. Our findings support predictive coding models of visual processing, in which higher-level predictions about motion explain away lower-level responses to expected sensory input.