Forecasting brain activity based on models of spatiotemporal brain dynamics: A comparison of graph neural network architectures

Comprehending the interplay between spatial and temporal characteristics of neural dynamics can contribute to our understanding of information processing in the human brain. Graph neural networks (GNNs) provide a new possibility to interpret graph-structured signals like those observed in complex brain networks. In our study we compare different spatiotemporal GNN architectures and study their ability to model neural activity distributions obtained in functional MRI (fMRI) studies. We evaluate the performance of the GNN models on a variety of scenarios in MRI studies and also compare it to a VAR model, which is currently often used for directed functional connectivity analysis. We show that by learning localized functional interactions on the anatomical substrate, GNN-based approaches are able to robustly scale to large network studies, even when available data are scarce. By including anatomical connectivity as the physical substrate for information propagation, such GNNs also provide a multimodal perspective on directed connectivity analysis, offering a novel possibility to investigate the spatiotemporal dynamics in brain networks.

A graph neural network framework for causal inference in brain networks

A central question in neuroscience is how self-organizing dynamic interactions in the brain emerge on their relatively static structural backbone. Due to the complexity of spatial and temporal dependencies between different brain areas, fully comprehending the interplay between structure and function is still challenging and an area of intense research. In this paper we present a graph neural network (GNN) framework, to describe functional interactions based on the structural anatomical layout. A GNN allows us to process graph-structured spatio-temporal signals, providing a possibility to combine structural information derived from diffusion tensor imaging (DTI) with temporal neural activity profiles, like that observed in functional magnetic resonance imaging (fMRI). Moreover, dynamic interactions between different brain regions discovered by this data-driven approach can provide a multi-modal measure of causal connectivity strength. We assess the proposed model's accuracy by evaluating its capabilities to replicate empirically observed neural activation profiles, and compare the performance to those of a vector auto regression (VAR), like that typically used in Granger causality. We show that GNNs are able to capture long-term dependencies in data and also computationally scale up to the analysis of large-scale networks. Finally we confirm that features learned by a GNN can generalize across MRI scanner types and acquisition protocols, by demonstrating that the performance on small datasets can be improved by pre-training the GNN on data from an earlier study. We conclude that the proposed multi-modal GNN framework can provide a novel perspective on the structure-function relationship in the brain. Accordingly this approach appears to be promising for the characterization of the information flow in brain networks.

Hybridizing EMD with cICA for fMRI Analysis of Patient Groups

Independent component analysis (ICA), as a data driven method, has shown to be a powerful tool for functional magnetic resonance imaging (fMRI) data analysis. One drawback of this multivariate approach is, that it is naturally not convenient for analysis of group studies. Therefore various techniques have been proposed in order to overcome this limitation of ICA. In this paper a novel ICA based work-flow for extracting resting state networks from fMRI group studies is proposed. An empirical mode decomposition (EMD) is used to generate reference signals in a data driven manner, which can be incorporated into a constrained version of ICA (cICA), what helps to overcome the inherent ambiguities. The results of the proposed workflow are then compared to those obtained by a widely used group ICA approach. It is demonstrated that intrinsic modes, extracted by EMD, are suitable to serve as references for cICA to obtain typical resting state patterns, which are consistent over subjects. This novel processing pipeline makes it transparent for the user, how comparable activity patterns across subjects emerge, and also the trade-off between similarity across subjects and preserving individual features can be well adjusted and adapted for different requirements in the new work-flow.

Spurious correlations in simultaneous EEG-fMRI driven by in-scanner movement

Simultaneous EEG-fMRI provides an increasingly attractive research tool to investigate cognitive processes with high temporal and spatial resolution. However, artifacts in EEG data introduced by the MR scanner still remain a major obstacle. This study, employing commonly used artifact correction steps, shows that head motion, one overlooked major source of artifacts in EEG-fMRI data, can cause plausible EEG effects and EEG-BOLD correlations. Specifically, low-frequency EEG (<20Hz) is strongly correlated with in-scanner movement. Accordingly, minor head motion (<0.2mm) induces spurious effects in a twofold manner: Small differences in task-correlated motion elicit spurious low-frequency effects, and, as motion concurrently influences fMRI data, EEG-BOLD correlations closely match motion-fMRI correlations. We demonstrate these effects in a memory encoding experiment showing that obtained theta power (~3-7Hz) effects and channel-level theta-BOLD correlations reflect motion in the scanner. These findings highlight an important caveat that needs to be addressed by future EEG-fMRI studies.

[The eye as a window to the pathophysiology in Parkinson's syndromes]

Although dysfunction of the visual system and dysfunctional eye movements during sporadic Parkinson's disease have been reported for more than 40 years, they have never been the focus of early and/or differential diagnosis. To date Parkinson's disease-related α-synuclein aggregates, i.e., Lewy pathology, are not known to develop either in the retina or in other components of the visual system. In a clinical context it is currently possible to test the involvement of the respective functional systems by means of optical coherence tomography and video oculography. Moreover, non-motor-related abnormalities are detectable both during psychophysical testing of visuospatial function as well as in the form of measurable deficits of color perception. These deficits of the visual and oculomotor systems could prove to be suitable candidates for diagnosing sporadic Parkinson's disease in its early phase in a non-invasive manner. This article is intended to provide an overview of the fundamental pathophysiological principles and clinical aspects of visual system involvement in sporadic Parkinson's disease together with currently available differential diagnostic options.

Juggling revisited - a voxel-based morphometry study with expert jugglers

Juggling is a highly interesting tool to investigate neuroplasticity associated with motor-learning. Several brain-imaging studies have reported changes in regional brain morphology in visual association cortices in individuals learning how to juggle a three-ball cascade. However, to our knowledge there are no studies that investigated expert jugglers, looking for specific features in regional brain morphology related to this highly specialized skill. Using T1-weighted images and voxel-based morphometry we investigated in a cross-sectional study design 16 expert jugglers, able to juggle at least five balls and an age- and gender-matched group of non-jugglers. We hypothesized that expert jugglers would show higher gray matter density in regions involved in visual motion perception and eye-hand coordination. Images were pre-processed and analyzed using SPM8. Age was included in the analyses as covariate of no interest. As compared to controls jugglers displayed several clusters of higher, regional gray matter density in the occipital and parietal lobes including the secondary visual cortex, the hMT+/V5 area bilaterally and the intraparietal sulcus bilaterally. Within the jugglers group we also found a correlation between performance and regional gray matter density in the right hMT+/V5 area. Our study provides evidence that expert jugglers show increased gray matter density in brain regions involved in visual motion perception and eye-hand coordination, i.e. brain areas that have previously been shown to undergo dynamic changes in terms of gray matter increases in subjects learning a basic three-ball cascade. The extent to which transient increases in beginners and the differences in experts and non-experts are based on the same neurobiological correlates remains to be fully elucidated.

Neural correlates of saccadic inhibition in healthy elderly and patients with amnestic mild cognitive impairment

Performance on tasks that require saccadic inhibition declines with age and altered inhibitory functioning has also been reported in patients with Alzheimer's disease. Although mild cognitive impairment (MCI) is assumed to be a high-risk factor for conversion to AD, little is known about changes in saccadic inhibition and its neural correlates in this condition. Our study determined whether the neural activation associated with saccadic inhibition is altered in persons with amnestic mild cognitive impairment (aMCI). Functional magnetic resonance imaging (fMRI) revealed decreased activation in parietal lobe in healthy elderly persons compared to young persons and decreased activation in frontal eye fields in aMCI patients compared to healthy elderly persons during the execution of anti-saccades. These results illustrate that the decline in inhibitory functions is associated with impaired frontal activation in aMCI. This alteration in function might reflect early manifestations of AD and provide new insights in the neural activation changes that occur in pathological ageing.

fMRI evidence for sensorimotor transformations in human cortex during smooth pursuit eye movements

Smooth pursuit eye movements (SP) are driven by moving objects. The pursuit system processes the visual input signals and transforms this information into an oculomotor output signal. Despite the object's movement on the retina and the eyes' movement in the head, we are able to locate the object in space implying coordinate transformations from retinal to head and space coordinates. To test for the visual and oculomotor components of SP and the possible transformation sites, we investigated three experimental conditions: (I) fixation of a stationary target with a second target moving across the retina (visual), (II) pursuit of the moving target with the second target moving in phase (oculomotor), (III) pursuit of the moving target with the second target remaining stationary (visuo-oculomotor). Precise eye movement data were simultaneously measured with the fMRI data. Visual components of activation during SP were located in the motion-sensitive, temporo-parieto-occipital region MT+ and the right posterior parietal cortex (PPC). Motor components comprised more widespread activation in these regions and additional activations in the frontal and supplementary eye fields (FEF, SEF), the cingulate gyrus and precuneus. The combined visuo-oculomotor stimulus revealed additional activation in the putamen. Possible transformation sites were found in MT+ and PPC. The MT+ activation evoked by the motion of a single visual dot was very localized, while the activation of the same single dot motion driving the eye was rather extended across MT+. The eye movement information appeared to be dispersed across the visual map of MT+. This could be interpreted as a transfer of the one-dimensional eye movement information into the two-dimensional visual map. Potentially, the dispersed information could be used to remap MT+ to space coordinates rather than retinal coordinates and to provide the basis for a motor output control. A similar interpretation holds for our results in the PPC region.

Differences in cortical activation during smooth pursuit and saccadic eye movements following cerebellar lesions

Current evidence supports the proposal that the cerebellum mediates the activity of other brain areas involved in the control of eye movements. Most of the evidence so far has concentrated on the vermis and flocculi as the cerebellar agents of oculomotor control. But there is also evidence for an involvement of the cerebellar hemispheres in eye movement control. Straube et al. (Ann Neurol 42:891-898, 1997) showed that lateral hemispheric lesions affect initiation of smooth pursuit (SPEM) and saccadic eye movements. Ron and Robinson (J Neurophysiol 36:1004-1022, 1973) evoked smooth pursuit and saccadic eye movements by electrical stimulation of crus I and II, as well as in the dentate nuclei of the monkey. Functional MRI studies also provide evidence that the cerebellar hemispheres play a significant role in SPEM and saccadic eye movements. To clarify the role of the cerebral hemispheres in eye movement control we compared the eye movement related blood oxygen level dependent (BOLD) responses of 12 patients with cerebellar lesions due to stroke with those of an aged-matched healthy control group. Six patients showed oculomotor abnormalities such as dysmetric saccades or saccadic SPEM during the experiment. The paradigm consisted of alternating blocks of fixation, visually guided saccades and visually guided SPEM. A nonparametric random-effects group analysis showed a degraded pattern of activation in the patient group during the performance of SPEM and saccadic eye movements in posterior parietal areas putatively containing the parietal eye fields.

Impaired working-memory after cerebellar infarcts paralleled by changes in BOLD signal of a cortico-cerebellar circuit

A considerable body of evidence supports the notion that cerebellar lesions lead to neuropsychological deficits, including impairments in working-memory, executive tasks and verbal fluency. Studies employing functional magnetic resonance imaging (fMRI) and anatomical tracing in primates provide evidence for a cortico-cerebellar circuitry as the functional substrate of working-memory. The present fMRI study explores the activation pattern during an n-back working-memory task in patients with an isolated cerebellar infarct. To determine each patient's cognitive impairment, neuropsychological tests of working-memory and attention were carried out. We conducted fMRI in nine patients and nine healthy age-matched controls while they performed a 2-back task in a blocked-design. In both groups we found bilateral activations in a widespread cortico-cerebellar network, consisting of the ventrolateral prefrontal cortex (BA 44, 45), dorsolateral prefrontal cortex (BA 9, 46), parietal cortex (BA 7, 40), pre-supplementary motor area (BA 6) anterior cingulate (BA 32). Relative to healthy controls, patients with isolated cerebellar infarcts demonstrated significantly more pronounced BOLD-activations in the precuneus and the angular gyrus during the 2-back task. The significant increase in activation in the posterior parietal areas of the cerebellar patients could be attributed to a compensatory recruitment to maintain task performance. We conclude that cerebellar lesions affect remote cortical regions that are part of a putative cortico-cerebellar network.

Functional magnetic resonance imaging evidence for binocular interactions in human visual cortex

Using functional magnetic resonance imaging (fMRI), we explored the binocular interactions occurring when subjects viewed dichoptically presented checkerboard stimuli. A flickering radial checkerboard was presented to each eye of the subject, while T2*-weighted images were acquired over the visual cortex with gradient-echo, echoplanar sequences. We compared responses in striate and extrastriate visual cortex under four conditions: both eyes were stimulated at the same time (binocular condition), each eye was stimulated in alternation (monocular condition) or first the one eye then the other eye was stimulated (left eye first - right eye trailing, or vice versa). The results indicate that only the striate area, in and near the calcarine fissure, shows significant differences for these stimulation conditions. These differences are not evident in more remote extrastriate or associational visual areas, although the BOLD response in the stimulation-rest comparison was robust. These results suggest that the effect could be related to inhibitory interactions across ocular dominance columns in striate visual cortex.

fMRI response during visual motion stimulation in patients with late whiplash syndrome

After whiplash trauma, up to one fourth of patients develop chronic symptoms including head and neck pain and cognitive disturbances. Resting perfusion single-photon-emission computed tomography (SPECT) found decreased temporoparietooccipital tracer uptake among these long-term symptomatic patients with late whiplash syndrome. As MT/MST (V5/V5a) are located in that area, this study addressed the question whether these patients show impairments in visual motion perception. We examined five symptomatic patients with late whiplash syndrome, five asymptomatic patients after whiplash trauma, and a control group of seven volunteers without the history of trauma. Tests for visual motion perception and functional magnetic resonance imaging (fMRI) measurements during visual motion stimulation were performed. Symptomatic patients showed a significant reduction in their ability to perceive coherent visual motion compared with controls, whereas the asymptomatic patients did not show this effect. fMRI activation was similar during random dot motion in all three groups, but was significantly decreased during coherent dot motion in the symptomatic patients compared with the other two groups. Reduced psychophysical motion performance and reduced fMRI responses in symptomatic patients with late whiplash syndrome both point to a functional impairment in cortical areas sensitive to coherent motion. Larger studies are needed to confirm these clinical and functional imaging results to provide a possible additional diagnostic criterion for the evaluation of patients with late whiplash syndrome.

Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex

Functional magnetic resonance imaging (fMRI) was used to estimate the average receptive field sizes of neurons in each of several striate and extrastriate visual areas of the human cerebral cortex. The boundaries of the visual areas were determined by retinotopic mapping procedures and were visualized on flattened representations of the occipital cortex. Estimates of receptive field size were derived from the temporal duration of the functional activation at each cortical location as a visual stimulus passed through the receptive fields represented at that location. Receptive fields are smallest in the primary visual cortex (V1). They are larger in V2, larger again in V3/VP and largest of all in areas V3A and V4. In all these areas, receptive fields increase in size with increasing stimulus eccentricity. The results are qualitatively in line with those obtained by others in macaque monkeys using neurophysiological methods.

Relationship between saccadic eye movements and cortical activity as measured by fMRI: quantitative and qualitative aspects

We investigated the quantitative relationship between saccadic activity (as reflected in frequency of occurrence and amplitude of saccades) and blood oxygenation level dependent (BOLD) changes in the cerebral cortex using functional magnetic resonance imaging (fMRI). Furthermore, we investigated quantitative changes in cortical activity associated with qualitative changes in the saccade task for comparable levels of saccadic activity. All experiments required the simultaneous acquisition of eye movement and fMRI data. For this purpose we used a new high-resolution limbus-tracking technique for recording eye movements in the magnetic resonance tomograph. In the first two experimental series we varied both frequency and amplitude of saccade stimuli (target jumps). In the third series we varied task difficulty; subjects performed either pro-saccades or anti-saccades. The brain volume investigated comprised the frontal and supplementary eye fields, parietal as well as striate cortex, and the motion sensitive area of the parieto-occipital cortex. All these regions showed saccade-related BOLD responses. The responses in these regions were highly correlated with saccade frequency, indicating that repeated processing of saccades is integrated over time in the BOLD response. In contrast, there was no comparable BOLD change with variation of saccade amplitude. This finding speaks for a topological rather than activity-dependent coding of saccade amplitudes in most cortical regions. In the experiments comparing pro- vs anti-saccades we found higher BOLD activation in the "anti" task than in the "pro" task. A comparison of saccade parameters revealed that saccade frequency and cumulative amplitude were comparable between the two tasks, whereas reaction times were longer in the "anti" task than the pro task. The latter finding is taken to indicate a more demanding cortical processing in the "anti" task than the "pro" task, which could explain the observed difference in BOLD activation. We hold that a quantitative analysis of saccade parameters (especially saccade frequency and latency) is important for the interpretation of the BOLD changes observed with visual stimuli in fMRI.

Functional MRI in patients with band heterotopia

Functional activation associated with a motor task (fist movements) was studied in three patients with band heterotopias by fMRI. In two patients, additional visual fMRI studies were performed using a flickering checkerboard stimulus. In all patients activation of the outer cortex and of the inner neuronal band could be found during performance of the motor task. Visual stimulation elicited a normal activation pattern without activation of the ectopic neuronal layer in one patient; in another patient activation extended toward the ventricular wall, i.e., along the route of embryonic neuronal migration. The potential participation of ectopic neuronal tissue in physiologic cerebral functions is of clinical impact in patients with neuronal heterotopias suffering from medically intractable seizures prior to epilepsy surgery.

What limits simultaneous discrimination accuracy?

Discrimination accuracy decreases when viewers simultaneously monitor two perceptually distinct stimulus components for changes in a common property, e.g. contrast [Magnussen & Greenlee (1997). Journal of Experimental Psychology: Human Perception and Performance, 23, 1603-1616; Olzak & Wickens (1997). Perception, 26, 1101-1120]. We ask whether the limitation is in monitoring two components or in making dual decisions about a single property. Using the same uncertainty paradigm as Magnussen and Greenlee, we find no evidence of a processing limitation when viewers simultaneously monitor one component (1.25 c/d) for a possible change in contrast and a second component (5 c/d) for a possible change in spatial frequency, regardless of whether the components are spatially separated or superimposed. The limitation is in making dual decisions about a single property.

Visual search and visual working memory in patients with chronic focal cortical lesions

Visually guided behavior is known to involve temporo-parietal, inferotemporal, and prefrontal cortex and each of these areas appears to contribute to visual working memory. We explored the extent to which chronic lesions in one of these cortical areas affect visually guided oculomotor performance. We also explore whether possible impairments become more pronounced with increasing memory load. With this aim we recorded saccadic eye movements in 19 patients with a chronic focal postsurgical lesion in either temporo-parietal, inferior temporal or prefrontal cortex. Their results are compared to those of 19 age-matched volunteers. The subjects performed three different visual search tasks with increasing memory load: Instructed search, cue-guided search and memory-guided search. In addition, the latter task was performed with a short (1 s) and a long (6 s) delay. All tasks required the subjects to make a saccade to a single target presented together with one or three distractors. The results indicate that patients with inferotemporal lesions make the most task-related errors. Saccadic reaction times (SRTs) were significantly prolonged in patients with temporo-parietal and prefrontal lesions, but were unaffected in the patients with lesions in the inferotemporal cortex. The spatial accuracy of saccades was lowest in patients with temporo-parietal lesions. An increase in memory load led to more errors, to longer reaction times and to lower saccadic precision. However, the effect was similar across the three patient groups and the controls. An error analysis indicated that both patients and controls tended to weight global (luminance contrast and form) features higher than local features (line-segment orientation) when making difficult perceptual decisions.

Changes in cortical activation during mirror reading before and after training: an fMRI study of procedural learning

The neural correlates of procedural learning were studied using functional magnetic resonance imaging (fMRI) and the mirror reading paradigm. The aim of the study was to investigate a presumed learning-related change of activation in cortical areas that are involved in the performance of a nonmotor skill. Changes in cortical blood oxygenation contrast were recorded in 10 healthy subjects while they alternatively read visually presented single mirror script words and normal script words. Responses in naive subjects were compared to those acquired after training of mirror script reading. The acquisition volume included the motor and premotor cortex, the parietal lobe and the occipital lobe including its inferior aspects. Striate and extrastriate visual areas, associative parietal cortex and the premotor cortex were bilaterally active during normal and mirror script reading. Significantly stronger activation during mirror reading was seen in BA7 and 40 (parietal associative cortex) and in BA6 (corresponding to the frontal eye fields). Simultaneous eye movement recordings indicated that activation in BA6 was related to processing components other than saccade frequency. After training, BA6 and BA7 exhibited a decrease of activation during mirror reading that significantly exceeded nonspecific changes observed in the normal script control condition. The present findings confirm the hypothesis of practice-related decrease of activation in task-related cortical areas during nonmotor procedural learning.

Cortical activation evoked by visual mental imagery as measured by fMRI

One of the major controversies in cognitive neuroscience is whether the primary visual cortex and nearby areas are involved in visual mental imagery. In an fMRI study we examined the brain activity of 10 healthy subjects under different task conditions: in the perception condition subjects saw complex geometrical shapes and had to decide whether other highlighted stimuli fell inside or outside the figure. In the imagery condition subjects saw only the highlighted stimuli and were instructed to imagine the previously studied geometrical shapes to solve the same task. Although the behavioral data show a distance effect that would be expected based on topographically organized mental images, the functional imaging data do not show increased activity in the primary visual cortex in the imagery condition. In the occipital cortex a slightly increased activity was found only in the visual association cortex (BA 19), whereas the highest activation was observed in the parietal cortex (BA 7 and 40). The results of the study do not support the assumption that the primary visual cortex is involved in visual mental imagery, but rather that a network of spatial subsystems and higher visual areas appears to be involved.

Spatiotemporal frequency and direction sensitivities of human visual areas measured using fMRI

Using functional magnetic resonance imaging (fMRI) we have studied the variation in response magnitude, in each visual area (V1-V5), as a function of spatial frequency (SF), temporal frequency (TF) and unidirectional motion versus counterphase flicker. Each visual area was identified in each subject using a combination of retinotopic mapping fMRI and cortical flattening techniques. A drifting (or counterphasing) sinusoidal grating was used as the stimulus in a study in which we parametrically varied SF between 0.4 and 7 cycles/degree and TF between 0 and 18 Hz. For each experiment we constructed fMRI amplitude tuning curves, averaged across subjects, for each visual area. The tuning curves that resulted are consistent with the known physiological properties of cells in the corresponding macaque visual areas, previous functional imaging studies, and in the case of V1, the psychophysically determined contrast sensitivity functions for spatial and temporal frequency. In the case of V3A, the SF tuning functions obtained were more similar to those found in single cell studies of macaque V3 rather than macaque V3A. All areas showed at least a moderate preference for directed versus counterphasing motion with V5 showing the largest preference. Visual areas V1, V2, V3, and V3A showed more direction sensitivity at low spatial frequencies, while VP, V4, and V5 had the highest drifting versus counterphasing ratios for higher spatial frequencies.

Brain activation during dichoptic presentation of optic flow stimuli

The processing of optic flow fields in motion-sensitive areas in human visual cortex was studied with BOLD (blood oxygen level dependent) contrast in functional magnetic resonance imaging (fMRI). Subjects binocularly viewed optic flow fields in plane (monoptic) or in stereo depth (dichoptic) with various degrees of disparity and increasing radial speed. By varying the directional properties of the stimuli (expansion, spiral motion, random), we explored whether the BOLD effect reflected neuronal responses to these different forms of optic flow. The results suggest that BOLD contrast as assessed by fMRI methods reflects the neural processing of optic flow information in motion-sensitive cortical areas. Furthermore, small but replicable disparity-selective responses were found in parts of Brodmann's area 19.

Visual memory for random block patterns defined by luminance and color contrast

We studied the ability of human subjects to memorize the visual information in computer-generated random block patterns defined either by luminance contrast, by color contrast, or by both. Memory performance declines rapidly with increasing inter-stimulus interval, showing a half-life of approximately 3 s. We further show that memory performance declines with eccentricity approximately as a Gaussian function of position. Memory decay functions did not depend on whether the patterns were defined by luminance or color contrast. Changing both luminance and color components of block patterns in conjunction did not improve performance suggesting a single memory mechanism is used to store luminance and color derived pattern information. Our results further suggest that color identity (hue, saturation) and pattern information extracted from color- or luminance-contrast are stored independently of each other.

Brain regions involved in spatial frequency discrimination: evidence from fMRI

The cortical areas underlying successive spatial-frequency discrimination were explored using functional magnetic resonance imaging (fMRI). In a steady-state, block-design paradigm, 12 subjects viewed a single fixation cross during a rest period, followed by an activation period consisting of the presentation of horizontal (distractors) and vertical (targets) sinewave gratings. Two tasks were performed: in the control task, subjects pressed a button after the second vertical grating was presented within each trial; in the discrimination task, subjects decided which target grating had the higher spatial frequency. Post-processing consisted of off-line image registration to correct for head motion, spatial and temporal smoothing, and cross-correlation between each voxel time course and a phase-shifted stimulus time profile. The results indicate that striate, extrastriate, parietal, and prefrontal areas show significant BOLD (blood oxygen level dependent) effects during both discrimination and control tasks, with consistently higher activity levels in the discrimination task.

Motion VEPs with simultaneous measurement of perceived velocity

The dependency of the N200 amplitude of the motion-onset VEP evoked by a parafoveal grating of variable speed (0.25-13.5 deg/s, corresponding to 0.5-27 Hz) and constant contrast (4%) was studied. Additional measurements were made with parafoveally presented gratings of constant speed (2 deg/s, corresponding to 4 Hz) and a variable contrast (0.5-64%) before and after adaptation to a stationary or drifting grating. In this latter experiment, simultaneous psychophysical measurements were made of the perceived speed. The amplitude of the N200 wave increased with increasing stimulus speed within the slow speed range up to 1.5 deg/s (corresponding to 3 Hz). Adaptation to a stationary grating had no significant effect on the relationship between the N200 amplitude and stimulus contrast. Contrary to this, adaptation to a slowly drifting grating (1 deg/s, corresponding to 2 Hz) or to a rapidly drifting grating (4 deg/s, corresponding to 8 Hz) reduced the N200 amplitude significantly. Adaptation to a stationary grating slightly reduced the perceived speed of subsequently viewed gratings. Adaptation to a slowly drifting grating increased the perceived speed of the subsequently viewed gratings, whereas adaptation to a rapidly drifting grating decreased the perceived speed. The findings can be best explained by a two-channel model of speed perception. While the motion VEP reflects the sum of both channel activities, the psychophysical measures point to the antagonistic encoding of low and high velocities.

Similarities and dissimilarities between pattern VEPs and motion VEPs

The contrast response functions (CRF) of pattern-appearance and motion-onset VEPs for periodic stimuli (gratings) were compared. The CRF for pattern-appearance is accelerative for the P100 component and compressive for the N200 component. Contrary to these results, the CRF for motion-onset shows an almost negligible slope for both components within the contrast range tested (0.5-64%). To better isolate the neural contributions to these different VEP components, we studied the effects of prior adaptation to stationary and moving gratings. Adaptation to stationary gratings has no effect on both VEP components for motion-onset and the P100 component for pattern-appearance, but did reduce the amplitude of the N200 for pattern-appearance. Adaptation to slow (1 deg/s) and fast (4 deg/s) gratings left the P100 amplitudes unaltered, while it significantly reduced the N200 amplitudes for both pattern-appearance and motion-onset. These results suggest that the N200 component of the motion-onset VEP is generated by motion-dependent neurons, whereas the same component for pattern-appearance arises from contrast-dependent neurons. The observed differences between P100 and N200 components appear to reflect the activity of both transient and sustained neural mechanisms.

Attentional suppression of activity in the human visual cortex

We have used fMRI to examine the nature of the changes that occur in the human visual cortex when an observer attends to a particular location in the visual image. Previous studies have shown that the magnitude of the response to a visual stimulus is increased when the observer attends to the stimulus. We show that, in addition, attention to a particular location results in a widespread suppression of activity levels at all other locations. This suggests that a key mechanism of attentional modulation may be that spontaneous (baseline) levels of neural activity are adjusted in a position-dependent manner across the entire visual field.

Visual contrast response functions in Parkinson's disease: evidence from electroretinograms, visually evoked potentials and psychophysics

OBJECTIVES

Visual contrast detection thresholds and suprathreshold contrast discrimination thresholds were compared to luminance and flash/pattern electroretinograms (ERG) and visually evoked potentials (VEP) in patients with Parkinson's disease (n = 31), patients with multiple system atrophy (n = 6), patients with progressive supranuclear palsy (n = 6) and control patients without central nervous disease (n = 33).

METHODS

The stimuli were luminance modulated full-field (flash) or horizontally oriented sinewave gratings (pattern), the latter having either a low (0.5 cycles/deg) or medium (4.0 cycles/deg) spatial frequency. Stimulus contrast ranged from 10 to 80% so that contrast response functions could be derived.

RESULTS

Contrast thresholds were higher in the patients with Parkinson's disease than in the control patients. Contrast discrimination thresholds were also somewhat elevated in patients with Parkinson's disease. Pattern ERG amplitudes were significantly reduced in patients with Parkinson's disease for the medium spatial frequency stimulus, but less for the low spatial frequency and flash stimuli.

CONCLUSIONS

Our results suggest that Parkinson's disease impairs contrast processing in the retina. VEP amplitudes did not significantly differ between the groups for the conditions tested. Patients with progressive supranuclear palsy also showed impaired contrast perception and reduced ERG amplitudes, whereas patients with multiple system atrophy were less impaired.

Human cortical areas underlying the perception of optic flow: brain imaging studies

In summary, we have reviewed electrophysiological and brain imaging studies of motion and optic-flow processing. Single-unit studies indicate that MST (V5a) is a site of optic-flow extraction and that this information can be used to guide pursuit eye movements and to estimate heading. The EEG and MEG studies point to a localized electrical dipole in occipitotemporal cortex evoked by visual motion. We have also discussed the evidence from functional imaging studies for response specificity of the rCBF and BOLD effects in posterior cortex to visual motion and optic flow. Focal attention modulates the amplitude of the BOLD signal evoked by visual motion stimulation. Retinotopic mapping techniques have been used to locate region borders within the visual cortex. Our results indicate that striate (V1) and extrastriate areas (V2, V3/V3a) respond robustly to optic flow. However, with exception of a more pronounced response in V3/V3a to random walk, we found little evidence for response selectivity with respect to flow type and disparity in these early visual areas. In a similar fashion, the human V5/V5a complex responds well to optic flow, but these responses do not vary significantly with the type of flow field and do not seem to depend on disparity. In contrast, the kinetic occipital area (KO/V3b) responds well to optic-flow information, and it is the only area that produces more pronounced activation to the disparity in the flow fields. These initial results are promising because they suggest that the fMRI method can be sensitive to changes in stimulus parameters that define flow fields. More work will be required to explore the extent to which these responses reflect the neuronal processing of optic flow. Eye position tracking is now possible during fMRI experiments. We have demonstrated that the eye movements affect the BOLD responses in motion-sensitive areas (Kimming et al., 1999). Further experiments in our laboratory are aimed at understanding the effects of eye movements on the neuronal coding of complex optic-flow fields (Schira et al., 1999).

Brain imaging in a patient with hemimicropsia

Hemimicropsia is an isolated misperception of the size of objects in one hemifield (objects appear smaller) which is, as a phenomenon of central origin, very infrequently reported in literature. We present a case of hemimicropsia as a selective deficit of size and distance perception in the left hemifield without hemianopsia caused by a cavernous angioma with hemorrhage in the right occipitotemporal area. The symptom occurred only intermittently and was considered the consequence of a local irritation by the hemorrhage. Imaging data including a volume-rendering MR data set of the patient's brain were transformed to the 3-D stereotactic grid system by Talairach and warped to a novel digital 3-D brain atlas. Imaging analysis included functional MRI (fMRI) to analyse the patient's visual cortex areas (mainly V5) in relation to the localization of the hemangioma to establish physiological landmarks with respect to visual stimulation. The lesion was localized in the peripheral visual association cortex, Brodmann area (BA) 19, adjacent to BA 37, both of which are part of the occipitotemporal visual pathway. Additional psychophysical measurements revealed an elevated threshold for perceiving coherent motion, which we relate to a partial loss of function in V5, a region adjacent to the cavernoma. In our study, we localized for the first time a cerebral lesion causing micropsia by digital mapping in Talairach space using a 3-D brain atlas and topologically related it to fMRI data for visual motion. The localization of the brain lesion affecting BA 19 and the occipitotemporal visual pathway is discussed with respect to experimental and case report findings about the neural basis of object size perception.

The psychophysics of perceptual memory

Psychophysical studies of short-term memory for attributes or dimensions of the visual stimulus known to be important in early visual processing--spatial frequency, orientation, contrast, motion--identify an early perceptual memory system. The proposed system, which may be part of the Schacter-Tulving perceptual representation system (PRS), is located early in the visual processing stream, prior to the structural description system responsible for shape priming but beyond primary visual cortex (V1), and consists of a series of parallel special-purpose perceptual mechanisms with independent but limited processing resources, where each mechanism is devoted to the analysis of a single stimulus dimension and is coupled to a memory store. The experimental evidence for this hypothesis is reviewed.

MR-eyetracker: a new method for eye movement recording in functional magnetic resonance imaging

We present a method for recording saccadic and pursuit eye movements in the magnetic resonance tomograph designed for visual functional magnetic resonance imaging (fMRI) experiments. To reliably classify brain areas as pursuit or saccade related it is important to carefully measure the actual eye movements. For this purpose, infrared light, created outside the scanner by light-emitting diodes (LEDs), is guided via optic fibers into the head coil and onto the eye of the subject. Two additional fiber optical cables pick up the light reflected by the iris. The illuminating and detecting cables are mounted in a plastic eyepiece that is manually lowered to the level of the eye. By means of differential amplification, we obtain a signal that covaries with the horizontal position of the eye. Calibration of eye position within the scanner yields an estimate of eye position with a resolution of 0.2 degrees at a sampling rate of 1000 Hz. Experiments are presented that employ echoplanar imaging with 12 image planes through visual, parietal and frontal cortex while subjects performed saccadic and pursuit eye movements. The distribution of BOLD (blood oxygen level dependent) responses is shown to depend on the type of eye movement performed. Our method yields high temporal and spatial resolution of the horizontal component of eye movements during fMRI scanning. Since the signal is purely optical, there is no interaction between the eye movement signals and the echoplanar images. This reasonably priced eye tracker can be used to control eye position and monitor eye movements during fMRI.

Spatial-frequency discrimination, brain lateralisation, and acute intake of alcohol

The effect of alcohol (breath-alcohol level of 0.1%) on perceptual discrimination of low (1.5 cycles deg-1) and high (8 cycles deg-1) spatial frequencies in the left and right visual field was measured in eighteen right-handed males, in a double-blind, balanced placebo design. Discrimination thresholds for briefly (180 ms) presented sinusoidal gratings were determined by two-alternative forced-choice judgments with four interleaving psychophysical staircases providing random trial-to-trial variation of reference spatial frequency and visual field, in addition to a random (+/- 10%) jitter of reference spatial frequency. Alcohol produced overall higher discrimination thresholds but did not alter the visual-field balance: no main effect of visual field was observed, but in both placebo and alcohol conditions spatial frequency interacted with visual field in the direction predicted by the spatial-frequency hypothesis of hemispheric asymmetry in visual-information processing, with left-visual-field/right-hemisphere superiority in discrimination of low spatial frequencies and right-visual-field/left-hemisphere superiority in discrimination of high spatial frequencies.

Impairment in preattentive visual processing in patients with Parkinson's disease

We explored the possibility of whether preattentive visual processing is impaired in Parkinson's disease. With this aim, visual discrimination thresholds for orientation texture stimuli were determined in two separate measurement sessions in 16 patients with idiopathic Parkinson's disease. The results were compared with those of 16 control subjects age-matched and 16 young healthy volunteers. Discrimination thresholds were measured in a four-alternative spatial forced-choice paradigm, in which subjects judged the location of a target embedded in a background of distractors. Four different stimulus configurations were employed: (i) a group of vertical targets among horizontal distractors ('vertical line targets'); (ii) targets with varying levels of orientation difference on a background of spatially filtered vertically oriented noise ('Gaussian filtered noise'); (iii) one 'L' among 43 '+' signs ('texton'), all of which assess preattentive visual processing; and (iv) control condition, of one 'L' among 43 'T' distractors ('non-texton' search target), which reflects attentive visual processing. In two of the preattentive tasks (filtered noise and texton), patients with Parkinson's disease required significantly greater orientation differences and longer stimulus durations, respectively. In contrast, their performance in the vertical line target and non-texton search target was comparable to that of the matched control subjects. These differences were more pronounced in the first compared with the second session. Duration of illness and age within the patient group correlated significantly with test performance. In all conditions tested, the young control subjects performed significantly better than the more elderly control group, further indicating an effect of age on this form of visual processing. The results suggest that, in addition to the well documented impairment in retinal processing, idiopathic Parkinson's disease is associated with a deficit in preattentive cortical visual processing.

Electrophysiological localization of brain regions involved in perceptual memory

Event-related potentials (ERP) were recorded during perceptual discrimination and short-term memory, varying the interstimulus interval (1-10 s) in delayed spatial frequency discrimination. Accuracy of discrimination remained unimpaired across this time interval, but choice reaction times increased. A brain source localization (BESA) model showed that the activity of the parietal and right temporal sources increased with long retention intervals in a sequential activation pattern where a long-latency component of the parietal source specific to the memory condition was observed, the latency of which matched a memory-related increase in choice reaction times in the cognitive task. It is suggested that the temporal sources are involved in encoding and storage of visual information, and the parietal source is involved in memory retrieval.

The processing of first- and second-order motion in human visual cortex assessed by functional magnetic resonance imaging (fMRI)

We have examined the activity levels produced in various areas of the human occipital cortex in response to various motion stimuli using functional magnetic resonance imaging (fMRI) methods. In addition to standard luminance-defined (first-order) motion, three types of second-order motion were used. The areas examined were the motion area V5 (MT) and the following areas that were delineated using retinotopic mapping procedures: V1, V2, V3, VP, V3A, and a new area that we refer to as V3B. Area V5 is strongly activated by second-order as well as by first-order motion. This activation is highly motion-specific. Areas V1 and V2 give good responses to all motion stimuli, but the activity seems to be related primarily to the local spatial and temporal structure in the image rather than to motion processing. Area V3 and its ventral counterpart VP also respond well to all our stimuli and show a slightly greater degree of motion specificity than do V1 and V2. Unlike V1 and V2, the response in V3 and VP is significantly greater for second-order motion than for first-order motion. This trend is evident, but less marked, in V3A and V3B and absent in V5. The results are consistent with the hypothesis that first-order motion sensitivity arises in V1, that second-order motion is first represented explicitly in V3 and VP, and that V5 (and perhaps also V3A and V3B) is involved in further processing of motion information, including the integration of motion signals of the two types.

Effect of eye movements on the magnitude of functional magnetic resonance imaging responses in extrastriate cortex during visual motion perception

We have studied the effects of pursuit eye movements on the functional magnetic resonance imaging (fMRI) responses in extrastriate visual areas during visual motion perception. Echoplanar imaging of 10-12 image planes through visual cortex was acquired in nine subjects while they viewed sequences of random-dot motion. Images obtained during stimulation periods were compared with baseline images, where subjects viewed a blank field. In a subsidiary experiment, responses to moving dots, viewed under conditions of fixation or pursuit, were compared with those evoked by static dots. Eye movements were recorded with MR-compatible electro-oculographic (EOG) electrodes. Our findings show an enhanced level of activation (as indexed by blood-oxygen level-dependent contrast) during pursuit compared with fixation in two extrastriate areas. The results support earlier findings on a motion-specific area in lateral occipitotemporal cortex (human V5). They also point to a further site of activation in a region approximately 12 mm dorsal of V5. The fMRI response in V5 during pursuit is significantly enhanced. This increased response may represent additional processing demands required for the control of eye movements.