Influence of meditation on anti-correlated networks in the brain

Human experiences can be broadly divided into those that are external and related to interaction with the environment, and experiences that are internal and self-related. The cerebral cortex appears to be divided into two corresponding systems: an "extrinsic" system composed of brain areas that respond more to external stimuli and tasks and an "intrinsic" system composed of brain areas that respond less to external stimuli and tasks. These two broad brain systems seem to compete with each other, such that their activity levels over time is usually anti-correlated, even when subjects are "at rest" and not performing any task. This study used meditation as an experimental manipulation to test whether this competition (anti-correlation) can be modulated by cognitive strategy. Participants either fixated without meditation (fixation), or engaged in non-dual awareness (NDA) or focused attention (FA) meditations. We computed inter-area correlations ("functional connectivity") between pairs of brain regions within each system, and between the entire extrinsic and intrinsic systems. Anti-correlation between extrinsic vs. intrinsic systems was stronger during FA meditation and weaker during NDA meditation in comparison to fixation (without mediation). However, correlation between areas within each system did not change across conditions. These results suggest that the anti-correlation found between extrinsic and intrinsic systems is not an immutable property of brain organization and that practicing different forms of meditation can modulate this gross functional organization in profoundly different ways.

Linear systems analysis of the fMRI signal

In 1995 when we began our investigations of the human visual system using fMRI, little was known about the temporal properties of the fMRI signal. Before we felt comfortable making quantitative estimates of neuronal responses with this new technique, we decided to first conduct a basic study of how the time-course of the fMRI response varied with stimulus timing and strength. The results ended up showing strong evidence that to a first approximation the hemodynamic transformation was linear in time. This was both important and remarkable: important because nearly all fMRI data analysis techniques assume or require linearity, and remarkable because the physiological basis of the hemodynamic transformation is so complex that we still have a far from complete understanding of it. In this paper, we provide highlights of the results of our original paper supporting the linear transform hypothesis. A reanalysis of the original data provides some interesting new insights into the published results. We also provide a detailed appendix describing of the properties and predictions of a linear system in time in the context of the transformation between neuronal responses and the BOLD signal.

Temporal eye movement strategies during naturalistic viewing

The deployment of eye movements to complex spatiotemporal stimuli likely involves a variety of cognitive factors. However, eye movements to movies are surprisingly reliable both within and across observers. We exploited and manipulated that reliability to characterize observers' temporal viewing strategies while they viewed naturalistic movies. Introducing cuts and scrambling the temporal order of the resulting clips systematically changed eye movement reliability. We developed a computational model that exhibited this behavior and provided an excellent fit to the measured eye movement reliability. The model assumed that observers searched for, found, and tracked a point of interest and that this process reset when there was a cut. The model did not require that eye movements depend on temporal context in any other way, and it managed to describe eye movements consistently across different observers and two movie sequences. Thus, we found no evidence for the integration of information over long time scales (greater than a second). The results are consistent with the idea that observers employ a simple tracking strategy even while viewing complex, engaging naturalistic stimuli.

Influence of meditation on anti-correlated networks in the brain

Human experiences can be broadly divided into those that are external and related to interaction with the environment, and experiences that are internal and self-related. The cerebral cortex appears to be divided into two corresponding systems: an "extrinsic" system composed of brain areas that respond more to external stimuli and tasks and an "intrinsic" system composed of brain areas that respond less to external stimuli and tasks. These two broad brain systems seem to compete with each other, such that their activity levels over time is usually anti-correlated, even when subjects are "at rest" and not performing any task. This study used meditation as an experimental manipulation to test whether this competition (anti-correlation) can be modulated by cognitive strategy. Participants either fixated without meditation (fixation), or engaged in non-dual awareness (NDA) or focused attention (FA) meditations. We computed inter-area correlations ("functional connectivity") between pairs of brain regions within each system, and between the entire extrinsic and intrinsic systems. Anti-correlation between extrinsic vs. intrinsic systems was stronger during FA meditation and weaker during NDA meditation in comparison to fixation (without mediation). However, correlation between areas within each system did not change across conditions. These results suggest that the anti-correlation found between extrinsic and intrinsic systems is not an immutable property of brain organization and that practicing different forms of meditation can modulate this gross functional organization in profoundly different ways.

Cross-orientation suppression in human visual cortex

Cross-orientation suppression was measured in human primary visual cortex (V1) to test the normalization model. Subjects viewed vertical target gratings (of varying contrasts) with or without a superimposed horizontal mask grating (fixed contrast). We used functional magnetic resonance imaging (fMRI) to measure the activity in each of several hypothetical channels (corresponding to subpopulations of neurons) with different orientation tunings and fit these orientation-selective responses with the normalization model. For the V1 channel maximally tuned to the target orientation, responses increased with target contrast but were suppressed when the horizontal mask was added, evident as a shift in the contrast gain of this channel's responses. For the channel maximally tuned to the mask orientation, a constant baseline response was evoked for all target contrasts when the mask was absent; responses decreased with increasing target contrast when the mask was present. The normalization model provided a good fit to the contrast-response functions with and without the mask. In a control experiment, the target and mask presentations were temporally interleaved, and we found no shift in contrast gain, i.e., no evidence for suppression. We conclude that the normalization model can explain cross-orientation suppression in human visual cortex. The approach adopted here can be applied broadly to infer, simultaneously, the responses of several subpopulations of neurons in the human brain that span particular stimulus or feature spaces, and characterize their interactions. In addition, it allows us to investigate how stimuli are represented by the inferred activity of entire neural populations.

Human primary visual cortex (V1) is selective for second-order spatial frequency

A variety of cues can differentiate objects from their surrounds. These include "first-order" cues such as luminance modulations and "second-order" cues involving modulations of orientation and contrast. Human sensitivity to first-order modulations is well described by a computational model involving spatially localized filters that are selective for orientation and spatial frequency (SF). It is widely held that first-order modulations are represented by the firing rates of simple and complex cells ("first-order" neurons) in primary visual cortex (V1) that, likewise, have spatially localized receptive fields that are selective for orientation- and SF. Human sensitivity to second-order modulations is well described by a filter-rectify-filter (FRF) model, with first- and second-order filters selective for orientation and SF. However, little is known about how neuronal activity in visual cortex represents second-order modulations. We tested the FRF model by using an functional (f)MRI-adaptation protocol to characterize the selectivity of activity in visual cortex to second-order, orientation-defined gratings of two different SFs. fMRI responses throughout early visual cortex exhibited selective adaptation to these stimuli. The low-SF grating was a more effective adapter than the high-SF grating, incompatible with the FRF model. To explain the results, we extended the FRF model by incorporating normalization, yielding a filter-rectify-normalize-filter model, in which normalization enhances selectivity for second-order SF but only for low spatial frequencies. We conclude that neurons in human visual cortex are selective for second-order SF, that normalization (surround suppression) contributes to this selectivity, and that the selectivity in higher visual areas is simply fed forward from V1.

Inter-area correlations in the ventral visual pathway reflect feature integration

During object perception, the brain integrates simple features into representations of complex objects. A perceptual phenomenon known as visual crowding selectively interferes with this process. Here, we use crowding to characterize a neural correlate of feature integration. Cortical activity was measured with functional magnetic resonance imaging, simultaneously in multiple areas of the ventral visual pathway (V1-V4 and the visual word form area, VWFA, which responds preferentially to familiar letters), while human subjects viewed crowded and uncrowded letters. Temporal correlations between cortical areas were lower for crowded letters than for uncrowded letters, especially between V1 and VWFA. These differences in correlation were retinotopically specific, and persisted when attention was diverted from the letters. But correlation differences were not evident when we substituted the letters with grating patches that were not crowded under our stimulus conditions. We conclude that inter-area correlations reflect feature integration and are disrupted by crowding. We propose that crowding may perturb the transformations between neural representations along the ventral pathway that underlie the integration of features into objects.

Motion-induced blindness and microsaccades: cause and effect

It has been suggested that subjective disappearance of visual stimuli results from a spontaneous reduction of microsaccade rate causing image stabilization, enhanced adaptation, and a consequent fading. In motion-induced blindness (MIB), salient visual targets disappear intermittently when surrounded by a moving pattern. We investigated whether changes in microsaccade rate can account for MIB. We first determined that the moving mask does not affect microsaccade metrics (rate, magnitude, and temporal distribution). We then compared the dynamics of microsaccades during reported illusory disappearance (MIB) and physical disappearance (Replay) of a salient peripheral target. We found large modulations of microsaccade rate following perceptual transitions, whether illusory (MIB) or real (Replay). For MIB, the rate also decreased prior to disappearance and increased prior to reappearance. Importantly, MIB persisted in the presence of microsaccades although sustained microsaccade rate was lower during invisible than visible periods. These results suggest that the microsaccade system reacts to changes in visibility, but microsaccades also modulate MIB. The latter modulation is well described by a Poisson model of the perceptual transitions assuming that the probability for reappearance and disappearance is modulated following a microsaccade. Our results show that microsaccades counteract disappearance but are neither necessary nor sufficient to account for MIB.

Differential roles for frontal eye fields (FEFs) and intraparietal sulcus (IPS) in visual working memory and visual attention

Cortical activity was measured with functional magnetic resonance imaging to probe the involvement of the superior precentral sulcus (including putative human frontal eye fields, FEFs) and the intraparietal sulcus (IPS) in visual short-term memory and visual attention. In two experimental tasks, human subjects viewed two visual stimuli separated by a variable delay period. The tasks placed differential demands on short-term memory and attention, but the stimuli were visually identical until after the delay period. An earlier study (S. Offen, D. Schluppeck, & D. J. Heeger, 2009) had found a dissociation in early visual cortex that suggested different computational mechanisms underlying the two processes. In contrast, the results reported here show that the patterns of activation in prefrontal and parietal cortex were different from one another but were similar for the two tasks. In particular, the FEF showed evidence for sustained delay period activity for both the working memory and the attention task, while the IPS did not show evidence for sustained delay period activity for either task. The results imply differential roles for the FEF and IPS in these tasks; the results also suggest that feedback of sustained activity from frontal cortex to visual cortex might be gated by task demands.

Orientation selectivity of motion-boundary responses in human visual cortex

Motion boundaries (local changes in visual motion direction) arise naturally when objects move relative to an observer. In human visual cortex, neuroimaging studies have identified a region (the kinetic occipital area [KO]) that responds more strongly to motion-boundary stimuli than to transparent-motion stimuli. However, some functional magnetic resonance imaging (fMRI) studies suggest that KO may encompass multiple visual areas and single-unit studies in macaque visual cortex have identified neurons selective for motion-boundary orientation in areas V2, V3, and V4, implying that motion-boundary selectivity may not be restricted to a single area. It is not known whether fMRI responses to motion boundaries are selective for motion-boundary orientation, as would be expected if these responses reflected the population activity of motion-boundary-selective neurons. We used an event-related fMRI adaptation protocol to measure orientation-selective responses to motion boundaries in human visual cortex. On each trial, we measured the response to a probe stimulus presented after an adapter stimulus (a vertical or horizontal motion-boundary grating). The probe stimulus was either a motion-boundary grating oriented parallel or orthogonal to the adapter stimulus or a transparent-motion stimulus. Orientation-selective adaptation for motion boundaries--smaller responses for trials in which test and adapter stimuli were parallel to each other--was observed in multiple extrastriate visual areas. The strongest adaptation, relative to the unadapted responses, was found in V3A, V3B, LO1, LO2, and V7. Most of the visual areas that exhibited orientation-selective adaptation in our data also showed response preference for motion boundaries over transparent motion, indicating that most of the human visual areas previously shown to respond to motion boundaries are also selective for motion-boundary orientation. These results suggest that neurons selective for motion-boundary orientation are distributed across multiple human visual cortical areas and argue against the existence of a single region or area specialized for motion-boundary processing.

Top-down flow of visual spatial attention signals from parietal to occipital cortex

Given the complexity of our visual environment, the ability to selectively attend to certain locations, while ignoring others, is crucial for reducing the amount of visual information to manageable levels and for optimizing behavioral performance. Sustained allocation of spatial attention causes persistent increases in functional magnetic resonance imaging (fMRI) signals in portions of early visual cortex that retinotopically represent the attended location, even in the absence of a visual stimulus. Here we test the hypothesis that topographically organized posterior parietal cortical areas IPS1 and IPS2 transmit top-down spatial attention signals to early visual cortex. We employed fMRI and coherency analysis to measure functional connectivity among cortical areas V1, V2, V3, V3A, V3B, V7, IPS1, and IPS2 during sustained visual spatial attention. Attention increased the magnitude of coherency for many pairs of areas in occipital and parietal cortex. Additionally, attention-related activity in IPS1 and IPS2 led activity in several visual cortical areas by a few hundred milliseconds. These results are consistent with transmission of top-down spatial attention signals from IPS1 and IPS2 to early visual cortex.

The role of early visual cortex in visual short-term memory and visual attention

We measured cortical activity with functional magnetic resonance imaging to probe the involvement of early visual cortex in visual short-term memory and visual attention. In four experimental tasks, human subjects viewed two visual stimuli separated by a variable delay period. The tasks placed differential demands on short-term memory and attention, but the stimuli were visually identical until after the delay period. Early visual cortex exhibited sustained responses throughout the delay when subjects performed attention-demanding tasks, but delay-period activity was not distinguishable from zero when subjects performed a task that required short-term memory. This dissociation reveals different computational mechanisms underlying the two processes.

Inter-ocular contrast normalization in human visual cortex

The brain combines visual information from the two eyes and forms a coherent percept, even when inputs to the eyes are different. However, it is not clear how inputs from the two eyes are combined in visual cortex. We measured fMRI responses to single gratings presented monocularly, or pairs of gratings presented monocularly or dichoptically with several combinations of contrasts. Gratings had either the same orientation or orthogonal orientations (i.e., plaids). Observers performed a demanding task at fixation to minimize top-down modulation of the stimulus-evoked responses. Dichoptic presentation of compatible gratings (same orientation) evoked greater activity than monocular presentation of a single grating only when contrast was low (<10%). A model that assumes linear summation of activity from each eye failed to explain binocular responses at 10% contrast or higher. However, a model with binocular contrast normalization, such that activity from each eye reduced the gain for the other eye, fitted the results very well. Dichoptic presentation of orthogonal gratings evoked greater activity than monocular presentation of a single grating for all contrasts. However, activity evoked by dichoptic plaids was equal to that evoked by monocular plaids. Introducing an onset asynchrony (stimulating one eye 500 ms before the other, which under attentive vision results in flash suppression) had no impact on the results; the responses to dichoptic and monocular plaids were again equal. We conclude that when attention is diverted, inter-ocular suppression in V1 can be explained by a normalization model in which the mutual suppression between orthogonal orientations does not depend on the eye of origin, nor on the onset times, and cross-orientation suppression is weaker than inter-ocular (same orientation) suppression.

The normalization model of attention

Attention has been found to have a wide variety of effects on the responses of neurons in visual cortex. We describe a model of attention that exhibits each of these different forms of attentional modulation, depending on the stimulus conditions and the spread (or selectivity) of the attention field in the model. The model helps reconcile proposals that have been taken to represent alternative theories of attention. We argue that the variety and complexity of the results reported in the literature emerge from the variety of empirical protocols that were used, such that the results observed in any one experiment depended on the stimulus conditions and the subject's attentional strategy, a notion that we define precisely in terms of the attention field in the model, but that has not typically been completely under experimental control.

Rapid and precise retinotopic mapping of the visual cortex obtained by voltage-sensitive dye imaging in the behaving monkey

Retinotopy is a fundamental organizing principle of the visual cortex. Over the years, a variety of techniques have been used to examine it. None of these techniques, however, provides a way to rapidly characterize retinotopy, at the submillimeter range, in alert, behaving subjects. Voltage-sensitive dye imaging (VSDI) can be used to monitor neuronal population activity at high spatial and temporal resolutions. Here we present a VSDI protocol for rapid and precise retinotopic mapping in the behaving monkey. Two monkeys performed a fixation task while thin visual stimuli swept periodically at a high speed in one of two possible directions through a small region of visual space. Because visual space is represented systematically across the cortical surface, each moving stimulus produced a traveling wave of activity in the cortex that could be precisely measured with VSDI. The time at which the peak of the traveling wave reached each location in the cortex linked this location with its retinotopic representation. We obtained detailed retinotopic maps from a region of about 1 cm(2) over the dorsal portion of areas V1 and V2. Retinotopy obtained during <4 min of imaging had a spatial precision of 0.11-0.19 mm, was consistent across experiments, and reliably predicted the locations of the response to small localized stimuli. The ability to rapidly obtain precise retinotopic maps in behaving monkeys opens the door for detailed analysis of the relationship between spatiotemporal dynamics of population responses in the visual cortex and perceptually guided behavior.

Hierarchy of cortical responses underlying binocular rivalry

During binocular rivalry, physical stimulation is dissociated from conscious visual awareness. Human brain imaging reveals a tight linkage between the neural events in human primary visual cortex (V1) and the dynamics of perceptual waves during transitions in dominance during binocular rivalry. Here, we report results from experiments in which observers' attention was diverted from the rival stimuli, implying that: competition between two rival stimuli involves neural circuits in V1, and attention is crucial for the consequences of this neural competition to advance to higher visual areas and promote perceptual waves.

Brain areas selective for both observed and executed movements

When observing a particular movement a subset of movement-selective visual and visuomotor neurons are active in the observer's brain, forming a representation of the observed movement. Similarly, when executing a movement a subset of movement-selective motor and visuomotor neurons are active, forming a representation of the executed movement. In this study we used an fMRI-adaptation protocol to assess cortical response selectivity to observed and executed movements simultaneously. Subjects freely played the rock-paper-scissors game against a videotaped opponent, sometimes repeatedly observing or executing the same movement on subsequent trials. Numerous brain areas exhibited adaptation (repetition suppression) during either repeated observations or repeated executions of the same movement. A subset of areas exhibited an overlap of both effects, containing neurons with selective responses for both executed and observed movements. We describe the function of these movement representation areas in the context of the human mirror system, which is expected to respond selectively to both observed and executed movements.

Rapid and precise retinotopic mapping of the visual cortex obtained by voltage-sensitive dye imaging in the behaving monkey

Retinotopy is a fundamental organizing principle of the visual cortex. Over the years, a variety of techniques have been used to examine it. None of these techniques, however, provides a way to rapidly characterize retinotopy, at the submillimeter range, in alert, behaving subjects. Voltage-sensitive dye imaging (VSDI) can be used to monitor neuronal population activity at high spatial and temporal resolutions. Here we present a VSDI protocol for rapid and precise retinotopic mapping in the behaving monkey. Two monkeys performed a fixation task while thin visual stimuli swept periodically at a high speed in one of two possible directions through a small region of visual space. Because visual space is represented systematically across the cortical surface, each moving stimulus produced a traveling wave of activity in the cortex that could be precisely measured with VSDI. The time at which the peak of the traveling wave reached each location in the cortex linked this location with its retinotopic representation. We obtained detailed retinotopic maps from a region of about 1 cm(2) over the dorsal portion of areas V1 and V2. Retinotopy obtained during <4 min of imaging had a spatial precision of 0.11-0.19 mm, was consistent across experiments, and reliably predicted the locations of the response to small localized stimuli. The ability to rapidly obtain precise retinotopic maps in behaving monkeys opens the door for detailed analysis of the relationship between spatiotemporal dynamics of population responses in the visual cortex and perceptually guided behavior.

Specificity of human cortical areas for reaches and saccades

Electrophysiological studies in monkeys have identified effector-related regions in the posterior parietal cortex (PPC). The lateral intraparietal area, for example, responds preferentially for saccades, whereas the parietal reach region responds preferentially for arm movements. However, the degree of effector selectivity actually observed is limited; each area contains neurons selective for the nonpreferred effector, and many neurons in both areas respond for both effectors. We used functional magnetic resonance imaging to assess the degree of effector preference at the population level, focusing on topographically organized regions in the human PPC [visual area V7, intraparietal sulcus 1 (IPS1), and IPS2]. An event-related design adapted from monkey experiments was used. In each trial, an effector cue preceded the appearance of a spatial target, after which a Go signal instructed subjects to produce the specified movement with the specified effector. Our results show that the degree of effector specificity is limited in many cortical areas and transitions gradually from saccade to reach preference as one moves through the hierarchy of areas in the occipital, parietal, and frontal cortices. Saccade preference was observed in visual cortex, including early areas and V7. IPS1 exhibited balanced activation to saccades and reaches, whereas IPS2 showed a weak but significant preference for reaches. In frontal cortex, areas near the central sulcus showed a clear and absolute preference for reaches, whereas the frontal eye field showed little or no effector selectivity. Although these results contradict many theoretical conclusions about effector specificity, they are compatible with the complex picture arising from electrophysiological studies and also with previous imaging studies that reported mostly overlapping saccade- and arm-related activation. The results are also compatible with theories of efficient coding in cortex.

Orientation-selective adaptation to illusory contours in human visual cortex

Humans can perceive illusory or subjective contours in the absence of any real physical boundaries. We used an adaptation protocol to look for orientation-selective neural responses to illusory contours defined by phase-shifted abutting line gratings in the human visual cortex. We measured functional magnetic resonance imaging (fMRI) responses to illusory-contour test stimuli after adapting to an illusory-contour adapter stimulus that was oriented parallel or orthogonal to the test stimulus. We found orientation-selective adaptation to illusory contours in early (V1 and V2) and higher-tier visual areas (V3, hV4, VO1, V3A/B, V7, LO1, and LO2). That is, fMRI responses were smaller for test stimuli parallel to the adapter than for test stimuli orthogonal to the adapter. In two control experiments using spatially jittered and phase-randomized stimuli, we demonstrated that this adaptation was not just in response to differences in the distribution of spectral power in the stimuli. Orientation-selective adaptation to illusory contours increased from early to higher-tier visual areas. Thus, both early and higher-tier visual areas contain neurons selective for the orientation of this type of illusory contour.

The effect of large veins on spatial localization with GE BOLD at 3 T: Displacement, not blurring

We used two different methods of region of interest (ROI) definition to investigate the spatial accuracy of functional magnetic resonance imaging (fMRI) at low and high spatial resolution. The "single-condition localizer" consisted of block alternation between a target stimulus and a mean gray background. The "differential localizer" consisted of block alternation between the target stimulus and another stimulus that filled the complement of the visual field. A separate series of scans, in which the target stimulus was presented briefly with long inter-stimulus intervals, was used to measure the hemodynamic impulse response function (HIRF). As expected, the differential localizer defined more restricted ROIs that better matched the predicted cortical representation of the target stimulus. However, at low resolution (3-mm isotropic) many voxels that responded positively to the target stimulus in the differential protocol responded negatively to the target stimulus in the single-condition localizer and in the HIRF measurements. The localization errors were attributed to voxels near large veins, which were identified based on low mean intensity and high variance. At high resolution (1.2-mm isotropic), the effects of large veins were present, but affected a smaller number of voxels. Thus, the use of differential localizers does not necessarily result in a more accurate indication of the underlying neural activity. Localization errors are reduced at higher spatial resolutions and can be eliminated by identification and removal of voxels dominated by large veins.

Two retinotopic visual areas in human lateral occipital cortex

We describe two visual field maps, lateral occipital areas 1 (LO1) and 2 (LO2), in the human lateral occipital cortex between the dorsal part of visual area V3 and visual area V5/MT+. Each map contained a topographic representation of the contralateral visual hemifield. The eccentricity representations were shared with V1/V2/V3. The polar angle representation in LO1 extended from the lower vertical meridian (at the boundary with dorsal V3) through the horizontal to the upper vertical meridian (at the boundary with LO2). The polar angle representation in LO2 was the mirror-reversal of that in LO1. LO1 and LO2 overlapped with the posterior part of the object-selective lateral occipital complex and the kinetic occipital region (KO). The retinotopy and functional properties of LO1 and LO2 suggest that they correspond to two new human visual areas, which lack exact homologues in macaque visual cortex. The topography, stimulus selectivity, and anatomical location of LO1 and LO2 indicate that they integrate shape information from multiple visual submodalities in retinotopic coordinates.

Neural correlates of sustained spatial attention in human early visual cortex

Attention is thought to enhance perceptual performance at attended locations through top-down attention signals that modulate activity in visual cortex. Here, we show that activity in early visual cortex is sustained during maintenance of attention in the absence of visual stimulation. We used functional magnetic resonance imaging (fMRI) to measure activity in visual cortex while human subjects performed a visual detection task in which a variable-duration delay period preceded target presentation. Portions of cortical areas V1, V2, and V3 representing the attended part of the visual field exhibited sustained increases in activity throughout the delay period. Portions of these cortical areas representing peripheral, unattended parts of the visual field displayed sustained decreases in activity. The data were well fit by a model that assumed the sustained neural activity was constant in amplitude over a time period equal to that of the actual delay period for each trial. These results demonstrate that sustained attention responses are present in early visual cortex (including primary visual cortex), in the absence of a visual stimulus, and that these responses correlate with the allocation of visuospatial attention in both the spatial and temporal domains.

Neural correlates of the visual vertical meridian asymmetry

Human visual performance is better below than above fixation along the vertical meridian-a phenomenon we refer to as vertical meridian asymmetry (VMA). Here, we used fMRI to investigate the neural correlates of the VMA. We presented stimuli of two possible sizes and spatial frequencies on the horizontal and vertical meridians and analyzed the fMRI data in subregions of early visual cortex (V1/V2) that corresponded retinotopically to the stimulus locations. Asymmetries in both the spatial extent and amplitude of the fMRI measurements correlated with the behavioral VMA. These results demonstrate that the VMA has a neural basis at the earliest stages of cortical visual processing and imply that visual performance is limited by the pooled sensory responses of large populations of neurons in the visual cortex.

Sustained activity in topographic areas of human posterior parietal cortex during memory-guided saccades

In a previous study, we identified three cortical areas in human posterior parietal cortex that exhibited topographic responses during memory-guided saccades [visual area 7 (V7), intraparietal sulcus 1 (IPS1), and IPS2], which are candidate homologs of macaque parietal areas such as the lateral intraparietal area and parietal reach region. Here, we show that these areas exhibit sustained delay-period activity, a critical physiological signature of areas in macaque parietal cortex. By varying delay duration, we disambiguated delay-period activity from sensory and motor responses. Mean time courses in the parietal areas were well fit by a linear model comprising three components representing responses to (1) the visual target, (2) the delay period, and (3) the eye movement interval. We estimated the contributions of each component: the response amplitude during the delay period was substantially smaller (<30%) than that elicited by the transient visual target. All three parietal regions showed comparable delay-period response amplitudes, with a trend toward larger responses from V7 to IPS1 and IPS2. Responses to the cue and during the delay period showed clear lateralization with larger responses to trials in which the target was placed in the contralateral visual field, suggesting that both of these components contributed to the topography we measured.

Orientation-selective adaptation to first- and second-order patterns in human visual cortex

Second-order textures-patterns that cannot be detected by mechanisms sensitive only to luminance changes-are ubiquitous in visual scenes, but the neuronal mechanisms mediating perception of such stimuli are not well understood. We used an adaptation protocol to measure neural activity in the human brain selective for the orientation of second-order textures. Functional MRI (fMRI) responses were measured in three subjects to presentations of first- and second-order probe gratings after adapting to a high-contrast first- or second-order grating that was either parallel or orthogonal to the probe gratings. First-order (LM) stimuli were generated by modulating the stimulus luminance. Second-order stimuli were generated by modulating the contrast (CM) or orientation (OM) of a first-order carrier. We used four combinations of adapter and probe stimuli: LM:LM, CM:CM, OM:OM, and LM:OM. The fourth condition tested for cross-modal adaptation with first-order adapter and second-order probe stimuli. Attention was diverted from the stimulus by a demanding task at fixation. Both first- and second-order stimuli elicited orientation-selective adaptation in multiple cortical visual areas, including V1, V2, V3, V3A/B, a newly identified visual area anterior to dorsal V3 that we have termed LO1, hV4, and VO1. For first-order stimuli (condition LM:LM), the adaptation was no larger in extrastriate areas than in V1, implying that the orientation-selective first-order (luminance) adaptation originated in V1. For second-order stimuli (conditions CM:CM and OM:OM), the magnitude of adaptation, relative to the absolute response magnitude, was significantly larger in VO1 (and for condition CM:CM, also in V3A/B and LO1) than in V1, suggesting that second-order stimulus orientation was extracted by additional processing after V1. There was little difference in the amplitude of adaptation between the second-order conditions. No consistent effect of adaptation was found in the cross-modal condition LM:OM, in agreement with psychophysical evidence for weak interactions between first- and second-order stimuli and computational models of separate mechanisms for first- and second-order visual processing.

Topographic maps of visual spatial attention in human parietal cortex

Functional magnetic resonance imaging (fMRI) was used to measure activity in human parietal cortex during performance of a visual detection task in which the focus of attention systematically traversed the visual field. Critically, the stimuli were identical on all trials (except for slight contrast changes in a fully randomized selection of the target locations) whereas only the cued location varied. Traveling waves of activity were observed in posterior parietal cortex consistent with shifts in covert attention in the absence of eye movements. The temporal phase of the fMRI signal in each voxel indicated the corresponding visual field location. Visualization of the distribution of temporal phases on a flattened representation of parietal cortex revealed at least two distinct topographically organized cortical areas within the intraparietal sulcus (IPS), each representing the contralateral visual field. Two cortical areas were proposed based on this topographic organization, which we refer to as IPS1 and IPS2 to indicate their locations within the IPS. This nomenclature is neutral with respect to possible homologies with well-established cortical areas in the monkey brain. The two proposed cortical areas exhibited relatively little response to passive visual stimulation in comparison with early visual areas. These results provide evidence for multiple topographic maps in human parietal cortex.

Topographic organization for delayed saccades in human posterior parietal cortex

Posterior parietal cortex (PPC) is thought to play a critical role in decision making, sensory attention, motor intention, and/or working memory. Research on the PPC in non-human primates has focused on the lateral intraparietal area (LIP) in the intraparietal sulcus (IPS). Neurons in LIP respond after the onset of visual targets, just before saccades to those targets, and during the delay period in between. To study the function of posterior parietal cortex in humans, it will be crucial to have a routine and reliable method for localizing specific parietal areas in individual subjects. Here, we show that human PPC contains at least two topographically organized regions, which are candidates for the human homologue of LIP. We mapped the topographic organization of human PPC for delayed (memory guided) saccades using fMRI. Subjects were instructed to fixate centrally while a peripheral target was briefly presented. After a further 3-s delay, subjects made a saccade to the remembered target location followed by a saccade back to fixation and a 1-s inter-trial interval. Targets appeared at successive locations "around the clock" (same eccentricity, approximately 30 degrees angular steps), to produce a traveling wave of activity in areas that are topographically organized. PPC exhibited topographic organization for delayed saccades. We defined two areas in each hemisphere that contained topographic maps of the contra-lateral visual field. These two areas were immediately rostral to V7 as defined by standard retinotopic mapping. The two areas were separated from each other and from V7 by reversals in visual field orientation. However, we leave open the possibility that these two areas will be further subdivided in future studies. Our results demonstrate that topographic maps tile the cortex continuously from V1 well into PPC.

Traveling waves of activity in primary visual cortex during binocular rivalry

When the two eyes view large, dissimilar patterns that induce binocular rivalry, alternating waves of visibility are experienced as one pattern sweeps the other out of conscious awareness. Here we combine psychophysics with functional magnetic resonance imaging to show tight linkage between dynamics of perceptual waves during rivalry and neural events in human primary visual cortex (V1).

Stereoscopic processing of absolute and relative disparity in human visual cortex

Stereoscopic vision relies mainly on relative depth differences between objects rather than on their absolute distance in depth from where the eyes fixate. However, relative disparities are computed from absolute disparities, and it is not known where these two stages are represented in the human brain. Using functional MRI (fMRI), we assessed absolute and relative disparity selectivity with stereoscopic stimuli consisting of pairs of transparent planes in depth in which the absolute and relative disparity signals could be independently manipulated (at a local spatial scale). In experiment 1, relative disparity was kept constant, while absolute disparity was varied in one-half the blocks of trials ("mixed" blocks) and kept constant in the remaining one-half ("same" blocks), alternating between blocks. Because neuronal responses undergo adaptation and reduce their firing rate following repeated presentation of an effective stimulus, the fMRI signal reflecting activity of units selective for absolute disparity is expected to be smaller during "same" blocks as compared with "mixed" ones. Experiment 2 similarly manipulated relative disparity rather than absolute disparity. The results from both experiments were consistent with adaptation with differential effects across visual areas such that 1) dorsal areas (V3A, MT+/V5, V7) showed more adaptation to absolute than to relative disparity; 2) ventral areas (hV4, V8/V4alpha) showed an equal adaptation to both; and 3) early visual areas (V1, V2, V3) showed a small effect in both experiments. These results indicate that processing in dorsal areas may rely mostly on information about absolute disparities, while ventral areas split neural resources between the two types of stereoscopic information so as to maintain an important representation of relative disparity.

Response suppression in v1 agrees with psychophysics of surround masking

When a target stimulus is embedded in a high contrast surround, the target appears reduced in contrast and is harder to detect, and neural responses in visual cortex are suppressed. We used functional magnetic resonance imaging (fMRI) and psychophysics to quantitatively compare these physiological and perceptual effects. Observers performed a contrast discrimination task on a contrast-reversing sinusoidal target grating. The target was either presented in isolation or embedded in a high-contrast surround. While observers performed the task, we also measured fMRI responses as a function of target contrast, both with and without a surround. We found that the surround substantially increased the psychophysical thresholds while reducing fMRI responses. The two data sets were compared, on the basis of the assumption that a fixed response difference is required for correct discrimination, and we found that the psychophysics accounted for 96.5% of the variance in the measured V1 responses. The suppression in visual areas V2 and V3 was stronger, too strong to agree with psychophysics. The good quantitative agreement between psychophysical thresholds and V1 responses suggests V1 as a plausible candidate for mediating surround masking.

Neuronal correlates of perception in early visual cortex

We used functional magnetic resonance imaging (fMRI) to measure activity in human early visual cortex (areas V1, V2 and V3) during a challenging contrast-detection task. Subjects attempted to detect the presence of slight contrast increments added to two kinds of background patterns. Behavioral responses were recorded so that the corresponding cortical activity could be grouped into the usual signal detection categories: hits, false alarms, misses and correct rejects. For both kinds of background patterns, the measured cortical activity was retinotopically specific. Hits and false alarms were associated with significantly more cortical activity than were correct rejects and misses. That false alarms evoked more activity than misses indicates that activity in early visual cortex corresponded to the subjects' percepts, rather than to the physically presented stimulus.

A synaptic explanation of suppression in visual cortex

The responses of neurons in the primary visual cortex (V1) are suppressed by mask stimuli that do not elicit responses if presented alone. This suppression is widely believed to be mediated by intracortical inhibition. As an alternative, we propose that it can be explained by thalamocortical synaptic depression. This explanation correctly predicts that suppression is monocular, immune to cortical adaptation, and occurs for mask stimuli that elicit responses in the thalamus but not in the cortex. Depression also explains other phenomena previously ascribed to intracortical inhibition. It explains why responses saturate at high stimulus contrast, whereas selectivity for orientation and spatial frequency is invariant with contrast. It explains why transient responses to flashed stimuli are nonlinear, whereas spatial summation is primarily linear. These results suggest that the very first synapses into the cortex, and not the cortical network, may account for important response properties of V1 neurons.

Spatiotemporal mechanisms for detecting and identifying image features in human vision

Our visual system constantly selects salient features in the environment, so that only those features are attended and targeted by further processing efforts to identify them. Models of feature detection hypothesize that salient features are localized based on contrast energy (local variance in intensity) in the visual stimulus. This hypothesis, however, has not been tested directly. We used psychophysical reverse correlation to study how humans detect and identify basic image features (bars and short line segments). Subjects detected a briefly flashed 'target bar' that was embedded in 'noise bars' that randomly changed in intensity over space and time. By studying how the intensity of the noise bars affected performance, we were able to dissociate two processing stages: an early 'detection' stage, whereby only locations of high-contrast energy in the image are selected, followed (after approximately 100 ms) by an 'identification' stage, whereby image intensity at selected locations is used to determine the identity (whether bright or dark) of the target.

What does fMRI tell us about neuronal activity?

In recent years, cognitive neuroscientists have taken great advantage of functional magnetic resonance imaging (fMRI) as a non-invasive method of measuring neuronal activity in the human brain. But what exactly does fMRI tell us? We know that its signals arise from changes in local haemodynamics that, in turn, result from alterations in neuronal activity, but exactly how neuronal activity, haemodynamics and fMRI signals are related is unclear. It has been assumed that the fMRI signal is proportional to the local average neuronal activity, but many factors can influence the relationship between the two. A clearer understanding of how neuronal activity influences the fMRI signal is needed if we are correctly to interpret functional imaging data.

Pattern-motion responses in human visual cortex

Physiological models of visual motion processing posit that 'pattern-motion cells' represent the direction of moving objects independent of their particular spatial pattern. We performed fMRI experiments to identify neuronal activity in the human brain selective for pattern motion. A protocol using adaptation to moving 'plaid' stimuli allowed us to separate pattern-motion responses from other types of motion-related activity within the same brain structures, and revealed strong pattern-motion selectivity in human visual area MT+. Reducing the perceptual coherence of the plaids yielded a corresponding decrease in pattern-motion responsivity, providing evidence that percepts of coherent motion are closely linked to the activity of pattern-motion cells in human MT+.

Neuronal basis of the motion aftereffect reconsidered

Several fMRI studies have reported MT+ response increases correlated with perception of the motion aftereffect (MAE). However, attention can strongly affect MT+ responses, and subjects may naturally attend more to the MAE than control trials without MAE. We found that requiring subjects to attend to motion on both MAE and control trials produced equal levels of MT+ response, suggesting that attention may have confounded the interpretation of previous experiments; in our data, attention accounts for the entire effect. After eliminating this confound, we observed that direction-selective motion adaptation produced a direction-selective imbalance in MT+ responses (and earlier visual areas), and yielded a corresponding asymmetry in speed discrimination thresholds. These findings provide physiological evidence that population level response imbalances underlie the MAE, and quantify the relative proportions of direction-selective neurons across human visual areas.

Human cortical activity correlates with stereoscopic depth perception

Stereoscopic depth perception is based on binocular disparities. Although neurons in primary visual cortex (V1) are selective for binocular disparity, their responses do not explicitly code perceived depth. The stereoscopic pathway must therefore include additional processing beyond V1. We used functional magnetic resonance imaging (fMRI) to examine stereo processing in V1 and other areas of visual cortex. We created stereoscopic stimuli that portrayed two planes of dots in depth, placed symmetrically about the plane of fixation, or else asymmetrically with both planes either nearer or farther than fixation. The interplane disparity was varied parametrically to determine the stereoacuity threshold (the smallest detectable disparity) and the upper depth limit (largest detectable disparity). fMRI was then used to quantify cortical activity across the entire range of detectable interplane disparities. Measured cortical activity covaried with psychophysical measures of stereoscopic depth perception. Activity increased as the interplane disparity increased above the stereoacuity threshold and dropped as interplane disparity approached the upper depth limit. From the fMRI data and an assumption that V1 encodes absolute retinal disparity, we predicted that the mean response of V1 neurons should be a bimodal function of disparity. A post hoc analysis of electrophysiological recordings of single neurons in macaques revealed that, although the average firing rate was a bimodal function of disparity (as predicted), the precise shape of the function cannot fully explain the fMRI data. Although there was widespread activity within the extrastriate cortex (consistent with electrophysiological recordings of single neurons), area V3A showed remarkable sensitivity to stereoscopic stimuli, suggesting that neurons in V3A may play a special role in the stereo pathway.

Measurement and modeling of center-surround suppression and enhancement

The apparent contrast of a central stimulus is affected by the presence of surrounding stimuli. For some stimulus conditions, the apparent contrast is suppressed and for other conditions the apparent contrast is enhanced. This report is intended to offer a coherent description of the stimulus factors that influence suppression and enhancement. Using a contrast-matching protocol, we measured the contrast dependence of center-surround interactions by systematically varying the suprathreshold contrasts of the central and surround gratings. Different spatial configurations of the surround stimuli were studied. Our results confirmed previous findings that (1) a surround stimulus could produce either contrast enhancement or contrast suppression depending on the balance of the central and surround contrasts; (2) suppression varied with the width of the surround stimulus and was strongly orientation-specific; and (3) enhancement was less sensitive to changes in surround configurations (in particular, enhancement did not depend on the colinearity of the central and surround gratings). Based on the experimental data, we developed a computational model to account for center-surround suppression and enhancement.

Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry

During binocular rivalry, two incompatible monocular images compete for perceptual dominance, with one pattern temporarily suppressed from conscious awareness. We measured fMRI signals in early visual cortex while subjects viewed rival dichoptic images of two different contrasts; the contrast difference served as a 'tag' for the neuronal representations of the two monocular images. Activity in primary visual cortex (V1) increased when subjects perceived the higher contrast pattern and decreased when subjects perceived the lower contrast pattern. These fluctuations in V1 activity during rivalry were about 55% as large as those evoked by alternately presenting the two monocular images without rivalry. The rivalry-related fluctuations in V1 activity were roughly equal to those observed in other visual areas (V2, V3, V3a and V4v). These results challenge the view that the neuronal mechanisms responsible for binocular rivalry occur primarily in later visual areas.

Activity in primary visual cortex predicts performance in a visual detection task

Visual attention can affect both neural activity and behavior in humans. To quantify possible links between the two, we measured activity in early visual cortex (V1, V2 and V3) during a challenging pattern-detection task. Activity was dominated by a large response that was independent of the presence or absence of the stimulus pattern. The measured activity quantitatively predicted the subject's pattern-detection performance: when activity was greater, the subject was more likely to correctly discern the presence or absence of the pattern. This stimulus-independent activity had several characteristics of visual attention, suggesting that attentional mechanisms modulate activity in early visual cortex, and that this attention-related activity strongly influences performance.

Task-related modulation of visual cortex

We performed a series of experiments to quantify the effects of task performance on cortical activity in early visual areas. Functional magnetic resonance imaging (fMRI) was used to measure cortical activity in several cortical visual areas including primary visual cortex (V1) and the MT complex (MT+) as subjects performed a variety of threshold-level visual psychophysical tasks. Performing speed, direction, and contrast discrimination tasks produced strong modulations of cortical activity. For example, one experiment tested for selective modulations of MT+ activity as subjects alternated between performing contrast and speed discrimination tasks. MT+ responses modulated in phase with the periods of time during which subjects performed the speed discrimination task; that is, MT+ activity was higher during speed discrimination than during contrast discrimination. Task-related modulations were consistent across repeated measurements in each subject; however, significant individual differences were observed between subjects. Together, the results suggest 1) that specific changes in the cognitive/behavioral state of a subject can exert selective and reliable modulations of cortical activity in early visual cortex, even in V1; 2) that there are significant individual differences in these modulations; and 3) that visual areas and pathways that are highly sensitive to small changes in a given stimulus feature (such as contrast or speed) are selectively modulated during discrimination judgments on that feature. Increasing the gain of the relevant neuronal signals in this way may improve their signal-to-noise to help optimize task performance.

Robust multiresolution alignment of MRI brain volumes

An algorithm for the automatic alignment of MRI volumes of the human brain was developed, based on techniques adopted from the computer vision literature for image motion estimation. Most image registration techniques rely on the assumption that corresponding voxels in the two volumes have equal intensity, which is not true for MRI volumes acquired with different coils and/or pulse sequences. Intensity normalization and contrast equalization were used to minimize the differences between the intensities of the two volumes. However, these preprocessing steps do not correct perfectly for the image differences when using different coils and/or pulse sequences. Hence, the alignment algorithm relies on robust estimation, which automatically ignores voxels where the intensities are sufficiently different in the two volumes. A multiresolution pyramid implementation enables the algorithm to estimate large displacements. The resulting algorithm is used routinely to align MRI volumes acquired using different protocols (3D SPGR and 2D fast spin echo) and different coils (surface and head) to subvoxel accuracy (better than 1 mm).

Center-surround interactions in foveal and peripheral vision

The perceived contrast of a central stimulus can be decreased (surround suppression) or increased (surround facilitation) by the presence of surround stimuli. In this report we examined center-surround interactions in foveal and peripheral vision using contrast-matching tasks. We found that: (1) surround suppression became markedly stronger as the center-surround stimulus was moved toward the periphery; (2) surround facilitation diminished in the periphery; and (3) the suppression in the periphery was less orientation- and frequency-specific than that in the fovea, so that significant suppression was induced even when the central and surround gratings had very different orientations and spatial frequencies. The different center-surround interactions in the fovea and periphery can not be accounted for by cortical magnification, suggesting that center-surround interactions in the fovea and periphery are incommensurable and play different functional roles in human image processing.

Motion opponency in visual cortex

Perceptual studies suggest that visual motion perception is mediated by opponent mechanisms that correspond to mutually suppressive populations of neurons sensitive to motions in opposite directions. We tested for a neuronal correlate of motion opponency using functional magnetic resonance imaging (fMRI) to measure brain activity in human visual cortex. There was strong motion opponency in a secondary visual cortical area known as the human MT complex (MT+), but there was little evidence of motion opponency in primary visual cortex. To determine whether the level of opponency in human and monkey are comparable, a variant of these experiments was performed using multiunit electrophysiological recording in areas MT and MST of the macaque monkey brain. Although there was substantial variability in the degree of opponency between recording sites, the monkey and human data were qualitatively similar on average. These results provide further evidence that: (1) direction-selective signals underly human MT+ responses, (2) neuronal signals in human MT+ support visual motion perception, (3) human MT+ is homologous to macaque monkey MT and adjacent motion sensitive brain areas, and (4) that fMRI measurements are correlated with average spiking activity.

Linking visual perception with human brain activity

The past year has seen great advances in the use of functional magnetic resonance imaging (fMRI) to study the functional organization of the human visual cortex, to measure the neuronal correlates of visual perception, and to test computational theories of vision. Activity in particular visual brain areas, as measured with fMRI, has been found to correlate with psychophysical performance, with visual attention, and with subjective perceptual experience.