Parametric reverse correlation reveals spatial linearity of retinotopic human V1 BOLD response

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Feature-space selection with banded ridge regression

Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.

Layer-specific, retinotopically-diffuse modulation in human visual cortex in response to viewing emotionally expressive faces

Viewing faces that are perceived as emotionally expressive evokes enhanced neural responses in multiple brain regions, a phenomenon thought to depend critically on the amygdala. This emotion-related modulation is evident even in primary visual cortex (V1), providing a potential neural substrate by which emotionally salient stimuli can affect perception. How does emotional valence information, computed in the amygdala, reach V1? Here we use high-resolution functional MRI to investigate the layer profile and retinotopic distribution of neural activity specific to emotional facial expressions. Across three experiments, human participants viewed centrally presented face stimuli varying in emotional expression and performed a gender judgment task. We found that facial valence sensitivity was evident only in superficial cortical layers and was not restricted to the retinotopic location of the stimuli, consistent with diffuse feedback-like projections from the amygdala. Together, our results provide a feedback mechanism by which the amygdala directly modulates activity at the earliest stage of visual processing.

Quantitative characterization of the human retinotopic map based on quasiconformal mapping

The retinotopic map depicts the cortical neurons' response to visual stimuli on the retina and has contributed significantly to our understanding of human visual system. Although recent advances in high field functional magnetic resonance imaging (fMRI) have made it possible to generate the in vivo retinotopic map with great detail, quantifying the map remains challenging. Existing quantification methods do not preserve surface topology and often introduce large geometric distortions to the map. In this study, we developed a new framework based on computational conformal geometry and quasiconformal Teichmüller theory to quantify the retinotopic map. Specifically, we introduced a general pipeline, consisting of cortical surface conformal parameterization, surface-spline-based cortical activation signal smoothing, and vertex-wise Beltrami coefficient-based map description. After correcting most of the violations of the topological conditions, the result was a "Beltrami coefficient map" (BCM) that rigorously and completely characterizes the retinotopic map by quantifying the local quasiconformal mapping distortion at each visual field location. The BCM provided topological and fully reconstructable retinotopic maps. We successfully applied the new framework to analyze the V1 retinotopic maps from the Human Connectome Project (n=181), the largest state of the art retinotopy dataset currently available. With unprecedented precision, we found that the V1 retinotopic map was quasiconformal and the local mapping distortions were similar across observers. The new framework can be applied to other visual areas and retinotopic maps of individuals with and without eye diseases, and improve our understanding of visual cortical organization in normal and clinical populations.

Imaging faster neural dynamics with fast fMRI: A need for updated models of the hemodynamic response

Fast fMRI enables the detection of neural dynamics over timescales of hundreds of milliseconds, suggesting it may provide a new avenue for studying subsecond neural processes in the human brain. The magnitudes of these fast fMRI dynamics are far greater than predicted by canonical models of the hemodynamic response. Several studies have established nonlinear properties of the hemodynamic response that have significant implications for fast fMRI. We first review nonlinear properties of the hemodynamic response function that may underlie fast fMRI signals. We then illustrate the breakdown of canonical hemodynamic response models in the context of fast neural dynamics. We will then argue that the canonical hemodynamic response function is not likely to reflect the BOLD response to neuronal activity driven by sparse or naturalistic stimuli or perhaps to spontaneous neuronal fluctuations in the resting state. These properties suggest that fast fMRI is capable of tracking surprisingly fast neuronal dynamics, and we discuss the neuroscientific questions that could be addressed using this approach.

Distinct saccade planning and endogenous visuospatial attention maps in parietal cortex: A basis for functional differences in sensory and motor attention

Parietal cortex activity contributes to higher-level cognitive processes, including endogenous visual attention and saccade planning. While visual attention is the process of selecting pertinent information from the environment, saccade planning may involve motor attention in the planning of a specific movement, including the process of selecting the correct path. We isolated areas in parietal cortex involved in saccade planning, while controlling visual attention, to understand the relationship between the two processes. Using our novel stimulus, participants performed a delayed saccade task and an endogenous covert visuospatial attention task with peripheral targets in identical locations. We compared multiple target locations across the two domains at the level of the individual to better understand variability in the relationship between these two maps. The anterior-posterior organization of saccade planning and visual attention maps varied among, but not within, participants, and 14-29% of the maps for each task overlapped one another across hemispheres. Interestingly, within the region of co-activation, over 67% of the voxels responded to the same location for both tasks. These cortical areas of overlap may represent regions of the brain specifically involved in the transfer of information from vision to action along the visuomotor pathway. These results further establish the relationship between maps associated with saccade planning and visual attention at the individual level, indicating the lack of a single saliency map in parietal cortex.

Population spatial frequency tuning in human early visual cortex

Neurons within early visual cortex are selective for basic image statistics, including spatial frequency. However, these neurons are thought to act as band-pass filters, with the window of spatial frequency sensitivity varying across the visual field and across visual areas. Although a handful of previous functional (f)MRI studies have examined human spatial frequency sensitivity using conventional designs and analysis methods, these measurements are time consuming and fail to capture the precision of spatial frequency tuning (bandwidth). In this study, we introduce a model-driven approach to fMRI analyses that allows for fast and efficient estimation of population spatial frequency tuning (pSFT) for individual voxels. Blood oxygen level-dependent (BOLD) responses within early visual cortex were acquired while subjects viewed a series of full-field stimuli that swept through a large range of spatial frequency content. Each stimulus was generated by band-pass filtering white noise with a central frequency that changed periodically between a minimum of 0.5 cycles/degree (cpd) and a maximum of 12 cpd. To estimate the underlying frequency tuning of each voxel, we assumed a log-Gaussian pSFT and optimized the parameters of this function by comparing our model output against the measured BOLD time series. Consistent with previous studies, our results show that an increase in eccentricity within each visual area is accompanied by a drop in the peak spatial frequency of the pSFT. Moreover, we found that pSFT bandwidth depends on eccentricity and is correlated with the pSFT peak; populations with lower peaks possess broader bandwidths in logarithmic scale, whereas in linear scale this relationship is reversed.NEW & NOTEWORTHY Spatial frequency selectivity is a hallmark property of early visuocortical neurons, and mapping these sensitivities gives us crucial insight into the hierarchical organization of information within visual areas. Due to technical obstacles, we lack a comprehensive picture of the properties of this sensitivity in humans. Here, we introduce a new method, coined population spatial frequency tuning mapping, which circumvents the limitations of the conventional neuroimaging methods, yielding a fuller visuocortical map of spatial frequency sensitivity.

Decoding natural images from evoked brain activities using encoding models with invertible mapping

Recent studies have built encoding models in the early visual cortex, and reliable mappings have been made between the low-level visual features of stimuli and brain activities. However, these mappings are irreversible, so that the features cannot be directly decoded. To solve this problem, we designed a sparse framework-based encoding model that predicted brain activities from a complete feature representation. Moreover, according to the distribution and activation rules of neurons in the primary visual cortex (V1), three key transformations were introduced into the basic feature to improve the model performance. In this setting, the mapping was simple enough that it could be inverted using a closed-form formula. Using this mapping, we designed a hybrid identification method based on the support vector machine (SVM), and tested it on a published functional magnetic resonance imaging (fMRI) dataset. The experiments confirmed the rationality of our encoding model, and the identification accuracies for 2 subjects increased from 92% and 72% to 98% and 92% with the chance level only 0.8%.

Manifold decoding for neural representations of face viewpoint and gaze direction using magnetoencephalographic data

The main challenge in decoding neural representations lies in linking neural activity to representational content or abstract concepts. The transformation from a neural-based to a low-dimensional representation may hold the key to encoding perceptual processes in the human brain. In this study, we developed a novel model by which to represent two changeable features of faces: face viewpoint and gaze direction. These features are embedded in spatiotemporal brain activity derived from magnetoencephalographic data. Our decoding results demonstrate that face viewpoint and gaze direction can be represented by manifold structures constructed from brain responses in the bilateral occipital face area and right superior temporal sulcus, respectively. Our results also show that the superposition of brain activity in the manifold space reveals the viewpoints of faces as well as directions of gazes as perceived by the subject. The proposed manifold representation model provides a novel opportunity to gain further insight into the processing of information in the human brain.

Optimization of functional MRI for detection, decoding and high-resolution imaging of the response patterns of cortical columns

The capacity of functional MRI (fMRI) to resolve cortical columns depends on several factors. These include the spatial scale of the columnar pattern, the point-spread of the fMRI response, the voxel size, and the signal-to-noise ratio (SNR) considering thermal and physiological noise. However, it remains unknown how these factors combine, and what is the voxel size that optimizes fMRI of cortical columns. Here we combine current knowledge into a quantitative model of fMRI of realistic patterns of cortical columns with different spatial scales and degrees of irregularity. We compare different approaches for identifying patterns of cortical columns, including univariate and multivariate based detection, multi-voxel pattern analysis (MVPA) based decoding, and high-resolution imaging and reconstruction of the pattern of cortical columns. We present the dependence of the performance of each approach on the parameters of the imaged pattern as well as those of the data acquisition. In addition, we predict voxel sizes that optimize fMRI of cortical columns under various scenarios. We found that all measures associated with multivariate detection and decoding could be approximately calculated from a measure we termed "multivariate contrast-to-noise ratio" (mv-CNR), which is a function of the contrast-to-noise ratio (CNR) and number of voxels. Furthermore, mv-CNR implied that the optimal voxel width for detection and decoding is independent of changes in response amplitude, SNR and imaged volume that are not caused by changes in voxel size. For regular patterns, optimal voxel widths for detection, decoding and imaging/reconstructing the pattern of cortical columns were approximately half the main cycle length of the organization. Optimal voxel widths for irregular patterns were less dependent on the main cycle length, and differed between univariate detection, multivariate detection and decoding, and reconstruction. We compared the effects of different factors of Gradient Echo fMRI at 3 Tesla (T), Gradient Echo fMRI at 7T, and Spin-Echo fMRI at 7T on the detection, decoding, and reconstruction measures considered and found that in all cases the width of the fMRI point-spread had the most significant effect. In contrast, different response amplitudes and noise characteristics played a relatively minor role. We recommend specific voxel widths for optimal univariate detection, for multivariate detection and decoding, and for high-resolution imaging of cortical columns under these three data-acquisition scenarios. Our study supports the planning, optimization, and interpretation of high-resolution fMRI of cortical columns and the decoding of information conveyed by these columns.

Modeling the Dynamics of Human Brain Activity with Recurrent Neural Networks

Encoding models are used for predicting brain activity in response to sensory stimuli with the objective of elucidating how sensory information is represented in the brain. Encoding models typically comprise a nonlinear transformation of stimuli to features (feature model) and a linear convolution of features to responses (response model). While there has been extensive work on developing better feature models, the work on developing better response models has been rather limited. Here, we investigate the extent to which recurrent neural network models can use their internal memories for nonlinear processing of arbitrary feature sequences to predict feature-evoked response sequences as measured by functional magnetic resonance imaging. We show that the proposed recurrent neural network models can significantly outperform established response models by accurately estimating long-term dependencies that drive hemodynamic responses. The results open a new window into modeling the dynamics of brain activity in response to sensory stimuli.

Positive and Negative Somatotopic BOLD Responses in Contralateral Versus Ipsilateral Penfield Homunculus

One of the basic properties of sensory cortices is their topographical organization. Most imaging studies explored this organization using the positive blood oxygenation level-dependent (BOLD) signal. Here, we studied the topographical organization of both positive and negative BOLD in contralateral and ipsilateral primary somatosensory cortex (S1). Using phase-locking mapping methods, we verified the topographical organization of contralateral S1, and further showed that different body segments elicit pronounced negative BOLD responses in both hemispheres. In the contralateral hemisphere, we found a sharpening mechanism in which stimulation of a given body segment triggered a gradient of activation with a significant deactivation in more remote areas. In the ipsilateral cortex, deactivation was not only located in the homolog area of the stimulated parts but rather was widespread across many parts of S1. Additionally, analysis of resting-state functional magnetic resonance imaging signal showed a gradient of connectivity to the neighboring contralateral body parts as well as to the ipsilateral homologous area for each body part. Taken together, our results indicate a complex pattern of baseline and activity-dependent responses in the contralateral and ipsilateral sides. Both primary sensory areas were characterized by unique negative BOLD responses, suggesting that they are an important component in topographic organization of sensory cortices.

Intersession reliability of population receptive field estimates

Population receptive field (pRF) analysis is a popular method to infer spatial selectivity of voxels in visual cortex. However, it remains largely untested how stable pRF estimates are over time. Here we measured the intersession reliability of pRF parameter estimates for the central visual field and near periphery, using a combined wedge and ring stimulus containing natural images. Sixteen healthy human participants completed two scanning sessions separated by 10-114 days. Individual participants showed very similar visual field maps for V1-V4 on both sessions. Intersession reliability for eccentricity and polar angle estimates was close to ceiling for most visual field maps (r>.8 for V1-3). PRF size and cortical magnification (CMF) estimates showed strong but lower overall intersession reliability (r≈.4-.6). Group level results for pRF size and CMF were highly similar between sessions. Additional control experiments confirmed that reliability does not depend on the carrier stimulus used and that reliability for pRF size and CMF is high for sessions acquired on the same day (r>.6). Our results demonstrate that pRF mapping is highly reliable across sessions.

Influence of BOLD Contributions to Diffusion fMRI Activation of the Visual Cortex

Reliance on the hemodynamic response as a surrogate marker of neural activity imposes an intrinsic limit on the spatial specificity of functional MRI. An alternative approach based on diffusion-weighted functional MRI (DfMRI) has been reported as a contrast less reliant on hemodynamic effects, however current evidence suggests that both hemodynamic and unique neural sources contribute to the diffusion signal. Here we compare activation patterns obtained with the standard blood oxygenation level-dependent (BOLD) contrast to DfMRI in order to gain a deeper understanding of how the BOLD proportion contributes to the observable diffusion signal. Both individual and group-level activation patterns obtained with DfMRI and BOLD to a visual field stimulation paradigm were analyzed. At the individual level, the DfMRI contrast showed a strong, positive relationship between the volumes of cortex activated in response to quadrant- and hemi-field visual stimulation. This was not observed in the corresponding BOLD experiment. Overall, the DfMRI response indicated less between-subject variability, with random effects analyses demonstrating higher statistical values at the peak voxel for DfMRI. Furthermore, the spatial extent of the activation was more restricted to the primary visual region for DfMRI than BOLD. However, the diffusion signal was sensitive to the hemodynamic response in a manner dependent on experimental manipulation. It was also limited by its low signal-to-noise ratio (SNR), demonstrating lower sensitivity than BOLD. Together these findings both support DfMRI as a contrast that bears a closer spatial relationship to the underlying neural activity than BOLD, and raise important caveats regarding its utilization. Models explaining the DfMRI signal change need to consider the dynamic vascular contributions that may vary with neural activity.

Spatial summation revealed in the earliest visual evoked component C1 and the effect of attention on its linearity

In natural scenes, multiple objects are usually presented simultaneously. How do specific areas of the brain respond to multiple objects based on their responses to each individual object? Previous functional magnetic resonance imaging (fMRI) studies have shown that the activity induced by a multiobject stimulus in the primary visual cortex (V1) can be predicted by the linear or nonlinear sum of the activities induced by its component objects. However, there has been little evidence from electroencephelogram (EEG) studies so far. Here we explored how V1 responded to multiple objects by comparing the EEG signals evoked by a three-grating stimulus with those evoked by its two components (the central grating and 2 flanking gratings). We focused on the earliest visual component C1 (onset latency of ∼50 ms) because it has been shown to reflect the feedforward responses of neurons in V1. We found that when the stimulus was unattended, the amplitude of the C1 evoked by the three-grating stimulus roughly equaled the sum of the amplitudes of the C1s evoked by its two components, regardless of the distances between these gratings. When the stimulus was attended, this linear spatial summation existed only when the three gratings were far apart from each other. When the three gratings were close to each other, the spatial summation became compressed. These results suggest that the earliest visual responses in V1 follow a linear summation rule when attention is not involved and that attention can affect the earliest interactions between multiple objects.

Frequency-dependent spatiotemporal profiles of visual responses recorded with subdural ECoG electrodes in awake monkeys: Differences between high- and low-frequency activity

Electrocorticography (ECoG) constitutes a powerful and promising neural recording modality in humans and animals. ECoG signals are often decomposed into several frequency bands, among which the so-called high-gamma band (80-250Hz) has been proposed to reflect local cortical functions near the cortical surface below the ECoG electrodes. It is typically assumed that the lower the frequency bands, the lower the spatial resolution of the signals; thus, there is not much to gain by analyzing the event-related changes of the ECoG signals in the lower-frequency bands. However, differences across frequency bands have not been systematically investigated. To address this issue, we recorded ECoG activity from two awake monkeys performing a retinotopic mapping task. We characterized the spatiotemporal profiles of the visual responses in the time-frequency domain. We defined the preferred spatial position, receptive field (RF), and response latencies of band-limited power (BLP) (i.e., alpha [3.9-11.7Hz], beta [15.6-23.4Hz], low [30-80Hz] and high [80-250Hz] gamma) for each electrode and compared them across bands and time-domain visual evoked potentials (VEPs). At the population level, we found that the spatial preferences were comparable across bands and VEPs. The high-gamma power showed a smaller RF than the other bands and VEPs. The response latencies for the alpha band were always longer than the latencies for the other bands and fastest in VEPs. Comparing the response profiles in both space and time for each cortical region (V1, V4+, and TEO/TE) revealed regional idiosyncrasies. Although the latencies of visual responses in the beta, low-, and high-gamma bands were almost identical in V1 and V4+, beta and low-gamma BLP occurred about 17ms earlier than high-gamma power in TEO/TE. Furthermore, TEO/TE exhibited a unique pattern in the spatial response profile: the alpha and high-gamma responses tended to prefer the foveal regions, whereas the beta and low-gamma responses preferred the peripheral visual fields with larger RFs. This suggests that neurons in TEO/TE first receive less selective spatial information via beta and low-gamma BLP but later receive more fine-tuned spatial foveal information via high-gamma power. This result is consistent with a hypothesis previously proposed by Nakamura et al. (1993) that states that visual processing in TEO/TE starts with coarse-grained information, which primes subsequent fine-grained information. Collectively, our results demonstrate that ECoG can be a potent tool for investigating the nature of the neural computations in each cortical region that cannot be fully understood by measuring only the spiking activity, through the incorporation of the knowledge of the spatiotemporal characteristics across all frequency bands.

Computational neuroimaging and population receptive fields

Functional magnetic resonance imaging (fMRI) noninvasively measures human brain activity at millimeter resolution. Scientists use different approaches to take advantage of the remarkable opportunities presented by fMRI. Here, we describe progress using the computational neuroimaging approach in human visual cortex, which aims to build models that predict the neural responses from the stimulus and task. We focus on a particularly active area of research, the use of population receptive field (pRF) models to characterize human visual cortex responses to a range of stimuli, in a variety of tasks and different subject populations.

Comparing different stimulus configurations for population receptive field mapping in human fMRI

Population receptive field (pRF) mapping is a widely used approach to measuring aggregate human visual receptive field properties by recording non-invasive signals using functional MRI. Despite growing interest, no study to date has systematically investigated the effects of different stimulus configurations on pRF estimates from human visual cortex. Here we compared the effects of three different stimulus configurations on a model-based approach to pRF estimation: size-invariant bars and eccentricity-scaled bars defined in Cartesian coordinates and traveling along the cardinal axes, and a novel simultaneous “wedge and ring” stimulus defined in polar coordinates, systematically covering polar and eccentricity axes. We found that the presence or absence of eccentricity scaling had a significant effect on goodness of fit and pRF size estimates. Further, variability in pRF size estimates was directly influenced by stimulus configuration, particularly for higher visual areas including V5/MT+. Finally, we compared eccentricity estimation between phase-encoded and model-based pRF approaches. We observed a tendency for more peripheral eccentricity estimates using phase-encoded methods, independent of stimulus size. We conclude that both eccentricity scaling and polar rather than Cartesian stimulus configuration are important considerations for optimal experimental design in pRF mapping. While all stimulus configurations produce adequate estimates, simultaneous wedge and ring stimulation produced higher fit reliability, with a significant advantage in reduced acquisition time.

The neural code for face orientation in the human fusiform face area

Humans recognize faces and objects with high speed and accuracy regardless of their orientation. Recent studies have proposed that orientation invariance in face recognition involves an intermediate representation where neural responses are similar for mirror-symmetric views. Here, we used fMRI, multivariate pattern analysis, and computational modeling to investigate the neural encoding of faces and vehicles at different rotational angles. Corroborating previous studies, we demonstrate a representation of face orientation in the fusiform face-selective area (FFA). We go beyond these studies by showing that this representation is category-selective and tolerant to retinal translation. Critically, by controlling for low-level confounds, we found the representation of orientation in FFA to be compatible with a linear angle code. Aspects of mirror-symmetric coding cannot be ruled out when FFA mean activity levels are considered as a dimension of coding. Finally, we used a parametric family of computational models, involving a biased sampling of view-tuned neuronal clusters, to compare different face angle encoding models. The best fitting model exhibited a predominance of neuronal clusters tuned to frontal views of faces. In sum, our findings suggest a category-selective and monotonic code of face orientation in the human FFA, in line with primate electrophysiology studies that observed mirror-symmetric tuning of neural responses at higher stages of the visual system, beyond the putative homolog of human FFA.

Comparison of randomized multifocal mapping and temporal phase mapping of visual cortex for clinical use

fMRI is becoming an important clinical tool for planning and guidance of surgery to treat brain tumors, arteriovenous malformations, and epileptic foci. For visual cortex mapping, the most popular paradigm by far is temporal phase mapping, although random multifocal stimulation paradigms have drawn increased attention due to their ability to identify complex response fields and their random properties. In this study we directly compared temporal phase and multifocal vision mapping paradigms with respect to clinically relevant factors including: time efficiency, mapping completeness, and the effects of noise. Randomized, multifocal mapping accurately decomposed the response of single voxels to multiple stimulus locations and made correct retinotopic assignments as noise levels increased despite decreasing sensitivity. Also, multifocal mapping became less efficient as the number of stimulus segments (locations) increased from 13 to 25 to 49 and when duty cycle was increased from 25% to 50%. Phase mapping, on the other hand, activated more extrastriate visual areas, was more time efficient in achieving statistically significant responses, and had better sensitivity as noise increased, though with an increase in systematic retinotopic mis-assignments. Overall, temporal phase mapping is likely to be a better choice for routine clinical applications though random multifocal mapping may offer some unique advantages for selected applications.

•

Phase mapping activates more extrastriate visual areas and is more efficient per run.

•

Random mapping can decompose the response of single voxels to multiple locations.

•

Efficiency of random mapping depends on number of stimulus regions and duty cycle.

•

Noise affects random- and phase-mapping differently.

Minimizing biases in estimating the reorganization of human visual areas with BOLD retinotopic mapping

There is substantial interest in using functional magnetic resonance imaging (fMRI) retinotopic mapping techniques to examine reorganization of the occipital cortex after vision loss in humans and nonhuman primates. However, previous reports suggest that standard phase encoding and the more recent population Receptive Field (pRF) techniques give biased estimates of retinotopic maps near the boundaries of retinal or cortical scotomas. Here we examine the sources of this bias and show how it can be minimized with a simple modification of the pRF method. In normally sighted subjects, we measured fMRI responses to a stimulus simulating a foveal scotoma; we found that unbiased retinotopic map estimates can be obtained in early visual areas, as long as the pRF fitting algorithm takes the scotoma into account and a randomized "multifocal" stimulus sequence is used.

Neural and hemodynamic responses elicited by forelimb- and photo-stimulation in channelrhodopsin-2 mice: insights into the hemodynamic point spread function

Hemodynamic responses are commonly used to map brain activity; however, their spatial limits have remained unclear because of the lack of a well-defined and malleable spatial stimulus. To examine the properties of neural activity and hemodynamic responses, multiunit activity, local field potential, cerebral blood volume (CBV)-sensitive optical imaging, and laser Doppler flowmetry were measured from the somatosensory cortex of transgenic mice expressing Channelrhodopsin-2 in cortex Layer 5 pyramidal neurons. The magnitude and extent of neural and hemodynamic responses were modulated using different photo-stimulation parameters and compared with those induced by somatosensory stimulation. Photo-stimulation-evoked spiking activity across cortical layers was similar to forelimb stimulation, although their activity originated in different layers. Hemodynamic responses induced by forelimb- and photo-stimulation were similar in magnitude and shape, although the former were slightly larger in amplitude and wider in extent. Altogether, the neurovascular relationship differed between these 2 stimulation pathways, but photo-stimulation-evoked changes in neural and hemodynamic activities were linearly correlated. Hemodynamic point spread functions were estimated from the photo-stimulation data and its full-width at half-maximum ranged between 103 and 175 µm. Therefore, submillimeter functional structures separated by a few hundred micrometers may be resolved using hemodynamic methods, such as optical imaging and functional magnetic resonance imaging.

A new method for estimating population receptive field topography in visual cortex

We introduce a new method for measuring visual population receptive fields (pRF) with functional magnetic resonance imaging (fMRI). The pRF structure is modeled as a set of weights that can be estimated by solving a linear model that predicts the Blood Oxygen Level-Dependent (BOLD) signal using the stimulus protocol and the canonical hemodynamic response function. This method does not make a priori assumptions about the specific pRF shape and is therefore a useful tool for uncovering the underlying pRF structure at different spatial locations in an unbiased way. We show that our method is more accurate than a previously described method (Dumoulin and Wandell, 2008) which directly fits a 2-dimensional isotropic Gaussian pRF model to predict the fMRI time-series. We demonstrate that direct-fit models do not fully capture the actual pRF shape, and can be prone to pRF center mislocalization when the pRF is located near the border of the stimulus space. A quantitative comparison demonstrates that our method outperforms the direct-fit methods in the pRF center modeling by achieving higher explained variance of the BOLD signal. This was true for direct-fit isotropic Gaussian, anisotropic Gaussian, and difference of isotropic Gaussians model. Importantly, our model is also capable of exploring a variety of pRF properties such as surround suppression, receptive field center elongation, orientation, location and size. Additionally, the proposed method is particularly attractive for monitoring pRF properties in the visual areas of subjects with lesions of the visual pathways, where it is difficult to anticipate what shape the reorganized pRF might take. Finally, the method proposed here is more efficient in computation time than direct-fit methods, which need to search for a set of parameters in an extremely large searching space. Instead, this method uses the pRF topography to constrain the space that needs to be searched for the subsequent modeling.

Compressive spatial summation in human visual cortex

Neurons within a small (a few cubic millimeters) region of visual cortex respond to stimuli within a restricted region of the visual field. Previous studies have characterized the population response of such neurons using a model that sums contrast linearly across the visual field. In this study, we tested linear spatial summation of population responses using blood oxygenation level-dependent (BOLD) functional MRI. We measured BOLD responses to a systematic set of contrast patterns and discovered systematic deviation from linearity: the data are more accurately explained by a model in which a compressive static nonlinearity is applied after linear spatial summation. We found that the nonlinearity is present in early visual areas (e.g., V1, V2) and grows more pronounced in relatively anterior extrastriate areas (e.g., LO-2, VO-2). We then analyzed the effect of compressive spatial summation in terms of changes in the position and size of a viewed object. Compressive spatial summation is consistent with tolerance to changes in position and size, an important characteristic of object representation.

Visual spatial attention enhances the amplitude of positive and negative fMRI responses to visual stimulation in an eccentricity-dependent manner

Endogenous visual spatial attention improves perception and enhances neural responses to visual stimuli at attended locations. Although many aspects of visual processing differ significantly between central and peripheral vision, little is known regarding the neural substrates of the eccentricity dependence of spatial attention effects. We measured amplitudes of positive and negative fMRI responses to visual stimuli as a function of eccentricity in a large number of topographically-organized cortical areas. Responses to each stimulus were obtained when the stimulus was attended and when spatial attention was directed to a stimulus in the opposite visual hemifield. Attending to the stimulus increased both positive and negative response amplitudes in all cortical areas we studied: V1, V2, V3, hV4, VO1, LO1, LO2, V3A/B, IPS0, TO1, and TO2. However, the eccentricity dependence of these effects differed considerably across cortical areas. In early visual, ventral, and lateral occipital cortex, attentional enhancement of positive responses was greater for central compared to peripheral eccentricities. The opposite pattern was observed in dorsal stream areas IPS0 and putative MT homolog TO1, where attentional enhancement of positive responses was greater in the periphery. Both the magnitude and the eccentricity dependence of attentional modulation of negative fMRI responses closely mirrored that of positive responses across cortical areas.

In vivo evaluation of the effect of stimulus distribution on FIR statistical efficiency in event-related fMRI

Technical developments in MRI have improved signal to noise, allowing use of analysis methods such as Finite impulse response (FIR) of rapid event related functional MRI (er-fMRI). FIR is one of the most informative analysis methods as it determines onset and full shape of the hemodynamic response function (HRF) without any a priori assumptions. FIR is however vulnerable to multicollinearity, which is directly related to the distribution of stimuli over time. Efficiency can be optimized by simplifying a design, and restricting stimuli distribution to specific sequences, while more design flexibility necessarily reduces efficiency. However, the actual effect of efficiency on fMRI results has never been tested in vivo. Thus, it is currently difficult to make an informed choice between protocol flexibility and statistical efficiency. The main goal of this study was to assign concrete fMRI signal to noise values to the abstract scale of FIR statistical efficiency. Ten subjects repeated a perception task with five random and m-sequence based protocol, with varying but, according to literature, acceptable levels of multicollinearity. Results indicated substantial differences in signal standard deviation, while the level was a function of multicollinearity. Experiment protocols varied up to 55.4% in standard deviation. Results confirm that quality of fMRI in an FIR analysis can significantly and substantially vary with statistical efficiency. Our in vivo measurements can be used to aid in making an informed decision between freedom in protocol design and statistical efficiency.

Spatial summation of neurometabolic coupling in the central visual pathway

Noninvasive neural imaging has become an important tool in both applied and theoretical applications. The hemodynamic properties that are measured in functional magnetic resonance imaging (fMRI), for example, are generally used to infer neuronal characteristics. In an attempt to provide empirical data to connect the hemodynamic measurements with neural function, we have conducted previous studies in which neural activity and tissue oxygen metabolic functions are determined together in co-localized regions of the central visual pathway. A basic question in this procedure is whether oxygen responses are coupled linearly in space and time with neural activity. We have previously examined temporal factors, and in the current study, spatial characteristics are addressed. We have recorded from neurons in the lateral geniculate nucleus (LGN) and striate cortex in anesthetized cats. In both structures, there is a classical receptive field (CRF) within which a neuron can be activated. There is also a region outside the CRF from which stimulation cannot activate the cell directly but can influence the response elicited from the CRF. In this investigation we have used several specific spatial stimulus patterns presented to either the CRF or the surrounding region or to both areas together in order to determine spatial response patterns. Within the CRF, we find that neural and metabolic responses sum in a nonlinear fashion but changes in these two measurements are closely coupled. For stimuli that extend beyond the CRF, neural activity is generally reduced while oxygen response exhibits uncoupled changes.

Cortical representation of animate and inanimate objects in complex natural scenes

The representations of animate and inanimate objects appear to be anatomically and functionally dissociated in the primate brain. How much of the variation in object-category tuning across cortical locations can be explained in terms of the animate/inanimate distinction? How is the distinction between animate and inanimate reflected in the arrangement of object representations along the cortical surface? To investigate these issues we recorded BOLD activity in visual cortex while subjects viewed streams of natural scenes. We then constructed an explicit model of object-category tuning for each voxel along the cortical surface. We verified that these models accurately predict responses to novel scenes for voxels located in anterior visual areas, and that they can be used to accurately decode multiple objects simultaneously from novel scenes. Finally, we used principal components analysis to characterize the variation in object-category tuning across voxels. Remarkably, we found that the first principal component reflects the distinction between animate and inanimate objects. This dimension accounts for between 50 and 60% of the total variation in object-category tuning across voxels in anterior visual areas. The importance of the animate-inanimate distinction is further reflected in the arrangement of voxels on the cortical surface: voxels that prefer animate objects tend to be located anterior to retinotopic visual areas and are flanked by voxels that prefer inanimate objects. Our explicit model of object-category tuning thus explains the anatomical and functional dissociation of animate and inanimate objects.

Spikes, BOLD, attention, and awareness: a comparison of electrophysiological and fMRI signals in V1

Early fMRI studies comparing results from fMRI and electrophysiological experiments support the notion that the blood oxygen level-dependent (BOLD) signal reliably follows the spiking activity of an underlying neuronal population averaged across a small region in space and a brief period in time. However, more recent studies focusing on higher level cognitive factors such as attention and visual awareness report striking discrepancies between the fMRI response in humans and electrophysiological signals in macaque early visual areas. Four hypotheses are discussed that can explain the discrepancies between the two methods: (1) the BOLD signal follows local field potential (LFP) signals closer than spikes, and only the LFP is modulated by top-down factors, (2) the BOLD signal is reflecting electrophysiological signals that are occurring later in time due to feedback delay, (3) the BOLD signal is more sensitive than traditional electrophysiological methods due to massive pooling by the hemodynamic coupling process, and finally (4) there is no real discrepancy, and instead, weak but reliable effects on firing rates may be obscured by differences in experimental design and interpretation of results across methods.

Nonlinear hemodynamic responses in human epilepsy: a multimodal analysis with fNIRS-EEG and fMRI-EEG

Functional magnetic resonance imaging (fMRI) combined with electroencephalography (fMRI-EEG) is a neuroimaging technique based on the blood oxygenation level dependent (BOLD) signal which has been shown to be useful in the study of epilepsy for the localization of the epileptogenic focus. Functional near-infrared spectroscopy (fNIRS) combined with EEG (fNIRS-EEG) is another imaging technique based on the measurement of oxygenated and deoxygenated hemoglobin with complementary clinical potential in epilepsy, for continuous patient monitoring, language lateralization, and focus localization. In this work fMRI-EEG and fNIRS-EEG are used to quantify nonlinear hemodynamic responses in three cases of human refractory focal epilepsy, by using the Volterra kernel expansion up to second order. Prior to analyzing real data, extensive simulations are carried out to show that nonlinearities are estimable. The Volterra methodology is then applied to multimodal data recorded from 3 epileptic patients selected for their frequent spiking activity. Care is taken to account for variability of hemodynamic responses due to other causes than Volterra nonlinearities. Statistically significant nonlinearities are observed for all patients and all modalities. Good concordance between fNIRS and fMRI is found for both the amplitude of the Volterra responses, and, with limitations, in the localization of the epileptic focus and regions of inverted responses (negative BOLD signals). In one patient, Volterra nonlinearities allowed epileptic focus identification with fMRI, while analyses without nonlinearities failed to see it. In simulations when nonlinearities were included, analysis without Volterra nonlinearities performed poorly. These two observations suggest routinely checking for nonlinearities in functional imaging of patients presenting with frequent spikes.

Adaptive Kalman filtering for real-time mapping of the visual field

This paper demonstrates the feasibility of real-time mapping of the visual field for clinical applications. Specifically, three aspects of this problem were considered: (1) experimental design, (2) statistical analysis, and (3) display of results. Proper experimental design is essential to achieving a successful outcome, particularly for real-time applications. A random-block experimental design was shown to have less sensitivity to measurement noise, as well as greater robustness to error in modeling of the hemodynamic impulse response function (IRF) and greater flexibility than common alternatives. In addition, random encoding of the visual field allows for the detection of voxels that are responsive to multiple, not necessarily contiguous, regions of the visual field. Due to its recursive nature, the Kalman filter is ideally suited for real-time statistical analysis of visual field mapping data. An important feature of the Kalman filter is that it can be used for nonstationary time series analysis. The capability of the Kalman filter to adapt, in real time, to abrupt changes in the baseline arising from subject motion inside the scanner and other external system disturbances is important for the success of clinical applications. The clinician needs real-time information to evaluate the success or failure of the imaging run and to decide whether to extend, modify, or terminate the run. Accordingly, the analytical software provides real-time displays of (1) brain activation maps for each stimulus segment, (2) voxel-wise spatial tuning profiles, (3) time plots of the variability of response parameters, and (4) time plots of activated volume.

The development and use of phase-encoded functional MRI designs

Phase-encoded designs advanced the early development of functional MRI, enabling the "killer app" of retinotopic mapping, which helped demonstrate fMRI's value to a skeptical scientific public. The design, also called "the traveling wave", remains in wide use today, due to its ability to easily measure neural activity in a parameterized set of experimental conditions. In phase-encoded designs, stimuli defined by a numerical parameter, for example visual eccentricity, are presented continuously in the order specified by the parameter. The stimulus parameter that produces maximum response can be recovered from the timing of neural activity, i.e. its phase. From the outset, phase-encoded designs were used for two related, but complementary purposes: 1) to measure aggregate response properties of neurons in a voxel, for example the average visual field location of receptive fields, and 2) to segregate the set of voxels that corresponds to an organized cortical region, for example a retinotopically mapped visual area. This short review will cover the history and current uses of phase-encoded fMRI, while noting the ongoing tension in the field between the brain mapping and computational neuroimaging approaches.

Reconstructing visual experiences from brain activity evoked by natural movies

Quantitative modeling of human brain activity can provide crucial insights about cortical representations [1, 2] and can form the basis for brain decoding devices [3-5]. Recent functional magnetic resonance imaging (fMRI) studies have modeled brain activity elicited by static visual patterns and have reconstructed these patterns from brain activity [6-8]. However, blood oxygen level-dependent (BOLD) signals measured via fMRI are very slow [9], so it has been difficult to model brain activity elicited by dynamic stimuli such as natural movies. Here we present a new motion-energy [10, 11] encoding model that largely overcomes this limitation. The model describes fast visual information and slow hemodynamics by separate components. We recorded BOLD signals in occipitotemporal visual cortex of human subjects who watched natural movies and fit the model separately to individual voxels. Visualization of the fit models reveals how early visual areas represent the information in movies. To demonstrate the power of our approach, we also constructed a Bayesian decoder [8] by combining estimated encoding models with a sampled natural movie prior. The decoder provides remarkable reconstructions of the viewed movies. These results demonstrate that dynamic brain activity measured under naturalistic conditions can be decoded using current fMRI technology.

Imaging retinotopic maps in the human brain

A quarter-century ago visual neuroscientists had little information about the number and organization of retinotopic maps in human visual cortex. The advent of functional magnetic resonance imaging (MRI), a non-invasive, spatially-resolved technique for measuring brain activity, provided a wealth of data about human retinotopic maps. Just as there are differences amongst non-human primate maps, the human maps have their own unique properties. Many human maps can be measured reliably in individual subjects during experimental sessions lasting less than an hour. The efficiency of the measurements and the relatively large amplitude of functional MRI signals in visual cortex make it possible to develop quantitative models of functional responses within specific maps in individual subjects. During this last quarter-century, there has also been significant progress in measuring properties of the human brain at a range of length and time scales, including white matter pathways, macroscopic properties of gray and white matter, and cellular and molecular tissue properties. We hope the next 25years will see a great deal of work that aims to integrate these data by modeling the network of visual signals. We do not know what such theories will look like, but the characterization of human retinotopic maps from the last 25years is likely to be an important part of future ideas about visual computations.

A neuronal basis for task-negative responses in the human brain

Neuroimaging studies have revealed a number of brain regions that show a reduced blood oxygenation level–dependent (BOLD) signal during externally directed tasks compared with a resting baseline. These regions constitute a network whose operation has become known as the default mode. The source of functional magnetic resonance imaging (fMRI) signal reductions in the default mode during task performance has not been resolved, however. It may be attributable to neuronal effects (neuronal firing), physiological effects (e.g., task vs. rest differences in respiration rate), or even increases in neuronal activity with an atypical blood response. To establish the source of signal decreases in the default mode, we used the calibrated fMRI method to quantify changes in the cerebral metabolic rate of oxygen (CMRO₂) and cerebral blood flow (CBF) in those regions that typically show reductions in BOLD signal during a demanding cognitive task. CBF:CMRO₂ coupling during task-negative responses were linear, with a coupling constant similar to that in task-positive regions, indicating a neuronal source for signal reductions in multiple brain areas. We also identify, for the first time, two modes of neuronal activity in this network; one in which greater deactivation (characterized by metabolic rate reductions) is associated with more effort and one where it is associated with less effort.

Laminar analysis of 7T BOLD using an imposed spatial activation pattern in human V1

With sufficient image encoding, high-resolution fMRI studies are limited by the biological point-spread of the hemodynamic signal. The extent of this spread is determined by the local vascular distribution and by the spatial specificity of blood flow regulation, as well as by measurement parameters that (i) alter the relative sensitivity of the acquisition to activation-induced hemodynamic changes and (ii) determine the image contrast as a function of vessel size. In particular, large draining vessels on the cortical surface are a major contributor to both the BOLD signal change and to the spatial bias of the BOLD activation away from the site of neuronal activity. In this work, we introduce a laminar surface-based analysis method and study the relationship between spatial localization and activation strength as a function of laminar depth by acquiring 1mm isotropic, single-shot EPI at 7 T and sampling the BOLD signal exclusively from the superficial, middle, or deep cortical laminae. We show that highly-accelerated EPI can limit image distortions to the point where a boundary-based registration algorithm accurately aligns the EPI data to the surface reconstruction. The spatial spread of the BOLD response tangential to the cortical surface was analyzed as a function of cortical depth using our surface-based analysis. Although sampling near the pial surface provided the highest signal strength, it also introduced the most spatial error. Thus, avoiding surface laminae improved spatial localization by about 40% at a cost of 36% in z-statistic, implying that optimal spatial resolution in functional imaging of the cortex can be achieved using anatomically-informed spatial sampling to avoid large pial vessels.

Attention and biased competition in multi-voxel object representations

The biased-competition theory accounts for attentional effects at the single-neuron level: It predicts that the neuronal response to simultaneously-presented stimuli is a weighted average of the response to isolated stimuli, and that attention biases the weights in favor of the attended stimulus. Perception, however, relies not on single neurons but on larger neuronal populations. The responses of such populations are in part reflected in large-scale multivoxel fMRI activation patterns. Because the pooling of neuronal responses into blood-oxygen-level-dependent signals is nonlinear, fMRI effects of attention need not mirror those observed at the neuronal level. Thus, to bridge the gap between neuronal responses and human perception, it is fundamental to understand attentional influences in large-scale multivariate representations of simultaneously-presented objects. Here, we ask how responses to simultaneous stimuli are combined in multivoxel fMRI patterns, and how attention affects the paired response. Objects from four categories were presented singly, or in pairs such that each category was attended, unattended, or attention was divided between the two. In a multidimensional voxel space, the response to simultaneously-presented categories was well described as a weighted average. The weights were biased toward the preferred category in category-selective regions. Consistent with single-unit reports, attention shifted the weights by approximately 30% in favor of the attended stimulus. These findings extend the biased-competition framework to the realm of large-scale multivoxel brain activations.

Comprehension of degraded speech sounds with m-sequence modulation: an fMRI study

In a recent electroencephalography (EEG) study (Takeichi et al., 2007a), we developed a new technique for assessing speech comprehension using speech degraded by m-sequence modulation and found a correlation peak with a 400-ms delay. This peak depended on the comprehensibility of the modulated speech sounds. Here we report the results of a functional magnetic resonance imaging (fMRI) experiment comparable to our previous EEG experiment. We examined brain areas related to verbal comprehension of the modulated speech sound to examine which neural system processes this modulated speech. A non-integer, alternating-block factorial design was used with 23 Japanese-speaking participants, with time reversal and m-sequence modulation as factors. A main effect of time reversal was found in the left temporal cortex along the superior temporal sulcus (BA21 and BA39), left precentral gyrus (BA6) and right inferior temporal gyrus (BA21). A main effect of modulation was found in the left postcentral gyrus (BA43) and the right medial frontal gyri (BA6) as an increase by modulation and in the left temporal cortex (BA21, 39), parahippocampal gyrus (BA34), posterior cingulate (BA23), caudate and thalamus and right superior temporal gyrus (BA38) as a decrease by modulation. An interaction effect associated specifically with non-modulated speech was found in the left frontal gyrus (BA47), left occipital cortex in the cuneus (BA18), left precuneus (BA7, 31), right precuneus (BA31) and right thalamus (forward>reverse). The other interaction effect associated specifically with modulation of speech sound was found in the inferior frontal gyrus in the opercular area (BA44) (forward>reverse). Estimated scalp projection of the component correlation function (Cao et al., 2002) for the corresponding EEG data (Takeichi et al., 2007a, showed leftward dominance. Hence, activities in the superior temporal sulcus (BA21 and BA39), which are commonly observed for speech processing, as well as left precentral gyrus (BA6) and left inferior frontal gyrus in the opercular area (BA44) is suggested to contribute to the comprehension-related EEG signal.

Visual image reconstruction from human brain activity using a combination of multiscale local image decoders

Perceptual experience consists of an enormous number of possible states. Previous fMRI studies have predicted a perceptual state by classifying brain activity into prespecified categories. Constraint-free visual image reconstruction is more challenging, as it is impractical to specify brain activity for all possible images. In this study, we reconstructed visual images by combining local image bases of multiple scales, whose contrasts were independently decoded from fMRI activity by automatically selecting relevant voxels and exploiting their correlated patterns. Binary-contrast, 10 x 10-patch images (2(100) possible states) were accurately reconstructed without any image prior on a single trial or volume basis by measuring brain activity only for several hundred random images. Reconstruction was also used to identify the presented image among millions of candidates. The results suggest that our approach provides an effective means to read out complex perceptual states from brain activity while discovering information representation in multivoxel patterns.

Quantitative multifocal fMRI shows active suppression in human V1

Multifocal functional magnetic resonance imaging has recently been introduced as an alternative method for retinotopic mapping, and it enables effective functional localization of multiple regions-of-interest in the visual cortex. In this study we characterized interactions in V1 with spatially and temporally identical stimuli presented alone, or as a part of a nine-region multifocal stimulus. We compared stimuli at different contrasts, collinear and orthogonal orientations and spatial frequencies one octave apart. Results show clear attenuation of BOLD signal from the central region in the multifocal condition. The observed modulation in BOLD signal could be produced either by neural suppression resulting from stimulation of adjacent regions of visual field, or alternatively by hemodynamic saturation or stealing effects in V1. However, we find that attenuation of the central response persists through a range of contrasts, and that its strength varies with relative orientation and spatial frequency of the central and surrounding stimulus regions, indicating active suppression mechanisms of neural origin. Our results also demonstrate that the extent of the signal spreading is commensurate with the extent of the horizontal connections in primate V1.

The multisensory attentional consequences of tool use: a functional magnetic resonance imaging study

Background

Tool use in humans requires that multisensory information is integrated across different locations, from objects seen to be distant from the hand, but felt indirectly at the hand via the tool. We tested the hypothesis that using a simple tool to perceive vibrotactile stimuli results in the enhanced processing of visual stimuli presented at the distal, functional part of the tool. Such a finding would be consistent with a shift of spatial attention to the location where the tool is used.

Methodology/Principal Findings

We tested this hypothesis by scanning healthy human participants' brains using functional magnetic resonance imaging, while they used a simple tool to discriminate between target vibrations, accompanied by congruent or incongruent visual distractors, on the same or opposite side to the tool. The attentional hypothesis was supported: BOLD response in occipital cortex, particularly in the right hemisphere lingual gyrus, varied significantly as a function of tool position, increasing contralaterally, and decreasing ipsilaterally to the tool. Furthermore, these modulations occurred despite the fact that participants were repeatedly instructed to ignore the visual stimuli, to respond only to the vibrotactile stimuli, and to maintain visual fixation centrally. In addition, the magnitude of multisensory (visual-vibrotactile) interactions in participants' behavioural responses significantly predicted the BOLD response in occipital cortical areas that were also modulated as a function of both visual stimulus position and tool position.

Conclusions/Significance

These results show that using a simple tool to locate and to perceive vibrotactile stimuli is accompanied by a shift of spatial attention to the location where the functional part of the tool is used, resulting in enhanced processing of visual stimuli at that location, and decreased processing at other locations. This was most clearly observed in the right hemisphere lingual gyrus. Such modulations of visual processing may reflect the functional importance of visuospatial information during human tool use.

Modeling low-frequency fluctuation and hemodynamic response timecourse in event-related fMRI

Functional magnetic resonance imaging (fMRI) suffers from many problems that make signal estimation difficult. These include variation in the hemodynamic response across voxels and low signal-to-noise ratio (SNR). We evaluate several analysis techniques that address these problems for event-related fMRI. (1) Many fMRI analyses assume a canonical hemodynamic response function, but this assumption may lead to inaccurate data models. By adopting the finite impulse response model, we show that voxel-specific hemodynamic response functions can be estimated directly from the data. (2) There is a large amount of low-frequency noise fluctuation (LFF) in blood oxygenation level dependent (BOLD) time-series data. To compensate for this problem, we use polynomials as regressors for LFF. We show that this technique substantially improves SNR and is more accurate than high-pass filtering of the data. (3) Model overfitting is a problem for the finite impulse response model because of the low SNR of the BOLD response. To reduce overfitting, we estimate a hemodynamic response timecourse for each voxel and incorporate the constraint of time-event separability, the constraint that hemodynamic responses across event types are identical up to a scale factor. We show that this technique substantially improves the accuracy of hemodynamic response estimates and can be computed efficiently. For the analysis techniques we present, we evaluate improvement in modeling accuracy via 10-fold cross-validation.

Identifying natural images from human brain activity

A challenging goal in neuroscience is to be able to read out, or decode, mental content from brain activity. Recent functional magnetic resonance imaging (fMRI) studies have decoded orientation, position and object category from activity in visual cortex. However, these studies typically used relatively simple stimuli (for example, gratings) or images drawn from fixed categories (for example, faces, houses), and decoding was based on previous measurements of brain activity evoked by those same stimuli or categories. To overcome these limitations, here we develop a decoding method based on quantitative receptive-field models that characterize the relationship between visual stimuli and fMRI activity in early visual areas. These models describe the tuning of individual voxels for space, orientation and spatial frequency, and are estimated directly from responses evoked by natural images. We show that these receptive-field models make it possible to identify, from a large set of completely novel natural images, which specific image was seen by an observer. Identification is not a mere consequence of the retinotopic organization of visual areas; simpler receptive-field models that describe only spatial tuning yield much poorer identification performance. Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone.

Visual field maps in human cortex

Much of the visual cortex is organized into visual field maps: nearby neurons have receptive fields at nearby locations in the image. Mammalian species generally have multiple visual field maps with each species having similar, but not identical, maps. The introduction of functional magnetic resonance imaging made it possible to identify visual field maps in human cortex, including several near (1) medial occipital (V1,V2,V3), (2) lateral occipital (LO-1,LO-2, hMT+), (3) ventral occipital (hV4, VO-1, VO-2), (4) dorsal occipital (V3A, V3B), and (5) posterior parietal cortex (IPS-0 to IPS-4). Evidence is accumulating for additional maps, including some in the frontal lobe. Cortical maps are arranged into clusters in which several maps have parallel eccentricity representations, while the angular representations within a cluster alternate in visual field sign. Visual field maps have been linked to functional and perceptual properties of the visual system at various spatial scales, ranging from the level of individual maps to map clusters to dorsal-ventral streams. We survey recent measurements of human visual field maps, describe hypotheses about the function and relationships between maps, and consider methods to improve map measurements and characterize the response properties of neurons comprising these maps.

Topographic organization in and near human visual area V4

The existence and location of a human counterpart of macaque visual area V4 are disputed. To resolve this issue, we used functional magnetic resonance imaging to obtain topographic maps from human subjects, using visual stimuli and tasks designed to maximize accuracy of topographic maps of the fovea and parafovea and to measure the effects of attention on topographic maps. We identified multiple topographic transitions, each clearly visible in > or = 75% of the maps, that we interpret as boundaries of distinct cortical regions. We call two of these regions dorsal V4 and ventral V4 (together comprising human area V4) because they share several defining characteristics with the macaque regions V4d and V4v (which together comprise macaque area V4). Ventral V4 is adjacent to V3v, and dorsal V4 is adjacent to parafoveal V3d. Ventral V4 and dorsal V4 meet in the foveal confluence shared by V1, V2, and V3. Ventral V4 and dorsal V4 represent complementary regions of the visual field, because ventral V4 represents the upper field and a subregion of the lower field, whereas dorsal V4 represents lower-field locations that are not represented by ventral V4. Finally, attentional modulation of spatial tuning is similar across dorsal and ventral V4, but attention has a smaller effect in V3d and V3v and a larger effect in a neighboring lateral occipital region.