GLMdenoise: a fast, automated technique for denoising task-based fMRI data

Unpacking the V1 map: Differential covariation of visual properties across spatial dimensions

Primary visual cortex (V1) has played a key role in understanding the organization of cerebral cortex. Both structural and functional properties vary sharply throughout the human V1 map. Despite large variation, underlying constancies computed from the covariation pattern of V1 properties have been proposed. Such constancies would imply that V1 is composed of multiple identical units whose visual properties differ only due to differences in their inputs. To test this, we used fMRI to investigate how V1 cortical magnification and preferred spatial frequency covary across eccentricity and polar angle, measured in 40 observers. V1 cortical magnification and preferred spatial frequency were strongly correlated across eccentricity and around polar angle, however their relation differed between these dimensions: they were proportional across eccentricity but not polar angle. The constant ratio of cortical magnification to preferred spatial frequency when measured as a function of eccentricity suggests a shared underlying cause of variation in the two properties, e.g., the gradient of retinal ganglion cell density across eccentricity. In contrast, the deviation from proportionality around polar angle implies that cortical variation differs from that in retina along this dimension. Thus, a constancy hypothesis is supported for one of the two spatial dimensions of V1, highlighting the importance of examining the full 2D-map to understand how V1 is organized.

Object representations drive emotion schemas across a large and diverse set of daily-life scenes

The rapid emotional evaluation of objects and events is essential in daily life. While visual scenes reliably evoke emotions, it remains unclear whether emotion schemas evoked by daily-life scenes depend on object processing systems or are extracted independently. To explore this, we collected emotion ratings for 4913 daily-life scenes from 300 participants, and predicted these ratings from representations in deep neural networks and functional magnetic resonance imaging (fMRI) activity patterns in visual cortex. AlexNet, an object-based model, outperformed EmoNet, an emotion-based model, in predicting emotion ratings for daily-life scenes, while EmoNet excelled for explicitly evocative scenes. Emotion information was processed hierarchically within the object recognition system, consistent with the visual cortex's organization. Activity patterns in the lateral occipital complex (LOC), an object-selective region, reliably predicted emotion ratings and outperformed other visual regions. These findings suggest that the emotional evaluation of daily-life scenes is mediated by visual object processing, with additional mechanisms engaged when object content is uninformative.

A neural geometry approach comprehensively explains apparently conflicting models of visual perceptual learning

Visual perceptual learning (VPL), defined as long-term improvement in a visual task, is considered a crucial tool for elucidating underlying visual and brain plasticity. Previous studies have proposed several neural models of VPL, including changes in neural tuning or in noise correlations. Here, to adjudicate different models, we propose that all neural changes at single units can be conceptualized as geometric transformations of population response manifolds in a high-dimensional neural space. Following this neural geometry approach, we identified neural manifold shrinkage due to reduced trial-by-trial population response variability, rather than tuning or correlation changes, as the primary mechanism of VPL. Furthermore, manifold shrinkage successfully explains VPL effects across artificial neural responses in deep neural networks, multivariate blood-oxygenation-level-dependent signals in humans and multiunit activities in monkeys. These converging results suggest that our neural geometry approach comprehensively explains a wide range of empirical results and reconciles previously conflicting models of VPL.

Feedback scales the spatial tuning of cortical responses during both visual working memory and long-term memory

Perception, working memory, and long-term memory each evoke neural responses in visual cortex. While previous neuroimaging research on the role of visual cortex in memory has largely emphasized similarities between perception and memory, we hypothesized that responses in visual cortex would differ depending on the origins of the inputs. Using fMRI, we quantified spatial tuning in visual cortex while participants (both sexes) viewed, maintained in working memory, or retrieved from long-term memory a peripheral target. In each condition, BOLD responses were spatially tuned and aligned with the target's polar angle in all measured visual field maps including V1. As expected given the increasing sizes of receptive fields, polar angle tuning during perception increased in width up the visual hierarchy from V1 to V2, V3, hV4, and beyond. In stark contrast, the tuned responses were broad across the visual hierarchy during long-term memory (replicating a prior result) and during working memory. This pattern is consistent with the idea that mnemonic responses in V1 stem from top-down sources, even when the stimulus was recently viewed and is held in working memory. Moreover, in long-term memory, trial-to-trial biases in these tuned responses (clockwise or counterclockwise of target), predicted matched biases in memory, suggesting that the reinstated cortical responses influence memory guided behavior. We conclude that feedback widens spatial tuning in visual cortex during memory, where earlier visual maps inherit broader tuning from later maps thereby impacting the precision of memory.Significance Statement We demonstrate that remembering a visual stimulus evokes responses in visual cortex that differ in spatial extent compared to seeing the same stimulus. Perception evokes tuned responses in early visual areas that increase in size up the visual hierarchy. Prior work showed that feedback inputs associated with long-term memory originate from later visual areas with larger receptive fields resulting in uniformly wide spatial tuning even in primary visual cortex. We replicate these results and show that the same pattern holds when maintaining in working memory a recently viewed stimulus. That trial-to-trial difficulty is reflected in the accuracy and precision of these representations suggests that visual cortex is flexibly used for processing visuospatial information, regardless of where that information originates.

Visual homogeneity computations in the brain enable solving property-based visual tasks

Most visual tasks involve looking for specific object features. But we also often perform property-based tasks where we look for specific property in an image, such as finding an odd item, deciding if two items are same, or if an object has symmetry. How do we solve such tasks? These tasks do not fit into standard models of decision making because their underlying feature space and decision process is unclear. Using well-known principles governing multiple object representations, we show that displays with repeating elements can be distinguished from heterogeneous displays using a property we define as visual homogeneity. In behavior, visual homogeneity predicted response times on visual search, same-different and symmetry tasks. Brain imaging during visual search and symmetry tasks revealed that visual homogeneity was localized to a region in the object-selective cortex. Thus, property-based visual tasks are solved in a localized region in the brain by computing visual homogeneity.

Spatial Frequency Maps in Human Visual Cortex: A Replication and Extension

In a step toward developing a model of human primary visual cortex, a recent study introduced a model of spatial frequency tuning in V1 (Broderick, Simoncelli, & Winawer, 2022). The model is compact, using just 9 parameters to predict BOLD response amplitude for locations across all of V1 as a function of stimulus orientation and spatial frequency. Here we replicated this analysis in a new dataset, the 'nsdsynthetic' supplement to the Natural Scenes Dataset (Allen et al., 2022), to assess generalization of model parameters. Furthermore, we extended the analyses to extrastriate maps V2 and V3. For each retinotopic map in the 8 NSD subjects, we fit the 9-parameter model. Despite many experimental differences between NSD and the original study, including stimulus size, experimental design, and MR field strength, there was good agreement in most model parameters. The dependence of preferred spatial frequency on eccentricity in V1 was similar between NSD and Broderick et al. Moreover, the effect of absolute stimulus orientation on spatial frequency maps was similar: higher preferred spatial frequency for horizontal and cardinal orientations compared to vertical and oblique orientations in both studies. The extension to extrastriate maps revealed that the biggest change in tuning between maps was in bandwidth: the bandwidth in spatial frequency tuning increased by 70% from V1 to V2 and 100% from V1 to V3, paralleling known increases in receptive field size. Together, the results show robust reproducibility and bring us closer to a systematic characterization of spatial encoding in the human visual system.

The neural link between stimulus duration and spatial location in the human visual hierarchy

Integrating spatial and temporal information is essential for our sensory experience. While psychophysical evidence suggests spatial dependencies in duration perception, few studies have directly tested the neural link between temporal and spatial processing. Using ultra-high-field functional MRI and neuronal-based modeling, we investigated how and where the processing and the representation of a visual stimulus duration is linked to that of its spatial location. Our results show a transition in duration coding: from monotonic and spatially-dependent in early visual cortex to unimodal and spatially-invariant in frontal cortex. Along the dorsal visual stream, particularly in the intraparietal sulcus (IPS), neuronal populations show common selective responses to both spatial and temporal information. In the IPS, spatial and temporal topographic organizations are also linked, although duration maps are smaller, less clustered, and more variable across participants. These findings help identify the mechanisms underlying human perception of visual duration and characterize the functional link between time and space processing, highlighting the importance of their interactions in shaping brain responses.

An fMRI dataset in response to large-scale short natural dynamic facial expression videos

Facial expression is among the most natural methods for human beings to convey their emotional information in daily life. Although the neural mechanisms of facial expression have been extensively studied employing lab-controlled images and a small number of lab-controlled video stimuli, how the human brain processes natural dynamic facial expression videos still needs to be investigated. To our knowledge, this type of data specifically on large-scale natural facial expression videos is currently missing. We describe here the natural Facial Expressions Dataset (NFED), an fMRI dataset including responses to 1,320 short (3-second) natural facial expression video clips. These video clips are annotated with three types of labels: emotion, gender, and ethnicity, along with accompanying metadata. We validate that the dataset has good quality within and across participants and, notably, can capture temporal and spatial stimuli features. NFED provides researchers with fMRI data for understanding of the visual processing of large number of natural facial expression videos.

Using imagination and the contents of memory to create new scene and object representations: A functional MRI study

Humans can use the contents of memory to construct scenarios and events that they have not encountered before, a process colloquially known as imagination. Much of our current understanding of the neural mechanisms mediating imagination is limited by paradigms that rely on participants' subjective reports of imagined content. Here, we used a novel behavioral paradigm that was designed to systematically evaluate the contents of an individual's imagination. Participants first learned the layout of four distinct rooms containing five wall segments with differing geometrical characteristics, each associated with a unique object. During functional MRI, participants were then shown two different wall segments or objects on each trial and asked to first, retrieve the associated objects or walls, respectively (retrieval phase) and then second, imagine the two objects side-by-side or combine the two wall segments (imagination phase). Importantly, the contents of each participant's imagination were interrogated by having them make a same/different judgment about the properties of the imagined objects or scenes. Using univariate and multivariate analyses, we observed widespread activity across occipito-temporal cortex for the retrieval of objects and for the imaginative creation of scenes. Interestingly, a classifier, whether trained on the imagination or retrieval data, was able to successfully differentiate the neural patterns associated with the imagination of scenes from that of objects. Our results reveal neural differences in the cued retrieval of object and scene memoranda, demonstrate that different representations underlie the creation and/or imagination of scene and object content, and highlight a novel behavioral paradigm that can be used to systematically evaluate the contents of an individual's imagination.

Sensory representations in primary visual cortex are not sufficient for subjective imagery

The contemporary definition of mental imagery is characterized by two aspects: a sensory representation that resembles, but does not result from, perception, and an associated subjective experience. Neuroimaging demonstrated imagery-related sensory representations in primary visual cortex (V1) that show striking parallels to perception. However, it remains unclear whether these representations always reflect subjective experience or if they can be dissociated from it. We addressed this question by comparing sensory representations and subjective imagery among visualizers and aphantasics, the latter with an impaired ability to experience imagery. Importantly, to test for the presence of sensory representations independently of the ability to generate imagery on demand, we examined both spontaneous and voluntary imagery forms. Using multivariate fMRI, we tested for decodable sensory representations in V1 and subjective visual imagery reports that occurred either spontaneously (during passive listening of evocative sounds) or in response to the instruction to voluntarily generate imagery of the sound content (always while blindfolded inside the scanner). Among aphantasics, V1 decoding of sound content was at chance during voluntary imagery, and lower than in visualizers, but it succeeded during passive listening, despite them reporting no imagery. In contrast, in visualizers, decoding accuracy in V1 was greater in voluntary than spontaneous imagery (while being positively associated with the reported vividness of both imagery types). Finally, for both conditions, decoding in precuneus was successful in visualizers but at chance for aphantasics. Together, our findings show that V1 representations can be dissociated from subjective imagery, while implicating a key role of precuneus in the latter.

NOise Reduction with DIstribution Corrected (NORDIC) principal component analysis improves brain activity detection across rodent and human functional MRI contexts

NOise Reduction with DIstribution Corrected (NORDIC) principal component analysis (PCA) has been shown to selectively suppress thermal noise and improve the temporal signal-to-noise ratio (tSNR) in human functional magnetic resonance imaging (fMRI). However, the feasibility to improve data quality for rodent fMRI using NORDIC PCA remains uncertain. NORDIC PCA may also be particularly beneficial for improving topological brain mapping, as conventional mapping requires precise spatiotemporal signals from large datasets (ideally ~1 hour acquisition) for individual representations. In this study, we evaluated the effects of NORDIC PCA compared with "Standard" processing in various rodent fMRI contexts that range from task-evoked optogenetic fMRI to resting-state fMRI. We also evaluated the effects of NORDIC PCA on human resting-state and retinotopic mapping fMRI via population receptive field (pRF) modeling. In rodent optogenetic fMRI, apart from doubling the tSNR, NORDIC PCA resulted in a larger number of activated voxels and a significant decrease in the variance of evoked brain responses without altering brain morphology. In rodent resting-state fMRI, we found that NORDIC PCA induced a nearly threefold increase in tSNR and preserved task-free relative cerebrovascular reactivity (rCVR) across cortical depth. NORDIC PCA further improved the detection of TGN020-induced aquaporin-4 inhibition on rCVR compared with Standard processing without NORDIC PCA. NORDIC PCA also increased the tSNR for both human resting-state and pRF fMRI, and for the latter also increased activation cluster sizes while retaining retinotopic organization. This suggests that NORDIC PCA preserves the spatiotemporal precision of fMRI signals needed for pRF analysis, and effectively captures small activity changes with high sensitivity. Taken together, these results broadly demonstrate the value of NORDIC PCA for the enhanced detection of neural dynamics across various rodent and human fMRI contexts. This can in turn play an important role in improving fMRI image quality and sensitivity for translational and preclinical neuroimaging research.

Improving source estimation of retinotopic MEG responses by combining data from multiple subjects

Magnetoencephalography (MEG) is a functional brain imaging modality, which measures the weak magnetic field arising from neuronal activity. The source amplitudes and locations are estimated from the sensor data by solving an ill-posed inverse problem. Commonly used solutions for these problems operate on data from individual subjects. Combining the measurements of multiple subjects has been suggested to increase the spatial resolution of MEG by leveraging the intersubject differences for increased information. In this article, we compare 3 multisubject analysis methods on a retinotopic mapping dataset recorded from 20 subjects. The compared methods are eLORETA with source-space averaging, minimum Wasserstein estimates (MWE), and MWE with source-space averaging. The results were quantified by the geodesic distances between early (60-100 ms) MEG peak activations and fMRI-based retinotopic target points in the primary visual cortex (V1). By increasing the subject count from 1 to 10, the median distances decreased by 6.6-9.4 mm (33-46%) compared with the single-subject median distances of around 20 mm. The observed peak activation locations with multisubject analysis also comply better with the established retinotopic maps of the primary visual cortex. Our results suggest that higher spatial accuracy can be achieved by pooling data from multiple subjects. The strength of MWE lies in individualized and sparse source estimates, but in our data, averaging eLORETA estimates across individuals in source space outperformed MWE in spatial accuracy.

Evaluating the effect of denoising submillimeter auditory fMRI data with NORDIC

Functional magnetic resonance imaging (fMRI) has emerged as an essential tool for exploring human brain function. Submillimeter fMRI, in particular, has emerged as a tool to study mesoscopic computations. The inherently low signal-to-noise ratio (SNR) at submillimeter resolutions warrants the use of denoising approaches tailored at reducing thermal noise-the dominant contributing noise component in high-resolution fMRI. NOise Reduction with DIstribution Corrected Principal Component Analysis (NORDIC PCA) is one of such approaches, and has been benchmarked against other approaches in several applications. Here, we investigate the effects that two versions of NORDIC denoising have on auditory submillimeter data. While investigating auditory functional responses poses unique challenges, we anticipated NORDIC to have a positive impact on the data on the basis of previous applications. Our results show that NORDIC denoising improves the detection sensitivity and the reliability of estimates in submillimeter auditory fMRI data. These effects can be explained by the reduction of the noise-induced signal variability. However, we did observe a reduction in the average response amplitude (percent signal change) within regions of interest, which may suggest that a portion of the signal of interest, which could not be distinguished from general i.i.d. noise, was also removed. We conclude that, while evaluating the effects of the signal reduction induced by NORDIC may be necessary for each application, using NORDIC in high-resolution auditory fMRI studies may be advantageous because of the large reduction in variability of the estimated responses.

Cortical Face-Selective Responses Emerge Early in Human Infancy

In human adults, multiple cortical regions respond robustly to faces, including the occipital face area (OFA) and fusiform face area (FFA), implicated in face perception, and the superior temporal sulcus (STS) and medial prefrontal cortex (MPFC), implicated in higher-level social functions. When in development, does face selectivity arise in each of these regions? Here, we combined two awake infant functional magnetic resonance imaging (fMRI) datasets to create a sample size twice the size of previous reports (n = 65 infants; 2.6-9.6 months). Infants watched movies of faces, bodies, objects, and scenes, while fMRI data were collected. Despite variable amounts of data from each infant, individual subject whole-brain activation maps revealed responses to faces compared to nonface visual categories in the approximate location of OFA, FFA, STS, and MPFC. To determine the strength and nature of face selectivity in these regions, we used cross-validated functional region of interest analyses. Across this larger sample size, face responses in OFA, FFA, STS, and MPFC were significantly greater than responses to bodies, objects, and scenes. Even the youngest infants (2-5 months) showed significantly face-selective responses in FFA, STS, and MPFC, but not OFA. These results demonstrate that face selectivity is present in multiple cortical regions within months of birth, providing powerful constraints on theories of cortical development.

Spatial attention alters visual cortical representation during target anticipation

Attention enables us to efficiently and flexibly interact with the environment by prioritizing some image features in preparation for responding to a stimulus. Using a concurrent psychophysics- fMRI experiment, we investigated how covert spatial attention affects responses in human visual cortex prior to target onset, and how it affects subsequent behavioral performance. Performance improved at cued locations and worsened at uncued locations, relative to distributed attention, demonstrating a selective tradeoff in processing. Pre-target BOLD responses in cortical visual field maps changed in two ways: First, there was a stimulus-independent baseline shift, positive in map locations near the cued location and negative elsewhere, paralleling the behavioral results. Second, population receptive field centers shifted toward the attended location. Both effects increased in higher visual areas. Together, the results show that spatial attention has large effects on visual cortex prior to target appearance, altering neural response properties throughout and across multiple visual field maps.

Can the neural representation of physical pain predict empathy for pain in others?

The question of whether physical pain and vicarious pain have some shared neural substrates is unresolved. Recent research has argued that physical and vicarious pain are represented by dissociable multivariate brain patterns by creating biomarkers for physical pain (Neurologic Pain Signature, NPS) and vicarious pain (Vicarious Pain Signature, VPS), respectively. In the current research, the NPS and two versions of the VPS were applied to three fMRI datasets (one new, two published) relating to vicarious pain which focused on between-subject differences in vicarious pain (Datasets 1 and 3) and within-subject manipulations of perspective taking (Dataset 2). Results show that (i) NPS can distinguish brain responses to images of pain vs no-pain and to a greater extent in vicarious pain responders who report experiencing pain when observing pain and (ii) neither version of the VPS mapped on to individual differences in vicarious pain and the two versions differed in their success in predicting vicarious pain overall. This study suggests that the NPS (created to detect physical pain) is, under some circumstances, sensitive to vicarious pain and there is significant variability in VPS measures (created to detect vicarious pain) to act as generalizable biomarkers of vicarious pain.

Linguistic modulation of the neural encoding of phonemes

Speech comprehension entails the neural mapping of the acoustic speech signal onto learned linguistic units. This acousto-linguistic transformation is bi-directional, whereby higher-level linguistic processes (e.g. semantics) modulate the acoustic analysis of individual linguistic units. Here, we investigated the cortical topography and linguistic modulation of the most fundamental linguistic unit, the phoneme. We presented natural speech and "phoneme quilts" (pseudo-randomly shuffled phonemes) in either a familiar (English) or unfamiliar (Korean) language to native English speakers while recording functional magnetic resonance imaging. This allowed us to dissociate the contribution of acoustic vs. linguistic processes toward phoneme analysis. We show that (i) the acoustic analysis of phonemes is modulated by linguistic analysis and (ii) that for this modulation, both of acoustic and phonetic information need to be incorporated. These results suggest that the linguistic modulation of cortical sensitivity to phoneme classes minimizes prediction error during natural speech perception, thereby aiding speech comprehension in challenging listening situations.

Evaluating the effect of denoising submillimeter auditory fMRI data with NORDIC

Functional magnetic resonance imaging (fMRI) has emerged as an essential tool for exploring human brain function. Submillimeter fMRI, in particular, has emerged as a tool to study mesoscopic computations. The inherently low signal-to-noise ratio (SNR) at submillimeter resolutions warrants the use of denoising approaches tailored at reducing thermal noise - the dominant contributing noise component in high resolution fMRI. NORDIC PCA is one of such approaches, and has been benchmarked against other approaches in several applications. Here, we investigate the effects that two versions of NORDIC denoising have on auditory submillimeter data. As investigating auditory functional responses poses unique challenges, we anticipated that the benefit of this technique would be especially pronounced. Our results show that NORDIC denoising improves the detection sensitivity and the reliability of estimates in submillimeter auditory fMRI data. These effects can be explained by the reduction of the noise-induced signal variability. However, we also observed a reduction in the average response amplitude (percent signal), which may suggest that a small amount of signal was also removed. We conclude that, while evaluating the effects of the signal reduction induced by NORDIC may be necessary for each application, using NORDIC in high resolution auditory fMRI studies may be advantageous because of the large reduction in variability of the estimated responses.

Denoising task-correlated head motion from motor-task fMRI data with multi-echo ICA

Motor-task functional magnetic resonance imaging (fMRI) is crucial in the study of several clinical conditions, including stroke and Parkinson's disease. However, motor-task fMRI is complicated by task-correlated head motion, which can be magnified in clinical populations and confounds motor activation results. One method that may mitigate this issue is multi-echo independent component analysis (ME-ICA), which has been shown to separate the effects of head motion from the desired blood oxygenation level dependent (BOLD) signal but has not been tested in motor-task datasets with high amounts of motion. In this study, we collected an fMRI dataset from a healthy population who performed a hand grasp task with and without task-correlated amplified head motion to simulate a motor-impaired population. We analyzed these data using three models: single-echo (SE), multi-echo optimally combined (ME-OC), and ME-ICA. We compared the models' performance in mitigating the effects of head motion on the subject level and group level. On the subject level, ME-ICA better dissociated the effects of head motion from the BOLD signal and reduced noise. Both ME models led to increased t-statistics in brain motor regions. In scans with high levels of motion, ME-ICA additionally mitigated artifacts and increased stability of beta coefficient estimates, compared to SE. On the group level, all three models produced activation clusters in expected motor areas in scans with both low and high motion, indicating that group-level averaging may also sufficiently resolve motion artifacts that vary by subject. These findings demonstrate that ME-ICA is a useful tool for subject-level analysis of motor-task data with high levels of task-correlated head motion. The improvements afforded by ME-ICA are critical to improve reliability of subject-level activation maps for clinical populations in which group-level analysis may not be feasible or appropriate, for example, in a chronic stroke cohort with varying stroke location and degree of tissue damage.

A hybrid spatiotemporal deep belief network and sparse representation-based framework reveals multilevel core functional components in decoding multitask fMRI signals

Decoding human brain activity on various task-based functional brain imaging data is of great significance for uncovering the functioning mechanism of the human mind. Currently, most feature extraction model-based methods for brain state decoding are shallow machine learning models, which may struggle to capture complex and precise spatiotemporal patterns of brain activity from the highly noisy fMRI raw data. Moreover, although decoding models based on deep learning methods benefit from their multilayer structure that could extract spatiotemporal features at multiscale, the relatively large populations of fMRI datasets are indispensable, and the explainability of their results is elusive. To address the above problems, we proposed a computational framework based on hybrid spatiotemporal deep belief network and sparse representations to differentiate multitask fMRI (tfMRI) signals. Using a relatively small cohort of tfMRI data as a test bed, our framework can achieve an average classification accuracy of 97.86% and define the multilevel temporal and spatial patterns of multiple cognitive tasks. Intriguingly, our model can characterize the key components for differentiating the multitask fMRI signals. Overall, the proposed framework can identify the interpretable and discriminative fMRI composition patterns at multiple scales, offering an effective methodology for basic neuroscience and clinical research with relatively small cohorts.

Hierarchical organization of social action features along the lateral visual pathway

Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content-consistent with hierarchical organization-along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.

Denoising task-correlated head motion from motor-task fMRI data with multi-echo ICA

Motor-task functional magnetic resonance imaging (fMRI) is crucial in the study of several clinical conditions, including stroke and Parkinson's disease. However, motor-task fMRI is complicated by task-correlated head motion, which can be magnified in clinical populations and confounds motor activation results. One method that may mitigate this issue is multi-echo independent component analysis (ME-ICA), which has been shown to separate the effects of head motion from the desired BOLD signal but has not been tested in motor-task datasets with high amounts of motion. In this study, we collected an fMRI dataset from a healthy population who performed a hand grasp task with and without task-correlated amplified head motion to simulate a motor-impaired population. We analyzed these data using three models: single-echo (SE), multi-echo optimally combined (ME-OC), and ME-ICA. We compared the models' performance in mitigating the effects of head motion on the subject level and group level. On the subject level, ME-ICA better dissociated the effects of head motion from the BOLD signal and reduced noise. Both ME models led to increased t-statistics in brain motor regions. In scans with high levels of motion, ME-ICA additionally mitigated artifacts and increased stability of beta coefficient estimates, compared to SE. On the group level, all three models produced activation clusters in expected motor areas in scans with both low and high motion, indicating that group-level averaging may also sufficiently resolve motion artifacts that vary by subject. These findings demonstrate that ME-ICA is a useful tool for subject-level analysis of motor-task data with high levels of task-correlated head motion. The improvements afforded by ME-ICA are critical to improve reliability of subject-level activation maps for clinical populations in which group-level analysis may not be feasible or appropriate, for example in a chronic stroke cohort with varying stroke location and degree of tissue damage.

Driving and suppressing the human language network using large language models

Transformer models such as GPT generate human-like language and are highly predictive of human brain responses to language. Here, using fMRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of brain response associated with each sentence. Then, we use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also noninvasively control neural activity in higher-level cortical areas, like the language network.

Human brain responses are modulated when exposed to optimized natural images or synthetically generated images

Understanding how human brains interpret and process information is important. Here, we investigated the selectivity and inter-individual differences in human brain responses to images via functional MRI. In our first experiment, we found that images predicted to achieve maximal activations using a group level encoding model evoke higher responses than images predicted to achieve average activations, and the activation gain is positively associated with the encoding model accuracy. Furthermore, anterior temporal lobe face area (aTLfaces) and fusiform body area 1 had higher activation in response to maximal synthetic images compared to maximal natural images. In our second experiment, we found that synthetic images derived using a personalized encoding model elicited higher responses compared to synthetic images from group-level or other subjects' encoding models. The finding of aTLfaces favoring synthetic images than natural images was also replicated. Our results indicate the possibility of using data-driven and generative approaches to modulate macro-scale brain region responses and probe inter-individual differences in and functional specialization of the human visual system.

Ultra-High Field Imaging of Human Visual Cognition

Functional magnetic resonance imaging (fMRI), the key methodology for mapping the functions of the human brain in a noninvasive manner, is limited by low temporal and spatial resolution. Recent advances in ultra-high field (UHF) fMRI provide a mesoscopic (i.e., submillimeter resolution) tool that allows us to probe laminar and columnar circuits, distinguish bottom-up versus top-down pathways, and map small subcortical areas. We review recent work demonstrating that UHF fMRI provides a robust methodology for imaging the brain across cortical depths and columns that provides insights into the brain's organization and functions at unprecedented spatial resolution, advancing our understanding of the fine-scale computations and interareal communication that support visual cognition.

A large-scale fMRI dataset for the visual processing of naturalistic scenes

One ultimate goal of visual neuroscience is to understand how the brain processes visual stimuli encountered in the natural environment. Achieving this goal requires records of brain responses under massive amounts of naturalistic stimuli. Although the scientific community has put a lot of effort into collecting large-scale functional magnetic resonance imaging (fMRI) data under naturalistic stimuli, more naturalistic fMRI datasets are still urgently needed. We present here the Natural Object Dataset (NOD), a large-scale fMRI dataset containing responses to 57,120 naturalistic images from 30 participants. NOD strives for a balance between sampling variation between individuals and sampling variation between stimuli. This enables NOD to be utilized not only for determining whether an observation is generalizable across many individuals, but also for testing whether a response pattern is generalized to a variety of naturalistic stimuli. We anticipate that the NOD together with existing naturalistic neuroimaging datasets will serve as a new impetus for our understanding of the visual processing of naturalistic stimuli.

Representational Geometries Reveal Differential Effects of Response Correlations on Population Codes in Neurophysiology and Functional Magnetic Resonance Imaging

Two sensory neurons usually display trial-by-trial spike-count correlations given the repeated representations of a stimulus. The effects of such response correlations on population-level sensory coding have been the focal contention in computational neuroscience over the past few years. In the meantime, multivariate pattern analysis (MVPA) has become the leading analysis approach in functional magnetic resonance imaging (fMRI), but the effects of response correlations among voxel populations remain underexplored. Here, instead of conventional MVPA analysis, we calculate linear Fisher information of population responses in human visual cortex (five males, one female) and hypothetically remove response correlations between voxels. We found that voxelwise response correlations generally enhance stimulus information, a result standing in stark contrast to the detrimental effects of response correlations reported in empirical neurophysiological studies. By voxel-encoding modeling, we further show that these two seemingly opposite effects actually can coexist within the primate visual system. Furthermore, we use principal component analysis to decompose stimulus information in population responses onto different principal dimensions in a high-dimensional representational space. Interestingly, response correlations simultaneously reduce and enhance information on higher- and lower-variance principal dimensions, respectively. The relative strength of the two antagonistic effects within the same computational framework produces the apparent discrepancy in the effects of response correlations in neuronal and voxel populations. Our results suggest that multivariate fMRI data contain rich statistical structures that are directly related to sensory information representation, and the general computational framework to analyze neuronal and voxel population responses can be applied in many types of neural measurements.SIGNIFICANCE STATEMENT Despite the vast research interest in the effect of spike-count noise correlations on population codes in neurophysiology, it remains unclear how the response correlations between voxels influence MVPA in human imaging. We used an information-theoretic approach and showed that unlike the detrimental effects of response correlations reported in neurophysiology, voxelwise response correlations generally improve sensory coding. We conducted a series of in-depth analyses and demonstrated that neuronal and voxel response correlations can coexist within the visual system and share some common computational mechanisms. These results shed new light on how the population codes of sensory information can be evaluated via different neural measurements.

Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations

Deep neural networks (DNNs) optimized for visual tasks learn representations that align layer depth with the hierarchy of visual areas in the primate brain. One interpretation of this finding is that hierarchical representations are necessary to accurately predict brain activity in the primate visual system. To test this interpretation, we optimized DNNs to directly predict brain activity measured with fMRI in human visual areas V1-V4. We trained a single-branch DNN to predict activity in all four visual areas jointly, and a multi-branch DNN to predict each visual area independently. Although it was possible for the multi-branch DNN to learn hierarchical representations, only the single-branch DNN did so. This result shows that hierarchical representations are not necessary to accurately predict human brain activity in V1-V4, and that DNNs that encode brain-like visual representations may differ widely in their architecture, ranging from strict serial hierarchies to multiple independent branches.

Evaluating increases in sensitivity from NORDIC for diverse fMRI acquisition strategies

As the neuroimaging field moves towards detecting smaller effects at higher spatial resolutions, and faster sampling rates, there is increased attention given to the deleterious contribution of unstructured, thermal noise. Here, we critically evaluate the performance of a recently developed reconstruction method, termed NORDIC, for suppressing thermal noise using datasets acquired with various field strengths, voxel sizes, sampling rates, and task designs. Following minimal preprocessing, statistical activation (t-values) of NORDIC processed data was compared to the results obtained with alternative denoising methods. Additionally, we examined the consistency of the estimates of task responses at the single-voxel, single run level, using a finite impulse response (FIR) model. To examine the potential impact on effective image resolution, the overall smoothness of the data processed with different methods was estimated. Finally, to determine if NORDIC alters or removes temporal information important for modeling responses, we employed an exhaustive leave-p-out cross validation approach, using FIR task responses to predict held out timeseries, quantified using R². After NORDIC, the t-values are increased, an improvement comparable to what could be achieved by 1.5 voxels smoothing, and task events are clearly visible and have less cross-run error. These advantages are achieved with smoothness estimates increasing by less than 4%, while 1.5 voxel smoothing is associated with increases of over 140%. Cross-validated R²s based on the FIR models show that NORDIC is not measurably distorting the temporal structure of the data under this approach and is the best predictor of non-denoised time courses. The results demonstrate that analyzing 1 run of data after NORDIC produces results equivalent to using 2 to 3 original runs and that NORDIC performs equally well across a diverse array of functional imaging protocols. Significance Statement: For functional neuroimaging, the increasing availability of higher field strengths and ever higher spatiotemporal resolutions has led to concomitant increase in concerns about the deleterious effects of thermal noise. Historically this noise source was suppressed using methods that reduce spatial precision such as image blurring or averaging over a large number of trials or sessions, which necessitates large data collection efforts. Here, we critically evaluate the performance of a recently developed reconstruction method, termed NORDIC, which suppresses thermal noise. Across datasets varying in field strength, voxel sizes, sampling rates, and task designs, NORDIC produces substantial gains in data quality. Both conventional t-statistics derived from general linear models and coefficients of determination for predicting unseen data are improved. These gains match or even exceed those associated with 1 voxel Full Width Half Max image smoothing, however, even such small amounts of smoothing are associated with a 52% reduction in estimates of spatial precision, whereas the measurable difference in spatial precision is less than 4% following NORDIC.

MP-PCA denoising of fMRI time-series data can lead to artificial activation "spreading"

MP-PCA denoising has become the method of choice for denoising MRI data since it provides an objective threshold to separate the signal components from unwanted thermal noise components. In rodents, thermal noise in the coils is an important source of noise that can reduce the accuracy of activation mapping in fMRI. Further confounding this problem, vendor data often contains zero-filling and other post-processing steps that may violate MP-PCA assumptions. Here, we develop an approach to denoise vendor data and assess activation "spreading" caused by MP-PCA denoising in rodent task-based fMRI data. Data was obtained from N = 3 mice using conventional multislice and ultrafast acquisitions (1 s and 50 ms temporal resolution, respectively), during visual stimulation. MP-PCA denoising produced SNR gains of 64% and 39% and Fourier Spectral Amplitude (FSA) increases in BOLD maps of 9% and 7% for multislice and ultrafast data, respectively, when using a small [2 2] denoising window. Larger windows provided higher SNR and FSA gains with increased spatial extent of activation that may or may not represent real activation. Simulations showed that MP-PCA denoising can incur activation "spreading" with increased false positive rate and smoother functional maps due to local "bleeding" of principal components, and that the optimal denoising window for improved specificity of functional mapping, based on Dice score calculations, depends on the data's tSNR and functional CNR. This "spreading" effect applies also to another recently proposed low-rank denoising method (NORDIC), although to a lesser degree. Our results bode well for enhancing spatial and/or temporal resolution in future fMRI work, while taking into account the sensitivity/specificity trade-offs of low-rank denoising methods.

Engaging in word recognition elicits highly specific modulations in visual cortex

A person's cognitive state determines how their brain responds to visual stimuli. The most common such effect is a response enhancement when stimuli are task relevant and attended rather than ignored. In this fMRI study, we report a surprising twist on such attention effects in the visual word form area (VWFA), a region that plays a key role in reading. We presented participants with strings of letters and visually similar shapes, which were either relevant for a specific task (lexical decision or gap localization) or ignored (during a fixation dot color task). In the VWFA, the enhancement of responses to attended stimuli occurred only for letter strings, whereas non-letter shapes evoked smaller responses when attended than when ignored. The enhancement of VWFA activity was accompanied by strengthened functional connectivity with higher-level language regions. These task-dependent modulations of response magnitude and functional connectivity were specific to the VWFA and absent in the rest of visual cortex. We suggest that language regions send targeted excitatory feedback into the VWFA only when the observer is trying to read. This feedback enables the discrimination of familiar and nonsense words and is distinct from generic effects of visual attention.

Robust high spatio-temporal line-scanning fMRI in humans at 7T using multi-echo readouts, denoising and prospective motion correction

BACKGROUND

Functional magnetic resonance imaging (fMRI), typically using blood oxygenation level-dependent (BOLD) contrast weighted imaging, allows the study of brain function with millimeter spatial resolution and temporal resolution of one to a few seconds. At a mesoscopic scale, neurons in the human brain are spatially organized in structures with dimensions of hundreds of micrometers, while they communicate at the millisecond timescale. For this reason, it is important to develop an fMRI method with simultaneous high spatial and temporal resolution. Line-scanning promises to reach this goal at the cost of volume coverage.

NEW METHOD

Here, we release a comprehensive update to human line-scanning fMRI. First, we investigated multi-echo line-scanning with five different protocols varying the number of echoes and readout bandwidth while keeping the TR constant. In these, we compared different echo combination approaches in terms of BOLD activation (sensitivity) and temporal signal-to-noise ratio. Second, we implemented an adaptation of NOise reduction with DIstribution Corrected principal component analysis (NORDIC) thermal noise removal for line-scanning fMRI data. Finally, we tested three image-based navigators for motion correction and investigated different ways of performing fMRI analysis on the timecourses which were influenced by the insertion of the navigators themselves.

RESULTS

The presented improvements are relatively straightforward to implement; multi-echo readout and NORDIC denoising together, significantly improve data quality in terms of tSNR and t-statistical values, while motion correction makes line-scanning fMRI more robust.

COMPARISON WITH EXISTING METHODS

Multi-echo acquisitions and denoising have previously been applied in 3D magnetic resonance imaging. Their combination and application to 1D line-scanning is novel. The current proposed method greatly outperforms the previous line-scanning acquisitions with single-echo acquisition, in terms of tSNR (4.0 for single-echo line-scanning and 36.2 for NORDIC-denoised multi-echo) and t-statistical values (3.8 for single-echo line-scanning and 25.1 for NORDIC-denoised multi-echo line-scanning).

CONCLUSIONS

Line-scanning fMRI was advanced compared to its previous implementation in order to improve sensitivity and reliability. The improved line-scanning acquisition could be used, in the future, for neuroscientific and clinical applications.

Color-biased regions in the ventral visual pathway are food selective

Color-biased regions have been found between face- and place-selective areas in the ventral visual pathway. To investigate the function of the color-biased regions in a pathway responsible for object recognition, we analyzed the natural scenes dataset (NSD), a large 7T fMRI dataset from 8 participants who each viewed up to 30,000 trials of images of colored natural scenes over more than 30 scanning sessions. In a whole-brain analysis, we correlated the average color saturation of the images with voxel responses, revealing color-biased regions that diverge into two streams, beginning in V4 and extending medially and laterally relative to the fusiform face area in both hemispheres. We drew regions of interest (ROIs) for the two streams and found that the images for each ROI that evoked the largest responses had certain characteristics: they contained food, circular objects, warmer hues, and had higher color saturation. Further analyses showed that food images were the strongest predictor of activity in these regions, implying the existence of medial and lateral ventral food streams (VFSs). We found that color also contributed independently to voxel responses, suggesting that the medial and lateral VFSs use both color and form to represent food. Our findings illustrate how high-resolution datasets such as the NSD can be used to disentangle the multifaceted contributions of many visual features to the neural representations of natural scenes.

The gradient model of brain organization in decisions involving “empathy for pain”

Influential models of cortical organization propose a close relationship between heteromodal association areas and highly connected hubs in the default mode network. The “gradient model” of cortical organization proposes a close relationship between these areas and highly connected hubs in the default mode network, a set of cortical areas deactivated by demanding tasks. Here, we used a decision-making task and representational similarity analysis with classic “empathy for pain” stimuli to probe the relationship between high-level representations of imminent pain in others and these areas. High-level representations were colocalized with task deactivations or the transitions from activations to deactivations. These loci belonged to 2 groups: those that loaded on the high end of the principal cortical gradient and were associated by meta-analytic decoding with the default mode network, and those that appeared to accompany functional repurposing of somatosensory cortex in the presence of visual stimuli. These findings suggest that task deactivations may set out cortical areas that host high-level representations. We anticipate that an increased understanding of the cortical correlates of high-level representations may improve neurobiological models of social interactions and psychopathology.

Personalized visual encoding model construction with small data

Quantifying population heterogeneity in brain stimuli-response mapping may allow insight into variability in bottom-up neural systems that can in turn be related to individual's behavior or pathological state. Encoding models that predict brain responses to stimuli are one way to capture this relationship. However, they generally need a large amount of fMRI data to achieve optimal accuracy. Here, we propose an ensemble approach to create encoding models for novel individuals with relatively little data by modeling each subject's predicted response vector as a linear combination of the other subjects' predicted response vectors. We show that these ensemble encoding models trained with hundreds of image-response pairs, achieve accuracy not different from models trained on 20,000 image-response pairs. Importantly, the ensemble encoding models preserve patterns of inter-individual differences in the image-response relationship. We also show the proposed approach is robust against domain shift by validating on data with a different scanner and experimental setup. Additionally, we show that the ensemble encoding models are able to discover the inter-individual differences in various face areas' responses to images of animal vs human faces using a recently developed NeuroGen framework. Our approach shows the potential to use existing densely-sampled data, i.e. large amounts of data collected from a single individual, to efficiently create accurate, personalized encoding models and, subsequently, personalized optimal synthetic images for new individuals scanned under different experimental conditions.

Improving the accuracy of single-trial fMRI response estimates using GLMsingle

Advances in artificial intelligence have inspired a paradigm shift in human neuroscience, yielding large-scale functional magnetic resonance imaging (fMRI) datasets that provide high-resolution brain responses to thousands of naturalistic visual stimuli. Because such experiments necessarily involve brief stimulus durations and few repetitions of each stimulus, achieving sufficient signal-to-noise ratio can be a major challenge. We address this challenge by introducing GLMsingle, a scalable, user-friendly toolbox available in MATLAB and Python that enables accurate estimation of single-trial fMRI responses (glmsingle.org). Requiring only fMRI time-series data and a design matrix as inputs, GLMsingle integrates three techniques for improving the accuracy of trial-wise general linear model (GLM) beta estimates. First, for each voxel, a custom hemodynamic response function (HRF) is identified from a library of candidate functions. Second, cross-validation is used to derive a set of noise regressors from voxels unrelated to the experiment. Third, to improve the stability of beta estimates for closely spaced trials, betas are regularized on a voxel-wise basis using ridge regression. Applying GLMsingle to the Natural Scenes Dataset and BOLD5000, we find that GLMsingle substantially improves the reliability of beta estimates across visually-responsive cortex in all subjects. Comparable improvements in reliability are also observed in a smaller-scale auditory dataset from the StudyForrest experiment. These improvements translate into tangible benefits for higher-level analyses relevant to systems and cognitive neuroscience. We demonstrate that GLMsingle: (i) helps decorrelate response estimates between trials nearby in time; (ii) enhances representational similarity between subjects within and across datasets; and (iii) boosts one-versus-many decoding of visual stimuli. GLMsingle is a publicly available tool that can significantly improve the quality of past, present, and future neuroimaging datasets sampling brain activity across many experimental conditions.

Selective attention sharpens population receptive fields in human auditory cortex

Selective attention enables the preferential processing of relevant stimulus aspects. Invasive animal studies have shown that attending a sound feature rapidly modifies neuronal tuning throughout the auditory cortex. Human neuroimaging studies have reported enhanced auditory cortical responses with selective attention. To date, it remains unclear how the results obtained with functional magnetic resonance imaging (fMRI) in humans relate to the electrophysiological findings in animal models. Here we aim to narrow the gap between animal and human research by combining a selective attention task similar in design to those used in animal electrophysiology with high spatial resolution ultra-high field fMRI at 7 Tesla. Specifically, human participants perform a detection task, whereas the probability of target occurrence varies with sound frequency. Contrary to previous fMRI studies, we show that selective attention resulted in population receptive field sharpening, and consequently reduced responses, at the attended sound frequencies. The difference between our results to those of previous fMRI studies supports the notion that the influence of selective attention on auditory cortex is diverse and may depend on context, stimulus, and task.

Denoise Functional Magnetic Resonance Imaging With Random Matrix Theory Based Principal Component Analysis

High-resolution functional MRI (fMRI) is largely hindered by random thermal noise. Random matrix theory (RMT)-based principal component analysis (PCA) is promising to reduce such noise in fMRI data. However, there is no consensus about the optimal strategy and practice in implementation. In this work, we propose a comprehensive RMT-based denoising method that consists of 1) rank and noise estimation based on a set of newly derived multiple criteria, and 2) optimal singular value shrinkage, with each module explained and implemented based on the RMT. By incorporating the variance stabilizing approach, the denoising method can deal with low signal-to-noise ratio (SNR) (such as <5) magnitude fMRI data with favorable performance compared to other state-of-the-art methods. Results from both simulation and in-vivo high-resolution fMRI data show that the proposed denoising method dramatically improves image restoration quality, promoting functional sensitivity at the same level of functional mapping blurring compared to existing denoising methods. Moreover, the denoising method can serve as a drop-in step in data preprocessing pipelines along with other procedures aimed at removal of structured physiological noises. We expect that the proposed denoising method will play an important role in leveraging high-quality, high-resolution task fMRI, which is desirable in many neuroscience and clinical applications.

Perception and memory have distinct spatial tuning properties in human visual cortex

Reactivation of earlier perceptual activity is thought to underlie long-term memory recall. Despite evidence for this view, it is unclear whether mnemonic activity exhibits the same tuning properties as feedforward perceptual activity. Here, we leverage population receptive field models to parameterize fMRI activity in human visual cortex during spatial memory retrieval. Though retinotopic organization is present during both perception and memory, large systematic differences in tuning are also evident. Whereas there is a three-fold decline in spatial precision from early to late visual areas during perception, this pattern is not observed during memory retrieval. This difference cannot be explained by reduced signal-to-noise or poor performance on memory trials. Instead, by simulating top-down activity in a network model of cortex, we demonstrate that this property is well explained by the hierarchical structure of the visual system. Together, modeling and empirical results suggest that computational constraints imposed by visual system architecture limit the fidelity of memory reactivation in sensory cortex.

Decoding perceptual awareness across the brain with a no-report fMRI masking paradigm

Does perceptual awareness arise within the sensory regions of the brain or within higher-level regions (e.g., the frontal lobe)? To answer this question, researchers traditionally compare neural activity when observers report being aware versus being unaware of a stimulus. However, it is unclear whether the resulting activations are associated with the conscious perception of the stimulus or the post-perceptual processes associated with reporting that stimulus. To address this limitation, we used both report and no-report conditions in a visual masking paradigm while participants were scanned using functional MRI (fMRI). We found that the overall univariate response to visible stimuli in the frontal lobe was robust in the report condition but disappeared in the no-report condition. However, using multivariate patterns, we could still decode in both conditions whether a stimulus reached conscious awareness across the brain, including in the frontal lobe. These results help reconcile key discrepancies in the recent literature and provide a path forward for identifying the neural mechanisms associated with perceptual awareness.

Is there evidence for a noisy computation deficit in developmental dyslexia?

The noisy computation hypothesis of developmental dyslexia (DD) is particularly appealing because it can explain deficits across a variety of domains, such as temporal, auditory, phonological, visual and attentional processes. A key prediction is that noisy computations lead to more variable and less stable word representations. A way to test this hypothesis is through repetition of words, that is, when there is noise in the system, the neural signature of repeated stimuli should be more variable. The hypothesis was tested in an functional magnetic resonance imaging experiment with dyslexic and typical readers by repeating words twelve times. Variability measures were computed both at the behavioral and neural levels. At the behavioral level, we compared the standard deviation of reaction time distributions of repeated words. At the neural level, in addition to standard univariate analyses and measures of intra-item variability, we also used multivariate pattern analyses (representational similarity and classification) to find out whether there was evidence for noisier representations in dyslexic readers compared to typical readers. Results showed that there were no significant differences between the two groups in any of the analyses despite robust results within each group (i.e., high representational similarity between repeated words, good classification of words vs. non-words). In summary, there was no evidence in favor of the idea that dyslexic readers would have noisier neural representations than typical readers.

Non-Neural Factors Influencing BOLD Response Magnitudes within Individual Subjects

To what extent is the size of the BOLD response influenced by factors other than neural activity? In a reanalysis of three neuroimaging datasets (male and female human participants), we find large systematic inhomogeneities in the BOLD response magnitude in primary visual cortex (V1): stimulus-evoked BOLD responses, expressed in units of percent signal change, are up to 50% larger along the representation of the horizontal meridian than the vertical meridian. To assess whether this surprising effect can be interpreted as differences in local neural activity, we quantified several factors that potentially contribute to the size of the BOLD response. We find relationships between BOLD response magnitude and cortical thickness, curvature, depth, and macrovasculature. These relationships are consistently found across subjects and datasets and suggest that variation in BOLD response magnitudes across cortical locations reflects, in part, differences in anatomy and vascularization. To compensate for these factors, we implement a regression-based correction method and show that, after correction, BOLD responses become more homogeneous across V1. The correction reduces the horizontal/vertical difference by about half, indicating that some of the difference is likely not because of neural activity differences. We conclude that interpretation of variation in BOLD response magnitude across cortical locations should consider the influence of the potential confounding factors of thickness, curvature, depth, and vascularization.SIGNIFICANCE STATEMENT The magnitude of the BOLD signal is often used as a surrogate of neural activity, but the exact factors that contribute to its strength have not been studied on a voxel-wise level. Here, we examined several anatomical and measurement-related factors to assess their relationship with BOLD signal magnitude. We find that BOLD magnitude correlates with cortical anatomy, depth, and macrovasculature. To remove the contribution of these factors, we propose a simple, data-driven correction method that can be used in any fMRI experiment. After accounting for the confounding factors, BOLD magnitude becomes more spatially homogeneous. Our correction method improves the ability to make more accurate inferences about local neural activity from fMRI data.

On the encoding of natural music in computational models and human brains

This article discusses recent developments and advances in the neuroscience of music to understand the nature of musical emotion. In particular, it highlights how system identification techniques and computational models of music have advanced our understanding of how the human brain processes the textures and structures of music and how the processed information evokes emotions. Musical models relate physical properties of stimuli to internal representations called features, and predictive models relate features to neural or behavioral responses and test their predictions against independent unseen data. The new frameworks do not require orthogonalized stimuli in controlled experiments to establish reproducible knowledge, which has opened up a new wave of naturalistic neuroscience. The current review focuses on how this trend has transformed the domain of the neuroscience of music.

Functional MRI in major depressive disorder: A review of findings, limitations, and future prospects

Objective diagnosis and prognosis in major depressive disorder (MDD) remains a challenge due to the absence of biomarkers based on physiological parameters or medical tests. Numerous studies have been conducted to identify functional magnetic resonance imaging‐based biomarkers of depression that either objectively differentiate patients with depression from healthy subjects, predict personalized treatment outcome, or characterize biological subtypes of depression. While there are some findings of consistent functional biomarkers, there is still lack of robust data acquisition and analysis methodology. According to current findings, primarily, the anterior cingulate cortex, prefrontal cortex, and default mode network play a crucial role in MDD. Yet, there are also less consistent results and the involvement of other regions or networks remains ambiguous. We further discuss image acquisition, processing, and analysis limitations that might underlie these inconsistencies. Finally, the current review aims to address and discuss possible remedies and future opportunities that could improve the search for consistent functional imaging biomarkers of depression. Novel acquisition techniques, such as multiband and multiecho imaging, and neural network‐based cleaning approaches can enhance the signal quality in limbic and frontal regions. More comprehensive analyses, such as directed or dynamic functional features or the identification of biological depression subtypes, can improve objective diagnosis or treatment outcome prediction and mitigate the heterogeneity of MDD. Overall, these improvements in functional MRI imaging techniques, processing, and analysis could advance the search for biomarkers and ultimately aid patients with MDD and their treatment course.

Texture-like representation of objects in human visual cortex

Humans are exquisitely sensitive to the spatial arrangement of visual features in objects and scenes, but not in visual textures. Category-selective regions in the visual cortex are widely believed to underlie object perception, suggesting such regions should distinguish natural images of objects from synthesized images containing similar visual features in scrambled arrangements. Contrarily, we demonstrate that representations in category-selective cortex do not discriminate natural images from feature-matched scrambles but can discriminate images of different categories, suggesting a texture-like encoding. We find similar insensitivity to feature arrangement in Imagenet-trained deep convolutional neural networks. This suggests the need to reconceptualize the role of category-selective cortex as representing a basis set of complex texture-like features, useful for a myriad of behaviors.

The human visual ability to recognize objects and scenes is widely thought to rely on representations in category-selective regions of the visual cortex. These representations could support object vision by specifically representing objects, or, more simply, by representing complex visual features regardless of the particular spatial arrangement needed to constitute real-world objects, that is, by representing visual textures. To discriminate between these hypotheses, we leveraged an image synthesis approach that, unlike previous methods, provides independent control over the complexity and spatial arrangement of visual features. We found that human observers could easily detect a natural object among synthetic images with similar complex features that were spatially scrambled. However, observer models built from BOLD responses from category-selective regions, as well as a model of macaque inferotemporal cortex and Imagenet-trained deep convolutional neural networks, were all unable to identify the real object. This inability was not due to a lack of signal to noise, as all observer models could predict human performance in image categorization tasks. How then might these texture-like representations in category-selective regions support object perception? An image-specific readout from category-selective cortex yielded a representation that was more selective for natural feature arrangement, showing that the information necessary for natural object discrimination is available. Thus, our results suggest that the role of the human category-selective visual cortex is not to explicitly encode objects but rather to provide a basis set of texture-like features that can be infinitely reconfigured to flexibly learn and identify new object categories.

Mapping spatial frequency preferences across human primary visual cortex

Neurons in primate visual cortex (area V1) are tuned for spatial frequency, in a manner that depends on their position in the visual field. Several studies have examined this dependency using functional magnetic resonance imaging (fMRI), reporting preferred spatial frequencies (tuning curve peaks) of V1 voxels as a function of eccentricity, but their results differ by as much as two octaves, presumably owing to differences in stimuli, measurements, and analysis methodology. Here, we characterize spatial frequency tuning at a millimeter resolution within the human primary visual cortex, across stimulus orientation and visual field locations. We measured fMRI responses to a novel set of stimuli, constructed as sinusoidal gratings in log-polar coordinates, which include circular, radial, and spiral geometries. For each individual stimulus, the local spatial frequency varies inversely with eccentricity, and for any given location in the visual field, the full set of stimuli span a broad range of spatial frequencies and orientations. Over the measured range of eccentricities, the preferred spatial frequency is well-fit by a function that varies as the inverse of the eccentricity plus a small constant. We also find small but systematic effects of local stimulus orientation, defined in both absolute coordinates and relative to visual field location. Specifically, peak spatial frequency is higher for pinwheel than annular stimuli and for horizontal than vertical stimuli.

NeuroGen: Activation optimized image synthesis for discovery neuroscience

Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network's ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.

Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants

Three of the most robust functional landmarks in the human brain are the selective responses to faces in the fusiform face area (FFA), scenes in the parahippocampal place area (PPA), and bodies in the extrastriate body area (EBA). Are the selective responses of these regions present early in development or do they require many years to develop? Prior evidence leaves this question unresolved. We designed a new 32-channel infant magnetic resonance imaging (MRI) coil and collected high-quality functional MRI (fMRI) data from infants (2-9 months of age) while they viewed stimuli from four conditions-faces, bodies, objects, and scenes. We find that infants have face-, scene-, and body-selective responses in the location of the adult FFA, PPA, and EBA, respectively, powerfully constraining accounts of cortical development.

Normal Retinotopy in Primary Visual Cortex in a Congenital Complete Unilateral Lesion of Lateral Geniculate Nucleus in Human: A Case Study

Impairment of the geniculostriate pathway results in scotomas in the corresponding part of the visual field. Here, we present a case of patient IB with left eye microphthalmia and with lesions in most of the left geniculostriate pathway, including the Lateral Geniculate Nucleus (LGN). Despite the severe lesions, the patient has a very narrow scotoma in the peripheral part of the lower-right-hemifield only (beyond 15° of eccentricity) and complete visual field representation in the primary visual cortex. Population receptive field mapping (pRF) of the patient’s visual field reveals orderly eccentricity maps together with contralateral activation in both hemispheres. With diffusion tractography, we revealed connections between superior colliculus (SC) and cortical structures in the hemisphere affected by the lesions, which could mediate the retinotopic reorganization at the cortical level. Our results indicate an astonishing case for the flexibility of the developing retinotopic maps where the contralateral thalamus receives fibers from both the nasal and temporal retinae.

A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence

Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.