Featural and temporal attention selectively enhance task-appropriate representations in human primary visual cortex

Goal-Dependent Use of Temporal Regularities to Orient Attention under Spatial and Action Uncertainty

The temporal regularities in our environments support the proactive dynamic anticipation of relevant events. In visual attention, one important outstanding question is whether temporal predictions must be linked to predictions about spatial locations or motor plans to facilitate behaviour. To test this, we developed a task for manipulating temporal expectations and task relevance of visual stimuli appearing within rapidly presented streams, while stimulus location and responding hand remained uncertain. Differently coloured stimuli appeared in one of two concurrent (left and right) streams with distinct temporal probability structures. Targets were defined by colour on a trial-by-trial basis and appeared equiprobably in either stream, requiring a localisation response. Across two experiments, participants were faster and more accurate at detecting temporally predictable targets compared to temporally unpredictable targets. We conclude that temporal expectations learned incidentally from temporal regularities can be called upon flexibly in a goal-driven manner to guide behaviour. Moreover, we show that visual temporal attention can facilitate performance in the absence of concomitant spatial or motor expectations in dynamically unfolding contexts.

Evaluating increases in sensitivity from NORDIC for diverse fMRI acquisition strategies

As the neuroimaging field moves towards detecting smaller effects at higher spatial resolutions, and faster sampling rates, there is increased attention given to the deleterious contribution of unstructured, thermal noise. Here, we critically evaluate the performance of a recently developed reconstruction method, termed NORDIC, for suppressing thermal noise using datasets acquired with various field strengths, voxel sizes, sampling rates, and task designs. Following minimal preprocessing, statistical activation (t-values) of NORDIC processed data was compared to the results obtained with alternative denoising methods. Additionally, we examined the consistency of the estimates of task responses at the single-voxel, single run level, using a finite impulse response (FIR) model. To examine the potential impact on effective image resolution, the overall smoothness of the data processed with different methods was estimated. Finally, to determine if NORDIC alters or removes temporal information important for modeling responses, we employed an exhaustive leave-p-out cross validation approach, using FIR task responses to predict held out timeseries, quantified using R². After NORDIC, the t-values are increased, an improvement comparable to what could be achieved by 1.5 voxels smoothing, and task events are clearly visible and have less cross-run error. These advantages are achieved with smoothness estimates increasing by less than 4%, while 1.5 voxel smoothing is associated with increases of over 140%. Cross-validated R²s based on the FIR models show that NORDIC is not measurably distorting the temporal structure of the data under this approach and is the best predictor of non-denoised time courses. The results demonstrate that analyzing 1 run of data after NORDIC produces results equivalent to using 2 to 3 original runs and that NORDIC performs equally well across a diverse array of functional imaging protocols. Significance Statement: For functional neuroimaging, the increasing availability of higher field strengths and ever higher spatiotemporal resolutions has led to concomitant increase in concerns about the deleterious effects of thermal noise. Historically this noise source was suppressed using methods that reduce spatial precision such as image blurring or averaging over a large number of trials or sessions, which necessitates large data collection efforts. Here, we critically evaluate the performance of a recently developed reconstruction method, termed NORDIC, which suppresses thermal noise. Across datasets varying in field strength, voxel sizes, sampling rates, and task designs, NORDIC produces substantial gains in data quality. Both conventional t-statistics derived from general linear models and coefficients of determination for predicting unseen data are improved. These gains match or even exceed those associated with 1 voxel Full Width Half Max image smoothing, however, even such small amounts of smoothing are associated with a 52% reduction in estimates of spatial precision, whereas the measurable difference in spatial precision is less than 4% following NORDIC.

Selective attention sharpens population receptive fields in human auditory cortex

Selective attention enables the preferential processing of relevant stimulus aspects. Invasive animal studies have shown that attending a sound feature rapidly modifies neuronal tuning throughout the auditory cortex. Human neuroimaging studies have reported enhanced auditory cortical responses with selective attention. To date, it remains unclear how the results obtained with functional magnetic resonance imaging (fMRI) in humans relate to the electrophysiological findings in animal models. Here we aim to narrow the gap between animal and human research by combining a selective attention task similar in design to those used in animal electrophysiology with high spatial resolution ultra-high field fMRI at 7 Tesla. Specifically, human participants perform a detection task, whereas the probability of target occurrence varies with sound frequency. Contrary to previous fMRI studies, we show that selective attention resulted in population receptive field sharpening, and consequently reduced responses, at the attended sound frequencies. The difference between our results to those of previous fMRI studies supports the notion that the influence of selective attention on auditory cortex is diverse and may depend on context, stimulus, and task.

Attention differentially modulates multiunit activity in the lateral geniculate nucleus and V1 of macaque monkeys

Attention promotes the selection of behaviorally relevant sensory signals from the barrage of sensory information available. Visual attention modulates the gain of neuronal activity in all visual brain areas examined, although magnitudes of gain modulations vary across areas. For example, attention gain magnitudes in the dorsal lateral geniculate nucleus (LGN) and primary visual cortex (V1) vary tremendously across fMRI measurements in humans and electrophysiological recordings in behaving monkeys. We sought to determine whether these discrepancies are due simply to differences in species or measurement, or more nuanced properties unique to each visual brain area. We also explored whether robust and consistent attention effects, comparable to those measured in humans with fMRI, are observable in the LGN or V1 of monkeys. We measured attentional modulation of multiunit activity in the LGN and V1 of macaque monkeys engaged in a contrast change detection task requiring shifts in covert visual spatial attention. Rigorous analyses of LGN and V1 multiunit activity revealed robust and consistent attentional facilitation throughout V1, with magnitudes comparable to those observed with fMRI. Interestingly, attentional modulation in the LGN was consistently negligible. These findings demonstrate that discrepancies in attention effects are not simply due to species or measurement differences. We also examined whether attention effects correlated with the feature selectivity of recorded multiunits. Distinct relationships suggest that attentional modulation of multiunit activity depends upon the unique structure and function of visual brain areas.

The time-course of feature-based attention effects dissociated from temporal expectation and target-related processes

Selective attention prioritises relevant information amongst competing sensory input. Time-resolved electrophysiological studies have shown stronger representation of attended compared to unattended stimuli, which has been interpreted as an effect of attention on information coding. However, because attention is often manipulated by making only the attended stimulus a target to be remembered and/or responded to, many reported attention effects have been confounded with target-related processes such as visual short-term memory or decision-making. In addition, attention effects could be influenced by temporal expectation about when something is likely to happen. The aim of this study was to investigate the dynamic effect of attention on visual processing using multivariate pattern analysis of electroencephalography (EEG) data, while (1) controlling for target-related confounds, and (2) directly investigating the influence of temporal expectation. Participants viewed rapid sequences of overlaid oriented grating pairs while detecting a "target" grating of a particular orientation. We manipulated attention, one grating was attended and the other ignored (cued by colour), and temporal expectation, with stimulus onset timing either predictable or not. We controlled for target-related processing confounds by only analysing non-target trials. Both attended and ignored gratings were initially coded equally in the pattern of responses across EEG sensors. An effect of attention, with preferential coding of the attended stimulus, emerged approximately 230 ms after stimulus onset. This attention effect occurred even when controlling for target-related processing confounds, and regardless of stimulus onset expectation. These results provide insight into the effect of feature-based attention on the dynamic processing of competing visual information.

The diachronic account of attentional selectivity

Many models of attention assume that attentional selection takes place at a specific moment in time that demarcates the critical transition from pre-attentive to attentive processing of sensory input. We argue that this intuitively appealing standard account of attentional selectivity is not only inaccurate, but has led to substantial conceptual confusion. As an alternative, we offer a 'diachronic' framework that describes attentional selectivity as a process that unfolds over time. Key to this view is the concept of attentional episodes, brief periods of intense attentional amplification of sensory representations that regulate access to working memory and response-related processes. We describe how attentional episodes are linked to earlier attentional mechanisms and to recurrent processing at the neural level. We review studies that establish the existence of attentional episodes, delineate the factors that determine if and when they are triggered, and discuss the costs associated with processing multiple events within a single episode. Finally, we argue that this framework offers new solutions to old problems in attention research that have never been resolved. It can provide a unified and conceptually coherent account of the network of cognitive and neural processes that produce the goal-directed selectivity in perceptual processing that is commonly referred to as 'attention'.

Population Models, Not Analyses, of Human Neuroscience Measurements

Selectivity for many basic properties of visual stimuli, such as orientation, is thought to be organized at the scale of cortical columns, making it difficult or impossible to measure directly with noninvasive human neuroscience measurement. However, computational analyses of neuroimaging data have shown that selectivity for orientation can be recovered by considering the pattern of response across a region of cortex. This suggests that computational analyses can reveal representation encoded at a finer spatial scale than is implied by the spatial resolution limits of measurement techniques. This potentially opens up the possibility to study a much wider range of neural phenomena that are otherwise inaccessible through noninvasive measurement. However, as we review in this article, a large body of evidence suggests an alternative hypothesis to this superresolution account: that orientation information is available at the spatial scale of cortical maps and thus easily measurable at the spatial resolution of standard techniques. In fact, a population model shows that this orientation information need not even come from single-unit selectivity for orientation tuning, but instead can result from population selectivity for spatial frequency. Thus, a categorical error of interpretation can result whereby orientation selectivity can be confused with spatial frequency selectivity. This is similarly problematic for the interpretation of results from numerous studies of more complex representations and cognitive functions that have built upon the computational techniques used to reveal stimulus orientation. We suggest in this review that these interpretational ambiguities can be avoided by treating computational analyses as models of the neural processes that give rise to measurement. Building upon the modeling tradition in vision science using considerations of whether population models meet a set of core criteria is important for creating the foundation for a cumulative and replicable approach to making valid inferences from human neuroscience measurements. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Temporal attention boosts perceptual effects of spatial attention and feature-based attention

Temporal attention, that is, the process of anticipating the occurrence of a stimulus at a given time point, has been shown to improve perceptual processing of visual stimuli. In the present study, we investigated whether and how temporal attention interacts with spatial attention and feature-based attention in visual selection. To monitor the influence of the three different attention dimensions on perceptual processing, we measured event-related potentials (ERPs). Our participants performed a visual search task, in which a colored singleton was presented amongst homogenous distractors. We manipulated spatial and feature-based attention by requiring participants to respond only to target singletons in a particular color and at a to-be-attended spatial location. We manipulated temporal attention by means of an explicit temporal cue that announced either validly or invalidly the occurrence of the search display. We obtained early ERP effects of spatial attention and feature-based attention at the validly cued but not at the invalidly cued time point. Taken together, our results suggest that temporal attention boosts early effects of spatial and feature-based attention.

The visual speech head start improves perception and reduces superior temporal cortex responses to auditory speech

Visual information about speech content from the talker's mouth is often available before auditory information from the talker's voice. Here we examined perceptual and neural responses to words with and without this visual head start. For both types of words, perception was enhanced by viewing the talker's face, but the enhancement was significantly greater for words with a head start. Neural responses were measured from electrodes implanted over auditory association cortex in the posterior superior temporal gyrus (pSTG) of epileptic patients. The presence of visual speech suppressed responses to auditory speech, more so for words with a visual head start. We suggest that the head start inhibits representations of incompatible auditory phonemes, increasing perceptual accuracy and decreasing total neural responses. Together with previous work showing visual cortex modulation (Ozker et al., 2018b) these results from pSTG demonstrate that multisensory interactions are a powerful modulator of activity throughout the speech perception network.

Stimulus vignetting and orientation selectivity in human visual cortex

Neural selectivity to orientation is one of the simplest and most thoroughly-studied cortical sensory features. Here, we show that a large body of research that purported to measure orientation tuning may have in fact been inadvertently measuring sensitivity to second-order changes in luminance, a phenomenon we term 'vignetting'. Using a computational model of neural responses in primary visual cortex (V1), we demonstrate the impact of vignetting on simulated V1 responses. We then used the model to generate a set of predictions, which we confirmed with functional MRI experiments in human observers. Our results demonstrate that stimulus vignetting can wholly determine the orientation selectivity of responses in visual cortex measured at a macroscopic scale, and suggest a reinterpretation of a well-established literature on orientation processing in visual cortex.

Mice use robust and common strategies to discriminate natural scenes

Mice use vision to navigate and avoid predators in natural environments. However, their visual systems are compact compared to other mammals, and it is unclear how well mice can discriminate ethologically relevant scenes. Here, we examined natural scene discrimination in mice using an automated touch-screen system. We estimated the discrimination difficulty using the computational metric structural similarity (SSIM), and constructed psychometric curves. However, the performance of each mouse was better predicted by the mean performance of other mice than SSIM. This high inter-mouse agreement indicates that mice use common and robust strategies to discriminate natural scenes. We tested several other image metrics to find an alternative to SSIM for predicting discrimination performance. We found that a simple, primary visual cortex (V1)-inspired model predicted mouse performance with fidelity approaching the inter-mouse agreement. The model involved convolving the images with Gabor filters, and its performance varied with the orientation of the Gabor filter. This orientation dependence was driven by the stimuli, rather than an innate biological feature. Together, these results indicate that mice are adept at discriminating natural scenes, and their performance is well predicted by simple models of V1 processing.

Local opposite orientation preferences in V1: fMRI sensitivity to fine-grained pattern information

The orientation of a visual grating can be decoded from human primary visual cortex (V1) using functional magnetic resonance imaging (fMRI) at conventional resolutions (2-3 mm voxel width, 3T scanner). It is unclear to what extent this information originates from different spatial scales of neuronal selectivity, ranging from orientation columns to global areal maps. According to the global-areal-map account, fMRI orientation decoding relies exclusively on fMRI voxels in V1 exhibiting a radial or vertical preference. Here we show, by contrast, that 2-mm isotropic voxels in a small patch of V1 within a quarterfield representation exhibit reliable opposite selectivities. Sets of voxels with opposite selectivities are locally intermingled and each set can support orientation decoding. This indicates that global areal maps cannot fully account for orientation information in fMRI and demonstrates that fMRI also reflects fine-grained patterns of neuronal selectivity.

The impact of ultra-high field MRI on cognitive and computational neuroimaging

The ability to measure functional brain responses non-invasively with ultra high field MRI (7 T and above) represents a unique opportunity in advancing our understanding of the human brain. Compared to lower fields (3 T and below), ultra high field MRI has an increased sensitivity, which can be used to acquire functional images with greater spatial resolution, and greater specificity of the blood oxygen level dependent (BOLD) signal to the underlying neuronal responses. Together, increased resolution and specificity enable investigating brain functions at a submillimeter scale, which so far could only be done with invasive techniques. At this mesoscopic spatial scale, perception, cognition and behavior can be probed at the level of fundamental units of neural computations, such as cortical columns, cortical layers, and subcortical nuclei. This represents a unique and distinctive advantage that differentiates ultra high from lower field imaging and that can foster a tighter link between fMRI and computational modeling of neural networks. So far, functional brain mapping at submillimeter scale has focused on the processing of sensory information and on well-known systems for which extensive information is available from invasive recordings in animals. It remains an open challenge to extend this methodology to uniquely human functions and, more generally, to systems for which animal models may be problematic. To succeed, the possibility to acquire high-resolution functional data with large spatial coverage, the availability of computational models of neural processing as well as accurate biophysical modeling of neurovascular coupling at mesoscopic scale all appear necessary.

Habituation of visual adaptation

Our sensory system adjusts its function driven by both shorter-term (e.g. adaptation) and longer-term (e.g. learning) experiences. Most past adaptation literature focuses on short-term adaptation. Only recently researchers have begun to investigate how adaptation changes over a span of days. This question is important, since in real life many environmental changes stretch over multiple days or longer. However, the answer to the question remains largely unclear. Here we addressed this issue by tracking perceptual bias (also known as aftereffect) induced by motion or contrast adaptation across multiple daily adaptation sessions. Aftereffects were measured every day after adaptation, which corresponded to the degree of adaptation on each day. For passively viewed adapters, repeated adaptation attenuated aftereffects. Once adapters were presented with an attentional task, aftereffects could either reduce for easy tasks, or initially show an increase followed by a later decrease for demanding tasks. Quantitative analysis of the decay rates in contrast adaptation showed that repeated exposure of the adapter appeared to be equivalent to adaptation to a weaker stimulus. These results suggest that both attention and a non-attentional habituation-like mechanism jointly determine how adaptation develops across multiple daily sessions.

Visual attention mitigates information loss in small- and large-scale neural codes

The visual system transforms complex inputs into robust and parsimonious neural codes that efficiently guide behavior. Because neural communication is stochastic, the amount of encoded visual information necessarily decreases with each synapse. This constraint requires that sensory signals are processed in a manner that protects information about relevant stimuli from degradation. Such selective processing--or selective attention--is implemented via several mechanisms, including neural gain and changes in tuning properties. However, examining each of these effects in isolation obscures their joint impact on the fidelity of stimulus feature representations by large-scale population codes. Instead, large-scale activity patterns can be used to reconstruct representations of relevant and irrelevant stimuli, thereby providing a holistic understanding about how neuron-level modulations collectively impact stimulus encoding.