Interacting roles of attention and visual salience in V4

Feature-Specific Salience Maps in Human Cortex

Priority map theory is a leading framework for understanding how various aspects of stimulus displays and task demands guide visual attention. Per this theory, the visual system computes a priority map, which is a representation of visual space indexing the relative importance, or priority, of locations in the environment. Priority is computed based on both salience, defined based on image-computable properties; and relevance, defined by an individual's current goals, and is used to direct attention to the highest-priority locations for further processing. Computational theories suggest that priority maps identify salient locations based on individual feature dimensions (e.g., color, motion), which are integrated into an aggregate priority map. While widely accepted, a core assumption of this framework, the existence of independent feature dimension maps in visual cortex, remains untested. Here, we tested the hypothesis that retinotopic regions selective for specific feature dimensions (color or motion) in human cortex act as neural feature dimension maps, indexing salient locations based on their preferred feature. We used fMRI activation patterns to reconstruct spatial maps while male and female human participants viewed stimuli with salient regions defined by relative color or motion direction. Activation in reconstructed spatial maps was localized to the salient stimulus position in the display. Moreover, the strength of the stimulus representation was strongest in the ROI selective for the salience-defining feature. Together, these results suggest that feature-selective extrastriate visual regions highlight salient locations based on local feature contrast within their preferred feature dimensions, supporting their role as neural feature dimension maps.SIGNIFICANCE STATEMENT Identifying salient information is important for navigating the world. For example, it is critical to detect a quickly approaching car when crossing the street. Leading models of computer vision and visual search rely on compartmentalized salience computations based on individual features; however, there has been no direct empirical demonstration identifying neural regions as responsible for performing these dissociable operations. Here, we provide evidence of a critical double dissociation that neural activation patterns from color-selective regions prioritize the location of color-defined salience while minimally representing motion-defined salience, whereas motion-selective regions show the complementary result. These findings reveal that specialized cortical regions act as neural "feature dimension maps" that are used to index salient locations based on specific features to guide attention.

How deep is the brain? The shallow brain hypothesis

Deep learning and predictive coding architectures commonly assume that inference in neural networks is hierarchical. However, largely neglected in deep learning and predictive coding architectures is the neurobiological evidence that all hierarchical cortical areas, higher or lower, project to and receive signals directly from subcortical areas. Given these neuroanatomical facts, today's dominance of cortico-centric, hierarchical architectures in deep learning and predictive coding networks is highly questionable; such architectures are likely to be missing essential computational principles the brain uses. In this Perspective, we present the shallow brain hypothesis: hierarchical cortical processing is integrated with a massively parallel process to which subcortical areas substantially contribute. This shallow architecture exploits the computational capacity of cortical microcircuits and thalamo-cortical loops that are not included in typical hierarchical deep learning and predictive coding networks. We argue that the shallow brain architecture provides several critical benefits over deep hierarchical structures and a more complete depiction of how mammalian brains achieve fast and flexible computational capabilities.

Eye movement evidence for the V1 Saliency Hypothesis and the Central-peripheral Dichotomy theory in an anomalous visual search task

Typically, searching for a target among uniformly tilted non-targets is easier when this target is perpendicular, rather than parallel, to the non-targets. The V1 Saliency Hypothesis (V1SH) - that V1 creates a saliency map to guide attention exogenously - predicts exactly the opposite in a special case: each target or non-target is a pair of equally-sized disks, a homo-pair of two disks of the same color, black or white, or a hetero-pair of two disks of the opposite color; the inter-disk displacement defines its orientation. This prediction - parallel advantage - was supported by the finding that parallel targets require shorter reaction times (RTs) to report targets' locations. Furthermore, it is stronger for targets further from the center of search images, as predicted by the Central-peripheral Dichotomy (CPD) theory entailing that saliency effects are stronger in peripheral than in central vision. However, the parallel advantage could arise from a shorter time required to recognize - rather than to shift attention to - the parallel target. By gaze tracking, the present study confirms that the parallel advantage is solely due to the RTs for the gaze to reach the target. Furthermore, when the gaze is sufficiently far from the target during search, saccade to a parallel, rather than perpendicular, target is more likely, demonstrating the Central-peripheral Dichotomy more directly. Parallel advantage is stronger among observers encouraged to let their search be guided by spontaneous gaze shifts, which are presumably guided by bottom-up saliency rather than top-down factors.

Nonadditive integration of visual information in ensemble processing

Statistically summarizing information from a stimulus array into an ensemble representation (e.g., the mean) improves the efficiency of visual processing. However, little is known about how the brain computes the ensemble statistics. Here, we propose that ensemble processing is realized by nonadditive integration, rather than linear averaging, of individual items. We used a linear regression model approach to extract EEG responses to three levels of information: the individual items, their local interactions, and their global interaction. The local and global interactions, representing nonadditive integration of individual items, elicited rapid and independent neural responses. Critically, only the neural representation of the global interaction predicted the precision of the ensemble perception at the behavioral level. Furthermore, spreading attention over the global pattern to enhance ensemble processing directly promoted rapid neural representation of the global interaction. Taken together, these findings advocate a global, nonadditive mechanism of ensemble processing in the brain.

Context-dependent attentional spotlight in pulvinar-V1 interaction

Spatial attention is often described as a mental spotlight that enhances information processing at the attended location. Using fMRI, we investigated background connectivity between the pulvinar and V1 in relation to focused versus diffused attention allocation, in weak and strong crowding contexts. Our findings revealed that focused attention led to enhanced correlations between the pulvinar and V1. Notably, this modulation was initiated by the pulvinar, and the strength of the modulation was dependent on the saliency of the target. These findings suggest that the pulvinar initiates information reweighting to V1, which underlies attentional selection in cluttered scenes.

Feedforward attentional selection in sensory cortex

Salient objects grab attention because they stand out from their surroundings. Whether this phenomenon is accomplished by bottom-up sensory processing or requires top-down guidance is debated. We tested these alternative hypotheses by measuring how early and in which cortical layer(s) neural spiking distinguished a target from a distractor. We measured synaptic and spiking activity across cortical columns in mid-level area V4 of male macaque monkeys performing visual search for a color singleton. A neural signature of attentional capture was observed in the earliest response in the input layer 4. The magnitude of this response predicted response time and accuracy. Errant behavior followed errant selection. Because this response preceded top-down influences and arose in the cortical layer not targeted by top-down connections, these findings demonstrate that feedforward activation of sensory cortex can underlie attentional priority.

Pathway and directional specificity of Hebbian plasticity in the cortical visual motion processing network

Cortico-cortical paired associative stimulation (ccPAS), which repeatedly pairs single-pulse transcranial magnetic stimulation (TMS) over two distant brain regions, is thought to modulate synaptic plasticity. We explored its spatial selectivity (pathway and direction specificity) and its nature (oscillatory signature and perceptual consequences) when applied along the ascending (Forward) and descending (Backward) motion discrimination pathway. We found unspecific connectivity increases in bottom-up inputs in the low gamma band, probably reflecting visual task exposure. A clear distinction in information transfer occurred in the re-entrant alpha signals, which were only modulated by Backward-ccPAS, and predictive of visual improvements in healthy participants. These results suggest a causal involvement of the re-entrant MT-to-V1 low-frequency inputs in motion discrimination and integration in healthy participants. Modulating re-entrant input activity could provide single-subject prediction scenarios for visual recovery. Visual recovery might indeed partly rely on these residual inputs projecting to spared V1 neurons.

Attention Modulates Human Visual Responses to Objects by Tuning Sharpening

Visual stimuli compete with each other for cortical processing and attention biases this competition in favor of the attended stimulus. How does the relationship between the stimuli affect the strength of this attentional bias? Here, we used functional MRI to explore the effect of target-distractor similarity in neural representation on attentional modulation in the human visual cortex using univariate and multivariate pattern analyses. Using stimuli from four object categories (human bodies, cats, cars and houses), we investigated attentional effects in the primary visual area V1, the object-selective regions LO and pFs, the body-selective region EBA, and the scene-selective region PPA. We demonstrated that the strength of the attentional bias towards the target is not fixed but decreases with increasing distractor-target similarity. Simulations provided evidence that this result pattern is explained by tuning sharpening rather than an increase in gain. Our findings provide a mechanistic explanation for behavioral effects of target-distractor similarity on attentional biases and suggest tuning sharpening as the underlying mechanism in object-based attention.

Which processes dominate visual search: Bottom-up feature contrast, top-down tuning or trial history?

Previous research has identified three mechanisms that guide visual attention: bottom-up feature contrasts, top-down tuning, and the trial history (e.g., priming effects). However, only few studies have simultaneously examined all three mechanisms. Hence, it is currently unclear how they interact or which mechanisms dominate over others. With respect to local feature contrasts, it has been claimed that a pop-out target can only be selected immediately in dense displays when the target has a high local feature contrast, but not when the displays are sparse, which leads to an inverse set-size effect. The present study critically evaluated this view by systematically varying local feature contrasts (i.e., set size), top-down knowledge, and the trial history in pop-out search. We used eye tracking to distinguish between early selection and later identification-related processes. The results revealed that early visual selection was mainly dominated by top-down knowledge and the trial history: When attention was biased to the target feature, either by valid pre-cueing (top-down) or automatic priming, the target could be localised immediately, regardless of display density. Bottom-up feature contrasts only modulated selection when the target was unknown and attention was biased to the non-targets. We also replicated the often-reported finding of reliable feature contrast effects in the mean RTs, but showed that these were due to later, target identification processes (e.g., in the target dwell times). Thus, contrary to the prevalent view, bottom-up feature contrasts in dense displays do not seem to directly guide attention, but only facilitate nontarget rejection, probably by facilitating nontarget grouping.

The angry versus happy recognition advantage: the role of emotional and physical properties

Facial emotional expressions are pivotal for social communication. Their fast and accurate recognition is crucial to promote adaptive responses to social demands, for the development of functional relationships, and for well-being. However, the literature has been inconsistent in showing differentiated recognition patterns for positive vs. negative facial expressions (e.g., happy and angry expressions, respectively), likely due to affective and perceptual factors. Accordingly, the present study explored differences in recognition performance between angry and happy faces, while specifically assessing the role of emotional intensity and global/regional low-level visual features. 98 participants categorized angry and happy faces morphed between neutral and emotional across 9 levels of expression intensity (10-90%). We observed a significantly higher recognition efficiency (higher accuracy and shorter response latencies) for angry compared to happy faces in lower levels of expression intensity, suggesting that our cognitive resources are biased to prioritize the recognition of potentially harmful stimuli, especially when briefly presented at an ambiguous stage of expression. Conversely, an advantage for happy faces was observed from the midpoint of expression intensity, regarding response speed. However, when compensating for the contribution of regional low-level properties of distinct facial key regions, the effect of emotion was maintained only for response accuracy. Altogether, these results shed new light on the processing of facial emotional stimuli, emphasizing the need to consider emotional intensity and regional low-level image properties in emotion recognition analysis.

When information security depends on font size: how the saliency of warnings affects protection behavior

Prior research on how to improve the effectiveness of information security warnings has predominantly focused on either the informational content of warnings or their visual saliency. In an online experiment (N = 1'486), we disentangle the effect of both manipulations and demonstrate that both factors simultaneously influence decision making. Our data indicate that the proportion of people who engage in protection behavior can be increased by roughly 65% by making a particular warning message more visually salient (i.e. a more conspicuous visual design is used). We also show that varying the message's saliency can make people behave very differently when confronted with the same threat or behave very similarly when confronted with threats that differ widely in terms of severity of outcomes. Our results suggest that the visual design of a warning may warrant at least as much attention as the informational content that the warning message conveys.

Preattentive facilitation of target trajectories in a dragonfly visual neuron

The ability to pursue targets in visually cluttered and distraction-rich environments is critical for predators such as dragonflies. Previously, we identified Centrifugal Small-Target Motion Detector 1 (CSTMD1), a dragonfly visual neuron likely involved in such target-tracking behaviour. CSTMD1 exhibits facilitated responses to targets moving along a continuous trajectory. Moreover, CSTMD1 competitively selects a single target out of a pair. Here, we conducted in vivo, intracellular recordings from CSTMD1 to examine the interplay between facilitation and selection, in response to the presentation of paired targets. We find that neuronal responses to both individual trajectories of simultaneous, paired targets are facilitated, rather than being constrained to the single, selected target. Additionally, switches in selection elicit suppression which is likely an important attribute underlying target pursuit. However, binocular experiments reveal these results are constrained to paired targets within the same visual hemifield, while selection of a target in one visual hemifield establishes ocular dominance that prevents facilitation or response to contralaterally presented targets. These results reveal that the dragonfly brain preattentively represents more than one target trajectory, to balance between attentional flexibility and resistance against distraction.

A dragonfly visual neuron independently facilitates responses to rival targets within the same visual field, mediating selective attention.

Eye-specific attentional bias driven by selection history

Attention helps in selection among competing stimuli, but attentional selection also biases subsequent information processing as a prior experience. Previous studies have demonstrated that intertrial repetition of target features or locations facilitates perceptual processing as selection history guides attention. In the current study, we found that eye selection history in binocular rivalry induces eye-specific attentional bias. In four experiments, participants responded to the target presented at one of the locations on either eye. The results showed that the target was detected faster when presented to the same eye as in the previous trial under binocular rivalry. However, the effect of eye repetition was not observed when the interocular conflict was reduced by presenting stimuli to only one eye on each trial. Our result indicates that eye selection history can affect eye dominance during binocular rivalry as attention amplifies selected information among competing inputs. These findings suggest that prior experience of attentional deployment modulates subsequent information processing owing to the residual effect of attentional amplification.

Dynamic causal interactions between occipital and parietal cortex explain how endogenous spatial attention and stimulus-driven salience jointly shape the distribution of processing priorities in 2D visual space

Visuo-spatial attention prioritizes the processing of relevant inputs via different types of signals, including current goals and stimulus salience. Complex mixtures of these signals engage in everyday life situations, but little is known about how these signals jointly modulate distributed patterns of activity across the occipital regions that represent visual space. Here, we measured spatio-topic, quadrant-specific occipital activity during the processing of visual displays containing both task-relevant targets and salient color-singletons. We computed spatial bias vectors indexing the effect of attention in 2D space, as coded by distributed activity in the occipital cortex. We found that goal-directed spatial attention biased activity towards the target and that salience further modulated this endogenous effect: salient distractors decreased the spatial bias, while salient targets increased it. Analyses of effective connectivity revealed that the processing of salient distractors relied on the modulation of the bidirectional connectivity between the occipital and the posterior parietal cortex, as well as the modulation of the lateral interactions within the occipital cortex. These findings demonstrate that goal-directed attention and salience jointly contribute to shaping processing priorities in the occipital cortex and highlight that multiple functional paths determine how spatial information about these signals is distributed across occipital regions.

The Ecological View of Selective Attention

Accumulating evidence is supporting the hypothesis that our selective attention is a manifestation of mechanisms that evolved early in evolution and are shared by many organisms from different taxa. This surge of new data calls for the re-examination of our notions about attention, which have been dominated mostly by human psychology. Here, we present an hypothesis that challenges, based on evolutionary grounds, a common view of attention as a means to manage limited brain resources. We begin by arguing that evolutionary considerations do not favor the basic proposition of the limited brain resources view of attention, namely, that the capacity of the sensory organs to provide information exceeds the capacity of the brain to process this information. Moreover, physiological studies in animals and humans show that mechanisms of selective attention are highly demanding of brain resources, making it paradoxical to see attention as a means to release brain resources. Next, we build on the above arguments to address the question why attention evolved in evolution. We hypothesize that, to a certain extent, limiting sensory processing is adaptive irrespective of brain capacity. We call this hypothesis the ecological view of attention (EVA) because it is centered on interactions of an animal with its environment rather than on internal brain resources. In its essence is the notion that inherently noisy and degraded sensory inputs serve the animal’s adaptive, dynamic interactions with its environment. Attention primarily functions to resolve behavioral conflicts and false distractions. Hence, we evolved to focus on a particular target at the expense of others, not because of internal limitations, but to ensure that behavior is properly oriented and committed to its goals. Here, we expand on this notion and review evidence supporting it. We show how common results in human psychophysics and physiology can be reconciled with an EVA and discuss possible implications of the notion for interpreting current results and guiding future research.

Dragonfly Neurons Selectively Attend to Targets Within Natural Scenes

Aerial predators, such as the dragonfly, determine the position and movement of their prey even when both are moving through complex, natural scenes. This task is likely supported by a group of neurons in the optic lobe which respond to moving targets that subtend less than a few degrees. These Small Target Motion Detector (STMD) neurons are tuned to both target size and velocity, whilst also exhibiting facilitated responses to targets traveling along continuous trajectories. When presented with a pair of targets, some STMDs generate spiking activity that represent a competitive selection of one target, as if the alternative does not exist (i.e., selective attention). Here, we describe intracellular responses of CSTMD1 (an identified STMD) to the visual presentation of targets embedded within cluttered, natural scenes. We examine CSTMD1 response changes to target contrast, as well as a range of target and background velocities. We find that background motion affects CSTMD1 responses via the competitive selection between features within the natural scene. Here, robust discrimination of our artificially embedded “target” is limited to scenarios when its velocity is matched to, or greater than, the background velocity. Additionally, the background’s direction of motion affects discriminability, though not in the manner observed in STMDs of other flying insects. Our results highlight that CSTMD1’s competitive responses are to those features best matched to the neuron’s underlying spatiotemporal tuning, whether from the embedded target or other features in the background clutter. In many scenarios, CSTMD1 responds robustly to targets moving through cluttered scenes. However, whether this neuronal system could underlie the task of competitively selecting slow moving prey against fast-moving backgrounds remains an open question.

Pop-out search instigates beta-gated feature selectivity enhancement across V4 layers

Visual search is a workhorse for investigating how attention interacts with processing of sensory information. Attentional selection has been linked to altered cortical sensory responses and feature preferences (i.e., tuning). However, attentional modulation of feature selectivity during search is largely unexplored. Here we map the spatiotemporal profile of feature selectivity during singleton search. Monkeys performed a search where a pop-out feature determined the target of attention. We recorded laminar neural responses from visual area V4. We first identified "feature columns" which showed preference for individual colors. In the unattended condition, feature columns were significantly more selective in superficial relative to middle and deep layers. Attending a stimulus increased selectivity in all layers but not equally. Feature selectivity increased most in the deep layers, leading to higher selectivity in extragranular layers as compared to the middle layer. This attention-induced enhancement was rhythmically gated in phase with the beta-band local field potential. Beta power dominated both extragranular laminar compartments, but current source density analysis pointed to an origin in superficial layers, specifically. While beta-band power was present regardless of attentional state, feature selectivity was only gated by beta in the attended condition. Neither the beta oscillation nor its gating of feature selectivity varied with microsaccade production. Importantly, beta modulation of neural activity predicted response times, suggesting a direct link between attentional gating and behavioral output. Together, these findings suggest beta-range synaptic activation in V4's superficial layers rhythmically gates attentional enhancement of feature tuning in a way that affects the speed of attentional selection.

Functional Segregation within the Dorsal Frontoparietal Network: A Multimodal Dynamic Causal Modeling Study

Discrimination and integration of motion direction requires the interplay of multiple brain areas. Theoretical accounts of perception suggest that stimulus-related (i.e., exogenous) and decision-related (i.e., endogenous) factors affect distributed neuronal processing at different levels of the visual hierarchy. To test these predictions, we measured brain activity of healthy participants during a motion discrimination task, using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). We independently modeled the impact of exogenous factors (task demand) and endogenous factors (perceptual decision-making) on the activity of the motion discrimination network and applied Dynamic Causal Modeling (DCM) to both modalities. DCM for event-related potentials (DCM-ERP) revealed that task demand impacted the reciprocal connections between the primary visual cortex (V1) and medial temporal areas (V5). With practice, higher visual areas were increasingly involved, as revealed by DCM-fMRI. Perceptual decision-making modulated higher levels (e.g., V5-to-Frontal Eye Fields, FEF), in a manner predictive of performance. Our data suggest that lower levels of the visual network support early, feature-based selection of responses, especially when learning strategies have not been implemented. In contrast, perceptual decision-making operates at higher levels of the visual hierarchy by integrating sensory information with the internal state of the subject.

A causal role for the right frontal eye fields in value comparison

Recent studies have suggested close functional links between overt visual attention and decision making. This suggests that the corresponding mechanisms may interface in brain regions known to be crucial for guiding visual attention - such as the frontal eye field (FEF). Here, we combined brain stimulation, eye tracking, and computational approaches to explore this possibility. We show that inhibitory transcranial magnetic stimulation (TMS) over the right FEF has a causal impact on decision making, reducing the effect of gaze dwell time on choice while also increasing reaction times. We computationally characterize this putative mechanism by using the attentional drift diffusion model (aDDM), which reveals that FEF inhibition reduces the relative discounting of the non-fixated option in the comparison process. Our findings establish an important causal role of the right FEF in choice, elucidate the underlying mechanism, and provide support for one of the key causal hypotheses associated with the aDDM.

Learning divisive normalization in primary visual cortex

Divisive normalization (DN) is a prominent computational building block in the brain that has been proposed as a canonical cortical operation. Numerous experimental studies have verified its importance for capturing nonlinear neural response properties to simple, artificial stimuli, and computational studies suggest that DN is also an important component for processing natural stimuli. However, we lack quantitative models of DN that are directly informed by measurements of spiking responses in the brain and applicable to arbitrary stimuli. Here, we propose a DN model that is applicable to arbitrary input images. We test its ability to predict how neurons in macaque primary visual cortex (V1) respond to natural images, with a focus on nonlinear response properties within the classical receptive field. Our model consists of one layer of subunits followed by learned orientation-specific DN. It outperforms linear-nonlinear and wavelet-based feature representations and makes a significant step towards the performance of state-of-the-art convolutional neural network (CNN) models. Unlike deep CNNs, our compact DN model offers a direct interpretation of the nature of normalization. By inspecting the learned normalization pool of our model, we gained insights into a long-standing question about the tuning properties of DN that update the current textbook description: we found that within the receptive field oriented features were normalized preferentially by features with similar orientation rather than non-specifically as currently assumed.

Voluntary and Involuntary Attention in Bistable Visual Perception: A MEG Study

In this study, voluntary and involuntary visual attention focused on different interpretations of a bistable image, were investigated using magnetoencephalography (MEG). A Necker cube with sinusoidally modulated pixels' intensity in the front and rear faces with frequencies 6.67 Hz (60/9) and 8.57 Hz (60/7), respectively, was presented to 12 healthy volunteers, who interpreted the cube as either left- or right-oriented. The tags of these frequencies and their second harmonics were identified in the average Fourier spectra of the MEG data recorded from the visual cortex. In the first part of the experiment, the subjects were asked to voluntarily control their attention by interpreting the cube orientation as either being on the left or right. Accordingly, we observed the dominance of the corresponding spectral component, and voluntary attention performance was measured. In the second part of the experiment, the subjects were asked to focus their gaze on a red marker at the center of the cube image without putting forth effort in its interpretation. The alternation of the dominant spectral energies at the second harmonics of the stimulation frequencies was treated as changes in the cube orientation. Based on the results of the first experimental stage and using a wavelet analysis, we developed a method which allowed us to identify the currently perceived cube orientation. Finally, we characterized involuntary attention using the distribution of dominance times when focusing attention on one of the cube orientations, which was related to voluntary attention performance and brain noise. In particular, we confirmed our hypothesis that higher attention performance is associated with stronger brain noise.

A Bottom-Up Validation of the IAPS, GAPED, and NAPS Affective Picture Databases: Differential Effects on Behavioral Performance

The concept of emotion is a complex neural and psychological phenomenon, central to the organization of human social behavior. As the result of subjective experience, emotions involve bottom-up cognitive styles responsible for efficient adaptation of human behavior to the environment based on salient goals. Indeed, bottom-up cognitive processes are mandatory for clarifying emotion-cognition interactions. Accordingly, a huge number of studies and standardized affective stimuli databases have been developed (i.e., International Affective Picture System (IAPS), Geneva Affective Picture Database (GAPED), and Nencki Affective Picture System (NAPS)). However, these neither accurately reflect the complex neural system underlying emotional responses nor do they offer a comprehensive framework for researchers. The present article aims to provide an additional bottom-up validation of affective stimuli that are independent from cognitive processing and control mechanisms, related to the implicit relevance and evolutionistic significance of stimuli. A subset of 360 images from the original NAPS, GAPED, and IAPS datasets was selected in order to proportionally cover the whole dimensional affective space. Among these, using a two-step analysis strategy, we identified three clusters (“good performance”, “poor performance”, and “false alarm”) of stimuli with similar cognitive response profiles. Results showed that the three clusters differed in terms of arousal and database membership, but not in terms of valence. The new database, with accompanying ratings and image parameters, allows researchers to select visual stimuli independent from dimensional/discrete-categories, and provides information on the implicit effects triggered by such stimuli.

Trypophobia: Heart rate, heart rate variability and cortical haemodynamic response

BACKGROUND

Trypophobia is a common condition in which sufferers are averse to images of small holes arranged in clusters.

METHODS

We used photo-plethysmography to examine cardiovascular correlates and near infrared spectroscopy to examine cortical correlates of the phenomenon in order to validate the Trypophobia Questionnaire and explore the several interlinked explanations of the disorder.

RESULTS

Trypophobic images were found to increase both heart rate and heart rate variability, but only in individuals with high scores on the Trypophobia Questionnaire. Trypophobic images were also found to elicit larger haemodynamic responses in posterior cortical areas, but again only in individuals with high scores.

LIMITATIONS

The results are consistent with a contribution from both parasympathetic and sympathetic systems.

CONCLUSION

The data demonstrate the validity of the Trypophobia Questionnaire and show an involvement not only of the autonomic system but cortical mechanisms including cortical hyperexcitability.

An Attempt at a Unified Theory of the Neocortical Microcircuit in Sensory Cortex

The neocortex performs a wide range of functions, including working memory, sensory perception, and motor planning. Despite this diversity in function, evidence suggests that the neocortex is made up of repeating subunits ("macrocolumns"), each of which is largely identical in circuitry. As such, the specific computations performed by these macrocolumns are of great interest to neuroscientists and AI researchers. Leading theories of this microcircuit include models of predictive coding, hierarchical temporal memory (HTM), and Adaptive Resonance Theory (ART). However, these models have not yet explained: (1) how microcircuits learn sequences input with delay (i.e., working memory); (2) how networks of columns coordinate processing on precise timescales; or (3) how top-down attention modulates sensory processing. I provide a theory of the neocortical microcircuit that extends prior models in all three ways. Additionally, this theory provides a novel working memory circuit that extends prior models to support simultaneous multi-item storage without disrupting ongoing sensory processing. I then use this theory to explain the functional origin of a diverse set of experimental findings, such as cortical oscillations.

Parietal Cortex Regulates Visual Salience and Salience-Driven Behavior

Unique stimuli stand out. Despite an abundance of competing sensory stimuli, the detection of the most salient ones occurs without effort, and that detection contributes to the guidance of adaptive behavior. Neurons sensitive to the salience of visual stimuli are widespread throughout the primate visual system and are thought to shape the selection of visual targets. However, a neural source of salience remains elusive. In an attempt to identify a source of visual salience, we reversibly inactivated parietal cortex and simultaneously recorded salience signals in prefrontal cortex. Inactivation of parietal cortex not only caused pronounced and selective reductions of salience signals in prefrontal cortex but also diminished the influence of salience on visually guided behavior. These observations demonstrate a causal role of parietal cortex in regulating salience signals within the brain and in controlling salience-driven behavior.

Functional Specialization in the Attention Network

Spatial attention is comprised of neural mechanisms that boost sensory processing at a behaviorally relevant location while filtering out competing information. The present review examines functional specialization in the network of brain regions that directs such preferential processing. This attention network includes both cortical (e.g., frontal and parietal cortices) and subcortical (e.g., the superior colliculus and the pulvinar nucleus of the thalamus) structures. Here, we piece together existing evidence that these various nodes of the attention network have dissociable functional roles by synthesizing results from electrophysiology and neuroimaging studies. We describe functional specialization across several dimensions (e.g., at different processing stages and within different behavioral contexts), while focusing on spatial attention as a dynamic process that unfolds over time. Functional contributions from each node of the attention network can change on a moment-to-moment timescale, providing the necessary cognitive flexibility for sampling from highly dynamic environments.

The resonant brain: How attentive conscious seeing regulates action sequences that interact with attentive cognitive learning, recognition, and prediction

This article describes mechanistic links that exist in advanced brains between processes that regulate conscious attention, seeing, and knowing, and those that regulate looking and reaching. These mechanistic links arise from basic properties of brain design principles such as complementary computing, hierarchical resolution of uncertainty, and adaptive resonance. These principles require conscious states to mark perceptual and cognitive representations that are complete, context sensitive, and stable enough to control effective actions. Surface-shroud resonances support conscious seeing and action, whereas feature-category resonances support learning, recognition, and prediction of invariant object categories. Feedback interactions between cortical areas such as peristriate visual cortical areas V2, V3A, and V4, and the lateral intraparietal area (LIP) and inferior parietal sulcus (IPS) of the posterior parietal cortex (PPC) control sequences of saccadic eye movements that foveate salient features of attended objects and thereby drive invariant object category learning. Learned categories can, in turn, prime the objects and features that are attended and searched. These interactions coordinate processes of spatial and object attention, figure-ground separation, predictive remapping, invariant object category learning, and visual search. They create a foundation for learning to control motor-equivalent arm movement sequences, and for storing these sequences in cognitive working memories that can trigger the learning of cognitive plans with which to read out skilled movement sequences. Cognitive-emotional interactions that are regulated by reinforcement learning can then help to select the plans that control actions most likely to acquire valued goal objects in different situations. Many interdisciplinary psychological and neurobiological data about conscious and unconscious behaviors in normal individuals and clinical patients have been explained in terms of these concepts and mechanisms.

A Target-Detecting Visual Neuron in the Dragonfly Locks on to Selectively Attended Targets

The visual world projects a complex and rapidly changing image onto the retina of many animal species. This presents computational challenges for those animals reliant on visual processing to provide an accurate representation of the world. One such challenge is parsing a visual scene for the most salient targets, such as the selection of prey amid a swarm. The ability to selectively prioritize processing of some stimuli over others is known as 'selective attention'. We recently identified a dragonfly visual neuron called 'Centrifugal Small Target Motion Detector 1' (CSTMD1) that exhibits selective attention when presented with multiple, equally salient targets. Here we conducted in vivo, electrophysiological recordings from CSTMD1 in wild-caught male dragonflies (Hemicordulia tau), while presenting visual stimuli on an LCD monitor. To identify the target selected in any given trial, we uniquely modulated the intensity of the moving targets (frequency tagging). We found that the frequency information of the selected target is preserved in the neuronal response, while the distracter is completely ignored. We also show that the competitive system that underlies selection in this neuron can be biased by the presentation of a preceding target on the same trajectory, even when it is of lower contrast than an abrupt, novel distracter. With this improved method for identifying and biasing target selection in CSTMD1, the dragonfly provides an ideal animal model system to probe the neuronal mechanisms underlying selective attention.SIGNIFICANCE STATEMENT We present the first application of frequency tagging to intracellular neuronal recordings, demonstrating that the frequency component of a stimulus is encoded in the spiking response of an individual neuron. Using this technique as an identifier, we demonstrate that CSTMD1 'locks on' to a selected target and encodes the absolute strength of this target, even in the presence of abruptly appearing, high-contrast distracters. The underlying mechanism also permits the selection mechanism to switch between targets mid-trial, even among equivalent targets. Together, these results demonstrate greater complexity in this selective attention system than would be expected in a winner-takes-all network. These results are in contrast to typical findings in the primate and avian brain, but display intriguing resemblance to observations in human psychophysics.

A Computational Model of Attention Control in Multi-Attribute, Context-Dependent Decision Making

Real-life decisions often require a comparison of multi-attribute options with various benefits and costs, and the evaluation of each option depends partly on the others in the choice set (i.e., the choice context). Although reinforcement learning models have successfully described choice behavior, how to account for multi-attribute information when making a context-dependent decision remains unclear. Here we develop a computational model of attention control that includes context effects on multi-attribute decisions, linking a context-dependent choice model with a reinforcement learning model. The overall model suggests that the distinctiveness of attributes guides an individual's preferences among multi-attribute options via an attention-control mechanism that determines whether choices are selectively biased toward the most distinctive attribute (selective attention) or proportionally distributed based on the relative distinctiveness of attributes (divided attention). To test the model, we conducted a behavioral experiment in rhesus monkeys, in which they made simple multi-attribute decisions over three conditions that manipulated the degree of distinctiveness between alternatives: (1) four foods of different size and calorie; (2) four pieces of the same food in different colors; and (3) four identical pieces of food. The model simulation of the choice behavior captured the preference bias (i.e., overall preference structure) and the choice persistence (repeated choices) in the empirical data, providing evidence for the respective influences of attention and memory on preference bias and choice persistence. Our study provides insights into computations underlying multi-attribute decisions, linking attentional control to decision-making processes.

A Non-cognitive Behavioral Model for Interpreting Functional Neuroimaging Studies

The dominant model for interpreting brain imaging experiments, which we refer to as the Standard Cognitive Model (SCM), assumes that the brain is organized in support of mental processes that control behavior. However, functional neuroimaging experiments of cognitive tasks have not shown clear anatomic segregation between mental processes originally proposed by this model. This failing has been blamed on limitations in imaging technology and non-linearity in the brain’s implementation of these processes. However, the validity of the underlying cognitive models used to describe the brain has rarely been questioned or directly tested against imaging results. We propose an alternative model of brain function, that we term the Non-cognitive Behavioral Model (NBM), which correlates observed human behavior directly with measured brain activity without making assumptions about intervening cognitive processes. Our model derives from behavioral psychology but is extended to include brain activity, in addition to behavior, as observables. A further extension is the role of neuroplasticity, as opposed to innate cognitive processes, in developing the brain’s support of cognitive behavior. We present the theoretical basis with which the SCM maps cognitive processes onto functional magnetic resonance and positron emission tomography images and compare and contrast with the NBM. We also describe how the NBM can be used experimentally to study how the brain supports behavior. Two applications are presented that support the usefulness of the NBM. In one, the NBM use of the total functional imaging signal (not just the differences between states) provides a stronger correlation of neural activity with the behavioral state of consciousness than the SCM approach in both anesthesia and coma. The second example reviews studies of facial and object recognition that provide evidence for the NBM proposal that neuroplasticity and experience play key roles in the brain’s support of recognition and other behaviors. The conclusions regarding neuroplasticity are then generalized to explain the incomplete functional segregation observed in the application of the SCM to neuroimaging.

Structure and function of dual-source cholinergic modulation in early vision

Behavioral states such as arousal and attention have profound effects on sensory processing, determining how-even whether-a stimulus is perceived. This state-dependence is believed to arise, at least in part, in response to inputs from subcortical structures that release neuromodulators such as acetylcholine, often nonsynaptically. The mechanisms that underlie the interaction between these nonsynaptic signals and the more point-to-point synaptic cortical circuitry are not well understood. This review highlights the state of the field, with a focus on cholinergic action in early visual processing. Key anatomical and physiological features of both the cholinergic and the visual systems are discussed. Furthermore, presenting evidence of cholinergic modulation in visual thalamus and primary visual cortex, we explore potential functional roles of acetylcholine and its effects on the processing of visual input over the sleep-wake cycle, sensory gain control during wakefulness, and consider evidence for cholinergic support of visual attention.

Contrast gain control in mouse auditory cortex

The neocortex is thought to employ a number of canonical computations, but little is known about whether these computations rely on shared mechanisms across different neural populations. In recent years, the mouse has emerged as a powerful model organism for the dissection of the circuits and mechanisms underlying various aspects of neural processing and therefore provides an important avenue for research into putative canonical computations. One such computation is contrast gain control, the systematic adjustment of neural gain in accordance with the contrast of sensory input, which helps to construct neural representations that are robust to the presence of background stimuli. Here, we characterized contrast gain control in the mouse auditory cortex. We performed laminar extracellular recordings in the auditory cortex of the anesthetized mouse while varying the contrast of the sensory input. We observed that an increase in stimulus contrast resulted in a compensatory reduction in the gain of neural responses, leading to representations in the mouse auditory cortex that are largely contrast invariant. Contrast gain control was present in all cortical layers but was found to be strongest in deep layers, indicating that intracortical mechanisms may contribute to these gain changes. These results lay a foundation for investigations into the mechanisms underlying contrast adaptation in the mouse auditory cortex. NEW & NOTEWORTHY We investigated whether contrast gain control, the systematic reduction in neural gain in response to an increase in sensory contrast, exists in the mouse auditory cortex. We performed extracellular recordings in the mouse auditory cortex while presenting sensory stimuli with varying contrasts and found this form of processing was widespread. This finding provides evidence that contrast gain control may represent a canonical cortical computation and lays a foundation for investigations into the underlying mechanisms.

Keyframe extraction from laparoscopic videos based on visual saliency detection

BACKGROUND AND OBJECTIVE

Laparoscopic surgery offers the potential for video recording of the operation, which is important for technique evaluation, cognitive training, patient briefing and documentation. An effective way for video content representation is to extract a limited number of keyframes with semantic information. In this paper we present a novel method for keyframe extraction from individual shots of the operational video.

METHODS

The laparoscopic video was first segmented into video shots using an objectness model, which was trained to capture significant changes in the endoscope field of view. Each frame of a shot was then decomposed into three saliency maps in order to model the preference of human vision to regions with higher differentiation with respect to color, motion and texture. The accumulated responses from each map provided a 3D time series of saliency variation across the shot. The time series was modeled as a multivariate autoregressive process with hidden Markov states (HMMAR model). This approach allowed the temporal segmentation of the shot into a predefined number of states. A representative keyframe was extracted from each state based on the highest state-conditional probability of the corresponding saliency vector.

RESULTS

Our method was tested on 168 video shots extracted from various laparoscopic cholecystectomy operations from the publicly available Cholec80 dataset. Four state-of-the-art methodologies were used for comparison. The evaluation was based on two assessment metrics: Color Consistency Score (CCS), which measures the color distance between the ground truth (GT) and the closest keyframe, and Temporal Consistency Score (TCS), which considers the temporal proximity between GT and extracted keyframes. About 81% of the extracted keyframes matched the color content of the GT keyframes, compared to 77% yielded by the second-best method. The TCS of the proposed and the second-best method was close to 1.9 and 1.4 respectively.

CONCLUSIONS

Our results demonstrated that the proposed method yields superior performance in terms of content and temporal consistency to the ground truth. The extracted keyframes provided highly semantic information that may be used for various applications related to surgical video content representation, such as workflow analysis, video summarization and retrieval.

Contextual-Dependent Attention Effect on Crowded Orientation Signals in Human Visual Cortex

A target becomes hard to identify with nearby visual stimuli. This phenomenon, known as crowding, places a fundamental limit on conscious perception and object recognition. To understand the neural representation of crowded stimuli, we used fMRI and a forward encoding model to reconstruct the target-specific feature from multivoxel activation patterns evoked by orientation patches. Orientation-selective response profiles were constructed in V1-V4 for a target embedded in different contexts. Subjects of both sexes either directed their attention over all the orientation patches or selectively to the target. In the context with a weak crowding effect, attending to the target enhanced the orientation selectivity of the response profile; such effect increased along the visual pathway. In the context with a strong crowding effect, attending to the target enhanced the orientation selectivity of the response profile in the earlier visual area, but not in V4. The increase and decrease of orientation selectivity along the visual hierarchy demonstrate a contextual-dependent attention effect on crowded orientation signals: in the context with a weak crowding effect, selective attention gradually resolves the target from nearby distractors along the hierarchy; in the context with a strong crowding effect, while selective attention maintains the target feature in the earlier visual area, its effect decreases in the downstream area. Our findings reveal how the human visual system represents the target-specific feature at multiple stages under the limit of attention selection in a cluttered scene.SIGNIFICANCE STATEMENT Using fMRI and a forward encoding model, we reconstructed orientation-selective response profiles for a target embedded in crowded contexts. In the context with a weak crowding effect, attention gradually resolves the target from nearby distractors along the visual hierarchy. In the context with a strong crowding effect, while the feature of the target is preserved in the early visual cortex, it degrades in the later visual processing stage. The increase and decrease of orientation selectivity along the visual hierarchy reveal how the human visual system strikes to present the target-specific feature under the limit of attention selection in a cluttered scene.

Enhancing the Visibility of Delamination during Pulsed Thermography of Carbon Fiber-Reinforced Plates Using a Stacked Autoencoder

The effectiveness of pulsed thermography (PT) for detecting delamination in carbon fiber-reinforced polymer (CFRP) plates has been widely verified. However, delaminations are usually characterized by weak visibility due to the influences of inspection factors and the delaminations with weak visibility are easily missed in real inspections. In this study, by introducing a deep learning algorithm—stacked autoencoder (SAE)—to PT, we propose a novel approach (SAE-PT) to enhance the visibility of delaminations. Based on the ability of SAE to learn unsupervised features from data, the thermal features of delaminations are extracted from the raw thermograms. The extracted features are then employed to construct SAE images, in which the visibility of delaminations is expected to be enhanced. To test the performance of SAE-PT, we inspected CFRP plates with prefabricated delaminations. By implementing SAE-PT on the raw inspection data, the delaminations were more clearly indicated in the constructed SAE images. We also compare SAE-PT to the widely used principal component thermography (PCT) method to further verify the validity of the proposed approach. The results reveal that compared to PCT, SAE-PT can show delaminations in CFRP with higher contrast. By effectively enhancing the delamination visibility, SAE-PT thus has potential for improving the inspection accuracy of PT for non-destructive testing (NDT) of CFRP.

Multiscale and Multifeature Segmentation of High-Spatial Resolution Remote Sensing Images Using Superpixels with Mutual Optimal Strategy

High spatial resolution (HSR) image segmentation is considered to be a major challenge for object-oriented remote sensing applications that have been extensively studied in the past. In this paper, we propose a fast and efficient framework for multiscale and multifeatured hierarchical image segmentation (MMHS). First, the HSR image pixels were clustered into a small number of superpixels using a simple linear iterative clustering algorithm (SLIC) on modern graphic processing units (GPUs), and then a region adjacency graph (RAG) and nearest neighbors graph (NNG) were constructed based on adjacent superpixels. At the same time, the RAG and NNG successfully integrated spectral information, texture information, and structural information from a small number of superpixels to enhance its expressiveness. Finally, a multiscale hierarchical grouping algorithm was implemented to merge these superpixels using local-mutual best region merging (LMM). We compared the experiments with three state-of-the-art segmentation algorithms, i.e., the watershed transform segmentation (WTS) method, the mean shift (MS) method, the multiresolution segmentation (MRS) method integrated in commercial software, eCognition9, on New York HSR image datasets, and the ISPRS Potsdam dataset. Computationally, our algorithm was dozens of times faster than the others, and it also had the best segmentation effect through visual assessment. The supervised and unsupervised evaluation results further proved the superiority of the MMHS algorithm.

Visual brain plasticity induced by central and peripheral visual field loss

Disorders that specifically affect central and peripheral vision constitute invaluable models to study how the human brain adapts to visual deafferentation. We explored cortical changes after the loss of central or peripheral vision. Cortical thickness (CoTks) and resting-state cortical entropy (rs-CoEn), as a surrogate for neural and synaptic complexity, were extracted in 12 Stargardt macular dystrophy, 12 retinitis pigmentosa (tunnel vision stage), and 14 normally sighted subjects. When compared to controls, both groups with visual loss exhibited decreased CoTks in dorsal area V3d. Peripheral visual field loss also showed a specific CoTks decrease in early visual cortex and ventral area V4, while central visual field loss in dorsal area V3A. Only central visual field loss exhibited increased CoEn in LO-2 area and FG1. Current results revealed biomarkers of brain plasticity within the dorsal and the ventral visual streams following central and peripheral visual field defects.

Bilateral Pathways from the Basal Forebrain to Sensory Cortices May Contribute to Synchronous Sensory Processing

Sensory processing in the cortex should integrate inputs arriving from receptive fields located on both sides of the body. This role could be played by the corpus callosum through precise projections between both hemispheres. However, different studies suggest that cholinergic projections from the basal forebrain (BF) could also contribute to the synchronization and integration of cortical activities. Using tracer injections and optogenetic techniques in transgenic mice, we investigated whether the BF cells project bilaterally to sensory cortical areas, and have provided anatomical evidence to support a modulatory role for the cholinergic projections in sensory integration. Application of the retrograde tracer Fluor-Gold or Fast Blue in both hemispheres of the primary somatosensory (S1), auditory or visual cortical areas showed labeled neurons in the ipsi- and contralateral areas of the diagonal band of Broca and substantia innominata. The nucleus basalis magnocellularis only showed ipsilateral projections to the cortex. Optogenetic stimulation of the horizontal limb of the diagonal band of Broca facilitated whisker responses in the S1 cortex of both hemispheres through activation of muscarinic cholinergic receptors and this effect was diminished by atropine injection. In conclusion, our findings have revealed that specific areas of the BF project bilaterally to sensory cortices and may contribute to the coordination of neuronal activity on both hemispheres.

Attention Priority Map of Face Images in Human Early Visual Cortex

Attention priority maps are topographic representations that are used for attention selection and guidance of task-related behavior during visual processing. Previous studies have identified attention priority maps of simple artificial stimuli in multiple cortical and subcortical areas, but investigating neural correlates of priority maps of natural stimuli is complicated by the complexity of their spatial structure and the difficulty of behaviorally characterizing their priority map. To overcome these challenges, we reconstructed the topographic representations of upright/inverted face images from fMRI BOLD signals in human early visual areas primary visual cortex (V1) and the extrastriate cortex (V2 and V3) based on a voxelwise population receptive field model. We characterized the priority map behaviorally as the first saccadic eye movement pattern when subjects performed a face-matching task relative to the condition in which subjects performed a phase-scrambled face-matching task. We found that the differential first saccadic eye movement pattern between upright/inverted and scrambled faces could be predicted from the reconstructed topographic representations in V1-V3 in humans of either sex. The coupling between the reconstructed representation and the eye movement pattern increased from V1 to V2/3 for the upright faces, whereas no such effect was found for the inverted faces. Moreover, face inversion modulated the coupling in V2/3, but not in V1. Our findings provide new evidence for priority maps of natural stimuli in early visual areas and extend traditional attention priority map theories by revealing another critical factor that affects priority maps in extrastriate cortex in addition to physical salience and task goal relevance: image configuration.SIGNIFICANCE STATEMENT Prominent theories of attention posit that attention sampling of visual information is mediated by a series of interacting topographic representations of visual space known as attention priority maps. Until now, neural evidence of attention priority maps has been limited to studies involving simple artificial stimuli and much remains unknown about the neural correlates of priority maps of natural stimuli. Here, we show that attention priority maps of face stimuli could be found in primary visual cortex (V1) and the extrastriate cortex (V2 and V3). Moreover, representations in extrastriate visual areas are strongly modulated by image configuration. These findings extend our understanding of attention priority maps significantly by showing that they are modulated, not only by physical salience and task-goal relevance, but also by the configuration of stimuli images.

The channel capacity of visual awareness divided among multiple moving objects

If attention is distributed among multiple moving objects, how does this divided attention affect the temporal process for detecting a specific target motion? Well-trained observers in three experiments monitored ongoing random motions of multiple objects, trying to rapidly detect non-random target motions. Response time hazard rates revealed a simple lawful structure of the detection processes. Target detection rates (hazard rates, in bits /s) were inversely proportional to the number of observed objects. Detection rates at any response time and in any condition equaled a product of two parallel (functionally independent and concurrent) visual processes: visual awareness and motion integration. The rate of visual awareness was inversely proportional to Set Size (n = 1-12), constant over time, and invariant with integrated motion information. Thus, a single rate parameter, indicating a constant channel capacity of visual awareness, described detection rates over a wide range of conditions and response times. During an initial interval of roughly 0.5 s, detection rates increased proportionally with the duration and length of motion; but after this initial integration, detection rates were constant, independent of the time the target remained undetected. The relationship between the quantity of visual information and detection rates was simpler than anticipated by contemporary theories of attention, perception, and performance.

A view not to be missed: Salient scene content interferes with cognitive restoration

Attention Restoration Theory (ART) states that built scenes place greater load on attentional resources than natural scenes. This is explained in terms of "hard" and "soft" fascination of built and natural scenes. Given a lack of direct empirical evidence for this assumption we propose that perceptual saliency of scene content can function as an empirically derived indicator of fascination. Saliency levels were established by measuring speed of scene category detection using a Go/No-Go detection paradigm. Experiment 1 shows that built scenes are more salient than natural scenes. Experiment 2 replicates these findings using greyscale images, ruling out a colour-based response strategy, and additionally shows that built objects in natural scenes affect saliency to a greater extent than the reverse. Experiment 3 demonstrates that the saliency of scene content is directly linked to cognitive restoration using an established restoration paradigm. Overall, these findings demonstrate an important link between the saliency of scene content and related cognitive restoration.

Spatially tuned normalization explains attention modulation variance within neurons

Spatial attention improves perception of attended parts of a scene, a behavioral enhancement accompanied by modulations of neuronal firing rates. These modulations vary in size across neurons in the same brain area. Models of normalization explain much of this variance in attention modulation with differences in tuned normalization across neurons (Lee J, Maunsell JHR. PLoS One 4: e4651, 2009; Ni AM, Ray S, Maunsell JHR. Neuron 73: 803-813, 2012). However, recent studies suggest that normalization tuning varies with spatial location both across and within neurons (Ruff DA, Alberts JJ, Cohen MR. J Neurophysiol 116: 1375-1386, 2016; Verhoef BE, Maunsell JHR. eLife 5: e17256, 2016). Here we show directly that attention modulation and normalization tuning do in fact covary within individual neurons, in addition to across neurons as previously demonstrated. We recorded the activity of isolated neurons in the middle temporal area of two rhesus monkeys as they performed a change-detection task that controlled the focus of spatial attention. Using the same two drifting Gabor stimuli and the same two receptive field locations for each neuron, we found that switching which stimulus was presented at which location affected both attention modulation and normalization in a correlated way within neurons. We present an equal-maximum-suppression spatially tuned normalization model that explains this covariance both across and within neurons: each stimulus generates equally strong suppression of its own excitatory drive, but its suppression of distant stimuli is typically less. This new model specifies how the tuned normalization associated with each stimulus location varies across space both within and across neurons, changing our understanding of the normalization mechanism and how attention modulations depend on this mechanism.NEW & NOTEWORTHY Tuned normalization studies have demonstrated that the variance in attention modulation size seen across neurons from the same cortical area can be largely explained by between-neuron differences in normalization strength. Here we demonstrate that attention modulation size varies within neurons as well and that this variance is largely explained by within-neuron differences in normalization strength. We provide a new spatially tuned normalization model that explains this broad range of observed normalization and attention effects.

Temporally evolving gain mechanisms of attention in macaque area V4

Cognitive attention and perceptual saliency jointly govern our interaction with the environment. Yet, we still lack a universally accepted account of the interplay between attention and luminance contrast, a fundamental dimension of saliency. We measured the attentional modulation of V4 neurons' contrast response functions (CRFs) in awake, behaving macaque monkeys and applied a new approach that emphasizes the temporal dynamics of cell responses. We found that attention modulates CRFs via different gain mechanisms during subsequent epochs of visually driven activity: an early contrast-gain, strongly dependent on prestimulus activity changes (baseline shift); a time-limited stimulus-dependent multiplicative modulation, reaching its maximal expression around 150 ms after stimulus onset; and a late resurgence of contrast-gain modulation. Attention produced comparable time-dependent attentional gain changes on cells heterogeneously coding contrast, supporting the notion that the same circuits mediate attention mechanisms in V4 regardless of the form of contrast selectivity expressed by the given neuron. Surprisingly, attention was also sometimes capable of inducing radical transformations in the shape of CRFs. These findings offer important insights into the mechanisms that underlie contrast coding and attention in primate visual cortex and a new perspective on their interplay, one in which time becomes a fundamental factor.NEW & NOTEWORTHY We offer an innovative perspective on the interplay between attention and luminance contrast in macaque area V4, one in which time becomes a fundamental factor. We place emphasis on the temporal dynamics of attentional effects, pioneering the notion that attention modulates contrast response functions of V4 neurons via the sequential engagement of distinct gain mechanisms. These findings advance understanding of attentional influences on visual processing and help reconcile divergent results in the literature.

Towards solving the hard problem of consciousness: The varieties of brain resonances and the conscious experiences that they support

The hard problem of consciousness is the problem of explaining how we experience qualia or phenomenal experiences, such as seeing, hearing, and feeling, and knowing what they are. To solve this problem, a theory of consciousness needs to link brain to mind by modeling how emergent properties of several brain mechanisms interacting together embody detailed properties of individual conscious psychological experiences. This article summarizes evidence that Adaptive Resonance Theory, or ART, accomplishes this goal. ART is a cognitive and neural theory of how advanced brains autonomously learn to attend, recognize, and predict objects and events in a changing world. ART has predicted that "all conscious states are resonant states" as part of its specification of mechanistic links between processes of consciousness, learning, expectation, attention, resonance, and synchrony. It hereby provides functional and mechanistic explanations of data ranging from individual spikes and their synchronization to the dynamics of conscious perceptual, cognitive, and cognitive-emotional experiences. ART has reached sufficient maturity to begin classifying the brain resonances that support conscious experiences of seeing, hearing, feeling, and knowing. Psychological and neurobiological data in both normal individuals and clinical patients are clarified by this classification. This analysis also explains why not all resonances become conscious, and why not all brain dynamics are resonant. The global organization of the brain into computationally complementary cortical processing streams (complementary computing), and the organization of the cerebral cortex into characteristic layers of cells (laminar computing), figure prominently in these explanations of conscious and unconscious processes. Alternative models of consciousness are also discussed.