The role of neural mechanisms of attention in solving the binding problem

Collective plasticity of binocular interactions in the adult visual system

Binocular visual plasticity can be initiated via either bottom-up or top-down mechanisms, but it is unknown if these two forms of adult plasticity can be independently combined. In seven participants with normal binocular vision, sensory eye dominance was assessed using a binocular rivalry task, before and after a period of monocular deprivation and with and without selective attention directed towards one eye. On each trial, participants reported the dominant monocular target and the inter-ocular contrast difference between the stimuli was systematically altered to obtain estimates of ocular dominance. We found that both monocular light- and pattern-deprivation shifted dominance in favour of the deprived eye. However, this shift was completely counteracted if the non-deprived eye's stimulus was selectively attended. These results reveal that shifts in ocular dominance, driven by bottom-up and top-down selection, appear to act independently to regulate the relative contrast gain between the two eyes.

A "Bandwidth" in cortical representations of multiple faces

The human ability to process multiple items simultaneously can be constrained by the extent to which those items are represented by distinct neural populations. In the current study, we used fMRI to investigate the cortical representation of multiple faces. We found that the addition of a second face to occupy both visual hemifields led to an increased response, whereas a further addition of faces within the same visual hemifield resulted in a decreased response. This pattern was widely observed in the occipital visual cortex, the intraparietal sulcus, and extended to the posterior inferotemporal cortex. A parallel trend was found in a behavioral change-detection task, revealing a perceptual "bandwidth" of multiface processing. The sensitivity to face clutter gradually decreased along the ventral pathway, supporting the notion of a buildup of clutter-tolerance representation. These cortical response patterns to face clutters suggest that adding signals with nonoverlapping cortical representation enhanced perception, while adding signals that competed for representation resources impaired perception.

Dynamic neural reconstructions of attended object location and features using EEG

Attention allows us to select relevant and ignore irrelevant information from our complex environments. What happens when attention shifts from one item to another? To answer this question, it is critical to have tools that accurately recover neural representations of both feature and location information with high temporal resolution. In the present study, we used human electroencephalography (EEG) and machine learning to explore how neural representations of object features and locations update across dynamic shifts of attention. We demonstrate that EEG can be used to create simultaneous time courses of neural representations of attended features (time point-by-time point inverted encoding model reconstructions) and attended location (time point-by-time point decoding) during both stable periods and across dynamic shifts of attention. Each trial presented two oriented gratings that flickered at the same frequency but had different orientations; participants were cued to attend one of them and on half of trials received a shift cue midtrial. We trained models on a stable period from Hold attention trials and then reconstructed/decoded the attended orientation/location at each time point on Shift attention trials. Our results showed that both feature reconstruction and location decoding dynamically track the shift of attention and that there may be time points during the shifting of attention when 1) feature and location representations become uncoupled and 2) both the previously attended and currently attended orientations are represented with roughly equal strength. The results offer insight into our understanding of attentional shifts, and the noninvasive techniques developed in the present study lend themselves well to a wide variety of future applications.NEW & NOTEWORTHY We used human EEG and machine learning to reconstruct neural response profiles during dynamic shifts of attention. Specifically, we demonstrated that we could simultaneously read out both location and feature information from an attended item in a multistimulus display. Moreover, we examined how that readout evolves over time during the dynamic process of attentional shifts. These results provide insight into our understanding of attention, and this technique carries substantial potential for versatile extensions and applications.

Dynamic saccade context triggers more stable object-location binding

Our visual systems rapidly perceive and integrate information about object identities and locations. There is long-standing debate about how we achieve world-centered (spatiotopic) object representations across eye movements, with many studies reporting persistent retinotopic (eye-centered) effects even for higher-level object-location binding. But these studies are generally conducted in fairly static experimental contexts. Might spatiotopic object-location binding only emerge in more dynamic saccade contexts? In the present study, we investigated this using the Spatial Congruency Bias paradigm in healthy adults. In the static (single saccade) context, we found purely retinotopic binding, as before. However, robust spatiotopic binding emerged in the dynamic (multiple frequent saccades) context. We further isolated specific factors that modulate retinotopic and spatiotopic binding. Our results provide strong evidence that dynamic saccade context can trigger more stable object-location binding in ecologically-relevant spatiotopic coordinates, perhaps via a more flexible brain state which accommodates improved visual stability in the dynamic world.

Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize

When we look at an image, its features are represented in our visual system in a highly distributed manner, calling for a mechanism that binds them into coherent object representations. There have been different proposals for the neuronal mechanisms that can mediate binding. One hypothesis is that binding is achieved by oscillations that synchronize neurons representing features of the same perceptual object. This view allows separate communication channels between different brain areas. Another hypothesis is that binding of features that are represented in different brain regions occurs when the neurons in these areas that respond to the same object simultaneously enhance their firing rate, which would correspond to directing object-based attention to these features. This review summarizes evidence in favor of and against these two hypotheses, examining the neuronal correlates of binding and assessing the time course of perceptual grouping. I conclude that enhanced neuronal firing rates bind features into coherent object representations, whereas oscillations and synchrony are unrelated to binding.

The role of spatial attention in crowding and feature binding

Crowding refers to the failure to identify a peripheral object due to nearby objects (flankers). A hallmark of crowding is inner-outer asymmetry; that is, the outer flanker (more peripheral) produces stronger interference than the inner one. Here, by manipulating attention, we tested the predictions of two competing accounts: the attentional account, which predicts a positive attentional effect on the inner-outer asymmetry (i.e., attention to the outer flanker will increase asymmetry) and the receptive field size account, which predicts a negative attentional effect. In Experiment 1, observers estimated a Gabor target orientation. A peripheral pre-cue drew attention to one of three locations: target, inner flanker, or outer flanker. Probabilistic mixture modeling demonstrated asymmetry by showing that observers often misreported the outer-flanker orientation as the target. Interestingly, the outer cue led to a higher misreport rate of the outer flanker, and the inner cue led to a lower misreport rate of the outer flanker. Experiment 2 tested the effect of crowding and attention on incoherent object reports (i.e., binding errors, reporting the tilt of one presented item with the color of another item). In each trial, observers estimated both the tilt and color of the target. Attention merely increased coherent target reports, but not coherent flanker reports. The results suggest that the locus of spatial attention plays an essential role in crowding, as well as inner-outer asymmetry, and demonstrate that crowding and feature binding are closely related. However, our findings are inconsistent with the view that covert attention automatically binds features together.

Figure-ground responsive fields of monkey V4 neurons estimated from natural image patches

Neurons in visual area V4 modulate their responses depending on the figure-ground (FG) organization in natural images containing a variety of shapes and textures. To clarify whether the responses depend on the extents of the figure and ground regions in and around the classical receptive fields (CRFs) of the neurons, we estimated the spatial extent of local figure and ground regions that evoked FG-dependent responses (RF-FGs) in natural images and their variants. Specifically, we applied the framework of spike triggered averaging (STA) to the combinations of neural responses and human-marked segmentation images (FG labels) that represent the extents of the figure and ground regions in the corresponding natural image stimuli. FG labels were weighted by the spike counts in response to the corresponding stimuli and averaged over. The bias due to the nonuniformity of FG labels was compensated by subtracting the ensemble average of FG labels from the weighted average. Approximately 50% of the neurons showed effective RF-FGs, and a large number exhibited structures that were similar to those observed in virtual neurons with ideal FG-dependent responses. The structures of the RF-FGs exhibited a subregion responsive to a preferred side (figure or ground) around the CRF center and a subregion responsive to a non-preferred side in the surroundings. The extents of the subregions responsive to figure were smaller than those responsive to ground in agreement with the Gestalt rule. We also estimated RF-FG by an adaptive filtering (AF) method, which does not require spherical symmetry (whiteness) in stimuli. RF-FGs estimated by AF and STA exhibited similar structures, supporting the veridicality of the proposed STA. To estimate the contribution of nonlinear processing in addition to linear processing, we estimated nonlinear RF-FGs based on the framework of spike triggered covariance (STC). The analyses of the models based on STA and STC did not show inconsiderable contribution of nonlinearity, suggesting spatial variance of FG regions. The results lead to an understanding of the neural responses that underlie the segregation of figures and the construction of surfaces in intermediate-level visual areas.

Cognitive Neural Mechanism of Backward Inhibition and Deinhibition: A Review

Task switching is one of the typical paradigms to study cognitive control. When switching back to a recently inhibited task (e.g., “A” in an ABA sequence), the performance is often worse compared to a task without N-2 task repetitions (e.g., CBA). This difference is called the backward inhibitory effect (BI effect), which reflects the process of overcoming residual inhibition from a recently performed task (i.e., deinhibition). The neural mechanism of backward inhibition and deinhibition has received a lot of attention in the past decade. Multiple brain regions, including the frontal lobe, parietal, basal ganglia, and cerebellum, are activated during deinhibition. The event-related potentials (ERP) studies have shown that deinhibition process is reflected in the P1/N1 and P3 components, which might be related to early attention control, context updating, and response selection, respectively. Future research can use a variety of new paradigms to separate the neural mechanisms of BI and deinhibition.

Musicianship and melodic predictability enhance neural gain in auditory cortex during pitch deviance detection

When listening to music, pitch deviations are more salient and elicit stronger prediction error responses when the melodic context is predictable and when the listener is a musician. Yet, the neuronal dynamics and changes in connectivity underlying such effects remain unclear. Here, we employed dynamic causal modeling (DCM) to investigate whether the magnetic mismatch negativity response (MMNm)-and its modulation by context predictability and musical expertise-are associated with enhanced neural gain of auditory areas, as a plausible mechanism for encoding precision-weighted prediction errors. Using Bayesian model comparison, we asked whether models with intrinsic connections within primary auditory cortex (A1) and superior temporal gyrus (STG)-typically related to gain control-or extrinsic connections between A1 and STG-typically related to propagation of prediction and error signals-better explained magnetoencephalography responses. We found that, compared to regular sounds, out-of-tune pitch deviations were associated with lower intrinsic (inhibitory) connectivity in A1 and STG, and lower backward (inhibitory) connectivity from STG to A1, consistent with disinhibition and enhanced neural gain in these auditory areas. More predictable melodies were associated with disinhibition in right A1, while musicianship was associated with disinhibition in left A1 and reduced connectivity from STG to left A1. These results indicate that musicianship and melodic predictability, as well as pitch deviations themselves, enhance neural gain in auditory cortex during deviance detection. Our findings are consistent with predictive processing theories suggesting that precise and informative error signals are selected by the brain for subsequent hierarchical processing.

The Global Configuration of Visual Stimuli Alters Co-Fluctuations of Cross-Hemispheric Human Brain Activity

We tested how a stimulus gestalt, defined by the neuronal interaction between local and global features of a stimulus, is represented within human primary visual cortex (V1). We used high-resolution fMRI, which serves as a surrogate of neuronal activation, to measure co-fluctuations within subregions of V1 as (male and female) subjects were presented with peripheral stimuli, each with different global configurations. We found stronger cross-hemisphere correlations when fine-scale V1 cortical subregions represented parts of the same object compared with different objects. This result was consistent with the vertical bias in global processing and, critically, was independent of the task and local discontinuities within objects. Thus, despite the relatively small receptive fields of neurons within V1, global stimulus configuration affects neuronal processing via correlated fluctuations between regions that represent different sectors of the visual field.SIGNIFICANCE STATEMENT We provide the first evidence for the impact of global stimulus configuration on cross-hemispheric fMRI fluctuations, measured in human primary visual cortex. Our results are consistent with changes in the level of γ-band synchrony, which has been shown to be affected by global stimulus configuration, being reflected in the level fMRI co-fluctuations. These data help narrow the gap between knowledge of global stimulus configuration encoding at the single-neuron level versus at the behavioral level.

Capturing the objects of vision with neural networks

Human visual perception carves a scene at its physical joints, decomposing the world into objects, which are selectively attended, tracked and predicted as we engage our surroundings. Object representations emancipate perception from the sensory input, enabling us to keep in mind that which is out of sight and to use perceptual content as a basis for action and symbolic cognition. Human behavioural studies have documented how object representations emerge through grouping, amodal completion, proto-objects and object files. By contrast, deep neural network models of visual object recognition remain largely tethered to sensory input, despite achieving human-level performance at labelling objects. Here, we review related work in both fields and examine how these fields can help each other. The cognitive literature provides a starting point for the development of new experimental tasks that reveal mechanisms of human object perception and serve as benchmarks driving the development of deep neural network models that will put the object into object recognition.

Visual Remapping

Our visual system is fundamentally retinotopic. When viewing a stable scene, each eye movement shifts object features and locations on the retina. Thus, sensory representations must be updated, or remapped, across saccades to align presaccadic and postsaccadic inputs. The earliest remapping studies focused on anticipatory, presaccadic shifts of neuronal spatial receptive fields. Over time, it has become clear that there are multiple forms of remapping and that different forms of remapping may be mediated by different neural mechanisms. This review attempts to organize the various forms of remapping into a functional taxonomy based on experimental data and ongoing debates about forward versus convergent remapping, presaccadic versus postsaccadic remapping, and spatial versus attentional remapping. We integrate findings from primate neurophysiological, human neuroimaging and behavioral, and computational modeling studies. We conclude by discussing persistent open questions related to remapping, with specific attention to binding of spatial and featural information during remapping and speculations about remapping's functional significance. Expected final online publication date for the Annual Review of Vision Science, Volume 7 is September 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Pulvinar Modulates Contrast Responses in the Visual Cortex as a Function of Cortical Hierarchy

The pulvinar is the largest extrageniculate visual nucleus in mammals. Given its extensive reciprocal connectivity with the visual cortex, it allows the cortico-thalamocortical transfer of visual information. Nonetheless, knowledge of the nature of the pulvinar inputs to the cortex remains elusive. We investigated the impact of silencing the pulvinar on the contrast response function of neurons in 2 distinct hierarchical cortical areas in the cat (areas 17 and 21a). Pulvinar inactivation altered the response gain in both areas, but with larger changes observed in area 21a. A theoretical model was proposed, simulating the pulvinar contribution to cortical contrast responses by modifying the excitation-inhibition balanced state of neurons across the cortical hierarchy. Our experimental and theoretical data showed that the pulvinar exerts a greater modulatory influence on neuronal activity in area 21a than in the primary visual cortex, indicating that the pulvinar impact on cortical visual neurons varies along the cortical hierarchy.

On the control of attentional processes in vision

The study of attentional processing in vision has a long and deep history. Recently, several papers have presented insightful perspectives into how the coordination of multiple attentional functions in the brain might occur. These begin with experimental observations and the authors propose structures, processes, and computations that might explain those observations. Here, we consider a perspective that past works have not, as a complementary approach to the experimentally-grounded ones. We approach the same problem as past authors but from the other end of the computational spectrum, from the problem nature, as Marr's Computational Level would prescribe. What problem must the brain solve when orchestrating attentional processes in order to successfully complete one of the myriad possible visuospatial tasks at which we as humans excel? The hope, of course, is for the approaches to eventually meet and thus form a complete theory, but this is likely not soon. We make the first steps towards this by addressing the necessity of attentional control, examining the breadth and computational difficulty of the visuospatial and attentional tasks seen in human behavior, and suggesting a sketch of how attentional control might arise in the brain. The key conclusions of this paper are that an executive controller is necessary for human attentional function in vision, and that there is a 'first principles' computational approach to its understanding that is complementary to the previous approaches that focus on modelling or learning from experimental observations directly.

A neural correlate of visual feature binding in primate lateral prefrontal cortex

We effortlessly perceive visual objects as unified entities, despite the preferential encoding of their various visual features in separate cortical areas. A 'binding' process is assumed to be required for creating this unified percept, but the underlying neural mechanism and specific brain areas are poorly understood. We investigated 'feature-binding' across two feature dimensions, using a novel stimulus configuration, designed to disambiguate whether a given combination of color and motion direction is perceived as bound or unbound. In the "bound" condition, two behaviorally relevant features (color and motion) belong to the same object, while in the "unbound" condition they belong to different objects. We recorded local field potentials from the lateral prefrontal cortex (lPFC) in macaque monkeys that actively monitored the different stimulus configurations. Our data show a neural representation of visual feature binding especially in the 4-12 Hz frequency band and a transmission of binding information between different lPFC neural subpopulations. This information is linked to the animal's reaction time, suggesting a behavioral relevance of the binding information. Together, our results document the involvement of the prefrontal cortex, targeted by the dorsal and ventral visual streams, in binding visual features from different dimensions, in a process that includes a dynamic modulation of low frequency inter-regional communication.

Vision, attention, and driving

Safe driving demands the coordination of multiple sensory and cognitive functions, such as vision and attention. Patients with neurologic or ophthalmic disease are exposed to selective pathophysiologic insults to driving-critical systems, placing them at a higher risk for unsafe driving and restricted driving privileges. Here, we evaluate how vision and attention contribute to unsafe driving across different patient populations. In ophthalmic disease, we focus on macular degeneration, glaucoma, diabetic retinopathy, and cataract; in neurologic disease, we focus on Alzheimer's disease, Parkinson's disease, and multiple sclerosis. Unsafe driving is generally associated with impaired vision and attention in ophthalmic and neurologic patients, respectively. Furthermore, patients with ophthalmic disease experience some degree of impairment in attention. Similarly, patients with neurologic disease experience some degree of impairment in vision. While numerous studies have demonstrated a relationship between impaired vision and unsafe driving in neurologic disease, there remains a dearth of knowledge regarding the relationship between impaired attention and unsafe driving in ophthalmic disease. In summary, this chapter confirms-and offers opportunities for future research into-the contribution of vision and attention to safe driving.

Voluntary and Involuntary Attention in Bistable Visual Perception: A MEG Study

In this study, voluntary and involuntary visual attention focused on different interpretations of a bistable image, were investigated using magnetoencephalography (MEG). A Necker cube with sinusoidally modulated pixels' intensity in the front and rear faces with frequencies 6.67 Hz (60/9) and 8.57 Hz (60/7), respectively, was presented to 12 healthy volunteers, who interpreted the cube as either left- or right-oriented. The tags of these frequencies and their second harmonics were identified in the average Fourier spectra of the MEG data recorded from the visual cortex. In the first part of the experiment, the subjects were asked to voluntarily control their attention by interpreting the cube orientation as either being on the left or right. Accordingly, we observed the dominance of the corresponding spectral component, and voluntary attention performance was measured. In the second part of the experiment, the subjects were asked to focus their gaze on a red marker at the center of the cube image without putting forth effort in its interpretation. The alternation of the dominant spectral energies at the second harmonics of the stimulation frequencies was treated as changes in the cube orientation. Based on the results of the first experimental stage and using a wavelet analysis, we developed a method which allowed us to identify the currently perceived cube orientation. Finally, we characterized involuntary attention using the distribution of dominance times when focusing attention on one of the cube orientations, which was related to voluntary attention performance and brain noise. In particular, we confirmed our hypothesis that higher attention performance is associated with stronger brain noise.

The Binding Problem after an eye movement

Spatial attention is thought to be the "glue" that binds features together (e.g., Treisman & Gelade, 1980, Psychology, 12[1], 97-136)-but attention is dynamic, constantly moving across multiple goals and locations. For example, when a person moves her eyes, visual inputs that are coded relative to the eyes (retinotopic) must be rapidly updated to maintain stable world-centered (spatiotopic) representations. Here, we examined how dynamic updating of spatial attention after a saccadic eye movement affects object-feature binding. Immediately after a saccade, participants were simultaneously presented with four colored and oriented bars (one at a precued spatiotopic target location) and instructed to reproduce both the color and orientation of the target item. Object-feature binding was assessed by applying probabilistic mixture models to the joint distribution of feature errors: feature reports for the target item could be correlated (and thus bound together) or independent. We found that compared with holding attention without an eye movement, attentional updating after an eye movement produced more independent errors, including illusory conjunctions, in which one feature of the item at the spatiotopic target location was misbound with the other feature of the item at the initial retinotopic location. These findings suggest that even when only one spatiotopic location is task relevant, spatial attention-and thus object-feature binding-is malleable across and after eye movements, heightening the challenge that eye movements pose for the binding problem and for visual stability.

A Self-Operating Time Crystal Model of the Human Brain: Can We Replace Entire Brain Hardware with a 3D Fractal Architecture of Clocks Alone?

Time crystal was conceived in the 1970s as an autonomous engine made of only clocks to explain the life-like features of a virus. Later, time crystal was extended to living cells like neurons. The brain controls most biological clocks that regenerate the living cells continuously. Most cognitive tasks and learning in the brain run by periodic clock-like oscillations. Can we integrate all cognitive tasks in terms of running clocks of the hardware? Since the existing concept of time crystal has only one clock with a singularity point, we generalize the basic idea of time crystal so that we could bond many clocks in a 3D architecture. Harvesting inside phase singularity is the key. Since clocks reset continuously in the brain–body system, during reset, other clocks take over. So, we insert clock architecture inside singularity resembling brain components bottom-up and top-down. Instead of one clock, the time crystal turns to a composite, so it is poly-time crystal. We used century-old research on brain rhythms to compile the first hardware-free pure clock reconstruction of the human brain. Similar to the global effort on connectome, a spatial reconstruction of the brain, we advocate a global effort for more intricate mapping of all brain clocks, to fill missing links with respect to the brain’s temporal map. Once made, reverse engineering the brain would remain a mere engineering challenge.

An Indexing Theory for Working Memory Based on Fast Hebbian Plasticity

Working memory (WM) is a key component of human memory and cognition. Computational models have been used to study the underlying neural mechanisms, but neglected the important role of short-term memory (STM) and long-term memory (LTM) interactions for WM. Here, we investigate these using a novel multiarea spiking neural network model of prefrontal cortex (PFC) and two parietotemporal cortical areas based on macaque data. We propose a WM indexing theory that explains how PFC could associate, maintain, and update multimodal LTM representations. Our simulations demonstrate how simultaneous, brief multimodal memory cues could build a temporary joint memory representation as an “index” in PFC by means of fast Hebbian synaptic plasticity. This index can then reactivate spontaneously and thereby also the associated LTM representations. Cueing one LTM item rapidly pattern completes the associated uncued item via PFC. The PFC–STM network updates flexibly as new stimuli arrive, thereby gradually overwriting older representations.

Spatial congruency bias in identifying objects is triggered by retinal position congruence: Examination using the Ternus-Pikler illusion

When two different objects are sequentially presented at the same location, the viewer tends to misjudge them as identical (spatial congruency bias). The present study examined whether the spatial congruency bias would involve not only retinotopic but also non-retinotopic processing using the Ternus-Pikler illusion. In the experiments, two objects (central and peripheral) appeared in an initial frame. The target object was presented in the central area of the display, while the peripheral object was either on the left or right side of the target object. In the second frame, the target object was again presented in the central area, and the peripheral object was on the opposite side. Two kinds of inter-stimulus intervals were used. In the no-blank condition, the target object was perceived as stationary, and the peripheral object appeared to move to the opposite side. However, in the long-blank condition, the two objects were perceived to move together. Participants judged whether the target objects in the two frames were identical. As a result, the spatial congruency bias occurred irrespective of the ISI conditions. Our findings suggest that the spatial congruency bias is mainly based on retinotopic processing.

Attentional capture alters feature perception

We live in a dynamic, distracting world. When distracting information captures attention, what are the consequences for perception? Previous literature has focused on effects such as reaction time (RT) slowing, accuracy decrements, and oculomotor capture by distractors. In the current study, we asked whether attentional capture by distractors can also more fundamentally alter target feature representations, and if so, whether participants are aware of such errors. Using a continuous report task and novel confidence range report paradigm, we discovered 2 types of feature-binding errors when a distractor was presented along with the target: First, when attention is strongly captured by the distractor, participants commit swapping errors (misreporting the color at the distractor location instead of the target color), which remarkably seem to occur without awareness. Second, when participants successfully resist capture, they tend to exhibit repulsion (perceptual distortion away from the color at the distractor location). Thus, we found that capture not only induces a spatial shift of attention, it also alters feature perception in striking ways. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Analysis of Biased Competition and Cooperation for Attention in the Cerebral Cortex

A new approach to understanding the interaction between cortical areas is provided by a mathematical analysis of biased competition, which describes many interactions between cortical areas, including those involved in top-down attention. The analysis helps to elucidate the principles of operation of such cortical systems, and in particular the parameter values within which biased competition operates. The analytic results are supported by simulations that illustrate the operation of the system with parameters selected from the analysis. The findings provide a detailed mathematical analysis of the operation of these neural systems with nodes connected by feedforward (bottom-up) and feedback (top-down) connections. The analysis provides the critical value of the top-down attentional bias that enables biased competition to operate for a range of input values to the network, and derives this as a function of all the parameters in the model. The critical value of the top-down bias depends linearly on the value of the other inputs, but the coefficients in the function reveal non-linear relations between the remaining parameters. The results provide reasons why the backprojections should not be very much weaker than the forward connections between two cortical areas. The major advantage of the analytical approach is that it discloses relations between all the parameters of the model.

Neural mechanisms of attending to items in working memory

Working memory, the ability to keep recently accessed information available for immediate manipulation, has been proposed to rely on two mechanisms that appear difficult to reconcile: self-sustained neural firing, or the opposite-activity-silent synaptic traces. Here we review and contrast models of these two mechanisms, and then show that both phenomena can co-exist within a unified system in which neurons hold information in both activity and synapses. Rapid plasticity in flexibly-coding neurons allows features to be bound together into objects, with an important emergent property being the focus of attention. One memory item is held by persistent activity in an attended or "focused" state, and is thus remembered better than other items. Other, previously attended items can remain in memory but in the background, encoded in activity-silent synaptic traces. This dual functional architecture provides a unified common mechanism accounting for a diversity of perplexing attention and memory effects that have been hitherto difficult to explain in a single theoretical framework.

Object-Feature Binding Survives Dynamic Shifts of Spatial Attention

Visual object perception requires integration of multiple features; spatial attention is thought to be critical to this binding. But attention is rarely static-how does dynamic attention impact object integrity? Here, we manipulated covert spatial attention and had participants (total N = 48) reproduce multiple properties (color, orientation, location) of a target item. Object-feature binding was assessed by applying probabilistic models to the joint distribution of feature errors: Feature reports for the same object could be correlated (and thus bound together) or independent. We found that splitting attention across multiple locations degrades object integrity, whereas rapid shifts of spatial attention maintain bound objects. Moreover, we document a novel attentional phenomenon, wherein participants exhibit unintentional fluctuations- lapses of spatial attention-yet nevertheless preserve object integrity at the wrong location. These findings emphasize the importance of a single focus of spatial attention for object-feature binding, even when that focus is dynamically moving across the visual field.

Until the demise of the functional field of view

Hulleman & Olivers (H&O) make a much-needed stride forward for a better understanding of visual search behavior by rejecting theories based on discrete stimulus items. I propose that the framework could be further enhanced by clearly delineating distinct mechanisms for attention guidance, selection, and enhancement during visual search, instead of conflating them into a single functional field of view.

Toward an Integration of Deep Learning and Neuroscience

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.

Disambiguation, Binding, and the Unity of Visual Consciousness

Recent findings in neuroscience strongly suggest that an object's features (e.g., its color, texture, shape, etc.) are represented in separate areas of the visual cortex. Although represented in separate neuronal areas, somehow the feature representations are brought together as a single, unified object of visual consciousness. This raises a question of binding: how do neural activities in separate areas of the visual cortex function to produce a feature-unified object of visual consciousness? Several prominent neuroscientists have adopted neural synchrony and attention-based approaches to explain object feature binding. I argue that although neural synchrony and/or attentional mechanisms might function to disambiguate an object's features, it is difficult to see how either of these mechanisms could fully explain the unity of an object's features at the level of visual consciousness. After presenting a detailed critique of neural synchrony and attention-based approaches to object feature binding, I propose interactive hierarchical structuralism (IHS) . This view suggests that a unified percept (i.e., a feature-unified object of visual consciousness) is not reducible to the activity of any cognitive capacity or to any localized neural area, but emerges out of the interaction of visual information organized by spatial structuring capacities correlated with lower, higher, and intermediate levels of the visual hierarchy. After clarifying different notions of emergence and elaborating evidence for IHS, I discuss how IHS can be tested through transcranial magnetic stimulation and masking. In the final section I present some further implications/advantages of IHS.

Localized Attentional Interference Affects Object Individuation, Not Feature Detection

Modern theorists conceptualize visual selective attention as a competition between object representations for the control of extrastriate receptive fields, an account supported by the finding that attentional selection of one stimulus can degrade processing of nearby stimuli. In the present study the conditions that produce reciprocal interference between attended stimuli are examined. Each display contained either no, one, or two feature-defined target items among an array of homogeneous distractors. Observers performed two tasks, feature detection and object individuation. The feature-detection task required observers to determine if any targets were present within the display. The object-individuation task required observers to determine if the number of targets was exactly two. Spatially mediated interference between target pairs occurred in the object-individuation task, but had no effect on feature detection. Results suggest that localized interference between attended stimuli occurs only when observers are required to resolve the features of individual objects, consistent with the competitive interaction models of attention.

More Gamma More Predictions: Gamma-Synchronization as a Key Mechanism for Efficient Integration of Classical Receptive Field Inputs with Surround Predictions

During visual stimulation, neurons in visual cortex often exhibit rhythmic and synchronous firing in the gamma-frequency (30–90 Hz) band. Whether this phenomenon plays a functional role during visual processing is not fully clear and remains heavily debated. In this article, we explore the function of gamma-synchronization in the context of predictive and efficient coding theories. These theories hold that sensory neurons utilize the statistical regularities in the natural world in order to improve the efficiency of the neural code, and to optimize the inference of the stimulus causes of the sensory data. In visual cortex, this relies on the integration of classical receptive field (CRF) data with predictions from the surround. Here we outline two main hypotheses about gamma-synchronization in visual cortex. First, we hypothesize that the precision of gamma-synchronization reflects the extent to which CRF data can be accurately predicted by the surround. Second, we hypothesize that different cortical columns synchronize to the extent that they accurately predict each other’s CRF visual input. We argue that these two hypotheses can account for a large number of empirical observations made on the stimulus dependencies of gamma-synchronization. Furthermore, we show that they are consistent with the known laminar dependencies of gamma-synchronization and the spatial profile of intercolumnar gamma-synchronization, as well as the dependence of gamma-synchronization on experience and development. Based on our two main hypotheses, we outline two additional hypotheses. First, we hypothesize that the precision of gamma-synchronization shows, in general, a negative dependence on RF size. In support, we review evidence showing that gamma-synchronization decreases in strength along the visual hierarchy, and tends to be more prominent in species with small V1 RFs. Second, we hypothesize that gamma-synchronized network dynamics facilitate the emergence of spiking output that is particularly information-rich and sparse.

Attentional Effects on Phenomenological Appearance: How They Change with Task Instructions and Measurement Methods

It has been reported that exogenous cues accentuate contrast appearance. The empirical finding is controversial because non-veridical perception challenges the idea that attention prioritizes processing resources to make perception better, and because philosophers have used the finding to challenge representational accounts of mental experience. The present experiments confirm that when evaluated with comparison paradigms exogenous cues increase the apparent contrast. In addition, contrast appearance was also changed by simply changing the purpose of a secondary task. When comparison and discrimination reports were combined in a single experiment there was a behavioral disassociation: contrast enhanced for comparison responses, but did not change for discrimination judgments, even when participants made both types of judgment for a single stimulus. That a single object can have multiple simultaneous appearances leads inescapably to the conclusion that our unitary mental experience is illusory.

Waking State: Rapid Variations Modulate Neural and Behavioral Responses

The state of the brain and body constantly varies on rapid and slow timescales. These variations contribute to the apparent noisiness of sensory responses at both the neural and the behavioral level. Recent investigations of rapid state changes in awake, behaving animals have provided insight into the mechanisms by which optimal sensory encoding and behavioral performance are achieved. Fluctuations in state, as indexed by pupillometry, impact both the "signal" (sensory evoked response) and the "noise" (spontaneous activity) of cortical responses. By taking these fluctuations into account, neural response (co)variability is significantly reduced, revealing the brain to be more reliable and predictable than previously thought.

Neuroplasticity subserving the operation of brain-machine interfaces

Neuroplasticity is key to the operation of brain machine interfaces (BMIs)-a direct communication pathway between the brain and a man-made computing device. Whereas exogenous BMIs that associate volitional control of brain activity with neurofeedback have been shown to induce long lasting plasticity, endogenous BMIs that use prolonged activity-dependent stimulation--and thus may curtail the time scale that governs natural sensorimotor integration loops--have been shown to induce short lasting plasticity. Here we summarize recent findings from studies using both categories of BMIs, and discuss the fundamental principles that may underlie their operation and the longevity of the plasticity they induce. We draw comparison to plasticity mechanisms known to mediate natural sensorimotor skill learning and discuss principles of homeostatic regulation that may constrain endogenous BMI effects in the adult mammalian brain. We propose that BMIs could be designed to facilitate structural and functional plasticity for the purpose of re-organization of target brain regions and directed augmentation of sensorimotor maps, and suggest possible avenues for future work to maximize their efficacy and viability in clinical applications.

Visual Contrast Sensitivity Improvement by Right Frontal High-Beta Activity Is Mediated by Contrast Gain Mechanisms and Influenced by Fronto-Parietal White Matter Microstructure

Behavioral and electrophysiological studies in humans and non-human primates have correlated frontal high-beta activity with the orienting of endogenous attention and shown the ability of the latter function to modulate visual performance. We here combined rhythmic transcranial magnetic stimulation (TMS) and diffusion imaging to study the relation between frontal oscillatory activity and visual performance, and we associated these phenomena to a specific set of white matter pathways that in humans subtend attentional processes. High-beta rhythmic activity on the right frontal eye field (FEF) was induced with TMS and its causal effects on a contrast sensitivity function were recorded to explore its ability to improve visual detection performance across different stimulus contrast levels. Our results show that frequency-specific activity patterns engaged in the right FEF have the ability to induce a leftward shift of the psychometric function. This increase in visual performance across different levels of stimulus contrast is likely mediated by a contrast gain mechanism. Interestingly, microstructural measures of white matter connectivity suggest a strong implication of right fronto-parietal connectivity linking the FEF and the intraparietal sulcus in propagating high-beta rhythmic signals across brain networks and subtending top-down frontal influences on visual performance.

Intracranial spectral amplitude dynamics of perceptual suppression in fronto-insular, occipito-temporal, and primary visual cortex

If conscious perception requires global information integration across active distant brain networks, how does the loss of conscious perception affect neural processing in these distant networks? Pioneering studies on perceptual suppression (PS) described specific local neural network responses in primary visual cortex, thalamus and lateral prefrontal cortex of the macaque brain. Yet the neural effects of PS have rarely been studied with intracerebral recordings outside these cortices and simultaneously across distant brain areas. Here, we combined (1) a novel experimental paradigm in which we produced a similar perceptual disappearance and also re-appearance by using visual adaptation with transient contrast changes, with (2) electrophysiological observations from human intracranial electrodes sampling wide brain areas. We focused on broadband high-frequency (50–150 Hz, i.e., gamma) and low-frequency (8–24 Hz) neural activity amplitude modulations related to target visibility and invisibility. We report that low-frequency amplitude modulations reflected stimulus visibility in a larger ensemble of recording sites as compared to broadband gamma responses, across distinct brain regions including occipital, temporal and frontal cortices. Moreover, the dynamics of the broadband gamma response distinguished stimulus visibility from stimulus invisibility earlier in anterior insula and inferior frontal gyrus than in temporal regions, suggesting a possible role of fronto-insular cortices in top–down processing for conscious perception. Finally, we report that in primary visual cortex only low-frequency amplitude modulations correlated directly with perceptual status. Interestingly, in this sensory area broadband gamma was not modulated during PS but became positively modulated after 300 ms when stimuli were rendered visible again, suggesting that local networks could be ignited by top–down influences during conscious perception.

External distraction impairs categorization performance in older adults

The detrimental influence of distraction on memory and attention is well established, yet it is not as clear whether irrelevant information impacts categorization abilities and whether this impact changes in aging. We examined categorization with morphed prototype stimuli in both younger and older adults, using an adaptive staircase approach to assess participants' performance in conditions with and without visual distractors. Results showed that distraction did not affect younger adults, but produced a negative impact on older adults' categorization such that there was an interaction of age and distraction. These results suggest a relationship between the increased susceptibility to visual distraction in normal aging and impairment in categorization.

Gamma Oscillations Underlie the Maintenance of Feature-Specific Information and the Contents of Visual Working Memory

Visual working memory (VWM) sustains information online as integrated object representations. Neuronal mechanisms supporting the maintenance of feature-specific information have remained unidentified. Synchronized oscillations in the gamma band (30-120 Hz) characterize VWM retention and predict task performance, but whether these oscillations are specific to memorized features and VWM contents or underlie general executive VWM functions is not known. In the present study, we investigated whether gamma oscillations reflect the maintenance of feature-specific information in VWM. Concurrent magneto- and electroencephalography was recorded while subjects memorized different object features or feature conjunctions in identical VWM experiments. Using a data-driven source analysis approach, we show that the strength, load-dependence, and source topographies of gamma oscillations in the visual cortex differentiate these memorized features. Load-dependence of gamma oscillations in feature-specific visual and prefrontal areas also predicts VWM accuracy. Furthermore, corroborating the hypothesis that gamma oscillations support the perceptual binding of feature-specific neuronal assemblies, we also show that VWM for color-location conjunctions is associated with stronger gamma oscillations than that for these features separately. Gamma oscillations hence support the maintenance of feature-specific information and reflect VWM contents. The results also suggest that gamma oscillations contribute to feature binding in the formation of memory representations.

Attentional modulation and selection--an integrated approach

Various models of the neural mechanisms of attentional modulation in the visual cortex have been proposed. In general, these models assume that an ‘attention’ parameter is provided separately. Its value as well as the selection of neuron(s) to which it applies are assumed, but its source and the selection mechanism are unspecified. Here we show how the Selective Tuning model of visual attention can account for the modulation of the firing rate at the single neuron level, and for the temporal pattern of attentional modulations in the visual cortex, in a self-contained formulation that simultaneously determines the stimulus elements to be attended while modulating the relevant neural processes.

Processing multiple visual objects is limited by overlap in neural channels

High-level visual categories (e.g., faces, bodies, scenes, and objects) have separable neural representations across the visual cortex. Here, we show that this division of neural resources affects the ability to simultaneously process multiple items. In a behavioral task, we found that performance was superior when items were drawn from different categories (e.g., two faces/two scenes) compared to when items were drawn from one category (e.g., four faces). The magnitude of this mixed-category benefit depended on which stimulus categories were paired together (e.g., faces and scenes showed a greater behavioral benefit than objects and scenes). Using functional neuroimaging (i.e., functional MRI), we showed that the size of the mixed-category benefit was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. These results suggest that the ability to process multiple items at once is limited by the extent to which those items are represented by separate neural populations.

The binding problem

The brain processes information in a distributed manner so that features of the sensory input are detected at different sites and subsets of these features are integrated into objects. The notion of 'binding' refers to the corresponding integration process, leading to perception of these objects as entities, and 'the binding problem' either refers to the scientific challenge of identifying mechanisms that may achieve binding or to the difficulty that mind and brain may have with binding in certain situations. This review concentrates on binding of properties in visual perception, but other varieties of the binding problem are also mentioned. The binding problem is reviewed from psychological, neurobiological, and computational perspectives. For further resources related to this article, please visit the WIREs website.

CONFLICT OF INTEREST

The author has declared no conflicts of interest for this article.

Theta band activity in response to emotional expressions and its relationship with gamma band activity as revealed by MEG and advanced beamformer source imaging

Neuronal oscillations in the theta and gamma bands have been shown to be important for cognition. Here we examined the temporal and spatial relationship between the two frequency bands in emotional processing using magnetoencephalography and an advanced dynamic beamformer source imaging method called synthetic aperture magnetometry. We found that areas including the amygdala, visual and frontal cortex showed significant event-related synchronization in both bands, suggesting a functional association of neuronal oscillations in the same areas in the two bands. However, while the temporal profile in both bands was similar in the amygdala, the peak in gamma band power was much earlier within both visual and frontal areas. Our results do not support a traditional view that the localizations of lower and higher frequencies are spatially distinct. Instead, they suggest that in emotional processing, neuronal oscillations in the gamma and theta bands may reflect, at least in visual and frontal cortex either different but related functional processes or, perhaps more probably, different computational components of the same functional process.

Cortical potentials in an auditory oddball task reflect individual differences in working memory capacity

This study determined whether auditory cortical responses associated with mechanisms of attention vary with individual differences in working memory capacity (WMC) and perceptual load. The operation span test defined subjects with low versus high WMC, who then discriminated target/nontarget tones while EEG was recorded. Infrequent white noise distracters were presented at midline or ±90° locations, and perceptual load was manipulated by varying nontarget frequency. Amplitude of the N100 to distracters was negatively correlated with WMC. Relative to targets, only high WMC subjects showed attenuated N100 amplitudes to nontargets. In the higher WMC group, increased perceptual load was associated with decreased P3a amplitudes to distracters and longer-lasting negative slow wave to nontargets. Results show that auditory cortical processing is associated with multiple facets of attention related to WMC and possibly higher-level cognition.

Changing the mind? Not really-activity and connectivity in the caudate correlates with changes of choice

Changes in preference are inherently subjective and internal psychological events. We have identified brain events that presage ultimate (rather than intervening) choices, and signal the finality of a choice. At the first exposure to a pair of faces, caudate activity reflected the face of final choice, even if an initial choice was different. Furthermore, the orbitofrontal cortex and hippocampus exhibited correlations only when the subject had made a choice that would not change.