A soft handoff of attention between cerebral hemispheres

Dynamic neural reconstructions of attended object location and features using EEG

Attention allows us to select relevant and ignore irrelevant information from our complex environments. What happens when attention shifts from one item to another? To answer this question, it is critical to have tools that accurately recover neural representations of both feature and location information with high temporal resolution. In the present study, we used human electroencephalography (EEG) and machine learning to explore how neural representations of object features and locations update across dynamic shifts of attention. We demonstrate that EEG can be used to create simultaneous time courses of neural representations of attended features (time point-by-time point inverted encoding model reconstructions) and attended location (time point-by-time point decoding) during both stable periods and across dynamic shifts of attention. Each trial presented two oriented gratings that flickered at the same frequency but had different orientations; participants were cued to attend one of them and on half of trials received a shift cue midtrial. We trained models on a stable period from Hold attention trials and then reconstructed/decoded the attended orientation/location at each time point on Shift attention trials. Our results showed that both feature reconstruction and location decoding dynamically track the shift of attention and that there may be time points during the shifting of attention when 1) feature and location representations become uncoupled and 2) both the previously attended and currently attended orientations are represented with roughly equal strength. The results offer insight into our understanding of attentional shifts, and the noninvasive techniques developed in the present study lend themselves well to a wide variety of future applications.NEW & NOTEWORTHY We used human EEG and machine learning to reconstruct neural response profiles during dynamic shifts of attention. Specifically, we demonstrated that we could simultaneously read out both location and feature information from an attended item in a multistimulus display. Moreover, we examined how that readout evolves over time during the dynamic process of attentional shifts. These results provide insight into our understanding of attention, and this technique carries substantial potential for versatile extensions and applications.

On target selection as reflected by posterior ERP components in feature-guided visual search

The N2pc event-related potential is a widely studied ERP component that reflects the covert deployment of visuo-spatial attention to target stimuli displayed laterally relative to fixation. Recently, an analogous ERP component, named N2pcb, has been proposed as a marker of the deployment of visuo-spatial attention to targets displayed on the vertical midline. Two studies that investigated the N2pcb component found analogous results, using however two different algorithms to compute the amplitude of N2pcb. One study subtracted the ipsilateral activity elicited by a lateral target from the bilateral activity elicited by a target displayed on the vertical midline, whereas the other study subtracted the bilateral activity elicited by target-absent displays from the bilateral activity elicited by a target displayed on the vertical midline. Here we show both algorithms estimate properly the N2pc as well as the N2pcb components. In addition, we explored whether the singleton detection positivity (SDP) component, a posterior bilateral positivity temporally concomitant to N2pc recently reported in studies using singleton search, could be observed in the present study in which a target was defined by a combination of features. Given that such component was indeed found using feature search, we named this component posterior processing positivity (PPP), and showed that bilateral activity elicited by target-absent displays is an adequate baseline for its correct isolation.

A bilateral SPCN is elicited by to-be-memorized visual stimuli displayed along the vertical midline

We recently showed that deploying attention to target stimuli displayed along the vertical meridian elicits a bilateral N2pc, that we labeled N2pcb (Psychophysiology). Here we investigated whether a different component, the sustained posterior contralateral negativity (SPCN), shows the same property when a varying number of visual stimuli are displayed either laterally or on the vertical meridian. We displayed one or two cues that designated candidate targets to be detected in a search array that was displayed after a retention interval. The cues were either on the horizontal meridian or on the vertical meridian. When the cues were on the horizontal meridian, we observed an N2pc followed by an SPCN in their classic form, as negativity increments contralateral to the cues. As expected, SPCN amplitude was greater when two cues had to be memorized than when only one cue had to be memorized. When the cues were on the vertical meridian, we observed an N2pcb followed by a bilateral SPCN (or SPCNb). Critically, like SPCN, SPCNb amplitude was greater when two cues had to be memorized than when only one cue had to be memorized. A series of additional parametrical and topographical comparisons between N2pcb and SPCNb revealed similarities but also some important differences between these two components that we interpreted as evidence for their distinct neural sources.

We challenge the view that the SPCN ERP component cannot track the memory maintenance of objects displayed along the vertical meridian. Owing to the receptive fields of posterior neurons straddling on the intersection of the two visual hemifields, bilateral N2pc (N2pcb) and SPCN (SPCNb) activity can be detected using a cued visual search design.

Singularity and consciousness: A neuropsychological contribution

In common sense experience based on introspection, consciousness is singular. There is only one ‘me’ and that is the one that is conscious. This means that ‘singularity’ is a defining aspect of ‘consciousness’. However, the three main theories of consciousness, Integrated Information, Global Workspace and Recurrent Processing theory, are generally not very clear on this issue. These theories have traditionally relied heavily on neuropsychological observations and have interpreted various disorders, such as anosognosia, neglect and split‐brain as impairments in conscious awareness without any reference to ‘the singularity’. In this review, we will re‐examine the theoretical implications of these impairments in conscious awareness and propose a new way how to conceptualize consciousness of singularity. We will argue that the subjective feeling of singularity can coexist with several disunified conscious experiences. Singularity awareness may only come into existence due to environmental response constraints. That is, perceptual, language, memory, attentional and motor processes may largely proceed unintegrated in parallel, whereas a sense of unity only arises when organisms need to respond coherently constrained by the affordances of the environment. Next, we examine from this perspective psychiatric disorders and psycho‐active drugs. Finally, we present a first attempt to test this hypothesis with a resting state imaging experiment in a split‐brain patient. The results suggest that there is substantial coherence of activation across the two hemispheres. These data show that a complete lesioning of the corpus callosum does not, in general, alter the resting state networks of the brain. Thus, we propose that we have separate systems in the brain that generate distributed conscious. The sense of singularity, the experience of a ‘Me‐ness’, emerges in the interaction between the world and response‐planning systems, and this leads to coherent activation in the different functional networks across the cortex.

Unified Visual Working Memory without the Anterior Corpus Callosum

One of the most fundamental, and most studied, human cognitive functions is working memory. Yet, it is currently unknown how working memory is unified. In other words, why does a healthy human brain have one integrated capacity of working memory, rather than one capacity per visual hemifield, for instance. Thus, healthy subjects can memorize roughly as many items, regardless of whether all items are presented in one hemifield, rather than throughout two visual hemifields. In this current research, we investigated two patients in whom either most, or the entire, corpus callosum has been cut to alleviate otherwise untreatable epilepsy. Crucially, in both patients the anterior parts connecting the frontal and most of the parietal cortices, are entirely removed. This is essential, since it is often posited that working memory resides in these areas of the cortex. We found that despite the lack of direct connections between the frontal cortices in these patients, working memory capacity is similar regardless of whether stimuli are all presented in one visual hemifield or across two visual hemifields. This indicates that in the absence of the anterior parts of the corpus callosum working memory remains unified. Moreover, it is important to note that memory performance was not similar across visual fields. In fact, capacity was higher when items appeared in the left visual hemifield than when they appeared in the right visual hemifield. Visual information in the left hemifield is processed by the right hemisphere and vice versa. Therefore, this indicates that visual working memory is not symmetric, with the right hemisphere having a superior visual working memory. Nonetheless, a (subcortical) bottleneck apparently causes visual working memory to be integrated, such that capacity does not increase when items are presented in two, rather than one, visual hemifield.

Low-Level Visual Information Is Maintained across Saccades, Allowing for a Postsaccadic Handoff between Visual Areas

Experience seems continuous and detailed despite saccadic eye movements changing retinal input several times per second. There is debate whether neural signals related to updating across saccades contain information about stimulus features, or only location pointers without visual details. We investigated the time course of low-level visual information processing across saccades by decoding the spatial frequency of a stationary stimulus that changed from one visual hemifield to the other because of a horizontal saccadic eye movement. We recorded magnetoencephalography while human subjects (both sexes) monitored the orientation of a grating stimulus, making spatial frequency task irrelevant. Separate trials, in which subjects maintained fixation, were used to train a classifier, whose performance was then tested on saccade trials. Decoding performance showed that spatial frequency information of the presaccadic stimulus remained present for ∼200 ms after the saccade, transcending retinotopic specificity. Postsaccadic information ramped up rapidly after saccade offset. There was an overlap of over 100 ms during which decoding was significant from both presaccadic and postsaccadic processing areas. This suggests that the apparent richness of perception across saccades may be supported by the continuous availability of low-level information with a "soft handoff" of information during the initial processing sweep of the new fixation.SIGNIFICANCE STATEMENT Saccades create frequent discontinuities in visual input, yet perception appears stable and continuous. How is this discontinuous input processed resulting in visual stability? Previous studies have focused on presaccadic remapping. Here we examined the time course of processing of low-level visual information (spatial frequency) across saccades with magnetoencephalography. The results suggest that spatial frequency information is not predictively remapped but also is not discarded. Instead, they suggest a soft handoff over time between different visual areas, making this information continuously available across the saccade. Information about the presaccadic stimulus remains available, while the information about the postsaccadic stimulus has also become available. The simultaneous availability of both the presaccadic and postsaccadic information could enable rich and continuous perception across saccades.

Hemifield-specific control of spatial attention and working memory: Evidence from hemifield crossover costs

Attentional tracking and working memory tasks are often performed better when targets are divided evenly between the left and right visual hemifields, rather than contained within a single hemifield (Alvarez & Cavanagh, 2005; Delvenne, 2005). However, this bilateral field advantage does not provide conclusive evidence of hemifield-specific control of attention and working memory, because it can be explained solely from hemifield-limited spatial interference at early stages of visual processing. If control of attention and working memory is specific to each hemifield, maintaining target information should become more difficult as targets move between the two hemifields. Observers in the present study maintained targets that moved either within or between the left and right hemifields, using either attention (Experiment 1) or working memory (Experiment 2). Maintaining spatial information was more difficult when target items moved between the hemifields compared with when target items moved within their original hemifields, consistent with hemifield-specific control of spatial attention and working memory. However, this pattern was not found for maintaining identity information (e.g., color) in working memory (Experiment 3). Together, these results provide evidence that control of spatial attention and working memory is specific to each hemifield, and that hemifield-specific control is a unique signature of spatial processing.

Visual Search for Letters in the Right Versus Left Visual Hemifields

The current study investigated the relationships between attention, word processing, and visual field asymmetries. There is a discussion on whether each brain hemisphere possesses its own attentional resources and on how attention allocation depends on hemispheric lateralization of functions. We used stimuli with lateralized processing in an attentional task presented across the two visual hemifields. Three experiments investigated the visual search for a prespecified letter in displays containing words or nonwords, placed left and right to fixation, with a variable target letter position within the strings. In Experiment 1, two letter strings of the same type (words or nonwords) were presented to both visual hemifields. In Experiment 2, there was only one letter string presented right or left to fixation. In Experiment 3, two letter strings of different type were presented to both hemifields. Response times and accuracy data were collected. The results of Experiment 1 provide evidence for letter-by-letter search within a word in the left visual field (LVF), within a nonword in the right visual field (RVF), and for position-independent access to letters within a nonword in LVF and within a word in RVF. Experiment 3 produced similar results except for letter-by-letter search within words in RVF. In Experiment 2, for all types of letter strings in both hemifields, we observed the same letter-by-letter search. These results demonstrate that presence of stimuli in both one or two hemifields and the readiness to process a certain string type might contribute to the search for a letter within a letter string.

Split-Brain: What We Know Now and Why This is Important for Understanding Consciousness

Recently, the discussion regarding the consequences of cutting the corpus callosum (“split-brain”) has regained momentum (Corballis, Corballis, Berlucchi, & Marzi, Brain, 141(6), e46, 2018; Pinto et al., Brain, 140(5), 1231–1237, 2017a; Pinto, Lamme, & de Haan, Brain, 140(11), e68, 2017; Volz & Gazzaniga, Brain, 140(7), 2051–2060, 2017; Volz, Hillyard, Miller, & Gazzaniga, Brain, 141(3), e15, 2018). This collective review paper aims to summarize the empirical common ground, to delineate the different interpretations, and to identify the remaining questions. In short, callosotomy leads to a broad breakdown of functional integration ranging from perception to attention. However, the breakdown is not absolute as several processes, such as action control, seem to remain unified. Disagreement exists about the responsible mechanisms for this remaining unity. The main issue concerns the first-person perspective of a split-brain patient. Does a split-brain harbor a split consciousness or is consciousness unified? The current consensus is that the body of evidence is insufficient to answer this question, and different suggestions are made with respect to how future studies might address this paucity. In addition, it is suggested that the answers might not be a simple yes or no but that intermediate conceptualizations need to be considered.

Shifts of Spatial Attention in Visual and Tactile Working Memory are Controlled by Independent Modality-Specific Mechanisms

The question whether the attentional control of working memory (WM) is shared across sensory modalities remains controversial. Here, we investigated whether attention shifts in visual and tactile WM are regulated independently. Participants memorized visual and tactile targets in a first memory sample set (S1) before encoding targets in a second sample set (S2). Importantly, visual or tactile S2 targets could appear on the same side as the corresponding S1 targets, or on opposite sides, thus, requiring shifts of spatial attention in visual or tactile WM. The activation of WM representations in modality-specific visual and somatosensory areas was tracked by recording visual and tactile contralateral delay activity (CDA/tCDA). CDA/tCDA components emerged contralateral to the side of visual or tactile S1 targets, and reversed polarity when S2 targets in the same modality appeared on the opposite side. Critically, the visual CDA was unaffected by the presence versus absence of concurrent attention shifts in tactile WM, and the tactile CDA remained insensitive to visual attention shifts. Visual and tactile WM performance was also not modulated by attention shifts in the other modality. These results show that the dynamic control of visual and tactile WM activation processes operates in an independent modality-specific fashion.

Graded Neuronal Modulations Related to Visual Spatial Attention

Studies of visual attention in monkeys typically measure neuronal activity when the stimulus event to be detected occurs at a cued location versus when it occurs at an uncued location. But this approach does not address how neuronal activity changes relative to conditions where attention is unconstrained by cueing. Human psychophysical studies have used neutral cueing conditions and found that neutrally cued behavioral performance is generally intermediate to that of cued and uncued conditions (Posner et al., 1978; Mangun and Hillyard, 1990; Montagna et al., 2009). To determine whether the neuronal correlates of visual attention during neutral cueing are similarly intermediate, we trained macaque monkeys to detect changes in stimulus orientation that were more likely to occur at one location (cued) than another (uncued), or were equally likely to occur at either stimulus location (neutral). Consistent with human studies, performance was best when the location was cued, intermediate when both locations were neutrally cued, and worst when the location was uncued. Neuronal modulations in visual area V4 were also graded as a function of cue validity and behavioral performance. By recording from both hemispheres simultaneously, we investigated the possibility of switching attention between stimulus locations during neutral cueing. The results failed to support a unitary "spotlight" of attention. Overall, our findings indicate that attention-related changes in V4 are graded to accommodate task demands.

SIGNIFICANCE STATEMENT

Studies of the neuronal correlates of attention in monkeys typically use visual cues to manipulate where attention is focused ("cued" vs "uncued"). Human psychophysical studies often also include neutrally cued trials to study how attention naturally varies between points of interest. But the neuronal correlates of this neutral condition are unclear. We measured behavioral performance and neuronal activity in cued, uncued, and neutrally cued blocks of trials. Behavioral performance and neuronal responses during neutral cueing were intermediate to those of the cued and uncued conditions. We found no signatures of a single mechanism of attention that switches between stimulus locations. Thus, attention-related changes in neuronal activity are largely hemisphere-specific and graded according to task demands.

Covert enaction at work: Recording the continuous movements of visuospatial attention to visible or imagined targets by means of Steady-State Visual Evoked Potentials (SSVEPs)

Whereas overt visuospatial attention is customarily measured with eye tracking, covert attention is assessed by various methods. Here we exploited Steady-State Visual Evoked Potentials (SSVEPs) - the oscillatory responses of the visual cortex to incoming flickering stimuli - to record the movements of covert visuospatial attention in a way operatively similar to eye tracking (attention tracking), which allowed us to compare motion observation and motion extrapolation with and without eye movements. Observers fixated a central dot and covertly tracked a target oscillating horizontally and sinusoidally. In the background, the left and the right halves of the screen flickered at two different frequencies, generating two SSVEPs in occipital regions whose size varied reciprocally as observers attended to the moving target. The two signals were combined into a single quantity that was modulated at the target frequency in a quasi-sinusoidal way, often clearly visible in single trials. The modulation continued almost unchanged when the target was switched off and observers mentally extrapolated its motion in imagery, and also when observers pointed their finger at the moving target during covert tracking, or imagined doing so. The amplitude of modulation during covert tracking was ∼25-30% of that measured when observers followed the target with their eyes. We used 4 electrodes in parieto-occipital areas, but similar results were achieved with a single electrode in Oz. In a second experiment we tested ramp and step motion. During overt tracking, SSVEPs were remarkably accurate, showing both saccadic-like and smooth pursuit-like modulations of cortical responsiveness, although during covert tracking the modulation deteriorated. Covert tracking was better with sinusoidal motion than ramp motion, and better with moving targets than stationary ones. The clear modulation of cortical responsiveness recorded during both overt and covert tracking, identical for motion observation and motion extrapolation, suggests to include covert attention movements in enactive theories of mental imagery.