Isolating endogenous visuo-spatial attentional effects using the novel visual-evoked spread spectrum analysis (VESPA) technique

Stimulus-evoked and resting-state alpha oscillations show a linked dependence on patterned visual experience for development

Persistent visual impairments after congenital blindness due to dense bilateral cataracts have been attributed to altered visual cortex development within a sensitive period. Occipital alpha (8-14 Hz) oscillations were found to be reduced after congenital cataract reversal, while participants performed visual motion tasks. However, it has been unclear whether reduced alpha oscillations were task-specific, or linked to impaired visual behavior in cataract-reversed individuals. Here, we compared resting-state and stimulus-evoked alpha activity between individuals who had been treated for dense bilateral congenital cataracts (CC, n = 13, mean duration of blindness = 11.0 years) and age-matched, normally sighted individuals (SC, n = 13). We employed the visual impulse response function, adapted from VanRullen and MacDonald (2012), to test for the characteristic alpha response to visual white noise. Participants observed white noise stimuli changing in luminance with equal power at frequencies between 0 and 30 Hz. Compared to SC individuals, CC individuals demonstrated a reduced likelihood of exhibiting an evoked alpha response. Moreover, stimulus-evoked alpha power was reduced and correlated with a corresponding reduction of resting-state alpha power in the same CC individuals. Finally, CC individuals with an above-threshold evoked alpha peak had better visual acuity than CC individual without an evoked alpha peak. Since alpha oscillations have been linked to feedback communication, we suggest that the concurrent impairment in resting-state and stimulus-evoked alpha oscillations indicates an altered interaction of top-down and bottom-up processing in the visual hierarchy, which likely contributes to incomplete behavioral recovery in individuals who experienced transient congenital blindness.

Voluntary and Involuntary Attention in Bistable Visual Perception: A MEG Study

In this study, voluntary and involuntary visual attention focused on different interpretations of a bistable image, were investigated using magnetoencephalography (MEG). A Necker cube with sinusoidally modulated pixels' intensity in the front and rear faces with frequencies 6.67 Hz (60/9) and 8.57 Hz (60/7), respectively, was presented to 12 healthy volunteers, who interpreted the cube as either left- or right-oriented. The tags of these frequencies and their second harmonics were identified in the average Fourier spectra of the MEG data recorded from the visual cortex. In the first part of the experiment, the subjects were asked to voluntarily control their attention by interpreting the cube orientation as either being on the left or right. Accordingly, we observed the dominance of the corresponding spectral component, and voluntary attention performance was measured. In the second part of the experiment, the subjects were asked to focus their gaze on a red marker at the center of the cube image without putting forth effort in its interpretation. The alternation of the dominant spectral energies at the second harmonics of the stimulation frequencies was treated as changes in the cube orientation. Based on the results of the first experimental stage and using a wavelet analysis, we developed a method which allowed us to identify the currently perceived cube orientation. Finally, we characterized involuntary attention using the distribution of dominance times when focusing attention on one of the cube orientations, which was related to voluntary attention performance and brain noise. In particular, we confirmed our hypothesis that higher attention performance is associated with stronger brain noise.

The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli

Understanding how brains process sensory signals in natural environments is one of the key goals of twenty-first century neuroscience. While brain imaging and invasive electrophysiology will play key roles in this endeavor, there is also an important role to be played by noninvasive, macroscopic techniques with high temporal resolution such as electro- and magnetoencephalography. But challenges exist in determining how best to analyze such complex, time-varying neural responses to complex, time-varying and multivariate natural sensory stimuli. There has been a long history of applying system identification techniques to relate the firing activity of neurons to complex sensory stimuli and such techniques are now seeing increased application to EEG and MEG data. One particular example involves fitting a filter—often referred to as a temporal response function—that describes a mapping between some feature(s) of a sensory stimulus and the neural response. Here, we first briefly review the history of these system identification approaches and describe a specific technique for deriving temporal response functions known as regularized linear regression. We then introduce a new open-source toolbox for performing this analysis. We describe how it can be used to derive (multivariate) temporal response functions describing a mapping between stimulus and response in both directions. We also explain the importance of regularizing the analysis and how this regularization can be optimized for a particular dataset. We then outline specifically how the toolbox implements these analyses and provide several examples of the types of results that the toolbox can produce. Finally, we consider some of the limitations of the toolbox and opportunities for future development and application.

Spatial decoupling of targets and flashing stimuli for visual brain-computer interfaces

OBJECTIVE

Recently, paradigms using code-modulated visual evoked potentials (c-VEPs) have proven to achieve among the highest information transfer rates for noninvasive brain-computer interfaces (BCIs). One issue with current c-VEP paradigms, and visual-evoked paradigms in general, is that they require direct foveal fixation of the flashing stimuli. These interfaces are often visually unpleasant and can be irritating and fatiguing to the user, thus adversely impacting practical performance. In this study, a novel c-VEP BCI paradigm is presented that attempts to perform spatial decoupling of the targets and flashing stimuli using two distinct concepts: spatial separation and boundary positioning.

APPROACH

For the paradigm, the flashing stimuli form a ring that encompasses the intended non-flashing targets, which are spatially separated from the stimuli. The user fixates on the desired target, which is classified using the changes to the EEG induced by the flashing stimuli located in the non-foveal visual field. Additionally, a subset of targets is also positioned at or near the stimulus boundaries, which decouples targets from direct association with a single stimulus. This allows a greater number of target locations for a fixed number of flashing stimuli.

MAIN RESULTS

Results from 11 subjects showed practical classification accuracies for the non-foveal condition, with comparable performance to the direct-foveal condition for longer observation lengths. Online results from 5 subjects confirmed the offline results with an average accuracy across subjects of 95.6% for a 4-target condition. The offline analysis also indicated that targets positioned at or near the boundaries of two stimuli could be classified with the same accuracy as traditional superimposed (non-boundary) targets.

SIGNIFICANCE

The implications of this research are that c-VEPs can be detected and accurately classified to achieve comparable BCI performance without requiring potentially irritating direct foveation of flashing stimuli. Furthermore, this study shows that it is possible to increase the number of targets beyond the number of stimuli without degrading performance. Given the superior information transfer rate of c-VEP paradigms, these results can lead to the development of more practical and ergonomic BCIs.

Modulation of early cortical processing during divided attention to non-contiguous locations

We often face the challenge of simultaneously attending to multiple non-contiguous regions of space. There is ongoing debate as to how spatial attention is divided under these situations. Whereas, for several years, the predominant view was that humans could divide the attentional spotlight, several recent studies argue in favor of a unitary spotlight that rhythmically samples relevant locations. Here, this issue was addressed by the use of high-density electrophysiology in concert with the multifocal m-sequence technique to examine visual evoked responses to multiple simultaneous streams of stimulation. Concurrently, we assayed the topographic distribution of alpha-band oscillatory mechanisms, a measure of attentional suppression. Participants performed a difficult detection task that required simultaneous attention to two stimuli in contiguous (undivided) or non-contiguous parts of space. In the undivided condition, the classic pattern of attentional modulation was observed, with increased amplitude of the early visual evoked response and increased alpha amplitude ipsilateral to the attended hemifield. For the divided condition, early visual responses to attended stimuli were also enhanced, and the observed multifocal topographic distribution of alpha suppression was in line with the divided attention hypothesis. These results support the existence of divided attentional spotlights, providing evidence that the corresponding modulation occurs during initial sensory processing time-frames in hierarchically early visual regions, and that suppressive mechanisms of visual attention selectively target distracter locations during divided spatial attention.

Comparing linear and quadratic models of the human auditory system using EEG

Recent studies have highlighted the importance of system identification as an approach for assessing sensory processing in humans using electroencephalography (EEG). These studies typically use linear impulse response estimates of visual and, more recently, auditory function. These methods, which are known as the VESPA and AESPA (Visual/Auditory Evoked Spread Spectrum Analysis) respectively, have been found to be useful for studying sensory processing in both healthy populations and clinical groups and for studying the effects of cognition on sensory processing. While a nonlinear extension of the VESPA has been previously described, no such extension has yet been examined for the AESPA. This paper investigates such an extension and quantifies the relative contribution of linear and quadratic processes to the EEG in response to novel auditory stimuli. While the ability to accurately predict novel EEG is poor, it is highly significant, with a slightly, but again significantly, greater ability to predict using a quadratic model (r=0.0418) over a linear model (r=0.0361).

Generation of the VESPA response to rapid contrast fluctuations is dominated by striate cortex: evidence from retinotopic mapping

The VESPA (visual-evoked spread spectrum analysis) method derives an impulse response function of the visual system from scalp electroencephalographic (EEG) data using the controlled modulation of some feature of a visual stimulus. Recent research using VESPA responses to modulations of stimulus contrast has provided new insights into both early visual attention mechanisms and the specificity of visual-processing deficits in schizophrenia. To allow a fuller interpretation of these and future findings, it is necessary to further characterize the VESPA in terms of its underlying cortical generators. To that end, we here examine spatio-temporal variations in the components of the VESPA as a function of stimulus location. We found that the first two VESPA components (C1/P1) each have a posterior dorsal midline focus and reverse in polarity across the horizontal meridian, consistent with retinotopic projections to calcarine cortex (V1) for the stimulus locations tested. Furthermore, the focal scalp topography of the VESPA was strikingly constant across the entire C1-P1 timeframe (50-120 ms) for each stimulus location, with negligible global scalp activity visible at the zero-crossing dividing the two. This indicates a common focal source underpinning both components, which was further supported by a significant correlation between C1 and P1 amplitudes across subjects (r=0.54; p<0.05). These results, along with factors implicit in the method of derivation of the contrast-VESPA, lead us to conclude that these responses are dominated by activity from striate cortex. We discuss the implications of this finding for previous and future research using the VESPA.

Isolating early cortical generators of visual-evoked activity: a systems identification approach

The VESPA (visual-evoked spread spectrum analysis) method estimates the impulse response of the visual system using a continuously varying stimulus. It has been used recently to address both basic cognitive and neurophysiologic questions as well as those surrounding clinical populations. Although the components of the average VESPA response are highly reminiscent of the early components of the visual-evoked potential (VEP) when measured over midline occipital locations, the two responses are acquired in different ways and, thus, they cannot be regarded as being equivalent. To further characterize the relationship between the VESPA and the VEP and the generative mechanisms underlying them, we recorded EEG from 31 subjects in response to checkerboard-based VEP and VESPA stimuli. We found that, across subjects, the amplitudes of the VEP C1 component and the VESPA C1 component were highly correlated, whereas the VEP P1 and the VESPA P1 bore no statistical relationship. Furthermore, we found that C1 and P1 amplitudes were significantly correlated in the VESPA but not in the VEP. We believe these findings point to the presence of common generators underlying the VESPA C1 and the VEP C1. We argue further that the VESPA P1, in light of its strong relationship to the VESPA C1, likely reflects further activation of the same cortical generators. Given the lack of correlation between the VEP P1 and each of these three other components, it is likely that the underlying generators of this particular component are more varied and widespread, as suggested previously. We discuss the implications of these relationships for basic and clinical research using the VESPA and for the assessment of additive-evoked versus phase-reset contributions to the VEP.

The elicitation of audiovisual steady-state responses: multi-sensory signal congruity and phase effects

Most ecologically natural sensory inputs are not limited to a single modality. While it is possible to use real ecological materials as experimental stimuli to investigate the neural basis of multi-sensory experience, parametric control of such tokens is limited. By using artificial bimodal stimuli composed of approximations to ecological signals, we aim to observe the interactions between putatively relevant stimulus attributes. Here we use MEG as an electrophysiological tool and employ as a measure the steady-state response (SSR), an experimental paradigm typically applied to unimodal signals. In this experiment we quantify the responses to a bimodal audio-visual signal with different degrees of temporal (phase) congruity, focusing on stimulus properties critical to audiovisual speech. An amplitude modulated auditory signal ('pseudo-speech') is paired with a radius-modulated ellipse ('pseudo-mouth'), with the envelope of low-frequency modulations occurring in phase or at offset phase values across modalities. We observe (i) that it is possible to elicit an SSR to bimodal signals; (ii) that bimodal signals exhibit greater response power than unimodal signals; and (iii) that the SSR power at specific harmonics and sensors differentially reflects the congruity between signal components. Importantly, we argue that effects found at the modulation frequency and second harmonic reflect differential aspects of neural coding of multisensory signals. The experimental paradigm facilitates a quantitative characterization of properties of multi-sensory speech and other bimodal computations.

Prepare for conflict: EEG correlates of the anticipation of target competition during overt and covert shifts of visual attention

When preparing to make a saccadic eye movement in a cued direction, perception of stimuli at the target location is enhanced, just as it is when attention is covertly deployed there. Accordingly, the timing and anatomical sources of preparatory brain activity accompanying shifts of covert attention and saccade preparation tend to exhibit a large degree of overlap. However, there is evidence that preparatory processes are modulated by the foreknowledge of visual distractor competition during covert attention, and it is unknown whether eye movement preparation undergoes equivalent modulation. Here we examine preparatory processes in the electroencephalogram of human participants during four blocked versions of a spatial cueing task, requiring either covert detection or saccade execution, and either containing a distractor or not. As in previous work, a typical pattern of spatially selective occipital, parietal and frontal activity was seen in all task versions. However, whereas distractor presence called on an enhancement of spatially selective visual cortical modulation during covert attention, it instead called on increased activity over frontomedial oculomotor areas in the case of overt saccade preparation. We conclude that, although advance orienting signals may be similar in character during overt and covert conditions, the pattern by which these signals are modulated to ameliorate the behavioral costs of distractor competition is highly distinct, pointing to a degree of separability between the overt and covert systems.

Early spatial attentional modulation of inputs to the fovea

Attending to a specific spatial location modulates responsivity of neurons with receptive fields processing that part of the environment. A major outstanding question is whether attentional modulation operates differently for the foveal (central) representation of the visual field than it does for the periphery. Indeed, recent animal electrophysiological recordings suggest that attention differentially affects spatial integration for central and peripheral receptive fields in primary visual cortex. In human electroencephalographic recordings, spatial attention to peripheral locations robustly modulates activity in early visual regions, but it has been claimed that this mechanism does not operate in foveal vision. Here, however, we show clear early attentional modulation of foveal stimulation with the same timing and cortical sources as seen for peripheral stimuli, demonstrating that attentional gain control operates similarly across the entire field of view. These results imply that covertly attending away from the center of gaze, which is a common paradigm in behavioral and electrophysiological studies of attention, results in a precisely timed push-pull mechanism. While the amplitude of the initial response to stimulation at attended peripheral locations is significantly increased beginning at 80 ms, the amplitude of the response to foveal stimulation begins to be attenuated.

Neural responses to uninterrupted natural speech can be extracted with precise temporal resolution

The human auditory system has evolved to efficiently process individual streams of speech. However, obtaining temporally detailed responses to distinct continuous natural speech streams has hitherto been impracticable using standard neurophysiological techniques. Here a method is described which provides for the estimation of a temporally precise electrophysiological response to uninterrupted natural speech. We have termed this response AESPA (Auditory Evoked Spread Spectrum Analysis) and it represents an estimate of the impulse response of the auditory system. It is obtained by assuming that the recorded electrophysiological function represents a convolution of the amplitude envelope of a continuous speech stream with the to-be-estimated impulse response. We present examples of these responses using both scalp and intracranially recorded human EEG, which were obtained while subjects listened to a binaurally presented recording of a male speaker reading naturally from a classic work of fiction. This method expands the arsenal of stimulation types that can now be effectively used to derive auditory evoked responses and allows for the use of considerably more ecologically valid stimulation parameters. Some implications for future research efforts are presented.

Resolving Precise Temporal Processing Properties of the Auditory System Using Continuous Stimuli

In natural environments complex and continuous auditory stimulation is virtually ubiquitous. The human auditory system has evolved to efficiently process an infinitude of everyday sounds, which range from short, simple bursts of noise to signals with a much higher order of information such as speech. Investigation of temporal processing in this system using the event-related potential (ERP) technique has led to great advances in our knowledge. However, this method is restricted by the need to present simple, discrete, repeated stimuli to obtain a useful response. Alternatively the continuous auditory steady-state response is used, although this method reduces the evoked response to its fundamental frequency component at the expense of useful information on the timing of response transmission through the auditory system. In this report, we describe a method for eliciting a novel ERP, which circumvents these limitations, known as the AESPA (auditory-evoked spread spectrum analysis). This method uses rapid amplitude modulation of audio carrier signals to estimate the impulse response of the auditory system. We show AESPA responses with high signal-to-noise ratios obtained using two types of carrier wave: a 1-kHz tone and broadband noise. To characterize these responses, they are compared with auditory-evoked potentials elicited using standard techniques. A number of similarities and differences between the responses are noted and these are discussed in light of the differing stimulation and analysis methods used. Data are presented that demonstrate the generalizability of the AESPA method and a number of applications are proposed.

Visual sensory processing deficits in Schizophrenia and their relationship to disease state

CONTEXT

Visual Evoked Potential (VEP) abnormalities have been a fairly consistent finding in patients with schizophrenia, and it has been suggested that electrophysiological markers of early sensory processing may be useful as trait markers for the illness, and for development as potential diagnostic measures.

OBJECTIVE

Clear amplitude reductions in the occipital P1 component of the VEP (approximately 100 ms), have been repeatedly demonstrated in patients with schizophrenia. Here, we investigated whether the extent of this deficit was related to age, clinical symptoms, medication status and length of illness, in a large cohort of ethnically homogenous patients.

DESIGN, SETTING AND PARTICIPANTS

VEP responses to simple isolated-check stimuli were examined in 52 DSM-IV diagnosed patients with schizophrenia, and compared with responses from 26 healthy age-matched control subjects. Using high-density electrical scalp recordings, we assessed the integrity of the visual P1 component across the two groups. This study was conducted at St.Vincent's Psychiatric Hospital in Fairview, Dublin, Ireland.

RESULTS

Substantially reduced P1 amplitude was demonstrated in the patient group compared to controls. The deficit was not linked to age, length of illness or medication status. A small positive correlation, accounting for about 11% of the variance, was found between P1 amplitude and clinical symptoms scales (BPRS and SANS). In addition, we found that a slightly later (~110 ms) fronto-central component was relatively increased in the patient group, and was inversely correlated with the occipital P1 amplitude in the patients, but not in the healthy control subjects.

CONCLUSIONS

Our findings clearly demonstrate a deficit in early visual processing in patients with schizophrenia (with a large effect size; Cohen's d = 0.7) that is unrelated to chronicity. The results are consistent with recent findings showing that the P1 deficit is endophenotypic of the disorder and related to genetic risk factors rather than the disease process itself.