Electrophysiological correlates of figure–ground segregation directly reflect perceptual saliency

Oxytocin modulates neural activity during early perceptual salience attribution

Leading hypotheses of oxytocin's (OT) role in human cognition posit that it enhances salience attribution. However, whether OT exerts its effects predominantly in social (vs non-social) contexts remains debatable, and the time-course of intranasal OT's effects' on salience attribution processing is still unknown. We used the social Salience Attribution Task modified (sSAT) in a double-blind, placebo-controlled intranasal OT (inOT) administration, between-subjects design, with 54 male participants, to test existing theories of OT's role in cognition. Namely, we aimed to test whether inOT would differently affect salience attribution processing of social stimuli (expressing fearfulness) and non-social stimuli (fruits) made relevant via monetary reinforcement, and its neural processing time-course. During electroencephalography (EEG) recording, participants made speeded responses to emotional social (fearful faces) and non-emotional non-social (fruits) stimuli - which were matched for task-relevant motivational salience through their (color-dependent) probability of monetary reinforcement. InOT affected early (rather than late, P3b and LPP) EEG components, increasing N170 amplitude (p = .041) and P2b latency (p .001; albeit not of P1), regardless of stimuli's (emotional) socialness or reinforcement probability. Fear-related socialness affected salience attribution processing EEG (p .05) across time (N170, P2b and P3b), being later modulated by reinforcement probability (LPP). Our data suggest that OT's effects on neural activity during early perception, may exist irrespective of fear-related social- or reward-contexts. This partially supports the tri-phasic model of OT (which posits OT enhances salience attribution in an early perception stage regardless of socialness), and not the social salience nor the general approach-withdrawal hypotheses of OT, for early salience processing event-related potentials.

Event-Related Potentials Reveal the Impact of Conflict Strength in a Numerical Stroop Paradigm

Numerical cognition provides an opportunity to study the underlying processes of selective attention to numerical information in the face of conflicting, non-numerical, information of different magnitudes. For instance, in the numerical Stroop paradigm, participants are asked to judge pairs of Arabic digits whose physical size can either be congruent (e.g., 3 vs. 5) or incongruent (e.g., 3 vs. 5) with numerical value. Congruency effects when deciding which of the two digits is numerically larger are thought to reflect the inhibition of the irrelevant physical size. However, few studies have investigated the impact of the salience of the irrelevant non-numerical information on these congruency effects and their neural substrates. EEG was recorded in 32 adults during a numerical Stroop task with two levels of salience (low, high) of the irrelevant size dimension. At the behavioral level, we observed larger congruency effects in the high salience condition (i.e., when the difference in size between the two digits is larger). At the neural level, at centro-parietal electrodes, we replicated previous studies showing a main effect of congruency on event-related potential (ERP) amplitudes between 280 and 370 ms post-stimulus, as well as a main effect of salience around 200 ms post-stimulus. Crucially, congruency and salience interacted both between 230 and 250 ms (P2), and between 290 and 340 ms (P3). These results provide support for separate processes underlying the increase in congruency effect, which can be attributed to higher demands in both the inhibition of the irrelevant dimension, and the attention to the relevant numerical information.

Synergy of spatial frequency and orientation bandwidth in texture segregation

Defining target textures by increased bandwidths in spatial frequency and orientation, we observed strong cue combination effects in a combined texture figure detection and discrimination task. Performance for double-cue targets was better than predicted by independent processing of either cue and even better than predicted from linear cue integration. Application of a texture-processing model revealed that the oversummative cue combination effect is captured by calculating a low-level summary statistic (\(\Delta CE_m\)), which describes the differential contrast energy to target and reference textures, from multiple scales and orientations, and integrating this statistic across channels with a winner-take-all rule. Modeling detection performance using a signal detection theory framework showed that the observers' sensitivity to single-cue and double-cue texture targets, measured in \(d^{\prime }\) units, could be reproduced with plausible settings for filter and noise parameters. These results challenge models assuming separate channeling of elementary features and their later integration, since oversummative cue combination effects appear as an inherent property of local energy mechanisms, at least for spatial frequency and orientation bandwidth-modulated textures.

Neural responses to dynamic adaptation reveal the dissociation between the processing of the shape of contours and textures

Shape-adaptation studies show that surround textures can inhibit the processing of contours. Using event-related potentials (ERP), we examined the time-course of neural processes involved in contour-shape and texture-shape processing following adaptation to contours and textures. Contours were made of Gabor strings whose orientations were either tangential or orthogonal to the contour path, while textures were made of a series of contours arranged in parallel. We focused on two ERP components -P1, related to low-level visual processes and N1, broadly indicative of mid-level vision- and, on ERP difference waves (no-adaptor minus with-adaptor) to isolate the effects of adaptation, which are fundamentally distinct from individual processes driving P1 and N1 components. We found that in the absence of adaptation, the N1 component for contour-tests peaked later and increased in amplitude compared to the N1 for texture-tests. Following adaptation, the ERP difference wave for contour-tests revealed an early and a late component that were differentially affected by the presence of surround texture, but critically not by its orientation. For texture-tests, the early component was of opposite polarity for contours compared to texture adaptors. From the temporal sequence of ERP modulations, we conclude that texture processing begins before contour processing and encompasses the stages of perceptual processing reflected in both the low-level P1 and the mid-level N1 vision-related components. Our study provides novel evidence on the nature of separable and temporally distinct texture and contour processing mechanisms, shown in two difference wave components, that highlights the multi-faceted nature of dynamic adaptation to shape when presented in isolation and in context.

Automatic Detection of Orientation Contrast Occurs at Early but Not Earliest Stages of Visual Cortical Processing in Humans

Orientation contrast is formed when some elements orient differently from their surroundings. Although orientation contrast can be processed in the absence of top-down attention, the underlying neural mechanism for this automatic processing in humans is controversial. In particular, whether automatic detection of orientation contrast occurs at the initial feedforward stage in the primary visual cortex (i.e., V1) remains unclear. Here, we used event-related potentials (ERPs) to examine the automatic processing of orientation contrast in humans. In three experiments, participants completed a task at fixation while orientation contrasts were presented in the periphery, either in the upper visual field (UVF) or the lower visual field (LVF). All experiments showed significant positive potentials evoked by orientation contrasts over occipital areas within 100 ms after stimulus onset. These contrast effects occurred 10-20 ms later than the C1 components evoked by identically located abrupt onset stimuli which indexes the initial feedforward activity in V1. Compared with those in the UVF, orientation contrasts in the LVF evoked earlier and stronger activities, probably reflecting a LVF advantage in processing of orientation contrast. Even when orientation contrasts were rendered almost invisible by backward masking (in Experiment 2), the early contrast effect in the LVF was not disrupted. These findings imply that automatic processing of orientation contrast could occur at early visual cortical processing stages, but was slightly later than the initial feedforward processing in human V1; such automatic processing may involve either recurrent processing in V1 or feedforward processing in early extrastriate visual cortex. Highlights -We examined the earliest automatic processing of orientation contrast in humans with ERPs.-Significant orientation contrast effect started within 100 ms in early visual areas.-The earliest orientation contrast effect occurred later than the C1 evoked by abrupt onset stimuli.-The earliest orientation contrast effect was independent of top-down attention and awareness.-Automatic detection of orientation contrast arises slightly after the initial feedforward processing in V1.

Interactions Elicited by the Contradiction Between Figure Direction Discrimination and Figure-Ground Segregation

Figure-ground (FG) segregation that separates an object from the rest of the image is a fundamental problem in vision science. A majority of neurons in monkey V2 showed the selectivity to border ownership (BO) that indicates which side of a contour owns the border. Although BO could be a precursor of FG segregation, the contribution of BO to FG segregation has not been clarified. Because FG segregation is the perception of the global region that belongs to an object, whereas BO determination provides the local direction of figure (DOF) along a contour, a spatial integration of BO might be expected for the generation of FG. To understand the mechanisms underlying the perception of figural regions, we investigated the interaction between the local BO determination and the global FG segregation through the quantitative analysis of the visual perception and the spatiotemporal characteristics of eye movements. We generated a set of novel stimuli in which translucency induces local DOF along the contour and global FG independently so that DOF and FG could be either consistent or contradictory. The perceptual responses showed better performance in DOF discrimination than FG segregation, supporting distinct mechanisms for the DOF discrimination and the FG segregation. We examined whether the contradiction between DOF and FG modulates the eye movement while participants judged DOF and FG. The duration of the first eye fixation was modulated by the contradiction during FG segregation but not DOF discrimination, suggesting a sequential processing from the BO determination to the FG segregation. These results of human perception and eye fixation provide important clues for understanding the visual processing for FG segregation.

EEG signatures accompanying auditory figure-ground segregation

In everyday acoustic scenes, figure-ground segregation typically requires one to group together sound elements over both time and frequency. Electroencephalogram was recorded while listeners detected repeating tonal complexes composed of a random set of pure tones within stimuli consisting of randomly varying tonal elements. The repeating pattern was perceived as a figure over the randomly changing background. It was found that detection performance improved both as the number of pure tones making up each repeated complex (figure coherence) increased, and as the number of repeated complexes (duration) increased - i.e., detection was easier when either the spectral or temporal structure of the figure was enhanced. Figure detection was accompanied by the elicitation of the object related negativity (ORN) and the P400 event-related potentials (ERPs), which have been previously shown to be evoked by the presence of two concurrent sounds. Both ERP components had generators within and outside of auditory cortex. The amplitudes of the ORN and the P400 increased with both figure coherence and figure duration. However, only the P400 amplitude correlated with detection performance. These results suggest that 1) the ORN and P400 reflect processes involved in detecting the emergence of a new auditory object in the presence of other concurrent auditory objects; 2) the ORN corresponds to the likelihood of the presence of two or more concurrent sound objects, whereas the P400 reflects the perceptual recognition of the presence of multiple auditory objects and/or preparation for reporting the detection of a target object.

The time course of pattern discrimination in the human brain

In electrophysiological experiments on visual pattern discrimination, decision difficulty was manipulated either via the physical characteristics of the test stimuli, or by changing the instruction given to the observer. Visual stimuli were rectangular matrices each composed of 100 Gabor patches having different orientations. Matrices differed in the number of Gabor patches with vertical, or horizontal, orientation. The observers' task was either to discriminate the dominant orientation or to detect collinear elements in the matrix. Relating task difficulty to performance, in the first experimental paradigm (detection of orientation) we obtained the conventional S-like psychometric function but in the second (detection of collinearity) the psychometric function showed a complicated U-curve. Matching between electrophysiological and psychophysical data and image statistical functions allowed us to establish the relative timing of the cortical processes underlying perception and decision making in relation to textural features. In the first 170ms after stimulus onset coding of the low-level properties of the image takes place. In the time interval 170-400ms, ERP amplitude correlated only with complex image properties, but not with task difficulty. The first effects arising from decision difficulty were observable at 400ms after stimulus onset, and therefore this is probably the earliest electrophysiological signature of the decision making processes, in the given experimental paradigm.

Asymmetrical white matter networks for attending to global versus local features

The ability to draw objects is a complex process depending on an array of cognitive mechanisms including routines for spatial coding, attention and the processing of both local and global features. Previous studies using both neuropsychological and neuroimaging data have reported hemispheric asymmetries in attending to local versus global features linked to a variety of cortical loci. However, it has not been examined to date whether such asymmetries exist at the level of white matter pathways sub-serving global/local attention. The current study provides a comprehensive analysis of brain-behaviour relationships in the processing of local versus global features based on data from a large cohort of sub-acute stroke patients (n = 248) and behavioural measures from a complex figure copy task. The data analysis used newly developed methods for automated delineation of stroke lesions combined with track-wise lesion deficits procedures. We found (i) that reproduction of local features in figure copying was supported by a neural network confined to the left hemisphere, consisting of cortical loci within parietal, occipital and insular lobes and interconnected by the inferior-fronto-occipital fasciculus (IFOF), and (ii) that global feature processing was associated with a right hemisphere network interconnected by the third branch of the superior longitudinal fasciculus and the long segment of the perisylvian network. The data support the argument that asymmetrical white matter disconnections within long-range association pathways predict poor complex figure drawing resulting from deficits in hierarchical representation. We conclude that hemispheric asymmetries in attending to local versus global features exist on the level of both cortical loci and the supporting white matter pathways.

Specificity of brain reactions to second-order visual stimuli

The second-order visual mechanisms perform the operation of integrating the spatially distributed local visual information. Their organization is traditionally considered within the framework of the filter-rectify-filter model. These are the second-order filters that provide the ability to detect texture gradients. However, the question of the mechanisms' selectivity to the modulation dimension remains open. The aim of this investigation is to answer the above question by using visual evoked potentials (VEPs). Stimuli were textures consisting of staggered Gabor patches. The base texture was nonmodulated (NM). Three other textures represented the base texture which was sinusoidally modulated in different dimensions: contrast, orientation, or spatial frequency. EEG was recorded with 20 electrodes. VEPs of 500 ms duration were obtained for each of the four textures. After that, VEP to the NM texture was subtracted from VEP to each modulated texture. As a result, three different waves (d-waves) were obtained for each electrode site. Each d-wave was then averaged across all the 48 observers. The revealed d-waves have a latency of about 200 ms and, in our opinion, reflect the second-order filters reactivation through the feedback connection. The d-waves for different modulation dimensions were compared with each other in time, amplitude, topography, and localization of the sources of activity that causes the d-wave (with sLORETA). We proceeded from the assumption that the d-wave (its first component) represents functioning of the second-order visual mechanisms and activity changes at the following processing stages. It was found that the d-waves for different modulation dimensions significantly differ in all parameters. The obtained results indicate that the spatial modulations of different texture parameters caused specific changes in the brain activity, which could be evidence supporting the specificity of the second-order visual mechanisms to modulation dimension.

Visual Processing of Contour Patterns under Conditions of Inattentional Blindness

An inattentional blindness paradigm was adapted to measure ERPs elicited by visual contour patterns that were or were not consciously perceived. In the first phase of the experiment, subjects performed an attentionally demanding task while task-irrelevant line segments formed square-shaped patterns or random configurations. After the square patterns had been presented 240 times, subjects' awareness of these patterns was assessed. More than half of all subjects, when queried, failed to notice the square patterns and were thus considered inattentionally blind during this first phase. In the second phase of the experiment, the task and stimuli were the same, but following this phase, all of the subjects reported having seen the patterns. ERPs recorded over the occipital pole differed in amplitude from 220 to 260 msec for the pattern stimuli compared with the random arrays regardless of whether subjects were aware of the patterns. At subsequent latencies (300–340 msec) however, ERPs over bilateral occipital-parietal areas differed between patterns and random arrays only when subjects were aware of the patterns. Finally, in a third phase of the experiment, subjects viewed the same stimuli, but the task was altered so that the patterns became task relevant. Here, the same two difference components were evident but were followed by a series of additional components that were absent in the first two phases of the experiment. We hypothesize that the ERP difference at 220–260 msec reflects neural activity associated with automatic contour integration whereas the difference at 300–340 msec reflects visual awareness, both of which are dissociable from task-related postperceptual processing.

Context modulates the ERP signature of contour integration

We investigated how the electrophysiological signature of contour integration is changed by the context in which a contour is embedded. Specifically, we manipulated the orientations of Gabor elements surrounding an embedded shape outline. The amplitudes of early visual components over posterior scalp regions were changed by the presence of a contour, and by the orientation of elements surrounding the contour. Differences in context type had an effect on the early P1 and N1 components, but not on the later P2 component. The presence of a contour had an effect on the N1 and P2 components, but not on the earlier P1 component. A modulatory effect of context on contour integration was observed on the N1 component. These results highlight the importance of the context in which contour integration takes place.

Visual exploration and object recognition by lattice deformation

Mechanisms of explicit object recognition are often difficult to investigate and require stimuli with controlled features whose expression can be manipulated in a precise quantitative fashion. Here, we developed a novel method (called "Dots"), for generating visual stimuli, which is based on the progressive deformation of a regular lattice of dots, driven by local contour information from images of objects. By applying progressively larger deformation to the lattice, the latter conveys progressively more information about the target object. Stimuli generated with the presented method enable a precise control of object-related information content while preserving low-level image statistics, globally, and affecting them only little, locally. We show that such stimuli are useful for investigating object recognition under a naturalistic setting--free visual exploration--enabling a clear dissociation between object detection and explicit recognition. Using the introduced stimuli, we show that top-down modulation induced by previous exposure to target objects can greatly influence perceptual decisions, lowering perceptual thresholds not only for object recognition but also for object detection (visual hysteresis). Visual hysteresis is target-specific, its expression and magnitude depending on the identity of individual objects. Relying on the particular features of dot stimuli and on eye-tracking measurements, we further demonstrate that top-down processes guide visual exploration, controlling how visual information is integrated by successive fixations. Prior knowledge about objects can guide saccades/fixations to sample locations that are supposed to be highly informative, even when the actual information is missing from those locations in the stimulus. The duration of individual fixations is modulated by the novelty and difficulty of the stimulus, likely reflecting cognitive demand.

From spatial frequency contrast to edge preponderance: the differential modulation of early visual evoked potentials by natural scene stimuli

The contrast response function of early visual evoked potentials elicited by sinusoidal gratings is known to exhibit characteristic potentials closely associated with the processes of parvocellular and magnocellular pathways. Specifically, the N1 component has been linked with parvocellular processes, while the P1 component has been linked with magnocellular processes. However, little is known regarding the response properties of the N1 and P1 components during the processing and encoding of complex (i.e., broadband) stimuli such as natural scenes. Here, we examine how established physical characteristics of natural scene imagery modulate the N1 and P1 components in humans by providing a systematic investigation of component modulation as visual stimuli are gradually built up from simple sinusoidal gratings to highly complex natural scene imagery. The results suggest that the relative dominance in signal output of the N1 and P1 components is dependent on spatial frequency (SF) luminance contrast for simple stimuli up to natural scene imagery possessing few edges. However, such a dependency shifts to a dominant N1 signal for natural scenes possessing abundant edge content and operates independently of SF luminance contrast.

Neural representation of feature synergy

Interactive non-linear cooperation of different feature dimensions, feature synergy, has been studied in psychophysics, but the neural mechanism is unknown. The present study investigated the neural representation of feature synergy of two second-order visual features by combining electroencephalography (EEG) with the signal detection theory (SDT). Two kinds of a 27-by-27 array of Gabor patches were presented in a random order; a reference stimulus which has no segregated region, and a target stimulus whose inner region differed in spatial frequency, orientation, or both from the surround. Subjects performed a Yes-No discrimination of whether the inner region was different from the surround, while EEG signals were recorded from 62 locations. When the SDT measure showed feature synergy, EEG activity showed a long-lasting enhancement starting at 130 ms around the inferior temporal region. In contrast, no EEG modulation was observed when feature synergy was not present. Thus, our combined approach demonstrates that non-linear cooperation between different features is represented by neural activity starting at 130 ms post-stimulus in the ventral visual stream.

Visual detection and identification are not the same: evidence from psychophysics and fMRI

Sometimes object detection as opposed to identification is sufficient to initiate the appropriate action. To explore the neural origin of behavioural differences between the two tasks, we combine psychophysical measurements and fMRI, specifically contrasting shape detection versus identification of a figure. This figure consisted of Gabor elements being oriented differently from those in the background. We equalized performance levels for detection and identification by adjusting orientation differences accordingly for each observer. Hence, stimulus saliency was constant for both tasks allowing a differentiation between the activations specific for detection versus identification processes. Identification yielded higher psychophysical thresholds, slower reaction times and increased hemodynamic activations in the lateral-occipital complex (LOC) and an adjacent area in the collateral sulcus (CoS). Additional analysis using cortex-based alignment revealed four voxel-clusters differentially activated by the tasks, situated in the inferior parietal lobe, the precuneus, the anterior cingulum and the medial frontal gyrus. Our results indicate partly separated cortical mechanisms for object detection and identification.