Early spatial frequency processing of natural images: an ERP study

Brain correlates of declarative memory atypicalities in autism: a systematic review of functional neuroimaging findings

The long-described atypicalities of memory functioning experienced by people with autism have major implications for daily living, academic learning, as well as cognitive remediation. Though behavioral studies have identified a robust profile of memory strengths and weaknesses in autism spectrum disorder (ASD), few works have attempted to establish a synthesis concerning their neural bases. In this systematic review of functional neuroimaging studies, we highlight functional brain asymmetries in three anatomical planes during memory processing between individuals with ASD and typical development. These asymmetries consist of greater activity of the left hemisphere than the right in ASD participants, of posterior brain regions-including hippocampus-rather than anterior ones, and presumably of the ventral (occipito-temporal) streams rather than the dorsal (occipito-parietal) ones. These functional alterations may be linked to atypical memory processes in ASD, including the pre-eminence of verbal over spatial information, impaired active maintenance in working memory, and preserved relational memory despite poor context processing in episodic memory.

Both low and high spatial frequencies drive the early posterior negativity in response to snake stimuli

Previous event-related potential (ERP) studies have shown that snake pictures elicit greater early posterior negativity (EPN) compared to other animal pictures. The EPN reflects early selective visual processing of emotionally significant stimuli. Evidence for the role that high and low spatial frequencies play in the early detection of snakes is still inconsistent. The current study aims to clarify this by studying the effect of high and low spatial frequencies on the elevated EPN for snakes separately. Using a rapid serial visual presentation paradigm, participants viewed images of snakes, spiders and birds in three different conditions of filtered spatial frequencies: high spatial frequency, low spatial frequency, and full spatial frequency (the original image). P1 and mean EPN activity in a time window of 225-300 ms after stimulus onset were measured at the occipital cluster (O1, O2, Oz). The results show smaller P1 amplitudes and shorter P1 latencies in response to full-spectrum snake pictures compared to full-spectrum spider and bird pictures, and an increased EPN in response to snake pictures compared to spider and bird pictures in all three filtering conditions. The EPN in response to full-spectrum snake pictures was larger than the EPN in response to filtered snake images. No difference in EPN was found between the snake pictures in the high and low spatial frequency conditions. The results suggest that the roles of high and low spatial frequencies in early automatic attention to snakes are equally important.

Taxonomic and thematic semantic relationships in picture naming as revealed by Laplacian-transformed event-related potentials

Semantically related concepts co-activate when we speak. Prior research reported both behavioral interference and facilitation due to co-activation during picture naming. Different word relationships may account for some of this discrepancy. Taxonomically related words (e.g., WOLF-DOG) have been associated with semantic interference; thematically related words (e.g., BONE-DOG) have been associated with facilitation. Although these different semantic relationships have been associated with opposite behavioral outcomes, electrophysiological studies have found inconsistent effects on event-related potentials. We conducted a picture-word interference electroencephalography experiment to examine word retrieval dynamics in these different semantic relationships. Importantly, we used traditional monopolar analysis as well as Laplacian transformation allowing us to examine spatially deblurred event-related components. Both analyses revealed greater negativity (150-250 ms) for unrelated than related taxonomic pairs, though more restricted in space for thematic pairs. Critically, Laplacian analyses revealed a larger negative-going component in the 300 to 500 ms time window in taxonomically related versus unrelated pairs which were restricted to a left frontal recording site. In parallel, an opposite effect was found in the same time window but localized to a left parietal site. Finding these opposite effects in the same time window was feasible thanks to the use of the Laplacian transformation and suggests that frontal control processes are concurrently engaged with cascading effects of the spread of activation through semantically related representations.

Perceiving fingerspelling via point-light displays: The stimulus and the perceiver both matter

Signed languages such as American Sign Language (ASL) rely on visuospatial information that combines hand and bodily movements, facial expressions, and fingerspelling. Signers communicate in a wide array of sub-optimal environments, such as in dim lighting or from a distance. While fingerspelling is a common and essential part of signed languages, the perception of fingerspelling in difficult visual environments is not well understood. The movement and spatial patterns of ASL are well-suited to representation by dynamic Point Light Display (PLD) stimuli in which human movement is shown as an array of moving dots affixed to joints on the body. We created PLD videos of fingerspelled location names. The location names were either Real (e.g., KUWAIT) or Pseudo-names (e.g., CLARTAND), and the PLDs showed either a High or a Low number of markers. In an online study, Deaf and Hearing ASL users (total N = 283) watched 27 PLD stimulus videos that varied by Word Type and Number of Markers. Participants watched the videos and typed the names they saw, along with how confident they were in their response. We predicted that when signers see ASL fingerspelling PLDs, language experience in ASL will be positively correlated with accuracy and self-rated confidence scores. We also predicted that Real location names would be understood better than Pseudo names. Our findings supported those predictions. We also discovered a significant interaction between Age and Word Type, which suggests that as people age, they use outside world knowledge to inform their fingerspelling success. Finally, we examined the accuracy and confidence in fingerspelling perception in early ASL users. Studying the relationship between language experience with PLD fingerspelling perception allows us to explore how hearing status, ASL fluency levels, and age of language acquisition affect the core abilities of understanding fingerspelling.

The Time Sequence of Face Spatial Frequency Differs During Working Memory Encoding and Retrieval Stages

Previous studies have found that P1 and P2 components were more sensitive to configural and featural face processing, respectively, when attentional resources were sufficient, suggesting that face processing follows a coarse-to-fine sequence. However, the role of working memory (WM) load in the time course of configural and featural face processing is poorly understood, especially whether it differs during encoding and retrieval stages. This study employed a delayed recognition task with varying WM load and face spatial frequency (SF). Our behavioral and ERP results showed that WM load modulated face SF processing. Specifically, for the encoding stage, P1 and P2 were more sensitive to broadband SF (BSF) faces, while N170 was more sensitive to low SF (LSF) and BSF faces. For the retrieval stage, P1 on the right hemisphere was more sensitive to BSF faces relative to HSF faces, N170 was more sensitive to LSF faces than HSF faces, especially under the load 1 condition, while P2 was more sensitive to high SF (HSF) faces than HSF faces, especially under load 3 condition. These results indicate that faces are perceived less finely during the encoding stage, whereas face perception follows a coarse-to-fine sequence during the retrieval stage, which is influenced by WM load. The coarse and fine information were processed especially under the low and high load conditions, respectively.

Spatial frequency impacts perceptual and attentional ERP components across cultures

Culture impacts visual perception in several ways.To identify stages of perceptual processing that differ between cultures, we usedelectroencephalography measures of perceptual and attentional responses to simple visual stimuli.Gabor patches of higher or lower spatialfrequencywere presented at high contrast to 25 American and 31 East Asian participants while they were watching for the onset of aninfrequent, oddball stimulus. Region of interest and mass univariate analyses assessed how cultural background and stimuli spatial frequency affected the visual evoked response potentials. Across both groups, the Gabor of lower spatial frequency produced stronger evoked response potentials in the anterior N1 and P3 than did the higher frequency Gabor. The mass univariate analyses also revealed effects of spatial frequency, including a frontal negativity around 150 ms and a widespread posterior positivity around 300 ms. The effects of spatial frequency generally differed little across cultures; although there was some evidence for cultural differences in the P3 response to different frequencies at the Pz electrode, this effect did not emerge in the mass univariate analyses. We discuss these results in relation to those from previous studies, and explore the potential advantages of mass univariate analyses for cultural neuroscience.

Do Humans and Deep Convolutional Neural Networks Use Visual Information Similarly for the Categorization of Natural Scenes?

The investigation of visual categorization has recently been aided by the introduction of deep convolutional neural networks (CNNs), which achieve unprecedented accuracy in picture classification after extensive training. Even if the architecture of CNNs is inspired by the organization of the visual brain, the similarity between CNN and human visual processing remains unclear. Here, we investigated this issue by engaging humans and CNNs in a two-class visual categorization task. To this end, pictures containing animals or vehicles were modified to contain only low/high spatial frequency (HSF) information, or were scrambled in the phase of the spatial frequency spectrum. For all types of degradation, accuracy increased as degradation was reduced for both humans and CNNs; however, the thresholds for accurate categorization varied between humans and CNNs. More remarkable differences were observed for HSF information compared to the other two types of degradation, both in terms of overall accuracy and image-level agreement between humans and CNNs. The difficulty with which the CNNs were shown to categorize high-passed natural scenes was reduced by picture whitening, a procedure which is inspired by how visual systems process natural images. The results are discussed concerning the adaptation to regularities in the visual environment (scene statistics); if the visual characteristics of the environment are not learned by CNNs, their visual categorization may depend only on a subset of the visual information on which humans rely, for example, on low spatial frequency information.

Symmetry Modulates the Amplitude Spectrum Slope Effect on Visual Preference

Within the spectrum of a natural image, the amplitude of modulation decreases with spatial frequency. The speed of such an amplitude decrease, or the amplitude spectrum slope, of an image affects the perceived aesthetic value. Additionally, a human observer would consider a symmetric image more appealing than they would an asymmetric one. We investigated how these two factors jointly affect aesthetic preferences by manipulating both the amplitude spectrum slope and the symmetric level of images to assess their effects on aesthetic preference on a 6-point Likert scale. Our results showed that the preference ratings increased with the symmetry level but had an inverted U-shaped relation to amplitude spectrum slope. In addition, a strong interaction existed between symmetry level and amplitude spectrum slope on preference rating, in that symmetry can amplify the amplitude spectrum slope’s effects. A quadratic function of the spectrum slope can describe such effects. That is, preference is an inverted U-shaped function of spectrum slope whose intercept is determined by the number of symmetry axes. The modulation depth of the quadratic function manifests the interaction between the two factors.

Increasing Perceptual Salience Diminishes the Motor Interference Effect From Dangerous Objects

Existing research has indicated that dangerous objects may conflict with an individual’s prepared motor actions and thus slow responses. This phenomenon is called the motor interference effect from dangerous objects. However, its origin remains arguable. The current study aimed to preclude an alternative origin and to investigate whether the efficiency of processing a prepared response toward a dangerous object could benefit from increasing the perceptual salience of the object by painting the object red. The design used a shape categorization task to emphasize the dangerous elements of target objects and manipulated target color (gray versus red), target dangerousness (safe versus dangerous) and prime-target congruency (congruent versus incongruent). The null effect of N2 amplitudes between the dangerous and safe conditions precluded the alternative origin and suggested that the motor interference effect did not originate from response inhibition. Furthermore, the results indicated a modulation effect of the motor interference effect in different colors. The classic motor interference effect was observed in the gray target condition, but it diminished in the red target condition. The underlying cognitive processes were reflected in ERPs. More positive P2 and frontal P3 amplitudes were identified in the red target condition than in the gray target condition, which indicated that deeper feature detection was assigned to and more attentional resources were automatically recruited for the red targets than for the gray targets. Analysis of the parietal P3 amplitudes identified a similar result pattern as the mean RTs. A more positive P3 amplitude was identified in the dangerous condition than in the safe condition when the targets were painted gray. In contrast, the P3 amplitudes were identical between the dangerous condition and the safe condition when the targets were painted red. The results indicated that the increased attentional resources facilitated the evaluation of red target dangerousness and thus accelerated reactions to the red dangerous targets; the reaction speeds to those targets were close to those for the reaction speeds to the red safe targets. Detailed processes that underline these components are discussed.

Influence of Artificially Generated Interocular Blur Difference on Fusion Stability Under Vergence Stress

The stability of fusion was evaluated by its breakage when interocular blur differences were presented under vergence demand to healthy subjects. We presumed that these blur differences cause suppression of the more blurred image (interocular blur suppression, IOBS), disrupt binocular fusion and suppressed eye leaves its forced vergent position. During dichoptic presentation of static grayscale images of natural scenes, the luminance contrast (mode B) or higher-spatial frequency content (mode C) or luminance contrast plus higher-spatial frequency content (mode A) were stepwise reduced in the image presented to the non-dominant eye. We studied the effect of these types of blur on fusion stability at various levels of the vergence demand. During the divergence demand, the fusion was disrupted with approximately half blur than during convergence. Various modes of blur influenced fusion differently. The mode C (isolated reduction of higher-spatial frequency content) violated fusion under the lowest vergence demand significantly more than either isolated or combined reduction of luminance contrast (mode B and A). According to our results, the image´s details (i.e. higher-spatial frequency content) protects binocular fusion from disruption by the lowest vergence demand.

Pattern matters: Snakes exhibiting triangular and diamond-shaped skin patterns modulate electrophysiological activity in human visual cortex

The neural and perceptual mechanisms that support the efficient visual detection of snakes in humans are still not fully understood. According to the Snake Detection Theory, selection pressures posed by snakes on early primates have shaped the development of the visual system. Previous studies in humans have investigated early visual electrophysiological activity in response to snake images vs. various alternative dangerous or non-dangerous stimuli. These studies have shown that the Early Posterior Negativity (EPN) component is selectively elicited by snake or snake-like images. Recent findings yielded the complementary/alternative hypothesis that early humans (and possibly other primates) evolved an aversion especially for potentially harmful triangular shapes, such as teeth, claws or spikes. In the present study we investigated the effect of triangular and diamond-shaped patterns in snake skins on the ERP correlates of visual processing in humans. In the first experiment, we employed pictures of snakes displaying either triangular/diamond-shaped patterns or no particular pattern on their skins, and pictures of frogs as control. Participants observed a random visual presentation of these pictures. Consistent with previous studies, snakes elicited an enhanced negativity between 225 and 300 ms (EPN) compared to frogs. However, snakes featuring triangular/diamond-shaped patterns on their skin produced an enhanced EPN compared to the snakes that did not display such patterns. In a second experiment we used pictures displaying only skin patterns of snakes and frogs. Results from the second experiment confirmed the results of the first experiment, suggesting that triangular snake-skin patterns modulate the activity in human visual cortex. Taken together, our results constitute an important contribution to the snake detection theory.

Categorization Goals Modulate the Use of Natural Scene Statistics

Understanding natural scenes involves the contribution of bottom–up analysis and top–down modulatory processes. However, the interaction of these processes during the categorization of natural scenes is not well understood. In the current study, we approached this issue using ERPs and behavioral and computational data. We presented pictures of natural scenes and asked participants to categorize them in response to different questions (Is it an animal/vehicle? Is it indoors/outdoors? Are there one/two foreground elements?). ERPs for target scenes requiring a “yes” response began to differ from those of nontarget scenes, beginning at 250 msec from picture onset, and this ERP difference was unmodulated by the categorization questions. Earlier ERPs showed category-specific differences (e.g., between animals and vehicles), which were associated with the processing of scene statistics. From 180 msec after scene onset, these category-specific ERP differences were modulated by the categorization question that was asked. Categorization goals do not modulate only later stages associated with target/nontarget decision but also earlier perceptual stages, which are involved in the processing of scene statistics.

Cultural specialization of visual cortex

A growing body of evidence suggests culture influences how individuals perceive the world around them. This study investigates whether these cultural differences extend to a simple object viewing task and visual cortex by examining voxel pattern representations with multi-voxel pattern analysis (MVPA). During functional magnetic resonance imaging scanning, 20 East Asian and 20 American participants viewed photos of everyday items, equated for familiarity and conceptual agreement across cultures. Whole brain searchlight mapping with non-parametric statistical evaluation tested whether these stimuli evoked multi-voxel patterns that were distinct between cultural groups. We found that participants' cultural identities were successfully predicted from stimuli representations in visual cortex Brodmann areas 18 and 19. This result demonstrates culturally specialized visual cortex during a basic perceptual task ubiquitous to everyday life.

Entrainment of visual steady-state responses is modulated by global spatial statistics

The rhythmic delivery of visual stimuli evokes large-scale neuronal entrainment in the form of steady-state oscillatory field potentials. The spatiotemporal properties of stimulus drive appear to constrain the relative degrees of neuronal entrainment. Specific frequency ranges, for example, are uniquely suited for enhancing the strength of stimulus-driven brain oscillations. When it comes to the nature of the visual stimulus itself, studies have used a plethora of inputs ranging from spatially unstructured empty fields to simple contrast patterns (checkerboards, gratings, stripes) and complex arrays (human faces, houses, natural scenes). At present, little is known about how the global spatial statistics of the input stimulus influence entrainment of scalp-recorded electrophysiological signals. In this study, we used rhythmic entrainment source separation of scalp EEG to compare stimulus-driven phase alignment for distinct classes of visual inputs, including broadband spatial noise ensembles with varying second-order statistics, natural scenes, and narrowband sine-wave gratings delivered at a constant flicker frequency. The relative magnitude of visual entrainment was modulated by the global properties of the driving stimulus. Entrainment was strongest for pseudo-naturalistic broadband visual noise patterns in which luminance contrast is greatest at low spatial frequencies (a power spectrum slope characterized by 1/ƒ-2).NEW & NOTEWORTHY Rhythmically modulated visual stimuli entrain the activity of neuronal populations, but the effect of global stimulus statistics on this entrainment is unknown. We assessed entrainment evoked by 1) visual noise ensembles with different spectral slopes, 2) complex natural scenes, and 3) narrowband sinusoidal gratings. Entrainment was most effective for broadband noise with naturalistic luminance contrast. This reveals some global properties shaping stimulus-driven brain oscillations in the human visual system.

Dissociating Attention Effects from Categorical Perception with ERP Functional Microstates

When faces appear in our visual environment we naturally attend to them, possibly to the detriment of other visual information. Evidence from behavioural studies suggests that faces capture attention because they are more salient than other types of visual stimuli, reflecting a category-dependent modulation of attention. By contrast, neuroimaging data has led to a domain-specific account of face perception that rules out the direct contribution of attention, suggesting a dedicated neural network for face perception. Here we sought to dissociate effects of attention from categorical perception using Event Related Potentials. Participants viewed physically matched face and butterfly images, with each category acting as a target stimulus during different blocks in an oddball paradigm. Using a data-driven approach based on functional microstates, we show that the locus of endogenous attention effects with ERPs occurs in the N1 time range. Earlier categorical effects were also found around the level of the P1, reflecting either an exogenous increase in attention towards face stimuli, or a putative face-selective measure. Both category and attention effects were dissociable from one another hinting at the role that faces may play in early capturing of attention before top-down control of attention is observed. Our data support the conclusion that certain object categories, in this experiment, faces, may capture attention before top-down voluntary control of attention is initiated.

Deficits in attentional processing of fearful facial expressions in schizophrenic patients

Impaired attentional processing of negative facial expressions is prominent in schizophrenia and has been shown to be associated with patients’ social dysfunctions. However, little is known about when and which specific attention deficits influence social functions. Given the dynamic feature of attention, it is necessary to investigate the attention deficits in negative emotional processing unfolding in time. The current study used event-related potentials (ERPs) to investigate the temporal dynamics of attention deficits in emotion perception and their potential relationship with emotional/social impairments in neuroleptic naive schizophrenic patients. Two specific attention deficits were identified and were found to be associated with emotional/social impairments. More specifically, the deficit in orienting attention (evidenced with the reduced P1 amplitude) was correlated with expressive deficits, while the deficit in executive control of attention (evidenced with the reduced P3 amplitude) was correlated with avolition/asociality. Together, these findings may provide novel insights into the core pathophysiological processes and offer objective biomarkers for specific emotional/social impairments in schizophrenia. It is also hoped that this study helps to bridge the gap between basic cognitive deficits and relative high-level social dysfunctions in schizophrenic patients.

Parallel processing of general and specific threat during early stages of perception

Differential processing of threat can consummate as early as 100 ms post-stimulus. Moreover, early perception not only differentiates threat from non-threat stimuli but also distinguishes among discrete threat subtypes (e.g. fear, disgust and anger). Combining spatial-frequency-filtered images of fear, disgust and neutral scenes with high-density event-related potentials and intracranial source estimation, we investigated the neural underpinnings of general and specific threat processing in early stages of perception. Conveyed in low spatial frequencies, fear and disgust images evoked convergent visual responses with similarly enhanced N1 potentials and dorsal visual (middle temporal gyrus) cortical activity (relative to neutral cues; peaking at 156 ms). Nevertheless, conveyed in high spatial frequencies, fear and disgust elicited divergent visual responses, with fear enhancing and disgust suppressing P1 potentials and ventral visual (occipital fusiform) cortical activity (peaking at 121 ms). Therefore, general and specific threat processing operates in parallel in early perception, with the ventral visual pathway engaged in specific processing of discrete threats and the dorsal visual pathway in general threat processing. Furthermore, selectively tuned to distinctive spatial-frequency channels and visual pathways, these parallel processes underpin dimensional and categorical threat characterization, promoting efficient threat response. These findings thus lend support to hybrid models of emotion.

Early and late effects of objecthood and spatial frequency on event-related potentials and gamma band activity

BACKGROUND

The visual system may process spatial frequency information in a low-to-high, coarse-to-fine sequence. In particular, low and high spatial frequency information may be processed via different pathways during object recognition, with LSF information projected rapidly to frontal areas and HSF processed later in visual ventral areas. In an electroencephalographic study, we examined the time course of information processing for images filtered to contain different ranges of spatial frequencies. Participants viewed either high spatial frequency (HSF), low spatial frequency (LSF), or unfiltered, broadband (BB) images of objects or non-object textures, classifying them as showing either man-made or natural objects, or non-objects. Event-related potentials (ERPs) and evoked and total gamma band activity (eGBA and tGBA) recorded using the electroencephalogram were compared for object and non-object images across the different spatial frequency ranges.

RESULTS

The visual P1 showed independent modulations by object and spatial frequency, while for the N1 these factors interacted. The P1 showed more positive amplitudes for objects than non-objects, and more positive amplitudes for BB than for HSF images, which in turn evoked more positive amplitudes than LSF images. The peak-to-peak N1 showed that the N1 was much reduced for BB non-objects relative to all other images, while HSF and LSF non-objects still elicited as negative an N1 as objects. In contrast, eGBA was influenced by spatial frequency and not objecthood, while tGBA showed a stronger response to objects than non-objects.

CONCLUSIONS

Different pathways are involved in the processing of low and high spatial frequencies during object recognition, as reflected in interactions between objecthood and spatial frequency in the visual N1 component. Total gamma band seems to be related to a late, probably high-level representational process.