Chapter 65 The speed of visual cognition

Animal cognition: Dogs build semantic expectations between spoken words and objects

A recent study has used scalp-recorded electroencephalography to obtain evidence of semantic processing of human speech and objects by domesticated dogs. The results suggest that dogs do comprehend the meaning of familiar spoken words, in that a word can evoke the mental representation of the object to which it refers.

Early visual processing relevant to the reduction of adaptation-induced perceptual bias

Visual perception is biased by the preceding visual environment. A well-known perceptual bias is the negative bias where a current percept is biased away from the preceding image (adaptor). The preceding adaptor induces augmentation of early visual evoked potential (the P1 enhancement) of the following test image; the adaptor may invoke certain visual processing for the subsequent test image. However, the visual mechanism underlying P1 enhancement remains unclear. The present study assessed what the P1 alteration reflects in relation to the occurrence of the negative bias. In terms of inter-individual differences, we report that the P1 enhancement of the Necker lattice significantly correlated with the reduction of the reverse-bias effect. Further analyses revealed that the P1 enhancement was insusceptible to neural adaptation to the adaptor at the level of perceptual configuration. The present study suggests that prolonged exposure to a visual image induces modulatory visual processing for the subsequent image (reflected in the P1 enhancement), which is relevant to counteraction of the negative bias.

Distinct brain representations of processed and unprocessed foods

Among all of the stimuli surrounding us, food is arguably the most rewarding for the essential role it plays in our survival. In previous visual recognition research, it has already been demonstrated that the brain not only differentiates edible stimuli from non-edible stimuli but also is endowed with the ability to detect foods' idiosyncratic properties such as energy content. Given the contribution of the cooked diet to human evolution, in the present study we investigated whether the brain is sensitive to the level of processing food underwent, based solely on its visual appearance. We thus recorded visual evoked potentials (VEPs) from normal-weight healthy volunteers who viewed color images of unprocessed and processed foods equated in caloric content. Results showed that VEPs and underlying neural sources differed as early as 130 ms post-image onset when participants viewed unprocessed versus processed foods, suggesting a within-category early discrimination of food stimuli. Responses to unprocessed foods engaged the inferior frontal and temporal regions and the premotor cortices. In contrast, viewing processed foods led to the recruitment of occipito-temporal cortices bilaterally, consistently with other motivationally relevant stimuli. This is the first evidence of diverging brain responses to food as a function of the transformation undergone during its preparation that provides insights on the spatiotemporal dynamics of food recognition.

Perceptual Discrimination of Basic Object Features Is Not Facilitated When Priming Stimuli Are Prevented From Reaching Awareness by Means of Visual Masking

Our understanding of how form, orientation and size are processed within and outside of awareness is limited and requires further investigation. Therefore, we investigated whether or not the visual discrimination of basic object features can be influenced by subliminal processing of stimuli presented beforehand. Visual masking was used to render stimuli perceptually invisible. Three experiments examined if visible and invisible primes could facilitate the subsequent feature discrimination of visible targets. The experiments differed in the kind of perceptual discrimination that participants had to make. Namely, participants were asked to discriminate visual stimuli on the basis of their form, orientation, or size. In all three experiments, we demonstrated reliable priming effects when the primes were visible but not when the primes were made invisible. Our findings underscore the importance of conscious awareness in facilitating the perceptual discrimination of basic object features.

Electrophysiological evidence for a specific neural correlate of musical violation expectation in primary-school children

The majority of studies on music processing in children used simple musical stimuli. Here, primary schoolchildren judged the appropriateness of musical closure in expressive polyphone music, while high-density electroencephalography was recorded. Stimuli ended either regularly or contained refined in-key harmonic transgressions at closure. The children discriminated the transgressions well above chance. Regular and transgressed endings evoked opposite scalp voltage configurations peaking around 400ms after stimulus onset with bilateral frontal negativity for regular and centro-posterior negativity (CPN) for transgressed endings. A positive correlation could be established between strength of the CPN response and rater sensitivity (d-prime). We also investigated whether the capacity to discriminate the transgressions was supported by auditory domain specific or general cognitive mechanisms, and found that working memory capacity predicted transgression discrimination. Latency and distribution of the CPN are reminiscent of the N400, typically observed in response to semantic incongruities in language. Therefore our observation is intriguing, as the CPN occurred here within an intra-musical context, without any symbols referring to the external world. Moreover, the harmonic in-key transgressions that we implemented may be considered syntactical as they transgress structural rules. Such structural incongruities in music are typically followed by an early right anterior negativity (ERAN) and an N5, but not so here. Putative contributive sources of the CPN were localized in left pre-motor, mid-posterior cingulate and superior parietal regions of the brain that can be linked to integration processing. These results suggest that, at least in children, processing of syntax and meaning may coincide in complex intra-musical contexts.

Gravity influences top-down signals in visual processing

Visual perception is not only based on incoming visual signals but also on information about a multimodal reference frame that incorporates vestibulo-proprioceptive input and motor signals. In addition, top-down modulation of visual processing has previously been demonstrated during cognitive operations including selective attention and working memory tasks. In the absence of a stable gravitational reference, the updating of salient stimuli becomes crucial for successful visuo-spatial behavior by humans in weightlessness. Here we found that visually-evoked potentials triggered by the image of a tunnel just prior to an impending 3D movement in a virtual navigation task were altered in weightlessness aboard the International Space Station, while those evoked by a classical 2D-checkerboard were not. Specifically, the analysis of event-related spectral perturbations and inter-trial phase coherency of these EEG signals recorded in the frontal and occipital areas showed that phase-locking of theta-alpha oscillations was suppressed in weightlessness, but only for the 3D tunnel image. Moreover, analysis of the phase of the coherency demonstrated the existence on Earth of a directional flux in the EEG signals from the frontal to the occipital areas mediating a top-down modulation during the presentation of the image of the 3D tunnel. In weightlessness, this fronto-occipital, top-down control was transformed into a diverging flux from the central areas toward the frontal and occipital areas. These results demonstrate that gravity-related sensory inputs modulate primary visual areas depending on the affordances of the visual scene.

Emotional expressions modulate low α and β oscillations in a cortically blind patient

Studies of cortical blindness have suggested that some residual visual function may persist without perceptual awareness, a condition known as blindsight. To investigate electrophysiological evidence of unconscious processing of emotional stimuli, we examined the event-related oscillations (EROs) in a 62year-old male patient (TN) with affective blindsight during random stimulation of three facial expressions (fearful, happy and neutral). Spectral power analysis in response to the different emotions revealed significant differences between fearful and happy faces over the right frontal regions at 7-8Hz (low α), and between emotional and neutral faces over the left frontal sites at 12-13Hz (low β) in a time period between 100-400ms after visual stimulus onset. These results demonstrate that emotional face processing occurs very early in time in the absence of any functional striate cortex, and further reveals the existence of specific oscillatory frequencies that reflect unconscious processing of facial expressions in affective blindsight.

TMS to the lateral occipital cortex disrupts object processing but facilitates scene processing

The study of brain-damaged patients and advancements in neuroimaging have lead to the discovery of discrete brain regions that process visual image categories, such as objects and scenes. However, how these visual image categories interact remains unclear. For example, is scene perception simply an extension of object perception, or can global scene "gist" be processed independently of its component objects? Specifically, when recognizing a scene such as an "office," does one need to first recognize its individual objects, such as the desk, chair, lamp, pens, and paper to build up the representation of an "office" scene? Here, we show that temporary interruption of object processing through repetitive TMS to the left lateral occipital cortex (LO), an area known to selectively process objects, impairs object categorization but surprisingly facilitates scene categorization. This result was replicated in a second experiment, which assessed the temporal dynamics of this disruption and facilitation. We further showed that repetitive TMS to left LO significantly disrupted object processing but facilitated scene processing when stimulation was administered during the first 180 msec of the task. This demonstrates that the visual system retains the ability to process scenes during disruption to object processing. Moreover, the facilitation of scene processing indicates disinhibition of areas involved in global scene processing, likely caused by disrupting inhibitory contributions from the LO. These findings indicate separate but interactive pathways for object and scene processing and further reveal a network of inhibitory connections between these visual brain regions.

Two phases of V1 activity for visual recognition of natural images

Present theories of visual recognition emphasize the role of interactive processing across populations of neurons within a given network, but the nature of these interactions remains unresolved. In particular, data describing the sufficiency of feedforward algorithms for conscious vision and studies revealing the functional relevance of feedback connections to the striate cortex seem to offer contradictory accounts of visual information processing. TMS is a good method to experimentally address this issue, given its excellent temporal resolution and its capacity to establish causal relations between brain function and behavior. We studied 20 healthy volunteers in a visual recognition task. Subjects were briefly presented with images of animals (birds or mammals) in natural scenes and were asked to indicate the animal category. MRI-guided stereotaxic single TMS pulses were used to transiently disrupt striate cortex function at different times after image onset (SOA). Visual recognition was significantly impaired when TMS was applied over the occipital pole at SOAs of 100 and 220 msec. The first interval has consistently been described in previous TMS studies and is explained as the interruption of the feedforward volley of activity. Given the late latency and discrete nature of the second peak, we hypothesize that it represents the disruption of a feedback projection to V1, probably from other areas in the visual network. These results provide causal evidence for the necessity of recurrent interactive processing, through feedforward and feedback connections, in visual recognition of natural complex images.

Electrical source dynamics in three functional localizer paradigms

The visual cortex exhibits functional specialization that can be routinely demonstrated using hemodynamic measures like fMRI and PET. To understand the dynamic nature of cortical processes, however, source imaging with a high temporal resolution is necessary. Here, we asked how well distributed EEG source localization (LAURA) identifies functionally specialized visual processes. We tested three stimulus paradigms commonly used in fMRI with the aim to localize striate cortex, motion-sensitive areas, and face-sensitive areas. EEG source localization showed initial activations in striate and extra-striate areas at around 70ms after stimulus onset. These were quickly followed by extensive cortical, as well as subcortical activation. Functional motion and face-selective areas were localized with margins of below 2cm, at around 170 and 150ms, respectively. The results furthermore show for the first time that the C1 component has generators in the insula and frontal eye fields, but also in subcortical areas like the parahippocampus and the thalamus.

The brain tracks the energetic value in food images

Do our brains implicitly track the energetic content of the foods we see? Using electrical neuroimaging of visual evoked potentials (VEPs) we show that the human brain can rapidly discern food's energetic value, vis à vis its fat content, solely from its visual presentation. Responses to images of high-energy and low-energy food differed over two distinct time periods. The first period, starting at approximately 165 ms post-stimulus onset, followed from modulations in VEP topography and by extension in the configuration of the underlying brain network. Statistical comparison of source estimations identified differences distributed across a wide network including both posterior occipital regions and temporo-parietal cortices typically associated with object processing, and also inferior frontal cortices typically associated with decision-making. During a successive processing stage (starting at approximately 300 ms), responses differed both topographically and in terms of strength, with source estimations differing predominantly within prefrontal cortical regions implicated in reward assessment and decision-making. These effects occur orthogonally to the task that is actually being performed and suggest that reward properties such as a food's energetic content are treated rapidly and in parallel by a distributed network of brain regions involved in object categorization, reward assessment, and decision-making.

Poor performance on serial visual tasks in persons with reading disabilities: impaired working memory?

The present study investigates the performance of persons with reading disabilities (PRD) on a variety of sequential visual-comparison tasks that have different working-memory requirements. In addition, mediating relationships between the sequential comparison process and attention and memory skills were looked for. Our findings suggest that PRD perform worse than normally achieving readers (NAR) when the task requires more than a minimal amount of working memory, unrelated to presentation rate. We also demonstrate high correlations between performance on the task with the most working-memory demands and reading-related skills, suggesting that poor working-memory abilities may be one of the underlying mechanisms of dyslexia. The mediating model analysis indicates that order judgment tasks are mediating to verbal working memory, suggesting that visual sequence memory precedes auditory sequence memory. We further suggest that visual tasks involving sequential comparisons could probe for poor working memory in PRD.

The time course of action and action-word comprehension in the human brain as revealed by neurophysiology

Numerous previous neuroimaging studies suggest an involvement of cortical motor areas not only in action execution but also in action recognition and understanding. Motor areas of the human brain have also been found to activate during the processing of written and spoken action-related words and sentences. Even more strikingly, stimuli referring to different bodily effectors produced specific somatotopic activation patterns in the motor areas. However, metabolic neuroimaging results can be ambiguous with respect to the processing stage they reflect. This is a serious limitation when hypotheses concerning linguistic processes are tested, since in this case it is usually crucial to distinguish early lexico-semantic processing from strategic effects or mental imagery that may follow lexico-semantic information access. Timing information is therefore pivotal to determine the functional significance of motor areas in action recognition and action-word comprehension. Here, we review attempts to reveal the time course of these processes using neurophysiological methods (EEG, MEG and TMS), in visual and auditory domains. We will highlight the importance of the choice of appropriate paradigms in combination with the corresponding method for the extraction of timing information. The findings will be discussed in the general context of putative brain mechanisms of word and object recognition.

Generating controlled image sets in cognitive neuroscience research

The investigation of perceptual and cognitive functions with non-invasive brain imaging methods critically depends on the careful selection of stimuli for use in experiments. For example, it must be verified that any observed effects follow from the parameter of interest (e.g. semantic category) rather than other low-level physical features (e.g. luminance, or spectral properties). Otherwise, interpretation of results is confounded. Often, researchers circumvent this issue by including additional control conditions or tasks, both of which are flawed and also prolong experiments. Here, we present some new approaches for controlling classes of stimuli intended for use in cognitive neuroscience, however these methods can be readily extrapolated to other applications and stimulus modalities. Our approach is comprised of two levels. The first level aims at equalizing individual stimuli in terms of their mean luminance. Each data point in the stimulus is adjusted to a standardized value based on a standard value across the stimulus battery. The second level analyzes two populations of stimuli along their spectral properties (i.e. spatial frequency) using a dissimilarity metric that equals the root mean square of the distance between two populations of objects as a function of spatial frequency along x- and y-dimensions of the image. Randomized permutations are used to obtain a minimal value between the populations to minimize, in a completely data-driven manner, the spectral differences between image sets. While another paper in this issue applies these methods in the case of acoustic stimuli (Aeschlimann et al., Brain Topogr 2008), we illustrate this approach here in detail for complex visual stimuli.

How the camel lost its hump: the impact of object typicality on event-related potential signals in object decision

Using an object decision task, event-related potentials (ERPs), and minimum norm current source estimates, we investigated early spatiotemporal aspects of cortical activation elicited by line drawings that were manipulated on two dimensions: authenticity and typicality. Authentic objects were those that match real-world experience, whereas nonauthentic objects were "doctored" by deletion or addition of features (e.g., a camel with its hump removed, a hammer with two handles). The main manipulation of interest for both authentic and nonauthentic objects was the degree of typicality in the object's structure: typical items are composed of parts that have tended to co-occur across many different objects in the perceiver's experience. The ERP pattern revealed a significant typicality effect at 116 msec after stimulus onset. Both atypical authentic objects (e.g., a camel with its hump) and atypical nonauthentic objects (e.g., a jackal with a hump) elicited stronger brain activation than did objects with typical structure. A significant effect of authenticity was observed at 480 msec, with stronger activation for the nonauthentic objects. The factors of typicality and authenticity interacted at 160 and 330 msec. The most prominent source of the typicality effect was the bilateral occipitotemporal cortex, whereas the interaction and the authenticity effects were mainly observed in the more anterior bilateral temporal cortex. These findings support the hypothesis that within the first few hundred milliseconds after stimulus presentation onset, visual-form-related perceptual and conceptual processes represent distinct but interacting stages in object recognition.

Boundary completion is automatic and dissociable from shape discrimination

Normal visual perception readily overcomes suboptimal or degraded viewing conditions through perceptual filling-in processes, enhancing object recognition and discrimination abilities. This study used visual evoked potential (VEP) recordings in conjunction with electrical neuroimaging analyses to determine the spatiotemporal brain dynamics of boundary completion and shape discrimination processes in healthy humans performing the so-called "thin/fat" discrimination task (Ringach and Shapley, 1996) with stimuli producing illusory contours. First, results suggest that boundary completion processes occur independent of subjects' accuracy on the discrimination task. Modulation of the VEP to the presence versus absence of illusory contours [the IC effect (Murray et al., 2002)] was indistinguishable in terms of response magnitude and scalp topography over the 124-186 ms poststimulus period, regardless of whether task performance was correct. This suggests that failure on this discrimination task is not primarily a consequence of failed boundary completion. Second, the electrophysiological correlates of thin/fat shape discrimination processes are temporally dissociable from those of boundary completion, occurring during a substantially later phase of processing (approximately 330-406 ms). The earlier IC effect was unaffected by whether the perceived contour produced a thin or fat shape. In contrast, later time periods of the VEP modulated according to perceived shape only in the case of stimuli producing illusory contours, but not for control stimuli for which performance was at near-chance levels. Collectively, these data provide further support for a multistage model of object processing under degraded viewing conditions.

Sex Differences in Semantic Processing: Event-Related Brain Potentials Distinguish between Lower and Higher Order Semantic Analysis during Word Reading

Behavioral studies suggest that women and men differ in the strategic elaboration of verbally encoded information especially in the absence of external task demand. However, measuring such covert processing requires other than behavioral data. The present study used event-related potentials to compare sexes in lower and higher order semantic processing during the passive reading of semantically related and unrelated word pairs. Women and men showed the same early context effect in the P1-N1 transition period. This finding indicates that the initial lexical-semantic access is similar in men and women. In contrast, sexes differed in higher order semantic processing. Women showed an earlier and longer lasting context effect in the N400 accompanied by larger signal strength in temporal networks similarly recruited by men and women. The results suggest that women spontaneously conduct a deeper semantic analysis. This leads to faster processing of related words in the active neural networks as reflected in a shorter stability of the N400 map in women. Taken together, the findings demonstrate that there is a selective sex difference in the controlled semantic analysis during passive word reading that is not reflected in different functional organization but in the depth of processing.

EEG source imaging

OBJECTIVE

Electroencephalography (EEG) is an important tool for studying the temporal dynamics of the human brain's large-scale neuronal circuits. However, most EEG applications fail to capitalize on all of the data's available information, particularly that concerning the location of active sources in the brain. Localizing the sources of a given scalp measurement is only achieved by solving the so-called inverse problem. By introducing reasonable a priori constraints, the inverse problem can be solved and the most probable sources in the brain at every moment in time can be accurately localized.

METHODS AND RESULTS

Here, we review the different EEG source localization procedures applied during the last two decades. Additionally, we detail the importance of those procedures preceding and following source estimation that are intimately linked to a successful, reliable result. We discuss (1) the number and positioning of electrodes, (2) the varieties of inverse solution models and algorithms, (3) the integration of EEG source estimations with MRI data, (4) the integration of time and frequency in source imaging, and (5) the statistical analysis of inverse solution results.

CONCLUSIONS AND SIGNIFICANCE

We show that modern EEG source imaging simultaneously details the temporal and spatial dimensions of brain activity, making it an important and affordable tool to study the properties of cerebral, neural networks in cognitive and clinical neurosciences.