Functional magnetic resonance imaging of human visual cortex during face matching: a comparison with positron emission tomography

Introduction to the special section on aging, cognition, and neuroimaging

In response to the recent increase in the number of laboratories engaged in the use of functional and structural neuroimaging to study cognitive aging, this special section has been compiled to serve as an entry into this area for the readers of Psychology and Aging. These articles are representative of the field and cover many of the issues faced by researchers in this area. This introduction presents some background into the techniques that are used and provides an overview of the articles.

Neural correlates of woman face processing by 2-month-old infants

The age of 2 months marks a turn in the development of face processing in humans with the emergence of recognition based on internal feature configuration. We studied the neural bases of this early cognitive expertise, critical for adaptive behavior in the social world, by mapping with positron emission tomography the brain activity of 2-month-old alert infants while looking at unknown woman faces. We observed the activation of a distributed network of cortical areas that largely overlapped the adult face-processing network, including the so-called fusiform face area. We also evidenced the activation of left superior temporal and inferior frontal gyri, regions associated, in adults, with language processing. These findings demonstrates that cognitive development proceeds early in functionally active interconnected cortical areas despite the fact they have not all yet reached full metabolic maturation.

Human neural systems for face recognition and social communication

Face perception is mediated by a distributed neural system in humans that consists of multiple, bilateral regions. The functional organization of this system embodies a distinction between the representation of invariant aspects of faces, which is the basis for recognizing individuals, and the representation of changeable aspects, such as eye gaze, expression, and lip movement, which underlies the perception of information that facilitates social communication. The system also has a hierarchical organization. A core system, consisting of occipitotemporal regions in extrastriate visual cortex, mediates the visual analysis of faces. An extended system consists of regions from neural systems for other cognitive functions that can act in concert with the core system to extract meaning from faces. Of regions in the extended system for face perception, the amygdala plays a central role in processing the social relevance of information gleaned from faces, particularly when that information may signal a potential threat.

Depersonalization disorder: thinking without feeling

Patients with depersonalization disorder (DP) experience a detachment from their own senses and surrounding events, as if they were outside observers. A particularly common symptom is emotional detachment from the surroundings. Using functional magnetic resonance imaging (fMRI), we compared neural responses to emotionally salient stimuli in DP patients, and in psychiatric and healthy control subjects. Six patients with DP, 10 with obsessive-compulsive disorder (OCD), and six volunteers were scanned whilst viewing standardized pictures of aversive and neutral scenes, matched for visual complexity. Pictures were then rated for emotional content. Both control groups rated aversive pictures as much more emotive, and demonstrated in response to these scenes significantly greater activation in regions important for disgust perception, the insula and occipito-temporal cortex, than DP patients (covarying for age, years of education and total extent of brain activation). In DP patients, aversive scenes activated the right ventral prefrontal cortex. The insula was activated only by neutral scenes in this group. Our findings indicate that a core phenomenon of depersonalization--absent subjective experience of emotion--is associated with reduced neural responses in emotion-sensitive regions, and increased responses in regions associated with emotion regulation.

Age-related changes in brain neuromagnetic responses to face perception in humans

In order to investigate the effects of ageing on face perception, we studied the magnetic responses to face images in 15 young (19-38 years) and 10 elderly (51-81 years) subjects. Face-specific responses (160mF), which originate in the inferior occipitotemporal cortices, and face non-specific responses (100m), which originate in the primary visual cortices, were evoked in all subjects. Averaged peak latency of the 160mF in the elderly group (174.0+/-9.1 ms) was significantly longer (P<0.0005) than that in the young group (161.5+/-5.1 ms), while no inter-group difference was found in the 100m latency. There was a significant correlation between age and 160mF latency (+0.35 ms/year, R=0.747) suggesting age-related decline of face perception.

Human cortical processing of colour and pattern

The present study investigates human visual processing of simple two-colour patterns using a delayed match to sample paradigm with positron emission tomography (PET). This study is unique in that we specifically designed the visual stimuli to be the same for both pattern and colour recognition with all patterns being abstract shapes not easily verbally coded composed of two-colour combinations. We did this to explore those brain regions required for both colour and pattern processing and to separate those areas of activation required for one or the other. We found that both tasks activated similar occipital regions, the major difference being more extensive activation in pattern recognition. A right-sided network that involved the inferior parietal lobule, the head of the caudate nucleus, and the pulvinar nucleus of the thalamus was common to both paradigms. Pattern recognition also activated the left temporal pole and right lateral orbital gyrus, whereas colour recognition activated the left fusiform gyrus and several right frontal regions.

The representation of objects in the human occipital and temporal cortex

Recently, we identified, using fMRI, three bilateral regions in the ventral temporal cortex that responded preferentially to faces, houses, and chairs [Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L., & Haxby, J. V. (1999). Distributed representation of objects in the human ventral visual pathway. Proceedings of the National Academy of Sciences, U.S.A., 96, 9379--9384]. Here, we report differential patterns of activation, similar to those seen in the ventral temporal cortex, in bilateral regions of the ventral occipital cortex. We also found category-related responses in the dorsal occipital cortex and in the superior temporal sulcus. Moreover, rather than activating discrete, segregated areas, each category was associated with its own differential pattern of response across a broad expanse of cortex. The distributed patterns of response were similar across tasks (passive viewing, delayed matching) and presentation formats (photographs, line drawings). We propose that the representation of objects in the ventral visual pathway, including both occipital and temporal regions, is not restricted to small, highly selective patches of cortex but, instead, is a distributed representation of information about object form. Within this distributed system, the representation of faces appears to be less extensive as compared to the representations of nonface objects.

Effects of attention and emotion on face processing in the human brain: an event-related fMRI study

We used event-related fMRI to assess whether brain responses to fearful versus neutral faces are modulated by spatial attention. Subjects performed a demanding matching task for pairs of stimuli at prespecified locations, in the presence of task-irrelevant stimuli at other locations. Faces or houses unpredictably appeared at the relevant or irrelevant locations, while the faces had either fearful or neutral expressions. Activation of fusiform gyri by faces was strongly affected by attentional condition, but the left amygdala response to fearful faces was not. Right fusiform activity was greater for fearful than neutral faces, independently of the attention effect on this region. These results reveal differential influences on face processing from attention and emotion, with the amygdala response to threat-related expressions unaffected by a manipulation of attention that strongly modulates the fusiform response to faces.

The neural system underlying Chinese logograph reading

Written Chinese as logographic script differs notably from alphabets such as English in visual form, orthography, phonology, and semantics. Thus, research on the Chinese language is important to advance our understanding of the universality and particularity of the organization of language systems in the brain. In this study, we examine the neural systems associated with logographic reading using functional magnetic resonance imaging. Two experimental tasks were devised, one based on semantic decision and the other on homophone decision. Compared to the fixation baseline, peak activations resulting from semantic as well as homophony decisions were localized in the left middle frontal gyrus (BA 9). Left inferior frontal cortex also mediated Chinese processing. In addition, more right hemisphere cortical regions (i.e., BAs 47/45, 7, 40/39, and the right visual system) were involved in reading Chinese relative to reading English. This is attributed to the square shape of the logograph which requires an elaborated analysis of the spatial information and locations of various strokes comprising the logographic character. We suggest that the left middle frontal area (BA 9) coordinates and integrates the intensive visuospatial analysis demanded by logographs' square configuration and the semantic (or phonological) analysis required by the present experimental tasks. Our study has implicated brain regions common to both logographic and alphabetic languages as well as brain regions specialized in processing logographs.

Center-periphery organization of human object areas

The organizing principles that govern the layout of human object-related areas are largely unknown. Here we propose a new organizing principle in which object representations are arranged according to a central versus peripheral visual field bias. The proposal is based on the finding that building-related regions overlap periphery-biased visual field representations, whereas face-related regions are associated with center-biased representations. Furthermore, the eccentricity maps encompass essentially the entire extent of object-related occipito-temporal cortex, indicating that most object representations are organized with respect to retinal eccentricity. A control experiment ruled out the possibility that the results are due exclusively to unequal feature distribution in these images. We hypothesize that brain regions representing object categories that rely on detailed central scrutiny (such as faces) are more strongly associated with processing of central information, compared to representations of objects that may be recognized by more peripheral information (such as buildings or scenes).

Left posterior BA37 is involved in object recognition: a TMS study

Functional imaging studies have proposed a role for left BA37 in phonological retrieval, semantic processing, face processing and object recognition. The present study targeted the posterior aspect of BA37 to see whether a deficit, specific to one of the above types of processing could be induced. Four conditions were investigated: word and nonword reading, colour naming and picture naming. Repetitive transcranial magnetic stimulation (rTMS) was delivered over posterior BA37 of the left and right hemispheres (lBA37 and rBA37, respectively) and over the vertex. Subjects were significantly slower to name pictures when TMS was given over lBA37 compared to vertex or rBA37. rTMS over lBA37 had no significant effect on word reading, nonword reading or colour naming. The picture naming deficit is suggested to result from a disruption to object recognition processes. This study corroborates the finding from a recent imaging study, that the most posterior part of left hemispheric BA37 has a necessary role in object recognition.

III. Electrophysiological studies of face processing in Williams syndrome

Williams Syndrome (WMS) is a genetically based disorder characterized by pronounced variability in performance across different domains of cognitive functioning. This study examined brain activity linked to face-processing abilities, which are typically spared in individuals with WMS. Subjects watched photographic pairs of upright or inverted faces and indicated if the second face matched or did not match the first face. Results from a previous study with normal adults showed dramatic differences in the timing and distribution of ERP effects linked to recognition of upright and inverted faces. In normal adults, upright faces elicited ERP differences to matched vs. mismatched faces at approximately 320 msec (N320) after the onset of the second stimulus. This "N320" effect was largest over anterior regions of the right hemisphere. In contrast, the mismatch/match effect for inverted faces consisted of a large positive component between 400 and 1000 msec (P500) that was largest over parietal regions and was symmetrical. In contrast to normal adults, WMS subjects showed an N320-mismatch effect for both upright and inverted faces. Additionally, the WMS subjects did not display the N320 right-hemisphere asymmetry observed in the normal adults. WMS subjects also displayed an abnormally small negativity at 100 msec (N100) and an abnormally large negativity at 200 msec (N200) to both upright and inverted faces. This ERP pattern was observed in all subjects with WMS but was not observed in the normal controls. These results may be linked to increased attention to faces in subjects with WMS and might be specific to the disorder. These results were consistent with our ERP studies of language processing in WMS, which suggested abnormal cerebral specialization for spared cognitive functions in individuals with WMS.

Hemispheric asymmetries for whole-based and part-based face processing in the human fusiform gyrus

Behavioral studies indicate a right hemisphere advantage for processing a face as a whole and a left hemisphere superiority for processing based on face features. The present PET study identifies the anatomical localization of these effects in well-defined regions of the middle fusiform gyri of both hemispheres. The right middle fusiform gyrus, previously described as a face-specific region, was found to be more activated when matching whole faces than face parts whereas this pattern of activity was reversed in the left homologous region. These lateralized differences appeared to be specific to faces since control objects processed either as wholes or parts did not induce any change of activity within these regions. This double dissociation between two modes of face processing brings new evidence regarding the lateralized localization of face individualization mechanisms in the human brain.

The distributed human neural system for face perception

Face perception, perhaps the most highly developed visual skill in humans, is mediated by a distributed neural system in humans that is comprised of multiple, bilateral regions. We propose a model for the organization of this system that emphasizes a distinction between the representation of invariant and changeable aspects of faces. The representation of invariant aspects of faces underlies the recognition of individuals, whereas the representation of changeable aspects of faces, such as eye gaze, expression, and lip movement, underlies the perception of information that facilitates social communication. The model is also hierarchical insofar as it is divided into a core system and an extended system. The core system is comprised of occipitotemporal regions in extrastriate visual cortex that mediate the visual analysis of faces. In the core system, the representation of invariant aspects is mediated more by the face-responsive region in the fusiform gyrus, whereas the representation of changeable aspects is mediated more by the face-responsive region in the superior temporal sulcus. The extended system is comprised of regions from neural systems for other cognitive functions that can be recruited to act in concert with the regions in the core system to extract meaning from faces.

Susceptibility-induced loss of signal: comparing PET and fMRI on a semantic task

Functional magnetic resonance imaging (fMRI) has become a popular tool for investigations into the neural correlates of cognitive activity. One limitation of fMRI, however, is that it has difficulty imaging regions near tissue interfaces due to distortions from macroscopic susceptibility effects which become more severe at higher magnetic field strengths. This difficulty can be particularly problematic for language tasks that engage regions of the temporal lobes near the air-filled sinuses. This paper investigates susceptibility-induced signal loss in the temporal lobes and proposes that by defining a priori regions of interest and using the small-volume statistical correction of K. J. Worsley, S. Marrett, P. Neelin, A. C. Vandal, K. J. Friston, and A. C. Evans (1996, Hum. Brain Mapp. 4: 58-83), activations in these areas can sometimes be detected by increasing the statistical power of the analysis. We conducted two experiments, one with PET and the other with fMRI, using almost identical semantic categorization paradigms and comparable methods of analysis. There were areas of overlap as well as differences between the PET and fMRI results. One anticipated difference was a lack of activation in two regions in the temporal lobe on initial analyses in the fMRI data set. With a specific region of interest, however, activation in one of the regions was detected. These experiments demonstrate three points: first, even for almost identical cognitive tasks such as those in this study, PET and fMRI may not produce identical results; second, differences between the two methods due to macroscopic susceptibility artifacts in fMRI can be overcome with appropriate statistical corrections, but only partially; and third, new data acquisition paradigms are necessary to fully deal with susceptibility-induced signal loss if the sensitivity of the fMRI experiment to temporal lobe activations is to be enhanced.

A stimulus-driven approach to object identity and location processing in the human brain

The primate visual system is considered to be segregated into ventral and dorsal streams specialized for processing object identity and location, respectively. We reexamined the dorsal/ventral model using a stimulus-driven approach to object identity and location processing. While looking at repeated presentations of a standard object at a standard location, subjects monitored for any infrequent "oddball" changes in object identity, location, or identity and location (conjunction). While the identity and location oddballs preferentially activated ventral and dorsal brain regions respectively, each oddball type activated both pathways. Furthermore, all oddball types recruited the lateral temporal cortex and the temporo-parietal junction. These findings suggest that a strict dorsal/ventral dual-stream model does not fully account for the perception of novel objects in space.

Electrophysiological correlates of recollecting faces of known and unknown individuals

We recorded brain potentials from healthy human subjects during a recognition test in order to monitor neural processing associated with face recollection. Subjects first attempted to memorize 40 faces; half were accompanied by a voice simulating that person speaking (e.g., "I'm Jimmy and I was a roadie for the Grateful Dead") and half were presented in silence. In the test phase, subjects attempted to discriminate both types of old faces (i.e., "named" and "unnamed" faces) from new faces. Recognition averaged 87% correct for named faces, 74% correct for unnamed faces, and 91% correct for new faces. Potentials to old faces were more positive than those to new faces from 300 to 600 ms after face onset. For named faces, the old-new ERP difference was observed at anterior and posterior scalp locations. For unnamed faces, the old-new ERP difference was observed only at posterior scalp locations. Results from a prior experiment suggest that these effects do not reflect perceptual priming of faces. The posterior portion of the old-new ERP difference was thus interpreted as a neural correlate of retrieval of visual face information and the anterior portion as an indication of retrieval of person-specific semantic information.

Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies

Positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have been extensively used to explore the functional neuroanatomy of cognitive functions. Here we review 275 PET and fMRI studies of attention (sustained, selective, Stroop, orientation, divided), perception (object, face, space/motion, smell), imagery (object, space/ motion), language (written/spoken word recognition, spoken/ no spoken response), working memory (verbal/numeric, object, spatial, problem solving), semantic memory retrieval (categorization, generation), episodic memory encoding (verbal, object, spatial), episodic memory retrieval (verbal, nonverbal, success, effort, mode, context), priming (perceptual, conceptual), and procedural memory (conditioning, motor, and nonmotor skill learning). To identify consistent activation patterns associated with these cognitive operations, data from 412 contrasts were summarized at the level of cortical Brodmann's areas, insula, thalamus, medial-temporal lobe (including hippocampus), basal ganglia, and cerebellum. For perception and imagery, activation patterns included primary and secondary regions in the dorsal and ventral pathways. For attention and working memory, activations were usually found in prefrontal and parietal regions. For language and semantic memory retrieval, typical regions included left prefrontal and temporal regions. For episodic memory encoding, consistently activated regions included left prefrontal and medial-temporal regions. For episodic memory retrieval, activation patterns included prefrontal, medial-temporal, and posterior midline regions. For priming, deactivations in prefrontal (conceptual) or extrastriate (perceptual) regions were consistently seen. For procedural memory, activations were found in motor as well as in non-motor brain areas. Analysis of regional activations across cognitive domains suggested that several brain regions, including the cerebellum, are engaged by a variety of cognitive challenges. These observations are discussed in relation to functional specialization as well as functional integration.

Imaging cognition II: An empirical review of 275 PET and fMRI studies

Positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have been extensively used to explore the functional neuroanatomy of cognitive functions. Here we review 275 PET and fMRI studies of attention (sustained, selective, Stroop, orientation, divided), perception (object, face, space/motion, smell), imagery (object, space/motion), language (written/spoken word recognition, spoken/no spoken response), working memory (verbal/numeric, object, spatial, problem solving), semantic memory retrieval (categorization, generation), episodic memory encoding (verbal, object, spatial), episodic memory retrieval (verbal, nonverbal, success, effort, mode, context), priming (perceptual, conceptual), and procedural memory (conditioning, motor, and nonmotor skill learning). To identify consistent activation patterns associated with these cognitive operations, data from 412 contrasts were summarized at the level of cortical Brodmann's areas, insula, thalamus, medial-temporal lobe (including hippocampus), basal ganglia, and cerebellum. For perception and imagery, activation patterns included primary and secondary regions in the dorsal and ventral pathways. For attention and working memory, activations were usually found in prefrontal and parietal regions. For language and semantic memory retrieval, typical regions included left prefrontal and temporal regions. For episodic memory encoding, consistently activated regions included left prefrontal and medial temporal regions. For episodic memory retrieval, activation patterns included prefrontal, medial temporal, and posterior midline regions. For priming, deactivations in prefrontal (conceptual) or extrastriate (perceptual) regions were consistently seen. For procedural memory, activations were found in motor as well as in non-motor brain areas. Analysis of regional activations across cognitive domains suggested that several brain regions, including the cerebellum, are engaged by a variety of cognitive challenges. These observations are discussed in relation to functional specialization as well as functional integration.

Brodmann's areas 17 and 18 brought into stereotaxic space-where and how variable?

Studies on structural-functional associations in the visual system require precise information on the location and variability of Brodmann's areas 17 and 18. Usually, these studies are based on the Talairach atlas, which does not rely on cytoarchitectonic observations, but on comparisons of macroscopic features in the Talairach brain and Brodmann's drawing. In addition, in this atlas are found only the approximate positions of cytoarchitectonic areas and not the exact borders. We have cytoarchitectonically mapped both areas in 10 human brains and marked their borders in corresponding computerized images. Borders were defined on the basis of quantitative cytoarchitecture and multivariate statistics. In addition to borders of areas 17 and 18, subparcellations within both areas were found. The cytoarchitectonically defined areas were 3-D reconstructed and transferred into the stereotaxic space of the standard reference brain. Surface rendering of the brains revealed high individual variability in size and shape of the areas and in the relationship to the free surface and sulci. Ranges and centers of gravity of both areas were calculated in Talairach coordinates. The positions of areas 17 and 18 in the stereotaxic space differed between the hemispheres. Both areas reached significantly more caudal and medial positions on the left than on the right. Probability maps were created in which the degree of overlap in each stereotaxic position was quantified. These maps of areas 17 and 18 are the first of their kind and contain precise stereotaxic information on both interhemispheric and interindividual differences.

FMRI and PET of self-paced finger movement: comparison of intersubject stereotaxic averaged data

We compared the intersubject-averaged functional anatomy of self-paced right index finger movement as revealed by (15)O water positron emission tomography (PET) and blood oxygen level-dependent functional magnetic resonance imaging (FMRI) at 1.5 T. Image data sets were acquired with both techniques on a group of eight subjects, spatially normalized in the stereotaxic space and subsequently processed in order to get identical smoothness and degrees of freedom. Intersubject-averaged PET and FMRI activation maps were found congruent in the left primary sensorimotor area (PSM), bilateral supplementary motor area, bilateral supra marginalis gyri, left operculum, left inferior parietal lobule, right middle frontal gyrus, and right cerebellum. In those regions the mean distance between PET and FMRI local maxima was 7.4 mm. FMRI detected additional activations in the right precentral gyrus, right rolandic operculum, right inferior parietal lobule, and bilateral insula, whereas PET demonstrated a higher detection sensitivity at the deep nuclei level. PET and FMRI percentage signal variations were found linearly related by a factor around 10, both within the PSM and across a set of distributed local extrema. However, in most cases, FMRI was more sensitive than PET, as assessed by t values. Finally the pattern of deactivations was markedly dissimilar between the two techniques, possibly due to differences in the "Rest" control task.

Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects

The cognitive and neural mechanisms underlying category-specific knowledge remain controversial. Here we report that, across multiple tasks (viewing, delayed match to sample, naming), pictures of animals and tools were associated with highly consistent, category-related patterns of activation in ventral (fusiform gyrus) and lateral (superior and middle temporal gyri) regions of the posterior temporal lobes. In addition, similar patterns of category-related activity occurred when subjects read the names of, and answered questions about, animals and tools. These findings suggest that semantic object information is represented in distributed networks that include sites for storing information about specific object attributes such as form (ventral temporal cortex) and motion (lateral temporal cortex).

Regional brain activity during shape recognition impaired by a scopolamine challenge to encoding

In the present positron emission tomography (PET) study, we examine the effect of a scopolamine-induced challenge to encoding upon the pattern of regional cerebral blood flow during recognition of a list of abstract visual shapes 3 days after encoding of these shapes. This study was conducted to test hypotheses concerning the fusiform and thalamic contributions to object recognition arising from a previous imaging study of impaired recognition. In that study, we demonstrated that activity in the fusiform cortex and the thalamus during shape recognition was modulated by memory challenges. These memory challenges included, on one hand, impaired storage as a consequence of diazepam administration during encoding, and, on the other hand, impaired retrieval caused by a perceptual challenge. Activation in the fusiform cortex decreased during impaired recognition, irrespective of the type of challenge. In contrast, thalamic activation increased only when the recognition deficit resulted from impaired memory storage. Based on these results, we hypothesized that fusiform activation during recognition reflects the matching of an incoming stimulus with a stored one, whereas thalamic activation reflects retrieval attempts. These hypotheses would receive considerable support if scopolamine, which also impairs memory storage, induced similar modulations of fusiform and thalamic activation. In the present study, we observed that a scopolamine challenge to encoding does indeed modulate the activity in the very same regions that were previously modulated by a diazepam challenge. Hence, a similar memory deficit, although primarily effected through different neurochemical pathways, was paralleled by a similar modulation of activity in the same set of nodes in the shape recognition network. In the fusiform cortex, scopolamine decreased recognition-related activity, as did the sensory challenge of retrieval. Furthermore, covariate analysis demonstrated that the level of fusiform activity linearly correlates with behavioural performance. In the thalamus, activation increased following impaired encoding. This is in accordance with the idea that enhanced thalamic activity reflects increased effort expended in retrieval. In addition, in the intraparietal sulcus, differential activation also increased following impaired memory storage, possibly reflecting enhanced visuospatial attention in an effort to compensate for impaired performance.

Distributed representation of objects in the human ventral visual pathway

Brain imaging and electrophysiological recording studies in humans have reported discrete cortical regions in posterior ventral temporal cortex that respond preferentially to faces, buildings, and letters. These findings suggest a category-specific anatomically segregated modular organization of the object vision pathway. Here we present data from a functional MRI study in which we found three distinct regions of ventral temporal cortex that responded preferentially to faces and two categories of other objects, namely houses and chairs, and had a highly consistent topological arrangement. Although the data could be interpreted as evidence for separate modules, we found that each category also evoked significant responses in the regions that responded maximally to other stimuli. Moreover, each category was associated with its own differential pattern of response across ventral temporal cortex. These results indicate that the representation of an object is not restricted to a region that responds maximally to that object, but rather is distributed across a broader expanse of cortex. We propose that the functional architecture of the ventral visual pathway is not a mosaic of category-specific modules but instead is a continuous representation of information about object form that has a highly consistent and orderly topological arrangement.

fMRI and EEG responses to periodic visual stimulation

EEG/VEP and fMRI responses to periodic visual stimulation are reported. The purpose of these experiments was to look for similar patterns in the time series produced by each method to help understand the relationship between the two. The stimulation protocol was the same for both sets of experiments and consisted of five complete cycles of checkerboard pattern reversal at 1.87 Hz for 30 s followed by 30 s of a stationary checkerboard. The fMRI data was analyzed using standard methods, while the EEG was analyzed with a new measurement of activation-the VEPEG. Both VEPEG and fMRI time series contain the fundamental frequency of the stimulus and quasi harmonic components-an unexplained double frequency commonly found in fMRI data. These results have prompted a reappraisal of the methods for analyzing fMRI data and have suggested a connection between our findings and much older published invasive electrophysiological measurements of blood flow and the partial pressures of oxygen and carbon dioxide. Overall our new analysis suggests that fMRI signals are strongly dependant on hydraulic blood flow effects. We distinguish three categories of fMRI signal corresponding to: focal activated regions of brain tissue; diffuse nonspecific regions of steal; and major cerebral vessels of arterial supply or venous drainage. Each category of signal has its own finger print in frequency, amplitude, and phase. Finally, we put forward the hypothesis that modulations in blood flow are not only the consequence but are also the cause of modulations in functional activity.

Selective visual and auditory attention toward utterances-a PET study

The purpose of this study was to reveal functional areas of the brain modulating processing of selective auditory or visual attention toward utterances. Regional cerebral blood flow was measured in six normal volunteers using positron emission tomography during two selective attention tasks and a control condition. The auditory task activated the auditory, inferior parietal, prefrontal, and anterior cingulate cortices. The visual task activated the visual association, inferior parietal, and prefrontal cortices. Both conditions activated the same area in the superior temporal sulcus. During the visual task, deactivation was observed in the auditory cortex. These results indicate that there exists a modality-dependent selective attention mechanism which activates or deactivates cortical areas in different ways.

Human cortical areas activated in relation to vergence eye movements-a PET study

Human cortical areas activated in relation to vergence eye movements were determined using positron emission tomography. Binocular disparity-driven visual stimuli were presented using a head-mounted display. Eye movements were monitored continuously by an infrared limbus tracker. A combination of a bar and a cross was used as the target. In the vergence task, subjects were instructed to follow an approaching bar, while ignoring a stationary cross. Activation in relation to vergence eye movement was discriminated from activation in relation to motion vision by using the ignore-bar task as the control. In the ignore-bar task, subjects were instructed to fixate on a stationary cross, while ignoring an approaching bar. The fixation task was used as the basic control for both the vergence and the ignore-bar tasks. Areas of activation in relation to vergence eye movements were found in the bilateral temporooccipital junction, the left inferior parietal lobule, and the right fusiform gyrus by comparing regional cerebral flow between the vergence and ignore-bar tasks and by the conjunctive analyses of vergence-vs-ignore comparison with vergence-vs-fixation comparison.

Human face perception traced by magneto- and electro-encephalography

The temporal and spatial processing of face perception in normal subjects was traced by magnetoencephalography (MEG) and electroencephalography (EEG). We used 5 different visual stimuli: (1) face with opened eyes, (2) face with closed eyes, (3) eyes, (4) scrambled face, and (5) hand, and they were shown in random order. Subjects were asked to count the number of hand stimuli. To analyze the complicated brain responses to visual stimuli, we used brain electric source analysis (BESA) as the spatio-temporal multiple source model. In MEG recording, the 1M and 2M components were identified in all subjects. The 1M component was recorded to all kinds of stimuli. The 2M component was clearly identified only to face stimulation in all subjects, but to eyes stimulation in only 3 subjects with a small amplitude. The 2M component was not identified to scrambled face nor hand stimulation. The 2M component was recorded from the right hemisphere in all subjects, but in only 5 of 10 subjects from the left hemisphere. The mean peak latencies of the 1M and 2M components were approximately 132 and 179 ms, respectively. The interpeak latency between 1M and 2M was approximately 47 ms on average but the interindividual difference was large. There was no significant difference of the 2M latency between face with opened eyes and face with closed eyes. The 1M component was generated in the primary visual cortex in the bilateral hemispheres, and the 2M component was generated in the inferior temporal cortex, around the fusiform gyrus. In the EEG recording, face-specific components, positive at the vertex, P200 (Cz), and the negative at the temporal areas, N190 (T5') and N190 (T6'), were clearly recorded. The EEG results were fundamentally compatible with the MEG results. The amplitude of the component recorded from the right hemisphere was significantly larger than that from the left hemisphere. These findings suggest that the fusiform gyrus is considered to play an important role in face perception in humans, and that the right hemisphere is more dominant. Face perception takes place approximately 47 ms after the primary response to visual stimulation in the primary visual cortex, but the period of information transfer to the fusiform gyrus is variable among subjects. Detailed temporal and spatial analyses of the processing of face perception can be achieved with MEG.

PET study of the human foveal fixation system

Positron emission tomography (PET) was used to investigate the functional anatomy of the foveal fixation system in 10 subjects scanned under three different conditions: at rest (REST), during the fixation of a central point (FIX), and while fixating the same foveal target during the presentation of peripheral visual distractors (DIS). Compared with the REST condition, both FIX and DIS tasks activated a common set of cortical areas. First, in addition to the involvement of the occipital visual cortex, both the frontal eye field (FEF) and the intraparietal sulcus (IPS) were bilaterally activated. Right frontal activation was also found in the dorsolateral prefrontal cortex, the inferior part of the precentral gyrus, and the inferior frontal gyrus. These results suggest that both FEF and IPS may constitute the main cortical regions subserving bilaterally the foveal fixation system in humans. The remaining right frontal activations may be considered as part of the anterior attentional network, supporting a role for the right frontal lobe in the allocation of the attentional mechanisms. Compared with the FIX condition, the DIS task also revealed the perceptual and cognitive processes related to the presence of peripheral visual distractors during foveal fixation. In addition to a bilateral activation of the V5/MT motion-sensitive area, a right FEF-IPS network was activated which may correspond to the engagement of the visuospatial attention. Finally, normalized regional cerebral blood flow (NrCBF) decreases were also observed during both DIS and FIX condition performance. Such NrCBF decreases were centered in the superior and middle temporal gyri, the prefrontal cortex, and the precuneus and the posterior retrosplenial part of the cingulate gyrus.

Differentiating cortical areas related to pain perception from stimulus identification: temporal analysis of fMRI activity

In a recent functional magnetic resonance imaging study (fMRI), we reported the cortical areas activated in a thermal painful task and compared the extent of overlap between this cortical network and those activated during a vibrotactile task and a motor task. In the present study we examine the temporal properties of the cortical activations for all three tasks and use linear systems identification techniques to functionally differentiate the cortical regions identified in the painful thermal task. Cortical activity was examined in the contralateral middle third of the brain of 10 right-handed subjects, using echo-planar imaging and a surface coil. In another eight subjects the temporal properties of the thermal task were examined psychophysically. The fMRI impulse response function was estimated from the cortical activations in the vibrotactile and motor tasks and shown to correspond to earlier reports. Given the fMRI impulse response function and the time courses for the thermal stimulus and the associated pain ratings, predictor functions were generated. The correlation between these predictor functions and cortical activations in the painful thermal task indicated a gradual transition of information processing anteroposteriorly in the parietal cortex. Within this region, activity in the anterior areas more closely reflected thermal stimulus parameters, whereas activity more posteriorly was better related to the temporal properties of pain perception. Insular cortex at the level of the anterior commissure was the region best related to the thermal stimulus, and Brodmann's area 5/7 was the region best related to the pain perception. The functional implications of these observations are discussed.

A direct comparison between whole-brain PET and BOLD fMRI measurements of single-subject activation response

We present the results of a direct comparison of single-subject activation using identical tasks for both functional PET and fMRI whole-brain studies. We examined the most commonly employed methods for each modality. For fMRI this is the blood oxygenation level-dependent (BOLD) contrast method with echo-planar imaging. In PET single-subject activation studies are based on the development of high sensitivity 3D imaging of regional cerebral blood flow from multiple [15O]water injections. The identical activation paradigm of a visually cued sequential finger opposition was used for PET and fMRI. For both modalities the entire brain volume difference images were smoothed to the same final resolution and the peak t value within the primary sensory/motor (PSM) area was then identified. All contiguous voxels in the PSM above a predetermined threshold of statistical significance were determined. Finally, the difference-weighted centroid location was calculated for the PSM region for each modality. These studies showed a very similar pattern of activation, with the volume of activation greater in fMRI and higher levels of statistical significance. The centroids of activation, however, differed by 9 +/- 3 mm between the modalities, with the fMRI centroid location dorsal to that for PET. These results were stable across all processing options including differing levels of image smoothing and thresholds of statistical significance. These results are consistent with the hypothesis that draining veins contribute a substantial signal for fMRI activation studies and indicate caution for the interpretation of BOLD fMRI images with activation sites near draining veins.

Brain waves following remembered faces index conscious recollection

At a glance, one can often determine whether a face belongs to a known individual. To investigate brain mechanisms underlying this memory feat, we recorded EEG signals time-locked to face presentations. In the study phase, 40 unknown faces were presented, 20 of which were accompanied by a voice simulating that person speaking. Instructions were to remember the faces with spoken biographical information (R-faces) and to forget the others (F-faces). In the test phase, famous and non-famous faces were presented in a visually degraded manner. Subjects made two-choice fame judgments and priming was observed in the form of faster and more accurate responses for old than for new non-famous faces. Priming did not differ between R-faces and F-faces. In a second experiment, faces were not degraded at test and behavioral responses were made only when faces were presented twice in immediate succession. Brain potentials elicited 300 to 900 ms after stimulus onset from frontal and parieto-occipital scalp regions were larger for R-faces than for F-faces. Recognition tested later was more accurate for R-faces than for F-faces. Because the study-phase manipulation influenced recognition but not priming, we conclude that this procedure succeeded in isolating neural correlates of recollective processing from more automatic uses of face memory as indexed by priming.

Interictal spikes increase cerebral glucose metabolism and blood flow: a PET study

PURPOSE

In patients with reflex epilepsy, it is sometimes possible to evoke interictal spikes predictably, thus providing an uncommon but important experimental paradigm for examining the physiological changes produced by epileptiform discharges.

METHODS

To examine the changes in regional cerebral blood flow (rCBF) and glucose consumption (rCMRglc) produced by interictal spikes, we performed positron emission tomography (PET) scans with the blood-flow tracer [15O]H20 and with [18F]fluorodeoxyglucose in a patient with fixation-off epilepsy. The scans were performed in states of high and low spike frequency produced by eye closure and opening, respectively.

RESULTS

The rCBF study revealed a focal increase in blood flow associated with the state of increased interictal spiking. The focus was in the posterior portion of the left superior parietal lobule (Talairach coordinates: x: -36, y: -71, z: 39; t = 4.5; p<0.05) and corresponded to the site of maximal ictal EEG abnormality recorded with implanted electrodes. In a volume of interest of 10-mm diameter centered on the t statistic peak in the rCBF study, the mean rCMRglc was 39.1 micromol/100 g/min with eyes open and 44.1 micromol/100 g/min (13% increase) with eyes closed. An identical activation paradigm was used in six normal subjects studied with functional magnetic resonance imaging. In the normal subjects, no significant activation was observed in the parieto-occipital region, indicating that the changes observed in the patient were due to interictal spiking rather than to task performance alone.

CONCLUSIONS

Interictal spiking produces focal increases in cerebral blood flow and glucose metabolism.

Searching for face-specific long latency ERPs: a topographic study of effects associated with mismatching features

In a previous study [E. Olivares, M.A. Bobes, E. Aubert, M. Valdés-Sosa, Associative ERPs effects with memories of artificial faces, Cogn. Brain Res. 2 (1994) 39-48] we reported the presence of a negativity associated with mismatching features when subjects carried out a face-feature matching task whilst their evoked potentials were recorded. Since the stimuli used were learned faces (realistic drawings), for which the subjects possessed no semantic information or associated verbal labels, the mismatch negativity obtained was considered a face-specific N400. In this work we present a new experiment to study the topographic distribution of these mismatch effects. As in the above-mentioned study, in each trial the subjects observed previously an incomplete (without the eyes/eyebrows fragment) familiar face, which served as a structural context for the face recognition. The face was then completed by grafting either matching (learned) features or mismatching features (from another face). In line with neuropsychological studies on prosopagnosia and electrophysiological findings in humans and non-human primates, we found as one of the most relevant items of data that the most-posterior (principally, left occipital) cortices appear to be a region in which are located the possible neural generators of the negativity associated with the detection of incongruencies in the structure of familiar faces. We also reported a late positivity, distributed in more anterior regions, which follows the mismatch negativity. This complex N-P is interpreted as reflecting a dual process of retrieval and integration of information in memory.

The effect of face inversion on activity in human neural systems for face and object perception

The differential effect of stimulus inversion on face and object recognition suggests that inverted faces are processed by mechanisms for the perception of other objects rather than by face perception mechanisms. We investigated the face inversion using functional magnetic resonance imaging (fMRI). The principal effect of face inversion on was an increased response in ventral extrastriate regions that respond preferentially to another class of objects (houses). In contrast, house inversion did not produce a similar change in face-selective regions. Moreover, stimulus inversion had equivalent, minimal effects for faces in in face-selective regions and for houses in house-selective regions. The results suggest that the failure of face perception systems with inverted faces leads to the recruitment of processing resources in object perception systems, but this failure is not reflected by altered activity in face perception systems.

Spatio-temporal dynamics of attention to color: evidence from human electrophysiology

This study characterized patterns of brain electrical activity associated with selective attention to the color of a stimulus. Multichannel recordings of event-related potentials (ERPs) were obtained while subjects viewed randomized sequences of checkerboards consisting of isoluminant red or blue checks superimposed on a grey background. Stimuli were presented foveally at a rapid rate, and subjects were required to attend to the red or blue checks in separate blocks of trials and to press a button each time they detected a dimmer target stimulus of the attended color. An early negative ERP component with an onset latency of 50 ms was sensitive to stimulus color but was unaffected by the attentional manipulation. ERPs elicited by attended and unattended stimuli began to diverge after approximately 100 ms following stimulus onset. Inverse dipole modelling of the attended-minus-unattended difference waveform indicated that an initial positive deflection with an onset latency of 100 ms had a source in lateral occipital cortex, while a subsequent negative deflection with an onset at 160 ms had a source in inferior occipito-temporal cortex. Longer-latency attention-sensitive components were localized to premotor frontal areas (onset at 190 ms) and to more anterior regions of the fusiform gyrus (onset at 240 ms). These source localizations correspond closely with cortical areas that were identified in previous neuroimaging studies as being involved in color-selective processing. The present ERP data thus provide information about the time course of stimulus selection processes in cortical areas that subserve attention to color.

The role of prefrontal cortex in working memory: examining the contents of consciousness

Working memory enables us to hold in our 'mind's eye' the contents of our conscious awareness, even in the absence of sensory input, by maintaining an active representation of information for a brief period of time. In this review we consider the functional organization of the prefrontal cortex and its role in this cognitive process. First, we present evidence from brain-imaging studies that prefrontal cortex shows sustained activity during the delay period of visual working memory tasks, indicating that this cortex maintains on-line representations of stimuli after they are removed from view. We then present evidence for domain specificity within frontal cortex based on the type of information, with object working memory mediated by more ventral frontal regions and spatial working memory mediated by more dorsal frontal regions. We also propose that a second dimension for domain specificity within prefrontal cortex might exist for object working memory on the basis of the type of representation, with analytic representations maintained preferentially in the left hemisphere and image-based representations maintained preferentially in the right hemisphere. Furthermore, we discuss the possibility that there are prefrontal areas brought into play during the monitoring and manipulation of information in working memory in addition to those engaged during the maintenance of this information. Finally, we consider the relationship of prefrontal areas important for working memory, both to posterior visual processing areas and to prefrontal areas associated with long-term memory.

Investigation of facial recognition memory and happy and sad facial expression perception: an fMRI study

We investigated facial recognition memory (for previously unfamiliar faces) and facial expression perception with functional magnetic resonance imaging (fMRI). Eight healthy, right-handed volunteers participated. For the facial recognition task, subjects made a decision as to the familiarity of each of 50 faces (25 previously viewed; 25 novel). We detected signal increase in the right middle temporal gyrus and left prefrontal cortex during presentation of familiar faces, and in several brain regions, including bilateral posterior cingulate gyri, bilateral insulae and right middle occipital cortex during presentation of unfamiliar faces. Standard facial expressions of emotion were used as stimuli in two further tasks of facial expression perception. In the first task, subjects were presented with alternating happy and neutral faces; in the second task, subjects were presented with alternating sad and neutral faces. During presentation of happy facial expressions, we detected a signal increase predominantly in the left anterior cingulate gyrus, bilateral posterior cingulate gyri, medial frontal cortex and right supramarginal gyrus, brain regions previously implicated in visuospatial and emotion processing tasks. No brain regions showed increased signal intensity during presentation of sad facial expressions. These results provide evidence for a distinction between the neural correlates of facial recognition memory and perception of facial expression but, whilst highlighting the role of limbic structures in perception of happy facial expressions, do not allow the mapping of a distinct neural substrate for perception of sad facial expressions.

Brain regions involved in the perception of gaze: a PET study

Mutual gaze may be described as a psychological process during which two persons have the feeling of a brief link between their two minds. In the monkey, specific cell assemblies in the superior temporal cortex of the brain are responsive to gaze. This suggests that the brain may have evolved mechanisms for interpreting direct eye contact. These mechanisms could depend on the activation of specific brain regions. Positron emission tomography was used to measure activity in brain regions in healthy volunteers while they were looking at faces featuring, respectively, eye contact, averted gaze, or no gaze. As expected a region known to be involved in face processing was found to be activated in the ventral occipito-temporal region, especially in the right hemisphere. Averted gaze and mutual gaze triggered blood flow responses in similar areas which were different from those involved in face processing. These areas included the occipital part of the fusiform gyrus, the right parietal lobule, the right inferior temporal gyrus, and the middle temporal gyrus in both hemispheres. These results are consistent with the hypothesis that perception of eyes regardless of the direction of the gaze is subserved by a distributed network. However, no conclusive evidence was found for specific area(s) devoted to mutual gaze processing.

fMRI study of face perception and memory using random stimulus sequences

A new functional magnetic resonance imaging (fMRI) method was used to investigate the functional neuroanatomy of face perception and memory. Whole-brain fMRI data were acquired while four types of stimuli were presented sequentially in an unpredictable pseudorandom order at a rate of 0.5 Hz. Stimulus types were a single repeated memorized target face, unrepeated novel faces, nonsense scrambled faces, and a blank screen. Random stimulus sequences were designed to generate a functional response to each stimulus type that was uncorrelated with responses to other stimuli. This allowed fMRI responses to each stimulus type to be examined separately using multiple regression. Signal increases were found for all stimuli in ventral posterior cortex. Responses to intact faces extended to more anterior locations of occipitotemporal cortex than did responses to scrambled faces, consistent with previous studies of face perception. Responses evoked by novel faces were in regions of ventral occipitotemporal cortex medial to regions in which significant responses were evoked by the target face. The repeated target face stimulus also evoked activity in widely distributed regions of frontal and parietal cortex. These results demonstrate that cortical hemodynamic responses to interleaved novel and repeated stimuli can be distinguished and measured using fMRI with appropriate stimulus sequences and data analysis methods. This method can now be used to examine the neural systems involved in cognitive tasks that were previously impossible to study using positron emission tomography or fMRI.

Temporal cortex activation in humans viewing eye and mouth movements

We sought to determine whether regions of extrastriate visual cortex could be activated in subjects viewing eye and mouth movements that occurred within a stationary face. Eleven subjects participated in three to five functional magnetic resonance imaging sessions in which they viewed moving eyes, moving mouths, or movements of check patterns that occurred in the same spatial location as the eyes or mouth. In each task, the stimuli were superimposed on a radial background pattern that continually moved inward to control for the effect of movement per se. Activation evoked by the radial background was assessed in a separate control task. Moving eyes and mouths activated a bilateral region centered in the posterior superior temporal sulcus (STS). The moving check patterns did not appreciably activate the STS or surrounding regions. The activation by moving eyes and mouths was distinct from that elicited by the moving radial background, which primarily activated the posterior-temporal-occipital fossa and the lateral occipital sulcus-a region corresponding to area MT/V5. Area MT/V5 was also strongly activated by moving eyes and to a lesser extent by other moving stimuli. These results suggest that a superior temporal region centered in the STS is preferentially involved in the perception of gaze direction and mouth movements. This region of the STS may be functionally related to nearby superior temporal regions thought to be involved in lip-reading and in the perception of hand and body movement.

Perceiving visually presented objects: recognition, awareness, and modularity

Object perception may involve seeing, recognition, preparation of actions, and emotional responses--functions that human brain imaging and neuropsychology suggest are localized separately. Perhaps because of this specialization, object perception is remarkably rapid and efficient. Representations of componential structure and interpolation from view-dependent images both play a part in object recognition. Unattended objects may be implicitly registered, but recent experiments suggest that attention is required to bind features, to represent three-dimensional structure, and to mediate awareness.