Behaviorally Relevant Abstract Object Identity Representation in the Human Parietal Cortex

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

The large-scale organization of shape processing in the ventral and dorsal pathways is dissociable from attention

The two visual pathways model posits that visual information is processed through two distinct cortical systems: The ventral pathway promotes visual recognition, while the dorsal pathway supports visuomotor control. Recent evidence suggests the dorsal pathway is also involved in shape processing and may contribute to object perception, but it remains unclear whether this sensitivity is independent of attentional mechanisms that were localized to overlapping cortical regions. To address this question, we conducted two fMRI experiments that utilized different parametric scrambling manipulations in which human participants viewed novel objects in different levels of scrambling and were instructed to attend to either the object or to another aspect of the image (e.g. color of the background). Univariate and multivariate analyses revealed that the large-scale organization of shape selectivity along the dorsal and ventral pathways was preserved regardless of the focus of attention. Attention did modulate shape sensitivity, but these effects were similar across the two pathways. These findings support the idea that shape processing is at least partially dissociable from attentional processes and relies on a distributed set of cortical regions across the visual pathways.

Developing cortex is functionally pluripotent: Evidence from blindness

How rigidly does innate architecture constrain function of developing cortex? What is the contribution of early experience? We review insights into these questions from visual cortex function in people born blind. In blindness, occipital cortices are active during auditory and tactile tasks. What 'cross-modal' plasticity tells us about cortical flexibility is debated. On the one hand, visual networks of blind people respond to higher cognitive information, such as sentence grammar, suggesting drastic repurposing. On the other, in line with 'metamodal' accounts, sighted and blind populations show shared domain preferences in ventral occipito-temporal cortex (vOTC), suggesting visual areas switch input modality but perform the same or similar perceptual functions (e.g., face recognition) in blindness. Here we bring these disparate literatures together, reviewing and synthesizing evidence that speaks to whether visual cortices have similar or different functions in blind and sighted people. Together, the evidence suggests that in blindness, visual cortices are incorporated into higher-cognitive (e.g., fronto-parietal) networks, which are a major source long-range input to the visual system. We propose the connectivity-constrained experience-dependent account. Functional development is constrained by innate anatomical connectivity, experience and behavioral needs. Infant cortex is pluripotent, the same anatomical constraints develop into different functional outcomes.

Early visual cortices reveal interrelated item and category representations in aging

Neural dedifferentiation, the finding that neural representations tend to be less distinct in older adults compared with younger adults, has been associated with age-related declines in memory performance. Most studies assessing the relation between memory and neural dedifferentiation have evaluated how age impacts the distinctiveness of neural representations for different visual categories (e.g., scenes and objects). However, how age impacts the quality of neural representations at the level of individual items is still an open question. Here, we present data from an age-comparative fMRI study that aimed to understand how the distinctiveness of neural representations for individual stimuli differs between younger and older adults and relates to memory outcomes. Pattern similarity searchlight analyses yielded indicators of neural dedifferentiation at the level of individual items as well as at the category level in posterior occipital cortices. We asked whether age differences in neural distinctiveness at each representational level were associated with inter- and/or intraindividual variability in memory performance. While age-related dedifferentiation at both the item and category level related to between-person differences in memory, neural distinctiveness at the category level also tracked within-person variability in memory performance. Concurrently, neural distinctiveness at the item level was strongly associated with neural distinctiveness at the category level both within and across participants, elucidating a potential representational mechanism linking item- and category-level distinctiveness. In sum, we provide evidence that age-related neural dedifferentiation co-exists across multiple representational levels and is related to memory performance.Significance Statement Age-related memory decline has been associated with neural dedifferentiation, the finding that older adults have less distinctive neural representations than younger adults. This has been mostly shown for category information, while evidence for age differences in the specificity of item representations is meager. We used pattern similarity searchlight analyses to find indicators of neural dedifferentiation at both levels of representation (category and item) and linked distinctiveness to memory performance. Both item- and category-level dedifferentiation in the calcarine cortex were related to interindividual differences in memory performance, while category-level distinctiveness further tracked intraindividual variability. Crucially, neural distinctiveness was strongly tied between the item and category levels, indicating that intersecting representational properties of posterior occipital cortices reflect both individual exemplars and categories.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Information processing in the dorsal pathway: from "where" to "what we do with it"

Visual processing in the brain has been understood as the ventral and dorsal pathways processing "what" and "where" information, respectively. Mocz et al. (Mocz V, Vaziri-Pashkam M, Chun M, Xu Y. J Cogn Neurosci 34: 2406-2435, 2022), however, report that the two pathways code object features in a parallel manner. These results support that information processing in the dorsal pathway is not strictly limited to "where" and that the two pathways work in parallel to process task-relevant information ("what we do with it").

Overwhelmed by the man in the moon? Pareidolic objects provoke increased amygdala activation in autism

An interesting feature of the primate face detection system results in the perception of illusory faces in objects, or pareidolia. These illusory faces do not per se contain social information, such as eye-gaze or specific identities, yet they activate the cortical brain face-processing network, possibly via the subcortical route, including the amygdala. In autism spectrum disorder (ASD), aversion to eye-contact is commonly reported, and so are alterations in face processing more generally, yet the underlying reasons are not clear. Here we show that in autistic participants (N=37), but not in non-autistic controls (N=34), pareidolic objects increase amygdala activation bilaterally (right amygdala peak: X = 26, Y = -6, Z = -16; left amygdala peak X = -24, Y = -6, Z = -20). In addition, illusory faces engage the face-processing cortical network significantly more in ASD than in controls. An early imbalance in the excitatory and inhibitory systems in autism, affecting typical brain maturation, may be at the basis of an overresponsive reaction to face configuration and to eye contact. Our data add to the evidence of an oversensitive subcortical face processing system in ASD.

Temporal asymmetries and interactions between dorsal and ventral visual pathways during object recognition

Despite their anatomical and functional distinctions, there is growing evidence that the dorsal and ventral visual pathways interact to support object recognition. However, the exact nature of these interactions remains poorly understood. Is the presence of identity-relevant object information in the dorsal pathway simply a byproduct of ventral input? Or, might the dorsal pathway be a source of input to the ventral pathway for object recognition? In the current study, we used high-density EEG-a technique with high temporal precision and spatial resolution sufficient to distinguish parietal and temporal lobes-to characterise the dynamics of dorsal and ventral pathways during object viewing. Using multivariate analyses, we found that category decoding in the dorsal pathway preceded that in the ventral pathway. Importantly, the dorsal pathway predicted the multivariate responses of the ventral pathway in a time-dependent manner, rather than the other way around. Together, these findings suggest that the dorsal pathway is a critical source of input to the ventral pathway for object recognition.

Preparatory attention to visual features primarily relies on non-sensory representation

Prior knowledge of behaviorally relevant information promotes preparatory attention before the appearance of stimuli. A key question is how our brain represents the attended information during preparation. A sensory template hypothesis assumes that preparatory signals evoke neural activity patterns that resembled the perception of the attended stimuli, whereas a non-sensory, abstract template hypothesis assumes that preparatory signals reflect the abstraction of attended stimuli. To test these hypotheses, we used fMRI and multivariate analysis to characterize neural activity patterns when human participants were prepared to attend a feature and then select it from a compound stimulus. In an fMRI experiment using basic visual feature (motion direction), we observed reliable decoding of the to-be-attended feature from the preparatory activity in both visual and frontoparietal areas. However, while the neural patterns constructed by a single feature from a baseline task generalized to the activity patterns during stimulus selection, they could not generalize to the activity patterns during preparation. Our findings thus suggest that neural signals during attentional preparation are predominantly non-sensory in nature that may reflect an abstraction of the attended feature. Such a representation could provide efficient and stable guidance of attention.

The transition from invariant to action-dependent visual object representation in human dorsal pathway

The human brain can efficiently process action-related visual information, which supports our ability to quickly understand and learn others' actions. The visual information of goal-directed action is extensively represented in the parietal and frontal cortex, but how actions and goal-objects are represented within this neural network is not fully understood. Specifically, which part of this dorsal network represents the identity of goal-objects? Is such goal-object information encoded at an abstract level or highly interactive with action representations? Here, we used functional magnetic resonance imaging with a large number of participants (n = 94) to investigate the neural representation of goal-objects and actions when participants viewed goal-directed action videos. Our results showed that the goal-directed action information could be decoded across much of the dorsal pathway, but in contrast, the invariant goal-object information independent of action was mainly localized in the early stage of dorsal pathway in parietal cortex rather than the down-stream areas of the parieto-frontal cortex. These results help us to understand the relationship between action and goal-object representations in the dorsal pathway, and the evolution of interactive representation of goal-objects and actions along the dorsal pathway during goal-directed action observation.

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Predicting Identity-Preserving Object Transformations in Human Posterior Parietal Cortex and Convolutional Neural Networks

Previous research shows that, within human occipito-temporal cortex (OTC), we can use a general linear mapping function to link visual object responses across nonidentity feature changes, including Euclidean features (e.g., position and size) and non-Euclidean features (e.g., image statistics and spatial frequency). Although the learned mapping is capable of predicting responses of objects not included in training, these predictions are better for categories included than those not included in training. These findings demonstrate a near-orthogonal representation of object identity and nonidentity features throughout human OTC. Here, we extended these findings to examine the mapping across both Euclidean and non-Euclidean feature changes in human posterior parietal cortex (PPC), including functionally defined regions in inferior and superior intraparietal sulcus. We additionally examined responses in five convolutional neural networks (CNNs) pretrained with object classification, as CNNs are considered as the current best model of the primate ventral visual system. We separately compared results from PPC and CNNs with those of OTC. We found that a linear mapping function could successfully link object responses in different states of nonidentity transformations in human PPC and CNNs for both Euclidean and non-Euclidean features. Overall, we found that object identity and nonidentity features are represented in a near-orthogonal, rather than complete-orthogonal, manner in PPC and CNNs, just like they do in OTC. Meanwhile, some differences existed among OTC, PPC, and CNNs. These results demonstrate the similarities and differences in how visual object information across an identity-preserving image transformation may be represented in OTC, PPC, and CNNs.

Functional imaging analyses reveal prototype and exemplar representations in a perceptual single-category task

Similarity-based categorization can be performed by memorizing category members as exemplars or by abstracting the central tendency of the category – the prototype. In similarity-based categorization of stimuli with clearly identifiable dimensions from two categories, prototype representations were previously located in the hippocampus and the ventromedial prefrontal cortex (vmPFC) and exemplar representations in areas supporting visual memory. However, the neural implementation of exemplar and prototype representations in perceptual similarity-based categorization of single categories is unclear. To investigate these representations, we applied model-based univariate and multivariate analyses of functional imaging data from a dot-pattern paradigm-based task. Univariate prototype and exemplar representations occurred bilaterally in visual areas. Multivariate analyses additionally identified prototype representations in parietal areas and exemplar representations in the hippocampus. Bayesian analyses supported the non-presence of prototype representations in the hippocampus and the vmPFC. We additionally demonstrate that some individuals form both representation types simultaneously, probably granting flexibility in categorization strategies.

Model-based univariate and multivariate analyses of fMRI data from 62 healthy participants in a dot-pattern paradigm-based task provide further insight into the neural basis of similarity-based categorization.

View Normalization of Object Size in the Right Parietal Cortex

Prior knowledge alters perception already on early levels of processing. For instance, judging the display size of an object is affected by its familiar size. Using functional magnetic resonance imaging, we investigated the neural processes involved in resolving ambiguities between familiar object size and physical object size in 33 healthy human subjects. The familiar size was either small or large, and the object was displayed as either small or large. Thus, the size of the displayed object was either congruent or incongruent with its internally stored canonical size representation. Subjects were asked to indicate where the stimuli appeared on the screen as quickly and accurately as possible, thereby ensuring that differential activations cannot be ascribed to explicit object size judgments. Incongruent (relative to congruent) object displays were associated with enhanced activation of the right intraparietal sulcus (IPS). These data are consistent with but extend previous patient studies, which found the right parietal cortex involved in matching visual objects presented atypically to prototypical object representations, suggesting that the right IPS supports view normalization of objects. In a second experiment, using a parametric design, a region-of-interest analysis supported this notion by showing that increases in size mismatch between the displayed size of an object and its familiar viewing size were associated with an increased right IPS activation. We conclude that the right IPS performs view normalization of mismatched information about the internally stored prototypical size and the current viewing size of an object.

The Dorsal Visual Pathway Represents Object-Centered Spatial Relations for Object Recognition

Although there is mounting evidence that input from the dorsal visual pathway is crucial for object processes in the ventral pathway, the specific functional contributions of dorsal cortex to these processes remain poorly understood. Here, we hypothesized that dorsal cortex computes the spatial relations among an object's parts, a process crucial for forming global shape percepts, and transmits this information to the ventral pathway to support object categorization. Using fMRI with human participants (females and males), we discovered regions in the intraparietal sulcus (IPS) that were selectively involved in computing object-centered part relations. These regions exhibited task-dependent functional and effective connectivity with ventral cortex, and were distinct from other dorsal regions, such as those representing allocentric relations, 3D shape, and tools. In a subsequent experiment, we found that the multivariate response of posterior (p)IPS, defined on the basis of part-relations, could be used to decode object category at levels comparable to ventral object regions. Moreover, mediation and multivariate effective connectivity analyses further suggested that IPS may account for representations of part relations in the ventral pathway. Together, our results highlight specific contributions of the dorsal visual pathway to object recognition. We suggest that dorsal cortex is a crucial source of input to the ventral pathway and may support the ability to categorize objects on the basis of global shape.SIGNIFICANCE STATEMENT Humans categorize novel objects rapidly and effortlessly. Such categorization is achieved by representing an object's global shape structure, that is, the relations among object parts. Yet, despite their importance, it is unclear how part relations are represented neurally. Here, we hypothesized that object-centered part relations may be computed by the dorsal visual pathway, which is typically implicated in visuospatial processing. Using fMRI, we identified regions selective for the part relations in dorsal cortex. We found that these regions can support object categorization, and even mediate representations of part relations in the ventral pathway, the region typically thought to support object categorization. Together, these findings shed light on the broader network of brain regions that support object categorization.

Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Regional Haemodynamic and Metabolic Coupling in Infants

Metabolic pathways underlying brain function remain largely unexplored during neurodevelopment, predominantly due to the lack of feasible techniques for use with awake infants. Broadband near-infrared spectroscopy (bNIRS) provides the opportunity to explore the relationship between cerebral energy metabolism and blood oxygenation/haemodynamics through the measurement of changes in the oxidation state of mitochondrial respiratory chain enzyme cytochrome-c-oxidase (ΔoxCCO) alongside haemodynamic changes. We used a bNIRS system to measure ΔoxCCO and haemodynamics during functional activation in a group of 42 typically developing infants aged between 4 and 7 months. bNIRS measurements were made over the right hemisphere over temporal, parietal and central cortical regions, in response to social and non-social visual and auditory stimuli. Both ΔoxCCO and Δ[HbO2] displayed larger activation for the social condition in comparison to the non-social condition. Integration of haemodynamic and metabolic signals revealed networks of stimulus-selective cortical regions that were not apparent from analysis of the individual bNIRS signals. These results provide the first spatially resolved measures of cerebral metabolic activity alongside haemodynamics during functional activation in infants. Measuring synchronised changes in metabolism and haemodynamics have the potential for uncovering the development of cortical specialisation in early infancy.

Representation of Color, Form, and their Conjunction across the Human Ventral Visual Pathway

Despite decades of research, our understanding of the relationship between color and form processing in the primate ventral visual pathway remains incomplete. Using fMRI multivoxel pattern analysis, we examined coding of color and form, using a simple form feature (orientation) and a mid-level form feature (curvature), in human ventral visual processing regions. We found that both color and form could be decoded from activity in early visual areas V1 to V4, as well as in the posterior color-selective region and shape-selective regions in ventral and lateral occipitotemporal cortex defined based on their univariate selectivity to color or shape, respectively (the central color region only showed color but not form decoding). Meanwhile, decoding biases towards one feature or the other existed in the color- and shape-selective regions, consistent with their univariate feature selectivity reported in past studies. Additional extensive analyses show that while all these regions contain independent (linearly additive) coding for both features, several early visual regions also encode the conjunction of color and the simple, but not the complex, form feature in a nonlinear, interactive manner. Taken together, the results show that color and form are encoded in a biased distributed and largely independent manner across ventral visual regions in the human brain.

A Distributed Model of Face and Body Integration

Separated face- and body-responsive brain networks have been identified that show strong responses when observers view faces and bodies. It has been proposed that face and body processing may be initially separated in the lateral occipitotemporal cortex and then combined into a whole person representation in the anterior temporal cortex, or elsewhere in the brain. However, in contrast to this proposal, our recent study identified a common coding of face and body orientation (ie, facing direction) in the lateral occipitotemporal cortex, demonstrating an integration of face and body information at an early stage of face and body processing. These results, in combination with findings that show integration of face and body identity in the lateral occipitotemporal, parahippocampal and superior parietal cortex, and face and body emotional expression in the posterior superior temporal sulcus and medial prefrontal cortex, suggest that face and body integration may be more distributed than previously considered. I propose a new model of face and body integration, where areas at the intersection of face- and body-responsive regions play a role in integrating specific properties of faces and bodies, and distributed regions across the brain contribute to high-level, abstract integration of shared face and body properties.

Spatially Adjacent Regions in Posterior Cingulate Cortex Represent Familiar Faces at Different Levels of Complexity

Extensive research has shown that perceptual information of faces is processed in a network of hierarchically-organized areas within ventral temporal cortex. For familiar and famous faces, perceptual processing of faces is normally accompanied by extraction of semantic knowledge about the social status of persons. Semantic processing of familiar faces could entail progressive stages of information abstraction. However, the cortical mechanisms supporting multistage processing of familiar faces have not been characterized. Here, using an event-related fMRI experiment, familiar faces from four celebrity groups (actors, singers, politicians, and football players) and unfamiliar faces were presented to the human subjects (both males and females) while they were engaged in a face categorization task. We systematically explored the cortical representations for faces, familiar faces, subcategories of familiar faces, and familiar face identities using whole-brain univariate analysis, searchlight-based multivariate pattern analysis (MVPA), and functional connectivity analysis. Convergent evidence from all these analyses revealed a set of overlapping regions within posterior cingulate cortex (PCC) that contained decodable fMRI responses for representing different levels of semantic knowledge about familiar faces. Our results suggest a multistage pathway in PCC for processing semantic information of faces, analogous to the multistage pathway in ventral temporal cortex for processing perceptual information of faces.SIGNIFICANCE STATEMENT Recognizing familiar faces is an important component of social communications. Previous research has shown that a distributed network of brain areas is involved in processing the semantic information of familiar faces. However, it is not clear how different levels of semantic information are represented in the brain. Here, we evaluated the multivariate response patterns across the entire cortex to discover the areas that contain information for familiar faces, subcategories of familiar faces, and identities of familiar faces. The searchlight maps revealed that different levels of semantic information are represented in topographically adjacent areas within posterior cingulate cortex (PCC). The results suggest that semantic processing of faces is mediated through progressive stages of information abstraction in PCC.

Transformative neural representations support long-term episodic memory

Memory is often conceived as a dynamic process that involves substantial transformations of mental representations. However, the neural mechanisms underlying these transformations and their role in memory formation and retrieval have only started to be elucidated. Combining intracranial EEG recordings with deep neural network models, we provide a detailed picture of the representational transformations from encoding to short-term memory maintenance and long-term memory retrieval that underlie successful episodic memory. We observed substantial representational transformations during encoding. Critically, more pronounced semantic representational formats predicted better subsequent long-term memory, and this effect was mediated by more consistent item-specific representations across encoding events. The representations were further transformed right after stimulus offset, and the representations during long-term memory retrieval were more similar to those during short-term maintenance than during encoding. Our results suggest that memory representations pass through multiple stages of transformations to achieve successful long-term memory formation and recall.

Understanding occipital and parietal contributions to visual working memory: Commentary on Xu (2020)

In her commentary, Xu (2020) admonishes the reader that "To have a full understanding of the cognitive mechanisms underlying VWM [visual working memory], both behavioral and neural evidence needs to be taken into account. This is a must, and not a choice, for any study that attempts to capture the nature of VWM" (p. 11). Although we don't disagree with this statement, our overall assessment of this commentary is that it, itself, fails to satisfy several "musts" and, consequently, does not pose a serious challenge for the sensory recruitment framework for understanding visual working memory. These "musts" include accurately characterizing the framework being critiqued, not favoring verbal models and intuition at the expense of formal quantitative models, and providing even-handed interpretation of the work of others. We'll conclude with a summary of how the sensory recruitment framework can be incorporated into a broader working model of visual working memory.

Getting to Know You: Emerging Neural Representations during Face Familiarization

The successful recognition of familiar persons is critical for social interactions. Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. In three EEG experiments on human participants of both sexes, we elucidated how representations of face familiarity and identity emerge from different qualities of familiarization: brief perceptual exposure (Experiment 1), extensive media familiarization (Experiment 2), and real-life personal familiarization (Experiment 3). Time-resolved representational similarity analysis revealed that familiarization quality has a profound impact on representations of face familiarity: they were strongly visible after personal familiarization, weaker after media familiarization, and absent after perceptual familiarization. Across all experiments, we found no enhancement of face identity representation, suggesting that familiarity and identity representations emerge independently during face familiarization. Our results emphasize the importance of extensive, real-life familiarization for the emergence of robust face familiarity representations, constraining models of face perception and recognition memory.SIGNIFICANCE STATEMENT Despite extensive research on the neural representations of familiar faces, we know little about how such representations unfold as someone becomes familiar. To elucidate how face representations change as we get familiar with someone, we conducted three EEG experiments where we used brief perceptual exposure, extensive media familiarization, or real-life personal familiarization. Using multivariate representational similarity analysis, we demonstrate that the method of familiarization has a profound impact on face representations, and emphasize the importance of real-life familiarization. Additionally, familiarization shapes representations of face familiarity and identity differently: as we get to know someone, familiarity signals seem to appear before the formation of identity representations.

Separated and overlapping neural coding of face and body identity

Recognising a person's identity often relies on face and body information, and is tolerant to changes in low‐level visual input (e.g., viewpoint changes). Previous studies have suggested that face identity is disentangled from low‐level visual input in the anterior face‐responsive regions. It remains unclear which regions disentangle body identity from variations in viewpoint, and whether face and body identity are encoded separately or combined into a coherent person identity representation. We trained participants to recognise three identities, and then recorded their brain activity using fMRI while they viewed face and body images of these three identities from different viewpoints. Participants' task was to respond to either the stimulus identity or viewpoint. We found consistent decoding of body identity across viewpoint in the fusiform body area, right anterior temporal cortex, middle frontal gyrus and right insula. This finding demonstrates a similar function of fusiform and anterior temporal cortex for bodies as has previously been shown for faces, suggesting these regions may play a general role in extracting high‐level identity information. Moreover, we could decode identity across fMRI activity evoked by faces and bodies in the early visual cortex, right inferior occipital cortex, right parahippocampal cortex and right superior parietal cortex, revealing a distributed network that encodes person identity abstractly. Lastly, identity decoding was consistently better when participants attended to identity, indicating that attention to identity enhances its neural representation. These results offer new insights into how the brain develops an abstract neural coding of person identity, shared by faces and bodies.

Priming of the Sander Parallelogram illusion separates perception from action

The two-visual stream hypothesis posits that the dorsal stream is less susceptible than the ventral stream to the effects of illusions and visual priming. While previous studies have separately examined these perceptual manipulations, the present study combined the effects of a visual illusion and priming to examine the possibility of dorsally guided actions being susceptible to the perceptual stimuli due to interactions between the two streams. Thirty-four participants were primed with a 'long' or 'short' version of the Sander Parallelogram illusion and were asked to either reach out and grasp or manually estimate the length of a rod placed on a version of the illusion that was on some trials the same as the prime (congruent) and on other trials was the inverse (incongruent). Due to the context-focused nature of ventral processing, we predicted that estimations would be more susceptible to the effects of the illusion and priming than grasps. Results showed that while participants' manual estimations were susceptible to both priming and the illusion, the grasps were only affected by the illusion, not by priming. The influence of the illusion on grip aperture was greater during manual estimations than it was during grasping. These findings support the notion that the functionally distinct dorsal and ventral streams interact under the current experimental paradigm. Outcomes of the study help better understand the nature of stimuli that promote interactions between the dorsal and ventral streams.

Differential neurodynamics and connectivity in the dorsal and ventral visual pathways during perception of emotional crowds and individuals: a MEG study

Reading the prevailing emotion of groups of people ("crowd emotion") is critical to understanding their overall intention and disposition. It alerts us to potential dangers, such as angry mobs or panicked crowds, giving us time to escape. A critical aspect of processing crowd emotion is that it must occur rapidly, because delays often are costly. Although knowing the timing of neural events is crucial for understanding how the brain guides behaviors using coherent signals from a glimpse of multiple faces, this information is currently lacking in the literature on face ensemble coding. Therefore, we used magnetoencephalography to examine the neurodynamics in the dorsal and ventral visual streams and the periamygdaloid cortex to compare perception of groups of faces versus individual faces. Forty-six participants compared two groups of four faces or two individual faces with varying emotional expressions and chose which group or individual they would avoid. We found that the dorsal stream was activated as early as 68 msec after the onset of stimuli containing groups of faces. In contrast, the ventral stream was activated later and predominantly for individual face stimuli. The latencies of the dorsal stream activation peaks correlated with participants' response times for facial crowds. We also found enhanced connectivity earlier between the periamygdaloid cortex and the dorsal stream regions for crowd emotion perception. Our findings suggest that ensemble coding of facial crowds proceeds rapidly and in parallel by engaging the dorsal stream to mediate adaptive social behaviors, via a distinct route from single face perception.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Transforming the Concept of Memory Reactivation

Reactivation refers to the phenomenon wherein patterns of neural activity expressed during perceptual experience are re-expressed at a later time, a putative neural marker of memory. Reactivation of perceptual content has been observed across many cortical areas and correlates with objective and subjective expressions of memory in humans. However, because reactivation emphasizes similarities between perceptual and memory-based representations, it obscures differences in how perceptual events and memories are represented. Here, we highlight recent evidence of systematic differences in how (and where) perceptual events and memories are represented in the brain. We argue that neural representations of memories are best thought of as spatially transformed versions of perceptual representations. We consider why spatial transformations occur and identify critical questions for future research.

Revisit once more the sensory storage account of visual working memory

Recent work has highlighted the role of early visual areas in visual working memory (VWM) storage and put forward a sensory storage account of VWM. Using a distractor interference paradigm, however, we previolsy showed that the contribution of early visual areas to VWM storage may not be essential. Instead, higher cortical regions such as the posterior parietal cortex may play a more significant role in VWM storage. This is consistent with reviews of other available behavioral, neuroimaging and neurophysiology results. Recently, a number of studies brought forward new evidence regarding this debate. Here I review these new pieces of evidence in detail and show that there is still no strong and definitive evidence supporting an essential role of the early visual areas in VWM storage. Instead, converging evidence suggests that early visual areas may contribute to the decision stage of a VWM task by facilitating target and probe comparison. Aside from further clarifying this debate, it is also important to note that whether or not VWM storage uses a sensory code depends on how it is defined, and that behavioral interactions between VWM and perception tasks do not necessarily support the involvement of sensory regions in VWM storage.

What Does Dorsal Cortex Contribute to Perception?

According to the influential "Two Visual Pathways" hypothesis, the cortical visual system is segregated into two pathways, with the ventral, occipitotemporal pathway subserving object perception, and the dorsal, occipitoparietal pathway subserving the visuomotor control of action. However, growing evidence suggests that the dorsal pathway also plays a functional role in object perception. In the current article, we present evidence that the dorsal pathway contributes uniquely to the perception of a range of visuospatial attributes that are not redundant with representations in ventral cortex. We describe how dorsal cortex is recruited automatically during perception, even when no explicit visuomotor response is required. Importantly, we propose that dorsal cortex may selectively process visual attributes that can inform the perception of potential actions on objects and environments, and we consider plausible developmental and cognitive mechanisms that might give rise to these representations. As such, we consider whether naturalistic stimuli, such as real-world solid objects, might engage dorsal cortex more so than simplified or artificial stimuli such as images that do not afford action, and how the use of suboptimal stimuli might limit our understanding of the functional contribution of dorsal cortex to visual perception.

Individual-specific and shared representations during episodic memory encoding and retrieval

Although human memories seem unique to each individual, they are shared to a great extent across individuals. Previous studies have examined, separately, subject-specific and cross-subject shared representations during memory encoding and retrieval, but how shared memories are formed from individually encoded representations is not clearly understood. Using a unique fMRI design involving memory encoding and retrieval, and representational similarity analysis to link representations from different individuals, brain regions, and processing stages, the current study revealed that distributed brain regions showed both subject-specific and shared neural representations during both memory encoding and retrieval. Furthermore, different brain regions showed stage-specific representational strength, with the visual cortex showing greater unique and shared representations during encoding, whereas the left angular gyrus showing greater unique and shared representations during retrieval. The neural representations during encoding were transformed during retrieval, as shown by smaller cross-subject encoding-retrieval similarity (ERS) than cross-subject similarity either during encoding or during retrieval. This cross-subject and cross-stage similarity was found both within and across regions, with strong pattern similarity between the encoded representation in VVC and the retrieved representation in the angular gyrus. Simulation analysis further suggested that these patterns could be achieved by incorporating stage-specific representational strength, and cross-region reinstatement from encoding to retrieval, but not by a common transformation from encoding to retrieval across subjects. Together, our results shed light on how memory representations are encoded and transformed to maintain individual characteristics and at the same time to create shared representations to facilitate interpersonal communication.

Cortical Transformation of Stimulus Space in Order to Linearize a Linearly Inseparable Task

The human brain is able to learn difficult categorization tasks, even ones that have linearly inseparable boundaries; however, it is currently unknown how it achieves this computational feat. We investigated this by training participants on an animal categorization task with a linearly inseparable prototype structure in a morph shape space. Participants underwent fMRI scans before and after 4 days of behavioral training. Widespread representational changes were found throughout the brain, including an untangling of the categories' neural patterns that made them more linearly separable after behavioral training. These neural changes were task dependent, as they were only observed while participants were performing the categorization task, not during passive viewing. Moreover, they were found to occur in frontal and parietal areas, rather than ventral temporal cortices, suggesting that they reflected attentional and decisional reweighting, rather than changes in object recognition templates. These results illustrate how the brain can flexibly transform neural representational space to solve computationally challenging tasks.

Mapping the anatomy of perceptual pseudoneglect. A multivariate approach

Fundamental to the understanding of the functions of spatial cognition and attention is to clarify the underlying neural mechanisms. It is clear that relatively right-dominant activity in ventral and dorsal parieto-frontal cortex is associated with attentional reorienting, certain forms of mental imagery and spatial working memory for higher loads, while lesions mostly to right ventral areas cause spatial neglect with pathological attentional biases to the right side. In contrast, complementary leftward biases in healthy people, called pseudoneglect, have been associated with varying patterns of cortical activity. Notably, this inconsistency may be explained, at least in part, by the fact that pseudoneglect studies have often employed experimental paradigms that do not control sufficiently for cognitive processes unrelated to pseudoneglect. To address this issue, here we administered a carefully designed continuum of pseudoneglect and control tasks in healthy adults while using functional magnetic resonance imaging (fMRI). Data submitted to partial least square (PLS) imaging analysis yielded a significant latent variable that identified a right-dominant network of brain regions along the intra-occipital and -parietal sulci, frontal eye fields and right ventral cortex in association with perceptual pseudoneglect. Our results shed new light on the interplay of attentional and cognitive systems in pseudoneglect.

Task modulation of the 2-pathway characterization of occipitotemporal and posterior parietal visual object representations

Recent studies have reported the existence of rich non-spatial visual object representations in both human and monkey posterior parietal cortex (PPC), similar to those found in occipito-temporal cortex (OTC). Despite this similarity, we recently showed that visual object representation still differ between OTC and PPC in two aspects. In one study, by manipulating whether object shape or color was task relevant, we showed that visual object representations were under greater top-down attention and task control in PPC than in OTC (Vaziri-Pashkam & Xu, 2017, J Neurosci). In another study, using a bottom-up data driven approach, we showed that there exists a large separation between PPC and OTC regions in the representational space, with OTC regions lining up hierarchically along an OTC pathway and PPC regions lining up hierarchically along an orthogonal PPC pathway (Vaziri-Pashkam & Xu, 2019, Cereb Cortex). To understand the interaction of goal-driven visual processing and the two-pathway structure in the representational space, here we performed a set of new analyses of the data from the three experiments of Vaziri-Pashkam and Xu (2017) and directly compared the two-pathway separation of OTC and PPC regions when object shapes were attended and task relevant and when they were not. We found that in all three experiments the correlation of visual object representational structure between superior IPS (a key PPC visual region) and lateral and ventral occipito-temporal regions (higher OTC visual regions) became greater when object shapes were attended than when they were not. This modified the two-pathway structure, with PPC regions moving closer to higher OTC regions and a compression of the PPC pathway towards the OTC pathway in the representational space when shapes were attended. Consistent with this observation, the correlation between neural and behavioral measures of visual representational structure was also higher in superior IPS when shapes were attended than when they were not. By comparing representational structures across experiments and tasks, we further showed that attention to object shape resulted in the formation of more similar object representations in superior IPS across experiments than between the two tasks within the same experiment despite noise and stimulus differences across the experiments. Overall, these results demonstrated that, despite the separation of the OTC and PPC pathways in the representational space, the visual representational structure of PPC is flexible and can be modulated by the task demand. This reaffirms the adaptive nature of visual processing in PPC and further distinguishes it from the more invariant nature of visual processing in OTC.

Decoding subcategories of human bodies from both body- and face-responsive cortical regions

Our visual system can easily categorize objects (e.g. faces vs. bodies) and further differentiate them into subcategories (e.g. male vs. female). This ability is particularly important for objects of social significance, such as human faces and bodies. While many studies have demonstrated category selectivity to faces and bodies in the brain, how subcategories of faces and bodies are represented remains unclear. Here, we investigated how the brain encodes two prominent subcategories shared by both faces and bodies, sex and weight, and whether neural responses to these subcategories rely on low-level visual, high-level visual or semantic similarity. We recorded brain activity with fMRI while participants viewed faces and bodies that varied in sex, weight, and image size. The results showed that the sex of bodies can be decoded from both body- and face-responsive brain areas, with the former exhibiting more consistent size-invariant decoding than the latter. Body weight could also be decoded in face-responsive areas and in distributed body-responsive areas, and this decoding was also invariant to image size. The weight of faces could be decoded from the fusiform body area (FBA), and weight could be decoded across face and body stimuli in the extrastriate body area (EBA) and a distributed body-responsive area. The sex of well-controlled faces (e.g. excluding hairstyles) could not be decoded from face- or body-responsive regions. These results demonstrate that both face- and body-responsive brain regions encode information that can distinguish the sex and weight of bodies. Moreover, the neural patterns corresponding to sex and weight were invariant to image size and could sometimes generalize across face and body stimuli, suggesting that such subcategorical information is encoded with a high-level visual or semantic code.

Spaced Learning Enhances Episodic Memory by Increasing Neural Pattern Similarity Across Repetitions

Spaced learning has been shown consistently to benefit memory compared with massed learning, yet the neural representations and processes underlying the spacing effect are still poorly understood. In particular, two influential models (i.e., the encoding variability hypothesis and the study-phase retrieval hypothesis) could both model behavioral performance very well, but they make opposite hypotheses regarding the spacing effect's neural mechanisms. The present study attempted to provide empirical neural evidence to adjudicate these competing hypotheses. Using spatiotemporal pattern similarity (STPS) analysis of EEG data, this study investigated whether and how repetition lags (massed/short-spaced/long-spaced) modulated the STPS's contribution to episodic memory encoding in male and female human participants. The results revealed that greater item-specific STPS in the right frontal electrodes at 543-727 ms after stimulus onset was associated with better memory performance. More importantly, this STPS was larger under the spaced-learning condition than the massed-learning condition and partially mediated the spacing effect on memory performance. In addition, we found that massed learning was associated with stronger repetition suppression in the N400 component that reflected momentary retrieval strength, but reduced activity in the late positive component that was associated with memory retrieval. These results suggest that spaced learning improves long-term memory by increasing retrieval effort and enhancing the pattern reinstatement of prior neural representations, which may be achieved by reducing the momentary retrieval strength as the extended repetition lags might help to eliminate the residual representation in working memory.SIGNIFICANCE STATEMENT As one of the most ubiquitous and fundamental phenomena in the history of memory research, the spacing effect provides an important window into understanding how enduring memory is formed in the brain and how different practice strategies could modulate these mechanisms to affect memory performance. By leveraging the neural representational analysis on scalp EEG data, the current study provides the first empirical data to show that spaced learning enhances memory by improving the spatiotemporal similarity that occurs at a late time window. Our results support the study-phase retrieval hypothesis but not the encoding variability hypothesis and emphasize the role of neural pattern reinstatement in strengthening memory via repeated study.

An Information-Driven 2-Pathway Characterization of Occipitotemporal and Posterior Parietal Visual Object Representations

Recent studies have demonstrated the existence of rich visual representations in both occipitotemporal cortex (OTC) and posterior parietal cortex (PPC). Using fMRI decoding and a bottom-up data-driven approach, we showed that although robust object category representations exist in both OTC and PPC, there is an information-driven 2-pathway separation among these regions in the representational space, with occipitotemporal regions arranging hierarchically along 1 pathway and posterior parietal regions along another pathway. We obtained 10 independent replications of this 2-pathway distinction, accounting for 58-81% of the total variance of the region-wise differences in visual representation. The separation of the PPC regions from higher occipitotemporal regions was not driven by a difference in tolerance to changes in low-level visual features, did not rely on the presence of special object categories, and was present whether or not object category was task relevant. Our information-driven 2-pathway structure differs from the well-known ventral-what and dorsal-where/how characterization of posterior brain regions. Here both pathways contain rich nonspatial visual representations. The separation we see likely reflects a difference in neural coding scheme used by PPC to represent visual information compared with that of OTC.

Rehearsal initiates systems memory consolidation, sleep makes it last

After encoding, memories undergo a transitional process termed systems memory consolidation. It allows fast acquisition of new information by the hippocampus, as well as stable storage in neocortical long-term networks, where memory is protected from interference. Whereas this process is generally thought to occur slowly over time and sleep, we recently found a rapid memory systems transition from hippocampus to posterior parietal cortex (PPC) that occurs over repeated rehearsal within one study session. Here, we use fMRI to demonstrate that this transition is stabilized over sleep, whereas wakefulness leads to a reset to naïve responses, such as observed during early encoding. The role of sleep therefore seems to go beyond providing additional rehearsal through memory trace reactivation, as previously thought. We conclude that repeated study induces systems consolidation, while sleep ensures that these transformations become stable and long lasting. Thus, sleep and repeated rehearsal jointly contribute to long-term memory consolidation.

Human frontoparietal cortex represents behaviorally relevant target status based on abstract object features

Searching for items that are useful given current goals, or "target" recognition, requires observers to flexibly attend to certain object properties at the expense of others. This could involve focusing on the identity of an object while ignoring identity-preserving transformations such as changes in viewpoint or focusing on its current viewpoint while ignoring its identity. To effectively filter out variation due to the irrelevant dimension, performing either type of task is likely to require high-level, abstract search templates. Past work has found target recognition signals in areas of ventral visual cortex and in subregions of parietal and frontal cortex. However, target status in these tasks is typically associated with the identity of an object, rather than identity-orthogonal properties such as object viewpoint. In this study, we used a task that required subjects to identify novel object stimuli as targets according to either identity or viewpoint, each of which was not predictable from low-level properties such as shape. We performed functional MRI in human subjects of both sexes and measured the strength of target-match signals in areas of visual, parietal, and frontal cortex. Our multivariate analyses suggest that the multiple-demand (MD) network, including subregions of parietal and frontal cortex, encodes information about an object's status as a target in the relevant dimension only, across changes in the irrelevant dimension. Furthermore, there was more target-related information in MD regions on correct compared with incorrect trials, suggesting a strong link between MD target signals and behavior. NEW & NOTEWORTHY Real-world target detection tasks, such as searching for a car in a crowded parking lot, require both flexibility and abstraction. We investigated the neural basis of these abilities using a task that required invariant representations of either object identity or viewpoint. Multivariate decoding analyses of our whole brain functional MRI data reveal that invariant target representations are most pronounced in frontal and parietal regions, and the strength of these representations is associated with behavioral performance.

Continuous and discrete representations of feature-based attentional priority in human frontoparietal network

Previous studies suggest that human frontoparietal network represents feature-based attentional priority, yet the precise nature of the priority signals remains unclear. Here, we examined whether priority signals vary continuously or discretely as a function of feature similarity. In an fMRI experiment, we presented two superimposed dot fields moving along two linear directions (leftward and rightward) while varying the angular separation between the two directions. Subjects were cued to attend to one of the two dot fields and respond to a possible speed-up in the cued direction. We used multivariate analysis to evaluate how priority representation of the attended direction changes with feature similarity. We found that in early visual areas as well as posterior intraparietal sulcus and inferior frontal junction, the patterns of neural activity became more different as the feature similarity decreased, indicating a continuous representation of the attended feature. In contrast, patterns of neural activity in anterior intraparietal sulcus and frontal eye field remained invariant to changes in feature similarity, indicating a discrete representation of the attended feature. Such distinct neural coding of attentional priority across the frontoparietal network may make complementary contributions to enable flexible attentional control.

Category-selective processing in the two visual pathways as a function of stimulus degradation by noise

Understanding the organising principles and functional properties of the primate brain's numerous visually responsive cortical regions is one of the major goals in cognitive neuroscience. Functional magnetic resonance imaging (fMRI) studies have revealed that neural responses in higher-order visual cortex are shaped by object categories, task context, and spatiotemporal regularities. Beyond these properties, visual processing in the ventral pathway has been shown to be tightly linked to perceptual awareness, while the evidence regarding dorsal visual processing and awareness is mixed. Most previous studies targeting the dorsal pathway have used dichotomous "visible versus invisible" experimental designs and interocular suppression paradigms to modulate stimulus visibility. In this fMRI study, we sought to investigate category-selective processing of faces and tools in the ventral and dorsal visual streams as a function of parametric stimulus degradation by noise. Both frequentist and Bayesian statistics provide strong evidence for a linear relationship between category-selective processing and stimulus information in both visual pathways. Overall, multivariate category decoding accuracies turned out to be lower in the dorsal pathway. We discuss our results within the context of the emerging notion of highly interconnected visual streams, and provide an outlook on how future studies may help to further refine our understanding of the functional role of the dorsal pathway in visual object processing.

Spatial Frequency Tolerant Visual Object Representations in the Human Ventral and Dorsal Visual Processing Pathways

Primate ventral and dorsal visual pathways both contain visual object representations. Dorsal regions receive more input from magnocellular system while ventral regions receive inputs from both magnocellular and parvocellular systems. Due to potential differences in the spatial sensitivites of manocellular and parvocellular systems, object representations in ventral and dorsal regions may differ in how they represent visual input from different spatial scales. To test this prediction, we asked observers to view blocks of images from six object categories, shown in full spectrum, high spatial frequency (SF), or low SF. We found robust object category decoding in all SF conditions as well as SF decoding in nearly all the early visual, ventral, and dorsal regions examined. Cross-SF decoding further revealed that object category representations in all regions exhibited substantial tolerance across the SF components. No difference between ventral and dorsal regions was found in their preference for the different SF components. Further comparisons revealed that, whereas differences in the SF component separated object category representations in early visual areas, such a separation was much smaller in downstream ventral and dorsal regions. In those regions, variations among the object categories played a more significant role in shaping the visual representational structures. Our findings show that ventral and dorsal regions are similar in how they represent visual input from different spatial scales and argue against a dissociation of these regions based on differential sensitivity to different SFs.

Three-stage processing of category and variation information by entangled interactive mechanisms of peri-occipital and peri-frontal cortices

Object recognition has been a central question in human vision research. The general consensus is that the ventral and dorsal visual streams are the major processing pathways undertaking objects' category and variation processing. This overlooks mounting evidence supporting the role of peri-frontal areas in category processing. Yet, many aspects of visual processing in peri-frontal areas have remained unattended including whether these areas play role only during active recognition and whether they interact with lower visual areas or process information independently. To address these questions, subjects were presented with a set of variation-controlled object images while their EEG were recorded. Considerable amounts of category and variation information were decodable from occipital, parietal, temporal and prefrontal electrodes. Using information-selectivity indices, phase and Granger causality analyses, three processing stages were identified showing distinct directions of information transaction between peri-frontal and peri-occipital areas suggesting their parallel yet interactive role in visual processing. A brain-plausible model supported the possibility of interactive mechanisms in peri-occipital and peri-frontal areas. These findings, while promoting the role of prefrontal areas in object recognition, extend their contributions from active recognition, in which peri-frontal to peri-occipital pathways are activated by higher cognitive processes, to the general sensory-driven object and variation processing.

The Posterior Parietal Cortex in Adaptive Visual Processing

Although the primate posterior parietal cortex (PPC) has been largely associated with space, attention, and action-related processing, a growing number of studies have reported the direct representation of a diverse array of action-independent nonspatial visual information in the PPC during both perception and visual working memory. By describing the distinctions and the close interactions of visual representation with space, attention, and action-related processing in the PPC, here I propose that we may understand these diverse PPC functions together through the unique contribution of the PPC to adaptive visual processing and form a more integrated and structured view of the role of the PPC in vision, cognition, and action.

Content-specific activity in frontoparietal and default-mode networks during prior-guided visual perception

How prior knowledge shapes perceptual processing across the human brain, particularly in the frontoparietal (FPN) and default-mode (DMN) networks, remains unknown. Using ultra-high-field (7T) functional magnetic resonance imaging (fMRI), we elucidated the effects that the acquisition of prior knowledge has on perceptual processing across the brain. We observed that prior knowledge significantly impacted neural representations in the FPN and DMN, rendering responses to individual visual images more distinct from each other, and more similar to the image-specific prior. In addition, neural representations were structured in a hierarchy that remained stable across perceptual conditions, with early visual areas and DMN anchored at the two extremes. Two large-scale cortical gradients occur along this hierarchy: first, dimensionality of the neural representational space increased along the hierarchy; second, prior’s impact on neural representations was greater in higher-order areas. These results reveal extensive and graded influences of prior knowledge on perceptual processing across the brain.

A Tale of Two Visual Systems: Invariant and Adaptive Visual Information Representations in the Primate Brain

Visual information processing contains two opposite needs. There is both a need to comprehend the richness of the visual world and a need to extract only pertinent visual information to guide thoughts and behavior at a given moment. I argue that these two aspects of visual processing are mediated by two complementary visual systems in the primate brain-specifically, the occipitotemporal cortex (OTC) and the posterior parietal cortex (PPC). The role of OTC in visual processing has been documented extensively by decades of neuroscience research. I review here recent evidence from human imaging and monkey neurophysiology studies to highlight the role of PPC in adaptive visual processing. I first document the diverse array of visual representations found in PPC. I then describe the adaptive nature of visual representation in PPC by contrasting visual processing in OTC and PPC and by showing that visual representations in PPC largely originate from OTC.