Differential processing of objects under various viewing conditions in the human lateral occipital complex

Setting the scene for boundary extension: Methods, findings, connections, and theories

A previously viewed scene is often remembered as containing a larger extent of the background than was actually present, and information that was likely present just outside the boundaries of that view is often incorporated into the representation of that scene. This has been referred to as boundary extension. Methodologies used in studies on boundary extension (terminology, stimulus presentation, response measures) are described. Empirical findings regarding effects of characteristics of the stimulus (whether the stimulus depicts a scene, semantics of the scene, view angle, object size, object cropping, object orientation, object color, number of objects, depth of field, object distance, viewpoint production, scene orientation, motion, faces, emotions, modality, whether the scene is multimodal), characteristics of the display (aperture shape, aperture size, target duration, retention interval), and characteristics of the observer (allocation of attention, imagination, age, expectations and strategies, eye fixation, eye movements, monocular or binocular view, vantage point, confinement, prior exposure, expertise, arousal, pathology) on boundary extension are reviewed. Connections of boundary extension to other cognitive phenomena and processes (evolutionary adaptation, Gestalt principles, illusions, psychophysics, invariant physical principles, aesthetics, temporal boundary extension, normalization) are noted, and theories and theoretical considerations regarding boundary extension (multisource model, boundary transformation, mental imagery, 4E cognition, cognitive modularity, neurological mechanisms of scene representation) are discussed.

Concurrent response and action effect representations across the somatomotor cortices during novel task preparation

Instructions allow us to fulfill novel and complex tasks on the first try. This skill has been linked to preparatory brain signals that encode upcoming demands in advance, facilitating novel performance. To deepen insight into these processes, we explored whether instructions pre-activated task-relevant motoric and perceptual neural states. Critically, we addressed whether these representations anticipated activity patterns guiding overt sensorimotor processing, which could reflect that internally simulating novel tasks facilitates the preparation. To do so, we collected functional magnetic resonance imaging data while female and male participants encoded and implemented novel stimulus-response associations. Participants also completed localizer tasks designed to isolate the neural representations of the mappings-relevant motor responses, perceptual consequences, and stimulus categories. Using canonical template tracking, we identified whether and where these sensorimotor representations were pre-activated. We found that response-related templates were encoded in advance in regions linked with action control, entailing not only the instructed responses but also their somatosensory consequences. This result was particularly robust in primary motor and somatosensory cortices. While, following our predictions, we found a systematic decrease in the irrelevant stimulus templates' representational strength compared to the relevant ones, this difference was due to below-zero estimates linked to the irrelevant category activity patterns. Overall, our findings reflect that instruction processing relies on the sensorimotor cortices to anticipate motoric and kinesthetic representations of prospective action plans, suggesting the engagement of motor imagery during novel task preparation. More generally, they stress that the somatomotor system could participate with higher-level frontoparietal regions during anticipatory task control.

Altered mechanisms of adaptation in social anxiety: differences in adapting to positive versus negative emotional faces

Social anxiety is characterised by fear of negative evaluation and negative perceptual biases; however, the cognitive mechanisms underlying these negative biases are not well understood. We investigated a possible mechanism which could maintain negative biases: altered adaptation to emotional faces. Heightened sensitivity to negative emotions could result from weakened adaptation to negative emotions, strengthened adaptation to positive emotions, or both mechanisms. We measured adaptation from repeated exposure to either positive or negative emotional faces, in individuals high versus low in social anxiety. We quantified adaptation strength by calculating the point of subjective equality (PSE) before and after adaptation for each participant. We hypothesised: (1) weaker adaptation to angry vs happy faces in individuals high in social anxiety, (2) no difference in adaptation to angry vs happy faces in individuals low in social anxiety, and (3) no difference in adaptation to sad vs happy faces in individuals high in social anxiety. Our results revealed a weaker adaptation to angry compared to happy faces in individuals high in social anxiety (Experiment 1), with no such difference in individuals low in social anxiety (Experiment 1), and no difference in adaptation strength to sad vs happy faces in individuals high in social anxiety (Experiment 2).

Contribution of the lateral occipital and parahippocampal cortices to pattern separation of objects and contexts

Contextual features are integral to episodic memories; yet, we know little about context effects on pattern separation, a hippocampal function promoting orthogonalization of overlapping memory representations. Recent studies suggested that various extrahippocampal brain regions support pattern separation; however, the specific role of the parahippocampal cortex-a region involved in context representation-in pattern separation has not yet been studied. Here, we investigated the contribution of the parahippocampal cortex (specifically, the parahippocampal place area) to context reinstatement effects on mnemonic discrimination, using functional magnetic resonance imaging. During scanning, participants saw object images on unique context scenes, followed by a recognition task involving the repetitions of encoded objects or visually similar lures on either their original context or a lure context. Context reinstatement at retrieval improved item recognition but hindered mnemonic discrimination. Crucially, our region of interest analyses of the parahippocampal place area and an object-selective visual area, the lateral occipital cortex indicated that while during successful mnemonic decisions parahippocampal place area activity decreased for old contexts compared to lure contexts irrespective of object novelty, lateral occipital cortex activity differentiated between old and lure objects exclusively. These results imply that pattern separation of contextual and item-specific memory features may be differentially aided by scene and object-selective cortical areas.

Neural representations of perspectival shapes and attentional effects: Evidence from fMRI and MEG

Does the human brain represent perspectival shapes, i.e., viewpoint-dependent object shapes, especially in relatively higher-level visual areas such as the lateral occipital cortex? What is the temporal profile of the appearance and disappearance of neural representations of perspectival shapes? And how does attention influence these neural representations? To answer these questions, we employed functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and multivariate decoding techniques to investigate spatiotemporal neural representations of perspectival shapes. Participants viewed rotated objects along with the corresponding objective shapes and perspectival shapes (i.e., rotated round, round, and oval) while we measured their brain activities. Our results revealed that shape classifiers trained on the basic shapes (i.e., round and oval) consistently identified neural representations in the lateral occipital cortex corresponding to the perspectival shapes of the viewed objects regardless of attentional manipulations. Additionally, this classification tendency toward the perspectival shapes emerged approximately 200 ms after stimulus presentation. Moreover, attention influenced the spatial dimension as the regions showing the perspectival shape classification tendency propagated from the occipital lobe to the temporal lobe. As for the temporal dimension, attention led to a more robust and enduring classification tendency towards perspectival shapes. In summary, our study outlines a spatiotemporal neural profile for perspectival shapes that suggests a greater degree of perspectival representation than is often acknowledged.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses between lower and higher visual brain areas and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Temporal dynamics of short-term neural adaptation across human visual cortex

Neural responses in visual cortex adapt to prolonged and repeated stimuli. While adaptation occurs across the visual cortex, it is unclear how adaptation patterns and computational mechanisms differ across the visual hierarchy. Here we characterize two signatures of short-term neural adaptation in time-varying intracranial electroencephalography (iEEG) data collected while participants viewed naturalistic image categories varying in duration and repetition interval. Ventral- and lateral-occipitotemporal cortex exhibit slower and prolonged adaptation to single stimuli and slower recovery from adaptation to repeated stimuli compared to V1-V3. For category-selective electrodes, recovery from adaptation is slower for preferred than non-preferred stimuli. To model neural adaptation we augment our delayed divisive normalization (DN) model by scaling the input strength as a function of stimulus category, enabling the model to accurately predict neural responses across multiple image categories. The model fits suggest that differences in adaptation patterns arise from slower normalization dynamics in higher visual areas interacting with differences in input strength resulting from category selectivity. Our results reveal systematic differences in temporal adaptation of neural population responses across the human visual hierarchy and show that a single computational model of history-dependent normalization dynamics, fit with area-specific parameters, accounts for these differences.

Feasibility study of functional near-infrared spectroscopy in the ventral visual pathway for real-life applications

Significance

fNIRS-based neuroenhancement depends on the feasible detection of hemodynamic responses in target brain regions. Using the lateral occipital complex (LOC) and the fusiform face area (FFA) in the ventral visual pathway as neurofeedback targets boosts performance in visual recognition. However, the feasibility of utilizing fNIRS to detect LOC and FFA activity in adults remains to be validated as the depth of these regions may exceed the detection limit of fNIRS.

Aim

This study aims to investigate the feasibility of using fNIRS to measure hemodynamic responses in the ventral visual pathway, specifically in the LOC and FFA, in adults.

Approach

We recorded the hemodynamic activities of the LOC and FFA regions in 35 subjects using a portable eight-channel fNIRS instrument. A standard one-back object and face recognition task was employed to elicit selective brain responses in the LOC and FFA regions. The placement of fNIRS optodes for LOC and FFA detection was guided by our group's transcranial brain atlas (TBA).

Results

Our findings revealed selective activation of the LOC target channel (CH2) in response to objects, whereas the FFA target channel (CH7) did not exhibit selective activation in response to faces.

Conclusions

Our findings indicate that, although fNIRS detection has limitations in capturing FFA activity, the LOC region emerges as a viable target for fNIRS-based detection. Furthermore, our results advocate for the adoption of the TBA-based method for setting the LOC target channel, offering a promising solution for optrode placement. This feasibility study stands as the inaugural validation of fNIRS for detecting cortical activity in the ventral visual pathway, underscoring its ecological validity. We suggest that our findings establish a pivotal technical groundwork for prospective real-life applications of fNIRS-based research.

Event Probabilities Have a Different Impact on Early and Late Electroencephalographic Measures Regarded as Metrics of Prediction

The oddball protocol has been used to study the neural and perceptual consequences of implicit predictions in the human brain. The protocol involves presenting a sequence of identical repeated events that are eventually broken by a novel "oddball" presentation. Oddball presentations have been linked to increased neural responding and to an exaggeration of perceived duration relative to repeated events. Because the number of repeated events in such protocols is circumscribed, as more repeats are encountered, the conditional probability of a further repeat decreases-whereas the conditional probability of an oddball increases. These facts have not been appreciated in many analyses of oddballs; repeats and oddballs have rather been treated as binary event categories. Here, we show that the human brain is sensitive to conditional event probabilities in an active, visual oddball paradigm. P300 responses (a relatively late component of visually evoked potentials measured with EEG) tended to be greater for less likely oddballs and repeats. By contrast, P1 responses (an earlier component) increased for repeats as a goal-relevant target presentation neared, but this effect occurred even when repeat probabilities were held constant, and oddball P1 responses were invariant. We also found that later, more likely oddballs seemed to last longer, and this effect was largely independent of the number of preceding repeats. These findings speak against a repetition suppression account of the temporal oddball effect. Overall, our data highlight an impact of event probability on later, rather than earlier, electroencephalographic measures previously related to predictive processes-and the importance of considering conditional probabilities in sequential presentation paradigms.

Where in the brain do internally generated and externally presented visual information interact?

Conscious experiences normally result from the flow of external input into our sensory systems. However, we can also create conscious percepts independently of sensory stimulation. These internally generated percepts are referred to as mental images, and they have many similarities with real visual percepts. Consequently, mental imagery is often referred to as "seeing in the mind's eye". While the neural basis of imagery has been widely studied, the interaction between internal and external sources of visual information has received little interest. Here we examined this question by using fMRI to record brain activity of healthy human volunteers while they were performing visual imagery that was distracted with a concurrent presentation of a visual stimulus. Multivariate pattern analysis (MVPA) was used to identify the brain basis of this interaction. Visual imagery was reflected in several brain areas in ventral temporal, lateral occipitotemporal, and posterior frontal cortices, with a left-hemisphere dominance. The key finding was that imagery content representations in the left lateral occipitotemporal cortex were disrupted when a visual distractor was presented during imagery. Our results thus demonstrate that the representations of internal and external visual information interact in brain areas associated with the encoding of visual objects and shapes.

A review of the impairments, preserved visual functions, and neuropathology in 21 patients with visual form agnosia - A unique defect with line drawings

We present a comprehensive review of the rare syndrome visual form agnosia (VFA). We begin by documenting its history, including the origins of the term, and the first case study labelled as VFA. The defining characteristics of the syndrome, as others have previously defined it, are then described. The impairments, preserved aspects of visual perception, and areas of brain damage in 21 patients who meet these defining characteristics are described in detail, including which tests were used to verify the presence or absence of key symptoms. From this, we note important similarities along with notable areas of divergence between patients. Damage to the occipital lobe (20/21), an inability to recognise line drawings (19/21), preserved colour vision (14/21), and visual field defects (16/21) were areas of consistency across most cases. We found it useful to distinguish between shape and form as distinct constructs when examining perceptual abilities in VFA patients. Our observations suggest that these patients often exhibit difficulties in processing simplified versions of form. Deficits in processing orientation and size were uncommon. Motion perception and visual imagery were not widely tested for despite being typically cited as defining features of the syndrome - although in the sample described, motion perception was never found to be a deficit. Moreover, problems with vision (e.g., poor visual acuity and the presence of hemianopias/scotomas in the visual fields) are more common than we would have thought and may also contribute to perceptual impairments in patients with VFA. We conclude that VFA is a perceptual disorder where the visual system has a reduced ability to synthesise lines together for the purposes of making sense of what images represent holistically.

Distinct patterns of neural response to faces from different races in humans and deep networks

Social categories such as the race or ethnicity of an individual are typically conveyed by the visual appearance of the face. The aim of this study was to explore how these differences in facial appearance are represented in human and artificial neural networks. First, we compared the similarity of faces from different races using a neural network trained to discriminate identity. We found that the differences between races were most evident in the fully connected layers of the network. Although these layers were also able to predict behavioural judgements of face identity from human participants, performance was biased toward White faces. Next, we measured the neural response in face-selective regions of the human brain to faces from different races in Asian and White participants. We found distinct patterns of response to faces from different races in face-selective regions. We also found that the spatial pattern of response was more consistent across participants for own-race compared to other-race faces. Together, these findings show that faces from different races elicit different patterns of response in human and artificial neural networks. These differences may underlie the ability to make categorical judgements and explain the behavioural advantage for the recognition of own-race faces.

The effect of context congruency on fMRI repetition suppression for objects

The recognition of objects is strongly facilitated when they are presented in the context of other objects (Biederman, 1972). Such contexts facilitate perception and induce expectations of context-congruent objects (Trapp and Bar, 2015). The neural mechanisms underlying these facilitatory effects of context on object processing, however, are not yet fully understood. In the present study, we investigate how context-induced expectations affect subsequent object processing. We used functional magnetic resonance imaging and measured repetition suppression as a proxy for prediction error processing. Participants viewed pairs of alternating or repeated object images which were preceded by context-congruent, context-incongruent or neutral cues. We found a stronger repetition suppression in congruent as compared to incongruent or neutral cues in the object sensitive lateral occipital cortex. Interestingly, this stronger effect was driven by enhanced responses to alternating stimulus pairs in the congruent contexts, rather than by suppressed responses to repeated stimulus pairs, which emphasizes the contribution of surprise-related response enhancement for the context modulation on RS when expectations are violated. In addition, in the congruent condition, we discovered significant functional connectivity between object-responsive and frontal cortical regions, as well as between object-responsive regions and the fusiform gyrus. Our findings indicate that prediction errors, reflected in enhanced brain responses to violated contextual expectations, underlie the facilitating effect of context during object perception.

A "Bandwidth" in cortical representations of multiple faces

The human ability to process multiple items simultaneously can be constrained by the extent to which those items are represented by distinct neural populations. In the current study, we used fMRI to investigate the cortical representation of multiple faces. We found that the addition of a second face to occupy both visual hemifields led to an increased response, whereas a further addition of faces within the same visual hemifield resulted in a decreased response. This pattern was widely observed in the occipital visual cortex, the intraparietal sulcus, and extended to the posterior inferotemporal cortex. A parallel trend was found in a behavioral change-detection task, revealing a perceptual "bandwidth" of multiface processing. The sensitivity to face clutter gradually decreased along the ventral pathway, supporting the notion of a buildup of clutter-tolerance representation. These cortical response patterns to face clutters suggest that adding signals with nonoverlapping cortical representation enhanced perception, while adding signals that competed for representation resources impaired perception.

ConnSearch: A framework for functional connectivity analysis designed for interpretability and effectiveness at limited sample sizes

Functional connectivity studies increasingly turn to machine learning methods, which typically involve fitting a connectome-wide classifier, then conducting post hoc interpretation analyses to identify the neural correlates that best predict a dependent variable. However, this traditional analytic paradigm suffers from two main limitations. First, even if classifiers are perfectly accurate, interpretation analyses may not identify all the patterns expressed by a dependent variable. Second, even if classifiers are generalizable, the patterns implicated via interpretation analyses may not replicate. In other words, this traditional approach can yield effective classifiers while falling short of most neuroscientists' goals: pinpointing the neural correlates of dependent variables. We propose a new framework for multivariate analysis, ConnSearch, which involves dividing the connectome into components (e.g., groups of highly connected regions) and fitting an independent model for each component (e.g., a support vector machine or a correlation-based model). Conclusions about the link between a dependent variable and the brain are based on which components yield predictive models rather than on interpretation analysis. We used working memory data from the Human Connectome Project (N = 50-250) to compare ConnSearch with four existing connectome-wide classification/interpretation methods. For each approach, the models attempted to classify examples as being from the high-load or low-load conditions (binary labels). Relative to traditional methods, ConnSearch identified neural correlates that were more comprehensive, had greater consistency with the WM literature, and better replicated across datasets. Hence, ConnSearch is well-positioned to be an effective tool for functional connectivity research.

The temporal visual oddball effect is not caused by repetition suppression

The oddball paradigm is commonly used to investigate human time perception. Trains of identical repeated events ('standards') are presented, only to be interrupted by a different 'oddball' that seems to have a relatively protracted duration. One theoretical account has been that this effect is driven by repetition suppression for repeated standards. The idea is that repeated events seem shorter as they incur a progressively reduced neural response, which is supported by the finding that oddball perceived duration increases linearly with the number of preceding repeated standards. However, typical oddball paradigms confound the probability of oddball presentations with variable numbers of standard repetitions on each trial, allowing people to increasingly anticipate an oddball presentation as more standards are presented. We eliminated this by making participants aware of what fixed number of standards they would encounter before a final test input and tested different numbers of standards in separate experimental sessions. The final event of sequences, the test event, was equally likely to be an oddball or another repeat. We found a positive linear relationship between the number of preceding repeated standards and the perceived duration of oddball test events. However, we also found this for repeat tests events, which speaks against the repetition suppression account of the temporal oddball effect.

Perceptual stability reflected in neuronal pattern similarities in human visual cortex

The magnitude of neuronal activation is commonly considered a critical factor for conscious perception of visual content. However, this dogma contrasts with the phenomenon of rapid adaptation, in which the magnitude of neuronal activation drops dramatically in a rapid manner while the visual stimulus and the conscious experience it elicits remain stable. Here, we report that the profiles of multi-site activation patterns and their relational geometry-i.e., the similarity distances between activation patterns, as revealed using intracranial electroencephalographic (iEEG) recordings-are sustained during extended visual stimulation despite the major magnitude decrease. These results are compatible with the hypothesis that conscious perceptual content is associated with the neuronal pattern profiles and their similarity distances, rather than the overall activation magnitude, in human visual cortex.

Spikiness and animacy as potential organizing principles of human ventral visual cortex

Considerable research has been devoted to understanding the fundamental organizing principles of the ventral visual pathway. A recent study revealed a series of 3-4 topographical maps arranged along the macaque inferotemporal (IT) cortex. The maps articulated a two-dimensional space based on the spikiness and animacy of visual objects, with "inanimate-spiky" and "inanimate-stubby" regions of the maps constituting two previously unidentified cortical networks. The goal of our study was to determine whether a similar functional organization might exist in human IT. To address this question, we presented the same object stimuli and images from "classic" object categories (bodies, faces, houses) to humans while recording fMRI activity at 7 Tesla. Contrasts designed to reveal the spikiness-animacy object space evoked extensive significant activation across human IT. However, unlike the macaque, we did not observe a clear sequence of complete maps, and selectivity for the spikiness-animacy space was deeply and mutually entangled with category-selectivity. Instead, we observed multiple new stimulus preferences in category-selective regions, including functional sub-structure related to object spikiness in scene-selective cortex. Taken together, these findings highlight spikiness as a promising organizing principle of human IT and provide new insights into the role of category-selective regions in visual object processing.

Altered functional connectivity of the ascending reticular activating system in obstructive sleep apnea

Repeated arousals during sleep in obstructive sleep apnea (OSA) may lead to altered functional connectivity (FC) of the ascending reticular activating system (ARAS). We evaluated resting-state FC between eight ARAS nuclei and 105 cortical/subcortical regions in OSA patients and healthy controls. Fifty patients with moderate to severe OSA and 20 controls underwent overnight polysomnography and resting-state functional magnetic resonance imaging. Seed-to-voxel analysis of ARAS-cortex FC was compared between OSA patients and controls. The ARAS nuclei included the locus coeruleus (LC), laterodorsal tegmental nucleus (LDTg), and ventral tegmental area (VTA). FC values of three ARAS nuclei (the LC, LDTg, and VTA) significantly differed between the groups. FC of the LC with the precuneus, posterior cingulate gyrus, and right lateral occipital cortex (LOC) was stronger in OSA patients than controls. FC between the LDTg and right LOC was stronger in OSA patients than controls, but FC between the VTA and right LOC was weaker. Average LC-cortex FC values positively correlated with the arousal, apnea, and apnea-hypopnea index in OSA patients. Alterations in ARAS-cortex FC were observed in OSA patients. The strength of LC-cortex noradrenergic FC was related to arousal or OSA severity in patients.

The neural correlates of domain-general visual ability

People vary in their general ability to compare, identify, and remember objects. Research using latent variable modeling identifies a domain-general visual recognition ability (called o) that reflects correlations among different visual tasks and categories. We measure associations between a psychometrically-sensitive measure of o and a neurometrically-sensitive measure of visual sensitivity to shape. We report evidence for distributed neural correlates of o using functional and anatomical regions-of-interest (ROIs) as well as whole brain analyses. Neural selectivity to shape is associated with o in several regions of the ventral pathway, as well as additional foci in parietal and premotor cortex. Multivariate analyses suggest the distributed effects in ventral cortex reflect a common mechanism. The network of brain areas where neural selectivity predicts o is similar to that evoked by the most informative features for object recognition in prior work, showing convergence of 2 different approaches on identifying areas that support the best object recognition performance. Because o predicts performance across many visual tasks for both novel and familiar objects, we propose that o could predict the magnitude of neural changes in task-relevant areas following experience with specific task and object category.

Understanding Human Object Vision: A Picture Is Worth a Thousand Representations

Objects are the core meaningful elements in our visual environment. Classic theories of object vision focus upon object recognition and are elegant and simple. Some of their proposals still stand, yet the simplicity is gone. Recent evolutions in behavioral paradigms, neuroscientific methods, and computational modeling have allowed vision scientists to uncover the complexity of the multidimensional representational space that underlies object vision. We review these findings and propose that the key to understanding this complexity is to relate object vision to the full repertoire of behavioral goals that underlie human behavior, running far beyond object recognition. There might be no such thing as core object recognition, and if it exists, then its importance is more limited than traditionally thought.

Statistical learning of distractor shape modulates attentional capture

Physically salient but task-irrelevant stimuli have high attentional priority, although observers are able to capitalize on statistical regularities in the environment to more efficiently ignore such stimuli. Physically salient distractors that more frequently appear in a particular location are less distracting when they appear in this high probability location. Likewise, colors and orientations that are frequently associated with distractors become preferentially ignored with learning. Such statistically learned distractor suppression has been examined with respect to the frequency of elementary features across trials, and less is known about how statistics concerning the composition of distractor features within a trial influence attention, particularly with respect to how orientations combine to form shapes. Color, orientation, and location are also represented very early in vision, whereas more complex features such as shape are represented further downstream in the visual system; it remains unclear whether statistically leaned distractor suppression can operate over such downstream visual representations. In the present study, we demonstrate attentional capture by physically salient, shape-defined distractors that is reduced in magnitude for a high probability shape. Our findings demonstrate that statistical learning can modulate attentional priority at least at the level of basic shapes and is not restricted to modulations of priority at the earliest stages of visual information processing tied to elementary features.

Independent repetition suppression in macaque area V2 and inferotemporal cortex

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.NEW & NOTEWORTHY When a complexly structured natural image is presented twice in rapid succession, neurons in inferotemporal area TE exhibit repetition suppression, responding less strongly to the second than to the first presentation. We have explored whether this phenomenon is confined to high-order areas where neurons respond selectively to such images and thus carry information relevant to recognizing a repeat. We have found surprisingly that repetition suppression occurs even in low-order visual area V2.

Effects of face repetition on ventral visual stream connectivity using dynamic causal modelling of fMRI data

Stimulus repetition normally causes reduced neural activity in brain regions that process that stimulus. Some theories claim that this "repetition suppression" reflects local mechanisms such as neuronal fatigue or sharpening within a region, whereas other theories claim that it results from changed connectivity between regions, following changes in synchrony or top-down predictions. In this study, we applied dynamic causal modeling (DCM) on a public fMRI dataset involving repeated presentations of faces and scrambled faces to test whether repetition affected local (self-connections) and/or between-region connectivity in left and right early visual cortex (EVC), occipital face area (OFA) and fusiform face area (FFA). Face "perception" (faces versus scrambled faces) modulated nearly all connections, within and between regions, including direct connections from EVC to FFA, supporting a non-hierarchical view of face processing. Face "recognition" (familiar versus unfamiliar faces) modulated connections between EVC and OFA/FFA, particularly in the left hemisphere. Most importantly, immediate and delayed repetition of stimuli were also best captured by modulations of connections between EVC and OFA/FFA, but not self-connections of OFA/FFA, consistent with synchronization or predictive coding theories, though also possibly reflecting local mechanisms like synaptic depression.

Does the brain's ventral visual pathway compute object shape?

A rich behavioral literature has shown that human object recognition is supported by a representation of shape that is tolerant to variations in an object's appearance. Such 'global' shape representations are achieved by describing objects via the spatial arrangement of their local features, or structure, rather than by the appearance of the features themselves. However, accumulating evidence suggests that the ventral visual pathway - the primary substrate underlying object recognition - may not represent global shape. Instead, ventral representations may be better described as a basis set of local image features. We suggest that this evidence forces a reevaluation of the role of the ventral pathway in object perception and posits a broader network for shape perception that encompasses contributions from the dorsal pathway.

Neural dissociation of the acoustic and cognitive representation of voice identity

Recognising a speaker's identity by the sound of their voice is important for successful interaction. The skill depends on our ability to discriminate minute variations in the acoustics of the vocal signal. Performance on voice identity assessments varies widely across the population. The neural underpinnings of this ability and its individual differences, however, remain poorly understood. Here we provide critical tests of a theoretical framework for the neural processing stages of voice identity and address how individual differences in identity discrimination mediate activation in this neural network. We scanned 40 individuals on an fMRI adaptation task involving voices drawn from morphed continua between two personally familiar identities. Analyses dissociated neuronal effects induced by repetition of acoustically similar morphs from those induced by a switch in perceived identity. Activation in temporal voice-sensitive areas decreased with acoustic similarity between consecutive stimuli. This repetition suppression effect was mediated by the performance on an independent voice assessment and this result highlights an important functional role of adaptive coding in voice expertise. Bilateral anterior insulae and medial frontal gyri responded to a switch in perceived voice identity compared to an acoustically equidistant switch within identity. Our results support a multistep model of voice identity perception.

Scene-selective brain regions respond to embedded objects of a scene

Objects are fundamental to scene understanding. Scenes are defined by embedded objects and how we interact with them. Paradoxically, scene processing in the brain is typically discussed in contrast to object processing. Using the BOLD5000 dataset (Chang et al., 2019), we examined whether objects within a scene predicted the neural representation of scenes, as measured by functional magnetic resonance imaging in humans. Stimuli included 1,179 unique scenes across 18 semantic categories. Object composition of scenes were compared across scene exemplars in different semantic scene categories, and separately, in exemplars of the same scene category. Neural representations in scene- and object-preferring brain regions were significantly related to which objects were in a scene, with the effect at times stronger in the scene-preferring regions. The object model accounted for more variance when comparing scenes within the same semantic category to scenes from different categories. Here, we demonstrate the function of scene-preferring regions includes the processing of objects. This suggests visual processing regions may be better characterized by the processes, which are engaged when interacting with the stimulus kind, such as processing groups of objects in scenes, or processing a single object in our foreground, rather than the stimulus kind itself.

Functional imaging analyses reveal prototype and exemplar representations in a perceptual single-category task

Similarity-based categorization can be performed by memorizing category members as exemplars or by abstracting the central tendency of the category – the prototype. In similarity-based categorization of stimuli with clearly identifiable dimensions from two categories, prototype representations were previously located in the hippocampus and the ventromedial prefrontal cortex (vmPFC) and exemplar representations in areas supporting visual memory. However, the neural implementation of exemplar and prototype representations in perceptual similarity-based categorization of single categories is unclear. To investigate these representations, we applied model-based univariate and multivariate analyses of functional imaging data from a dot-pattern paradigm-based task. Univariate prototype and exemplar representations occurred bilaterally in visual areas. Multivariate analyses additionally identified prototype representations in parietal areas and exemplar representations in the hippocampus. Bayesian analyses supported the non-presence of prototype representations in the hippocampus and the vmPFC. We additionally demonstrate that some individuals form both representation types simultaneously, probably granting flexibility in categorization strategies.

Model-based univariate and multivariate analyses of fMRI data from 62 healthy participants in a dot-pattern paradigm-based task provide further insight into the neural basis of similarity-based categorization.

Neuroscout, a unified platform for generalizable and reproducible fMRI research

Functional magnetic resonance imaging (fMRI) has revolutionized cognitive neuroscience, but methodological barriers limit the generalizability of findings from the lab to the real world. Here, we present Neuroscout, an end-to-end platform for analysis of naturalistic fMRI data designed to facilitate the adoption of robust and generalizable research practices. Neuroscout leverages state-of-the-art machine learning models to automatically annotate stimuli from dozens of fMRI studies using naturalistic stimuli—such as movies and narratives—allowing researchers to easily test neuroscientific hypotheses across multiple ecologically-valid datasets. In addition, Neuroscout builds on a robust ecosystem of open tools and standards to provide an easy-to-use analysis builder and a fully automated execution engine that reduce the burden of reproducible research. Through a series of meta-analytic case studies, we validate the automatic feature extraction approach and demonstrate its potential to support more robust fMRI research. Owing to its ease of use and a high degree of automation, Neuroscout makes it possible to overcome modeling challenges commonly arising in naturalistic analysis and to easily scale analyses within and across datasets, democratizing generalizable fMRI research.

The nature of neural object representations during dynamic occlusion

Objects disappearing briefly from sight due to occlusion is an inevitable occurrence in everyday life. Yet we generally have a strong experience that occluded objects continue to exist, despite the fact that they objectively disappear. This indicates that neural object representations must be maintained during dynamic occlusion. However, it is unclear what the nature of such representation is and in particular whether it is perception-like or more abstract, for example, reflecting limited features such as position or movement direction only. In this study, we address this question by examining how different object features such as object shape, luminance, and position are represented in the brain when a moving object is dynamically occluded. We apply multivariate decoding methods to Magnetoencephalography (MEG) data to track how object representations unfold over time. Our methods allow us to contrast the representations of multiple object features during occlusion and enable us to compare the neural responses evoked by visible and occluded objects. The results show that object position information is represented during occlusion to a limited extent while object identity features are not maintained through the period of occlusion. Together, this suggests that the nature of object representations during dynamic occlusion is different from visual representations during perception.

The representation of shape and texture in category-selective regions of ventral-temporal cortex

Neuroimaging studies using univariate and multivariate approaches have shown that the fusiform face area (FFA) and parahippocampal place area (PPA) respond selectively to images of faces and places. The aim of this study was to determine the extent to which this selectivity to faces or places is based on the shape or texture properties of the images. Faces and houses were filtered to manipulate their texture properties, while preserving the shape properties (spatial envelope) of the images. In Experiment 1, multivariate pattern analysis (MVPA) showed that patterns of fMRI response to faces and houses in FFA and PPA were predicted by the shape properties, but not by the texture properties of the image. In Experiment 2, a univariate analysis (fMR‐adaptation) showed that responses in the FFA and PPA were sensitive to changes in both the shape and texture properties of the image. These findings can be explained by the spatial scale of the representation of images in the FFA and PPA. At a coarser scale (revealed by MVPA), the neural selectivity to faces and houses is sensitive to variation in the shape properties of the image. However, at a finer scale (revealed by fMR‐adaptation), the neural selectivity is sensitive to the texture properties of the image. By combining these neuroimaging paradigms, our results provide insights into the spatial scale of the neural representation of faces and places in the ventral‐temporal cortex.

The emergence of view-symmetric neural responses to familiar and unfamiliar faces

Successful recognition of familiar faces is thought to depend on the ability to integrate view-dependent representations of a face into a view-invariant representation. It has been proposed that a key intermediate step in achieving view invariance is the representation of symmetrical views. However, key unresolved questions remain, such as whether these representations are specific for naturally occurring changes in viewpoint and whether view-symmetric representations exist for familiar faces. To address these issues, we compared behavioural and neural responses to natural (canonical) and unnatural (noncanonical) rotations of the face. Similarity judgements revealed that symmetrical viewpoints were perceived to be more similar than non-symmetrical viewpoints for both canonical and non-canonical rotations. Next, we measured patterns of neural response from early to higher level regions of visual cortex. Early visual areas showed a view-dependent representation for natural or canonical rotations of the face, such that the similarity between patterns of response were related to the difference in rotation. View symmetric patterns of neural response to canonically rotated faces emerged in higher visual areas, particularly in face-selective regions. The emergence of a view-symmetric representation from a view-dependent representation for canonical rotations of the face was also evident for familiar faces, suggesting that view-symmetry is an important intermediate step in generating view-invariant representations. Finally, we measured neural responses to unnatural or non-canonical rotations of the face. View-symmetric patterns of response were also found in face-selective regions. However, in contrast to natural or canonical rotations of the face, these view-symmetric responses did not arise from an initial view-dependent representation in early visual areas. This suggests differences in the way that view-symmetrical representations emerge with canonical or non-canonical rotations. The similarity in the neural response to canonical views of familiar and unfamiliar faces in the core face network suggests that the neural correlates of familiarity emerge at later stages of processing.

Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism

Deep learning (DL) technologies have recently shown great potential in emotion recognition based on electroencephalography (EEG). However, existing DL-based EEG emotion recognition methods are built on single-task learning, i.e., learning arousal, valence, and dominance individually, which may ignore the complementary information of different tasks. In addition, single-task learning involves a new round of training every time a new task appears, which is time consuming. To this end, we propose a novel method for EEG-based emotion recognition based on multi-task learning with capsule network (CapsNet) and attention mechanism. First, multi-task learning can learn multiple tasks simultaneously while exploiting commonalities and differences across tasks, it can also obtain more data from different tasks, which can improve generalization and robustness. Second, the innovative structure of the CapsNet enables it to effectively characterize the intrinsic relationship among various EEG channels. Finally, the attention mechanism can change the weight of different channels to extract important information. In the DEAP dataset, the average accuracy reached 97.25%, 97.41%, and 98.35% on arousal, valence, and dominance, respectively. In the DREAMER dataset, average accuracy reached 94.96%, 95.54%, and 95.52% on arousal, valence, and dominance, respectively. Experimental results demonstrate the efficiency of the proposed method for EEG emotion recognition.

Incongruence in Lighting Impairs Face Identification

The effect of uniform lighting on face identity processing is little understood, despite its potential influence on our ability to recognize faces. Here, we investigated how changes in uniform lighting level affected face identification performance during face memory tests. Observers were tasked with learning a series of faces, followed by a memory test where observers judged whether the faces presented were studied before or novel. Face stimuli were presented under uniform bright or dim illuminations, and lighting across the face learning and the memory test sessions could be the same (“congruent”) or different (“incongruent”). This led to four experimental conditions: (1) Bright/Dim (learning bright faces, testing on dim faces); (2) Bright/Bright; (3) Dim/Bright; and (4) Dim/Dim. Our results revealed that incongruent lighting levels across sessions (Bright/Dim and Dim/Bright) significantly reduced sensitivity (d’) to faces and introduced conservative biases compared to congruent lighting levels (Bright/Bright and Dim/Dim). No significant differences in performance were detected between the congruent lighting conditions (Bright/Bright vs. Dim/Dim) and between the incongruent lighting conditions (Bright/Dim vs. Dim/Bright). Thus, incongruent lighting deteriorated performance in face identification. These findings implied that the level of uniform lighting should be considered in an illumination-specific face representation and potential applications such as eyewitness testimony.

Larger images are better remembered during naturalistic encoding

We are constantly exposed to multiple visual scenes, and while freely viewing them without an intentional effort to memorize or encode them, only some are remembered. It has been suggested that image memory is influenced by multiple factors, such as depth of processing, familiarity, and visual category. However, this is typically investigated when people are instructed to perform a task (e.g., remember or make some judgment about the images), which may modulate processing at multiple levels and thus, may not generalize to naturalistic visual behavior. Visual memory is assumed to rely on high-level visual perception that shows a level of size invariance and therefore is not assumed to be highly dependent on image size. Here, we reasoned that during naturalistic vision, free of task-related modulations, bigger images stimulate more visual system processing resources (from retina to cortex) and would, therefore, be better remembered. In an extensive set of seven experiments, naïve participants (n = 182) were asked to freely view presented images (sized 3° to 24°) without any instructed encoding task. Afterward, they were given a surprise recognition test (midsized images, 50% already seen). Larger images were remembered better than smaller ones across all experiments (∼20% higher accuracy or ∼1.5 times better). Memory was proportional to image size, faces were better remembered, and outdoors the least. Results were robust even when controlling for image set, presentation order, screen resolution, image scaling at test, or the amount of information. While multiple factors affect image memory, our results suggest that low- to high-level processes may all contribute to image memory.

Unique, Shared, and Dominant Brain Activation in Visual Word Form Area and Lateral Occipital Complex during Reading and Picture Naming

Identifying printed words and pictures concurrently is ubiquitous in daily tasks, and so it is important to consider the extent to which reading words and naming pictures may share a cognitive-neurophysiological functional architecture. Two functional magnetic resonance imaging (fMRI) experiments examined whether reading along the left ventral occipitotemporal region (vOT; often referred to as a visual word form area, VWFA) has activation that is overlapping with referent pictures (i.e., both conditions significant and shared, or with one significantly more dominant) or unique (i.e., one condition significant, the other not), and whether picture naming along the right lateral occipital complex (LOC) has overlapping or unique activation relative to referent words. Experiment 1 used familiar regular and exception words (to force lexical reading) and their corresponding pictures in separate naming blocks, and showed dominant activation for pictures in the LOC, and shared activation in the VWFA for exception words and their corresponding pictures (regular words did not elicit significant VWFA activation). Experiment 2 controlled for visual complexity by superimposing the words and pictures and instructing participants to either name the word or the picture, and showed primarily shared activation in the VWFA and LOC regions for both word reading and picture naming, with some dominant activation for pictures in the LOC. Overall, these results highlight the importance of including exception words to force lexical reading when comparing to picture naming, and the significant shared activation in VWFA and LOC serves to challenge specialized models of reading or picture naming.

Anatomy of the temporal lobe: From macro to micro

The temporal cortex encompasses a large number of different areas ranging from the six-layered isocortex to the allocortex. The areas support auditory, visual, and language processing, as well as emotions and memory. The primary auditory cortex is found at the Heschl gyri, which develop early in ontogeny with the Sylvian fissure, a deep and characteristic fissure that separates the temporal lobe from the parietal and frontal lobes. Gyri and sulci as well as brain areas vary between brains and between hemispheres, partly linked to the functional organization of language and lateralization. Interindividual variability in anatomy makes a direct comparison between different brains in structure-functional analysis often challenging, but can be addressed by applying cytoarchitectonic probability maps of the Julich-Brain atlas. We review the macroanatomy of the temporal lobe, its variability and asymmetry at the macro- and the microlevel, discuss the relationship to brain areas and their microstructure, and emphasize the advantage of a multimodal approach to address temporal lobe organization. We review recent data on combined cytoarchitectonic and molecular architectonic studies of temporal areas, and provide links to their function.

From Action to Cognition: Neural Reuse, Network Theory and the Emergence of Higher Cognitive Functions

The aim of this article is to discuss the logic and assumptions behind the concept of neural reuse, to explore its biological advantages and to discuss the implications for the cognition of a brain that reuses existing circuits and resources. We first address the requirements that must be fulfilled for neural reuse to be a biologically plausible mechanism. Neural reuse theories generally take a developmental approach and model the brain as a dynamic system composed of highly flexible neural networks. They often argue against domain-specificity and for a distributed, embodied representation of knowledge, which sets them apart from modular theories of mental processes. We provide an example of reuse by proposing how a phylogenetically more modern mental capacity (mental rotation) may appear through the reuse and recombination of existing resources from an older capacity (motor planning). We conclude by putting arguments into context regarding functional modularity, embodied representation, and the current ontology of mental processes.

Preparatory attention incorporates contextual expectations

Humans are remarkably proficient at finding objects within complex visual scenes. According to current theories of attention,^1-3 visual processing of an object of interest is favored through the preparatory activation of object-specific representations in visual cortex.^4-15 One key problem that is inherent to real-world visual search but is not accounted for by current theories is that a given object will produce a dramatically different retinal image depending on its location, which is unknown in advance. For instance, the color of the retinal image depends on the illumination on the object, its shape depends on the viewpoint, and (most critically) its size can vary by several orders of magnitude, depending on the distance to the observer. In order to benefit search, preparatory activity thus needs to incorporate contextual expectations. In the current study, we measured fMRI blood-oxygen-level-dependent (BOLD) activity in human observers while they prepared to search for objects at different distances in indoor-scene photographs. First, we established that observers instantiated preparatory object representations: activity patterns in object-selective cortex evoked during search preparation (while no objects were presented) resembled activity patterns evoked by viewing those objects in isolation. Second, we demonstrated that these preparatory object representations were systematically modulated by expectations derived from scene context: activity patterns reflected the predicted retinal image of the object at each distance (i.e., distant search evoking smaller object representations and nearby search evoking larger object representations). These findings reconcile current theories of attentional selection with the challenges of real-world vision.

The neural coding of face and body orientation in occipitotemporal cortex

Face and body orientation convey important information for us to understand other people's actions, intentions and social interactions. It has been shown that several occipitotemporal areas respond differently to faces or bodies of different orientations. However, whether face and body orientation are processed by partially overlapping or completely separate brain networks remains unclear, as the neural coding of face and body orientation is often investigated separately. Here, we recorded participants' brain activity using fMRI while they viewed faces and bodies shown from three different orientations, while attending to either orientation or identity information. Using multivoxel pattern analysis we investigated which brain regions process face and body orientation respectively, and which regions encode both face and body orientation in a stimulus-independent manner. We found that patterns of neural responses evoked by different stimulus orientations in the occipital face area, extrastriate body area, lateral occipital complex and right early visual cortex could generalise across faces and bodies, suggesting a stimulus-independent encoding of person orientation in occipitotemporal cortex. This finding was consistent across functionally defined regions of interest and a whole-brain searchlight approach. The fusiform face area responded to face but not body orientation, suggesting that orientation responses in this area are face-specific. Moreover, neural responses to orientation were remarkably consistent regardless of whether participants attended to the orientation of faces and bodies or not. Together, these results demonstrate that face and body orientation are processed in a partially overlapping brain network, with a stimulus-independent neural code for face and body orientation in occipitotemporal cortex.

Contrast Adaptation in Face Perception Revealed Through EEG and Behavior

Exposure to a face can produce biases in the perception of subsequent faces. Typically, these face aftereffects are studied by adapting to an individual face or category (e.g., faces of a given gender) and can result in renormalization of perceptions such that the adapting face appears more neutral. These shifts are analogous to chromatic adaptation, where a renormalization for the average adapting color occurs. However, in color vision, adaptation can also adjust to the variance or range of colors in the distribution. We examined whether this variance or contrast adaptation also occurs for faces, using an objective EEG measure to assess response changes following adaptation. An average female face was contracted or expanded along the horizontal or vertical axis to form four images. Observers viewed a 20 s sequence of the four images presented in a fixed order at a rate of 6 Hz, while responses to the faces were recorded with EEG. A 6 Hz signal was observed over right occipito-temporal channels, indicating symmetric responses to the four images. This test sequence was repeated after 20 s adaptation to alternations between two of the faces (e.g., horizontal contracted and expanded). This adaptation resulted in an additional signal at 3 Hz, consistent with asymmetric responses to adapted and non-adapted test faces. Adapting pairs have the same mean (undistorted) as the test sequence and thus should not bias responses driven only by the mean. Instead, the results are consistent with selective adaptation to the distortion axis. A 3 Hz signal was also observed after adapting to face pairs selected to induce a mean bias (e.g., expanded vertical and expanded horizontal), and this signal was not significantly different from that observed following adaption to a single image that did not form part of the test sequence (e.g., a single image expanded both vertically and horizontally). In a further experiment, we found that this variance adaptation can also be observed behaviorally. Our results suggest that adaptation calibrates face perception not only for the average characteristics of the faces we experience but also for the gamut of faces to which we are exposed.

Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations

How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.

The extrastriate symmetry response is robust to variation in visual memory load

An Event Related Potential response to visual symmetry, known as the Sustained Posterior Negativity (SPN), is generated whether symmetry is task relevant or not, and whether symmetry is attended or not. However, no study has yet examined interference from concurrent memory tasks. To answer this fundamental question, we investigated whether the SPN is robust to variation in Visual Working Memory (VWM) load. In Experiment 1 (N = 24), each trial involved a sample display, a probe and a test display. Sample and test displays contained either four colors or four black shapes, and the probe was either a symmetrical or random pattern. We compared a memory task and a passive viewing task. In the memory task, participants held color or shape information in VWM when the probe was presented. In the passive viewing task, there were no memory demands. Contrary to our predictions, there was no evidence that VWM interfered with the symmetry response. Instead, there was a general SPN enhancement during both color and shape memory tasks compared to passive viewing. In Experiment 2 (N = 24), we used symmetrical patterns themselves as sample and test to maximize interference. Again, the SPN was enhanced in the memory task compared to passive viewing. We conclude that the visual symmetry response is not impaired by concurrent VWM tasks, even when these tasks involve remembering symmetry itself. It seems that the SPN is not only attention-proof, but also memory-proof. This adds to evidence that symmetry perception is robust and automatic.