Rapid and dynamic processing of face pareidolia in the human brain

Neural representations of perspectival shapes and attentional effects: Evidence from fMRI and MEG

Does the human brain represent perspectival shapes, i.e., viewpoint-dependent object shapes, especially in relatively higher-level visual areas such as the lateral occipital cortex? What is the temporal profile of the appearance and disappearance of neural representations of perspectival shapes? And how does attention influence these neural representations? To answer these questions, we employed functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and multivariate decoding techniques to investigate spatiotemporal neural representations of perspectival shapes. Participants viewed rotated objects along with the corresponding objective shapes and perspectival shapes (i.e., rotated round, round, and oval) while we measured their brain activities. Our results revealed that shape classifiers trained on the basic shapes (i.e., round and oval) consistently identified neural representations in the lateral occipital cortex corresponding to the perspectival shapes of the viewed objects regardless of attentional manipulations. Additionally, this classification tendency toward the perspectival shapes emerged approximately 200 ms after stimulus presentation. Moreover, attention influenced the spatial dimension as the regions showing the perspectival shape classification tendency propagated from the occipital lobe to the temporal lobe. As for the temporal dimension, attention led to a more robust and enduring classification tendency towards perspectival shapes. In summary, our study outlines a spatiotemporal neural profile for perspectival shapes that suggests a greater degree of perspectival representation than is often acknowledged.

Beyond faces: the contribution of the amygdala to visual processing in the macaque brain

The amygdala is present in a diverse range of vertebrate species, such as lizards, rodents, and primates; however, its structure and connectivity differs across species. The increased connections to visual sensory areas in primate species suggests that understanding the visual selectivity of the amygdala in detail is critical to revealing the principles underlying its function in primate cognition. Therefore, we designed a high-resolution, contrast-agent enhanced, event-related fMRI experiment, and scanned 3 adult rhesus macaques, while they viewed 96 naturalistic stimuli. Half of these stimuli were social (defined by the presence of a conspecific), the other half were nonsocial. We also nested manipulations of emotional valence (positive, neutral, and negative) and visual category (faces, nonfaces, animate, and inanimate) within the stimulus set. The results reveal widespread effects of emotional valence, with the amygdala responding more on average to inanimate objects and animals than faces, bodies, or social agents in this experimental context. These findings suggest that the amygdala makes a contribution to primate vision that goes beyond an auxiliary role in face or social perception. Furthermore, the results highlight the importance of stimulus selection and experimental design when probing the function of the amygdala and other visually responsive brain regions.

Visual periodicity reveals distinct attentional signatures for face and non-face categories

Observers can selectively deploy attention to regions of space, moments in time, specific visual features, individual objects, and even specific high-level categories-for example, when keeping an eye out for dogs while jogging. Here, we exploited visual periodicity to examine how category-based attention differentially modulates selective neural processing of face and non-face categories. We combined electroencephalography with a novel frequency-tagging paradigm capable of capturing selective neural responses for multiple visual categories contained within the same rapid image stream (faces/birds in Exp 1; houses/birds in Exp 2). We found that the pattern of attentional enhancement and suppression for face-selective processing is unique compared to other object categories: Where attending to non-face objects strongly enhances their selective neural signals during a later stage of processing (300-500 ms), attentional enhancement of face-selective processing is both earlier and comparatively more modest. Moreover, only the selective neural response for faces appears to be actively suppressed by attending towards an alternate visual category. These results underscore the special status that faces hold within the human visual system, and highlight the utility of visual periodicity as a powerful tool for indexing selective neural processing of multiple visual categories contained within the same image sequence.

Movement trajectories as a window into the dynamics of emerging neural representations

The rapid transformation of sensory inputs into meaningful neural representations is critical to adaptive human behaviour. While non-invasive neuroimaging methods are the de-facto method for investigating neural representations, they remain expensive, not widely available, time-consuming, and restrictive. Here we show that movement trajectories can be used to measure emerging neural representations with fine temporal resolution. By combining online computer mouse-tracking and publicly available neuroimaging data via representational similarity analysis (RSA), we show that movement trajectories track the unfolding of stimulus- and category-wise neural representations along key dimensions of the human visual system. We demonstrate that time-resolved representational structures derived from movement trajectories overlap with those derived from M/EEG (albeit delayed) and those derived from fMRI in functionally-relevant brain areas. Our findings highlight the richness of movement trajectories and the power of the RSA framework to reveal and compare their information content, opening new avenues to better understand human perception.

A behavioral advantage for the face pareidolia illusion in peripheral vision

Investigation of visual illusions helps us understand how we process visual information. For example, face pareidolia, the misperception of illusory faces in objects, could be used to understand how we process real faces. However, it remains unclear whether this illusion emerges from errors in face detection or from slower, cognitive processes. Here, our logic is straightforward; if examples of face pareidolia activate the mechanisms that rapidly detect faces in visual environments, then participants will look at objects more quickly when the objects also contain illusory faces. To test this hypothesis, we sampled continuous eye movements during a fast saccadic choice task-participants were required to select either faces or food items. During this task, pairs of stimuli were positioned close to the initial fixation point or further away, in the periphery. As expected, the participants were faster to look at face targets than food targets. Importantly, we also discovered an advantage for food items with illusory faces but, this advantage was limited to the peripheral condition. These findings are among the first to demonstrate that the face pareidolia illusion persists in the periphery and, thus, it is likely to be a consequence of erroneous face detection.

Dynamic brain communication underwriting face pareidolia

Face pareidolia is a tendency to seeing faces in nonface images that reflects high tuning to a face scheme. Yet, studies of the brain networks underwriting face pareidolia are scarce. Here, we examined the time course and dynamic topography of gamma oscillatory neuromagnetic activity while administering a task with nonface images resembling a face. Images were presented either with canonical orientation or with display inversion that heavily impedes face pareidolia. At early processing stages, the peaks in gamma activity (40 to 45 Hz) to images either triggering or not face pareidolia originate mainly from the right medioventral and lateral occipital cortices, rostral and caudal cuneus gyri, and medial superior occipital gyrus. Yet, the difference occurred at later processing stages in the high-frequency range of 80 to 85 Hz over a set of the areas constituting the social brain. The findings speak rather for a relatively late neural network playing a key role in face pareidolia. Strikingly, a cutting-edge analysis of brain connectivity unfolding over time reveals mutual feedforward and feedback intra- and interhemispheric communication not only within the social brain but also within the extended large-scale network of down- and upstream regions. In particular, the superior temporal sulcus and insula strongly engage in communication with other brain regions either as signal transmitters or recipients throughout the whole processing of face-pareidolia images.

A limited visual search advantage for illusory faces

The human visual system is very sensitive to the presence of faces in the environment, so much so that it can produce the perception of illusory faces in everyday objects. Growing research suggests that illusory faces and real faces are processed by similar perceptual and neural mechanisms, but whether this similarity extends to visual attention is less clear. A visual search study showed that illusory faces have a search advantage over objects when the types of objects vary to match the objects in the illusory faces (e.g., chair, pepper, clock) (Keys et al., 2021). Here, we examine whether the search advantage for illusory faces over objects remains when compared against objects that belong to a single category (flowers). In three experiments, we compared visual search of illusory faces, real faces, variable objects, and uniform objects (flowers). Search for real faces was best compared with all other types of targets. In contrast, search for illusory faces was only better than search for variable objects, not uniform objects. This result shows a limited visual search advantage for illusory faces and suggests that illusory faces may not be processed like real faces in visual attention.

The phenomenology of pareidolia in healthy subjects and patients with left- or right-hemispheric stroke

Pareidolia are perceptions of recognizable images or meaningful patterns where none exist. In recent years, this phenomenon has been increasingly studied in healthy subjects and patients with neurological or psychiatric diseases. The current study examined pareidolia production in a group of 53 stroke patients and 82 neurologically healthy controls who performed a natural images task. We found a significant reduction of absolute pareidolia production in left- and right-hemispheric stroke patients, with right-hemispheric patients producing overall fewest pareidolic output. Responses were categorized into 28 distinct categories, with 'Animal', 'Human', 'Face', and 'Body parts' being the most common, accounting for 72% of all pareidolia. Regarding the percentages of the different categories of pareidolia, we found a significant reduction for the percentage of "Body parts" pareidolia in the left-hemispheric patient group as compared to the control group, while the percentage of this pareidolia type was not significantly reduced in right-hemispheric patients compared to healthy controls. These results support the hypothesis that pareidolia production may be influenced by local-global visual processing with the left hemisphere being involved in local and detailed analytical visual processing to a greater extent. As such, a lesion to the right hemisphere, that is believed to be critical for global visual processing, might explain the overall fewest pareidolic output produced by the right-hemispheric patients.

The anterior fusiform gyrus: The ghost in the cortical face machine

Face-selective regions in the human ventral occipito-temporal cortex (VOTC) have been defined for decades mainly with functional magnetic resonance imaging. This face-selective VOTC network is traditionally divided in a posterior 'core' system thought to subtend face perception, and regions of the anterior temporal lobe as a semantic memory component of an extended general system. In between these two putative systems lies the anterior fusiform gyrus and surrounding sulci, affected by magnetic susceptibility artifacts. Here we suggest that this methodological gap overlaps with and contributes to a conceptual gap between (visual) perception and semantic memory for faces. Filling this gap with intracerebral recordings and direct electrical stimulation reveals robust face-selectivity in the anterior fusiform gyrus and a crucial role of this region, especially in the right hemisphere, in identity recognition for both familiar and unfamiliar faces. Based on these observations, we propose an integrated theoretical framework for human face (identity) recognition according to which face-selective regions in the anterior fusiform gyrus join the dots between posterior and anterior cortical face memories.

Asymmetric stimulus representations bias visual perceptual learning

The primate visual cortex contains various regions that exhibit specialization for different stimulus properties, such as motion, shape, and color. Within each region, there is often further specialization, such that particular stimulus features, such as horizontal and vertical orientations, are over-represented. These asymmetries are associated with well-known perceptual biases, but little is known about how they influence visual learning. Most theories would predict that learning is optimal, in the sense that it is unaffected by these asymmetries. However, other approaches to learning would result in specific patterns of perceptual biases. To distinguish between these possibilities, we trained human observers to discriminate between expanding and contracting motion patterns, which have a highly asymmetrical representation in the visual cortex. Observers exhibited biased percepts of these stimuli, and these biases were affected by training in ways that were often suboptimal. We simulated different neural network models and found that a learning rule that involved only adjustments to decision criteria, rather than connection weights, could account for our data. These results suggest that cortical asymmetries influence visual perception and that human observers often rely on suboptimal strategies for learning.

Distinct patterns of neural response to faces from different races in humans and deep networks

Social categories such as the race or ethnicity of an individual are typically conveyed by the visual appearance of the face. The aim of this study was to explore how these differences in facial appearance are represented in human and artificial neural networks. First, we compared the similarity of faces from different races using a neural network trained to discriminate identity. We found that the differences between races were most evident in the fully connected layers of the network. Although these layers were also able to predict behavioural judgements of face identity from human participants, performance was biased toward White faces. Next, we measured the neural response in face-selective regions of the human brain to faces from different races in Asian and White participants. We found distinct patterns of response to faces from different races in face-selective regions. We also found that the spatial pattern of response was more consistent across participants for own-race compared to other-race faces. Together, these findings show that faces from different races elicit different patterns of response in human and artificial neural networks. These differences may underlie the ability to make categorical judgements and explain the behavioural advantage for the recognition of own-race faces.

When the whole is only the parts: non-holistic object parts predominate face-cell responses to illusory faces

Humans are inclined to perceive faces in everyday objects with a face-like configuration. This illusion, known as face pareidolia, is often attributed to a specialized network of 'face cells' in primates. We found that face cells in macaque inferotemporal cortex responded selectively to pareidolia images, but this selectivity did not require a holistic, face-like configuration, nor did it encode human faceness ratings. Instead, it was driven mostly by isolated object parts that are perceived as eyes only within a face-like context. These object parts lack usual characteristics of primate eyes, pointing to the role of lower-level features. Our results suggest that face-cell responses are dominated by local, generic features, unlike primate visual perception, which requires holistic information. These findings caution against interpreting neural activity through the lens of human perception. Doing so could impose human perceptual biases, like seeing faces where none exist, onto our understanding of neural activity.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Preliminary evidence of an increased susceptibility to face pareidolia in postpartum women

As primates, we are hypersensitive to faces and face-like patterns in the visual environment, hence we often perceive illusory faces in otherwise inanimate objects, such as burnt pieces of toast and the surface of the moon. Although this phenomenon, known as face pareidolia, is a common experience, it is unknown whether our susceptibility to face pareidolia is static across our lifespan or what factors would cause it to change. Given the evidence that behaviour towards face stimuli is modulated by the neuropeptide oxytocin (OT), we reasoned that participants in stages of life associated with high levels of endogenous OT might be more susceptible to face pareidolia than participants in other stages of life. We tested this hypothesis by assessing pareidolia susceptibility in two groups of women; pregnant women (low endogenous OT) and postpartum women (high endogenous OT). We found evidence that postpartum women report seeing face pareidolia more easily than women who are currently pregnant. These data, collected online, suggest that our sensitivity to face-like patterns is not fixed and may change throughout adulthood, providing a crucial proof of concept that requires further research.

Children perceive illusory faces in objects as male more often than female

Face pareidolia is the experience of seeing illusory faces in inanimate objects. While children experience face pareidolia, it is unknown whether they perceive gender in illusory faces, as their face evaluation system is still developing in the first decade of life. In a sample of 412 children and adults from 4 to 80 years of age we found that like adults, children perceived many illusory faces in objects to have a gender and had a strong bias to see them as male rather than female, regardless of their own gender identification. These results provide evidence that the male bias for face pareidolia emerges early in life, even before the ability to discriminate gender from facial cues alone is fully developed. Further, the existence of a male bias in children suggests that any social context that elicits the cognitive bias to see faces as male has remained relatively consistent across generations.

Overwhelmed by the man in the moon? Pareidolic objects provoke increased amygdala activation in autism

An interesting feature of the primate face detection system results in the perception of illusory faces in objects, or pareidolia. These illusory faces do not per se contain social information, such as eye-gaze or specific identities, yet they activate the cortical brain face-processing network, possibly via the subcortical route, including the amygdala. In autism spectrum disorder (ASD), aversion to eye-contact is commonly reported, and so are alterations in face processing more generally, yet the underlying reasons are not clear. Here we show that in autistic participants (N=37), but not in non-autistic controls (N=34), pareidolic objects increase amygdala activation bilaterally (right amygdala peak: X = 26, Y = -6, Z = -16; left amygdala peak X = -24, Y = -6, Z = -20). In addition, illusory faces engage the face-processing cortical network significantly more in ASD than in controls. An early imbalance in the excitatory and inhibitory systems in autism, affecting typical brain maturation, may be at the basis of an overresponsive reaction to face configuration and to eye contact. Our data add to the evidence of an oversensitive subcortical face processing system in ASD.

The representational hierarchy in human and artificial visual systems in the presence of object-scene regularities

Human vision is still largely unexplained. Computer vision made impressive progress on this front, but it is still unclear to which extent artificial neural networks approximate human object vision at the behavioral and neural levels. Here, we investigated whether machine object vision mimics the representational hierarchy of human object vision with an experimental design that allows testing within-domain representations for animals and scenes, as well as across-domain representations reflecting their real-world contextual regularities such as animal-scene pairs that often co-occur in the visual environment. We found that DCNNs trained in object recognition acquire representations, in their late processing stage, that closely capture human conceptual judgements about the co-occurrence of animals and their typical scenes. Likewise, the DCNNs representational hierarchy shows surprising similarities with the representational transformations emerging in domain-specific ventrotemporal areas up to domain-general frontoparietal areas. Despite these remarkable similarities, the underlying information processing differs. The ability of neural networks to learn a human-like high-level conceptual representation of object-scene co-occurrence depends upon the amount of object-scene co-occurrence present in the image set thus highlighting the fundamental role of training history. Further, although mid/high-level DCNN layers represent the category division for animals and scenes as observed in VTC, its information content shows reduced domain-specific representational richness. To conclude, by testing within- and between-domain selectivity while manipulating contextual regularities we reveal unknown similarities and differences in the information processing strategies employed by human and artificial visual systems.

A database of heterogeneous faces for studying naturalistic expressions

Facial expressions are thought to be complex visual signals, critical for communication between social agents. Most prior work aimed at understanding how facial expressions are recognized has relied on stimulus databases featuring posed facial expressions, designed to represent putative emotional categories (such as 'happy' and 'angry'). Here we use an alternative selection strategy to develop the Wild Faces Database (WFD); a set of one thousand images capturing a diverse range of ambient facial behaviors from outside of the laboratory. We characterized the perceived emotional content in these images using a standard categorization task in which participants were asked to classify the apparent facial expression in each image. In addition, participants were asked to indicate the intensity and genuineness of each expression. While modal scores indicate that the WFD captures a range of different emotional expressions, in comparing the WFD to images taken from other, more conventional databases, we found that participants responded more variably and less specifically to the wild-type faces, perhaps indicating that natural expressions are more multiplexed than a categorical model would predict. We argue that this variability can be employed to explore latent dimensions in our mental representation of facial expressions. Further, images in the WFD were rated as less intense and more genuine than images taken from other databases, suggesting a greater degree of authenticity among WFD images. The strong positive correlation between intensity and genuineness scores demonstrating that even the high arousal states captured in the WFD were perceived as authentic. Collectively, these findings highlight the potential utility of the WFD as a new resource for bridging the gap between the laboratory and real world in studies of expression recognition.

Pareidolic faces receive prioritized attention in the dot-probe task

Face pareidolia occurs when random or ambiguous inanimate objects are perceived as faces. While real faces automatically receive prioritized attention compared with nonface objects, it is unclear whether pareidolic faces similarly receive special attention. We hypothesized that, given the evolutionary importance of broadly detecting animacy, pareidolic faces may have enough faceness to activate a broad face template, triggering prioritized attention. To test this hypothesis, and to explore where along the faceness continuum pareidolic faces fall, we conducted a series of dot-probe experiments in which we paired pareidolic faces with other images directly competing for attention: objects, animal faces, and human faces. We found that pareidolic faces elicited more prioritized attention than objects, a process that was disrupted by inversion, suggesting this prioritized attention was unlikely to be driven by low-level features. However, unexpectedly, pareidolic faces received more privileged attention compared with animal faces and showed similar prioritized attention to human faces. This attentional efficiency may be due to pareidolic faces being perceived as not only face-like, but also as human-like, and having larger facial features-eyes and mouths-compared with real faces. Together, our findings suggest that pareidolic faces appear automatically attentionally privileged, similar to human faces. Our findings are consistent with the proposal of a highly sensitive broad face detection system that is activated by pareidolic faces, triggering false alarms (i.e., illusory faces), which, evolutionarily, are less detrimental relative to missing potentially relevant signals (e.g., conspecific or heterospecific threats). In sum, pareidolic faces appear "special" in attracting attention.

Behavioral and physiological sensitivity to natural sick faces

The capacity to rapidly detect and avoid sick people may be adaptive. Given that faces are reliably available, as well as rapidly detected and processed, they may provide health information that influences social interaction. Prior studies used faces that were manipulated to appear sick (e.g., editing photos, inducing inflammatory response); however, responses to naturally sick faces remain largely unexplored. We tested whether adults detected subtle cues of genuine, acute, potentially contagious illness in face photos compared to the same individuals when healthy. We tracked illness symptoms and severity with the Sickness Questionnaire and Common Cold Questionnaire. We also checked that sick and healthy photos were matched on low-level features. We found that participants (N = 109) rated sick faces, compared to healthy faces, as sicker, more dangerous, and eliciting more unpleasant feelings. Participants (N = 90) rated sick faces as more likely to be avoided, more tired, and more negative in expression than healthy faces. In a passive-viewing eye-tracking task, participants (N = 50) looked longer at healthy than sick faces, especially the eye region, suggesting people may be more drawn to healthy conspecifics. When making approach-avoidance decisions, participants (N = 112) had greater pupil dilation to sick than healthy faces, and more pupil dilation was associated with greater avoidance, suggesting elevated arousal to threat. Across all experiments, participants' behaviors correlated with the degree of sickness, as reported by the face donors, suggesting a nuanced, fine-tuned sensitivity. Together, these findings suggest that humans may detect subtle threats of contagion from sick faces, which may facilitate illness avoidance. By better understanding how humans naturally avoid illness in conspecifics, we may identify what information is used and ultimately improve public health.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

Face pareidolia is enhanced by 40 Hz transcranial alternating current stimulation (tACS) of the face perception network

Pareidolia refers to the perception of ambiguous sensory patterns as carrying a specific meaning. In its most common form, pareidolia involves human-like facial features, where random objects or patterns are illusionary recognized as faces. The current study investigated the neurophysiological correlates of face pareidolia via transcranial alternating current stimulation (tACS). tACS was delivered at gamma (40 Hz) frequency over critical nodes of the "face perception" network (i.e., right lateral occipito-temporal and left prefrontal cortex) of 75 healthy participants while completing four face perception tasks ('Mooney test' for faces, 'Toast test', 'Noise pareidolia test', 'Pareidolia task') and an object perception task ('Mooney test' for objects). In this single-blind, sham-controlled between-subjects study, participants received 35 min of either Sham, Online, (40Hz-tACS_ON), or Offline (40Hz-tACS_PRE) stimulation. Results showed that face pareidolia was causally enhanced by 40Hz-tACS_PRE in the Mooney test for faces in which, as compared to sham, participants more often misperceived scrambled stimuli as faces. In addition, as compared to sham, participants receiving 40Hz-tACS_PRE showed similar reaction times (RTs) when perceiving illusory faces and correctly recognizing noise stimuli in the Toast test, thus not exhibiting hesitancy in identifying faces where there were none. Also, 40Hz-tACS_ON induced slower rejections of face pareidolia responses in the Noise pareidolia test. The current study indicates that 40 Hz tACS can enhance pareidolic illusions in healthy individuals and, thus, that high frequency (i.e., gamma band) oscillations are critical in forming coherent and meaningful visual perception.

Emotion recognition in autism spectrum disorder across age groups: A cross-sectional investigation of various visual and auditory communicative domains

Previous research on emotion processing in autism spectrum disorder (ASD) has predominantly focused on human faces and speech prosody, with little attention paid to other domains such as nonhuman faces and music. In addition, emotion processing in different domains was often examined in separate studies, making it challenging to evaluate whether emotion recognition difficulties in ASD generalize across domains and age cohorts. The present study investigated: (i) the recognition of basic emotions (angry, scared, happy, and sad) across four domains (human faces, face-like objects, speech prosody, and song) in 38 autistic and 38 neurotypical (NT) children, adolescents, and adults in a forced-choice labeling task, and (ii) the impact of pitch and visual processing profiles on this ability. Results showed similar recognition accuracy between the ASD and NT groups across age groups for all domains and emotion types, although processing speed was slower in the ASD compared to the NT group. Age-related differences were seen in both groups, which varied by emotion, domain, and performance index. Visual processing style was associated with facial emotion recognition speed and pitch perception ability with auditory emotion recognition in the NT group but not in the ASD group. These findings suggest that autistic individuals may employ different emotion processing strategies compared to NT individuals, and that emotion recognition difficulties as manifested by slower response times may result from a generalized, rather than a domain-specific underlying mechanism that governs emotion recognition processes across domains in ASD.

Understanding Human Object Vision: A Picture Is Worth a Thousand Representations

Objects are the core meaningful elements in our visual environment. Classic theories of object vision focus upon object recognition and are elegant and simple. Some of their proposals still stand, yet the simplicity is gone. Recent evolutions in behavioral paradigms, neuroscientific methods, and computational modeling have allowed vision scientists to uncover the complexity of the multidimensional representational space that underlies object vision. We review these findings and propose that the key to understanding this complexity is to relate object vision to the full repertoire of behavioral goals that underlie human behavior, running far beyond object recognition. There might be no such thing as core object recognition, and if it exists, then its importance is more limited than traditionally thought.

Sustained neural representations of personally familiar people and places during cued recall

The recall and visualization of people and places from memory is an everyday occurrence, yet the neural mechanisms underpinning this phenomenon are not well understood. In particular, the temporal characteristics of the internal representations generated by active recall are unclear. Here, we used magnetoencephalography (MEG) and multivariate pattern analysis to measure the evolving neural representation of familiar places and people across the whole brain when human participants engage in active recall. To isolate self-generated imagined representations, we used a retro-cue paradigm in which participants were first presented with two possible labels before being cued to recall either the first or second item. We collected personalized labels for specific locations and people familiar to each participant. Importantly, no visual stimuli were presented during the recall period, and the retro-cue paradigm allowed the dissociation of responses associated with the labels from those corresponding to the self-generated representations. First, we found that following the retro-cue it took on average ∼1000 ms for distinct neural representations of freely recalled people or places to develop. Second, we found distinct representations of personally familiar concepts throughout the 4 s recall period. Finally, we found that these representations were highly stable and generalizable across time. These results suggest that self-generated visualizations and recall of familiar places and people are subserved by a stable neural mechanism that operates relatively slowly when under conscious control.

Do chimpanzees see a face on Mars? A search for face pareidolia in chimpanzees

We sometimes perceive meaningful patterns or images in random arrangements of colors and shapes. This phenomenon is called pareidolia and has recently been studied intensively, especially face pareidolia. In contrast, there are few comparative-cognitive studies on face pareidolia with nonhuman primates. This study explored behavioral evidence for face pareidolia in chimpanzees using visual search and matching tasks. Faces are processed in a configural manner, and their perception and recognition are hampered by inversion and misalignment of top and bottom parts. We investigated whether the same effect occurs in a visual search for face-like objects. The results showed an effect of misalignment. On the other hand, consistent results were not obtained with the photographs of fruits. When only the top or bottom half of the face-like object was presented, chimpanzees showed better performance for the top-half condition, suggesting the importance of the eye area in face pareidolia. In the positive-control experiments, chimpanzees received the same experiment using human faces and human participants with face-like objects and fruits. As a result, chimpanzees showed an inefficient search for inverted and misaligned faces and humans for manipulated face-like objects. Finally, to examine the role of face awareness, we tested matching a human face to a face-like object in chimpanzees but obtained no substantial evidence that they saw the face-like object as a "face." Based on these results, we discussed the extents and limits of face pareidolia in chimpanzees.

Brain-inspired Predictive Coding Improves the Performance of Machine Challenging Tasks

Backpropagation has been regarded as the most favorable algorithm for training artificial neural networks. However, it has been criticized for its biological implausibility because its learning mechanism contradicts the human brain. Although backpropagation has achieved super-human performance in various machine learning applications, it often shows limited performance in specific tasks. We collectively referred to such tasks as machine-challenging tasks (MCTs) and aimed to investigate methods to enhance machine learning for MCTs. Specifically, we start with a natural question: Can a learning mechanism that mimics the human brain lead to the improvement of MCT performances? We hypothesized that a learning mechanism replicating the human brain is effective for tasks where machine intelligence is difficult. Multiple experiments corresponding to specific types of MCTs where machine intelligence has room to improve performance were performed using predictive coding, a more biologically plausible learning algorithm than backpropagation. This study regarded incremental learning, long-tailed, and few-shot recognition as representative MCTs. With extensive experiments, we examined the effectiveness of predictive coding that robustly outperformed backpropagation-trained networks for the MCTs. We demonstrated that predictive coding-based incremental learning alleviates the effect of catastrophic forgetting. Next, predictive coding-based learning mitigates the classification bias in long-tailed recognition. Finally, we verified that the network trained with predictive coding could correctly predict corresponding targets with few samples. We analyzed the experimental result by drawing analogies between the properties of predictive coding networks and those of the human brain and discussing the potential of predictive coding networks in general machine learning.

Processing visual ambiguity in fractal patterns: Pareidolia as a sign of creativity

Creativity is a highly valued and beneficial skill that empirical research typically probes using “divergent thinking” (DT) tasks such as problem solving and novel idea generation. Here, in contrast, we examine the perceptual aspect of creativity by asking whether creative individuals are more likely to perceive recognizable forms in ambiguous stimuli –a phenomenon known as pareidolia. To this end, we designed a visual task in which participants were asked to identify as many recognizable forms as possible in cloud-like fractal images. We found that pareidolic perceptions arise more often and more rapidly in highly creative individuals. Furthermore, high-creatives report pareidolia across a broader range of image contrasts and fractal dimensions than do low creatives. These results extend the established body of work on DT by introducing divergent perception as a complementary manifestation of the creative mind, thus clarifying the perception-creation link while opening new paths for studying creative behavior in humans.

•

Creativity has been linked to divergent thinking

•

Creativity is associated with enhanced pareidolia (i.e., divergent perception)

•

High-creatives report pareidolia across a broader range of image fractal dimensions

•

Divergent perception constitutes a promising phenomenon for the study of creativity

Human faces and face-like stimuli are more memorable

We have previously suggested a distinction in the brain processes governing biological and artifactual stimuli. One of the best examples of the biological category consists of human faces, the perception of which appears to be determined by inherited mechanisms or ones rapidly acquired after birth. In extending this work, we inquire here whether there is a higher memorability for images of human faces and whether memorability declines with increasing departure from human faces; if so, the implication would add to the growing evidence of differences in the processing of biological versus artifactual stimuli. To do so, we used images and memorability scores from a large data set of 58,741 images to compare the relative memorability of the following image categories: real human faces versus buildings, and extending this to a comparison of real human faces with five image categories that differ in their grade of resemblance to a real human face. Our findings show that, in general, when we compare the biological category of faces to the artifactual category of buildings, the former is more memorable. Furthermore, there is a gradient in which the more an image resembles a real human face the more memorable it is. Thus, the previously identified differences in biological and artifactual images extend to the field of memory.

Clutter Substantially Reduces Selectivity for Peripheral Faces in the Macaque Brain

According to a prominent view in neuroscience, visual stimuli are coded by discrete cortical networks that respond preferentially to specific categories, such as faces or objects. However, it remains unclear how these category-selective networks respond when viewing conditions are cluttered, i.e., when there is more than one stimulus in the visual field. Here, we asked three questions: (1) Does clutter reduce the response and selectivity for faces as a function of retinal location? (2) Is the preferential response to faces uniform across the visual field? And (3) Does the ventral visual pathway encode information about the location of cluttered faces? We used fMRI to measure the response of the face-selective network in awake, fixating macaques (two female, five male). Across a series of four experiments, we manipulated the presence and absence of clutter, as well as the location of the faces relative to the fovea. We found that clutter reduces the response to peripheral faces. When presented in isolation, without clutter, the selectivity for faces is fairly uniform across the visual field, but, when clutter is present, there is a marked decrease in the selectivity for peripheral faces. We also found no evidence of a contralateral visual field bias when faces were presented in clutter. Nonetheless, multivariate analyses revealed that the location of cluttered faces could be decoded from the multivoxel response of the face-selective network. Collectively, these findings demonstrate that clutter blunts the selectivity of the face-selective network to peripheral faces, although information about their retinal location is retained.SIGNIFICANCE STATEMENT Numerous studies that have measured brain activity in macaques have found visual regions that respond preferentially to faces. Although these regions are thought to be essential for social behavior, their responses have typically been measured while faces were presented in isolation, a situation atypical of the real world. How do these regions respond when faces are presented with other stimuli? We report that, when clutter is present, the preferential response to foveated faces is spared but preferential response to peripheral faces is reduced. Our results indicate that the presence of clutter changes the response of the face-selective network.

Are you for real? Decoding realistic AI-generated faces from neural activity

Can we trust our eyes? Until recently, we rarely had to question whether what we see is indeed what exists, but this is changing. Artificial neural networks can now generate realistic images that challenge our perception of what is real. This new reality can have significant implications for cybersecurity, counterfeiting, fake news, and border security. We investigated how the human brain encodes and interprets realistic artificially generated images using behaviour and brain imaging. We found that we could reliably decode AI generated faces using people's neural activity. However, while at a group level people performed near chance classifying real and realistic fakes, participants tended to interchange the labels, classifying real faces as realistic fakes and vice versa. Understanding this difference between brain and behavioural responses may be key in determining the 'real' in our new reality. Stimuli, code, and data for this study can be found at https://osf.io/n2z73/.

EEG decoding of multidimensional information from emotional faces

Humans can detect and recognize faces quickly, but there has been little research on the temporal dynamics of the different dimensional face information that is extracted. The present study aimed to investigate the time course of neural responses to the representation of different dimensional face information, such as age, gender, emotion, and identity. We used support vector machine decoding to obtain representational dissimilarity matrices of event-related potential responses to different faces for each subject over time. In addition, we performed representational similarity analysis with the model representational dissimilarity matrices that contained different dimensional face information. Three significant findings were observed. First, the extraction process of facial emotion occurred before that of facial identity and lasted for a long time, which was specific to the right frontal region. Second, arousal was preferentially extracted before valence during the processing of facial emotional information. Third, different dimensional face information exhibited representational stability during different periods. In conclusion, these findings reveal the precise temporal dynamics of multidimensional information processing in faces and provide powerful support for computational models on emotional face perception.

The cortical and subcortical correlates of face pareidolia in the macaque brain

Face detection is a foundational social skill for primates. This vital function is thought to be supported by specialized neural mechanisms; however, although several face-selective regions have been identified in both humans and nonhuman primates, there is no consensus about which region(s) are involved in face detection. Here, we used naturally occurring errors of face detection (i.e. objects with illusory facial features referred to as examples of 'face pareidolia') to identify regions of the macaque brain implicated in face detection. Using whole-brain functional magnetic resonance imaging to test awake rhesus macaques, we discovered that a subset of face-selective patches in the inferior temporal cortex, on the lower lateral edge of the superior temporal sulcus, and the amygdala respond more to objects with illusory facial features than matched non-face objects. Multivariate analyses of the data revealed differences in the representation of illusory faces across the functionally defined regions of interest. These differences suggest that the cortical and subcortical face-selective regions contribute uniquely to the detection of facial features. We conclude that face detection is supported by a multiplexed system in the primate brain.

Face neurons encode nonsemantic features

Significance Face neurons, which fire more strongly in response to images of faces than to other objects, are a paradigmatic example of object selectivity in the visual cortex. We asked whether such neurons represent the semantic concept of faces or, rather, visual features that are present in faces but do not necessarily count as a face. We created synthetic stimuli that strongly activated face neurons and showed that these stimuli were perceived as clearly distinct from real faces. At the same time, these synthetic stimuli were slightly more often associated with faces than other objects were. These results suggest that so-called face neurons do not represent a semantic category but, rather, represent visual features that correlate with faces.

What happens to our representation of identity as familiar faces age? Evidence from priming and identity aftereffects

Matching identity in images of unfamiliar faces is error prone, but we can easily recognize highly variable images of familiar faces – even images taken decades apart. Recent theoretical development based on computational modelling can account for how we recognize extremely variable instances of the same identity. We provide complementary behavioural data by examining older adults’ representation of older celebrities who were also famous when young. In Experiment 1, participants completed a long‐lag repetition priming task in which primes and test stimuli were the same age or different ages. In Experiment 2, participants completed an identity after effects task in which the adapting stimulus was an older or young photograph of one celebrity and the test stimulus was a morph between the adapting identity and a different celebrity; the adapting stimulus was the same age as the test stimulus on some trials (e.g., both old) or a different age (e.g., adapter young, test stimulus old). The magnitude of priming and identity after effects were not influenced by whether the prime and adapting stimulus were the same age or different age as the test face. Collectively, our findings suggest that humans have one common mental representation for a familiar face (e.g., Paul McCartney) that incorporates visual changes across decades, rather than multiple age‐specific representations. These findings make novel predictions for state‐of‐the‐art algorithms (e.g., Deep Convolutional Neural Networks).

Staring death in the face: chimpanzees' attention towards conspecific skulls and the implications of a face module guiding their behaviour

Chimpanzees exhibit a variety of behaviours surrounding their dead, although much less is known about how they respond towards conspecific skeletons. We tested chimpanzees' visual attention to images of conspecific and non-conspecific stimuli (cat/chimp/dog/rat), shown simultaneously in four corners of a screen in distinct orientations (frontal/diagonal/lateral) of either one of three types (faces/skulls/skull-shaped stones). Additionally, we compared their visual attention towards chimpanzee-only stimuli (faces/skulls/skull-shaped stones). Lastly, we tested their attention towards specific regions of chimpanzee skulls. We theorized that chimpanzee skulls retaining face-like features would be perceived similarly to chimpanzee faces and thus be subjected to similar biases. Overall, supporting our hypotheses, the chimpanzees preferred conspecific-related stimuli. The results showed that chimpanzees attended: (i) significantly longer towards conspecific skulls than other species skulls (particularly in forward-facing and to a lesser extent diagonal orientations); (ii) significantly longer towards conspecific faces than other species faces at forward-facing and diagonal orientations; (iii) longer towards chimpanzee faces compared with chimpanzee skulls and skull-shaped stones, and (iv) attended significantly longer to the teeth, similar to findings for elephants. We suggest that chimpanzee skulls retain relevant, face-like features that arguably activate a domain-specific face module in chimpanzees' brains, guiding their attention.

The role of animal faces in the animate-inanimate distinction in the ventral temporal cortex

Animate and inanimate objects elicit distinct response patterns in the human ventral temporal cortex (VTC), but the exact features driving this distinction are still poorly understood. One prominent feature that distinguishes typical animals from inanimate objects and that could potentially explain the animate-inanimate distinction in the VTC is the presence of a face. In the current fMRI study, we investigated this possibility by creating a stimulus set that included animals with faces, faceless animals, and inanimate objects, carefully matched in order to minimize other visual differences. We used both searchlight-based and ROI-based representational similarity analysis (RSA) to test whether the presence of a face explains the animate-inanimate distinction in the VTC. The searchlight analysis revealed that when animals with faces were removed from the analysis, the animate-inanimate distinction almost disappeared. The ROI-based RSA revealed a similar pattern of results, but also showed that, even in the absence of faces, information about agency (a combination of animal's ability to move and think) is present in parts of the VTC that are sensitive to animacy. Together, these analyses showed that animals with faces do elicit a stronger animate/inanimate response in the VTC, but that faces are not necessary in order to observe high-level animacy information (e.g., agency) in parts of the VTC. A possible explanation could be that this animacy-related activity is driven not by faces per se, or the visual features of faces, but by other factors that correlate with face presence, such as the capacity for self-movement and thought. In short, the VTC might treat the face as a proxy for agency, a ubiquitous feature of familiar animals.

Illusory faces are more likely to be perceived as male than female

Face pareidolia is the phenomenon of perceiving illusory faces in inanimate objects. Here we show that illusory faces engage social perception beyond the detection of a face: they have a perceived age, gender, and emotional expression. Additionally, we report a striking bias in gender perception, with many more illusory faces perceived as male than female. As illusory faces do not have a biological sex, this bias is significant in revealing an asymmetry in our face evaluation system given minimal information. Our result demonstrates that the visual features that are sufficient for face detection are not generally sufficient for the perception of female. Instead, the perception of a nonhuman face as female requires additional features beyond that required for face detection.

Despite our fluency in reading human faces, sometimes we mistakenly perceive illusory faces in objects, a phenomenon known as face pareidolia. Although illusory faces share some neural mechanisms with real faces, it is unknown to what degree pareidolia engages higher-level social perception beyond the detection of a face. In a series of large-scale behavioral experiments (n_total = 3,815 adults), we found that illusory faces in inanimate objects are readily perceived to have a specific emotional expression, age, and gender. Most strikingly, we observed a strong bias to perceive illusory faces as male rather than female. This male bias could not be explained by preexisting semantic or visual gender associations with the objects, or by visual features in the images. Rather, this robust bias in the perception of gender for illusory faces reveals a cognitive bias arising from a broadly tuned face evaluation system in which minimally viable face percepts are more likely to be perceived as male.

Rapid neural categorization of facelike objects predicts the perceptual awareness of a face (face pareidolia)

The human brain rapidly and automatically categorizes faces vs. other visual objects. However, whether face-selective neural activity predicts the subjective experience of a face - perceptual awareness - is debated. To clarify this issue, here we use face pareidolia, i.e., the illusory perception of a face, as a proxy to relate the neural categorization of a variety of facelike objects to conscious face perception. In Experiment 1, scalp electroencephalogram (EEG) is recorded while pictures of human faces or facelike objects - in different stimulation sequences - are interleaved every second (i.e., at 1 Hz) in a rapid 6-Hz train of natural images of nonface objects. Participants do not perform any explicit face categorization task during stimulation, and report whether they perceived illusory faces post-stimulation. A robust categorization response to facelike objects is identified at 1 Hz and harmonics in the EEG frequency spectrum with a facelike occipito-temporal topography. Across all individuals, the facelike categorization response is of about 20% of the response to human faces, but more strongly right-lateralized. Critically, its amplitude is much larger in participants who report having perceived illusory faces. In Experiment 2, facelike or matched nonface objects from the same categories appear at 1 Hz in sequences of nonface objects presented at variable stimulation rates (60 Hz to 12 Hz) and participants explicitly report after each sequence whether they perceived illusory faces. The facelike categorization response already emerges at the shortest stimulus duration (i.e., 17 ms at 60 Hz) and predicts the behavioral report of conscious perception. Strikingly, neural facelike-selectivity emerges exclusively when participants report illusory faces. Collectively, these experiments characterize a neural signature of face pareidolia in the context of rapid categorization, supporting the view that face-selective brain activity reliably predicts the subjective experience of a face from a single glance at a variety of stimuli.

Objects that induce face pareidolia are prioritized by the visual system

The human visual system has evolved specialized neural mechanisms to rapidly detect faces. Its broad tuning for facial features is thought to underlie the illusory perception of faces in inanimate objects, a phenomenon called face pareidolia. Recent studies on face pareidolia suggest that the mechanisms underlying face processing, at least at the early stages of visual encoding, may treat objects that resemble faces as real faces; prioritizing their detection. In our study, we used breaking continuous flash suppression (b-CFS) to examine whether the human visual system prioritizes the detection of objects that induce face pareidolia over stimuli matched for object content. Similar to previous b-CFS results using real face stimuli, we found that participants detected the objects with pareidolia faces faster than object-matched control stimuli. Given that face pareidolia has been more frequently reported amongst individuals prone to hallucinations, we also explored whether this rapid prioritization is intact in individuals with schizophrenia, and found evidence suggesting that it was. Our findings suggest that face pareidolia engages a broadly tuned mechanism that facilitates rapid face detection. This may involve the proposed fast subcortical pathway that operates outside of visual awareness.

"I Spy with my Little Eye, Something that is a Face…": A Brain Network for Illusory Face Detection

The most basic aspect of face perception is simply detecting the presence of a face, which requires the extraction of features that it has in common with other faces. Putatively, it is caused by matching high-dimensional sensory input with internal face templates, achieved through a top-down mediated coupling between prefrontal regions and brain areas in the occipito-temporal cortex ("core system of face perception"). Illusory face detection tasks can be used to study these top-down influences. In the present functional magnetic resonance imaging study, we showed that illusory face perception activated just as real faces the core system, albeit with atypical left-lateralization of the occipital face area. The core system was coupled with two distinct brain regions in the lateral prefrontal (inferior frontal gyrus, IFG) and orbitofrontal cortex (OFC). A dynamic causal modeling (DCM) analysis revealed that activity in the core system during illusory face detection was upregulated by a modulatory face-specific influence of the IFG, not as previously assumed by the OFC. Based on these findings, we were able to develop the most comprehensive neuroanatomical framework of illusory face detection until now.

Single-Unit Recordings Reveal the Selectivity of a Human Face Area

The exquisite capacity of primates to detect and recognize faces is crucial for social interactions. Although disentangling the neural basis of human face recognition remains a key goal in neuroscience, direct evidence at the single-neuron level is limited. We recorded from face-selective neurons in human visual cortex in a region characterized by functional magnetic resonance imaging (fMRI) activations for faces compared with objects. The majority of visually responsive neurons in this fMRI activation showed strong selectivity at short latencies for faces compared with objects. Feature-scrambled faces and face-like objects could also drive these neurons, suggesting that this region is not tightly tuned to the visual attributes that typically define whole human faces. These single-cell recordings within the human face processing system provide vital experimental evidence linking previous imaging studies in humans and invasive studies in animal models.SIGNIFICANCE STATEMENT We present the first recordings of face-selective neurons in or near an fMRI-defined patch in human visual cortex. Our unbiased multielectrode array recordings (i.e., no selection of neurons based on a search strategy) confirmed the validity of the BOLD contrast (faces-objects) in humans, a finding with implications for all human imaging studies. By presenting faces, feature-scrambled faces, and face-pareidolia (perceiving faces in inanimate objects) stimuli, we demonstrate that neurons at this level of the visual hierarchy are broadly tuned to the features of a face, independent of spatial configuration and low-level visual attributes.

Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations

How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.

A shared mechanism for facial expression in human faces and face pareidolia

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

A visual search advantage for illusory faces in objects

Face detection is a priority of both the human and primate visual system. However, occasionally we misperceive faces in inanimate objects -- "face pareidolia". A key feature of these 'false positives' is that face perception occurs in the absence of visual features typical of real faces. Human faces are known to be located faster than objects in visual search. Here we used a visual search paradigm to test whether illusory faces share this advantage. Search times were faster for illusory faces than for matched objects amongst both matched (Experiment 1) and diverse (Experiment 2) distractors, however search times for real human faces were faster and more efficient than objects with or without an illusory face. Importantly, this result indicates that illusory faces are processed quickly enough by the human brain to confer a visual search advantage, suggesting the engagement of a broadly-tuned mechanism that facilitates rapid face detection in cluttered environments.

Color Space Geometry Uncovered with Magnetoencephalography

The geometry that describes the relationship among colors, and the neural mechanisms that support color vision, are unsettled. Here, we use multivariate analyses of measurements of brain activity obtained with magnetoencephalography to reverse-engineer a geometry of the neural representation of color space. The analyses depend upon determining similarity relationships among the spatial patterns of neural responses to different colors and assessing how these relationships change in time. We evaluate the approach by relating the results to universal patterns in color naming. Two prominent patterns of color naming could be accounted for by the decoding results: the greater precision in naming warm colors compared to cool colors evident by an interaction of hue and lightness, and the preeminence among colors of reddish hues. Additional experiments showed that classifiers trained on responses to color words could decode color from data obtained using colored stimuli, but only at relatively long delays after stimulus onset. These results provide evidence that perceptual representations can give rise to semantic representations, but not the reverse. Taken together, the results uncover a dynamic geometry that provides neural correlates for color appearance and generates new hypotheses about the structure of color space.