Knowledge is power: how conceptual knowledge transforms visual cognition

Behavioral and neural measures of semantic conflict monitoring: Findings from a novel picture-word interference task

Conflict monitoring has been studied extensively using experimental paradigms that manipulate perceptual dimensions of stimuli and responses. The picture-word interference (PWI) task has historically been used to examine semantic conflict, but primarily for the purpose of examining lexical retrieval. In this study, we utilized two novel PWI tasks to assess conflict monitoring in the context of semantic conflict. Participants included nineteen young adults (14F, age = 20.79 ± 3.14) who completed two tasks: Animals and Objects. Task and conflict effects were assessed by examining behavioral (reaction time and accuracy) and neurophysiological (oscillations in theta, alpha, and beta band) measures. Results revealed conflict effects within both tasks, but the pattern of findings differed across the two semantic categories. Participants were slower to respond to unmatched versus matched trials on the Objects task only and were less accurate responding to matched versus unmatched trials in the Animals task only. We also observed task differences, with participants responding more accurately on conflict trials for Animals compared to Objects. Differences in neural oscillations were observed, including between-task differences in low beta oscillations and within-task differences in theta, alpha, and low beta. We also observed significant correlations between task performance and standard measures of cognitive control. This work provides new insights into conflict monitoring, highlighting the importance of examining conflict across different semantic categories, especially in the context of animacy. The findings serve as a benchmark to assess conflict monitoring using PWI tasks across populations of varying cognitive ability.

An automatic red-female association tested by the Stroop task

Colors have been reported to be associated with genders (e.g., reddish color-females). Here, we investigated color-gender associations (red-female/green-male) among Japanese participants using two Stroop-word categorization tasks. Ten Japanese gendered words were chosen as visual stimuli. In Experiment 1 (N = 23), participants were instructed to indicate whether a target word presented in either red, green, or gray font color was a masculine or feminine word. Results showed a congruency effect of red-female association that red font color facilitated the categorization of feminine words and inhibited the categorization of masculine words compared to other colors. No significant effect of green-male association was observed. Experiment 2 (N = 23 newly recruited participants) examined whether the congruency effect of color-gender associations could bias perceptual font color categorization. Participants were asked to discriminate whether the font color, with low saturation, was red or green while ignoring the word's meaning. Results showed that participants responded faster and made fewer errors when categorizing red font colors for feminine words than masculine words. Additionally, a congruent effect of green-male association on performance accuracy was observed, although it did not affect response times. These results indicate the existence of an automatically activated red-female association, which can influence both conceptual gender categorization and perceptual color processing.

Red background color biases gender categorization of human faces

Color carries gender information (e.g., red-female). This study explored whether background color could influence the gender categorization of human faces. Visual stimuli were generated from faces whose sexually dimorphic content was morphed monotonically from female to male perception. The face stimulus was presented upright (Experiment 1) and inverted (Experiment 2) with three background colors (i.e., red, green, and gray). Participants were instructed to categorize the gender of the face stimulus as male or female by pressing one of two labelled keys. Results showed that a red background could bias the gender of an ambiguous upright face toward a female compared with green and gray background colors (Experiment 1). However, this red effect was diminished when the face stimulus was inverted (Experiment 2). These results suggest that red background color interacting with facial configuration features biases gender perception toward a female face, possibly through top-down processing of learned associations between the color red and femininity.

The role of the ventrolateral anterior temporal lobes in social cognition

A key challenge for neurobiological models of social cognition is to elucidate whether brain regions are specialised for that domain. In recent years, discussion surrounding the role of anterior temporal regions epitomises such debates; some argue the anterior temporal lobe (ATL) is part of a domain‐specific network for social processing, while others claim it comprises a domain‐general hub for semantic representation. In the present study, we used ATL‐optimised fMRI to map the contribution of different ATL structures to a variety of paradigms frequently used to probe a crucial social ability, namely ‘theory of mind’ (ToM). Using multiple tasks enables a clearer attribution of activation to ToM as opposed to idiosyncratic features of stimuli. Further, we directly explored whether these same structures are also activated by a non‐social task probing semantic representations. We revealed that common to all of the tasks was activation of a key ventrolateral ATL region that is often invisible to standard fMRI. This constitutes novel evidence in support of the view that the ventrolateral ATL contributes to social cognition via a domain‐general role in semantic processing and against claims of a specialised social function.

Integrating unsupervised and reinforcement learning in human categorical perception: A computational model

Categorical perception identifies a tuning of human perceptual systems that can occur during the execution of a categorisation task. Despite the fact that experimental studies and computational models suggest that this tuning is influenced by task-independent effects (e.g., based on Hebbian and unsupervised learning, UL) and task-dependent effects (e.g., based on reward signals and reinforcement learning, RL), no model studies the UL/RL interaction during the emergence of categorical perception. Here we have investigated the effects of this interaction, proposing a system-level neuro-inspired computational architecture in which a perceptual component integrates UL and RL processes. The model has been tested with a categorisation task and the results show that a balanced mix of unsupervised and reinforcement learning leads to the emergence of a suitable categorical perception and the best performance in the task. Indeed, an excessive unsupervised learning contribution tends to not identify task-relevant features while an excessive reinforcement learning contribution tends to initially learn slowly and then to reach sub-optimal performance. These results are consistent with the experimental evidence regarding categorical activations of extrastriate cortices in healthy conditions. Finally, the results produced by the two extreme cases of our model can explain the existence of several factors that may lead to sensory alterations in autistic people.

School-Aged Children Learn Novel Categories on the Basis of Distributional Information

Categorization of sensory stimuli is a vital process in understanding the world. In this paper we show that distributional learning plays a role in learning novel object categories in school-aged children. An 11-step continuum was constructed based on two novel animate objects by morphing one object into the other in 11 equal steps. Forty-nine children (7–9 years old) were subjected to one of two familiarization conditions during which they saw tokens from the continuum. The conditions differed in the position of the distributional peaks along the continuum. After familiarization it was tested how the children categorized the stimuli. Results show that, in line with our expectations, familiarization condition influenced categorization during the test phase, indicating that the frequency distribution of tokens in the input had induced novel object category formation. These results suggest that distributional learning could play an important role in categorizing sensory stimuli throughout life.

Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain

In our everyday life, we continuously get to know people, dominantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory information refines person representation, and finally, one reaches a stage where the effortless identification of very well-known persons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adaptation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the information emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activation of long-term memory and emotion processing areas is essential for correct identification.

Impressions of HIV risk online: Brain potentials while viewing online dating profiles

There is an increasing trend to use online dating to meet potential partners. Previous studies in off-line contexts indicate that people may judge the risk of sexually transmitted infections based on a person's appearance. Online dating profiles commonly present profile pictures and verbal self-descriptions. To examine the integration of verbal and visual risk information, the current event-related potential (ERP) study used a simulated dating platform in which verbal-descriptive information (low vs. high verbal risk) was presented, followed by a photograph (low vs. high visual risk). Results indicated main effects of verbal and visual risk. Specifically, high-risk compared with low-risk verbal profiles elicited a relative negative shift over occipitoparietal sensor sites between 260 ms and 408 ms. Furthermore, a sustained occipital negativity (132-500 ms) and central positivity (156-272 ms) was observed for high as compared with low visual risk profiles. There was also evidence for the integration of verbal and visual risk formation, as indicated by distinct positive ERP shift occurred between 272 ms and 428 ms over anterior temporal regions when a high-risk photograph was preceded by high-risk verbal information. This suggests that verbal-descriptive information is integrated with visual appearance early in the processing stream. The distinct response for high verbal and visual information extends the notion of an alarm function ascribed to risk perception by demonstrating integration about multiple sources. Simulating online dating platforms provides a useful tool to examine intuitive risk perception.

Dynamic interactive theory as a domain-general account of social perception

The perception of social categories, emotions, and personality traits from others' faces each have been studied extensively but in relative isolation. We synthesize emerging findings suggesting that, in each of these domains of social perception, both a variety of bottom-up facial features and top-down social cognitive processes play a part in driving initial perceptions. Among such top-down processes, social-conceptual knowledge in particular can have a fundamental structuring role in how we perceive others' faces. Extending the Dynamic Interactive framework (Freeman & Ambady, 2011), we outline a perspective whereby the perception of social categories, emotions, and traits from faces can all be conceived as emerging from an integrated system relying on domain-general cognitive properties. Such an account of social perception would envision perceptions to be a rapid, but gradual, process of negotiation between the variety of visual cues inherent to a person and the social cognitive knowledge an individual perceiver brings to the perceptual process. We describe growing evidence in support of this perspective as well as its theoretical implications for social psychology.

The effect of symbolic meaning of speed on time to contact

The effects of moving task-irrelevant objects on time-to-contact (TTC) judgments are examined in six experiments. In particular, we investigated the effects of the symbolic meaning of speed on TTC by presenting images of objects recalling the symbolic meaning of high speed (motorbike, rocket, formula one, rabbit, cheetah and flying Superman) and low speed (bicycle, hot-air balloon, tank, turtle, elephant and static Superman). In all experiments, participants judged the TTC of these moving objects with a black line, indicating the end of the occlusion. Experiment 7 was conducted to disambiguate whether the effects on TTC, found in the previous experiments, were either a by-product of a speed illusion or they were rather elicited by the implicit timing task. In a two-interval forced choice task, participants were instructed to judge if "high-speed objects" moved actually faster than "slow-speed objects". The results revealed no consistent speed illusion. Taken together the results showed shorter TTC estimated with stimuli recalling the meaning of high compared to low speed, but only with the long occlusion duration (3.14 s). At shorter occlusion durations, the pattern was reversed (participant tend to have shorter TTC with stimuli recalling the meaning of low speed). We suggest that the symbolic meaning of speed works mainly at low speed and long TTC, because the semantic elaboration of the stimulus needs a deeper cognitive elaboration. On the other hand, at higher speeds, a small erroneous perceptual judgment affects the TTC, perhaps due to a speed expectancy violation of the expected "slow object".

Categorization training changes the visual representation of face identity

Previous research suggests that learning to categorize faces along a new dimension changes the perceptual representation of that dimension, but little is known about how the representation of specific face identities changes after such category learning. Here, we trained participants to categorize faces that varied along two morphing dimensions. One dimension was relevant to the categorization task and the other was irrelevant. We used reverse correlation to estimate the internal templates used to identify the two faces at the extremes of the relevant dimension, both before and after training, and at two different levels of the irrelevant dimension. Categorization training changed the internal templates used for face identification, even though identification and categorization tasks impose different demands on the observers. After categorization training, the internal templates became more invariant across changes in the irrelevant dimension. These results suggest that the representation of face identity can be modified by categorization experience.

Semantic Knowledge Enhances Conscious Awareness of Visual Objects

It is becoming increasingly established that information from long-term memory can influence early perceptual processing, a finding that is in line with recent theoretical approaches to cognition such as the predictive coding framework. Notwithstanding, the impact of semantic knowledge on conscious perception and the temporal dynamics of such an influence remain unclear. To address this question, we presented pictures of novel objects to participants as the second of two targets in an attentional blink paradigm. We found that associating newly acquired semantic knowledge to objects increased overall conscious detection in comparison to objects associated with minimal knowledge while controlling for object familiarity. Additionally, event-related brain potentials revealed a corresponding modulation beginning 100 msec after stimulus presentation in the P1 component. Furthermore, the size of this modulation was correlated with participant's subjective reports of conscious perception. These findings suggest that semantic knowledge can shape the contents of consciousness by affecting early stages of perceptual processing.

Novel representations that support rule-based categorization are acquired on-the-fly during category learning

Humans learn categorization rules that are aligned with separable dimensions through a rule-based learning system, which makes learning faster and easier to generalize than categorization rules that require integration of information from different dimensions. Recent research suggests that learning to categorize objects along a completely novel dimension changes its perceptual representation, making it more separable and discriminable. Here, we asked whether such newly learned dimensions could support rule-based category learning. One group received extensive categorization training and a second group did not receive such training. Later, both groups were trained in a task that made use of the category-relevant dimension, and then tested in an analogical transfer task (Experiment 1) and a button-switch interference task (Experiment 2). We expected that only the group with extensive pre-training (with well-learned dimensional representations) would show evidence of rule-based behavior in these tasks. Surprisingly, both groups performed as expected from rule-based learning. A third experiment tested whether a single session (less than 1 h) of training in a categorization task would facilitate learning in a task requiring executive function. There was a substantial learning advantage for a group with brief pre-training with the relevant dimension. We hypothesize that extensive experience with separable dimensions is not required for rule-based category learning; rather, the rule-based system may learn representations "on the fly" that allow rule application. We discuss what kind of neurocomputational model might explain these data best.

Cognitive penetration of early vision in face perception

Cognitive and affective penetration of perception refers to the influence that higher mental states such as beliefs and emotions have on perceptual systems. Psychological and neuroscientific studies appear to show that these states modulate the visual system at the visuomotor, attentional, and late levels of processing. However, empirical evidence showing that similar consequences occur in early stages of visual processing seems to be scarce. In this paper, I argue that psychological evidence does not seem to be either sufficient or necessary to argue in favour of or against the cognitive penetration of perception in either late or early vision. In order to do that we need to have recourse to brain imaging techniques. Thus, I introduce a neuroscientific study and argue that it seems to provide well-grounded evidence for the cognitive penetration of early vision in face perception. I also examine and reject alternative explanations to my conclusion.

Social and Contextual Constraints on Embodied Perception

A number of papers have challenged research on physiological and psychological influences on perception by claiming to show that such findings can be explained by nonperceptual factors such as demand characteristics. Relatedly, calls for separating perception from judgment have been issued. However, such efforts fail to consider key processes known to shape judgment processes: people's inability to report accurately on their judgments, conversational dynamics of experimental research contexts, and misattribution and discounting processes. Indeed, the fact that initially observed effects of embodied influences disappear is predicted by an extensive amount of literature on judgments studied within social psychology. Thus, findings from such studies suggest that the initially presumed underlying processes are at work-namely, functional considerations that are informative in the context of preparing the body for action. In this article, I provide suggestions on how to conduct research on perception within the social constraints of experimental contexts.

Self-prioritization and perceptual matching: The effects of temporal construal

Recent research has revealed that self-referential processing enhances perceptual judgments - the so-called self-prioritization effect. The extent and origin of this effect remains unknown, however. Noting the multifaceted nature of the self, here we hypothesized that temporal influences on self-construal (i.e., past/future-self continuity) may serve as an important determinant of stimulus prioritization. Specifically, as representations of the self increase in abstraction as a function of temporal distance (i.e., distance from now), self-prioritization may only emerge when stimuli are associated with the current self. The results of three experiments supported this prediction. Self-relevance only enhanced performance in a standard perceptual-matching task when stimuli (i.e., geometric shapes) were connected with the current self; representations of the self in the future (Expts. 1 & 2) and past (Expt. 3) failed to facilitate decision making. To identify the processes underlying task performance, data were interrogated using a hierarchical drift diffusion model (HDDM) approach. Results of these analyses revealed that self-prioritization was underpinned by a stimulus bias (i.e., rate of information uptake). Collectively, these findings elucidate when and how self-relevance influences decisional processing.

The iterative nature of person construal: Evidence from event-related potentials

Recently, a dynamic-interactive model of person construal (DI model) has been proposed, whereby the social categories a person represents are determined on the basis of an iterative integration of bottom-up and top-down influences. The current study sought to test this model by leveraging the high temporal resolution of event-related brain potentials (ERPs) as 65 participants viewed male faces that varied by race (White vs Black), fixating either between the eyes or on the forehead. Within face presentations, the effect of fixation, meant to vary bottom-up visual input, initially was large but decreased across early latency neural responses identified by a principal components analysis (PCA). In contrast, the effect of race, reflecting a combination of top-down and bottom-up factors, initially was small but increased across early latency principal components. These patterns support the DI model prediction that bottom-up and top-down processes are iteratively integrated to arrive at a stable construal within 230 ms. Additionally, exploratory multilevel modeling of single trial ERP responses representing a component linked to outgroup categorization (the P2) suggests change in effects of the manipulations over the course of the experiment. Implications of the findings for the DI model are considered.

Effect of Affective Personality Information on Face Processing: Evidence from ERPs

This study explored the extent to which there are the neural correlates of the affective personality influence on face processing using event-related potentials (ERPs). In the learning phase, participants viewed a target individual’s face (expression neutral or faint smile) paired with either negative, neutral or positive sentences describing previous typical behavior of the target. In the following EEG testing phase, participants completed gender judgments of the learned faces. Statistical analyses were conducted on measures of neural activity during the gender judgment task. Repeated measures ANOVA of ERP data showed that faces described as having a negative personality elicited larger N170 than did those with a neutral or positive description. The early posterior negativity (EPN) showed the same result pattern, with larger amplitudes for faces paired with negative personality than for others. The size of the late positive potential was larger for faces paired with positive personality than for those with neutral and negative personality. The current study indicates that affective personality information is associated with an automatic, top–down modulation on face processing.

Can you experience 'top-down' effects on perception?: The case of race categories and perceived lightness

A recent surge of research has revived the notion that higher-level cognitive states such as beliefs, desires, and categorical knowledge can directly change what we see. The force of such claims, however, has been undercut by an absence of visually apparent demonstrations of the form so often appealed to in vision science: such effects may be revealed by statistical analyses of observers' responses, but you cannot literally experience the alleged top-down effects yourself. A singular exception is an influential report that racial categorization alters the perceived lightness of faces, a claim that was bolstered by a striking visual demonstration that Black faces appear darker than White faces, even when matched for mean luminance. Here, we show that this visually compelling difference is explicable in terms of purely low-level factors. Observers who viewed heavily blurred versions of the original Black and White faces still judged the Black face to be darker and the White face to be lighter even when these observers could not perceive the races of the faces, and even when they explicitly judged the faces to be of the same race. We conclude that the best subjectively appreciable evidence for top-down influences on perception does not reflect a genuinely top-down effect after all: instead, such effects arise from more familiar (if subtle) bottom-up factors within visual processing.

Linking person perception and person knowledge in the human brain

Neuroscience research has examined separately how we detect human agents on the basis of their face and body (person perception) and how we reason about their thoughts, traits or intentions (person knowledge). Neuroanatomically distinct networks have been associated with person perception and person knowledge, but it remains unknown how multiple features of a person (e.g. thin and kind) are linked to form a holistic identity representation. In this fMRI experiment, we investigated the hypothesis that when encountering another person specialised person perception circuits would be functionally coupled with circuits involved in person knowledge. In a factorial design, we paired bodies or names with trait-based or neutral statements, and independent localiser scans identified body-selective and mentalising networks. When observing a body paired with a trait-implying statement, functional connectivity analyses demonstrated that body-selective patches in bilateral fusiform gyri were functionally coupled with nodes of the mentalising network. We demonstrate that when forming a representation of a person circuits for representing another person's physical appearance are linked to circuits that are engaged when reasoning about trait-based character. These data support the view that a 'who' system for social cognition involves communication between perceptual and inferential mechanisms when forming a representation of another's identity.

Neural representation for object recognition in inferotemporal cortex

We suggest that population representation of objects in inferotemporal cortex lie on a continuum between a purely structural, parts-based description and a purely holistic description. The intrinsic dimensionality of object representation is estimated to be around 100, perhaps with lower dimensionalities for object representations more toward the holistic end of the spectrum. Cognitive knowledge in the form of semantic information and task information feed back to inferotemporal cortex from perirhinal and prefrontal cortex respectively, providing high-level multimodal-based expectations that assist in the interpretation of object stimuli. Integration of object information across eye movements may also contribute to object recognition through a process of active vision.

Hallucinations and mental imagery demonstrate top-down effects on visual perception

In this commentary, we present two examples where perception is not only influenced by, but also in fact driven by, top-down effects: hallucinations and mental imagery. Crucially, both examples avoid all six of the potential confounds that Firestone & Scholl (F&S) raised as arguments against previous studies claiming to demonstrate the influence of top-down effects on perception.

Cognition does not affect perception: Evaluating the evidence for “top-down” effects

What determines what we see? In contrast to the traditional “modular” understanding of perception, according to which visual processing is encapsulated from higher-level cognition, a tidal wave of recent research alleges that states such as beliefs, desires, emotions, motivations, intentions, and linguistic representations exert direct, top-down influences on what we see. There is a growing consensus that such effects are ubiquitous, and that the distinction between perception and cognition may itself be unsustainable. We argue otherwise: None of these hundreds of studies – either individually or collectively – provides compelling evidence for true top-down effects on perception, or “cognitive penetrability.” In particular, and despite their variety, we suggest that these studies all fall prey to only a handful of pitfalls. And whereas abstract theoretical challenges have failed to resolve this debate in the past, our presentation of these pitfalls is empirically anchored: In each case, we show not only how certain studies could be susceptible to the pitfall (in principle), but also how several alleged top-down effects actually are explained by the pitfall (in practice). Moreover, these pitfalls are perfectly general, with each applying to dozens of other top-down effects. We conclude by extracting the lessons provided by these pitfalls into a checklist that future work could use to convincingly demonstrate top-down effects on visual perception. The discovery of substantive top-down effects of cognition on perception would revolutionize our understanding of how the mind is organized; but without addressing these pitfalls, no such empirical report will license such exciting conclusions.

Reprint of: Semantic impairment disrupts perception, memory, and naming of secondary but not primary colours

To investigate how basic aspects of perception are shaped by acquired knowledge about the world, we assessed colour perception and cognition in patients with semantic dementia (SD), a disorder that progressively erodes conceptual knowledge. We observed a previously undocumented pattern of impairment to colour perception and cognition characterized by: (i) a normal ability to discriminate between only subtly different colours but an impaired ability to group different colours into categories, (ii) normal perception and memory for the colours red, green, and blue but impaired perception and memory for colours lying between these regions of a fully-saturated and luminant spectrum, and (iii) normal naming of polar colours in the opponent-process colour system (red, green, blue, yellow, white, and black) but impaired naming of other basic colours (brown, gray, pink, and orange). The results suggest that fundamental aspects of perception can be shaped by acquired knowledge about the world, but only within limits.

Category Learning Stretches Neural Representations in Visual Cortex

We review recent work that shows how learning to categorize objects changes how those objects are represented in the mind and the brain. After category learning, visual perception of objects is enhanced along perceptual dimensions that were relevant to the learned categories, an effect we call dimensional modulation (DM). DM stretches object representations along category-relevant dimensions and shrinks them along category-irrelevant dimensions. The perceptual advantage for category-relevant dimensions extends beyond categorization and can be observed during visual discrimination and other tasks that do not depend on the learned categories. fMRI shows that category learning causes ventral stream neural populations in visual cortex representing objects along a category-relevant dimension to become more distinct. These results are consistent with a view that specific aspects of cognitive tasks associated with objects can account for how our visual system responds to objects.

Semantic impairment disrupts perception, memory, and naming of secondary but not primary colours

To investigate how basic aspects of perception are shaped by acquired knowledge about the world, we assessed colour perception and cognition in patients with semantic dementia (SD), a disorder that progressively erodes conceptual knowledge. We observed a previously undocumented pattern of impairment to colour perception and cognition characterized by: (i) a normal ability to discriminate between only subtly different colours but an impaired ability to group different colours into categories, (ii) normal perception and memory for the colours red, green, and blue but impaired perception and memory for colours lying between these regions of a fully-saturated and luminant spectrum, and (iii) normal naming of polar colours in the opponent-process colour system (red, green, blue, yellow, white, and black) but impaired naming of other basic colours (brown, gray, pink, and orange). The results suggest that fundamental aspects of perception can be shaped by acquired knowledge about the world, but only within limits.

•

Study of colour perception and cognition in patients with semantic dementia.

•

Colour discrimination is normal but categorization is impaired.

•

Patients and controls show opposing accuracy patterns in memory and perception.

•

Naming is more impaired for secondary than primary colours.