Sensory and decision-making processes underlying perceptual adaptation

Dynamics of sensory and decisional biases in perceptual decision making: Insights from the face distortion illusion

Bias in perceptual decision making can have both sensory and decisional origins. These distinct sources of bias are typically seen as static and stable over time. However, human behavior is dynamic and constantly adapting. Yet it remains unclear how sensory and decisional biases progress in distinct ways over time. We addressed this question by tracking the dynamics of sensory and decisional biases during a task that involves a visual illusion. Observers saw multiple pairs of peripherally presented faces that induce a strong illusion making the faces appear distorted and grotesque. The task was to judge whether one of the last two faces had true physical distortion (experimentally introduced in half of the trials). Initially, participants classified most faces as distorted as exemplified by a liberal response bias. However, over the course of the experiment, this response bias gradually disappeared even though the distortion illusion remained equally strong, as demonstrated by a separate subjective rating task without artificially distorted faces. The results suggest that the sensory bias was progressively countered by an opposite decisional bias. This transition was accompanied by an increase in reaction times and a decrease in confidence relative to a condition that does not induce the visual illusion. All results were replicated in a second experiment with inverted faces. These findings demonstrate that participants dynamically adjust their decisional bias to compensate for sensory biases, and that these two biases together determine how humans make perceptual decisions.

Affective norms for 501 Chinese words from three emotional dimensions rated by depressive disorder patients

Introduction

Emotional words are often used as stimulus material to explore the cognitive and emotional characteristics of individuals with depressive disorder, while normal individuals mostly rate the scores of affective words. Given that individuals with depressive disorder exhibit a negative cognitive bias, it is possible that their depressive state could influence the ratings of affective words. To enhance the validity of the stimulus material, we specifically recruited patients with depression to provide these ratings.

Methods

This study provided subjective ratings for 501 Chinese affective norms, incorporating 167 negative words selected from depressive disorder patients' Sino Weibo blogs, and 167 neutral words and 167 positive words selected from the Chinese Affective Word System. The norms are based on the assessments made by 91 patients with depressive disorder and 92 normal individuals, by using the paper-and-pencil quiz on a 9-point scale.

Results

Regardless of the group, the results show high reliability and validity. We identified group differences in three dimensions: valence, arousal, and self-relevance: the depression group rated negative words higher, but positive and neutral words lower than the normal control group.

Conclusion

The emotional perception affected the individual's perception of words, to some extent, this database expanded the ratings and provided a reference for exploring norms for individuals with different emotional states.

Introspective inference counteracts perceptual distortion

Introspective agents can recognize the extent to which their internal perceptual experiences deviate from the actual states of the external world. This ability, also known as insight, is critically required for reality testing and is impaired in psychosis, yet little is known about its cognitive underpinnings. We develop a Bayesian modeling framework and a psychophysics paradigm to quantitatively characterize this type of insight while people experience a motion after-effect illusion. People can incorporate knowledge about the illusion into their decisions when judging the actual direction of a motion stimulus, compensating for the illusion (and often overcompensating). Furthermore, confidence, reaction-time, and pupil-dilation data all show signatures consistent with inferential adjustments in the Bayesian insight model. Our results suggest that people can question the veracity of what they see by making insightful inferences that incorporate introspective knowledge about internal distortions.

Positive hysteresis in emotion recognition: Face processing visual regions are involved in perceptual persistence, which mediates interactions between anterior insula and medial prefrontal cortex

Facial emotion perception can be studied from the point of view of dynamic systems whose output may depend not only on current input but also on prior history - a phenomenon known as hysteresis. In cognitive neuroscience, hysteresis has been described as positive (perceptual persistence) or negative (fatigue of current percept) depending on whether perceptual switching occurs later or earlier than actual physical stimulus changes. However, its neural correlates remain elusive. We used dynamic transitions between emotional expressions and combined behavioral assessment with functional magnetic resonance imaging (fMRI) to investigate the underlying circuitry of perceptual hysteresis in facial emotion recognition. Our findings revealed the involvement of face-selective visual areas - fusiform face area (FFA) and superior temporal sulcus (STS) - in perceptual persistence as well as the right anterior insula. Moreover, functional connectivity analyses revealed an interplay between the right anterior insula and medial prefrontal cortex, which showed to be dependent on the presence of positive hysteresis. Our results support the hypothesis that high-order regions are involved in perceptual stabilization and decision during perceptual persistence (positive hysteresis) and add evidence to the role of the anterior insula as a hub of sensory information in perceptual decision-making.

Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

The implied motion aftereffect changes decisions, but not confidence

Viewing static images depicting movement can result in a motion aftereffect: people tend to categorise direction signals as moving in the opposite direction relative to the implied motion in still photographs. This finding could indicate that inferred motion direction can penetrate sensory processing and change perception. Equally possible, however, is that inferred motion changes decision processes, but not perception. Here we test these two possibilities. Since both categorical decisions and subjective confidence are informed by sensory information, confidence can be informative about whether an aftereffect probably results from changes to perceptual or decision processes. We therefore used subjective confidence as an additional measure of the implied motion aftereffect. In Experiment 1 (implied motion), we find support for decision-level changes only, with no change in subjective confidence. In Experiment 2 (real motion), we find equal changes to decisions and confidence. Our results suggest the implied motion aftereffect produces a bias in decision-making, but leaves perceptual processing unchanged.

Adaptation aftereffects reveal how categorization training changes the encoding of face identity

Previous research suggests that learning to categorize faces along a novel dimension changes the perceptual representation of such dimension, increasing its discriminability, its invariance, and the information used to identify faces varying along the dimension. A common interpretation of these results is that categorization training promotes the creation of novel dimensions, rather than simply the enhancement of already existing representations. Here, we trained a group of participants to categorize faces that varied along two morphing dimensions, one of them relevant to the categorization task and the other irrelevant to the task. An untrained group did not receive such categorization training. In three experiments, we used face adaptation aftereffects to explore how categorization training changes the encoding of face identities at the extremes of the category-relevant dimension and whether such training produces encoding of the category-relevant dimension as a preferred direction in face space. The pattern of results suggests that categorization training enhances the already existing norm-based coding of face identity, rather than creating novel category-relevant representations. We formalized this conclusion in a model that explains the most important results in our experiments and serves as a working hypothesis for future work in this area.

How positive emotional content overrules perceptual history effects: Hysteresis in emotion recognition

The human visual system is constantly processing multiple and often conflicting sensory cues to make perceptual decisions. Given the nonlinear nature of emotion recognition, this often leads to different percepts of the same physical facial expression. Moreover, the state of the emotion recognition system might depend on the trajectory of temporal context, potentially leading to a phenomenon known as perceptual hysteresis. Here, we aimed to explore temporal context-related mechanisms underlying perceptual hysteresis during emotion recognition. We hypothesized that dependence on recent perceptual experience might reveal important clues about the role of short-term memory on the perception of emotional stimuli. Behavioral data were acquired using reality-based, changing emotion expressions morphed from a source to a target emotion with different valences, always passing through a neutral expression. Participants identified the onset and offset of what they perceived as the neutral expression interval. Our results showed that current perception of emotional expression is affected by recent temporal context, thus revealing perceptual hysteresis. We also found a relation between recent perceptual history effects and stimulus emotional Content: The positive valence of the stimulus emotional content appeared to abolish perceptual history effects, whereas negatively loaded stimuli induced clear short-term memory effects and positive hysteresis. Our findings show direct competition between recent perceptual experience and stimulus emotional content during decision making, which affects the formation of current percepts in emotion recognition.

Does the motor system contribute to the perception of changes in objects visual attributes? The neural dynamics of sensory binding by action

The contribution of the motor system to perception has been highlighted in research investigating the effect of performing an action on the conscious processing of information received from the sensory systems. For example, the perceptual temporal asynchrony observed when passively reporting changes in visual object attributes (e.g., colour and position) was found to disappear virtually when the changes resulted from a voluntary motor action. Although the spatio-temporal constraints of sensory binding by action have been broadly investigated, the underlying neural correlates are still largely unknown. In the present study, participants performed temporal order judgments of changes in the colour and position of a visual stimulus, while adapting to a 750 ms delay between a sound (perceptual condition) or the end of a manual reaching action (motor condition), and the visual changes. Behavioural observations indicated that temporal asynchrony (-30.2 ms) decreased in the motor condition (2.7 ms), as a result of sensorimotor adaptation, but not in the perceptual condition (-29.6 ms). EEG-evoked potentials on posterior visual regions showed that early components were altered by sensorimotor adaptation, with in particular a broad reduction in the amplitude of the early P1 component. Furthermore, time-frequency analysis of EEG signals during the 350 ms period preceding the visual changes revealed an increase of the 15-25 Hz frequency band amplitude in the central region and a decrease of the 8-12 Hz frequency band amplitude in the posterior region. Overall the results suggest that sensory binding by action depends on an early top-down modulation of the visual regions by the motor system - in agreement with the pre-activation theory of action-perception coupling - associated with an increase of attentional resources.

Confidence as a diagnostic tool for perceptual aftereffects

Perceptual judgements are, by nature, a product both of sensation and the cognitive processes responsible for interpreting and reporting subjective experiences. Changed perceptual judgements may thus result from changes in how the world appears (perception), or subsequent interpretation (judgement). This ambiguity has led to persistent debates about how to interpret changes in decision-making, and if higher-order cognitions can change how the world looks, or sounds, or feels. Here we introduce an approach that can help resolve these ambiguities. In three motion-direction experiments, we measured perceptual judgements and subjective confidence. We show that each measure is sensitive to sensory information and can index sensory adaptation. Each measure is also sensitive to decision biases, but response bias impacts the central tendency of decision and confidence distributions differently. Our findings show that subjective confidence, when measured in addition to perceptual decisions, can supply important diagnostic information about the cause of aftereffects.