Motion illusions as optimal percepts

Hierarchical motion perception as causal inference

Motion can only be defined relative to a reference frame; yet it remains unclear which reference frame guides perception. A century of psychophysical studies has produced conflicting evidence: retinotopic, egocentric, world-centric, or even object-centric. We introduce a hierarchical Bayesian model mapping retinal velocities to perceived velocities. Our model mirrors the structure in the world, in which visual elements move within causally connected reference frames. Friction renders velocities in these reference frames mostly stationary, formalized by an additional delta component (at zero) in the prior. Inverting this model automatically segments visual inputs into groups, groups into supergroups, progressively inferring structured reference frames and "perceives" motion in the appropriate reference frame. Critical model predictions are supported by two experiments, and fitting our model to the data allows us to infer the subjective set of reference frames used by individual observers. Our model provides a quantitative normative justification for key Gestalt principles providing inspiration for building better models of visual processing in general.

Intact Susceptibility to Visual Illusions in Autistic Individuals

Altered sensory perception, a core characteristic of autism, has been attributed to attenuated use of stimuli context or prior information in perception. Reduced susceptibility to perceptual illusions was extensively used to support these accounts for autistic perception. However, empirical evidence has been inconsistent. The current study systematically investigated susceptibility to size illusions in autistic and non-autistic individuals using a standardized psychophysical battery. Eighty-one participants, 41 autistic and 40 non-autistic individuals, completed the Ben-Gurion University Test for Perceptual Illusions (BTPI), measuring susceptibility to the Ponzo, Ebbinghaus, and Height-width illusions. The results demonstrate clear evidence for susceptibility to illusions in the perception of size both in the autistic and non-autistic groups. No significant differences were found between groups in the magnitude of illusion on the perceived size, or on the perceptual resolutions of size (discrimination thresholds) in any of the illusory settings tested. The results challenge current theories suggesting reduced reliance on priors or enhanced sensory measurement in autism. Instead, using robust psychophysical methods, the study provides clear evidence for autistic people forming priors and using long-term knowledge in perception.

The probabilistic and dynamic nature of perception in human generalization behavior

Generalization theories traditionally overlook how our mental representations dynamically change in the process of transferring learned knowledge to new contexts. We integrated perceptual and generalization theories into a computational model using data from 80 participants who underwent Pavlovian fear conditioning experiments. The model analyzed continuous measures of perception and fear generalization to understand their relationship. Our findings revealed large individual variations in perceptual processes that directly influence generalization patterns. By examining how perceptual and generalization mechanisms work together, we uncovered their combined role in producing generalization behavior. This research illuminates the probabilistic perceptual foundations underlying individual differences in generalization, emphasizing the crucial integration between perceptual and generalization processes. Understanding this relationship enhances our knowledge of generalization behavior and has potential implications for various cognitive domains including categorization, motor learning, language processing, and face recognition-all of which rely on generalization as a fundamental cognitive process.

Holistic and local processing occur simultaneously for inverted faces: Evidence from behavior and computational modeling

Whether inverted faces are processed locally or involve holistic processing has been debated for several years. This study conducted two experiments to explore the extent of holistic processing of inverted faces. Experiment 1 adopted a face-congruency paradigm that orthogonally manipulated stimulus congruency and orientation. Experiment 2 employed the complete congruency paradigm to test whether misalignment effects of inverted faces are related to holistic processing. The results of both experiments consistently demonstrated that inverted faces are processed not only locally but also holistically, and that misalignment disrupts the holistic processing of inverted faces. Subsequently computational modeling showed that in the congruent condition, the contributions of holistic and local information in inverted face processing performance were 24% and 76%, respectively, whereas in the incongruent condition, they were 10% and 90%, respectively. Together, the present study reveals that also inverted faces are processed holistically, albeit to a lower degree than upright faces.

Bayesian inference by visuomotor neurons in the prefrontal cortex

Perceptual judgments of the environment emerge from the concerted activity of neural populations in decision-making areas downstream of the sensory cortex. When the sensory input is ambiguous, perceptual judgments can be biased by prior expectations shaped by environmental regularities. These effects are examples of Bayesian inference, a reasoning method in which prior knowledge is leveraged to optimize uncertain decisions. However, it is not known how decision-making circuits combine sensory signals and prior expectations to form a perceptual decision. Here, we study neural population activity in the prefrontal cortex of macaque monkeys trained to report perceptual judgments of ambiguous visual stimuli under two different stimulus distributions. We isolate the component of the neural population response that represents the formation of the perceptual decision (the decision variable, DV), and find that its dynamical evolution reflects the integration of sensory signals and prior expectations. Prior expectations impact the DV's trajectory both before and during stimulus presentation such that DV trajectories with a smaller dynamic range result in more biased and less sensitive perceptual decisions. We show that these results resemble a specific variant of Bayesian inference known as approximate hierarchical inference. Our findings expand our understanding of the mechanisms by which prefrontal circuits can execute Bayesian inference.

Sensorimotor adaptation reveals systematic biases in 3D perception

The existence of biases in visual perception and their impact on visually guided actions has long been a fundamental yet unresolved question. Evidence revealing perceptual or visuomotor biases has typically been disregarded because such biases in spatial judgments can often be attributed to experimental measurement confounds. To resolve this controversy, we leveraged the visuomotor system's adaptation mechanism - triggered only by a discrepancy between visual estimates and sensory feedback - to directly indicate whether systematic errors in perceptual and visuomotor spatial judgments exist. To resolve this controversy, we leveraged the adaptive mechanisms of the visuomotor system to directly reveal whether systematic biases or errors in perceptual and visuomotor spatial judgments exist. In a within-subject study (N=24), participants grasped a virtual 3D object with varying numbers of depth cues (single vs. multiple) while receiving haptic feedback. The resulting visuomotor adaptations and aftereffects demonstrated that the planned grip size, determined by the visually perceived depth of the object, was consistently overestimated. This overestimation intensified when multiple cues were present, despite no actual change in physical depth. These findings conclusively confirm the presence of inherent biases in visual estimates for both perception and action, and highlight the potential use of visuomotor adaptation as a novel tool for understanding perceptual biases.

Representational geometry explains puzzling error distributions in behavioral tasks

Measuring and interpreting errors in behavioral tasks is critical for understanding cognition. Conventional wisdom assumes that encoding/decoding errors for continuous variables in behavioral tasks should naturally have Gaussian distributions, so that deviations from normality in the empirical data indicate the presence of more complex sources of noise. This line of reasoning has been central for prior research on working memory. Here, we reassess this assumption and find that even in ideal observer models with Gaussian encoding noise, the error distribution is generally non-Gaussian, contrary to the commonly held belief. Critically, we find that the shape of the error distribution is determined by the geometrical structure of the encoding manifold via a simple rule. In the case of a high-dimensional geometry, the error distributions naturally exhibit flat tails. Using this insight, we apply our theory to visual short-term memory tasks, and find that it can account for a large array of experimental data with only two free parameters. Our results challenge the dominant view in the mechanisms and capacity constraints of working memory systems. They instead suggest that the Bayesian framework, which explains various aspects of perceptual behavior, also provides an excellent account of working memory. Overall, our results establish a direct connection between neural manifold geometry and behavior, and call attention to the geometry of the representation as a critically important, yet underappreciated factor in determining the character of errors in human behavior.

The interplay of uncertainty, relevance and learning influences auditory categorization

Auditory perception requires categorizing sound sequences, such as speech or music, into classes, such as syllables or notes. Auditory categorization depends not only on the acoustic waveform, but also on variability and uncertainty in how the listener perceives the sound - including sensory and stimulus uncertainty, the listener's estimated relevance of the particular sound to the task, and their ability to learn the past statistics of the acoustic environment. Whereas these factors have been studied in isolation, whether and how these factors interact to shape categorization remains unknown. Here, we measured human participants' performance on a multi-tone categorization task and modeled each participant's behavior using a Bayesian framework. Task-relevant tones contributed more to category choice than task-irrelevant tones, confirming that participants combined information about sensory features with task relevance. Conversely, participants' poor estimates of task-relevant tones or high-sensory uncertainty adversely impacted category choice. Learning the statistics of sound category over both short and long timescales also affected decisions, biasing the decisions toward the overrepresented category. The magnitude of this effect correlated inversely with participants' relevance estimates. Our results demonstrate that individual participants idiosyncratically weigh sensory uncertainty, task relevance, and statistics over both short and long timescales, providing a novel understanding of and a computational framework for how sensory decisions are made under several simultaneous behavioral demands.

The representational instability in the generalization of fear learning

Perception and perceptual memory play crucial roles in fear generalization, yet their dynamic interaction remains understudied. This research (N = 80) explored their relationship through a classical differential conditioning experiment. Results revealed that while fear context perception fluctuates over time with a drift effect, perceptual memory remains stable, creating a disjunction between the two systems. Surprisingly, this disjunction does not significantly impact fear generalization behavior. Although most participants demonstrated generalization aligned with perceptual rather than physical stimulus distances, incorporating perceptual memory data into perceptual distance calculations did not enhance model performance. This suggests a potential shift in the mapping of the perceptual memory component of fear context, occurring alongside perceptual dynamics. Overall, this work provides evidence for understanding fear generalization behavior through different stimulus representational processes. Such mechanistic investigations can enhance our understanding of how individuals behave when facing threats and potentially aid in developing mechanism-specific diagnoses and treatments.

Models optimized for real-world tasks reveal the task-dependent necessity of precise temporal coding in hearing

Neurons encode information in the timing of their spikes in addition to their firing rates. Spike timing is particularly precise in the auditory nerve, where action potentials phase lock to sound with sub-millisecond precision, but its behavioral relevance remains uncertain. We optimized machine learning models to perform real-world hearing tasks with simulated cochlear input, assessing the precision of auditory nerve spike timing needed to reproduce human behavior. Models with high-fidelity phase locking exhibited more human-like sound localization and speech perception than models without, consistent with an essential role in human hearing. However, the temporal precision needed to reproduce human-like behavior varied across tasks, as did the precision that benefited real-world task performance. These effects suggest that perceptual domains incorporate phase locking to different extents depending on the demands of real-world hearing. The results illustrate how optimizing models for realistic tasks can clarify the role of candidate neural codes in perception.

Listening with generative models

Perception has long been envisioned to use an internal model of the world to explain the causes of sensory signals. However, such accounts have historically not been testable, typically requiring intractable search through the space of possible explanations. Using auditory scenes as a case study, we leveraged contemporary computational tools to infer explanations of sounds in a candidate internal generative model of the auditory world (ecologically inspired audio synthesizers). Model inferences accounted for many classic illusions. Unlike traditional accounts of auditory illusions, the model is applicable to any sound, and exhibited human-like perceptual organization for real-world sound mixtures. The combination of stimulus-computability and interpretable model structure enabled 'rich falsification', revealing additional assumptions about sound generation needed to account for perception. The results show how generative models can account for the perception of both classic illusions and everyday sensory signals, and illustrate the opportunities and challenges involved in incorporating them into theories of perception.

Multidigit tactile perception II: perceptual weighting during integration follows a leading-finger priority

When we run our hand across a surface, each finger typically repeats the sensory stimulation that the leading finger has already experienced. Because of this redundancy, the leading finger may attract more attention and contribute more strongly when tactile signals are integrated across fingers to form an overall percept. To test this hypothesis, we re-analyzed data collected in a previous study (Arslanova I, Takamuku S, Gomi H, Haggard P, J Neurophysiol 128: 418-433, 2022), where two probes were moved in different directions on two different fingerpads and participants reported the probes' average direction. Here, we evaluate the relative contribution of each finger to the percept and examine whether multidigit integration gives priority to the leading finger. Although the hand actually remained static in these experiments, a "functional leading finger" could be defined with reference to the average direction of the stimuli and the direction of hand-object relative motion that this implied. When participants averaged the motion direction across fingers of the same hand, the leading finger received a higher weighting than the nonleading finger, even though this biased estimate of average direction. Importantly, this bias disappeared when averaging motion direction across the two hands. Both the reported average direction and its systematic relation to the difference between the individual stimulus directions were explained by a model of motion integration in which the sensory weighting of stimuli depends on the directions of the applied stimuli. Our finding supports the hypothesis that the leading finger, which often receives novel information in natural hand-object interactions, is prioritized in forming our tactile perception.NEW & NOTEWORTHY The capacity of the tactile system to process multiple simultaneous stimuli is restricted. One solution could be to prioritize input from more informative sources. Here, we show that sensory weighting accorded to each finger during multidigit touch is biased in a direction-dependent manner when different motions are delivered to the fingers of the same hand. We argue that tactile inputs are weighted based on purely geometric information to prioritize "novel" information from the leading finger.

A dissociation between the use of implicit and explicit priors in perceptual inference

The brain constantly uses prior knowledge of the statistics of its environment to shape perception. These statistics are often implicit (not directly observable) and learned incrementally from observation, but they can also be explicitly communicated to the observer, especially in humans. Here, we show that priors are used differently in human perceptual inference depending on whether they are explicit or implicit in the environment. Bayesian modeling of learning and perception revealed that the weight of the sensory likelihood in perceptual decisions was highly correlated across participants between tasks with implicit and explicit priors, and slightly stronger in the implicit task. By contrast, the weight of priors was much less correlated across tasks, and it was markedly smaller for explicit priors. The model comparison also showed that different computations underpinned perceptual decisions depending on the origin of the priors. This dissociation may resolve previously conflicting results about the appropriate use of priors in human perception.

Sparse-Coding Variational Autoencoders

The sparse coding model posits that the visual system has evolved to efficiently code natural stimuli using a sparse set of features from an overcomplete dictionary. The original sparse coding model suffered from two key limitations; however: (1) computing the neural response to an image patch required minimizing a nonlinear objective function via recurrent dynamics and (2) fitting relied on approximate inference methods that ignored uncertainty. Although subsequent work has developed several methods to overcome these obstacles, we propose a novel solution inspired by the variational autoencoder (VAE) framework. We introduce the sparse coding variational autoencoder (SVAE), which augments the sparse coding model with a probabilistic recognition model parameterized by a deep neural network. This recognition model provides a neurally plausible feedforward implementation for the mapping from image patches to neural activities and enables a principled method for fitting the sparse coding model to data via maximization of the evidence lower bound (ELBO). The SVAE differs from standard VAEs in three key respects: the latent representation is overcomplete (there are more latent dimensions than image pixels), the prior is sparse or heavy-tailed instead of gaussian, and the decoder network is a linear projection instead of a deep network. We fit the SVAE to natural image data under different assumed prior distributions and show that it obtains higher test performance than previous fitting methods. Finally, we examine the response properties of the recognition network and show that it captures important nonlinear properties of neurons in the early visual pathway.

An inductive bias for slowly changing features in human reinforcement learning

Identifying goal-relevant features in novel environments is a central challenge for efficient behaviour. We asked whether humans address this challenge by relying on prior knowledge about common properties of reward-predicting features. One such property is the rate of change of features, given that behaviourally relevant processes tend to change on a slower timescale than noise. Hence, we asked whether humans are biased to learn more when task-relevant features are slow rather than fast. To test this idea, 295 human participants were asked to learn the rewards of two-dimensional bandits when either a slowly or quickly changing feature of the bandit predicted reward. Across two experiments and one preregistered replication, participants accrued more reward when a bandit's relevant feature changed slowly, and its irrelevant feature quickly, as compared to the opposite. We did not find a difference in the ability to generalise to unseen feature values between conditions. Testing how feature speed could affect learning with a set of four function approximation Kalman filter models revealed that participants had a higher learning rate for the slow feature, and adjusted their learning to both the relevance and the speed of feature changes. The larger the improvement in participants' performance for slow compared to fast bandits, the more strongly they adjusted their learning rates. These results provide evidence that human reinforcement learning favours slower features, suggesting a bias in how humans approach reward learning.

No evidence for a difference in Bayesian reasoning for egocentric versus allocentric spatial cognition

Bayesian reasoning (i.e. prior integration, cue combination, and loss minimization) has emerged as a prominent model for some kinds of human perception and cognition. The major theoretical issue is that we do not yet have a robust way to predict when we will or will not observe Bayesian effects in human performance. Here we tested a proposed divide in terms of Bayesian reasoning for egocentric spatial cognition versus allocentric spatial cognition (self-centered versus world-centred). The proposal states that people will show stronger Bayesian reasoning effects when it is possible to perform the Bayesian calculations within the egocentric frame, as opposed to requiring an allocentric frame. Three experiments were conducted with one egocentric-allowing condition and one allocentric-requiring condition but otherwise matched as closely as possible. No difference was found in terms of prior integration (Experiment 1), cue combination (Experiment 2), or loss minimization (Experiment 3). The contrast in previous reports, where Bayesian effects are present in many egocentric-allowing tasks while they are absent in many allocentric-requiring tasks, is likely due to other differences between the tasks-for example, the way allocentric-requiring tasks are often more complex and memory intensive.

The distinct development of stimulus and response serial dependence

Serial dependence (SD) is a phenomenon wherein current perceptions are biased by the previous stimulus and response. This helps to attenuate perceptual noise and variability in sensory input and facilitates stable ongoing perceptions of the environment. However, little is known about the developmental trajectory of SD. This study investigates how the stimulus and response biases of the SD effect develop across three age groups. Conventional analyses, in which previous stimulus and response biases were assessed separately, revealed significant changes in the biases over time. Previous stimulus bias shifted from repulsion to attraction, while previous response bias evolved from attraction to greater attraction. However, there was a strong correlation between stimulus and response orientations. Therefore, a generalized linear mixed-effects (GLME) analysis that simultaneously considered both previous stimulus and response, outperformed separate analyses. This revealed that previous stimulus and response resulted in two distinct biases with different developmental trajectories. The repulsion bias of previous stimulus remained relatively stable across all age groups, whereas the attraction bias of previous response was significantly stronger in adults than in children and adolescents. These findings demonstrate that the repulsion bias towards preceding stimuli is established early in the developing brain (at least by around 10 years old), while the attraction bias towards responses is not fully developed until adulthood. Our findings provide new insights into the development of the SD phenomenon and how humans integrate two opposing mechanisms into their perceptual responses to external input during development.

Electrical Signaling Beyond Neurons

Neural action potentials (APs) are difficult to interpret as signal encoders and/or computational primitives. Their relationships with stimuli and behaviors are obscured by the staggering complexity of nervous systems themselves. We can reduce this complexity by observing that "simpler" neuron-less organisms also transduce stimuli into transient electrical pulses that affect their behaviors. Without a complicated nervous system, APs are often easier to understand as signal/response mechanisms. We review examples of nonneural stimulus transductions in domains of life largely neglected by theoretical neuroscience: bacteria, protozoans, plants, fungi, and neuron-less animals. We report properties of those electrical signals-for example, amplitudes, durations, ionic bases, refractory periods, and particularly their ecological purposes. We compare those properties with those of neurons to infer the tasks and selection pressures that neurons satisfy. Throughout the tree of life, nonneural stimulus transductions time behavioral responses to environmental changes. Nonneural organisms represent the presence or absence of a stimulus with the presence or absence of an electrical signal. Their transductions usually exhibit high sensitivity and specificity to a stimulus, but are often slow compared to neurons. Neurons appear to be sacrificing the specificity of their stimulus transductions for sensitivity and speed. We interpret cellular stimulus transductions as a cell's assertion that it detected something important at that moment in time. In particular, we consider neural APs as fast but noisy detection assertions. We infer that a principal goal of nervous systems is to detect extremely weak signals from noisy sensory spikes under enormous time pressure. We discuss neural computation proposals that address this goal by casting neurons as devices that implement online, analog, probabilistic computations with their membrane potentials. Those proposals imply a measurable relationship between afferent neural spiking statistics and efferent neural membrane electrophysiology.

Optimization in Visual Motion Estimation

Sighted animals use visual signals to discern directional motion in their environment. Motion is not directly detected by visual neurons, and it must instead be computed from light signals that vary over space and time. This makes visual motion estimation a near universal neural computation, and decades of research have revealed much about the algorithms and mechanisms that generate directional signals. The idea that sensory systems are optimized for performance in natural environments has deeply impacted this research. In this article, we review the many ways that optimization has been used to quantitatively model visual motion estimation and reveal its underlying principles. We emphasize that no single optimization theory has dominated the literature. Instead, researchers have adeptly incorporated different computational demands and biological constraints that are pertinent to the specific brain system and animal model under study. The successes and failures of the resulting optimization models have thereby provided insights into how computational demands and biological constraints together shape neural computation.

Perceptual axioms are irreconcilable with Euclidean geometry

There are different definitions of axioms, but the one that seems to have general approval is that axioms are statements whose truths are universally accepted but cannot be proven; they are the foundation from which further propositional truths are derived. Previous attempts, led by David Hilbert, to show that all of mathematics can be built into an axiomatic system that is complete and consistent failed when Kurt Gödel proved that there will always be statements which are known to be true but can never be proven within the same axiomatic system. But Gödel and his followers took no account of brain mechanisms that generate and mediate logic. In this largely theoretical paper, but backed by previous experiments and our new ones reported below, we show that in the case of so-called 'optical illusions', there exists a significant and irreconcilable difference between their visual perception and their description according to Euclidean geometry; when participants are asked to adjust, from an initial randomised state, the perceptual geometric axioms to conform to the Euclidean description, the two never match, although the degree of mismatch varies between individuals. These results provide evidence that perceptual axioms, or statements known to be perceptually true, cannot be described mathematically. Thus, the logic of the visual perceptual system is irreconcilable with the cognitive (mathematical) system and cannot be updated even when knowledge of the difference between the two is available. Hence, no one brain reality is more 'objective' than any other.

The precision of signals encoding active self-movement

Everyday actions like moving the head, walking around, and grasping objects are typically self-controlled. This presents a problem when studying the signals encoding such actions because active self-movement is difficult to control experimentally. Available techniques demand repeatable trials, but each action is unique, making it difficult to measure fundamental properties like psychophysical thresholds. We present a novel paradigm that recovers both precision and bias of self-movement signals with minimal constraint on the participant. The paradigm relies on linking image motion to previous self-movement, and two experimental phases to extract the signal encoding the latter. The paradigm takes care of a hidden source of external noise not previously accounted for in techniques that link display motion to self-movement in real time (e.g., virtual reality). We use head rotations as an example of self-movement, and show that the precision of the signals encoding head movement depends on whether they are being used to judge visual motion or auditory motion. We find that perceived motion is slowed during head movement in both cases. The "nonimage" signals encoding active head rotation (motor commands, proprioception, and vestibular cues) are therefore biased toward lower speeds and/or displacements. In a second experiment, we trained participants to rotate their heads at different rates and found that the imprecision of the head rotation signal rises proportionally with head speed (Weber's law). We discuss the findings in terms of the different motion cues used by vision and hearing, and the implications they have for Bayesian models of motion perception.NEW & NOTEWORTHY We present a psychophysical technique for measuring the precision of signals encoding active self-movements. Using head movements, we show that 1) precision is greater when active head rotation is performed using visual comparison stimuli versus auditory; 2) precision decreases with head speed (Weber's law); 3) perceived speed is lower during head rotation. The findings may reflect the steps needed to convert different cues into common units, and challenge standard Bayesian models of motion perception.

Age-related changes in the susceptibility to visual illusions of size

As the global population ages, understanding of the effect of aging on visual perception is of growing importance. This study investigates age-related changes in adulthood along size perception through the lens of three visual illusions: the Ponzo, Ebbinghaus, and Height-width illusions. Utilizing the Bayesian conceptualization of the aging brain, which posits increased reliance on prior knowledge with age, we explored potential differences in the susceptibility to visual illusions across different age groups in adults (ages 20-85 years). To this end, we used the BTPI (Ben-Gurion University Test for Perceptual Illusions), an online validated battery of visual illusions developed in our lab. The findings revealed distinct patterns of age-related changes for each of the illusions, challenging the idea of a generalized increase in reliance on prior knowledge with age. Specifically, we observed a systematic reduction in susceptibility to the Ebbinghaus illusion with age, while susceptibility to the Height-width illusion increased with age. As for the Ponzo illusion, there were no significant changes with age. These results underscore the complexity of age-related changes in visual perception and converge with previous findings to support the idea that different visual illusions of size are mediated by distinct perceptual mechanisms.

The relativity of reaching: Motion of the touched surface alters the trajectory of hand movements

For dexterous control of the hand, humans integrate sensory information and prior knowledge regarding their bodies and the world. We studied the role of touch in hand motor control by challenging a fundamental prior assumption-that self-motion of inanimate objects is unlikely upon contact. In a reaching task, participants slid their fingertips across a robotic interface, with their hand hidden from sight. Unbeknownst to the participants, the robotic interface remained static, followed hand movement, or moved in opposition to it. We considered two hypotheses. Either participants were able to account for surface motion or, if the stationarity assumption held, they would integrate the biased tactile cues and proprioception. Motor errors consistent with the latter hypothesis were observed. The role of visual feedback, tactile sensitivity, and friction was also investigated. Our study carries profound implications for human-machine collaboration in a world where objects may no longer conform to the stationarity assumption.

How does V1 population activity inform perceptual certainty?

Neural population activity in sensory cortex informs our perceptual interpretation of the environment. Oftentimes, this population activity will support multiple alternative interpretations. The larger the spread of probability over different alternatives, the more uncertain the selected perceptual interpretation. We test the hypothesis that the reliability of perceptual interpretations can be revealed through simple transformations of sensory population activity. We recorded V1 population activity in fixating macaques while presenting oriented stimuli under different levels of nuisance variability and signal strength. We developed a decoding procedure to infer from V1 activity the most likely stimulus orientation as well as the certainty of this estimate. Our analysis shows that response magnitude, response dispersion, and variability in response gain all offer useful proxies for orientation certainty. Of these three metrics, the last one has the strongest association with the decoder's uncertainty estimates. These results clarify that the nature of neural population activity in sensory cortex provides downstream circuits with multiple options to assess the reliability of perceptual interpretations.

Abstract deliberation by visuomotor neurons in prefrontal cortex

During visually guided behavior, the prefrontal cortex plays a pivotal role in mapping sensory inputs onto appropriate motor plans. When the sensory input is ambiguous, this involves deliberation. It is not known whether the deliberation is implemented as a competition between possible stimulus interpretations or between possible motor plans. Here we study neural population activity in the prefrontal cortex of macaque monkeys trained to flexibly report perceptual judgments of ambiguous visual stimuli. We find that the population activity initially represents the formation of a perceptual choice before transitioning into the representation of the motor plan. Stimulus strength and prior expectations both bear on the formation of the perceptual choice, but not on the formation of the action plan. These results suggest that prefrontal circuits involved in action selection are also used for the deliberation of abstract propositions divorced from a specific motor plan, thus providing a crucial mechanism for abstract reasoning.

Incongruent active head rotations increase visual motion detection thresholds

Attributing a visual motion signal to its correct source-be that external object motion, self-motion, or some combination of both-seems effortless, and yet often involves disentangling a complex web of motion signals. Existing literature focuses on either translational motion (heading) or eye movements, leaving much to be learnt about the influence of a wider range of self-motions, such as active head rotations, on visual motion perception. This study investigated how active head rotations affect visual motion detection thresholds, comparing conditions where visual motion and head-turn direction were either congruent or incongruent. Participants judged the direction of a visual motion stimulus while rotating their head or remaining stationary, using a fixation-locked Virtual Reality display with integrated head-movement recordings. Thresholds to perceive visual motion were higher in both active-head rotation conditions compared to stationary, though no differences were found between congruent or incongruent conditions. Participants also showed a significant bias to report seeing visual motion travelling in the same direction as the head rotation. Together, these results demonstrate active head rotations increase visual motion perceptual thresholds, particularly in cases of incongruent visual and active vestibular stimulation.

A paradigm for characterizing motion misperception in people with typical vision and low vision

PURPOSE

We aimed to develop a paradigm that can efficiently characterize motion percepts in people with low vision and compare their responses with well-known misperceptions made by people with typical vision when targets are hard to see.

METHODS

We recruited a small cohort of individuals with reduced acuity and contrast sensitivity (n = 5) as well as a comparison cohort with typical vision (n = 5) to complete a psychophysical study. Study participants were asked to judge the motion direction of a tilted rhombus that was either high or low contrast. In a series of trials, the rhombus oscillated vertically, horizontally, or diagonally. Participants indicated the perceived motion direction using a number wheel with 12 possible directions, and statistical tests were used to examine response biases.

RESULTS

All participants with typical vision showed systematic misperceptions well predicted by a Bayesian inference model. Specifically, their perception of vertical or horizontal motion was biased toward directions orthogonal to the long axis of the rhombus. They had larger biases for hard-to-see (low contrast) stimuli. Two participants with low vision had a similar bias, but with no difference between high- and low-contrast stimuli. The other participants with low vision were unbiased in their percepts or biased in the opposite direction.

CONCLUSIONS

Our results suggest that some people with low vision may misperceive motion in a systematic way similar to people with typical vision. However, we observed large individual differences. Future work will aim to uncover reasons for such differences and identify aspects of vision that predict susceptibility.

Representing stimulus motion with waves in adaptive neural fields

Traveling waves of neural activity emerge in cortical networks both spontaneously and in response to stimuli. The spatiotemporal structure of waves can indicate the information they encode and the physiological processes that sustain them. Here, we investigate the stimulus-response relationships of traveling waves emerging in adaptive neural fields as a model of visual motion processing. Neural field equations model the activity of cortical tissue as a continuum excitable medium, and adaptive processes provide negative feedback, generating localized activity patterns. Synaptic connectivity in our model is described by an integral kernel that weakens dynamically due to activity-dependent synaptic depression, leading to marginally stable traveling fronts (with attenuated backs) or pulses of a fixed speed. Our analysis quantifies how weak stimuli shift the relative position of these waves over time, characterized by a wave response function we obtain perturbatively. Persistent and continuously visible stimuli model moving visual objects. Intermittent flashes that hop across visual space can produce the experience of smooth apparent visual motion. Entrainment of waves to both kinds of moving stimuli are well characterized by our theory and numerical simulations, providing a mechanistic description of the perception of visual motion.

Conflicts between short- and long-term experiences affect visual perception through modulating sensory or motor response systems: Evidence from Bayesian inference models

The independent effects of short- and long-term experiences on visual perception have been discussed for decades. However, no study has investigated whether and how these experiences simultaneously affect our visual perception. To address this question, we asked participants to estimate their self-motion directions (i.e., headings) simulated from optic flow, in which a long-term experience learned in everyday life (i.e., straight-forward motion being more common than lateral motion) plays an important role. The headings were selected from three distributions that resembled a peak, a hill, and a flat line, creating different short-term experiences. Importantly, the proportions of headings deviating from the straight-forward motion gradually increased in the peak, hill, and flat distributions, leading to a greater conflict between long- and short-term experiences. The results showed that participants biased their heading estimates towards the straight-ahead direction and previously seen headings, which increased with the growing experience conflict. This suggests that both long- and short-term experiences simultaneously affect visual perception. Finally, we developed two Bayesian models (Model 1 vs. Model 2) based on two assumptions that the experience conflict altered the likelihood distribution of sensory representation or the motor response system. The results showed that both models accurately predicted participants' estimation biases. However, Model 1 predicted a higher variance of serial dependence compared to Model 2, while Model 2 predicted a higher variance of the bias towards the straight-ahead direction compared to Model 1. This suggests that the experience conflict can influence visual perception by affecting both sensory and motor response systems. Taken together, the current study systematically revealed the effects of long- and short-term experiences on visual perception and the underlying Bayesian processing mechanisms.

Structural and Functional Analysis of the Amorphous Calcium Carbonate-Binding Protein Paramyosin in the Shell of the Pearl Oyster, Pinctada fucata

Amorphous calcium carbonate (ACC) is an important precursor phase for the formation of aragonite crystals in the shells of Pinctada fucata. To identify the ACC-binding protein in the inner aragonite layer of the shell, extracts from the shell were used in the ACC-binding experiments. Semiquantitative analyses using liquid chromatography-mass spectrometry revealed that paramyosin was strongly associated with ACC in the shell. We discovered that paramyosin, a major component of the adductor muscle, was included in the myostracum, which is the microstructure of the shell attached to the adductor muscle. Purified paramyosin accumulates calcium carbonate and induces the prism structure of aragonite crystals, which is related to the morphology of prism aragonite crystals in the myostracum. Nuclear magnetic resonance measurements revealed that the Glu-rich region was bound to ACC. Activity of the Glu-rich region was stronger than that of the Asp-rich region. These results suggest that paramyosin in the adductor muscle is involved in the formation of aragonite prisms in the myostracum.

Stereoscopic slant contrast revisited

The perceived slant of a stereoscopic surface is altered by the presence of a surrounding surface, a phenomenon termed stereo slant contrast. Previous studies have shown that a slanted surround causes a fronto-parallel surface to appear slanted in the opposite direction, an instance of "bidirectional" contrast. A few studies have examined slant contrast using slanted as opposed to fronto-parallel test surfaces, and these also have shown slant contrast. Here, we use a matching method to examine slant contrast over a wide range of combinations of surround and test slants, one aim being to determine whether stereo slant contrast transfers across opposite directions of test and surround slant. We also examine the effect of the test on the perceived slant of the surround. Test slant contrast was found to be bidirectional in virtually all test-surround combinations and transferred across opposite test and surround slants, with little or no decline in magnitude as the test-surround slant difference approached the limit. There was a weak bidirectional effect of the test slant on the perceived slant of the surround. We consider how our results might be explained by four mechanisms: (a) normalization of stereo slant to vertical; (b) divisive normalization of stereo slant channels in a manner analogous to the tilt illusion; (c) interactions between center and surround disparity-gradient detectors; and (d) uncertainty in slant estimation. We conclude that the third of these (interactions between center and surround disparity-gradient detectors) is the most likely cause of stereo slant contrast.

A unifying theory explains seemingly contradictory biases in perceptual estimation

Perceptual biases are widely regarded as offering a window into the neural computations underlying perception. To understand these biases, previous work has proposed a number of conceptually different, and even seemingly contradictory, explanations, including attraction to a Bayesian prior, repulsion from the prior due to efficient coding and central tendency effects on a bounded range. We present a unifying Bayesian theory of biases in perceptual estimation derived from first principles. We demonstrate theoretically an additive decomposition of perceptual biases into attraction to a prior, repulsion away from regions with high encoding precision and regression away from the boundary. The results reveal a simple and universal rule for predicting the direction of perceptual biases. Our theory accounts for, and yields, new insights regarding biases in the perception of a variety of stimulus attributes, including orientation, color and magnitude. These results provide important constraints on the neural implementations of Bayesian computations.

Perceptual transitions between object rigidity and non-rigidity: Competition and cooperation among motion energy, feature tracking, and shape-based priors

Why do moving objects appear rigid when projected retinal images are deformed non-rigidly? We used rotating rigid objects that can appear rigid or non-rigid to test whether shape features contribute to rigidity perception. When two circular rings were rigidly linked at an angle and jointly rotated at moderate speeds, observers reported that the rings wobbled and were not linked rigidly, but rigid rotation was reported at slow speeds. When gaps, paint, or vertices were added, the rings appeared rigidly rotating even at moderate speeds. At high speeds, all configurations appeared non-rigid. Salient features thus contribute to rigidity at slow and moderate speeds but not at high speeds. Simulated responses of arrays of motion-energy cells showed that motion flow vectors are predominantly orthogonal to the contours of the rings, not parallel to the rotation direction. A convolutional neural network trained to distinguish flow patterns for wobbling versus rotation gave a high probability of wobbling for the motion-energy flows. However, the convolutional neural network gave high probabilities of rotation for motion flows generated by tracking features with arrays of MT pattern-motion cells and corner detectors. In addition, circular rings can appear to spin and roll despite the absence of any sensory evidence, and this illusion is prevented by vertices, gaps, and painted segments, showing the effects of rotational symmetry and shape. Combining convolutional neural network outputs that give greater weight to motion energy at fast speeds and to feature tracking at slow speeds, with the shape-based priors for wobbling and rolling, explained rigid and non-rigid percepts across shapes and speeds (R2 = 0.95). The results demonstrate how cooperation and competition between different neuronal classes lead to specific states of visual perception and to transitions between the states.

Bayesian Decision-Making Under Uncertainty in Borderline Personality Disorder

Bayesian decision theory suggests that optimal decision-making should use and weigh prior beliefs with current information, according to their relative uncertainties. However, some characteristics of borderline personality disorder (BPD) patients, such as fast, drastic changes in the overall perception of themselves and others, suggest they may be underrelying on priors. Here, we investigated if BPD patients have a general deficit in relying on or combining prior with current information. We analyzed this by having BPD patients (n = 23) and healthy controls (n = 18) perform a coin-catching sensorimotor task with varying levels of prior and current information uncertainty. Our results indicate that BPD patients learned and used prior information and combined it with current information in a qualitatively Bayesian-like way. Our results show that, at least in a lower-level, nonsocial sensorimotor task, BPD patients can appropriately use both prior and current information, illustrating that potential deficits using priors may not be widespread or domain-general.

Effect of motion state variability on error-related potentials during continuous feedback paradigms and their consequences for classification

BACKGROUND

An erroneous motion would elicit the error-related potential (ErrP) when humans monitor the behavior of the external devices. This EEG modality has been largely applied to brain-computer interface in an active or passive manner with discrete visual feedback. However, the effect of variable motion state on ErrP morphology and classification performance raises concerns when the interaction is conducted with continuous visual feedback.

NEW METHOD

In the present study, we designed a cursor control experiment. Participants monitored a continuously moving cursor to reach the target on one side of the screen. Motion state varied multiple times with two factors: (1) motion direction and (2) motion speed. The effects of these two factors on the morphological characteristics and classification performance of ErrP were analyzed. Furthermore, an offline simulation was performed to evaluate the effectiveness of the proposed extended ErrP-decoder in resolving the interference by motion direction changes.

RESULTS

The statistical analyses revealed that motion direction and motion speed significantly influenced the amplitude of feedback-ERN and frontal-Pe components, while only motion direction significantly affected the classification performance.

COMPARISON WITH EXISTING METHODS

Significant deviation was found in ErrP detection utilizing classical correct-versus-erroneous event training. However, this bias can be alleviated by 16% by the extended ErrP-decoder.

CONCLUSION

The morphology and classification performance of ErrP signal can be affected by motion state variability during continuous feedback paradigms. The results enhance the comprehension of ErrP morphological components and shed light on the detection of BCI's error behavior in practical continuous control.

From neural noise to co-adaptability: Rethinking the multifaceted architecture of motor variability

In the last decade, the source and the functional meaning of motor variability have attracted considerable attention in behavioral and brain sciences. This construct classically combined different levels of description, variable internal robustness or coherence, and multifaceted operational meanings. We provide here a comprehensive review of the literature with the primary aim of building a precise lexicon that goes beyond the generic and monolithic use of motor variability. In the pars destruens of the work, we model three domains of motor variability related to peculiar computational elements that influence fluctuations in motor outputs. Each domain is in turn characterized by multiple sub-domains. We begin with the domains of noise and differentiation. However, the main contribution of our model concerns the domain of adaptability, which refers to variation within the same exact motor representation. In particular, we use the terms learning and (social)fitting to specify the portions of motor variability that depend on our propensity to learn and on our largely constitutive propensity to be influenced by external factors. A particular focus is on motor variability in the context of the sub-domain named co-adaptability. Further groundbreaking challenges arise in the modeling of motor variability. Therefore, in a separate pars construens, we attempt to characterize these challenges, addressing both theoretical and experimental aspects as well as potential clinical implications for neurorehabilitation. All in all, our work suggests that motor variability is neither simply detrimental nor beneficial, and that studying its fluctuations can provide meaningful insights for future research.

Serial dependence bias can predict the overall estimation error in visual perception

Although visual feature estimations are accurate and precise, overall estimation errors (i.e., the difference between estimates and actual values) tend to show systematic patterns. For example, estimates of orientations are systematically biased away from horizontal and vertical orientations, showing an oblique illusion. Additionally, many recent studies have demonstrated that estimations of current visual features are systematically biased toward previously seen features, showing a serial dependence. However, no study examined whether the overall estimation errors were correlated with the serial dependence bias. To address this question, we enrolled three groups of participants to estimate orientation, motion speed, and point-light-walker direction. The results showed that the serial dependence bias explained over 20% of overall estimation errors in the three tasks, indicating that we could use the serial dependence bias to predict the overall estimation errors. The current study first demonstrated that the serial dependence bias was not independent from the overall estimation errors. This finding could inspire researchers to investigate the neural bases underlying the visual feature estimation and serial dependence.

Perceiving depth and motion in depth from successive occlusion

Occlusion, or interposition, is one of the strongest and best-known pictorial cues to depth. Furthermore, the successive occlusions of previous objects by newly presented objects produces an impression of increasing depth. Although the perceived motion associated with this illusion has been studied, the depth percept has not. To investigate, participants were presented with two piles of disks with one always static and the other either a static pile or a stacking pile where a new disk was added every 200 ms. We found static piles with equal number of disks appeared equal in height. In contrast, the successive presentation of disks in the stacking condition appeared to enhance the perceived height of the stack-fewer disks were needed to match the static pile. Surprisingly, participants were also more precise when comparing stacking versus static piles of disks. Reversing the stacking by removing rather than adding disks reversed the bias and degraded precision. In follow-up experiments, we used nonoverlapping static and dynamic configurations to show that the effects are not due to simple differences in perceived numerosity. In sum, our results show that successive occlusions generate a greater sense of height than occlusion alone, and we posit that dynamic occlusion may be an underappreciated source of depth information.

Rational inferences about social valuation

The decisions made by other people can contain information about the value they assign to our welfare-for example how much they are willing to sacrifice to make us better off. An emerging body of research suggests that we extract and use this information, responding more favorably to those who sacrifice more even if they provide us with less. The magnitude of their trade-offs governs our social responses to them-including partner choice, giving, and anger. This implies that people have well-designed cognitive mechanisms for estimating the weight someone else assigns to their welfare, even when the amounts at stake vary and the information is noisy or sparse. We tested this hypothesis in two studies (N=200; US samples) by asking participants to observe a partner make two trade-offs, and then predict the partner's decisions in other trials. Their predictions were compared to those of a model that uses statistically optimal procedures, operationalized as a Bayesian ideal observer. As predicted, (i) the estimates people made from sparse evidence matched those of the ideal observer, and (ii) lower welfare trade-offs elicited more anger from participants, even when their total payoffs were held constant. These results support the view that people efficiently update their representations of how much others value them. They also provide the most direct test to date of a key assumption of the recalibrational theory of anger: that anger is triggered by cues of low valuation, not by the infliction of costs.

Causal inference during closed-loop navigation: parsing of self- and object-motion

A key computation in building adaptive internal models of the external world is to ascribe sensory signals to their likely cause(s), a process of causal inference (CI). CI is well studied within the framework of two-alternative forced-choice tasks, but less well understood within the cadre of naturalistic action-perception loops. Here, we examine the process of disambiguating retinal motion caused by self- and/or object-motion during closed-loop navigation. First, we derive a normative account specifying how observers ought to intercept hidden and moving targets given their belief about (i) whether retinal motion was caused by the target moving, and (ii) if so, with what velocity. Next, in line with the modelling results, we show that humans report targets as stationary and steer towards their initial rather than final position more often when they are themselves moving, suggesting a putative misattribution of object-motion to the self. Further, we predict that observers should misattribute retinal motion more often: (i) during passive rather than active self-motion (given the lack of an efference copy informing self-motion estimates in the former), and (ii) when targets are presented eccentrically rather than centrally (given that lateral self-motion flow vectors are larger at eccentric locations during forward self-motion). Results support both of these predictions. Lastly, analysis of eye movements show that, while initial saccades toward targets were largely accurate regardless of the self-motion condition, subsequent gaze pursuit was modulated by target velocity during object-only motion, but not during concurrent object- and self-motion. These results demonstrate CI within action-perception loops, and suggest a protracted temporal unfolding of the computations characterizing CI. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Two Sides of the Same Coin: Efficient and Predictive Neural Coding

Some visual properties are consistent across a wide range of environments, while other properties are more labile. The efficient coding hypothesis states that many of these regularities in the environment can be discarded from neural representations, thus allocating more of the brain's dynamic range to properties that are likely to vary. This paradigm is less clear about how the visual system prioritizes different pieces of information that vary across visual environments. One solution is to prioritize information that can be used to predict future events, particularly those that guide behavior. The relationship between the efficient coding and future prediction paradigms is an area of active investigation. In this review, we argue that these paradigms are complementary and often act on distinct components of the visual input. We also discuss how normative approaches to efficient coding and future prediction can be integrated.

Task feedback suggests a post-perceptual component to serial dependence

Decisions across a range of perceptual tasks are biased toward past stimuli. Such serial dependence is thought to be an adaptive low-level mechanism that promotes perceptual stability across time. However, recent studies suggest post-perceptual mechanisms may also contribute to serially biased responses, calling into question a single locus of serial dependence and the nature of integration of past and present sensory inputs. We measured serial dependence in the context of a three-dimensional (3D) motion perception task where uncertainty in the sensory information varied substantially from trial to trial. We found that serial dependence varied with stimulus properties that impact sensory uncertainty on the current trial. Reduced stimulus contrast was associated with an increased bias toward the stimulus direction of the previous trial. Critically, performance feedback, which reduced sensory uncertainty, abolished serial dependence. These results provide clear evidence for a post-perceptual locus of serial dependence in 3D motion perception and support the role of serial dependence as a response strategy in the face of substantial sensory uncertainty.

Humans display interindividual differences in the latent mechanisms underlying fear generalization behaviour

Human generalization research aims to understand the processes underlying the transfer of prior experiences to new contexts. Generalization research predominantly relies on descriptive statistics, assumes a single generalization mechanism, interprets generalization from mono-source data, and disregards individual differences. Unfortunately, such an approach fails to disentangle various mechanisms underlying generalization behaviour and can readily result in biased conclusions regarding generalization tendencies. Therefore, we combined a computational model with multi-source data to mechanistically investigate human generalization behaviour. By simultaneously modelling learning, perceptual and generalization data at the individual level, we revealed meaningful variations in how different mechanisms contribute to generalization behaviour. The current research suggests the need for revising the theoretical and analytic foundations in the field to shift the attention away from forecasting group-level generalization behaviour and toward understanding how such phenomena emerge at the individual level. This raises the question for future research whether a mechanism-specific differential diagnosis may be beneficial for generalization-related psychiatric disorders.

Environmental dynamics shape perceptual decision bias

To interpret the sensory environment, the brain combines ambiguous sensory measurements with knowledge that reflects context-specific prior experience. But environmental contexts can change abruptly and unpredictably, resulting in uncertainty about the current context. Here we address two questions: how should context-specific prior knowledge optimally guide the interpretation of sensory stimuli in changing environments, and do human decision-making strategies resemble this optimum? We probe these questions with a task in which subjects report the orientation of ambiguous visual stimuli that were drawn from three dynamically switching distributions, representing different environmental contexts. We derive predictions for an ideal Bayesian observer that leverages knowledge about the statistical structure of the task to maximize decision accuracy, including knowledge about the dynamics of the environment. We show that its decisions are biased by the dynamically changing task context. The magnitude of this decision bias depends on the observer's continually evolving belief about the current context. The model therefore not only predicts that decision bias will grow as the context is indicated more reliably, but also as the stability of the environment increases, and as the number of trials since the last context switch grows. Analysis of human choice data validates all three predictions, suggesting that the brain leverages knowledge of the statistical structure of environmental change when interpreting ambiguous sensory signals.

Retinal motion statistics during natural locomotion

Walking through an environment generates retinal motion, which humans rely on to perform a variety of visual tasks. Retinal motion patterns are determined by an interconnected set of factors, including gaze location, gaze stabilization, the structure of the environment, and the walker's goals. The characteristics of these motion signals have important consequences for neural organization and behavior. However, to date, there are no empirical in situ measurements of how combined eye and body movements interact with real 3D environments to shape the statistics of retinal motion signals. Here, we collect measurements of the eyes, the body, and the 3D environment during locomotion. We describe properties of the resulting retinal motion patterns. We explain how these patterns are shaped by gaze location in the world, as well as by behavior, and how they may provide a template for the way motion sensitivity and receptive field properties vary across the visual field.

Motion duration is overestimated behind an occluder in action and perception tasks

Motion estimation behind an occluder is a common task in situations like crossing the street or passing another car. People tend to overestimate the duration of an object's motion when it gets occluded for subsecond motion durations. Here, we explored (a) whether this bias depended on the type of interceptive action: discrete keypress versus continuous reach and (b) whether it was present in a perception task without an interceptive action. We used a prediction-motion task and presented a bar moving across the screen with a constant velocity that later became occluded. In the action task, participants stopped the occluded bar when they thought the bar reached the goal position via keypress or reach. They were more likely to stop the bar after it passed the goal position regardless of the action type, suggesting that the duration of occluded motion was overestimated (or its speed was underestimated). In the perception task, where participants judged whether a tone was presented before or after the bar reached the goal position, a similar bias was observed. In both tasks, the bias was near constant across motion durations and directions and grew over trials. We speculate that this robust bias may be due to a temporal illusion, Bayesian slow-motion prior, or the processing of the visible-occluded boundary crossing. Understanding its exact mechanism, the conditions on which it depends, and the relative roles of speed and time perception requires further research.

Perceptual history biases in serial ensemble representation

Ensemble perception refers to the visual system's ability to efficiently represent groups of similar objects as a unified percept using their summary statistical information. Most studies focused on extraction of current trial averages, giving little attention to prior experience effects, although a few recent studies found that ensemble mean estimations contract toward previously presented stimuli, with most of these focusing on explicit perceptual averaging of simultaneously presented item ensembles. Yet, the time element is crucial in real dynamic environments, where we encounter ensemble items over time, aggregating information until reaching summary representations. Moreover, statistical information of objects and scenes is learned over time and often implicitly and then used for predictions that shape perception, promoting environmental stability. Therefore, we now focus on temporal aspects of ensemble statistics and test whether prior information, beyond the current trial, biases implicit perceptual decisions. We designed methods to separate current trial biases from those of previously seen trial ensembles. In each trial, six circles of different sizes were presented serially, followed by two test items. Participants were asked to choose which was present in the sequence. Participants unconsciously rely on ensemble statistics, choosing stimuli closer to the ensemble mean. To isolate the influence of earlier trials, the two test items were sometimes equidistant from the current trial mean. Results showed membership judgment biases toward current trial mean, when informative (largest effect). On equidistant trials, judgments were biased toward previously experienced stimulus statistics. Comparison of similar conditions with a shifted stimulus distribution ruled out a bias toward an earlier, presession, prototypical diameter. We conclude that ensemble perception, even for temporally experienced ensembles, is influenced not only by current trial mean but also by means of recently seen ensembles and that these influences are somewhat correlated on a participant-by-participant basis.

A dynamical model of visual motion processing for arbitrary stimuli including type II plaids

To explore the operating principle of visual motion processing in the brain underlying perception and eye movements, we model the information processing of velocity estimate of the visual stimulus at the algorithmic level using the dynamical system approach. In this study, we formulate the model as an optimization process of an appropriately defined objective function. The model is applicable to arbitrary visual stimuli. We find that our theoretical predictions qualitatively agree with time evolution of eye movement reported by previous works across various types of stimulus. Our results suggest that the brain implements the present framework as the internal model of motion vision. We anticipate our model to be a promising building block for more profound understanding of visual motion processing as well as for the development of robotics.

Recurrent networks endowed with structural priors explain suboptimal animal behavior

The strategies found by animals facing a new task are determined both by individual experience and by structural priors evolved to leverage the statistics of natural environments. Rats quickly learn to capitalize on the trial sequence correlations of two-alternative forced choice (2AFC) tasks after correct trials but consistently deviate from optimal behavior after error trials. To understand this outcome-dependent gating, we first show that recurrent neural networks (RNNs) trained in the same 2AFC task outperform rats as they can readily learn to use across-trial information both after correct and error trials. We hypothesize that, although RNNs can optimize their behavior in the 2AFC task without any a priori restrictions, rats' strategy is constrained by a structural prior adapted to a natural environment in which rewarded and non-rewarded actions provide largely asymmetric information. When pre-training RNNs in a more ecological task with more than two possible choices, networks develop a strategy by which they gate off the across-trial evidence after errors, mimicking rats' behavior. Population analyses show that the pre-trained networks form an accurate representation of the sequence statistics independently of the outcome in the previous trial. After error trials, gating is implemented by a change in the network dynamics that temporarily decouple the categorization of the stimulus from the across-trial accumulated evidence. Our results suggest that the rats' suboptimal behavior reflects the influence of a structural prior that reacts to errors by isolating the network decision dynamics from the context, ultimately constraining the performance in a 2AFC laboratory task.

Priming of probabilistic attentional templates

Attentional priming has a dominating influence on vision, speeding visual search, releasing items from crowding, reducing masking effects, and during free-choice, primed targets are chosen over unprimed ones. Many accounts postulate that templates stored in working memory control what we attend to and mediate the priming. But what is the nature of these templates (or representations)? Analyses of real-world visual scenes suggest that tuning templates to exact color or luminance values would be impractical since those can vary greatly because of changes in environmental circumstances and perceptual interpretation. Tuning templates to a range of the most probable values would be more efficient. Recent evidence does indeed suggest that the visual system represents such probability, gradually encoding statistical variation in the environment through repeated exposure to input statistics. This is consistent with evidence from neurophysiology and theoretical neuroscience as well as computational evidence of probabilistic representations in visual perception. I argue that such probabilistic representations are the unit of attentional priming and that priming of, say, a repeated single-color value simply involves priming of a distribution with no variance. This "priming of probability" view can be modelled within a Bayesian framework where priming provides contextual priors. Priming can therefore be thought of as learning of the underlying probability density function of the target or distractor sets in a given continuous task.