Perceiving the Present and a Systematization of Illusions

Illusory light drives pupil responses in primates

In humans, the eye pupils respond to both physical light sensed by the retina and mental representations of light produced by the brain. Notably, our pupils constrict when a visual stimulus is illusorily perceived brighter, even if retinal illumination is constant. However, it remains unclear whether such perceptual penetrability of pupil responses is an epiphenomenon unique to humans or whether it represents an adaptive mechanism shared with other animals to anticipate variations in retinal illumination between successive eye fixations. To address this issue, we measured the pupil responses of both humans and macaque monkeys exposed to three chromatic versions (cyan, magenta, and yellow) of the Asahi brightness illusion. We found that the stimuli illusorily perceived brighter or darker trigger differential pupil responses that are very similar in macaques and human participants. Additionally, we show that this phenomenon exhibits an analogous cyan bias in both primate species. Beyond evincing the macaque monkey as a relevant model to study the perceptual penetrability of pupil responses, our results suggest that this phenomenon is tuned to ecological conditions because the exposure to a "bright cyan-bluish sky" may be associated with increased risks of dazzle and retinal damages.

A time-causal and time-recursive scale-covariant scale-space representation of temporal signals and past time

This article presents an overview of a theory for performing temporal smoothing on temporal signals in such a way that: (i) temporally smoothed signals at coarser temporal scales are guaranteed to constitute simplifications of corresponding temporally smoothed signals at any finer temporal scale (including the original signal) and (ii) the temporal smoothing process is both time-causal and time-recursive, in the sense that it does not require access to future information and can be performed with no other temporal memory buffer of the past than the resulting smoothed temporal scale-space representations themselves. For specific subsets of parameter settings for the classes of linear and shift-invariant temporal smoothing operators that obey this property, it is shown how temporal scale covariance can be additionally obtained, guaranteeing that if the temporal input signal is rescaled by a uniform temporal scaling factor, then also the resulting temporal scale-space representations of the rescaled temporal signal will constitute mere rescalings of the temporal scale-space representations of the original input signal, complemented by a shift along the temporal scale dimension. The resulting time-causal limit kernel that obeys this property constitutes a canonical temporal kernel for processing temporal signals in real-time scenarios when the regular Gaussian kernel cannot be used, because of its non-causal access to information from the future, and we cannot additionally require the temporal smoothing process to comprise a complementary memory of the past beyond the information contained in the temporal smoothing process itself, which in this way also serves as a multi-scale temporal memory of the past. We describe how the time-causal limit kernel relates to previously used temporal models, such as Koenderink's scale-time kernels and the ex-Gaussian kernel. We do also give an overview of how the time-causal limit kernel can be used for modelling the temporal processing in models for spatio-temporal and spectro-temporal receptive fields, and how it more generally has a high potential for modelling neural temporal response functions in a purely time-causal and time-recursive way, that can also handle phenomena at multiple temporal scales in a theoretically well-founded manner. We detail how this theory can be efficiently implemented for discrete data, in terms of a set of recursive filters coupled in cascade. Hence, the theory is generally applicable for both: (i) modelling continuous temporal phenomena over multiple temporal scales and (ii) digital processing of measured temporal signals in real time. We conclude by stating implications of the theory for modelling temporal phenomena in biological, perceptual, neural and memory processes by mathematical models, as well as implications regarding the philosophy of time and perceptual agents. Specifically, we propose that for A-type theories of time, as well as for perceptual agents, the notion of a non-infinitesimal inner temporal scale of the temporal receptive fields has to be included in representations of the present, where the inherent nonzero temporal delay of such time-causal receptive fields implies a need for incorporating predictions from the actual time-delayed present in the layers of a perceptual hierarchy, to make it possible for a representation of the perceptual present to constitute a representation of the environment with timing properties closer to the actual present.

The lateralized flash-lag illusion: A psychophysical and pupillometry study

The flash-lag illusion (FLI) is a visual phenomenon where a flashed object, either co-localized or in physical alignment with another continuously moving object, is perceived to lag behind the path of the moving object. In the present study, we reveal an anisotropy of the FLI between the lateral visual fields that was expressed psychophysically as different points of subjective equality, depending on the hemifield in which the stimuli appeared. Specifically, the study confirmed that, as seen in two previous studies, the FLI was significantly larger in the left visual field (LVF) than in the right (RVF). In addition, pupil dilations were larger in the RVF than in the LVF as well as returning to baseline levels more rapidly in the LVF. We interpret these findings as converging on revealing more efficient spatial and attentional processing and, in turn, extrapolation of motion in the LVF/right hemisphere than in the RVF/left hemisphere.

The Eye Pupil Adjusts to Illusorily Expanding Holes

Some static patterns evoke the perception of an illusory expanding central region or “hole.” We asked observers to rate the magnitudes of illusory motion or expansion of black holes, and these predicted the degree of dilation of the pupil, measured with an eye tracker. In contrast, when the “holes” were colored (including white), i.e., emitted light, these patterns constricted the pupils, but the subjective expansions were also weaker compared with the black holes. The change rates of pupil diameters were significantly related to the illusory motion phenomenology only with the black holes. These findings can be accounted for within a perceiving-the-present account of visual illusions, where both the illusory motion and the pupillary adjustments represent compensatory mechanisms to the perception of the next moment, based on shared experiences with the ecological regularities of light.

Predictive coding feedback results in perceived illusory contours in a recurrent neural network

Modern feedforward convolutional neural networks (CNNs) can now solve some computer vision tasks at super-human levels. However, these networks only roughly mimic human visual perception. One difference from human vision is that they do not appear to perceive illusory contours (e.g. Kanizsa squares) in the same way humans do. Physiological evidence from visual cortex suggests that the perception of illusory contours could involve feedback connections. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we equip a deep feedforward convolutional network with brain-inspired recurrent dynamics. The network was first pretrained with an unsupervised reconstruction objective on a natural image dataset, to expose it to natural object contour statistics. Then, a classification decision head was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar "illusory contour" configuration: inducer shapes oriented to form an illusory square. Compared with feedforward baselines, the iterative "predictive coding" feedback resulted in more illusory contours being classified as physical squares. The perception of the illusory contour was measurable in the luminance profile of the image reconstructions produced by the model, demonstrating that the model really "sees" the illusion. Ablation studies revealed that natural image pretraining and feedback error correction are both critical to the perception of the illusion. Finally we validated our conclusions in a deeper network (VGG): adding the same predictive coding feedback dynamics again leads to the perception of illusory contours.

The perceived present: What is it, and what is it there for?

It is proposed that the perceived present is not a moment in time, but an information structure comprising an integrated set of products of perceptual processing. All information in the perceived present carries an informational time marker identifying it as "present". This marker is exclusive to information in the perceived present. There are other kinds of time markers, such as ordinality ("this stimulus occurred before that one") and duration ("this stimulus lasted for 50 ms"). These are different from the "present" time marker and may be attached to information regardless of whether it is in the perceived present or not. It is proposed that the perceived present is a very short-term and very high-capacity holding area for perceptual information. The maximum holding time for any given piece of information is ~100 ms: This is affected by the need to balance the value of informational persistence for further processing against the problem of obsolescence of the information. The main function of the perceived present is to facilitate access by other specialized, automatic processes.

Symmetric Networks with Geometric Constraints as Models of Visual Illusions

Multistable illusions occur when the visual system interprets the same image in two different ways. We model illusions using dynamic systems based on Wilson networks, which detect combinations of levels of attributes of the image. In most examples presented here, the network has symmetry, which is vital to the analysis of the dynamics. We assume that the visual system has previously learned that certain combinations are geometrically consistent or inconsistent, and model this knowledge by adding suitable excitatory and inhibitory connections between attribute levels. We first discuss 4-node networks for the Necker cube and the rabbit/duck illusion. The main results analyze a more elaborate model for the Necker cube, a 16-node Wilson network whose nodes represent alternative orientations of specific segments of the image. Symmetric Hopf bifurcation is used to show that a small list of natural local geometric consistency conditions leads to alternation between two global percepts: cubes in two different orientations. The model also predicts brief transitional states in which the percept involves impossible rectangles analogous to the Penrose triangle. A tristable illusion generalizing the Necker cube is modelled in a similar manner.

In the interest of saving time: a critique of discrete perception

A recently proposed model of sensory processing suggests that perceptual experience is updated in discrete steps. We show that the data advanced to support discrete perception are in fact compatible with a continuous account of perception. Physiological and psychophysical constraints, moreover, as well as our awake-primate imaging data, imply that human neuronal networks cannot support discrete updates of perceptual content at the maximal update rates consistent with phenomenology. A more comprehensive approach to understanding the physiology of perception (and experience at large) is therefore called for, and we briefly outline our take on the problem.

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, 'Geometrical' and, in particular, 'Tilt Illusions' are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as 'Anchoring theory' and 'Perceptual grouping'.

Perceptual similarity and the neural correlates of geometrical illusions in human brain structure

Geometrical visual illusions are an intriguing phenomenon, in which subjective perception consistently misjudges the objective, physical properties of the visual stimulus. Prominent theoretical proposals have been advanced attempting to find common mechanisms across illusions. But empirically testing the similarity between illusions has been notoriously difficult because illusions have very different visual appearances. Here we overcome this difficulty by capitalizing on the variability of the illusory magnitude across participants. Fifty-nine healthy volunteers participated in the study that included measurement of individual illusion magnitude and structural MRI scanning. We tested the Muller-Lyer, Ebbinghaus, Ponzo, and vertical-horizontal geometrical illusions as well as a non-geometrical, contrast illusion. We found some degree of similarity in behavioral judgments of all tested geometrical illusions, but not between geometrical illusions and non-geometrical, contrast illusion. The highest similarity was found between Ebbinghaus and Muller-Lyer geometrical illusions. Furthermore, the magnitude of all geometrical illusions, and particularly the Ebbinghaus and Muller-Lyer illusions, correlated with local gray matter density in the parahippocampal cortex, but not in other brain areas. Our findings suggest that visuospatial integration and scene construction processes might partly mediate individual differences in geometric illusory perception. Overall, these findings contribute to a better understanding of the mechanisms behind geometrical illusions.

Monoaminergic Control of Cellular Glucose Utilization by Glycogenolysis in Neocortex and Hippocampus

Brainstem nuclei are the principal sites of monoamine (MA) innervation to major forebrain structures. In the cortical grey matter, increased secretion of MA neuromodulators occurs in response to a wealth of environmental and homeostatic challenges, whose onset is associated with rapid, preparatory changes in neural activity as well as with increases in energy metabolism. Blood-borne glucose is the main substrate for energy production in the brain. Once entered the tissue, interstitial glucose is equally accessible to neurons and astrocytes, the two cell types accounting for most of cellular volume and energy metabolism in neocortex and hippocampus. Astrocytes also store substantial amounts of glycogen, but non-stimulated glycogen turnover is very small. The rate of cellular glucose utilization in the brain is largely determined by hexokinase, which under basal conditions is more than 90 % inhibited by its product glucose-6-phosphate (Glc-6-P). During rapid increases in energy demand, glycogen is a primary candidate in modulating the intracellular level of Glc-6-P, which can occur only in astrocytes. Glycogenolysis can produce Glc-6-P at a rate higher than uptake and phosphorylation of glucose. MA neurotransmitter are released extrasinaptically by brainstem neurons projecting to neocortex and hippocampus, thus activating MA receptors located on both neuronal and astrocytic plasma membrane. Importantly, MAs are glycogenolytic agents and thus they are exquisitely suitable for regulation of astrocytic Glc-6-P concentration, upstream substrate flow through hexokinase and hence cellular glucose uptake. Conforming to such mechanism, Gerald A. Dienel and Nancy F. Cruz recently suggested that activation of noradrenergic locus coeruleus might reversibly block astrocytic glucose uptake by stimulating glycogenolysis in these cells, thereby anticipating the rise in glucose need by active neurons. In this paper, we further develop the idea that the whole monoaminergic system modulates both function and metabolism of forebrain regions in a manner mediated by glycogen mobilization in astrocytes.

Length and orientation constancy learning in 2-dimensions with auditory sensory substitution: the importance of self-initiated movement

A subset of sensory substitution (SS) devices translate images into sounds in real time using a portable computer, camera, and headphones. Perceptual constancy is the key to understanding both functional and phenomenological aspects of perception with SS. In particular, constancies enable object externalization, which is critical to the performance of daily tasks such as obstacle avoidance and locating dropped objects. In order to improve daily task performance by the blind, and determine if constancies can be learned with SS, we trained blind (N = 4) and sighted (N = 10) individuals on length and orientation constancy tasks for 8 days at about 1 h per day with an auditory SS device. We found that blind and sighted performance at the constancy tasks significantly improved, and attained constancy performance that was above chance. Furthermore, dynamic interactions with stimuli were critical to constancy learning with the SS device. In particular, improved task learning significantly correlated with the number of spontaneous left-right head-tilting movements while learning length constancy. The improvement from previous head-tilting trials even transferred to a no-head-tilt condition. Therefore, not only can SS learning be improved by encouraging head movement while learning, but head movement may also play an important role in learning constancies in the sighted. In addition, the learning of constancies by the blind and sighted with SS provides evidence that SS may be able to restore vision-like functionality to the blind in daily tasks.

Activity of somatosensory-responsive neurons in high subdivisions of SI cortex during locomotion

Responses of neurons in the primary somatosensory cortex during movements are poorly understood, even during such simple tasks as walking on a flat surface. In this study, we analyzed spike discharges of neurons in the rostral bank of the ansate sulcus (areas 1-2) in 2 cats while the cats walked on a flat surface or on a horizontal ladder, a complex task requiring accurate stepping. All neurons (n = 82) that had receptive fields (RFs) on the contralateral forelimb exhibited frequency modulation of their activity that was phase locked to the stride cycle during simple locomotion. Neurons with proximal RFs (upper arm/shoulder) and pyramidal tract-projecting neurons (PTNs) with fast-conducting axons tended to fire at peak rates in the middle of the swing phase, whereas neurons with RFs on the distal limb (wrist/paw) and slow-conducting PTNs typically showed peak firing at the transition between swing and stance phases. Eleven of 12 neurons with tactile RFs on the volar forepaw began firing toward the end of swing, with peak activity occurring at the moment of foot contact with floor, thereby preceding the evoked sensory volley from touch receptors. Requirement to step accurately on the ladder affected 91% of the neurons, suggesting their involvement in control of accuracy of stepping. During both tasks, neurons exhibited a wide variety of spike distributions within the stride cycle, suggesting that, during either simple or ladder locomotion, they represent the cycling somatosensory events in their activity both predictively before and reflectively after these events take place.

Active inference, eye movements and oculomotor delays

This paper considers the problem of sensorimotor delays in the optimal control of (smooth) eye movements under uncertainty. Specifically, we consider delays in the visuo-oculomotor loop and their implications for active inference. Active inference uses a generalisation of Kalman filtering to provide Bayes optimal estimates of hidden states and action in generalised coordinates of motion. Representing hidden states in generalised coordinates provides a simple way of compensating for both sensory and oculomotor delays. The efficacy of this scheme is illustrated using neuronal simulations of pursuit initiation responses, with and without compensation. We then consider an extension of the generative model to simulate smooth pursuit eye movements-in which the visuo-oculomotor system believes both the target and its centre of gaze are attracted to a (hidden) point moving in the visual field. Finally, the generative model is equipped with a hierarchical structure, so that it can recognise and remember unseen (occluded) trajectories and emit anticipatory responses. These simulations speak to a straightforward and neurobiologically plausible solution to the generic problem of integrating information from different sources with different temporal delays and the particular difficulties encountered when a system-like the oculomotor system-tries to control its environment with delayed signals.

Postdiction: its implications on visual awareness, hindsight, and sense of agency

There are a few postdictive perceptual phenomena known, in which a stimulus presented later seems causally to affect the percept of another stimulus presented earlier. While backward masking provides a classical example, the flash lag effect stimulates theorists with a variety of intriguing findings. The TMS-triggered scotoma together with “backward filling-in” of it offer a unique neuroscientific case. Findings suggest that various visual attributes are reorganized in a postdictive fashion to be consistent with each other, or to be consistent in a causality framework. In terms of the underlying mechanisms, four prototypical models have been considered: the “catch up,” the “reentry,” the “different pathway” and the “memory revision” models. By extending the list of postdictive phenomena to memory, sensory-motor and higher-level cognition, one may note that such a postdictive reconstruction may be a general principle of neural computation, ranging from milliseconds to months in a time scale, from local neuronal interactions to long-range connectivity, in the complex brain. The operational definition of the “postdictive phenomenon” can be applicable to such a wide range of sensory/cognitive effects across a wide range of time scale, even though the underlying neural mechanisms may vary across them. This has significant implications in interpreting “free will” and “sense of agency” in functional, psychophysical and neuroscientific terms.

Spatial warping by oriented line detectors can counteract neural delays

The slow speed of neural transmission necessitates that cortical visual information from dynamic scenes will lag reality. The “perceiving the present” (PTP) hypothesis suggests that the visual system can mitigate the effect of such delays by spatially warping scenes to look as they will in ~100 ms from now (Changizi, 2001). We here show that the Hering illusion, in which straight lines appear bowed, can be induced by a background of optic flow, consistent with the PTP hypothesis. However, importantly, the bowing direction is the same whether the flow is inward or outward. This suggests that if the warping is meant to counteract latencies, it is accomplished by a simple strategy that is insensitive to motion direction, and that works only under typical (forward-moving) circumstances. We also find that the illusion strengthens with longer pulses of optic flow, demonstrating motion integration over ~80 ms. The illusion is identical whether optic flow precedes or follows the flashing of bars, exposing the spatial warping to be equally postdictive and predictive, i.e., peri-dictive. Additionally, the illusion is diminished by cues which suggest the bars are independent of the background movement. Collectively, our findings are consistent with a role for networks of visual orientation-tuned neurons (e.g., simple cells in primary visual cortex) in spatial warping. We conclude that under the common condition of forward ego-motion, spatial warping counteracts the disadvantage of neural latencies. It is not possible to prove that this is the purpose of spatial warping, but our findings at minimum place constraints on the PTP hypothesis, demonstrating that any spatial warping for the purpose of counteracting neural delays is not a precise, on-the-fly computation, but instead a heuristic achieved by a simple mechanism that succeeds under normal circumstances.

Bright illusions reduce the eye's pupil

We recorded by use of an infrared eye-tracker the pupil diameters of participants while they observed visual illusions of lightness or brightness. Four original illusions {based on Gaetano Kanisza's [Kanizsa G (1976) Subjective contours. Sci Am 234:48-52] and Akiyoshi Kitaoka's [Kitaoka A. (2005) Trick Eyes (Barnes & Noble, New Providence, NJ).] examples} were manipulated to obtain control conditions in which the perceived illusory luminance was either eliminated or reduced. All stimuli were equiluminant so that constrictions in pupillary size could not be ascribed to changes in light energy. We found that the pupillary diameter rapidly varied according to perceived brightness and lightness strength. Differences in local contrast information could be ruled out as an explanation because, in a second experiment, the observers maintained eye fixation in the center of the display; thus, differential stimulation of the fovea by local contrast changes could not be responsible for the pupillary differences. Hence, the most parsimonious explanation for the present findings is that pupillary responses to ambient light reflect the perceived brightness or lightness of the scene and not simply the amount of physical light energy entering the eye. Thus, the pupillary physiological response reflects the subjective perception of light and supports the idea that the brain's visual circuitry is shaped by visual experience with images and their possible sources.

A computational intelligibility model for assessment and compression of American sign language video

Real-time, two-way transmission of American Sign Language (ASL) video over cellular networks provides natural communication among members of the Deaf community. As a communication tool, compressed ASL video must be evaluated according to the intelligibility of the conversation, not according to conventional definitions of video quality. Guided by linguistic principles and human perception of ASL, this paper proposes a full-reference computational model of intelligibility for ASL (CIM-ASL) that is suitable for evaluating compressed ASL video. The CIM-ASL measures distortions only in regions relevant for ASL communication, using spatial and temporal pooling mechanisms that vary the contribution of distortions according to their relative impact on the intelligibility of the compressed video. The model is trained and evaluating using ground truth experimental data collected in three separate studies. The CIM-ASL provides accurate estimates of subjective intelligibility and demonstrates statistically significant improvements over computational models traditionally used to estimate video quality. The CIM-ASL is incorporated into an H.264 compliant video coding framework, creating a closed-loop encoding system optimized explicitly for ASL intelligibility. The ASL-optimized encoder achieves bitrate reductions between 10% and 42%, without reducing intelligibility, when compared to a general purpose H.264 encoder.

Perceiving the present: systematization of illusions or illusion of systematization?

Mark Changizi et al. (2008) claim that it is possible systematically to organize more than 50 kinds of illusions in a 7 × 4 matrix of 28 classes. This systematization, they further maintain, can be explained by the operation of a single visual processing latency correction mechanism that they call "perceiving the present" (PTP). This brief report raises some concerns about the way a number of illusions are classified by the proposed systematization. It also poses two general problems-one empirical and one conceptual-for the PTP approach.

Response to briscoe (2010)

In an earlier paper my colleagues and I put forth a theory called "perceiving-the-present" that predicts a systematic pattern across a large variety of illusions, and we presented evidence that the systematic pattern exists. Briscoe puts forth arguments against the theory and the existence of the systematic pattern. Here I provide counterarguments to his criticisms of the theory, and I explain why his arguments do not bear on the existence of the systematic pattern.

Illusory force perception following a voluntary limb movement

We present a novel illusion in which participants report constant forces on their hand as steadily increasing. Participants made discrete reaching movements perturbed by a lateral force that increased with the distance moved; when stationary at the end of the movement, a true constant force was perceived to increase. We tested perceived subjective equality by increasing or decreasing the force. The illusion was significantly stronger when the perturbation was applied during active movement. We conclude that the unusual context of moving against lateral spring forces results in participants failing to predict steady lateral forces at the end of their movement, and causes an illusion of increasing forces even after movement termination. This result further emphasizes the role of action prediction in sensory perception.