Natural-scene geometry predicts the perception of angles and line orientation

Complexity of mental geometry for 3D pose perception

Biological visual systems rely on pose estimation of 3D objects to navigate and interact with their environment, but the neural mechanisms and computations for inferring 3D poses from 2D retinal images are only partially understood, especially where stereo information is missing. We previously presented evidence that humans infer the poses of 3D objects lying centered on the ground by using the geometrical back-transform from retinal images to viewer-centered world coordinates. This model explained the almost veridical estimation of poses in real scenes and the illusory rotation of poses in obliquely viewed pictures, which includes the "pointing out of the picture" phenomenon. Here we test this model for more varied configurations and find that it needs to be augmented. Five observers estimated poses of sloped, elevated, or off-center 3D sticks in each of 16 different poses displayed on a monitor in frontal and oblique views. Pose estimates in scenes and pictures showed remarkable accuracy and agreement between observers, but with a systematic fronto-parallel bias for oblique poses similar to the ground condition. The retinal projection of the pose of an object sloped wrt the ground depends on the slope. We show that observers' estimates can be explained by the back-transform derived for close to the correct slope. The back-transform explanation also applies to obliquely viewed pictures and to off-center objects and elevated objects, making it more likely that observers use internalized perspective geometry to make 3D pose inferences while actively incorporating inferences about other aspects of object placement.

Response to geometrical visual illusions in non-human animals: a meta-analysis

Visual illusions have been studied in many non-human species, spanning a wide range of biological and methodological variables. While early reviews have proved useful in providing an overview of the field, they have not been accompanied by quantitative analysis to systematically evaluate the contribution of biological and methodological moderators on the proportion of illusory choice. In the current meta-analytical study, we confirm that geometrical visual illusion perception is a general phenomenon among non-human animals. Additionally, we found that studies testing birds report stronger illusion perception compared to other classes, as do those on animals with lateral-positioned eyes compared to animals with forward-facing eyes. In terms of methodological choices, we found a positive correlation between the number of trials during training or testing and the effect sizes, while studies with larger samples report smaller effect sizes. Despite studies that trained animals with artificial stimuli showing larger effect sizes compared with those using spontaneous testing with naturalistic stimuli, like food, we found more recent studies prefer spontaneous choice over training. We discuss the challenges and bottlenecks in this area of study, which, if addressed, could lead to more successful advances in the future.

Analogue magnitude representation of angles and its relation to geometric expertise

The distance effect (comparing objects becomes easier with increasing differences in their magnitude) is observed in tasks ranging across domains, and its existence has been interpreted as evidence for analogue magnitude representation. Similarly, associations between response side and magnitude (faster left/right-sided responses to small/large objects, respectively) are observed across domains. We investigated the analogue processing of angles and the association between angle magnitude and response side in relation to geometric expertise. We compared the behavioural pattern of two groups-architects and controls-in a direct angle magnitude classification task (i.e., judge whether a presented angle was greater or less than 90°) and in an indirect task (i.e., judge whether an angle was drawn with a dashed or continuous line). We found a robust distance effect for reaction times and accuracy at the whole sample level and in each group separately. Architects revealed a smaller distance effect for accuracy than controls. This could be interpreted as an argument for a more precise analogue representation of angles in experts compared to non-experts. However, we did not find evidence for an association between angle magnitude and response side in any group.

The box-circle illusion

A novel geometrical optical illusion is reported in this article: the horizontal distances of the contextual structures distort the perceived vertical positions of observed objects. Specifically, the illusion manifests in the form of connected boxes of varying widths but equal heights, each containing a circle at the center. Despite identical vertical positioning of the circles, they appear misaligned. The illusion diminishes when the boxes are removed. Potential underlying mechanisms are discussed.

EXPRESS: Perceptual scale expansion: A natural design for improving the precision of motor control

Space perception is systematically biased. Few theories of spatial bias address the possible functional advantages of mechanisms that produce spatial biases. The scale expansion hypothesis proposes that many spatial biases are due to the perceptual expansion of visual angles, which acts somewhat like a natural magnifying glass in vision. The present study examined the idea that visual expansion may improve motor precision (i.e., reduce motor variability) in movements when using closed-loop control but not when using open-loop control. Experiment 1 tested this idea in an online tracking task (closed-loop control), whereas Experiment 2 tested it in a fast-hitting task (open-loop control). The results were consistent with the hypothesis. To rule out the effect of the task difference (i.e., tracking vs. fast hitting), Experiment 3 examined the effect of visual expansion on the variability of motor performance in a line-reproduction task. The control type (closed-loop or open-loop) was manipulated by the form of visual feedback (online or offline). The results were again consistent with the present assumption. Taken together, the present data suggest that perceptual expansion in vision improves motor control precision when using closed-loop control (but not when using open-loop control), which supports the scale-expansion hypothesis. In addition, the present findings also improve our understanding of how visual error amplification affects motor control.

Geometry intuitions without vision? A study in blind children and adults

Geometry intuitions seem to be rooted in a non-verbal system that humans possess since early age. However, the mechanisms underlying the comprehension of basic geometric concepts remain elusive. Some authors have suggested that the starting point of geometry development could be found in the visual perception of specific features in our environment, thus conferring to vision a foundational role in the acquisition of geometric skills. To examine this assumption, a test probing intuitive understanding of basic geometric concepts was presented to congenitally blind children and adults. Participants had to detect the intruder among four different shapes, from which three instantiated a specific geometrical concept and one (the intruder) violated it. Although they performed above the chance level, the blind presented poorer performance than the sighted participants who did the task in the visual modality (i.e., with the eyes open), but performed equally well than the sighted who did the task in the tactile modality (i.e., with a blindfold). We therefore provide evidence that geometric abilities are impacted by the lack of vision.

Local biases drive, but do not determine, the perception of illusory trajectories

When a dot moves horizontally across a set of tilted lines of alternating orientations, the dot appears to be moving up and down along its trajectory. This perceptual phenomenon, known as the slalom illusion, reveals a mismatch between the veridical motion signals and the subjective percept of the motion trajectory, which has not been comprehensively explained. In the present study, we investigated the empirical boundaries of the slalom illusion using psychophysical methods. The phenomenon was found to occur both under conditions of smooth pursuit eye movements and constant fixation, and to be consistently amplified by intermittently occluding the dot trajectory. When the motion direction of the dot was not constant, however, the stimulus display did not elicit the expected illusory percept. These findings confirm that a local bias towards perpendicularity at the intersection points between the dot trajectory and the tilted lines cause the illusion, but also highlight that higher-level cortical processes are involved in interpreting and amplifying the biased local motion signals into a global illusion of trajectory perception.

Mental geometry of perceiving 3D size in pictures

We show that the classical problem of three-dimensional (3D) size perception in obliquely viewed pictures can be understood by comparing human performance to the optimal geometric solution. A photograph seen from the camera position, can form the same retinal projection as the physical 3D scene, but retinal projections of sizes and shapes are distorted in oblique viewing. For real scenes, we previously showed that size and shape inconstancy result despite observers using the correct geometric back-transform, because some retinal images evoke misestimates of object slant or viewing elevation. Now, we examine how observers estimate 3D sizes in oblique views of pictures of objects lying on the ground in different poses. Compared to estimates for real scenes, in oblique views of pictures, sizes were seriously underestimated for objects at frontoparallel poses, but there was almost no change for objects perceived as pointing toward the viewer. The inverse of the function relating projected length to pose, camera elevation and viewing azimuth, gives the optimal correction factor for inferring correct 3D lengths if the elevation and azimuth are estimated accurately. Empirical correction functions had similar shapes to optimal, but lower amplitude. Measurements revealed that observers systematically underestimated viewing azimuth, similar to the frontoparallel bias for object pose perception. A model that adds underestimation of viewing azimuth to the geometrical back-transform, provided good fits to estimated 3D lengths from oblique views. These results add to accumulating evidence that observers use internalized projective geometry to perceive sizes, shapes, and poses in 3D scenes and their pictures.

The uphill battle for action-specific perception

The action-specific account of perception states that a perceiver's ability to act influences the perception of the environment. For example, participants tend to perceive distances as farther when presented up hills than on the flat ground. This tendency is known as the distance-on-hill effect. However, there is debate as to whether these types of effects are truly perceptual. Critics of the action-specific account of perception claim that the effects could be due to participants guessing the hypothesis and trying to comply with the experimental demands. The present study aims to explore the distance-on-hill effect and determine whether it is truly perceptual or whether past results were due to response bias. Participants judged the relative distance to targets on a hill and the flat ground. We found the distance-on-hill effect in virtual reality using a visual matching task. The distance-on-hill effect persisted even when participants were given explicit feedback about their estimates. We also found that the effect went away, as predicted by a perceptual explanation, when participants had to match the distance between two cones that were both on hills. These results offer important steps toward the painstaking task of determining whether action's effect on perception is truly perceptual.

Picture perception reveals mental geometry of 3D scene inferences

Pose estimation of objects in real scenes is critically important for biological and machine visual systems, but little is known of how humans infer 3D poses from 2D retinal images. We show unexpectedly remarkable agreement in the 3D poses different observers estimate from pictures. We further show that all observers apply the same inferential rule from all viewpoints, utilizing the geometrically derived back-transform from retinal images to actual 3D scenes. Pose estimations are altered by a fronto-parallel bias, and by image distortions that appear to tilt the ground plane. We used pictures of single sticks or pairs of joined sticks taken from different camera angles. Observers viewed these from five directions, and matched the perceived pose of each stick by rotating an arrow on a horizontal touchscreen. The projection of each 3D stick to the 2D picture, and then onto the retina, is described by an invertible trigonometric expression. The inverted expression yields the back-projection for each object pose, camera elevation, and observer viewpoint. We show that a model that uses the back-projection, modulated by just two free parameters, explains 560 pose estimates per observer. By considering changes in retinal image orientations due to position and elevation of limbs, the model also explains perceived limb poses in a complex scene of two bodies lying on the ground. The inferential rules simply explain both perceptual invariance and dramatic distortions in poses of real and pictured objects, and show the benefits of incorporating projective geometry of light into mental inferences about 3D scenes.

Interactions between visual working memory representations

We investigated whether the representations of different objects are maintained independently in working memory or interact with each other. Observers were shown two sequentially presented orientations and required to reproduce each orientation after a delay. The sequential presentation minimized perceptual interactions so that we could isolate interactions between memory representations per se. We found that similar orientations were repelled from each other whereas dissimilar orientations were attracted to each other. In addition, when one of the items was given greater attentional priority by means of a cue, the representation of the high-priority item was not influenced very much by the orientation of the low-priority item, but the representation of the low-priority item was strongly influenced by the orientation of the high-priority item. This indicates that attention modulates the interactions between working memory representations. In addition, errors in the reported orientations of the two objects were positively correlated under some conditions, suggesting that representations of distinct objects may become grouped together in memory. Together, these results demonstrate that working-memory representations are not independent but instead interact with each other in a manner that depends on attentional priority.

On the role of visual experience in mathematical development: Evidence from blind mathematicians

Advanced mathematical reasoning, regardless of domain or difficulty, activates a reproducible set of bilateral brain areas including intraparietal, inferior temporal and dorsal prefrontal cortex. The respective roles of genetics, experience and education in the development of this math-responsive network, however, remain unresolved. Here, we investigate the role of visual experience by studying the exceptional case of three professional mathematicians who were blind from birth (n = 1) or became blind during childhood (n = 2). Subjects were scanned with fMRI while they judged the truth value of spoken mathematical and nonmathematical statements. Blind mathematicians activated the classical network of math-related areas during mathematical reflection, similar to that found in a group of sighted professional mathematicians. Thus, brain networks for advanced mathematical reasoning can develop in the absence of visual experience. Additional activations were found in occipital cortex, even in individuals who became blind during childhood, suggesting that either mental imagery or a more radical repurposing of visual cortex may occur in blind mathematicians.

Event-related brain potentials during the visuomotor mental rotation task: The contingent negative variation scales to angle of rotation

Perceptual judgments about the angular disparity of a character from its standard upright (i.e., mental rotation task) result in a concurrent increase in reaction time (RT) and modulation of the amplitude of the P300 event-related brain potential (ERP). It has therefore been proposed that the P300 represents the neural processes associated with a visual rotation. In turn, the visuomotor mental rotation (VMR) task requires reaching to a location that deviates from a target by a predetermined angle. Although the VMR task exhibits a linear increase in RT with increasing oblique angles of rotation, work has not examined whether the task is supported via a visual rotation analogous to its mental rotation task counterpart. This represents a notable issue because seminal work involving non-human primates has ascribed VMR performance to the motor-related rotation of directionally tuned neurons in the primary motor cortex. Here we examined the concurrent behavioral and ERP characteristics of a standard reaching task and VMR tasks of 35°, 70°, and 105° of rotation. Results showed that the P300 amplitude was larger for the standard compared to each VMR task--an effect independent of the angle of rotation. In turn, the amplitude of the contingent negative variation (CNV)--an ERP related to cognitive and visuomotor integration for movement preparation--was systematically modulated with angle of rotation. Thus, we propose that the CNV represents an ERP correlate related to the cognitive and/or visuomotor transformation demands of increasing the angular separation between a stimulus and a movement goal.

Neural extrapolation of motion for a ball rolling down an inclined plane

It is known that humans tend to misjudge the kinematics of a target rolling down an inclined plane. Because visuomotor responses are often more accurate and less prone to perceptual illusions than cognitive judgments, we asked the question of how rolling motion is extrapolated for manual interception or drawing tasks. In three experiments a ball rolled down an incline with kinematics that differed as a function of the starting position (4 different positions) and slope (30°, 45° or 60°). In Experiment 1, participants had to punch the ball as it fell off the incline. In Experiment 2, the ball rolled down the incline but was stopped at the end; participants were asked to imagine that the ball kept moving and to punch it. In Experiment 3, the ball rolled down the incline and was stopped at the end; participants were asked to draw with the hand in air the trajectory that would be described by the ball if it kept moving. We found that performance was most accurate when motion of the ball was visible until interception and haptic feedback of hand-ball contact was available (Experiment 1). However, even when participants punched an imaginary moving ball (Experiment 2) or drew in air the imaginary trajectory (Experiment 3), they were able to extrapolate to some extent global aspects of the target motion, including its path, speed and arrival time. We argue that the path and kinematics of a ball rolling down an incline can be extrapolated surprisingly well by the brain using both visual information and internal models of target motion.

Generic properties of curvature sensing through vision and touch

Generic properties of curvature representations formed on the basis of vision and touch were examined as a function of mathematical properties of curved objects. Virtual representations of the curves were shown on a computer screen for visual scaling by sighted observers (experiment 1). Their physical counterparts were placed in the two hands of blindfolded and congenitally blind observers for tactile scaling. The psychophysical data show that curvature representations in congenitally blind individuals, who never had any visual experience, and in sighted observers, who rely on vision most of the time, are statistically linked to the same mathematical properties of the curves. The perceived magnitude of object curvature, sensed through either vision or touch, is related by a mathematical power law, with similar exponents for the two sensory modalities, to the aspect ratio of the curves, a scale invariant geometric property. This finding supports biologically motivated models of sensory integration suggesting a universal power law for the adaptive brain control and balance of motor responses to environmental stimuli from any sensory modality.

The Müller-Lyer illusion in ant foraging

The Müller-Lyer illusion is a classical geometric illusion in which the apparent (perceived) length of a line depends on whether the line terminates in an arrow tail or arrowhead. This effect may be caused by economic compensation for the gap between the physical stimulus and visual fields. Here, we show that the Müller-Lyer illusion can also be produced by the foraging patterns of garden ants (Lasius niger) and that the pattern obtained can be explained by a simple, asynchronously updated foraging ant model. Our results suggest that the geometric illusion may be a byproduct of the foraging process, in which local interactions underlying efficient exploitation can also give rise to global exploration, and that visual information processing in human could implement similar modulation between local efficient processing and widespread computation.

Spatial warping by oriented line detectors can counteract neural delays

The slow speed of neural transmission necessitates that cortical visual information from dynamic scenes will lag reality. The “perceiving the present” (PTP) hypothesis suggests that the visual system can mitigate the effect of such delays by spatially warping scenes to look as they will in ~100 ms from now (Changizi, 2001). We here show that the Hering illusion, in which straight lines appear bowed, can be induced by a background of optic flow, consistent with the PTP hypothesis. However, importantly, the bowing direction is the same whether the flow is inward or outward. This suggests that if the warping is meant to counteract latencies, it is accomplished by a simple strategy that is insensitive to motion direction, and that works only under typical (forward-moving) circumstances. We also find that the illusion strengthens with longer pulses of optic flow, demonstrating motion integration over ~80 ms. The illusion is identical whether optic flow precedes or follows the flashing of bars, exposing the spatial warping to be equally postdictive and predictive, i.e., peri-dictive. Additionally, the illusion is diminished by cues which suggest the bars are independent of the background movement. Collectively, our findings are consistent with a role for networks of visual orientation-tuned neurons (e.g., simple cells in primary visual cortex) in spatial warping. We conclude that under the common condition of forward ego-motion, spatial warping counteracts the disadvantage of neural latencies. It is not possible to prove that this is the purpose of spatial warping, but our findings at minimum place constraints on the PTP hypothesis, demonstrating that any spatial warping for the purpose of counteracting neural delays is not a precise, on-the-fly computation, but instead a heuristic achieved by a simple mechanism that succeeds under normal circumstances.

Depth compression based on mis-scaling of binocular disparity may contribute to angular expansion in perceived optical slant

Three studies, involving a total of 145 observers examined quantitative theories of the overestimation of perceived optical slant. The first two studies investigated the depth/width anisotropies on positive and negative slant in both pitch and yaw at 2 and 8 m using calibrated immersive virtual environments. Observers made judgments of the relative lengths of extents that were frontal with those that were in depth. The physical aspect ratio that was perceived as 1:1 was determined for each slant. The observed anisotropies can be modeled by assuming overestimation in perceived slant. Three one-parameter slant perception models (angular expansion, affine depth compression caused by mis-scaling of binocular disparity, and intrinsic bias) were compared. The angular expansion and the affine depth compression models provided significantly better fits to the aspect ratio data than the intrinsic bias model did. The affine model required depth compression at the 2 m distance; however, that was much more than the depth compression measured directly in the third study using the same apparatus. The present results suggest that depth compression based on mis-scaling of binocular disparity may contribute to slant overestimation, especially as viewing distance increases, but also suggest that a functional rather than mechanistic account may be more appropriate for explaining biases in perceived slant in near space.

"Seeing" and "feeling" architecture: how bodily self-consciousness alters architectonic experience and affects the perception of interiors

Over the centuries architectural theory evolved several notions of embodiment, proposing in the nineteenth and twentieth century that architectonic experience is related to physiological responses of the observer. Recent advances in the cognitive neuroscience of embodiment (or bodily self-consciousness) enable empirical studies of architectonic embodiment. Here, we investigated how architecture modulates bodily self-consciousness by adapting a video-based virtual reality (VR) setup previously used to investigate visuo-tactile mechanisms of bodily self-consciousness. While standing in two different interiors, participants were filmed from behind and watched their own virtual body online on a head-mounted display (HMD). Visuo-tactile strokes were applied in synchronous or asynchronous mode to the participants and their virtual body. Two interiors were simulated in the laboratory by placing the sidewalls either far or near from the participants, generating a large and narrow room. We tested if bodily self-consciousness was differently modulated when participants were exposed to both rooms and whether these changes depend on visuo-tactile stimulation. We measured illusory touch, self-identification, and performed length estimations. Our data show that synchronous stroking of the physical and the virtual body induces illusory touch and self-identification with the virtual body, independent of room-size. Moreover, in the narrow room we observed weak feelings of illusory touch with the sidewalls and of approaching walls. These subjective changes were complemented by a stroking-dependent modulation of length estimation only in the narrow room with participants judging the room-size more accurately during conditions of illusory self-identification. We discuss our findings and previous notions of architectonic embodiment in the context of the cognitive neuroscience of bodily self-consciousness and propose an empirical framework grounded in architecture, cognitive neuroscience, and VR.

An Empirical Model of Slope Ratio Comparisons

Comparing slopes is a fundamental graph reading task and the aspect ratio chosen for a plot influences how easy these comparisons are to make. According to Banking to 45°, a classic design guideline first proposed and studied by Cleveland et al., aspect ratios that center slopes around 45° minimize errors in visual judgments of slope ratios. This paper revisits this earlier work. Through exploratory pilot studies that expand Cleveland et al.'s experimental design, we develop an empirical model of slope ratio estimation that fits more extreme slope ratio judgments and two common slope ratio estimation strategies. We then run two experiments to validate our model. In the first, we show that our model fits more generally than the one proposed by Cleveland et al. and we find that, in general, slope ratio errors are not minimized around 45°. In the second experiment, we explore a novel hypothesis raised by our model: that visible baselines can substantially mitigate errors made in slope judgments. We conclude with an application of our model to aspect ratio selection.

Relative luminance and binocular disparity preferences are correlated in macaque primary visual cortex, matching natural scene statistics

Humans excel at inferring information about 3D scenes from their 2D images projected on the retinas, using a wide range of depth cues. One example of such inference is the tendency for observers to perceive lighter image regions as closer. This psychophysical behavior could have an ecological basis because nearer regions tend to be lighter in natural 3D scenes. Here, we show that an analogous association exists between the relative luminance and binocular disparity preferences of neurons in macaque primary visual cortex. The joint coding of relative luminance and binocular disparity at the neuronal population level may be an integral part of the neural mechanisms for perceptual inference of depth from images.

Cognitive architecture of perceptual organization: from neurons to gnosons

What, if anything, is cognitive architecture and how is it implemented in neural architecture? Focusing on perceptual organization, this question is addressed by way of a pluralist approach which, supported by metatheoretical considerations, combines complementary insights from representational, connectionist, and dynamic systems approaches to cognition. This pluralist approach starts from a representationally inspired model which implements the intertwined but functionally distinguishable subprocesses of feedforward feature encoding, horizontal feature binding, and recurrent feature selection. As sustained by a review of neuroscientific evidence, these are the subprocesses that are believed to take place in the visual hierarchy in the brain. Furthermore, the model employs a special form of processing, called transparallel processing, whose neural signature is proposed to be gamma-band synchronization in transient horizontal neural assemblies. In neuroscience, such assemblies are believed to mediate binding of similar features. Their formal counterparts in the model are special input-dependent distributed representations, called hyperstrings, which allow many similar features to be processed in a transparallel fashion, that is, simultaneously as if only one feature were concerned. This form of processing does justice to both the high combinatorial capacity and the high speed of the perceptual organization process. A naturally following proposal is that those temporarily synchronized neural assemblies are "gnosons", that is, constituents of flexible self-organizing cognitive architecture in between the relatively rigid level of neurons and the still elusive level of consciousness.

Perceptual scale expansion: an efficient angular coding strategy for locomotor space

Whereas most sensory information is coded on a logarithmic scale, linear expansion of a limited range may provide a more efficient coding for the angular variables important to precise motor control. In four experiments, we show that the perceived declination of gaze, like the perceived orientation of surfaces, is coded on a distorted scale. The distortion seems to arise from a nearly linear expansion of the angular range close to horizontal/straight ahead and is evident in explicit verbal and nonverbal measures (Experiments 1 and 2), as well as in implicit measures of perceived gaze direction (Experiment 4). The theory is advanced that this scale expansion (by a factor of about 1.5) may serve a functional goal of coding efficiency for angular perceptual variables. The scale expansion of perceived gaze declination is accompanied by a corresponding expansion of perceived optical slants in the same range (Experiments 3 and 4). These dual distortions can account for the explicit misperception of distance typically obtained by direct report and exocentric matching, while allowing for accurate spatial action to be understood as the result of calibration.

Manual matching of perceived surface orientation is affected by arm posture: evidence of calibration between proprioception and visual experience in near space

Proprioception of hand orientation (orientation production using the hand) is compared with manual matching of visual orientation (visual surface matching using the hand) in two experiments. In experiment 1, using self-selected arm postures, the proportions of wrist and elbow flexion spontaneously used to orient the pitch of the hand (20 and 80%, respectively) are relatively similar across both manual matching tasks and manual orientation production tasks for most participants. Proprioceptive error closely matched perceptual biases previously reported for visual orientation perception, suggesting calibration of proprioception to visual biases. A minority of participants, who attempted to use primarily wrist flexion while holding the forearm horizontal, performed poorly at the manual matching task, consistent with proprioceptive error caused by biomechanical constraints of their self-selected posture. In experiment 2, postural choices were constrained to primarily wrist or elbow flexion without imposing biomechanical constraints (using a raised forearm). Identical relative offsets were found between the two constraint groups in manual matching and manual orientation production. The results support two claims: (1) manual orientation matching to visual surfaces is based on manual proprioception and (2) calibration between visual and proprioceptive experiences guarantees relatively accurate manual matching for surfaces within reach, despite systematic visual biases in perceived surface orientation.

Understanding vision in wholly empirical terms

This article considers visual perception, the nature of the information on which perceptions seem to be based, and the implications of a wholly empirical concept of perception and sensory processing for vision science. Evidence from studies of lightness, brightness, color, form, and motion all indicate that, because the visual system cannot access the physical world by means of retinal light patterns as such, what we see cannot and does not represent the actual properties of objects or images. The phenomenology of visual perceptions can be explained, however, in terms of empirical associations that link images whose meanings are inherently undetermined to their behavioral significance. Vision in these terms requires fundamentally different concepts of what we see, why, and how the visual system operates.

The perception of 2D orientation is categorically biased

Three experimental paradigms were used to investigate the perception of orientation relative to internal categorical standards of vertical and horizontal. In Experiment 1, magnitude estimation of orientation (in degrees) relative to vertical and horizontal replicated a previously reported spatial orientation bias also measured using verbal report: Orientations appear farther from horizontal than they are, whether numeric judgments are made relative to vertical or to horizontal. Analyses of verbal response patterns, however, suggested that verbal reports underestimate the true spatial bias. A non-verbal orientation bisection task (Experiment 2) confirmed that spatial errors are not due to numeric coding and are larger than the 6° error replicated using verbal methods. A spatial error of 8.6° was found in the bisection task, such that an orientation of about 36.4° from horizontal appears equidistant from vertical and horizontal. Finally, using a categorization ("ABX") paradigm in Experiment 3, it was found that there is less memory confusability for orientations near horizontal than for orientations near vertical. Thus, three different types of measures, two of them non-verbal, provide converging evidence that the coding of orientation relative to the internal standards of horizontal and vertical is asymmetrically biased and that horizontal appears to be the privileged axis.

Exact feature probabilities in images with occlusion

To understand the computations of our visual system, it is important to understand also the natural environment it evolved to interpret. Unfortunately, existing models of the visual environment are either unrealistic or too complex for mathematical description. Here we describe a naturalistic image model and present a mathematical solution for the statistical relationships between the image features and model variables. The world described by this model is composed of independent, opaque, textured objects, which occlude each other. This simple structure allows us to calculate the joint probability distribution of image values sampled at multiple arbitrarily located points, without approximation. This result can be converted into probabilistic relationships between observable image features as well as between the unobservable properties that caused these features, including object boundaries and relative depth. We show that the image model is sufficient to explain a wide range of natural scene properties. Finally, we discuss the implications of this description of natural scenes for the study of vision.

The 'bookshelf illusion'--a real-world Zöllner-type illusion?

We discovered an interesting perceptual distortion in our office where an upright lamp in front of a bookshelf was noticeably curved to form several subtle S-shaped bends. We realized that the books in the bookshelf fell in a particular manner, leaning in alternative directions, which caused the straight lamp to appear bent, creating what may be a real-world example of the Zöllner illusion. Evidence for the production of the illusion diagrammatically and an explanation for the effect are provided.

Dissociable perceptual effects of visual adaptation

Neurons in the visual cortex are responsive to the presentation of oriented and curved line segments, which are thought to act as primitives for the visual processing of shapes and objects. Prolonged adaptation to such stimuli gives rise to two related perceptual effects: a slow change in the appearance of the adapting stimulus (perceptual drift), and the distortion of subsequently presented test stimuli (adaptational aftereffects). Here we used a psychophysical nulling technique to dissociate and quantify these two classical observations in order to examine their underlying mechanisms and their relationship to one another. In agreement with previous work, we found that during adaptation horizontal and vertical straight lines serve as attractors for perceived orientation and curvature. However, the rate of perceptual drift for different stimuli was not predictive of the corresponding aftereffect magnitudes, indicating that the two perceptual effects are governed by distinct neural processes. Finally, the rate of perceptual drift for curved line segments did not depend on the spatial scale of the stimulus, suggesting that its mechanisms lie outside strictly retinotopic processing stages. These findings provide new evidence that the visual system relies on statistically salient intrinsic reference stimuli for the processing of visual patterns, and point to perceptual drift as an experimental window for studying the mechanisms of visual perception.

Memory-prediction errors and their consequences in schizophrenia

Cognitive deficits play a central role in the onset of schizophrenia. Cognitive impairment precedes the onset of psychosis in at least a subgroup of patients, and accounts for considerable dysfunction. Yet cognitive deficits as currently measured are not significantly related to hallucinations and delusions. Part of this counterintuitive absence of a relationship may be caused by the lack of an organizing principle of cognitive impairment in schizophrenia research. We review literature suggesting that a system of memory-based prediction is central to human perception, thought and action , and forward the notion that many of the symptoms of schizophrenia are a result of a failure of this system.

The brightness of colour

Background

The perception of brightness depends on spatial context: the same stimulus can appear light or dark depending on what surrounds it. A less well-known but equally important contextual phenomenon is that the colour of a stimulus can also alter its brightness. Specifically, stimuli that are more saturated (i.e. purer in colour) appear brighter than stimuli that are less saturated at the same luminance. Similarly, stimuli that are red or blue appear brighter than equiluminant yellow and green stimuli. This non-linear relationship between stimulus intensity and brightness, called the Helmholtz-Kohlrausch (HK) effect, was first described in the nineteenth century but has never been explained. Here, we take advantage of the relative simplicity of this ‘illusion’ to explain it and contextual effects more generally, by using a simple Bayesian ideal observer model of the human visual ecology. We also use fMRI brain scans to identify the neural correlates of brightness without changing the spatial context of the stimulus, which has complicated the interpretation of related fMRI studies.

Results

Rather than modelling human vision directly, we use a Bayesian ideal observer to model human visual ecology. We show that the HK effect is a result of encoding the non-linear statistical relationship between retinal images and natural scenes that would have been experienced by the human visual system in the past. We further show that the complexity of this relationship is due to the response functions of the cone photoreceptors, which themselves are thought to represent an efficient solution to encoding the statistics of images. Finally, we show that the locus of the response to the relationship between images and scenes lies in the primary visual cortex (V1), if not earlier in the visual system, since the brightness of colours (as opposed to their luminance) accords with activity in V1 as measured with fMRI.

Conclusions

The data suggest that perceptions of brightness represent a robust visual response to the likely sources of stimuli, as determined, in this instance, by the known statistical relationship between scenes and their retinal responses. While the responses of the early visual system (receptors in this case) may represent specifically the statistics of images, post receptor responses are more likely represent the statistical relationship between images and scenes. A corollary of this suggestion is that the visual cortex is adapted to relate the retinal image to behaviour given the statistics of its past interactions with the sources of retinal images: the visual cortex is adapted to the signals it receives from the eyes, and not directly to the world beyond.

Space and time in visual context

No sensory stimulus is an island unto itself; rather, it can only properly be interpreted in light of the stimuli that surround it in space and time. This can result in entertaining illusions and puzzling results in psychological and neurophysiological experiments. We concentrate on perhaps the best studied test case, namely orientation or tilt, which gives rise to the notorious tilt illusion and the adaptation tilt after-effect. We review the empirical literature and discuss the computational and statistical ideas that are battling to explain these conundrums, and thereby gain favour as more general accounts of cortical processing.

Transient cortical excitation at the onset of visual fixation

Primates actively examine the visual world by rapidly shifting gaze (fixation) over the elements in a scene. Despite this fact, we typically study vision by presenting stimuli with gaze held constant. To better understand the dynamics of natural vision, we examined how the onset of visual fixation affects ongoing neuronal activity in the absence of visual stimulation. We used multiunit activity and current source density measurements to index neuronal firing patterns and underlying synaptic processes in macaque V1. Initial averaging of neural activity synchronized to the onset of fixation suggested that a brief period of cortical excitation follows each fixation. Subsequent single-trial analyses revealed that 1) neuronal oscillation phase transits from random to a highly organized state just after the fixation onset, 2) this phase concentration is accompanied by increased spectral power in several frequency bands, and 3) visual response amplitude is enhanced at the specific oscillatory phase associated with fixation. We hypothesize that nonvisual inputs are used by the brain to increase cortical excitability at fixation onset, thus "priming" the system for new visual inputs generated at fixation. Despite remaining mechanistic questions, it appears that analysis of fixation-related responses may be useful in studying natural vision.

Evolution of visually guided behavior in artificial agents

Recent work on brightness, color, and form has suggested that human visual percepts represent the probable sources of retinal images rather than stimulus features as such. Here we investigate the plausibility of this empirical concept of vision by allowing autonomous agents to evolve in virtual environments based solely on the relative success of their behavior. The responses of evolved agents to visual stimuli indicate that fitness improves as the neural network control systems gradually incorporate the statistical relationship between projected images and behavior appropriate to the sources of the inherently ambiguous images. These results: (1) demonstrate the merits of a wholly empirical strategy of animal vision as a means of contending with the inverse optics problem; (2) argue that the information incorporated into biological visual processing circuitry is the relationship between images and their probable sources; and (3) suggest why human percepts do not map neatly onto physical reality.

Comparison of Bayesian and empirical ranking approaches to visual perception

Much current vision research is predicated on the idea--and a rapidly growing body of evidence--that visual percepts are generated according to the empirical significance of light stimuli rather than their physical characteristics. As a result, an increasing number of investigators have asked how visual perception can be rationalized in these terms. Here, we compare two different theoretical frameworks for predicting what observers actually see in response to visual stimuli: Bayesian decision theory and empirical ranking theory. Deciding which of these approaches has greater merit is likely to determine how the statistical operations that apparently underlie visual perception are eventually understood.

Bayesian inference of form and shape

The ability to visually perceive two-dimensional (2D) form and three-dimensional (3D) shape is one of our most fundamental faculties. This ability relies on considerable prior knowledge about the way edge elements in an image are likely to be connected together into a contour as well as the way these 2D contours relate to 3D shapes. The interaction of prior knowledge with image information is well modeled within a Bayesian framework. We review here the experimental evidence of shape perception seen as a Bayesian inference problem.

The Poggendorff illusion explained by natural scene geometry

One of the most intriguing of the many discrepancies between perceived spatial relationships and the physical structure of visual stimuli is the Poggendorff illusion, when an obliquely oriented line that is interrupted no longer appears collinear. Although many different theories have been proposed to explain this effect, there has been no consensus about its cause. Here, we use a database of range images (i.e., images that include the distance from the image plane of every pixel in the scene) to show that the probability distribution of the possible locations of line segments across an interval in natural environments can fully account for all of the behavior of this otherwise puzzling phenomenon.

The Müller-Lyer illusion explained by the statistics of image-source relationships

The Müller-Lyer effect, the apparent difference in the length of a line as the result of its adornment with arrowheads or arrow tails, is the best known and most controversial of the classical geometrical illusions. By sampling a range-image database of natural scenes, we show that the perceptual effects elicited by the Müller-Lyer stimulus and its major variants are correctly predicted by the probability distributions of the possible physical sources underlying the relevant retinal images. These results support the conclusion that the Müller-Lyer illusion is a manifestation of the probabilistic strategy of visual processing that has evolved to contend with the uncertain provenance of retinal stimuli.