Aligning pictorial descriptions: an approach to object recognition

Interpreting the orientation of objects: A cross-disciplinary review

Is object orientation an inherent aspect of the shape of the object or is it represented separately and bound to the object shape in a similar way to other features, such as colour? This review brings together findings from neuropsychological studies of patients with agnosia for object orientation and experimental studies of object perception in healthy individuals that provide converging evidence of separate processing of object identity and orientation. Individuals with agnosia for object orientation, which typically results from damage to the right parietal lobe, can recognize objects presented in a range of orientations yet are unable to interpret or discriminate the objects' orientation. Healthy individuals tested with briefly presented objects demonstrate a similar dissociation: object identity is extracted rapidly in an orientation-invariant way, whereas processing the object's orientation is slower, requires attention and is influenced by the degree of departure from the canonical orientation. This asymmetry in processing can sometimes lead to incorrect bindings between the identity and orientation of objects presented in close temporal proximity. Overall, the available evidence indicates that object recognition is achieved in a largely orientation-invariant manner and that interpreting the object's orientation requires an additional step of mapping this orientation-invariant representation to a spatial reference frame.

The roles of symmetry and elongation in developing reference frames

Previous studies showed that elongation and symmetry (two ubiquitous aspects of natural stimuli) are important attributes in object perception and recognition, which in turn suggests that these geometrical factors may contribute to the selection of perceptual reference-frames. However, whether and how these attributes guide the selection of reference-frames is still poorly understood. The goal of this study was to examine systematically the roles of elongation and symmetry, as well as their combination, in the selection of reference axis and how these axes are developed for unfamiliar objects. We designed our experiments to eliminate two potential confounding factors: (i) extraneous environmental cues, such as edges of the screen, etc. (by using VR) and (ii) pre-learned cues for familiar objects and shapes (by using reinforcement learning of novel shapes). We used algorithmically generated textures with different orientations having specified levels of symmetry and elongation as the stimuli. In each trial, we presented only one stimulus and asked observers to report if the stimulus was in its original form or a flipped (mirror-image) one. Feedback was provided at the end of each trial. Based on previous studies on mental rotation, we hypothesized that the selection of a reference-frame defined by symmetry and/or elongation would be revealed by a linear relationship between reaction-times and the angular-deviation from either the most symmetrical or the most elongated orientation. Our results are consistent with this hypothesis. We found that subjects performed mental rotation to transform images to their reference axes and used the most symmetrical or elongated orientation as the reference axis when only one factor was presented, and they used a "winner-take-all" strategy when both factors were presented, with elongation being more dominant than symmetry. We discuss theoretical implications of these findings, in particular in the context of "canonical sensorimotor theory."

The attentive reconstruction of objects facilitates robust object recognition

Humans are extremely robust in our ability to perceive and recognize objects-we see faces in tea stains and can recognize friends on dark streets. Yet, neurocomputational models of primate object recognition have focused on the initial feed-forward pass of processing through the ventral stream and less on the top-down feedback that likely underlies robust object perception and recognition. Aligned with the generative approach, we propose that the visual system actively facilitates recognition by reconstructing the object hypothesized to be in the image. Top-down attention then uses this reconstruction as a template to bias feedforward processing to align with the most plausible object hypothesis. Building on auto-encoder neural networks, our model makes detailed hypotheses about the appearance and location of the candidate objects in the image by reconstructing a complete object representation from potentially incomplete visual input due to noise and occlusion. The model then leverages the best object reconstruction, measured by reconstruction error, to direct the bottom-up process of selectively routing low-level features, a top-down biasing that captures a core function of attention. We evaluated our model using the MNIST-C (handwritten digits under corruptions) and ImageNet-C (real-world objects under corruptions) datasets. Not only did our model achieve superior performance on these challenging tasks designed to approximate real-world noise and occlusion viewing conditions, but also better accounted for human behavioral reaction times and error patterns than a standard feedforward Convolutional Neural Network. Our model suggests that a complete understanding of object perception and recognition requires integrating top-down and attention feedback, which we propose is an object reconstruction.

Perceptual similarity judgments do not predict the distribution of errors in working memory

Population coding models provide a quantitative account of visual working memory (VWM) retrieval errors with a plausible link to the response characteristics of sensory neurons. Recent work has provided an important new perspective linking population coding to variables of signal detection, including d-prime, and put forward a new hypothesis: that the distribution of recall errors on, for example, a color wheel, is a consequence of the psychological similarity between points in that stimulus space, such that the exponential-like psychophysical distance scaling function can fulfil the role of population tuning and obviate the need to fit a tuning width parameter to recall data. Using four different visual feature spaces, we measured psychophysical similarity and memory errors in the same participants. Our results revealed strong evidence for a common source of variability affecting similarity judgments and recall estimates but did not support any consistent relationship between psychophysical similarity functions and VWM errors. At the group level, the responsiveness functions obtained from the psychophysical similarity task diverged strongly from those that provided the best fit to working memory errors. At the individual level, we found convincing evidence against an association between observed and best-fitting similarity functions. Finally, our results show that the newly proposed exponential-like responsiveness function has in general no advantage over the canonical von Mises (circular normal) function assumed by previous population coding models. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Modal and amodal cognition: an overarching principle in various domains of psychology

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Use of superordinate labels yields more robust and human-like visual representations in convolutional neural networks

Human visual recognition is outstandingly robust. People can recognize thousands of object classes in the blink of an eye (50–200 ms) even when the objects vary in position, scale, viewpoint, and illumination. What aspects of human category learning facilitate the extraction of invariant visual features for object recognition? Here, we explore the possibility that a contributing factor to learning such robust visual representations may be a taxonomic hierarchy communicated in part by common labels to which people are exposed as part of natural language. We did this by manipulating the taxonomic level of labels (e.g., superordinate-level [mammal, fruit, vehicle] and basic-level [dog, banana, van]), and the order in which these training labels were used during learning by a Convolutional Neural Network. We found that training the model with hierarchical labels yields visual representations that are more robust to image transformations (e.g., position/scale, illumination, noise, and blur), especially when images were first trained with superordinate labels and then fine-tuned with basic labels. We also found that Superordinate-label followed by Basic-label training best predicts functional magnetic resonance imaging responses in visual cortex and behavioral similarity judgments recorded while viewing naturalistic images. The benefits of training with superordinate labels in the earlier stages of category learning is discussed in the context of representational efficiency and generalization.

From Objects to Unified Minds

The involvement of top-down processes in perception and cognition is widely acknowledged by now. In fields of research from predictions to inhibition, and from attentional guidance to affect, a great deal has already been charted. Integrating this newer understanding with accumulated findings from the past has made it clear that human experience is determined by a combination of both bottom-up and top-down processes. It has been proposed that the ongoing balance between their relative contribution affects a person’s entire state of mind, an overarching framework that encompasses the breadth of mental activity. According to this proposal, state of mind, in which multiple facets of mind are clumped together functionally and dynamically, orients us to the optimal state for the given circumstances. These ideas are examined here by connecting a broad array of domains in which the balance between top-down and bottom-up processes is apparent. These domains range from object recognition to contextual associations, from pattern of thought to tolerance for uncertainty, and from the default-mode network to mood. From this synthesis emerge numerous hypotheses, implications, and directions for future research in cognitive psychology, psychiatry, and neuroscience.

Does stereopsis improve face identification? A study using a virtual reality display with integrated eye-tracking and pupillometry

Stereopsis is a powerful depth cue for humans, which may also contribute to object recognition. In particular, we surmise that face identification would benefit from the availability of stereoscopic depth cues, since facial perception may be based on three-dimensional (3D) representations. In this study, a virtual reality (VR) headset with integrated eye-tracking was used to present stereoscopic images of faces. As a monoscopic contrast condition, identical images of faces were displayed to the two eyes. We monitored the participants' gaze behavior and pupil diameters while they performed a sample-to-match face identification task. We found that accuracy was superior in the stereoscopic condition compared to the monoscopic condition for frontal and intermediate views, but not profiles. Moreover, pupillary diameters were smaller when identifying stereoscopically seen faces than when viewing them without stereometric cues, which we interpret as lower processing load for the former than the latter conditions. The analysis of gaze showed that participants tended to focus on regions of the face rich in volumetric information, more so in the stereoscopic condition than the monoscopic condition. Together, these findings suggest that a 3D representation of faces may be the natural format used by the visual system when assessing face identity. Stereoscopic information, by providing depth information, assists the construction of robust facial representations in memory.

Rethinking Space: A Review of Perception, Attention, and Memory in Scene Processing

Scene processing is fundamentally influenced and constrained by spatial layout and spatial associations with objects. However, semantic information has played a vital role in propelling our understanding of real-world scene perception forward. In this article, we review recent advances in assessing how spatial layout and spatial relations influence scene processing. We examine the organization of the larger environment and how we take full advantage of spatial configurations independently of semantic information. We demonstrate that a clear differentiation of spatial from semantic information is necessary to advance research in the field of scene processing.

Brain Processes While Struggling With Evidence Accumulation During Facial Emotion Recognition: An ERP Study

The human brain is tuned to recognize emotional facial expressions in faces having a natural upright orientation. The relative contributions of featural, configural, and holistic processing to decision-making are as yet poorly understood. This study used a diffusion decision model (DDM) of decision-making to investigate the contribution of early face-sensitive processes to emotion recognition from physiognomic features (the eyes, nose, and mouth) by determining how experimental conditions tapping those processes affect early face-sensitive neuroelectric reflections (P100, N170, and P250) of processes determining evidence accumulation at the behavioral level. We first examined the effects of both stimulus orientation (upright vs. inverted) and stimulus type (photographs vs. sketches) on behavior and neuroelectric components (amplitude and latency). Then, we explored the sources of variance common to the experimental effects on event-related potentials (ERPs) and the DDM parameters. Several results suggest that the N170 indicates core visual processing for emotion recognition decision-making: (a) the additive effect of stimulus inversion and impoverishment on N170 latency; and (b) multivariate analysis suggesting that N170 neuroelectric activity must be increased to counteract the detrimental effects of face inversion on drift rate and of stimulus impoverishment on the stimulus encoding component of non-decision times. Overall, our results show that emotion recognition is still possible even with degraded stimulation, but at a neurocognitive cost, reflecting the extent to which our brain struggles to accumulate sensory evidence of a given emotion. Accordingly, we theorize that: (a) the P100 neural generator would provide a holistic frame of reference to the face percept through categorical encoding; (b) the N170 neural generator would maintain the structural cohesiveness of the subtle configural variations in facial expressions across our experimental manipulations through coordinate encoding of the facial features; and (c) building on the previous configural processing, the neurons generating the P250 would be responsible for a normalization process adapting to the facial features to match the stimulus to internal representations of emotional expressions.

A surface-based code contributes to visual shape perception

Considerable uncertainty remains regarding the types of features human vision uses for shape representation. Visual-search experiments are reported which assessed the hypothesis of a surface-based (i.e., edge-bounded polygons) code for shape representation in human vision. The results indicate slower search rates and/or longer response times when the target shape shares its constituent surfaces with distractors (conjunction condition) than when the target surfaces are unique in the display (nonconjunction condition). This demonstration is made using test conditions that strictly control any potential artifact pertaining to target-distractor similarity. The surface-based code suggested by this surface-conjunction effect is strictly 2-D, since the effect occurs even when the surfaces are shared between the target and distractors in the 2-D image but not in their 3-D instantiation. Congruently, this latter finding is unaltered by manipulations of the richness of the depth information offered by the stimuli. It is proposed that human vision uses a 2-D surface-based code for shape representation which, considering other key findings in the field, probably coexists with an alternative representation mode based on a type of structural description that can integrate information pertaining to the 3-D aspect of shapes.

Seeing structure: Shape skeletons modulate perceived similarity

An intrinsic part of seeing objects is seeing how similar or different they are relative to one another. This experience requires that objects be mentally represented in a common format over which such comparisons can be carried out. What is that representational format? Objects could be compared in terms of their superficial features (e.g., degree of pixel-by-pixel overlap), but a more intriguing possibility is that they are compared on the basis of a deeper structure. One especially promising candidate that has enjoyed success in the computer vision literature is the shape skeleton-a geometric transformation that represents objects according to their inferred underlying organization. Despite several hints that shape skeletons are computed in human vision, it remains unclear how much they actually matter for subsequent performance. Here, we explore the possibility that shape skeletons help mediate the ability to extract visual similarity. Observers completed a same/different task in which two shapes could vary either in their skeletal structure (without changing superficial features such as size, orientation, and internal angular separation) or in large surface-level ways (without changing overall skeletal organization). Discrimination was better for skeletally dissimilar shapes: observers had difficulty appreciating even surprisingly large differences when those differences did not reorganize the underlying skeletons. This pattern also generalized beyond line drawings to 3-D volumes whose skeletons were less readily inferable from the shapes' visible contours. These results show how shape skeletons may influence the perception of similarity-and more generally, how they have important consequences for downstream visual processing.

Binding identity and orientation in object recognition

We tested whether an object's orientation is inherently bound to its identity in a holistic view-based representation at the early stages of visual identification, or whether identity and orientation are represented separately. Observers saw brief and masked stimulus sequences containing two rotated objects. They had to detect if a previously cued object was present in the sequence and report its orientation. In Experiments 1 and 2, the objects were presented sequentially in the same spatial location for 70 ms each, whereas in Experiments 3 and 4 they were presented simultaneously in different spatial locations for 70 ms and 140 ms, respectively. Across all experiments, observers reported the correct orientation for approximately 70% of the positively identified objects, and were at chance in reporting the orientation when they had not recognized the object. This finding suggests that orientation information is accessed after an object has been identified. In addition, when the two objects were presented sequentially in the same spatial location, orientation errors were not random-observers tended to report the orientation of the alternative object in the sequence, indicating misbindings between the identities and orientations of objects that share spatial location. This susceptibility to binding errors was not observed when the objects were in different spatial locations. These results suggest that identity and orientation may be prone to misbinding, and that spatial location may serve to protect their joint integrity.

The nature of shape constancy mechanisms as revealed by shape priming

Five shape priming experiments are reported in which the target was either a five- or six-sided line-drawn figure and participants made a speeded two-alternative forced-choice judgment about the target's number of sides. On priming trials, the target was preceded by a briefly presented smaller line figure (the prime) and performance on these trials was gauged relative to a no-prime condition. In the first two experiments, primes were rendered invisible by the presentation of a backwards visual noise mask, respectively for a short (∼40 ms) or long duration (∼93 ms). No reliable priming effects arose under masked conditions. When these experiments were repeated without the mask, participants were speeded when the prime and target were related by a rigid through-the-plane rotation but not when the prime was a nonrigid, stretched version of the target. The same pattern of priming effects arose when, in a final experiment, novel irregular shapes were used. Collectively, the data reveal the operation of shape constancy mechanisms that are particularly sensitive to shape rigidity. The findings suggest that the visual system attempts to secure a correspondence between the rapid and successive presentations of the prime and the target by matching shapes according to a rigidity constraint.

Orientation Encoding and Viewpoint Invariance in Face Recognition: Inferring Neural Properties from Large-Scale Signals

Viewpoint-invariant face recognition is thought to be subserved by a distributed network of occipitotemporal face-selective areas that, except for the human anterior temporal lobe, have been shown to also contain face-orientation information. This review begins by highlighting the importance of bilateral symmetry for viewpoint-invariant recognition and face-orientation perception. Then, monkey electrophysiological evidence is surveyed describing key tuning properties of face-selective neurons-including neurons bimodally tuned to mirror-symmetric face-views-followed by studies combining functional magnetic resonance imaging (fMRI) and multivariate pattern analyses to probe the representation of face-orientation and identity information in humans. Altogether, neuroimaging studies suggest that face-identity is gradually disentangled from face-orientation information along the ventral visual processing stream. The evidence seems to diverge, however, regarding the prevalent form of tuning of neural populations in human face-selective areas. In this context, caveats possibly leading to erroneous inferences regarding mirror-symmetric coding are exposed, including the need to distinguish angular from Euclidean distances when interpreting multivariate pattern analyses. On this basis, this review argues that evidence from the fusiform face area is best explained by a view-sensitive code reflecting head angular disparity, consistent with a role of this area in face-orientation perception. Finally, the importance is stressed of explicit models relating neural properties to large-scale signals.

Subliminal Priming-State of the Art and Future Perspectives

The influence of subliminal priming (behavior outside of awareness) in humans is an interesting phenomenon and its understanding is crucial as it can impact behavior, choices, and actions. Given this, research about the impact of priming continues to be an area of investigative interest, and this paper provides a technical overview of research design strengths and issues in subliminal priming research. Efficient experiments and protocols, as well as associated electroencephalographic and eye movement data analyses, are discussed in detail. We highlight the strengths and weaknesses of different priming experiments that have measured affective (emotional) and cognitive responses. Finally, very recent approaches and findings are described to summarize and emphasize state-of-the-art methods and potential future directions in research marketing and other commercial applications.

Attention to distinguishing features in object recognition: An interactive-iterative framework

This article advances a framework that casts object recognition as a process of discrimination between alternative object identities, in which top-down and bottom-up processes interact-iteratively when necessary-with attention to distinguishing features playing a critical role. In two experiments, observers discriminated between different types of artificial fish. In parallel, a secondary, variable-SOA visual-probe detection task was used to examine the dynamics of visual attention. In Experiment 1, the fish varied in three distinguishing features: one indicating the general category (saltwater, freshwater), and one of the two other features indicating the specific type of fish within each category. As predicted, in the course of recognizing each fish, attention was allocated iteratively to the distinguishing features in an optimal manner: first to the general category feature, and then, based on its value, to the second feature that identified the specific fish. In Experiment 2, two types of fish could be discriminated on the basis of either of two distinguishing features, one more visually discriminable than the other. On some of the trials, one of the two alternative distinguishing features was occluded. As predicted, in the course of recognizing each fish, attention was directed initially to the more discriminable distinguishing feature, but when this feature was occluded, it was then redirected to the less discriminable feature. The implications of these findings, and the interactive-iterative framework they support, are discussed with regard to several fundamental issues having a long history in the literatures on object recognition, object categorization, and visual perception in general.

Integration of objects and space in perception and memory

Distinct processing of objects and space has been an organizing principle for studying higher-level vision and medial temporal lobe memory. Here, however, we discuss how object and spatial information are in fact closely integrated in vision and memory. The ventral, object-processing visual pathway carries precise spatial information, transformed from retinotopic coordinates into relative dimensions. At the final stages of the ventral pathway, including the dorsal anterior temporal lobe (TEd), object-sensitive neurons are intermixed with neurons that process large-scale environmental space. TEd projects primarily to perirhinal cortex (PRC), which in turn projects to lateral entorhinal cortex (LEC). PRC and LEC also combine object and spatial information. For example, PRC and LEC neurons exhibit place fields that are evoked by landmark objects or the remembered locations of objects. Thus, spatial information, on both local and global scales, is deeply integrated into the ventral (temporal) object-processing pathway in vision and memory.

Effects of age and type of picture on visuospatial working memory assessed with a computerized jigsaw-puzzle task

We investigated the effect of age and color in a computerized version of the jigsaw-puzzle task. In Experiment 1, young and older adults were presented with puzzles in color and black-and-white line drawings, varying in difficulty from 4 to 9 pieces. Older adults performed the task better with the black-and-white stimuli and younger adults performed better with the color ones. In Experiment 2, new older and young adults identified the same fragmented pictures as fast and accurately as possible. The older group identified the black-and-white stimuli faster than those presented in color, while the younger adults identified both similarly. In Experiment 3A, new older and young groups performed the puzzle task with the same color pictures and their monochrome versions. In Experiment 3B, participants performed a speeded identification task with the two sets. The findings of these experiments showed that older adults have a memory not a perceptual difficulty.

When does Human Object Recognition use a Viewer-Centered Reference Frame?

How do people recognize an object in different orientations? One theory is that the visual system describes the object relative to a reference frame centered on the object, resulting in a representation that is invariant across orientations. Chronometric data show that this is true only when an object can be identified uniquely by the arrangement of its parts along a single dimension. When an object can only be distinguished by an arrangement of its parts along more than one dimension, people mentally rotate it to a familiar orientation. This finding suggests that the human visual reference frame is tied to egocentric coordinates.

Interrelated and Isolated Self-Concepts

We propose aframework for conceptualizing different ways of representing concepts of the self. Interrelated self-concepts are concepts that are defined by connections to concepts of other (real or prototypic) individuals; isolated self-concepts do not depend upon other person conceptsfor their mental characterization. This distinction between ways of representing self-concepts is similar to the distinction between interrelated and isolated concepts recently proposed by Goldstone (1993b, 1996). In this article, the extant self literature is evaluated in terms of the interrelated-isolated distinction. Methods for manipulating and diagnosing interrelated and isolated self-concepts are also proposed. Results of 3 studies show that interrelated self-concepts contain less abstract features than do isolated self-concepts. The former concepts also contain more diagnosticfeatures than the latter. Discussion focuses on predictions about other differences in isolatedSnd interrelated self-concepts. The conditions under which different types of self-concepts might change and the implications of interrelated and isolated self-concepts for information processing, memory, self-esteem, and mental health are considered.

Event-Related Potential Effects of Object Repetition Depend on Attention and Part-Whole Configuration

The effects of spatial attention and part-whole configuration on recognition of repeated objects were investigated with behavioral and event-related potential (ERP) measures. Short-term repetition effects were measured for probe objects as a function of whether a preceding prime object was shown as an intact image or coarsely scrambled (split into two halves) and whether or not it had been attended during the prime display. In line with previous behavioral experiments, priming effects were observed from both intact and split primes for attended objects, but only from intact (repeated same-view) objects when they were unattended. These behavioral results were reflected in ERP waveforms at occipital–temporal locations as more negative-going deflections for repeated items in the time window between 220 and 300 ms after probe onset (N250r). Attended intact images showed generally more enhanced repetition effects than split ones. Unattended images showed repetition effects only when presented in an intact configuration, and this finding was limited to the right-hemisphere electrodes. Repetition effects in earlier (before 200 ms) time windows were limited to attended conditions at occipito-temporal sites during the N1, a component linked to the encoding of object structure, while repetition effects at central locations during the same time window (P150) were found for attended and unattended probes but only when repeated in the same intact configuration. The data indicate that view-generalization is mediated by a combination of analytic (part-based) representations and automatic view-dependent representations.

Visual completion from 2D cross-sections: Implications for visual theory and STEM education and practice

Accurately inferring three-dimensional (3D) structure from only a cross-section through that structure is not possible. However, many observers seem to be unaware of this fact. We present evidence for a 3D amodal completion process that may explain this phenomenon and provide new insights into how the perceptual system processes 3D structures. Across four experiments, observers viewed cross-sections of common objects and reported whether regions visible on the surface extended into the object. If they reported that the region extended, they were asked to indicate the orientation of extension or that the 3D shape was unknowable from the cross-section. Across Experiments 1, 2, and 3, participants frequently inferred 3D forms from surface views, showing a specific prior to report that regions in the cross-section extend straight back into the object, with little variance in orientation. In Experiment 3, we examined whether 3D visual inferences made from cross-sections are similar to other cases of amodal completion by examining how the inferences were influenced by observers’ knowledge of the objects. Finally, in Experiment 4, we demonstrate that these systematic visual inferences are unlikely to result from demand characteristics or response biases. We argue that these 3D visual inferences have been largely unrecognized by the perception community, and have implications for models of 3D visual completion and science education.

Novel Visual Illusions Related to Vasarely's ‘Nested Squares’ Show That Corner Salience Varies with Corner Angle

Vasarely's ‘nested-squares’ illusion shows that 90° corners can be more salient perceptually than straight edges. On the basis of this illusion we have developed a novel visual illusion, the ‘Alternating Brightness Star’, which shows that sharp corners are more salient than shallow corners (an effect we call ‘corner angle salience variation’) and that the same corner can be perceived as either bright or dark depending on the polarity of the angle (ie whether concave or convex: ‘corner angle brightness reversal’). Here we quantify the perception of corner angle salience variation and corner angle brightness reversal effects in twelve naive human subjects, in a two-alternative forced-choice brightness discrimination task. The results show that sharp corners generate stronger percepts than shallow corners, and that corner gradients appear bright or dark depending on whether the corner is concave or convex. Basic computational models of center – surround receptive fields predict the results to some degree, but not fully.

Distinct Patterns of Viewpoint-Dependent BOLD Activity during Common-Object Recognition and Mental Rotation

A fundamental but unanswered question about the human visual system concerns the way in which misoriented objects are recognized. One hypothesis maintains that representations of incoming stimuli are transformed via parietally based spatial normalization mechanisms (eg mental rotation) to match view-specific representations in long-term memory. Using fMRI, we tested this hypothesis by directly comparing patterns of brain activity evoked during classic mental rotation and misoriented object recognition involving everyday objects. BOLD activity increased systematically with stimulus rotation within the ventral visual stream during object recognition and within the dorsal visual stream during mental rotation. More specifically, viewpoint-dependent activity was significantly greater in the right superior parietal lobule during mental rotation than during object recognition. In contrast, viewpoint-dependent activity was significantly greater in the right fusiform gyrus during object recognition than during mental rotation. In addition to these differences in viewpoint-dependent activity, object recognition and mental rotation produced distinct patterns of brain activity, independent of stimulus rotation: object recognition resulted in greater overall activity within ventral stream visual areas and mental rotation resulted in greater overall activity within dorsal stream visual areas. The present results are inconsistent with the hypothesis that misoriented object recognition is mediated by structures within the parietal lobe that are known to be involved in mental rotation.

Efficient Extrapolation of the View with a Dynamic and Predictive Stimulus

In the real-world, the retinal projection of an object changes as we move, or as a moving object passes in front of us. We have to recognise objects, despite such retinal-projection changes. Many studies have shown that the time required to identify objects after a change in the retinal projection is longer than when there is no retinal-projection change. This recognition cost is referred to as the view-dependent effect. Previous researchers have studied the view-dependent effect while disregarding the predictability of retinal-projection changes. Here, we demonstrate that there is no view-dependent effect when the predictability is introduced, in the case where participants track moving objects by head-turning or eye-movement in a virtual environment. Violation of the predictability, such as an unpredictable retinal-projection change or a movement of the first stimulus that was inconsistent with a subsequent retinal-projection change, caused a view-dependent effect. Moreover, we found that extraretinal information such as head-turning or eye-movement was unnecessary for view-independent recognition. These results indicate that humans can extrapolate to the tested view from the studied view in a view-independent way when retinal-projection change is predictable from the visual stimulus.

Major Axes as a Moderately Abstract Model for Object Recognition

These experiments provide evidence that the major axes of an object's components can activate a moderately abstract model that is used during recognition Major-axis primes were presented immediately before target airplanes, the total stimulus durations were about 220 ms Major axes facilitated identification accuracy relative to primes controlling for attention and for a general reference frame In addition, axis primes facilitated identification with incongruently oriented targets and across small shifts in the target's spatial position The configuration of the axes was crucial because facilitation did not occur with primes containing similar elements in a different relationship

Perceiving Real-World Viewpoint Changes

Retinal images vary as observers move through the environment, but observers seem to have little difficulty recognizing objects and scenes across changes in view. Although real-world view changes can be produced both by object rotations (orientation changes) and by observer movements (viewpoint changes), research on recognition across views has relied exclusively on display rotations. However, research on spatial reasoning suggests a possible dissociation between orientation and viewpoint. Here we demonstrate that scene recognition in the real world depends on more than the retinal projection of the visible array; viewpoint changes have little effect on detection of layout changes, but equivalent orientation changes disrupt performance significantly. Findings from our three experiments suggest that scene recognition across view changes relies on a mechanism that updates a viewer-centered representation during observer movements, a mechanism not available for orientation changes. These results link findings from spatial tasks to work on object and scene recognition and highlight the importance of considering the mechanisms underlying recognition in real environments.

Viewpoint Dependence in Scene Recognition

Two experiments investigated the viewpoint dependence of spatial memories In Experiment 1, participants learned the locations of objects on a desktop from a single perspective and then took part in a recognition test, test scenes included familiar and novel views of the layout Recognition latency was a linear function of the angular distance between a test view and the study view In Experiment 2, participants studied a layout from a single view and then learned to recognize the layout from three additional training views A final recognition test showed that the study view and the training views were represented in memory, and that latency was a linear function of the angular distance to the nearest study or training view These results indicate that interobject spatial relations are encoded in a viewpoint-dependent manner, and that recognition of novel views requires normalization to the most similar representation in memory These findings parallel recent results in visual object recognition.

The Visual System's Measurement of Invariants Need Not Itself Be Invariant

When two shapes that differ in orientation or size have to be compared or objects have to be recognized from different viewpoints, the response time and error rate are systematically affected by the size of the geometric difference In this report, we argue that these effects are not necessarily solid evidence for the use of mental transformations and against the use of invariants by the visual system We report an experiment in which observers were asked to give affine-invariant coordinates of a point located in an affine frame defined by three other points The angle subtended by the coordinate axes and the ratio of the lengths of their unit vectors systematically affected the measurement errors This finding demonstrates that the visual system's measurement of invariants need not itself be invariant

Learning the 3-D structure of objects from 2-D views depends on shape, not format

Humans can learn to recognize new objects just from observing example views. However, it is unknown what structural information enables this learning. To address this question, we manipulated the amount of structural information given to subjects during unsupervised learning by varying the format of the trained views. We then tested how format affected participants' ability to discriminate similar objects across views that were rotated 90° apart. We found that, after training, participants' performance increased and generalized to new views in the same format. Surprisingly, the improvement was similar across line drawings, shape from shading, and shape from shading + stereo even though the latter two formats provide richer depth information compared to line drawings. In contrast, participants' improvement was significantly lower when training used silhouettes, suggesting that silhouettes do not have enough information to generate a robust 3-D structure. To test whether the learned object representations were format-specific or format-invariant, we examined if learning novel objects from example views transfers across formats. We found that learning objects from example line drawings transferred to shape from shading and vice versa. These results have important implications for theories of object recognition because they suggest that (a) learning the 3-D structure of objects does not require rich structural cues during training as long as shape information of internal and external features is provided and (b) learning generates shape-based object representations independent of the training format.

Developmental Commonalities between Object and Face Recognition in Adolescence

In the visual perception literature, the recognition of faces has often been contrasted with that of non-face objects, in terms of differences with regard to the role of parts, part relations and holistic processing. However, recent evidence from developmental studies has begun to blur this sharp distinction. We review evidence for a protracted development of object recognition that is reminiscent of the well-documented slow maturation observed for faces. The prolonged development manifests itself in a retarded processing of metric part relations as opposed to that of individual parts and offers surprising parallels to developmental accounts of face recognition, even though the interpretation of the data is less clear with regard to holistic processing. We conclude that such results might indicate functional commonalities between the mechanisms underlying the recognition of faces and non-face objects, which are modulated by different task requirements in the two stimulus domains.

Investigating shape representation using sensitivity to part- and axis-based transformations

Part- and axis-based approaches organize shape representations in terms of simple parts and their spatial relationships. Shape transformations that alter qualitative part structure have been shown to be more detectable than those that preserve it. We compared sensitivity to various transformations that change quantitative properties of parts and their spatial relationships, while preserving qualitative part structure. Shape transformations involving changes in length, width, curvature, orientation and location were applied to a small part attached to a larger base of a two-part shape. Increment thresholds were estimated for each transformation using a 2IFC procedure. Thresholds were converted into common units of shape difference to enable comparisons across transformations. Higher sensitivity was consistently found for transformations involving a parameter of a single part (length, width, curvature) than those involving spatial relations between two parts (relative orientation and location), suggesting a single-part superiority effect. Moreover, sensitivity to shifts in part location - a biomechanically implausible shape transformation - was consistently poorest. The influence of region-based geometry was investigated via stereoscopic manipulation of figure and ground. Sensitivity was compared across positive parts (protrusions) and negative parts (indentations) for transformations involving a change in orientation or location. For changes in part orientation (biomechanically plausible), sensitivity was better for positive than negative parts; whereas for changes in part location (biomechanically implausible), no systematic difference was observed.

The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex

Is visual cortex made up of general-purpose information processing machinery, or does it consist of a collection of specialized modules? If prior knowledge, acquired from learning a set of objects is only transferable to new objects that share properties with the old, then the recognition system's optimal organization must be one containing specialized modules for different object classes. Our analysis starts from a premise we call the invariance hypothesis: that the computational goal of the ventral stream is to compute an invariant-to-transformations and discriminative signature for recognition. The key condition enabling approximate transfer of invariance without sacrificing discriminability turns out to be that the learned and novel objects transform similarly. This implies that the optimal recognition system must contain subsystems trained only with data from similarly-transforming objects and suggests a novel interpretation of domain-specific regions like the fusiform face area (FFA). Furthermore, we can define an index of transformation-compatibility, computable from videos, that can be combined with information about the statistics of natural vision to yield predictions for which object categories ought to have domain-specific regions in agreement with the available data. The result is a unifying account linking the large literature on view-based recognition with the wealth of experimental evidence concerning domain-specific regions.

The lemon illusion: seeing curvature where there is none

Curvature is a highly informative visual cue for shape perception and object recognition. We introduce a novel illusion—the Lemon Illusion—in which subtle illusory curvature is perceived along contour regions that are devoid of physical curvature. We offer several perceptual demonstrations and observations that lead us to conclude that the Lemon Illusion is an instance of a more general illusory curvature phenomenon, one in which the presence of contour curvature discontinuities lead to the erroneous extension of perceived curvature. We propose that this erroneous extension of perceived curvature results from the interaction of neural mechanisms that operate on spatially local contour curvature signals with higher-tier mechanisms that serve to establish more global representations of object shape. Our observations suggest that the Lemon Illusion stems from discontinuous curvature transitions between rectilinear and curved contour segments. However, the presence of curvature discontinuities is not sufficient to produce the Lemon Illusion, and the minimal conditions necessary to elicit this subtle and insidious illusion are difficult to pin down.

Non-holistic coding of objects in lateral occipital complex with and without attention

A fundamental issue in visual cognition is whether high-level visual areas code objects in a part-based or a view-based (holistic) format. Previous behavioral and neuroimaging studies that examined the viewpoint invariance of object recognition have yielded ambiguous results, providing evidence for either type of representational format. A critical factor distinguishing the two formats could be the availability of attentional resources, as a number of priming studies have found greater viewpoint invariance for attended compared to unattended objects. It has therefore been suggested that the activation of part-based representations requires attention, whereas the activation of holistic representations occurs automatically irrespective of attention. Using functional magnetic resonance imaging in combination with a novel multivariate pattern analysis approach, the present study probed the format of object representations in human lateral occipital complex and its dependence on attention. We presented human participants with intact and half-split versions of objects that were either attended or unattended. Cross-classifying between intact and split objects, we found that the object-related information coded in activation patterns of intact objects is fully preserved in the patterns of split objects and vice versa. Importantly, the generalization between intact and split objects did not depend on attention. We conclude that lateral occipital complex codes objects in a non-holistic format, both in the presence and absence of attention.

Size precedes view: developmental emergence of invariant object representations in lateral occipital complex

Although object perception involves encoding a wide variety of object properties (e.g., size, color, viewpoint), some properties are irrelevant for identifying the object. The key to successful object recognition is having an internal representation of the object identity that is insensitive to these properties while accurately representing important diagnostic features. Behavioral evidence indicates that the formation of these kinds of invariant object representations takes many years to develop. However, little research has investigated the developmental emergence of invariant object representations in the ventral visual processing stream, particularly in the lateral occipital complex (LOC) that is implicated in object processing in adults. Here, we used an fMR adaptation paradigm to evaluate age-related changes in the neural representation of objects within LOC across variations in size and viewpoint from childhood through early adulthood. We found a dissociation between the neural encoding of object size and object viewpoint within LOC: by age of 5-10 years, area LOC demonstrates adaptation across changes in size, but not viewpoint, suggesting that LOC responses are invariant to size variations, but that adaptation across changes in view is observed in LOC much later in development. Furthermore, activation in LOC was correlated with behavioral indicators of view invariance across the entire sample, such that greater adaptation was correlated with better recognition of objects across changes in viewpoint. We did not observe similar developmental differences within early visual cortex. These results indicate that LOC acquires the capacity to compute invariance specific to different sources of information at different time points over the course of development.

Shape Analysis of Planar Multiply-Connected Objects Using Conformal Welding

Shape analysis is a central problem in the field of computer vision. In 2D shape analysis, classification and recognition of objects from their observed silhouettes are extremely crucial but difficult. It usually involves an efficient representation of 2D shape space with a metric, so that its mathematical structure can be used for further analysis. Although the study of 2D simply-connected shapes has been subject to a corpus of literatures, the analysis of multiply-connected shapes is comparatively less studied. In this work, we propose a representation for general 2D multiply-connected domains with arbitrary topologies using conformal welding. A metric can be defined on the proposed representation space, which gives a metric to measure dissimilarities between objects. The main idea is to map the exterior and interior of the domain conformally to unit disks and circle domains (unit disk with several inner disks removed), using holomorphic 1-forms. A set of diffeomorphisms of the unit circle S(1) can be obtained, which together with the conformal modules are used to define the shape signature. A shape distance between shape signatures can be defined to measure dissimilarities between shapes. We prove theoretically that the proposed shape signature uniquely determines the multiply-connected objects under suitable normalization. We also introduce a reconstruction algorithm to obtain shapes from their signatures. This completes our framework and allows us to move back and forth between shapes and signatures. With that, a morphing algorithm between shapes can be developed through the interpolation of the Beltrami coefficients associated with the signatures. Experiments have been carried out on shapes extracted from real images. Results demonstrate the efficacy of our proposed algorithm as a stable shape representation scheme.

A biologically based model for recognition of 2-D occluded patterns

In this work, we present a biologically inspired model for recognition of occluded patterns. The general architecture of the model is based on the two visual information processing pathways of the human visual system, i.e. the ventral and the dorsal pathways. The proposed hierarchically structured model consists of three parallel processing channels. The main channel learns invariant representations of the input patterns and is responsible for pattern recognition task. But, it is limited to process one pattern at a time. The direct channel represents the biologically based direct connection from the lower to the higher processing level in the human visual cortex. It computes rapid top-down pattern-specific cues to modulate processing in the other two channels. The spatial channel mimics the dorsal pathway of the visual cortex. It generates a combined saliency map of the input patterns and, later, segments the part of the map representing the occluded pattern. This segmentation process is based on our hypothesis that the dorsal pathway, in addition to encoding spatial properties, encodes the shape representations of the patterns as well. The lateral interaction between the main and the spatial channels at appropriate processing levels and top-down, pattern-specific modulation of the these two channels by the direct channel strengthen the locations and features representing the occluded pattern. Consequently, occluded patterns become focus of attention in the ventral channel and also the pattern selected for further processing along this channel for final recognition.

Earlier development of analytical than holistic object recognition in adolescence

BACKGROUND

Previous research has shown that object recognition may develop well into late childhood and adolescence. The present study extends that research and reveals novel differences in holistic and analytic recognition performance in 7-12 year olds compared to that seen in adults. We interpret our data within a hybrid model of object recognition that proposes two parallel routes for recognition (analytic vs. holistic) modulated by attention.

METHODOLOGY/PRINCIPAL FINDINGS

Using a repetition-priming paradigm, we found in Experiment 1 that children showed no holistic priming, but only analytic priming. Given that holistic priming might be thought to be more 'primitive', we confirmed in Experiment 2 that our surprising finding was not because children's analytic recognition was merely a result of name repetition.

CONCLUSIONS/SIGNIFICANCE

Our results suggest a developmental primacy of analytic object recognition. By contrast, holistic object recognition skills appear to emerge with a much more protracted trajectory extending into late adolescence.

The power of connectivity: identity preserving transformations on visual streams in the spike domain

We investigate neural architectures for identity preserving transformations (IPTs) on visual stimuli in the spike domain. The stimuli are encoded with a population of spiking neurons; the resulting spikes are processed and finally decoded. A number of IPTs are demonstrated including faithful stimulus recovery, as well as simple transformations on the original visual stimulus such as translations, rotations and zoomings. We show that if the set of receptive fields satisfies certain symmetry properties, then IPTs can easily be realized and additionally, the same basic stimulus decoding algorithm can be employed to recover the transformed input stimulus. Using group theoretic methods we advance two different neural encoding architectures and discuss the realization of exact and approximate IPTs. These are realized in the spike domain processing block by a "switching matrix" that regulates the input/output connectivity between the stimulus encoding and decoding blocks. For example, for a particular connectivity setting of the switching matrix, the original stimulus is faithfully recovered. For other settings, translations, rotations and dilations (or combinations of these operations) of the original video stream are obtained. We evaluate our theoretical derivations through extensive simulations on natural video scenes, and discuss implications of our results on the problem of invariant object recognition in the spike domain.

Biologically inspired face recognition: toward pose-invariance

A small change in image will cause a dramatic change in signals. Visual system is required to be able to ignore these changes, yet specific enough to perform recognition. This work intends to provide biological-backed insights into 2D translation and scaling invariance and 3D pose-invariance without imposing strain on memory and with biological justification. The model can be divided into lower and higher visual stages. Lower visual stage models the visual pathway from retina to the striate cortex (V1), whereas the modeling of higher visual stage is mainly based on current psychophysical evidences.

Renewing the respect for similarity

In psychology, the concept of similarity has traditionally evoked a mixture of respect, stemming from its ubiquity and intuitive appeal, and concern, due to its dependence on the framing of the problem at hand and on its context. We argue for a renewed focus on similarity as an explanatory concept, by surveying established results and new developments in the theory and methods of similarity-preserving associative lookup and dimensionality reduction-critical components of many cognitive functions, as well as of intelligent data management in computer vision. We focus in particular on the growing family of algorithms that support associative memory by performing hashing that respects local similarity, and on the uses of similarity in representing structured objects and scenes. Insofar as these similarity-based ideas and methods are useful in cognitive modeling and in AI applications, they should be included in the core conceptual toolkit of computational neuroscience. In support of this stance, the present paper (1) offers a discussion of conceptual, mathematical, computational, and empirical aspects of similarity, as applied to the problems of visual object and scene representation, recognition, and interpretation, (2) mentions some key computational problems arising in attempts to put similarity to use, along with their possible solutions, (3) briefly states a previously developed similarity-based framework for visual object representation, the Chorus of Prototypes, along with the empirical support it enjoys, (4) presents new mathematical insights into the effectiveness of this framework, derived from its relationship to locality-sensitive hashing (LSH) and to concomitant statistics, (5) introduces a new model, the Chorus of Relational Descriptors (ChoRD), that extends this framework to scene representation and interpretation, (6) describes its implementation and testing, and finally (7) suggests possible directions in which the present research program can be extended in the future.

Implicit learning of viewpoint-independent spatial layouts

We usually perceive things in our surroundings as unchanged despite viewpoint changes caused by self-motion. The visual system therefore must have a function to process objects independently of viewpoint. In this study, we examined whether viewpoint-independent spatial layout can be obtained implicitly. For this purpose, we used a contextual cueing effect, a learning effect of spatial layout in visual search displays known to be an implicit effect. We investigated the transfer of the contextual cueing effect to images from a different viewpoint by using visual search displays of 3D objects. For images from a different viewpoint, the contextual cueing effect was maintained with self-motion but disappeared when the display changed without self-motion. This indicates that there is an implicit learning effect in environment-centered coordinates and suggests that the spatial representation of object layouts can be obtained and updated implicitly. We also showed that binocular disparity plays an important role in the layout representations.