Situating vision in the world

Individuation of pairs of objects in infancy

Looking-time studies examined whether 11-month-old infants can individuate two pairs of objects using only shape information. In order to test individuation, the object pairs were presented sequentially. Infants were familiarized either with the sequential pairs, disk-triangle/disk-triangle (XY/XY), whose shapes differed within but not across pairs, or with the sequential pairs, disk-disk/triangle-triangle (XX/YY), whose shapes differed across but not within pairs. The XY/XY presentation looked to adults like a single pair of objects presented repeatedly, whereas the XX/YY presentation looked like different pairs of objects. Following familiarization to these displays, infants were given a series of test trials in which the screen was removed, revealing two pairs of objects in one of two outcomes, XYXY or XXYY. On the first test trial, infants familiarized with the identical pairs (XY/XY) apparently expected a single pair to be revealed because they looked longer than infants familiarized with the distinct pairs (XX/YY). Infants who had seen the distinct pairs apparently expected a double pair outcome. A second experiment showed outcomes of a single XY pair. This outcome is unexpected for XX/YY-familiarized infants but expected for XY/XY-familiarized infants, the reverse of Experiment 1. This time looking times were longer for XX/YY infants. Eleven-month-olds appear to be able to represent not just individual objects but also pairs of objects. These results suggest that if they can group the objects into sets, infants may be able to track more objects than their numerosity limit or available working memory slots would normally allow. We suggest possible small exact numerosity representations that would allow tracking of such sets.

A study in the cognition of individuals’ identity: Solving the problem of singular cognition in object and agent tracking

This article compares the ability to track individuals lacking mental states with the ability to track intentional agents. It explains why reference to individuals raises the problem of explaining how cognitive agents track unique individuals and in what sense reference is based on procedures of perceptual-motor and epistemic tracking. We suggest applying the notion of singular-files from theories in perception and semantics to the problem of tracking intentional agents. In order to elucidate the nature of agent-files, three views of the relation between object- and agent-tracking are distinguished: the Independence, Deflationary and Organism-Dependence Views. The correct view is argued to be the latter, which states that perceptual and epistemic tracking of a unique human organism requires tracking both its spatio-temporal object-properties and its agent-properties.

Ape metaphysics: object individuation without language

Developmental research suggests that whereas very young infants individuate objects purely on spatiotemporal grounds, from (at latest) around 1 year of age children are capable of individuating objects according to the kind they belong to and the properties they instantiate. As the latter ability has been found to correlate with language, some have speculated whether it might be essentially language dependent and therefore uniquely human. Existing studies with non-human primates seem to speak against this hypothesis, but fail to present conclusive evidence due to methodological shortcomings. In the present experiments we set out to test non-linguistic object individuation in three great ape species with a refined manual search methodology. Experiment 1 tested for spatiotemporal object individuation: Subjects saw 1 or 2 objects simultaneously being placed inside a box in which they could reach, and then in both conditions only found 1 object. After retrieval of the 1 object, subjects reached again significantly more often when they had seen 2 than when they had seen 1 object. Experiment 2 tested for object individuation according to property/kind information only: Subjects saw 1 object being placed inside the box, and then either found that object (expected) or an object of a different kind (unexpected). Analogously to Experiment 1, after retrieval of the 1 object, subjects reached again significantly more often in the unexpected than in the expected condition. These results thus confirm previous findings suggesting that individuating objects according to their property/kind is neither uniquely human nor essentially language dependent. It remains to be seen, however, whether this kind of object individuation requires sortal concepts as human linguistic thinkers use them, or whether some simpler form of tracking properties is sufficient.

Relational spatial reasoning by a nonhuman: the example of capuchin monkeys

The authors review spontaneous manipulation and spatial problem solving by capuchin monkeys to illuminate the nature of relational reasoning (wherein two or more elements of a problem or situation are considered together to arrive at a course of action) that these monkeys use in goal-directed activity. Capuchin monkeys master problems with one, two, or three spatial relations, and if more than one relation, at least two relations may be managed concurrently. They can master static and dynamic relations and, with sufficient practice, can produce specific spatial relations through both direct and distal action. Examining capuchins' spatial problem-solving behavior with objects in the framework of a spatial relational reasoning model leads to new interpretations of previous studies with these monkeys and other nonhuman animals. The model produces a variety of testable predictions concerning the contribution of relational properties to spatial reasoning. It also provides conceptual linkages with neurological processes and cognitive analyses of physical reasoning. Understanding relational spatial reasoning, including tool use, in a wider view is vital to informed, principled comparison of problem solving and the use of technology across species, across ages within species, and across eras in human prehistory.

Multiple object tracking in people with Williams syndrome and in normally developing children

Multiple object tracking is hypothesized to utilize visual indexes, which may provide rapid, parallel access to a limited number of visual objects, thereby supporting a variety of spatial tasks. We examined whether faulty indexing might play a role in the severe visuospatial deficits found in Williams syndrome. We asked observers to track from one to four targets in a display of eight identical objects. Objects remained stationary (static condition) or moved randomly and independently (moving condition) for 6 s, after which observers pointed to the objects they thought were targets. People with Williams syndrome were impaired in the moving condition, but not the static condition, compared with mental-age-matched control participants. Normal children who were younger than the mental-age-matched control children did not show the same profile as individuals with Williams syndrome, which suggests that the difference between the tasks in Williams syndrome did not reflect simple developmental immaturity. Error analysis revealed that all groups had "slippery" indexes, falsely identifying target neighbors, and further suggested that people with Williams syndrome deploy fewer indexes than do people without this disorder.

Effects of color heterogeneity on subitization

Small numbers of items can be enumerated rapidly and accurately via a process termed subitizing. In two experiments, we examined the effect of target color heterogeneity on subitizing efficiency. In contrast to the findings of Puts and de Weert (1997), we found that observers were no less efficient at subitizing displays containing red and green items than they were at subitizing displays of a single color. We propose that these findings are consistent with subitization operating on items represented within a location master map that codes where objects are but not what they are. The data are discussed in relation to recent findings concerning the detection of single-feature targets and the functional architecture of early visual processing.

Effects of Alzheimer's disease on visual enumeration

Speeded enumeration of visual objects typically produces fast and accurate performance for up to 3 to 4 items (subitization) but slower and less accurate performance thereafter (counting). We investigated enumeration ability in patients with Alzheimer's disease (AD) and in age-matched controls. AD patients were slower overall than controls. The subitizing span was significantly reduced in AD patients compared with controls (2.3 vs 3.5 items) and counting rate was significantly slower (451 vs 349 ms/item). Error rates were similar in the two groups except at numerosity 3, when AD patients made errors but controls did not (consistent with their subitizing spans). Within the AD patient group, several aspects of performance correlated significantly with Mini-Mental State Examination scores. Together, the results provide a striking contrast with studies showing preservation of enumeration ability in normal aging.

The Efficiency of Feature-Based Subitization and Counting

The enumeration of small numbers of objects (approximately 4) proceeds rapidly, accurately, and with little effort via a process termed subitization. Four experiments examined whether it was possible to subitize the number of features rather than objects present in a display. Overall, the findings showed that when features are presented randomly and are uncorrelated with object numerosity, efficient enumeration is not possible. This suggests that the visual system does not have parallel access to multiple feature maps and that subitization processes operate exclusively on representations coding the locations of objects. The data are discussed with respect to theories of visual enumeration and search.

Visual cognition: a new look at the two-visual systems model

In this paper, we argue that no valid comparison between visual representations can arise unless provision is made for three critical properties: their direction of fit, their direction of causation and the level of their conceptual content. The conceptual content in turn is a function of the level of processing. Representations arising from earlier stages of processing of visual input have very little or no conceptual content. Higher order representations get their conceptual content from the connections between visual cognition and other parts of the human cognitive system. The two other critical properties of visual representations are their mind/world direction of fit and their mind/world direction of causation. The output of the semantic processing of visual input has a full mind-to-world direction of fit and a full world-to-mind direction of causation: it visually registers the way the world is and is caused by what it represents. The output of the pragmatic processing yields information for the benefit of intentions, which clearly have a world-to-mind direction of fit and a mind-to-world direction of causation. An intention is both the representation of a goal and a cause of the transformation of a goal into a fact. These properties segregate representations specialized for perception from those specialized for action. Perception implies comparison between simultaneously represented and analyzed objects: hence, object perception presupposes the representation of spatial relationships among objects in a coordinate system independent from the perceiver. Spatial relationships carry cues for attributing meaning to an object, so that their processing is actually part of semantic processing of visual information. These considerations lead to a re-evaluation of the role of the two classical pathways of the human visual system: the ventral and the dorsal cortical pathways. The parietal lobe, which has been identified with the dorsal pathway, cannot be considered as a unitary entity with a single function. The superior parietal lobule carries visuomotor processing, a non-lateralized process. The right inferior parietal lobule contributes to the perception of spatial relationships, a process with a mind-to-world direction of fit and a world-to-mind direction of causation. Finally, the left inferior parietal lobule contributes to still another type of representation, related to visually goal-directed action, i.e., with both a world-to-mind direction of fit and a mind-to-world direction of causation.

Event files: feature binding in and across perception and action

The primate brain codes perceived events in a distributed fashion, which raises the question of how the codes referring to the same event are related to each other. Recent findings suggest that they are integrated into 'object files', episodic bindings of object-related information. However, the problem of integrating distributed codes is not restricted to perception but applies to action planning and sensorimotor processing as well. Here I argue that the brain addresses these problems by creating multi-layered networks of bindings - 'event files' - that temporarily link codes of perceptual events, the current task context, and the actions performed therein. These bindings produce systematic but often surprising and counter-intuitive interactions between, and impairments in, perception and action planning.

Biocultural orchestration of developmental plasticity across levels: the interplay of biology and culture in shaping the mind and behavior across the life span

The author reviews reemerging coconstructive conceptions of development and recent empirical findings of developmental plasticity at different levels spanning several fields of developmental and life sciences. A cross-level dynamic biocultural coconstructive framework is endorsed to understand cognitive and behavioral development across the life span. This framework integrates main conceptions of earlier views into a unifying frame, viewing the dynamics of life span development as occurring simultaneously within different time scales (i.e., moment-to-moment microgenesis, life span ontogeny, and human phylogeny) and encompassing multiple levels (i.e., neurobiological, cognitive, behavioral, and sociocultural). Viewed through this metatheoretical framework, new insights of potential interfaces for reciprocal cultural and experiential influences to be integrated with behavioral genetics and cognitive neuroscience research can be more easily prescribed.

Return of the mental image: are there really pictures in the brain?

In the past decade there has been renewed interest in the study of mental imagery. Emboldened by new findings from neuroscience, many people have revived the idea that mental imagery involves a special format of thought, one that is pictorial in nature. But the evidence and the arguments that exposed deep conceptual and empirical problems in the picture theory over the past 300 years have not gone away. I argue that the new evidence from neural imaging and clinical neuropsychology does little to justify this recidivism because it does not address the format of mental images. I also discuss some reasons why the picture theory is so resistant to counterarguments and suggest ways in which non-pictorial theories might account for the apparent spatial nature of images.

Does partial difficult search help difficult search?

Olds, Cowan, and Jolicoeur (2000a, 2000b) showed that exposure to a display that affords pop-out search a target among distractors of only one color) can assist processing of a related display that requires difficult search. They added distractors of an additional color to the initial simple display and analyzed response time distributions to show that exposure to the initial display aided subsequent search in the difficult portion (this finding was called search assistance). To test whether search assistance depends on perceptual grouping of the initial items, we presented initial items that were more difficult to group (two colors of distractors, instead of just one). The target appeared (on 50% of the trials) among distractors of two colors, and then after a delay, more distractors of those two colors were added to the display. Exposure to the initial easier portion of the display did not assist processing of the second portion of the display when the initial display contained a large number of items; we found tentative evidence for assistance with small numbers of initial items. In the Olds et al. (2000a, 2000b) studies, it was easy to group the initial distractor items, because they were all the same color. In contrast, in the present study, it was difficult to group the heterogeneous initial distractor items. Search assistance is found only when initial item grouping is relatively easy, and thus we conclude that search assistance depends on grouping.

Eye movements and visual memory: detecting changes to saccade targets in scenes

Saccade-contingent change detection provides a powerful tool for investigating scene representation and scene memory. In the present study, critical objects presented within color images of naturalistic scenes were changed during a saccade toward or away from the target. During the saccade,the critical object was changed to another object type, to a visually different token of the same object type, or was deleted from the scene. There were three main results. First, the deletion of a saccade target was special: Detection performance for saccade target deletions was very good, and this level of performance did not decline with the amplitude of the saccade. In contrast, detection of type and token changes at the saccade target, and of all changes including deletions at a location that had just been fixated but was not the saccade target, decreased as the amplitude of the saccade increased. Second, detection performance for type and token changes, both when the changing object was the target of the saccade and when the object had just been fixated but was not the saccade target, was well above chance. Third, mean gaze durations were reliably elevated for those trials in which the change was not overtly detected. The results suggest that the presence of the saccade target plays a special role in trassaccadic integration, and together with other recent findings, suggest more generally that a relatively rich scene representation is retained across saccades and stored in visual memory.

Engagement of gaze in capturing targets for future sequential manual actions

We investigated the role of saccadic gaze fixations in encoding target locations for planning a future manual task consisting of a sequence of discrete target-oriented actions. We hypothesized that fixations of the individual targets are necessary for accurate encoding of target locations and that there is a transfer of sequence information from visual encoding to manual recall. Subjects viewed four targets presented at random positions on a screen. After various delays following target extinction, the subjects marked the remembered target locations on the screen with the tip of a hand-held stick. When the targets were presented simultaneously among distracting elements, the overall accuracy of marking increased with presentation time and total number of targets fixated because the subjects had to serially fixate the individual targets to locate them. Without distractors, the marking accuracy was similarly high regardless of duration of target presentation (0.25-8 s) and number of targets fixated; it was comparable to that with distractors when all four targets had been fixated. This indicates parallel encoding of target locations largely based on peripheral vision. Location memory was stable in these tasks over the delay periods investigated (0.5-8 s). With parallel encoding there was a "shrinkage" in the visuomotor transformation, i.e., the distances between the markings were systematically smaller than the corresponding inter-target distances. When the targets were presented sequentially without distractors, marking accuracy improved with the total number of targets fixated and shrinkage in the visuomotor transformation occurred only with parallel encoding, i.e., when subjects did not fixate the targets. In all experimental conditions for trials in which targets were fixated during encoding, there was little correspondence between the marking sequence and the sequence in which the targets were fixated. We conclude that subjects benefit from fixating targets for subsequent target-oriented manual actions when the targets are presented among distractors and when presented sequentially; when distinct targets are presented simultaneously against a blank background, they are efficiently encoded in parallel largely by peripheral vision.

Conceiving of entities as objects and as stuff

What is the difference between conceiving of an entity as an object of some kind and conceiving of it as an amount of solid stuff? We propose that the difference lies in how one thinks about the entity's structure. Object construals require thinking of the structure as being nonarbitrary, whereas substance construals require thinking of the structure as being arbitrary. We report six experiments that provide empirical support for this proposal. Regularity of structure, repetition of structure, and the existence of structure-dependent functions, all of which provide reasons to consider the structure of an entity to be nonarbitrary, were shown to bias participants towards object construals. We also discuss how the proposed account of what it means to construe an entity as an object or as some stuff can account for a range of findings in the literature on lexical development. These include the relation between cohesiveness and ontological category, shape and ontological category, and complexity of shape and ontological category. Finally, we discuss the nature of construals and the relation of object and substance construals to the physical, design, and intentional stances.

The spatial resolution of visual attention

Two tasks were used to evaluate the grain of visual attention, the minimum spacing at which attention can select individual items. First, observers performed a tracking task at many viewing distances. When the display subtended less than 1 degrees in size, tracking was no longer possible even though observers could resolve the items and their motions: The items were visible but could not be individuated one from the other. The limiting size for selection was roughly the same whether tracking one or three targets, suggesting that the resolution limit acts independently of the capacity limit of attention. Second, the closest spacing that still allowed individuation of single items in dense, static displays was examined. This critical spacing was about 50% coarser in the radial direction compared to the tangential direction and was coarser in the upper as opposed to the lower visual field. The results suggest that no more than about 60 items can be arrayed in the central 30 degrees of the visual field while still allowing attentional access to each individually. Our data show that selection has a coarse grain, much coarser than visual resolution. These measures of the resolution of attention are based solely on the selection of location and are not confounded with preattentive feature interactions that may contribute to measures from flanker and crowding tasks. The results suggest that the parietal area is the most likely locus of this selection mechanism and that it acts by pointing to the spatial coordinates (or cortical coordinates) of items of interest rather than by holding a representation of the items themselves.

The in vivo/in vitro approach to cognition: the case of analogy

A new way of investigating cognition is proposed that combines naturalistic (in vivo) and experimental (in vitro) methods into a single approach. Investigating analogical reasoning by scientists 'live' at laboratory meetings (in vivo) we found that the types of analogies they use change with their goals. Unlike subjects in many cognitive 'reminding' experiments, scientists frequently used structural similarities in their analogies, as opposed to superficial similarities. By subsequently using the in vitro approach, we found that the use of structural information was associated with being asked to generate analogies and with the use of complex scenarios. Similar results are obtained in other areas of reasoning such as hypothesis testing and causal reasoning.

Visual indexes, preconceptual objects, and situated vision

This paper argues that a theory of situated vision, suited for the dual purposes of object recognition and the control of action, will have to provide something more than a system that constructs a conceptual representation from visual stimuli: it will also need to provide a special kind of direct (preconceptual, unmediated) connection between elements of a visual representation and certain elements in the world. Like natural language demonstratives (such as 'this' or 'that') this direct connection allows entities to be referred to without being categorized or conceptualized. Several reasons are given for why we need such a preconceptual mechanism which individuates and keeps track of several individual objects in the world. One is that early vision must pick out and compute the relation among several individual objects while ignoring their properties. Another is that incrementally computing and updating representations of a dynamic scene requires keeping track of token individuals despite changes in their properties or locations. It is then noted that a mechanism meeting these requirements has already been proposed in order to account for a number of disparate empirical phenomena, including subitizing, search-subset selection and multiple object tracking (Pylyshyn et al., Canadian Journal of Experimental Psychology 48(2) (1994) 260). This mechanism, called a visual index or FINST, is briefly discussed and it is argued that viewing it as performing a demonstrative or preconceptual reference function has far-reaching implications not only for a theory of situated vision, but also for suggesting a new way to look at why the primitive individuation of visual objects, or proto-objects, is so central in computing visual representations. Indexing visual objects is also, according to this view, the primary means for grounding visual concepts and is a potentially fruitful way to look at the problem of visual integration across time and across saccades, as well as to explain how infants' numerical capacity might arise.

Object-oriented models of cognitive processing

Information-processing models of vision and cognition are inspired by procedural programming languages. Models that emphasize object-based representations are closely related to object-oriented programming languages. The concepts underlying object-oriented languages provide a theoretical framework for cognitive processing that differs markedly from that offered by procedural languages. This framework is well-suited to a system designed to deal flexibly with discrete objects and unpredictable events in the world.

Extending the classical view of representation

Representation has always been a central part of models in cognitive science, but this idea has come under attack. Researchers advocating the alternative approaches of perceptual symbol systems, situated action, embodied cognition, and dynamical systems have argued against central assumptions of the classical representational approach to mind. We review the core assumptions of the representational view and these four suggested alternatives. We argue that representation should remain a core part of cognitive science, but that the insights from these alternative approaches must be incorporated into models of cognitive processing.