Evidence of fixed capacity in visual object categorization

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Rethinking simultaneous suppression in visual cortex via compressive spatiotemporal population receptive fields

When multiple visual stimuli are presented simultaneously in the receptive field, the neural response is suppressed compared to presenting the same stimuli sequentially. The prevailing hypothesis suggests that this suppression is due to competition among multiple stimuli for limited resources within receptive fields, governed by task demands. However, it is unknown how stimulus-driven computations may give rise to simultaneous suppression. Using fMRI, we find simultaneous suppression in single voxels, which varies with both stimulus size and timing, and progressively increases up the visual hierarchy. Using population receptive field (pRF) models, we find that compressive spatiotemporal summation rather than compressive spatial summation predicts simultaneous suppression, and that increased simultaneous suppression is linked to larger pRF sizes and stronger compressive nonlinearities. These results necessitate a rethinking of simultaneous suppression as the outcome of stimulus-driven compressive spatiotemporal computations within pRFs, and open new opportunities to study visual processing capacity across space and time.

Divided attention effects in visual search are caused by objects not by space

Divided attention effects have been observed across a variety of stimuli and perceptual tasks, which have given rise to both object-based and space-based theories of divided attention. Object-based theories assert that processing information from multiple objects is limited, whereas space-based theories assert that processing information from multiple locations is limited. Extant results in the literature are collectively inconsistent with both simple object-based theories and simple space-based theories of divided attention. Using a visual search task with the extended simultaneous-sequential method to reveal capacity limitations, we found evidence of limited-capacity processing of object properties and unlimited-capacity processing of feature contrast. We found no evidence of a separate spatial limitation. A multiple pathway processing theory can account for these and a large body of previous results. According to this theory, tasks that require object processing must follow a limited-capacity pathway and therefore incur divided attention effects. Tasks that depend on only feature contrast can follow a separate unlimited-capacity processing pathway and therefore do not incur divided attention effects.

A major role for retrieval and/or comparison in the set-size effects of change detection

Set-size effects in change detection have been attributed to capacity limits in a variety of processes, including perception, memory encoding, memory storage, memory retrieval, comparison, and decision. In this study, we investigated the locus of the effect of increasing set size from 1 to 2. The task was to detect a 90 degree change in the orientation of 1 or 2 briefly presented Gabor patterns in noise. To measure purely attentional effects and not another phenomena, such as crowding, a precue was used to manipulate relevant set size while keeping the display constant. The locus of the capacity limit was determined by varying when observers were cued to a single relevant stimulus. To begin, we measured the baseline set-size effect for change detection. Next, a dual-task procedure and a 100% valid postcue was added to test for an effect of decision: This modification did not reliably change the set-size effects. In the critical experiments, a 100% valid cue was provided during the retention interval between displays, or only one stimulus was presented in the second display (local recognition). For both of these conditions, there was only a relatively small set-size effect. These results are consistent with the bulk of capacity limits being in memory retrieval or comparison and not in perception, memory encoding, or memory storage.

To Grasp the World at a Glance: The Role of Attention in Visual and Semantic Associative Processing

Associative relations among words, concepts and percepts are the core building blocks of high-level cognition. When viewing the world ‘at a glance’, the associative relations between objects in a scene, or between an object and its visual background, are extracted rapidly. The extent to which such relational processing requires attentional capacity, however, has been heavily disputed over the years. In the present manuscript, I review studies investigating scene–object and object–object associative processing. I then present a series of studies in which I assessed the necessity of spatial attention to various types of visual–semantic relations within a scene. Importantly, in all studies, the spatial and temporal aspects of visual attention were tightly controlled in an attempt to minimize unintentional attention shifts from ‘attended’ to ‘unattended’ regions. Pairs of stimuli—either objects, scenes or a scene and an object—were briefly presented on each trial, while participants were asked to detect a pre-defined target category (e.g., an animal, a nonsense shape). Response times (RTs) to the target detection task were registered when visual attention spanned both stimuli in a pair vs. when attention was focused on only one of two stimuli. Among non-prioritized stimuli that were not defined as to-be-detected targets, findings consistently demonstrated rapid associative processing when stimuli were fully attended, i.e., shorter RTs to associated than unassociated pairs. Focusing attention on a single stimulus only, however, largely impaired this relational processing. Notably, prioritized targets continued to affect performance even when positioned at an unattended location, and their associative relations with the attended items were well processed and analyzed. Our findings portray an important dissociation between unattended task-irrelevant and task-relevant items: while the former require spatial attentional resources in order to be linked to stimuli positioned inside the attentional focus, the latter may influence high-level recognition and associative processes via feature-based attentional mechanisms that are largely independent of spatial attention.

Is there a serial bottleneck in visual object recognition?

Divided attention has little effect for simple tasks, such as luminance detection, but it has large effects for complex tasks, such as semantic categorization of masked words. Here, we asked whether the semantic categorization of visual objects shows divided attention effects as large as those observed for words, or as small as those observed for simple feature judgments. Using a dual-task paradigm with nameable object stimuli, performance was compared with the predictions of serial and parallel models. At the extreme, parallel processes with unlimited capacity predict no effect of divided attention; alternatively, an all-or-none serial process makes two predictions: a large divided attention effect (lower accuracy for dual-task trials, compared to single-task trials) and a negative response correlation in dual-task trials (a given response is more likely to be incorrect when the response about the other stimulus is correct). These predictions were tested in two experiments examining object judgments. In both experiments, there was a large divided attention effect and a small negative correlation in responses. The magnitude of these effects was larger than for simple features, but smaller than for words. These effects were consistent with serial models, and rule out some but not all parallel models. More broadly, the results help establish one of the first examples of likely serial processing in perception.

Vision at a glance: The role of attention in processing object-to-object categorical relations

When viewing a scene at a glance, the visual and categorical relations between objects in the scene are extracted rapidly. In the present study, the involvement of spatial attention in the processing of such relations was investigated. Participants performed a category detection task (e.g., "is there an animal") on briefly flashed object pairs. In one condition, visual attention spanned both stimuli, and in another, attention was focused on a single object while its counterpart object served as a task-irrelevant distractor. The results showed that when participants attended to both objects, a categorical relation effect was obtained (Exp. 1). Namely, latencies were shorter to objects from the same category than to those from different superordinate categories (e.g., clothes, vehicles), even if categories were not prioritized by the task demands. Focusing attention on only one of two stimuli, however, largely eliminated this effect (Exp. 2). Some relational processing was seen when categories were narrowed to the basic level and were highly distinct from each other (Exp. 3), implying that categorical relational processing necessitates attention, unless the unattended input is highly predictable. Critically, when a prioritized (to-be-detected) object category, positioned in a distractor's location, differed from an attended object, a robust distraction effect was consistently observed, regardless of category homogeneity and/or of response conflict factors (Exp. 4). This finding suggests that object relations that involve stimuli that are highly relevant to the task settings may survive attentional deprivation at the distractor location. The involvement of spatial attention in object-to-object categorical processing is most critical in situations that include wide categories that are irrelevant to one's current goals.

The Influence of Each Facial Feature on How We Perceive and Interpret Human Faces

Facial information is processed by our brain in such a way that we immediately make judgments about, for example, attractiveness or masculinity or interpret personality traits or moods of other people. The appearance of each facial feature has an effect on our perception of facial traits. This research addresses the problem of measuring the size of these effects for five facial features (eyes, eyebrows, nose, mouth, and jaw). Our proposal is a mixed feature-based and image-based approach that allows judgments to be made on complete real faces in the categorization tasks, more than on synthetic, noisy, or partial faces that can influence the assessment. Each facial feature of the faces is automatically classified considering their global appearance using principal component analysis. Using this procedure, we establish a reduced set of relevant specific attributes (each one describing a complete facial feature) to characterize faces. In this way, a more direct link can be established between perceived facial traits and what people intuitively consider an eye, an eyebrow, a nose, a mouth, or a jaw. A set of 92 male faces were classified using this procedure, and the results were related to their scores in 15 perceived facial traits. We show that the relevant features greatly depend on what we are trying to judge. Globally, the eyes have the greatest effect. However, other facial features are more relevant for some judgments like the mouth for happiness and femininity or the nose for dominance.

Automatic classification of human facial features based on their appearance

Classification or typology systems used to categorize different human body parts have existed for many years. Nevertheless, there are very few taxonomies of facial features. Ergonomics, forensic anthropology, crime prevention or new human-machine interaction systems and online activities, like e-commerce, e-learning, games, dating or social networks, are fields in which classifications of facial features are useful, for example, to create digital interlocutors that optimize the interactions between human and machines. However, classifying isolated facial features is difficult for human observers. Previous works reported low inter-observer and intra-observer agreement in the evaluation of facial features. This work presents a computer-based procedure to automatically classify facial features based on their global appearance. This procedure deals with the difficulties associated with classifying features using judgements from human observers, and facilitates the development of taxonomies of facial features. Taxonomies obtained through this procedure are presented for eyes, mouths and noses.

Primary or secondary tasks? Dual-task interference between cyclist hazard perception and cadence control using cross-modal sensory aids with rider assistance bike computers

This research investigated the risks involved in bicycle riding while using various sensory modalities to deliver training information. To understand the risks associated with using bike computers, this study evaluated hazard perception performance through lab-based simulations of authentic riding conditions. Analysing hazard sensitivity (d') of signal detection theory, the rider's response time, and eye glances provided insights into the risks of using bike computers. In this study, 30 participants were tested with eight hazard perception tasks while they maintained a cadence of 60 ± 5 RPM and used bike computers with different sensory displays, namely visual, auditory, and tactile feedback signals. The results indicated that synchronously using different sense organs to receive cadence feedback significantly affects hazard perception performance; direct visual information leads to the worst rider distraction, with a mean sensitivity to hazards (d') of -1.03. For systems with multiple interacting sensory aids, auditory aids were found to result in the greatest reduction in sensitivity to hazards (d' mean = -0.57), whereas tactile sensory aids reduced the degree of rider distraction (d' mean = -0.23). Our work complements existing work in this domain by advancing the understanding of how to design devices that deliver information subtly, thereby preventing disruption of a rider's perception of road hazards.

The perceptual processing of fused multi-spectral imagery

Multi-spectral imagery can enhance decision-making by supplying multiple complementary sources of information. However, overloading an observer with information can deter decision-making. Hence, it is critical to assess multi-spectral image displays using human performance. Accuracy and response times (RTs) are fundamental for assessment, although without sophisticated empirical designs, they offer little information about why performance is better or worse. Systems factorial technology (SFT) is a framework for study design and analysis that examines observers' processing mechanisms, not just overall performance. In the current work, we use SFT to compare a display with two sensor images alongside each another with a display in which there is a single composite image. In our first experiment, the SFT results indicated that both display approaches suffered from limited workload capacity and more so for the composite imagery. In the second experiment, we examined the change in observer performance over the course of multiple days of practice. Participants' accuracy and RTs improved with training, but their capacity limitations were unaffected. Using SFT, we found that the capacity limitation was not due to an inefficient serial examination of the imagery by the participants. There are two clear implications of these results: Observers are less efficient with multi-spectral images than single images and the side-by-side display of source images is a viable alternative to composite imagery. SFT was necessary for these conclusions because it provided an appropriate mechanism for comparing single-source images to multi-spectral images and because it ruled out serial processing as the source of the capacity limitation.

Information processing capacity in psychopathy: Effects of anomalous attention

Hamilton and colleagues (2015) recently proposed that an integrative deficit in psychopathy restricts simultaneous processing, thereby leaving fewer resources available for information encoding, narrowing the scope of attention, and undermining associative processing. The current study evaluated this parallel processing deficit proposal using the Simultaneous-Sequential paradigm. This investigation marks the first a priori test of the Hamilton et al.'s theoretical framework. We predicted that psychopathy would be associated with inferior performance (as indexed by lower accuracy and longer response time) on trials requiring simultaneous processing of visual information relative to trials necessitating sequential processing. Results were consistent with these predictions, supporting the proposal that psychopathy is characterized by a reduced capacity to process multicomponent perceptual information concurrently. We discuss the potential implications of impaired simultaneous processing for the conceptualization of the psychopathic deficit. (PsycINFO Database Record

Losing the trees for the forest in dynamic visual search

Representing temporally continuous objects across change (e.g., in position) requires integration of newly sampled visual information with existing object representations. We asked what consequences representational updating has for visual search. In this dynamic visual search task, bars rotated around their central axis. Observers searched for a single episodic target state (oblique bar among vertical and horizontal bars). Search was efficient when the target display was presented as an isolated static display. Performance declined to near chance, however, when the same display was a single state of a dynamically changing scene (Experiment 1), as though temporal selection of the target display from the stream of stimulation failed entirely (Experiment 3). The deficit is attributable neither to masking (Experiment 2), nor to a lack of temporal marker for the target display (Experiment 4). The deficit was partially reduced by visually marking the target display with unique feature information (Experiment 5). We suggest that representational updating causes a loss of access to instantaneous state information in search. Similar to spatially crowded displays that are perceived as textures (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), we propose a temporal version of the trees (instantaneous orientation information) being lost for the forest (rotating bars).

The capacity limitations of orientation summary statistics

The simultaneous-sequential method was used to test the processing capacity of establishing mean orientation summaries. Four clusters of oriented Gabor patches were presented in the peripheral visual field. One of the clusters had a mean orientation that was tilted either left or right, whereas the mean orientations of the other three clusters were roughly vertical. All four clusters were presented at the same time in the simultaneous condition, whereas the clusters appeared in temporal subsets of two in the sequential condition. Performance was lower when the means of all four clusters had to be processed concurrently than when only two had to be processed in the same amount of time. The advantage for establishing fewer summaries at a given time indicates that the processing of mean orientation engages limited-capacity processes (Exp. 1). This limitation cannot be attributed to crowding, low target-distractor discriminability, or a limited-capacity comparison process (Exps. 2 and 3). In contrast to the limitations of establishing multiple summary representations, establishing a single summary representation unfolds without interference (Exp. 4). When interpreted in the context of recent work on the capacity of summary statistics, these findings encourage a reevaluation of the view that early visual perception consists of creating summary statistic representations that unfold independently across multiple areas of the visual field.

The bandwidth of consolidation into visual short-term memory (VSTM) depends on the visual feature

We investigated the nature of the bandwidth limit in the consolidation of visual information into visual short-term memory. In the first two experiments, we examined whether previous results showing differential consolidation bandwidth for color and orientation resulted from methodological differences by testing the consolidation of color information with methods used in prior orientation experiments. We briefly presented two color patches with masks, either sequentially or simultaneously, followed by a location cue indicating the target. Participants identified the target color via button-press (Experiment 1) or by clicking a location on a color wheel (Experiment 2). Although these methods have previously demonstrated that two orientations are consolidated in a strictly serial fashion, here we found equivalent performance in the sequential and simultaneous conditions, suggesting that two colors can be consolidated in parallel. To investigate whether this difference resulted from different consolidation mechanisms or a common mechanism with different features consuming different amounts of bandwidth, Experiment 3 presented a color patch and an oriented grating either sequentially or simultaneously. We found a lower performance in the simultaneous than the sequential condition, with orientation showing a larger impairment than color. These results suggest that consolidation of both features share common mechanisms. However, it seems that color requires less information to be encoded than orientation. As a result two colors can be consolidated in parallel without exceeding the bandwidth limit, whereas two orientations or an orientation and a color exceed the bandwidth and appear to be consolidated serially.

Summary statistics of size: fixed processing capacity for multiple ensembles but unlimited processing capacity for single ensembles

We assessed the processing capacity of establishing statistical summary representations (SSRs) of mean size in visual displays using the simultaneous-sequential method. Four clusters of stimuli, each composed of several circles with various diameters, were presented around fixation. Observers searched for the cluster with the largest or smallest mean size. In the simultaneous condition, all four clusters were presented concurrently; in the sequential condition, the clusters appeared two at a time. We found that the processing capacity of SSRs for multiple ensembles was as extreme as a fixed-rate bottleneck process (Experiment 1). A control experiment confirmed that this was not caused by having to compare the results of multiple averaging processes (Experiment 2). In contrast to computing SSRs across ensembles, computing SSRs for a single ensemble using the same stimuli was consistent with unlimited-capacity processing (Experiment 3). Contrary to existing claims, summary representations appear to be extracted independently for items within single ensembles but not multiple ensembles. A developing understanding of capacity limitations in perceptual processing is discussed.

Evidence for a fixed capacity limit in attending multiple locations

A classic question concerns whether humans can attend multiple locations or objects at once. Although it is generally agreed that the answer to this question is "yes," the limits on this ability are subject to extensive debate. According to one view, attentional resources can be flexibly allocated to a variable number of locations, with an inverse relationship between the number of selected locations and the quality of information processing at each location. Alternatively, these resources might be quantized in a "discrete" fashion that enables concurrent access to a small number of locations. Here, we report a series of experiments comparing these alternatives. In each experiment, we cued participants to attend a variable number of spatial locations and asked them to report the orientation of a single, briefly presented target. In all experiments, participants' orientation report errors were well-described by a model that assumes a fixed upper limit in the number of locations that can be attended. Conversely, report errors were poorly described by a flexible-resource model that assumes no fixed limit on the number of locations that can be attended. Critically, we showed that these discrete limits were predicted by cue-evoked neural activity elicited before the onset of the target array, suggesting that performance was limited by selection processes that began prior to subsequent encoding and memory storage. Together, these findings constitute novel evidence supporting the hypothesis that human observers can attend only a small number of discrete locations at an instant.

Evidence of unlimited-capacity surface completion

Capacity limitations of perceptual surface completion were assessed using a simultaneous-sequential method. Observers searched among multiple surfaces requiring perceptual completion in front of other objects (modal completion) or behind other objects (amodal completion). In the simultaneous condition, all surfaces were presented at once, whereas in the sequential condition, they appeared in subsets of 2 at a time. For both modal and amodal surface completion, performance was as good in the simultaneous condition as in the sequential condition, indicating that surface completion unfolds independently for multiple surfaces across the visual field (i.e., has unlimited capacity). We confirmed this was due to the formation of surfaces defined by the pacmen inducers, and not simply to the detection of individual features of the pacmen inducers. These results provide evidence that surface-completion processes can be engaged and unfold independently for multiple surfaces across the visual field. In other words, surface completion can occur through unlimited-capacity processes. These results contribute to a developing understanding of capacity limitations in perceptual processing more generally.

Serial consolidation of orientation information into visual short-term memory

Previous research suggests that there is a limit to the rate at which items can be consolidated in visual short-term memory (VSTM). This limit could be due to either a serial or a limited-capacity parallel process. Historically, it has proven difficult to distinguish between these two types of processes. In the present experiment, we took a novel approach that allowed us to do so. Participants viewed two oriented gratings either sequentially or simultaneously and reported one of the gratings' orientation via method of adjustment. Performance was worse for the simultaneous than for the sequential condition. We fit the data with a mixture model that assumes performance is limited by a noisy memory representation plus random guessing. Critically, the serial and limited-capacity parallel processes made distinct predictions regarding the model's guessing and memory-precision parameters. We found strong support for a serial process, which implies that one can consolidate only a single orientation into VSTM at a time.

Divided attention limits perception of 3-D object shapes

Can one perceive multiple object shapes at once? We tested two benchmark models of object shape perception under divided attention: an unlimited-capacity and a fixed-capacity model. Under unlimited-capacity models, shapes are analyzed independently and in parallel. Under fixed-capacity models, shapes are processed at a fixed rate (as in a serial model). To distinguish these models, we compared conditions in which observers were presented with simultaneous or sequential presentations of a fixed number of objects (The extended simultaneous-sequential method: Scharff, Palmer, & Moore, 2011a, 2011b). We used novel physical objects as stimuli, minimizing the role of semantic categorization in the task. Observers searched for a specific object among similar objects. We ensured that non-shape stimulus properties such as color and texture could not be used to complete the task. Unpredictable viewing angles were used to preclude image-matching strategies. The results rejected unlimited-capacity models for object shape perception and were consistent with the predictions of a fixed-capacity model. In contrast, a task that required observers to recognize 2-D shapes with predictable viewing angles yielded an unlimited capacity result. Further experiments ruled out alternative explanations for the capacity limit, leading us to conclude that there is a fixed-capacity limit on the ability to perceive 3-D object shapes.

Statistical measures for workload capacity analysis

A critical component of how we understand a mental process is given by measuring the effect of varying the workload. The capacity coefficient (Townsend & Nozawa, 1995; Townsend & Wenger, 2004) is a measure on response times for quantifying changes in performance due to workload. Despite its precise mathematical foundation, until now rigorous statistical tests have been lacking. In this paper, we demonstrate statistical properties of the components of the capacity measure and propose a significance test for comparing the capacity coefficient to a baseline measure or two capacity coefficients to each other.

Exploring the relationship between perceptual learning and top-down attentional control

Here, we review the role of top-down attention in both the acquisition and the expression of perceptual learning, as well as the role of learning in more efficiently guiding attentional modulations. Although attention often mediates learning at the outset of training, many of the characteristic behavioral and neural changes associated with learning can be observed even when stimuli are task irrelevant and ignored. However, depending on task demands, attention can override the effects of perceptual learning, suggesting that even if top-down factors are not strictly necessary to observe learning, they play a critical role in determining how learning-related changes in behavior and neural activity are ultimately expressed. In turn, training may also act to optimize the effectiveness of top-down attentional control by improving the efficiency of sensory gain modulations, regulating intrinsic noise, and altering the read-out of sensory information.