A New Multiple Object Awareness Paradigm Shows that Imperfect Knowledge of Object Location Is Still Knowledge

Six ways of failing to see (and why the differences matter)

Sometimes we look but fail to see: our car keys on a cluttered desk, a repeated word in a carefully proofread email, or a motorcycle at an intersection. Wolfe and colleagues present a unifying, mechanistic framework for understanding these "Looked But Failed to See" errors, explaining how such misses arise from natural constraints on human visual processing. Here, we offer a conceptual taxonomy of six distinct ways we might be said to fail to see, and explore: how these relate to processes in Wolfe et al.'s model; how they can be distinguished experimentally; and, why the differences matter.

Modeling mean estimation tasks in within-trial and across-trial contexts

The mean estimation task, which explicitly asks observers to estimate the mean feature value of multiple stimuli, is a fundamental paradigm in research areas such as ensemble coding and cue integration. The current study uses computational models to formalize how observers summarize information in mean estimation tasks. We compare model predictions from our Fidelity-based Integration Model (FIM) and other models on their ability to simulate observed patterns in within-trial weight distribution, across-trial information integration, and set-size effects on mean estimation accuracy. Experiments show non-equal weighting within trials in both sequential and simultaneous mean estimation tasks. Observers implicitly overestimated trial means below the global mean and underestimated trial means above the global mean. Mean estimation performance declined and stabilized with increasing set sizes. FIM successfully simulated all observed patterns, while other models failed. FIM's information sampling structure provides a new way to interpret the capacity limit in visual working memory and sub-sampling strategies. As a model framework, FIM offers task-dependent modeling for various ensemble coding paradigms, facilitating research synthesis across different studies in the literature.

Humans represent the precision and utility of information acquired across fixations

Our environment contains an abundance of objects which humans interact with daily, gathering visual information using sequences of eye-movements to choose which object is best-suited for a particular task. This process is not trivial, and requires a complex strategy where task affordance defines the search strategy, and the estimated precision of the visual information gathered from each object may be used to track perceptual confidence for object selection. This study addresses the fundamental problem of how such visual information is metacognitively represented and used for subsequent behaviour, and reveals a complex interplay between task affordance, visual information gathering, and metacogntive decision making. People fixate higher-utility objects, and most importantly retain metaknowledge about how much information they have gathered about these objects, which is used to guide perceptual report choices. These findings suggest that such metacognitive knowledge is important in situations where decisions are based on information acquired in a temporal sequence.

Finding meaning in "wrong responses": The multiple object-awareness paradigm shows that visual awareness is probabilistic

Visual information that observers perceive and remember at any given moment guides behavior in daily life. However, binary alternative-forced choice responses, often used in visual research, limit the report of the visual information that observers perceive and remember. We used a new multiple object-awareness paradigm where observers can use multiple clicks to find a target. We calculated visual awareness capacity based on the first-attempt accuracy and the total number of clicks, respectively. Results showed that the capacity estimated by the clicks in guessing from N was significantly greater than that estimated by the first-attempt accuracy. Further, analysis found that if observers could not locate the target in their first attempt, they were more likely to click closer to the target or on stimuli that matched its color. In addition, we found that even when observers used the same number of clicks to find a target (2 or 3), the average distance was shorter when observers reported high-level subjective visibility. The findings are compatible with the partial awareness hypothesis, and the visual ensembles and summary statistics hypothesis, which hold that visual awareness is probabilistic. These results also support the visual short-term memory models where many items are stored but with a resolution or noise level that depends on the number of items in memory.

From non-conscious processing to conscious events: a minimalist approach

The minimalist approach that we develop here is a framework that allows to appreciate how non-conscious processing and conscious contents shape human cognition, broadly defined. It is composed of three simple principles. First, cognitive processes are inherently non-conscious, while their inputs and (interim) outputs may be consciously experienced. Second, non-conscious processes and elements of the cognitive architecture prioritize information for conscious experiences. Third, conscious events are composed of series of conscious contents and non-conscious processes, with increased duration leading to more opportunity for processing. The narrowness of conscious experiences is conceptualized here as a solution to the problem of channeling the plethora of non-conscious processes into action and communication processes that are largely serial. The framework highlights the importance of prioritization for consciousness, and we provide an illustrative review of three main factors that shape prioritization-stimulus strength, motivational relevance and mental accessibility. We further discuss when and how this framework (i) is compatible with previous theories, (ii) enables new understandings of established findings and models, and (iii) generates new predictions and understandings.

Did you see it? Robust individual differences in the speed with which meaningful visual stimuli break suppression

Perceptual conscious experiences result from non-conscious processes that precede them. We document a new characteristic of the cognitive system: the speed with which visual meaningful stimuli are prioritized to consciousness over competing noise in visual masking paradigms. In ten experiments (N = 399) we find that an individual's non-conscious visual prioritization speed (NVPS) is ubiquitous across a wide variety of stimuli, and generalizes across visual masks, suppression tasks, and time. We also find that variation in NVPS is unique, in that it cannot be explained by variation in general speed, perceptual decision thresholds, short-term visual memory, or three networks of attention (alerting, orienting and executive). Finally, we find that NVPS is correlated with subjective measures of sensitivity, as they are measured by the Highly Sensitive Person scale. We conclude by discussing the implications of variance in NVPS for understanding individual variance in behavior and the neural substrates of consciousness.

Effectiveness of "rescue saccades" on the accuracy of tracking multiple moving targets: An eye-tracking study on the effects of target occlusions

Occlusion is one of the main challenges in tracking multiple moving objects. In almost all real-world scenarios, a moving object or a stationary obstacle occludes targets partially or completely for a short or long time during their movement. A previous study (Zelinsky & Todor, 2010) reported that subjects make timely saccades toward the object in danger of being occluded. Observers make these so-called “rescue saccades” to prevent target swapping. In this study, we examined whether these saccades are helpful. To this aim, we used as the stimuli recorded videos from natural movement of zebrafish larvae swimming freely in a circular container. We considered two main types of occlusion: object-object occlusions that naturally exist in the videos, and object-occluder occlusions created by adding a stationary doughnut-shape occluder in some videos. Four different scenarios were studied: (1) no occlusions, (2) only object-object occlusions, (3) only object-occluder occlusion, or (4) both object-object and object-occluder occlusions. For each condition, two set sizes (two and four) were applied. Participants’ eye movements were recorded during tracking, and rescue saccades were extracted afterward. The results showed that rescue saccades are helpful in handling object-object occlusions but had no reliable effect on tracking through object-occluder occlusions. The presence of occlusions generally increased visual sampling of the scenes; nevertheless, tracking accuracy declined due to occlusion.

Looking ahead: When do you find the next item in foraging visual search?

Many real-world visual tasks involve searching for multiple instances of a target (e.g., picking ripe berries). What strategies do observers use when collecting items in this type of search? Do they wait to finish collecting the current item before starting to look for the next target, or do they search ahead for future targets? We utilized behavioral and eye-tracking measures to distinguish between these two possibilities in foraging search. Experiment 1 used a color wheel technique in which observers searched for T shapes among L shapes while all items independently cycled through a set of colors. Trials were abruptly terminated, and observers reported both the color and location of the next target that they intended to click. Using observers’ color reports to infer target-finding times, we demonstrate that observers found the next item before the time of the click on the current target. We validated these results in Experiment 2 by recording fixation locations around the time of each click. Experiment 3 utilized a different procedure, in which all items were intermittently occluded during the trial. We then calculated a distribution of when targets were visible around the time of each click, allowing us to infer when they were most likely found. In a fourth and final experiment, observers indicated the locations of multiple future targets after the search was abruptly terminated. Together, our results provide converging evidence to demonstrate that observers can find the next target before collecting the current target and can typically forage one to two items ahead.

Multiple-target tracking in human and machine vision

Humans are able to track multiple objects at any given time in their daily activities—for example, we can drive a car while monitoring obstacles, pedestrians, and other vehicles. Several past studies have examined how humans track targets simultaneously and what underlying behavioral and neural mechanisms they use. At the same time, computer-vision researchers have proposed different algorithms to track multiple targets automatically. These algorithms are useful for video surveillance, team-sport analysis, video analysis, video summarization, and human–computer interaction. Although there are several efficient biologically inspired algorithms in artificial intelligence, the human multiple-target tracking (MTT) ability is rarely imitated in computer-vision algorithms. In this paper, we review MTT studies in neuroscience and biologically inspired MTT methods in computer vision and discuss the ways in which they can be seen as complementary.

Multiple-target tracking (MTT) is a challenging task vital for both a human’s daily life and for many artificial intelligent systems, such as those used for urban traffic control. Neuroscientists are interested in discovering the underlying neural mechanisms that successfully exploit cognitive resources, e.g., spatial attention or memory, during MTT. Computer-vision specialists aim to develop powerful MTT algorithms based on advanced models or data-driven computational methods. In this paper, we review MTT studies from both communities and discuss how findings from cognitive studies can inspire developers to construct higher performing MTT algorithms. Moreover, some directions have been proposed through which MTT algorithms could raise new questions in the cognitive science domain, and answering them can shed light on neural processes underlying MTT.

Neuronal correlates of full and partial visual conscious perception

Stimuli may induce only partial consciousness-an intermediate between null and full consciousness-where the presence but not identity of an object can be reported. The differences in the neuronal basis of full and partial consciousness are poorly understood. We investigated if evoked and oscillatory activity could dissociate full from partial conscious perception. We recorded human cortical activity with magnetoencephalography (MEG) during a visual perception task in which stimulus could be either partially or fully perceived. Partial consciousness was associated with an early increase in evoked activity and theta/low-alpha-band oscillations while full consciousness was also associated with late evoked activity and beta-band oscillations. Full from partial consciousness was dissociated by stronger evoked activity and late increase in theta oscillations that were localized to higher-order visual regions and posterior parietal and prefrontal cortices. Our results reveal both evoked activity and theta oscillations dissociate partial and full consciousness.

What Is the True Capacity of Visual Cognition?

How much can we perceive and remember at a time? Results from various paradigms traditionally show that observers are aware of surprisingly little of the world around them. However, a recent study by Wu and Wolfe (Curr. Biol. 2018;28:3430-3434) uses a novel technique to reveal that observers have more knowledge of the visual world than previously believed.