Are summary statistics enough? Evidence for the importance of shape in guiding visual search

Peripheral material perception

Humans can rapidly identify materials, such as wood or leather, even within a complex visual scene. Given a single image, one can easily identify the underlying "stuff," even though a given material can have highly variable appearance; fabric comes in unlimited variations of shape, pattern, color, and smoothness, yet we have little trouble categorizing it as fabric. What visual cues do we use to determine material identity? Prior research suggests that simple "texture" features of an image, such as the power spectrum, capture information about material properties and identity. Few studies, however, have tested richer and biologically motivated models of texture. We compared baseline material classification performance to performance with synthetic textures generated from the Portilla-Simoncelli model and several common image degradations. The textures retain statistical information but are otherwise random. We found that performance with textures and most degradations was well below baseline, suggesting insufficient information to support foveal material perception. Interestingly, modern research suggests that peripheral vision might use a statistical, texture-like representation. In a second set of experiments, we found that peripheral performance is more closely predicted by texture and other image degradations. These findings delineate the nature of peripheral material classification.

Why did Rubens add a parrot to Titian's The Fall of Man? A pictorial manipulation of joint attention

Almost 400 years ago, Rubens copied Titian's The Fall of Man, albeit with important changes. Rubens altered Titian's original composition in numerous ways, including by changing the gaze directions of the depicted characters and adding a striking red parrot to the painting. Here, we quantify the impact of Rubens's choices on the viewer's gaze behavior. We displayed digital copies of Rubens's and Titian's artworks-as well as a version of Rubens's painting with the parrot digitally removed-on a computer screen while recording the eye movements produced by observers during free visual exploration of each image. To assess the effects of Rubens's changes to Titian's composition, we directly compared multiple gaze parameters across the different images. We found that participants gazed at Eve's face more frequently in Rubens's painting than in Titian's. In addition, gaze positions were more tightly focused for the former than for the latter, consistent with different allocations of viewer interest. We also investigated how gaze fixation on Eve's face affected the perceptual visibility of the parrot in Rubens's composition and how the parrot's presence versus its absence impacted gaze dynamics. Taken together, our results demonstrate that Rubens's critical deviations from Titian's painting have powerful effects on viewers' oculomotor behavior.

What the Neuroscience and Psychology of Magic Reveal about Misinformation

When we believe misinformation, we have succumbed to an illusion: our perception or interpretation of the world does not match reality. We often trust misinformation for reasons that are unrelated to an objective, critical interpretation of the available data: Key facts go unnoticed or unreported. Overwhelming information prevents the formulation of alternative explanations. Statements become more believable every time they are repeated. Events are reframed or given "spin" to mislead audiences. In magic shows, illusionists apply similar techniques to convince spectators that false and even seemingly impossible events have happened. Yet, many magicians are "honest liars," asking audiences to suspend their disbelief only during the performance, for the sole purpose of entertainment. Magic misdirection has been studied in the lab for over a century. Psychological research has sought to understand magic from a scientific perspective and to apply the tools of magic to the understanding of cognitive and perceptual processes. More recently, neuroscientific investigations have also explored the relationship between magic illusions and their underlying brain mechanisms. We propose that the insights gained from such studies can be applied to understanding the prevalence and success of misinformation. Here, we review some of the common factors in how people experience magic during a performance and are subject to misinformation in their daily lives. Considering these factors will be important in reducing misinformation and encouraging critical thinking in society.

Negative cues minimize visual search specificity effects

Prior target knowledge (i.e., positive cues) improves visual search performance. However, there is considerable debate about whether distractor knowledge (i.e., negative cues) can guide search. Some studies suggest the active suppression of negatively cued search items, while others suggest the initial capture of attention by negatively cued items. Prior work has used pictorial or specific text cues but has not explicitly compared them. We build on that work by comparing positive and negative cues presented pictorially and as categorical text labels using photorealistic objects and eye movement measures. Search displays contained a target (cued on positive trials), a lure from the target category (cued on negative trials), and four categorically-unrelated distractors. Search performance with positive cues resulted in stronger attentional guidance and faster object recognition for pictorial relative to categorical cues (i.e., a pictorial advantage, suggesting specific visual details afforded by pictorial cues improved search). However, in most search performance metrics, negative cues mitigate the pictorial advantage. Given that the negatively cued items captured attention, generated target guidance but mitigated the pictorial advantage, these results are partly consistent with both existing theories. Specific visual details provided in positive cues produce a large pictorial advantage in all measures, whereas specific visual details in negative cues only produce a small pictorial advantage for object recognition but not for attentional guidance. This asymmetry in the pictorial advantage suggests that the down-weighting of specific negatively cued visual features is less efficient than the up-weighting of specific positively cued visual features.

Mandating Limits on Workload, Duty, and Speed in Radiology

Research has not yet quantified the effects of workload or duty hours on the accuracy of radiologists. With the exception of a brief reduction in imaging studies during the 2020 peak of the COVID-19 pandemic, the workload of radiologists in the United States has seen relentless growth in recent years. One concern is that this increased demand could lead to reduced accuracy. Behavioral studies in species ranging from insects to humans have shown that decision speed is inversely correlated to decision accuracy. A potential solution is to institute workload and duty limits to optimize radiologist performance and patient safety. The concern, however, is that any prescribed mandated limits would be arbitrary and thus no more advantageous than allowing radiologists to self-regulate. Specific studies have been proposed to determine whether limits reduce error, and if so, to provide a principled basis for such limits. This could determine the precise susceptibility of individual radiologists to medical error as a function of speed during image viewing, the maximum number of studies that could be read during a work shift, and the appropriate shift duration as a function of time of day. Before principled recommendations for restrictions are made, however, it is important to understand how radiologists function both optimally and at the margins of adequate performance. This study examines the relationship between interpretation speed and error rates in radiology, the potential influence of artificial intelligence on reading speed and error rates, and the possible outcomes of imposed limits on both caseload and duty hours. This review concludes that the scientific evidence needed to make meaningful rules is lacking and notes that regulating workloads without scientific principles can be more harmful than not regulating at all.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

Understanding Collections of Related Datasets Using Dependent MMD Coresets

Understanding how two datasets differ can help us determine whether one dataset under-represents certain sub-populations, and provides insights into how well models will generalize across datasets. Representative points selected by a maximum mean discrepancy (MMD) coreset can provide interpretable summaries of a single dataset, but are not easily compared across datasets. In this paper, we introduce dependent MMD coresets, a data summarization method for collections of datasets that facilitates comparison of distributions. We show that dependent MMD coresets are useful for understanding multiple related datasets and understanding model generalization between such datasets.

Gaze mechanisms enabling the detection of faint stars in the night sky

For millennia, people have used "averted vision" to improve their detection of faint celestial objects, a technique first documented around 325 BCE. Yet, no studies have assessed gaze location during averted vision to determine what pattern best facilitates perception. Here, we characterized averted vision while recording eye-positions of dark-adapted human participants, for the first time. We simulated stars of apparent magnitudes 3.3 and 3.5, matching their brightness to Megrez (the dimmest star in the Big Dipper) and Tau Ceti. Participants indicated whether each star was visible from a series of fixation locations, providing a comprehensive map of detection performance in all directions. Contrary to prior predictions, maximum detection was first achieved at ~8° from the star, much closer to the fovea than expected from rod-cone distributions alone. These findings challenge the assumption of optimal detection at the rod density peak and provide the first systematic assessment of an age-old facet of human vision.

Towards Interactive Search: Investigating Visual Search in a Novel Real-World Paradigm

An overwhelming majority of studies on visual search and selective attention were conducted using computer screens. There are arguably shortcomings in transferring knowledge from computer-based studies to real-world search behavior as findings are based on viewing static pictures on computer screens. This does not go well with the dynamic and interactive nature of vision in the real world. It is crucial to take visual search research to the real world in order to study everyday visual search processes. The aim of the present study was to develop an interactive search paradigm that can serve as a "bridge" between classical computerized search and everyday interactive search. We based our search paradigm on simple LEGO® bricks arranged on tabletop trays to ensure comparability with classical computerized visual search studies while providing room for easily increasing the complexity of the search environment. We found that targets were grasped slower when there were more distractors (Experiment 1) and there were sizable differences between various search conditions (Experiment 2), largely in line with classical visual search research and revealing similarities to research in natural scenes. Therefore, our paradigm can be seen as a valuable asset complementing visual search research in an environment between computerized search and everyday search.

What do radiologists look for? Advances and limitations of perceptual learning in radiologic search

Supported by guidance from training during residency programs, radiologists learn clinically relevant visual features by viewing thousands of medical images. Yet the precise visual features that expert radiologists use in their clinical practice remain unknown. Identifying such features would allow the development of perceptual learning training methods targeted to the optimization of radiology training and the reduction of medical error. Here we review attempts to bridge current gaps in understanding with a focus on computational saliency models that characterize and predict gaze behavior in radiologists. There have been great strides toward the accurate prediction of relevant medical information within images, thereby facilitating the development of novel computer-aided detection and diagnostic tools. In some cases, computational models have achieved equivalent sensitivity to that of radiologists, suggesting that we may be close to identifying the underlying visual representations that radiologists use. However, because the relevant bottom-up features vary across task context and imaging modalities, it will also be necessary to identify relevant top-down factors before perceptual expertise in radiology can be fully understood. Progress along these dimensions will improve the tools available for educating new generations of radiologists, and aid in the detection of medically relevant information, ultimately improving patient health.

Specifying the precision of guiding features for visual search

Visual search is the task of finding things with uncertain locations. Despite decades of research, the features that guide visual search remain poorly specified, especially in realistic contexts. This study tested the role of two features-shape and orientation-both in the presence and absence of hue information. We conducted five experiments to describe preview-target mismatch effects, decreases in performance caused by differences between the image of the target as it appears in the preview and as it appears in the actual search display. These mismatch effects provide direct measures of feature importance, with larger performance decrements expected for more important features. Contrary to previous conclusions, our data suggest that shape and orientation only guide visual search when color is not available. By varying the probability of mismatch in each feature dimension, we also show that these patterns of feature guidance do not change with the probability that the previewed feature will be invalid. We conclude that the target representations used to guide visual search are much less precise than previously believed, with participants encoding and using color and little else. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The role of scene summary statistics in object recognition

Objects that are semantically related to the visual scene context are typically better recognized than unrelated objects. While context effects on object recognition are well studied, the question which particular visual information of an object's surroundings modulates its semantic processing is still unresolved. Typically, one would expect contextual influences to arise from high-level, semantic components of a scene but what if even low-level features could modulate object processing? Here, we generated seemingly meaningless textures of real-world scenes, which preserved similar summary statistics but discarded spatial layout information. In Experiment 1, participants categorized such textures better than colour controls that lacked higher-order scene statistics while original scenes resulted in the highest performance. In Experiment 2, participants recognized briefly presented consistent objects on scenes significantly better than inconsistent objects, whereas on textures, consistent objects were recognized only slightly more accurately. In Experiment 3, we recorded event-related potentials and observed a pronounced mid-central negativity in the N300/N400 time windows for inconsistent relative to consistent objects on scenes. Critically, inconsistent objects on textures also triggered N300/N400 effects with a comparable time course, though less pronounced. Our results suggest that a scene's low-level features contribute to the effective processing of objects in complex real-world environments.

Occluded information is restored at preview but not during visual search

Objects often appear with some amount of occlusion. We fill in missing information using local shape features even before attending to those objects-a process called amodal completion. Here we explore the possibility that knowledge about common realistic objects can be used to "restore" missing information even in cases where amodal completion is not expected. We systematically varied whether visual search targets were occluded or not, both at preview and in search displays. Button-press responses were longest when the preview was unoccluded and the target was occluded in the search display. This pattern is consistent with a target-verification process that uses the features visible at preview but does not restore missing information in the search display. However, visual search guidance was weakest whenever the target was occluded in the search display, regardless of whether it was occluded at preview. This pattern suggests that information missing during the preview was restored and used to guide search, thereby resulting in a feature mismatch and poor guidance. If this process were preattentive, as with amodal completion, we should have found roughly equivalent search guidance across all conditions because the target would always be unoccluded or restored, resulting in no mismatch. We conclude that realistic objects are restored behind occluders during search target preview, even in situations not prone to amodal completion, and this restoration does not occur preattentively during search.

Five Factors that Guide Attention in Visual Search

How do we find what we are looking for? Fundamental limits on visual processing mean that even when the desired target is in our field of view, we often need to search, because it is impossible to recognize everything at once. Searching involves directing attention to objects that might be the target. This deployment of attention is not random. It is guided to the most promising items and locations by five factors discussed here: Bottom-up salience, top-down feature guidance, scene structure and meaning, the previous history of search over time scales from msec to years, and the relative value of the targets and distractors. Modern theories of search need to specify how all five factors combine to shape search behavior. An understanding of the rules of guidance can be used to improve the accuracy and efficiency of socially-important search tasks, from security screening to medical image perception.

Search performance is better predicted by tileability than presence of a unique basic feature

Traditional models of visual search such as feature integration theory (FIT; Treisman & Gelade, 1980), have suggested that a key factor determining task difficulty consists of whether or not the search target contains a "basic feature" not found in the other display items (distractors). Here we discriminate between such traditional models and our recent texture tiling model (TTM) of search (Rosenholtz, Huang, Raj, Balas, & Ilie, 2012b), by designing new experiments that directly pit these models against each other. Doing so is nontrivial, for two reasons. First, the visual representation in TTM is fully specified, and makes clear testable predictions, but its complexity makes getting intuitions difficult. Here we elucidate a rule of thumb for TTM, which enables us to easily design new and interesting search experiments. FIT, on the other hand, is somewhat ill-defined and hard to pin down. To get around this, rather than designing totally new search experiments, we start with five classic experiments that FIT already claims to explain: T among Ls, 2 among 5s, Q among Os, O among Qs, and an orientation/luminance-contrast conjunction search. We find that fairly subtle changes in these search tasks lead to significant changes in performance, in a direction predicted by TTM, providing definitive evidence in favor of the texture tiling model as opposed to traditional views of search.