Basic-level categorization of intermediate complexity fragments reveals top-down effects of expertise in visual perception

Task-dependent neural representations of visual object categories

What do we perceive in a glance of an object? If we are questioned about it, will our perception be affected? How does the task demand influence visual processing in the brain and, consequently, our behaviour? To address these questions, we conducted an object categorisation experiment with three tasks, one at the superordinate level ('animate/inanimate') and two at the basic levels ('face/body' and 'animal/human face') along with a passive task in which participants were not required to categorise objects. To control bottom-up information and eliminate the effect of sensory-driven dissimilarity, we used a particular set of animal face images as the identical target stimuli across all tasks. We then investigated the impact of top-down task demands on behaviour and brain representations. Behavioural results demonstrated a superordinate advantage in the reaction time, while the accuracy was similar for all categorisation levels. The event-related potentials (ERPs) for all categorisation levels were highly similar except for about 170 ms and after 300 ms from stimulus onset. In these time windows, the animal/human face categorisation, which required fine-scale discrimination, elicited a differential ERP response. Similarly, decoding analysis over all electrodes showed the highest peak value of task decoding around 170 ms, followed by a few significant timepoints, generally after 300 ms. Moreover, brain responses revealed task-related neural modulation during categorisation tasks compared with the passive task. Overall, these findings demonstrate different task-related effects on the behavioural response and brain representations. The early and late components of neural modulation could be linked to perceptual and top-down processing of object categories, respectively.

Children are sensitive to mutual information in intermediate-complexity face and non-face features

Understanding developmental changes in children's use of specific visual information for recognizing object categories is essential for understanding how experience shapes recognition. Research on the development of face recognition has focused on children's use of low-level information (e.g. orientation sub-bands), or high-level information. In face categorization tasks, adults also exhibit sensitivity to intermediate complexity features that are diagnostic of the presence of a face. Do children also use intermediate complexity features for categorizing faces and objects, and, if so, how does their sensitivity to such features change during childhood? Intermediate-complexity features bridge the gap between low- and high-level processing: they have computational benefits for object detection and segmentation, and are known to drive neural responses in the ventral visual system. Here, we have investigated the developmental trajectory of children's sensitivity to diagnostic category information in intermediate-complexity features. We presented children (5–10 years old) and adults with image fragments of faces (Experiment 1) and cars (Experiment 2) varying in their mutual information, which quantifies a fragment's diagnosticity of a specific category. Our goal was to determine whether children were sensitive to the amount of mutual information in these fragments, and if their information usage is different from adults. We found that despite better overall categorization performance in adults, all children were sensitive to fragment diagnosticity in both categories, suggesting that intermediate representations of appearance are established early in childhood. Moreover, children's usage of mutual information was not limited to face fragments, suggesting the extracting intermediate-complexity features is a process that is not specific only to faces. We discuss the implications of our findings for developmental theories of face and object recognition.

Neural Sensitivity to Mutual Information in Intermediate-Complexity Face Features Changes during Childhood

One way in which face recognition develops during infancy and childhood is with regard to the visual information that contributes most to recognition judgments. Adult face recognition depends on critical features spanning a hierarchy of complexity, including low-level, intermediate, and high-level visual information. To date, the development of adult-like information biases for face recognition has focused on low-level features, which are computationally well-defined but low in complexity, and high-level features, which are high in complexity, but not defined precisely. To complement this existing literature, we examined the development of children's neural responses to intermediate-level face features characterized using mutual information. Specifically, we examined children's and adults' sensitivity to varying levels of category diagnosticity at the P100 and N170 components. We found that during middle childhood, sensitivity to mutual information shifts from early components to later ones, which may indicate a critical restructuring of face recognition mechanisms that takes place over several years. This approach provides a useful bridge between the study of low- and high-level visual features for face recognition and suggests many intriguing questions for further investigation.

Ultra-rapid object categorization in real-world scenes with top-down manipulations

Humans are able to achieve visual object recognition rapidly and effortlessly. Object categorization is commonly believed to be achieved by interaction between bottom-up and top-down cognitive processing. In the ultra-rapid categorization scenario where the stimuli appear briefly and response time is limited, it is assumed that a first sweep of feedforward information is sufficient to discriminate whether or not an object is present in a scene. However, whether and how feedback/top-down processing is involved in such a brief duration remains an open question. To this end, here, we would like to examine how different top-down manipulations, such as category level, category type and real-world size, interact in ultra-rapid categorization. We have constructed a dataset comprising real-world scene images with a built-in measurement of target object display size. Based on this set of images, we have measured ultra-rapid object categorization performance by human subjects. Standard feedforward computational models representing scene features and a state-of-the-art object detection model were employed for auxiliary investigation. The results showed the influences from 1) animacy (animal, vehicle, food), 2) level of abstraction (people, sport), and 3) real-world size (four target size levels) on ultra-rapid categorization processes. This had an impact to support the involvement of top-down processing when rapidly categorizing certain objects, such as sport at a fine grained level. Our work on human vs. model comparisons also shed light on possible collaboration and integration of the two that may be of interest to both experimental and computational vision researches. All the collected images and behavioral data as well as code and models are publicly available at https://osf.io/mqwjz/.

Eye-tracking for assessing medical image interpretation: A pilot feasibility study comparing novice vs expert cardiologists

INTRODUCTION

As specialized medical professionals such as radiologists, pathologists, and cardiologists gain education and experience, their diagnostic efficiency and accuracy change, and they show altered eye movement patterns during medical image interpretation. Existing research in this area is limited to interpretation of static medical images, such as digitized whole slide biopsies, making it difficult to understand how expertise development might manifest during dynamic image interpretation, such as with angiograms or volumetric scans.

METHODS

A two-group (novice, expert) comparative pilot study examined the feasibility and utility of tracking and interpreting eye movement patterns while cardiologists viewed video-based coronary angiograms. A non-invasive eye tracking system recorded cardiologists' (n = 8) visual behaviour while they viewed and diagnosed a series of eight angiogram videos. Analyses assessed frame-by-frame video navigation behaviour, eye fixation behaviour, and resulting diagnostic decision making.

RESULTS

Relative to novices, expert cardiologists demonstrated shorter and less variable video review times, fewer eye fixations and saccadic eye movements, and less time spent paused on individual video frames. Novices showed repeated eye fixations on critical image frames and regions, though these were not predictive of accurate diagnostic decisions.

DISCUSSION

These preliminary results demonstrate interpretive decision errors among novices, suggesting they identify and process critical diagnostic features, but sometimes fail to accurately interpret those features. Results also showcase the feasibility of tracking and understanding eye movements during video-based coronary angiogram interpretation and suggest that eye tracking may be valuable for informing assessments of competency progression during medical education and training.

Monitoring Processes in Visual Search Enhanced by Professional Experience: The Case of Orange Quality-Control Workers

Visual search tasks have often been used to investigate how cognitive processes change with expertise. Several studies have shown visual experts' advantages in detecting objects related to their expertise. Here, we tried to extend these findings by investigating whether professional search experience could boost top-down monitoring processes involved in visual search, independently of advantages specific to objects of expertise. To this aim, we recruited a group of quality-control workers employed in citrus farms. Given the specific features of this type of job, we expected that the extensive employment of monitoring mechanisms during orange selection could enhance these mechanisms even in search situations in which orange-related expertise is not suitable. To test this hypothesis, we compared performance of our experimental group and of a well-matched control group on a computerized visual search task. In one block the target was an orange (expertise target) while in the other block the target was a Smurfette doll (neutral target). The a priori hypothesis was to find an advantage for quality-controllers in those situations in which monitoring was especially involved, that is, when deciding the presence/absence of the target required a more extensive inspection of the search array. Results were consistent with our hypothesis. Quality-controllers were faster in those conditions that extensively required monitoring processes, specifically, the Smurfette-present and both target-absent conditions. No differences emerged in the orange-present condition, which resulted to mainly rely on bottom-up processes. These results suggest that top-down processes in visual search can be enhanced through immersive real-life experience beyond visual expertise advantages.

Invariant object recognition is a personalized selection of invariant features in humans, not simply explained by hierarchical feed-forward vision models

One key ability of human brain is invariant object recognition, which refers to rapid and accurate recognition of objects in the presence of variations such as size, rotation and position. Despite decades of research into the topic, it remains unknown how the brain constructs invariant representations of objects. Providing brain-plausible object representations and reaching human-level accuracy in recognition, hierarchical models of human vision have suggested that, human brain implements similar feed-forward operations to obtain invariant representations. However, conducting two psychophysical object recognition experiments on humans with systematically controlled variations of objects, we observed that humans relied on specific (diagnostic) object regions for accurate recognition which remained relatively consistent (invariant) across variations; but feed-forward feature-extraction models selected view-specific (non-invariant) features across variations. This suggests that models can develop different strategies, but reach human-level recognition performance. Moreover, human individuals largely disagreed on their diagnostic features and flexibly shifted their feature extraction strategy from view-invariant to view-specific when objects became more similar. This implies that, even in rapid object recognition, rather than a set of feed-forward mechanisms which extract diagnostic features from objects in a hard-wired fashion, the bottom-up visual pathways receive, through top-down connections, task-related information possibly processed in prefrontal cortex.

A brain-based account of "basic-level" concepts

This study provides a brain-based account of how object concepts at an intermediate (basic) level of specificity are represented, offering an enriched view of what it means for a concept to be a basic-level concept, a research topic pioneered by Rosch and others (Rosch et al., 1976). Applying machine learning techniques to fMRI data, it was possible to determine the semantic content encoded in the neural representations of object concepts at basic and subordinate levels of abstraction. The representation of basic-level concepts (e.g. bird) was spatially broad, encompassing sensorimotor brain areas that encode concrete object properties, and also language and heteromodal integrative areas that encode abstract semantic content. The representation of subordinate-level concepts (robin) was less widely distributed, concentrated in perceptual areas that underlie concrete content. Furthermore, basic-level concepts were representative of their subordinates in that they were neurally similar to their typical but not atypical subordinates (bird was neurally similar to robin but not woodpecker). The findings provide a brain-based account of the advantages that basic-level concepts enjoy in everyday life over subordinate-level concepts: the basic level is a broad topographical representation that encompasses both concrete and abstract semantic content, reflecting the multifaceted yet intuitive meaning of basic-level concepts.

Body emotion recognition disproportionately depends on vertical orientations during childhood

Children’s ability to recognize emotional expressions from faces and bodies develops during childhood. However, the low-level features that support accurate body emotion recognition during development have not been well characterized. This is in marked contrast to facial emotion recognition, which is known to depend upon specific spatial frequency and orientation sub-bands during adulthood, biases that develop during childhood. Here, we examined whether children’s reliance on vertical vs. horizontal orientation energy for recognizing emotional expressions in static images of bodies changed during middle childhood (5 to 10 years old). We found that while children of all ages had an adult-like bias favoring vertical orientation energy, this effect was larger at younger ages. We conclude that in terms of information use, a key feature of the development of emotion recognition is improved performance with sub-optimal features for recognition – that is, learning to use less diagnostic features of the image is a slower process than learning to use more useful features.

Perceptual expertise improves category detection in natural scenes

There is much debate about how detection, categorization, and within-category identification relate to one another during object recognition. Whether these tasks rely on partially shared perceptual mechanisms may be determined by testing whether training on one of these tasks facilitates performance on another. In the present study we asked whether expertise in discriminating objects improves the detection of these objects in naturalistic scenes. Self-proclaimed car experts (N = 34) performed a car discrimination task to establish their level of expertise, followed by a visual search task where they were asked to detect cars and people in hundreds of photographs of natural scenes. Results revealed that expertise in discriminating cars was strongly correlated with car detection accuracy. This effect was specific to objects of expertise, as there was no influence of car expertise on person detection. These results indicate a close link between object discrimination and object detection performance, which we interpret as reflecting partially shared perceptual mechanisms and neural representations underlying these tasks: the increased sensitivity of the visual system for objects of expertise – as a result of extensive discrimination training – may benefit both the discrimination and the detection of these objects. Alternative interpretations are also discussed.

What's the meaning of this? A behavioral and neurophysiological investigation into the principles behind the classification of visual emotional stimuli

Two experiments were performed to investigate the principles by which emotional stimuli are classified on the dimensions of valence and arousal. In Experiment 1, a large sample of healthy participants rated emotional stimuli according to both broad dimensions. Hierarchical cluster analyses performed on these ratings revealed that stimuli were clustered according to their semantic content at the beginning of the agglomerative process. Example semantic themes include food, violence, nudes, death, and objects. Importantly, this pattern occurred in a parallel fashion for ratings on both dimensions. In Experiment 2, we investigated if the same semantic clusters were differentiated at the neurophysiological level. Intracerebral EEG was recorded from 18 patients with intractable epilepsy who viewed the same set of stimuli. Not only did electrocortical responses differentiate between these data-defined semantic clusters, they converged with the behavioral measurements to highlight the importance of categories associated with survival and reproduction. These findings provide strong evidence that the semantic content of affective material influences their classification along the broad dimensions of valence and arousal, and this principle of categorization exerts an effect on the evoked emotional response. Future studies should consider data-driven techniques rather than normative ratings to identify more specific, semantically related emotional images.

What is special about expertise? Visual expertise reveals the interactive nature of real-world object recognition

Ever since Diamond and Carey (1986. J. Exp. Psychol.: Gen., vol. 115, pp. 107-117) seminal work, the main model for studying expertise in visual object recognition ("visual expertise") has been face perception. The underlying assumption was that since faces may be considered the ultimate domain of visual expertise, any face-processing signature might actually be a general characteristic of visual expertise. However, while humans are clearly experts in face recognition, visual expertise is not restricted to faces and can be observed in a variety of domains. This raises the question of whether face recognition is in fact the right model to study visual expertise, and if not, what are the common cognitive and neural characteristics of visual expertise. The current perspective article addresses this question by revisiting past and recent neuroimaging and behavioural works on visual expertise. The view of visual expertise that emerges from these works is that expertise is a unique phenomenon, with distinctive neural and cognitive characteristics. Specifically, visual expertise is a controlled, interactive process that develops from the reciprocal interactions between the visual system and multiple top-down factors, including semantic knowledge, top-down attentional control, and task relevance. These interactions enable the ability to flexibly access domain-specific information at multiple scales and levels guided by multiple recognition goals. Extensive visual experience with a given object category culminates in the recruitment of these multiple systems, and is reflected in widespread neural activity, extending well beyond visual cortex, to include higher-level cortical areas.

A face detection bias for horizontal orientations develops in middle childhood

Faces are complex stimuli that can be described via intuitive facial features like the eyes, nose, and mouth, "configural" features like the distances between facial landmarks, and features that correspond to computations performed in the early visual system (e.g., oriented edges). With regard to this latter category of descriptors, adult face recognition relies disproportionately on information in specific spatial frequency and orientation bands: many recognition tasks are performed more accurately when adults have access to mid-range spatial frequencies (8-16 cycles/face) and horizontal orientations (Dakin and Watt, 2009). In the current study, we examined how this information bias develops in middle childhood. We recruited children between the ages of 5-10 years-old to participate in a simple categorization task that required them to label images according to whether they depicted a face or a house. Critically, children were presented with face and house images comprised either of primarily horizontal orientation energy, primarily vertical orientation energy, or both horizontal and vertical orientation energy. We predicted that any bias favoring horizontal information over vertical should be more evident in faces than in houses, and also that older children would be more likely to show such a bias than younger children. We designed our categorization task to be sufficiently easy that children would perform at near-ceiling accuracy levels, but with variation in response times that would reflect how they rely on different orientations as a function of age and object category. We found that horizontal bias for face detection (but not house detection) correlated significantly with age, suggesting an emergent category-specific bias for horizontal orientation energy that develops during middle childhood. These results thus suggest that the tuning of high-level recognition to specific low-level visual features takes place over several years of visual development.

Eye movements as an index of pathologist visual expertise: a pilot study

A pilot study examined the extent to which eye movements occurring during interpretation of digitized breast biopsy whole slide images (WSI) can distinguish novice interpreters from experts, informing assessments of competency progression during training and across the physician-learning continuum. A pathologist with fellowship training in breast pathology interpreted digital WSI of breast tissue and marked the region of highest diagnostic relevance (dROI). These same images were then evaluated using computer vision techniques to identify visually salient regions of interest (vROI) without diagnostic relevance. A non-invasive eye tracking system recorded pathologists’ (N = 7) visual behavior during image interpretation, and we measured differential viewing of vROIs versus dROIs according to their level of expertise. Pathologists with relatively low expertise in interpreting breast pathology were more likely to fixate on, and subsequently return to, diagnostically irrelevant vROIs relative to experts. Repeatedly fixating on the distracting vROI showed limited value in predicting diagnostic failure. These preliminary results suggest that eye movements occurring during digital slide interpretation can characterize expertise development by demonstrating differential attraction to diagnostically relevant versus visually distracting image regions. These results carry both theoretical implications and potential for monitoring and evaluating student progress and providing automated feedback and scanning guidance in educational settings.

Innate pattern recognition and categorization in a jumping spider

The East African jumping spider Evarcha culicivora feeds indirectly on vertebrate blood by preferentially preying upon blood-fed Anopheles mosquitoes, the vectors of human malaria¹, using the distinct resting posture and engorged abdomen characteristic of these specific prey as key elements for their recognition. To understand perceptual categorization of objects by these spiders, we investigated their predatory behavior toward different digital stimuli - abstract ‘stick figure’ representations of Anopheles constructed solely by known key identification elements, disarranged versions of these, as well as non-prey items and detailed images of alternative prey. We hypothesized that the abstract images representing Anopheles would be perceived as potential prey, and would be preferred to those of non-preferred prey. Spiders perceived the abstract stick figures of Anopheles specifically as their preferred prey, attacking them significantly more often than non-preferred prey, even when the comprising elements of the Anopheles stick figures were disarranged and disconnected from each other. However, if the relative angles between the elements of the disconnected stick figures of Anopheles were altered, the otherwise identical set of elements was no longer perceived as prey. These data show that E. culicivora is capable of making discriminations based on abstract concepts, such as the hypothetical angle formed by discontinuous elements. It is this inter-element angle rather than resting posture that is important for correct identification of Anopheles. Our results provide a glimpse of the underlying processes of object recognition in animals with minute brains, and suggest that these spiders use a local processing approach for object recognition, rather than a holistic or global approach. This study provides an excellent basis for a comparative analysis on feature extraction and detection by animals as diverse as bees and mammals.

Beyond perceptual expertise: revisiting the neural substrates of expert object recognition

Real-world expertise provides a valuable opportunity to understand how experience shapes human behavior and neural function. In the visual domain, the study of expert object recognition, such as in car enthusiasts or bird watchers, has produced a large, growing, and often-controversial literature. Here, we synthesize this literature, focusing primarily on results from functional brain imaging, and propose an interactive framework that incorporates the impact of high-level factors, such as attention and conceptual knowledge, in supporting expertise. This framework contrasts with the perceptual view of object expertise that has concentrated largely on stimulus-driven processing in visual cortex. One prominent version of this perceptual account has almost exclusively focused on the relation of expertise to face processing and, in terms of the neural substrates, has centered on face-selective cortical regions such as the Fusiform Face Area (FFA). We discuss the limitations of this face-centric approach as well as the more general perceptual view, and highlight that expert related activity is: (i) found throughout visual cortex, not just FFA, with a strong relationship between neural response and behavioral expertise even in the earliest stages of visual processing, (ii) found outside visual cortex in areas such as parietal and prefrontal cortices, and (iii) modulated by the attentional engagement of the observer suggesting that it is neither automatic nor driven solely by stimulus properties. These findings strongly support a framework in which object expertise emerges from extensive interactions within and between the visual system and other cognitive systems, resulting in widespread, distributed patterns of expertise-related activity across the entire cortex.

Ultra rapid object categorization: effects of level, animacy and context

It is widely agreed that in object categorization bottom-up and top-down influences interact. How top-down processes affect categorization has been primarily investigated in isolation, with only one higher level process at a time being manipulated. Here, we investigate the combination of different top-down influences (by varying the level of category, the animacy and the background of the object) and their effect on rapid object categorization. Subjects participated in a two-alternative forced choice rapid categorization task, while we measured accuracy and reaction times. Subjects had to categorize objects on the superordinate, basic or subordinate level. Objects belonged to the category animal or vehicle and each object was presented on a gray, congruent (upright) or incongruent (inverted) background. The results show that each top-down manipulation impacts object categorization and that they interact strongly. The best categorization was achieved on the superordinate level, providing no advantage for basic level in rapid categorization. Categorization between vehicles was faster than between animals on the basic level and vice versa on the subordinate level. Objects in homogenous gray background (context) yielded better overall performance than objects embedded in complex scenes, an effect most prominent on the subordinate level. An inverted background had no negative effect on object categorization compared to upright scenes. These results show how different top-down manipulations, such as category level, category type and background information, are related. We discuss the implications of top-down interactions on the interpretation of categorization results.

Neuroanatomical correlates of visual car expertise

Expertise in non-visual domains such as musical performance is associated with differences in gray matter volume of particular regions of the human brain. Whether this is also the case for expertise in visual object recognition is unknown. Here we tested whether individual variability in the ability to recognize car models, from novice performance to high level of expertise, is associated with specific structural changes in gray matter volume. We found that inter-individual variability in expertise with cars was significantly and selectively correlated with gray matter volume in prefrontal cortex. Inter-individual differences in the recognition of airplanes, that none of the participants had expertise with, were correlated with structural variability of regions bordering the visual cortex. These results highlight the role of prefrontal regions outside the visual cortex in accessing and processing visual knowledge about objects from the domain of expertise and suggest that expertise in visual object recognition may entail structural changes in regions associated with semantic knowledge.