Perceptual Expertise and Attention: An Exploration using Deep Neural Networks

Perceptual expertise and attention are two important factors that enable superior object recognition and task performance. While expertise enhances knowledge and provides a holistic understanding of the environment, attention allows us to selectively focus on task-related information and suppress distraction. It has been suggested that attention operates differently in experts and in novices, but much remains unknown. This study investigates the relationship between perceptual expertise and attention using convolutional neural networks (CNNs), which are shown to be good models of primate visual pathways. Two CNN models were trained to become experts in either face or scene recognition, and the effect of attention on performance was evaluated in tasks involving complex stimuli, such as superimposed images containing superimposed faces and scenes. The goal was to explore how feature-based attention (FBA) influences recognition within and outside the domain of expertise of the models. We found that each model performed better in its area of expertise-and that FBA further enhanced task performance, but only within the domain of expertise, increasing performance by up to 35% in scene recognition, and 15% in face recognition. However, attention had reduced or negative effects when applied outside the models' expertise domain. Neural unit-level analysis revealed that expertise led to stronger tuning towards category-specific features and sharper tuning curves, as reflected in greater representational dissimilarity between targets and distractors, which, in line with the biased competition model of attention, leads to enhanced performance by reducing competition. These findings highlight the critical role of neural tuning at single as well as network level neural in distinguishing the effects of attention in experts and in novices and demonstrate that CNNs can be used fruitfully as computational models for addressing neuroscience questions not practical with the empirical methods.

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these “upfixes” (i.e., upwards affixes) involve a productive schema enabling both stored and novel face–upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Multisensory Information Facilitates the Categorization of Untrained Stimuli

Although it has been demonstrated that multisensory information can facilitate object recognition and object memory, it remains unclear whether such facilitation effect exists in category learning. To address this issue, comparable car images and sounds were first selected by a discrimination task in Experiment 1. Then, those selected images and sounds were utilized in a prototype category learning task in Experiments 2 and 3, in which participants were trained with auditory, visual, and audiovisual stimuli, and were tested with trained or untrained stimuli within the same categories presented alone or accompanied with a congruent or incongruent stimulus in the other modality. In Experiment 2, when low-distortion stimuli (more similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual trials, but no significant difference between audiovisual and auditory trials. During testing, accuracy was significantly higher for congruent trials than unisensory or incongruent trials, and the congruency effect was larger for untrained high-distortion stimuli than trained low-distortion stimuli. In Experiment 3, when high-distortion stimuli (less similar to the prototypes) were trained, there was higher accuracy for audiovisual trials than visual or auditory trials, and the congruency effect was larger for trained high-distortion stimuli than untrained low-distortion stimuli during testing. These findings demonstrated that higher degree of stimuli distortion resulted in more robust multisensory effect, and the categorization of not only trained but also untrained stimuli in one modality could be influenced by an accompanying stimulus in the other modality.

Extended categorization of conjunction object stimuli decreases the latency of attentional feature selection and recruits orthography-linked ERPs

The role of attention in driving perceptual expertise effects is controversial. The current study addressed the effect of training on ERP components related to and independent of attentional feature selection. Participants learned to categorize cartoon animals over six training sessions (8,800 trials) after which ERPs were recorded during a target detection task performed on trained and untrained stimulus sets. The onset of the selection negativity, an ERP component indexing attentional modulation, was about 60 msec earlier for trained than untrained stimuli. Trained stimuli also elicited centro-parietal N200 and N320 components that were insensitive to attentional feature selection. The scalp distribution and timecourse of these components were better matched by studies of orthography than object expertise. Source localization using eLORETA suggested that the strongest neural sources of the selection negativity were in right ventral temporal cortex whereas the strongest sources of the N200/N320 components were in left ventral temporal cortex, again consistent with the hypothesis that training recruited orthography related areas. Overall, training altered neural processes related to attentional selection, but also affected neural processes that were independent of feature selection.

Quantifying evidential value and selective reporting in recent and 10-year past psychophysiological literature: A pre-registered P-curve analysis

Selective reporting (i.e., only reporting significant findings as opposed to all analyses or results) is a questionable research practice that undermines the integrity of published research. Psychophysiology research may be susceptible to selective reporting, given the high number of decision points and methodological complexity in analyses of psychophysiology data. We aimed to assess the presence of selective reporting and evidential value (i.e., that significant results are due to true underlying effects) in recent and past psychophysiological research by utilizing p-curve analyses. Study protocols and methods were pre-registered on the Open Science Framework (OSF). P-values and the associated test statistics were extracted from articles in the most recent issue (as of January 2018) and 10-year previous counterpart issue of three major psychophysiology journals: Psychophysiology, International Journal of Psychophysiology, and Journal of Psychophysiology. Using the p-curve application, 10 primary p-curves were conducted: all recent articles, all past articles, recent articles split by journal, past articles split by journal, recent cognitive electrophysiology articles, and past cognitive electrophysiology articles. Evidential value and generally adequate average power (≥78% average power) were present in all p-curves, except those that only included articles from the Journal of Psychophysiology because of the small number of articles published in the journal. Findings provide some positive news and indicate that, generally, results were not selectively reported, and selective reporting may not be a primary issue for this sample of psychophysiological research. Future p-curve analyses examining sub-disciplines of psychophysiology are recommended.

Bilateral engagement of the occipito-temporal cortex in response to dance kinematics in experts

Previous evidence has shown neuroplastic changes in brain anatomy and connectivity associated with the acquisition of professional visuomotor skills. Reduced hemispherical asymmetry was found in the sensorimotor and visual areas in expert musicians and athletes compared with non-experts. Moreover, increased expertise with faces, body, and objects resulted in an enhanced engagement of the occipito-temporal cortex (OTC) during stimulus observation. The present study aimed at investigating whether intense and extended practice with dance would result in an enhanced symmetric response of OTC at an early stage of action processing. Expert ballet dancers and non-dancer controls were presented with videos depicting ballet steps during EEG recording. The observation of the moving dancer elicited a posterior N2 component, being larger over the left hemisphere in dancers than controls. The source reconstruction (swLORETA) of the negativity showed the engagement of the bilateral inferior and middle temporal regions in experts, while right-lateralized activity was found in controls. The dancers also showed an early P2 and enhanced P300 responses, indicating faster stimulus processing and subsequent recognition. This evidence seemed to suggest expertise-related increased sensitivity of the OTC in encoding body kinematics. Thus, we speculated that long-term whole-body practice would result in enriched and refined action processing.

Neural and behavioral effects of subordinate-level training of novel objects across manipulations of color and spatial frequency

Perceptual expertise is marked by subordinate-level recognition of objects in the expert domain. In this study, participants learned one family of full-color, artificial objects at the subordinate (species) level and another family at the basic (family) level. Discrimination of trained and untrained exemplars was tested before and after training across several image manipulations [full-color, grayscale, low spatial frequency (LSF) and high spatial frequency (HSF)] while event-related potentials (ERPs) were recorded. Regardless of image manipulation, discrimination (indexed by d') of trained and of untrained exemplars was enhanced after subordinate-level training, but not after basic-level training. Enhanced discrimination after subordinate-level training generalized to untrained exemplars and to grayscale images and images in which LSF or HSF information was removed. After training, the N170 and N250, recorded over occipital and occipitotemporal brain regions, were both more enhanced after subordinate-level training than after basic-level training. However, the topographic distribution of enhanced responses differed across components. The N170 latency predicted reaction time after both basic-level training and subordinate-level training, highlighting an association between behavioral and neural responses. These findings further elucidate the role of the N170 and N250 as ERP indices of subordinate-level expert object processing and demonstrate how low-level manipulations of color and spatial frequency impact behavior and the N170 and N250 components independent of training or expertise.