Visual prediction and perceptual expertise

Effects of expectation on face perception and its association with expertise

Perceptual decisions are derived from the combination of priors and sensorial input. While priors are broadly understood to reflect experience/expertise developed over one's lifetime, the role of perceptual expertise at the individual level has seldom been directly explored. Here, we manipulate probabilistic information associated with a high and low expertise category (faces and cars respectively), while assessing individual level of expertise with each category. 67 participants learned the probabilistic association between a color cue and each target category (face/car) in a behavioural categorization task. Neural activity (EEG) was then recorded in a similar paradigm in the same participants featuring the previously learned contingencies without the explicit task. Behaviourally, perception of the higher expertise category (faces) was modulated by expectation. Specifically, we observed facilitatory and interference effects when targets were correctly or incorrectly expected, which were also associated with independently measured individual levels of face expertise. Multivariate pattern analysis of the EEG signal revealed clear effects of expectation from 100 ms post stimulus, with significant decoding of the neural response to expected vs. not stimuli, when viewing identical images. Latency of peak decoding when participants saw faces was directly associated with individual level facilitation effects in the behavioural task. The current results not only provide time sensitive evidence of expectation effects on early perception but highlight the role of higher-level expertise on forming priors.

Enhanced neural synchrony associated with long-term ballroom dance training

Long-term dance training offers numerous benefits, including improvements in physical health, posture, body coordination, and mental health and well-being. Since dance is an art form of body-to-body communication, professional dancers may share feelings and thoughts on dance with their partners, owing to their shared training experiences. Considering this perspective, one may expect that professional dancers would demonstrate pronounced neural similarities when viewing dancing videos, which could be associated with their training duration. To test these hypotheses, we collected functional magnetic resonance imaging (fMRI) data while presenting ballroom dancing and neutral video clips with long durations (∼100 s each) to 41 professional ballroom dancers (19 pairs of dance partners) and 39 age- and sex-matched nondancers. Our findings revealed that dancers exhibited broader and stronger neural similarities across the whole brain when watching dancing video clips, as compared to the control group. These increased neural similarities could be interpreted in at least two distinct ways. First, neural similarities in certain brain regions within the motor control circuit (i.e., frontal cortical-basal ganglia-thalamic circuit) were significantly correlated with dance-related information (e.g., dance partners' cooperation duration), which reinforced the impact of long-term dance training on neural synchronization. Second, neural similarities in other brain regions (e.g., memory-related brain regions) were significantly correlated with subjects' impression of the viewed videos (i.e., whether they have watched before, familiarity, and liking), which may not necessarily be directly linked to long-term dance training. Altogether, our study provided solid evidence for synchronized neural mechanisms in professional dancers due to long-term dance training.

An investigation of feed-forward and feedback eye movement training in immersive virtual reality

The control of eye gaze is critical to the execution of many skills. The observation that task experts in many domains exhibit more efficient control of eye gaze than novices has led to the development of gaze training interventions that teach these behaviours. We aimed to extend this literature by i) examining the relative benefits of feed-forward (observing an expert's eye movements) versus feed-back (observing your own eye movements) training, and ii) automating this training within virtual reality. Serving personnel from the British Army and Royal Navy were randomised to either feed-forward or feed-back training within a virtual reality simulation of a room search and clearance task. Eye movement metrics - including visual search, saccade direction, and entropy - were recorded to quantify the efficiency of visual search behaviours. Feed-forward and feed-back eye movement training produced distinct learning benefits, but both accelerated the development of efficient gaze behaviours. However, we found no evidence that these more efficient search behaviours transferred to better decision making in the room clearance task. Our results suggest integrating eye movement training principles within virtual reality training simulations may be effective, but further work is needed to understand the learning mechanisms.

Visual expertise modulates baseline brain activity: a preliminary resting-state fMRI study using expertise model of radiologists

Background

visual expertise and experience modulate evoked brain activity in response to training-related stimuli. However, few studies have considered how the visual experience is represented in the resting state brain activity. This study tried to investigate the way visual experience, i.e., visual recognition expertise, modulates baseline brain neuronal activity in the resting state using the model of radiologists.

Methods

The amplitude of low-frequency (< 0.08 Hz) fluctuation (ALFF) was used as the metric of baseline brain activity and a visual expertise model of radiologists to investigated this question. The visual recognition skill enables them to accurately identify pathological information in medical images. After the behavior measurement, a cohort group of radiology interns (n = 22) and a group of matched layperson (n = 22) were selected for inclusion in the study. The resting state functional magnetic resonance imaging (fMRI) scans were performed for all of the subjects.

Results

Higher ALFF in the right fusiform gyrus and the left orbitofrontal cortex were observed, and the ALFF in the fusiform gyrus was correlated with the intern radiologists’ behavioral expertise(all results corrected for multiple comparisons).

Conclusions

Visual experience modulates the baseline brain activity in both high-level visual cortex and high-order cognitive cortex, indicating the engagement of both top-down and bottom-up facilitation. We provide a novel perspective to how visual experience modulated cortical brain activity by introducing the resting state changes. Also, we propose that our current study may provide novel ideas for the development of new training protocols in medical school.

Repetition Probability Effects for Chinese Characters and German Words in the Visual Word Form Area

The magnitude of repetition suppression (RS), measured by fMRI, is modulated by the probability of repetitions (P(rep)) for various sensory stimulus categories. It has been suggested that for visually presented simple letters this P(rep) effect depends on the prior practices of the participants with the stimuli. Here we tested further if previous experiences affect the neural mechanisms of RS, leading to the modulatory effects of stimulus P(rep), for more complex lexical stimuli as well. We measured the BOLD signal in the Visual Word Form Area (VWFA) of native Chinese and German participants and estimated the P(rep) effects for Chinese characters and German words. The results showed a significant P(rep) effect for stimuli of the mother tongue in both participant groups. Interestingly, Chinese participants, learning German as a second language, also showed a significant P(rep) modulation of RS for German words while the German participants who had no prior experiences with the Chinese characters showed no such effects. Our findings suggest that P(rep) effects on RS are manifest for visual word processing as well, but only for words of a language with which participants are highly familiar. These results support further the idea that predictive processes, estimated by P(rep) modulations of RS, require prior experiences.

Interaction of top-down category-level expectation and bottom-up sensory input in early stages of visual-orthographic processing

How and when top-down information modulates visual-orthographic processing is an essential question in reading research. In a previous study, we showed that task modulation of print-tuning started at around 170 ms after stimulus presentation in the N1 offset of the ERP, while the N1 onset was yet unaffected. Here we test how prior category-level expectation affects visual-orthographic processing. Familiar, left/right-structured Chinese characters and stroke number matched, unfamiliar Korean characters were presented, while expectation about the upcoming stimuli was manipulated with green and blue colored frames (high Chinese vs. high Korean expectation). EEG data of 18 native Chinese speakers were recorded while participants performed an expectation judgment task. Results from occipito-temporal and whole map analyses revealed that effects of prior expectation changed throughout the N1. Accordingly in the N1 onset, a print tuning main effect was found, with a stronger N1 to Chinese characters than Korean characters, irrespective of expectation. In the N1 offset, an expectation-by-character interaction was observed at the whole map level, with a more negative N1 to Korean characters than Chinese characters when expecting a Chinese character, but no such difference when expecting a Korean character. Moreover, the expectation-by-character interaction continued to the N250, with similar responses to Chinese and Korean characters under the Chinese expectation condition, while less negative N250 to Korean than Chinese under the Korean expectation condition. Taken together, the current study provides evidence that prior category-level expectation starts to take effect at an early stage even within 200 ms by facilitating the processing of expected stimuli, suggesting that category-level expectation can influence early visual-orthographic processing during word recognition.

Parsing rooms: the role of the PPA and RSC in perceiving object relations and spatial layout

The perception of a scene involves grasping the global space of the scene, usually called the spatial layout, as well as the objects in the scene and the relations between them. The main brain areas involved in scene perception, the parahippocampal place area (PPA) and retrosplenial cortex (RSC), are supposed to mostly support the processing of spatial layout. Here we manipulated the objects and their relations either by arranging objects within rooms in a common way or by scattering them randomly. The rooms were then varied for spatial layout by keeping or removing the walls of the room, a typical layout manipulation. We then combined a visual search paradigm, where participants actively search for an object within the room, with multivariate pattern analysis (MVPA). Both left and right PPA were sensitive to the layout properties, but the right PPA was also sensitive to the object relations even when the information about objects and their relations is used in the cross-categorization procedure on novel stimuli. The left and right RSC were sensitive to both spatial layout and object relations, but could only use the information about object relations for cross-categorization to novel stimuli. These effects were restricted to the PPA and RSC, as other control brain areas did not display the same pattern of results. Our results underline the importance of employing paradigms that require participants to explicitly retrieve domain-specific processes and indicate that objects and their relations are processed in the scene areas to a larger extent than previously assumed.

Prior Precision Modulates the Minimization of Auditory Prediction Error

The predictive coding model of perception proposes that successful representation of the perceptual world depends upon canceling out the discrepancy between prediction and sensory input (i.e., prediction error). Recent studies further suggest a distinction to be made between prediction error triggered by non-predicted stimuli of different prior precision (i.e., inverse variance). However, it is not fully understood how prediction error with different precision levels is minimized in the predictive process. Here, we conducted a magnetoencephalography (MEG) experiment which orthogonally manipulated prime-probe relation (for contextual precision) and stimulus repetition (for perceptual learning which decreases prediction error). We presented participants with cycles of tone quartets which consisted of three prime tones and one probe tone of randomly selected frequencies. Within each cycle, the three prime tones remained identical while the probe tones changed once at some point (e.g., from repetition of 123X to repetition of 123Y). Therefore, the repetition of probe tones can reveal the development of perceptual inferences in low and high precision contexts depending on their position within the cycle. We found that the two conditions resemble each other in terms of N1m modulation (as both were associated with N1m suppression) but differ in terms of N2m modulation. While repeated probe tones in low precision context did not exhibit any modulatory effect, repeated probe tones in high precision context elicited a suppression and rebound of the N2m source power. The differentiation suggested that the minimization of prediction error in low and high precision contexts likely involves distinct mechanisms.

Electrophysiological correlates of aesthetic processing of webpages: a comparison of experts and laypersons

We investigated whether design experts or laypersons evaluate webpages differently. Twenty participants, 10 experts and 10 laypersons, judged the aesthetic value of a webpage in an EEG-experiment. Screenshots of 150 webpages, judged as aesthetic or as unaesthetic by another 136 participants, served as stimulus material. Behaviorally, experts and laypersons evaluated unaesthetic webpages similarly, but they differed in their evaluation of aesthetic ones: experts evaluated aesthetic webpages as unaesthetic more often than laypersons did. The ERP-data show main effects of level of expertise and of aesthetic value only. There was no interaction of expertise and aesthetics. In a time-window of 110–130 ms after stimulus onset, aesthetic webpages elicited a more negative EEG-amplitude than unaesthetic webpages. In the same time window, experts had more negative EEG-amplitudes than laypersons. This patterning of results continued until a time window of 600–800 ms in which group and aesthetic differences diminished. An interaction of perceiver characteristics and object properties that several interactionist theories postulate was absent in the EEG-data. Experts seem to process the stimuli in a more thorough manner than laypersons. The early activation differences between aesthetic and unaesthetic webpages is in contrast with some theories of aesthetic processing and has not been reported before.

Fusing multiple neuroimaging modalities to assess group differences in perception-action coupling

In the last few decades, non-invasive neuroimaging has revealed macro-scale brain dynamics that underlie perception, cognition and action. Advances in non-invasive neuroimaging target two capabilities; 1) increased spatial and temporal resolution of measured neural activity, and 2) innovative methodologies to extract brain-behavior relationships from evolving neuroimaging technology. We target the second. Our novel methodology integrated three neuroimaging methodologies and elucidated expertise-dependent differences in functional (fused EEG-fMRI) and structural (dMRI) brain networks for a perception-action coupling task. A set of baseball players and controls performed a Go/No-Go task designed to mimic the situation of hitting a baseball. In the functional analysis, our novel fusion methodology identifies 50ms windows with predictive EEG neural correlates of expertise and fuses these temporal windows with fMRI activity in a whole-brain 2mm voxel analysis, revealing time-localized correlations of expertise at a spatial scale of millimeters. The spatiotemporal cascade of brain activity reflecting expertise differences begins as early as 200ms after the pitch starts and lasting up to 700ms afterwards. Network differences are spatially localized to include motor and visual processing areas, providing evidence for differences in perception-action coupling between the groups. Furthermore, an analysis of structural connectivity revealed that the players have significantly more connections between cerebellar and left frontal/motor regions, and many of the functional activation differences between the groups are located within structurally defined network modules that differentiate expertise. In short, our novel method illustrates how multimodal neuroimaging can provide specific macro-scale insights into the functional and structural correlates of expertise development.

Knowing when not to swing: EEG evidence that enhanced perception-action coupling underlies baseball batter expertise

Given a decision that requires less than half a second for evaluating the characteristics of the incoming pitch and generating a motor response, hitting a baseball potentially requires unique perception-action coupling to achieve high performance. We designed a rapid perceptual decision-making experiment modeled as a Go/No-Go task yet tailored to reflect a real scenario confronted by a baseball hitter. For groups of experts (Division I baseball players) and novices (non-players), we recorded electroencephalography (EEG) while they performed the task. We analyzed evoked EEG single-trial variability, contingent negative variation (CNV), and pre-stimulus alpha power with respect to the expert vs. novice groups. We found strong evidence for differences in inhibitory processes between the two groups, specifically differential activity in supplementary motor areas (SMA), indicative of enhanced inhibitory control in the expert (baseball player) group. We also found selective activity in the fusiform gyrus (FG) and orbital gyrus in the expert group, suggesting an enhanced perception-action coupling in baseball players that differentiates them from matched controls. In sum, our results show that EEG correlates of decision formation can be used to identify neural markers of high-performance athletes.

Segmentation of dance movement: effects of expertise, visual familiarity, motor experience and music

According to event segmentation theory, action perception depends on sensory cues and prior knowledge, and the segmentation of observed actions is crucial for understanding and memorizing these actions. While most activities in everyday life are characterized by external goals and interaction with objects or persons, this does not necessarily apply to dance-like actions. We investigated to what extent visual familiarity of the observed movement and accompanying music influence the segmentation of a dance phrase in dancers of different skill level and non-dancers. In Experiment 1, dancers and non-dancers repeatedly watched a video clip showing a dancer performing a choreographed dance phrase and indicated segment boundaries by key press. Dancers generally defined less segment boundaries than non-dancers, specifically in the first trials in which visual familiarity with the phrase was low. Music increased the number of segment boundaries in the non-dancers and decreased it in the dancers. The results suggest that dance expertise reduces the number of perceived segment boundaries in an observed dance phrase, and that the ways visual familiarity and music affect movement segmentation are modulated by dance expertise. In a second experiment, motor experience was added as factor, based on empirical evidence suggesting that action perception is modified by visual and motor expertise in different ways. In Experiment 2, the same task as in Experiment 1 was performed by dance amateurs, and was repeated by the same participants after they had learned to dance the presented dance phrase. Less segment boundaries were defined in the middle trials after participants had learned to dance the phrase, and music reduced the number of segment boundaries before learning. The results suggest that specific motor experience of the observed movement influences its perception and anticipation and makes segmentation broader, but not to the same degree as dance expertise on a professional level.

The predictive mind and the experience of visual art work

Among the main challenges of the predictive brain/mind concept is how to link prediction at the neural level to prediction at the cognitive-psychological level and finding conceptually robust and empirically verifiable ways to harness this theoretical framework toward explaining higher-order mental and cognitive phenomena, including the subjective experience of aesthetic and symbolic forms. Building on the tentative prediction error account of visual art, this article extends the application of the predictive coding framework to the visual arts. It does so by linking this theoretical discussion to a subjective, phenomenological account of how a work of art is experienced. In order to engage more deeply with a work of art, viewers must be able to tune or adapt their prediction mechanism to recognize art as a specific class of objects whose ontological nature defies predictability, and they must be able to sustain a productive flow of predictions from low-level sensory, recognitional to abstract semantic, conceptual, and affective inferences. The affective component of the process of predictive error optimization that occurs when a viewer enters into dialog with a painting is constituted both by activating the affective affordances within the image and by the affective consequences of prediction error minimization itself. The predictive coding framework also has implications for the problem of the culturality of vision. A person's mindset, which determines what top-down expectations and predictions are generated, is co-constituted by culture-relative skills and knowledge, which form hyperpriors that operate in the perception of art.

Repetition probability effects depend on prior experiences

The magnitude of repetition suppression (RS) in the Fusiform Face Area is influenced by the probability of repetitions of faces (Summerfield et al., 2008), implying that perceptual expectations affect repetition-related processes. Surprisingly, however, macaque single-cell (Kaliukhovich and Vogels, 2011) and human fMRI (Kovács et al., 2013) studies have failed to find repetition probability [P(rep)] modulations of RS with nonface stimuli in the occipitotemporal cortex, suggesting that the effect is face specific. One possible explanation of this category selectivity is that the extensive experience humans have with faces affects the neural mechanisms of RS specifically, creating P(rep) modulatory effects. To address this question, we used fMRI to test the P(rep) effects for another well trained stimulus category, upright letters of the roman alphabet as well as for unfamiliar false fonts. We observed significant RS for both stimulus sets in the Letter Form Area as well as in the caudodorsal part of the lateral occipital complex. Interestingly, the influence of P(rep) on RS was dependent on the stimulus: while we observed P(rep) modulations for the roman letters, no such effects were found for the unfamiliar false fonts in either area. Our findings suggest that P(rep) effects on RS are manifest for nonface stimuli as well, but that they depend on the experience of the subjects with the stimulus category. This shows, for the first time, that prior experience affects the influence of contextual predictive information on RS in the human occipitotemporal cortex.

Fingerprints of Learned Object Recognition Seen in the fMRI Activation Patterns of Lateral Occipital Complex

One feature of visual processing in the ventral stream is that cortical responses gradually depart from the physical aspects of the visual stimulus and become correlated with perceptual experience. Thus, unlike early retinotopic areas, the responses in the object-related lateral occipital complex (LOC) are typically immune to parameter changes (e.g., contrast, location, etc.) when these do not affect recognition. Here, we use a complementary approach to highlight changes in brain activity following a shift in the perceptual state (in the absence of any alteration in the physical image). Specifically, we focus on LOC and early visual cortex (EVC) and compare their functional magnetic resonance imaging (fMRI) responses to degraded object images, before and after fast perceptual learning that renders initially unrecognized objects identifiable. Using 3 complementary analyses, we find that, in LOC, unlike EVC, learned recognition is associated with a change in the multivoxel response pattern to degraded object images, such that the response becomes significantly more correlated with that evoked by the intact version of the same image. This provides further evidence that the coding in LOC reflects the recognition of visual objects.

The role of the parahippocampal cortex in cognition

The parahippocampal cortex (PHC) has been associated with many cognitive processes, including visuospatial processing and episodic memory. To characterize the role of PHC in cognition, a framework is required that unifies these disparate processes. An overarching account was proposed whereby the PHC is part of a network of brain regions that processes contextual associations. Contextual associations are the principal element underlying many higher-level cognitive processes, and thus are suitable for unifying the PHC literature. Recent findings are reviewed that provide support for the contextual associations account of PHC function. In addition to reconciling a vast breadth of literature, the synthesis presented expands the implications of the proposed account and gives rise to new and general questions about context and cognition.

Prediction, perception and agency

The articles in this special issue provide a rich and thoughtful perspective on the brain as an inference machine. They illuminate key aspects of the internal or generative models the brain might use for perception. Furthermore, they explore the implications for a sense of agency and the nature of false inference in neuropsychiatric syndromes. In this review, I try to gather together some of the themes that emerge in this special issue and use them to illustrate how far one can take the notion of predictive coding in understanding behaviour and agency.