The effects of visual object priming on brain activation before and after recognition

Neural segregation and integration of sensory, decision, and action processes during object categorization

Neural and computational evidence suggests that perceptual decisions depend on an evidence accumulation process. The gradual reveal fMRI method, which prolongs a decision to match the slow temporal resolution of fMRI measurements, has classified dorsal visual stream regions as "Action" (alternatively, "Moment of Recognition" or "Commitment") and ventral visual stream regions as "Accumulator." Previous gradual reveal fMRI studies, however, only tested actions that were in response to decisions and, thus, related to evidence accumulation. To fully dissociate the contribution of sensory, decision, and motor components to Action and Accumulator regions in the dorsal and ventral visual streams, we extended the gradual reveal paradigm to also include responses made to cues where no decision was necessary. We found that the lateral occipital cortex in the ventral visual stream showed a highly selective Accumulator profile, whereas regions in the fusiform gyrus were influenced by action generation. Dorsal visual stream regions showed strikingly similar profiles as classical motor regions and also as regions of the salience network. These results suggest that the dorsal and ventral visual streams may appear highly segregated because they include a small number of regions that are highly selective for Accumulator or Action. However, the streams may be more integrated than previously thought and this integration may be accomplished by regions with graded responses that are less selective (i.e., more distributed).

Twenty years of investigation with the case of prosopagnosia PS to understand human face identity recognition. Part II: Neural basis

Patient PS sustained her dramatic brain injury in 1992, the same year as the first report of a neuroimaging study of human face recognition. The present paper complements the review on the functional nature of PS's prosopagnosia (part I), illustrating how her case study directly, i.e., through neuroimaging investigations of her brain structure and activity, but also indirectly, through neural studies performed on other clinical cases and neurotypical individuals, inspired and constrained neural models of human face recognition. In the dominant right hemisphere for face recognition in humans, PS's main lesion concerns (inputs to) the inferior occipital gyrus (IOG), in a region where face-selective activity is typically found in normal individuals ('Occipital Face Area', OFA). Her case study initially supported the criticality of this region for face identity recognition (FIR) and provided the impetus for transcranial magnetic stimulation (TMS), intracerebral electrical stimulation, and cortical surgery studies that have generally supported this view. Despite PS's right IOG lesion, typical face-selectivity is found anteriorly in the middle portion of the fusiform gyrus, a hominoid structure (termed the right 'Fusiform Face Area', FFA) that is widely considered to be the most important region for human face recognition. This finding led to the original proposal of direct anatomico-functional connections from early visual cortices to the FFA, bypassing the IOG/OFA (lulu), a hypothesis supported by further neuroimaging studies of PS, other neurological cases and neuro-typical individuals with original visual stimulation paradigms, data recordings and analyses. The proposal of a lack of sensitivity to face identity in PS's right FFA due to defective reentrant inputs from the IOG/FFA has also been supported by other cases, functional connectivity and cortical surgery studies. Overall, neural studies of, and based on, the case of prosopagnosia PS strongly question the hierarchical organization of the human neural face recognition system, supporting a more flexible and dynamic view of this key social brain function.

Corticothalamic projections deliver enhanced responses to medial geniculate body as a function of the temporal reliability of the stimulus

Ageing and challenging signal-in-noise conditions are known to engage the use of cortical resources to help maintain speech understanding. Extensive corticothalamic projections are thought to provide attentional, mnemonic and cognitive-related inputs in support of sensory inferior colliculus (IC) inputs to the medial geniculate body (MGB). Here we show that a decrease in modulation depth, a temporally less distinct periodic acoustic signal, leads to a jittered ascending temporal code, changing MGB unit responses from adapting responses to responses showing repetition enhancement, posited to aid identification of important communication and environmental sounds. Young-adult male Fischer Brown Norway rats, injected with the inhibitory opsin archaerhodopsin T (ArchT) into the primary auditory cortex (A1), were subsequently studied using optetrodes to record single-units in MGB. Decreasing the modulation depth of acoustic stimuli significantly increased repetition enhancement. Repetition enhancement was blocked by optical inactivation of corticothalamic terminals in MGB. These data support a role for corticothalamic projections in repetition enhancement, implying that predictive anticipation could be used to improve neural representation of weakly modulated sounds. KEY POINTS: In response to a less temporally distinct repeating sound with low modulation depth, medial geniculate body (MGB) single units show a switch from adaptation towards repetition enhancement. Repetition enhancement was reversed by blockade of MGB inputs from the auditory cortex. Collectively, these data argue that diminished acoustic temporal cues such as weak modulation engage cortical processes to enhance coding of those cues in auditory thalamus.

Repetition Suppression in Visual and Auditory Modalities Affects the Sound-Induced Flash Illusion

Sound-induced flash illusion (SiFI) refers to the illusion that the number of visual flashes is equal to the number of auditory sounds when the visual flashes are accompanied by an unequal number of auditory sounds presented within 100 ms. The effect of repetition suppression (RS), an adaptive effect caused by stimulus repetition, upon the SiFI has not been investigated. Based on the classic SiFI paradigm, the present study investigated whether RS would affect the SiFI differently by adding preceding stimuli in visual and auditory modalities prior to the appearance of audiovisual stimuli. The results showed the auditory RS effect on the SiFI varied with the number of preceding auditory stimuli. The hit rate was higher with two preceding auditory stimuli than one preceding auditory stimulus in fission illusion, but it did not affect the size of the fusion illusion. However, the visual RS had no effect on the size of the fission and fusion illusions. The present study suggested that RS could affect the SiFI, indicating that the RS effect in different modalities would differentially affect the magnitude of the SiFI. In the process of multisensory integration, the visual and auditory modalities had asymmetrical RS effects.

Enhanced inter-regional coupling of neural responses and repetition suppression provide separate contributions to long-term behavioral priming

Stimulus identification commonly improves with repetition over long delays ("repetition priming"), whereas neural activity commonly decreases ("repetition suppression"). Multiple models have been proposed to explain this brain-behavior relationship, predicting alterations in functional and/or effective connectivity (Synchrony and Predictive Coding models), in the latency of neural responses (Facilitation model), and in the relative similarity of neural representations (Sharpening model). Here, we test these predictions with fMRI during overt and covert naming of repeated and novel objects. While we find partial support for predictions of the Facilitation and Sharpening models in the left fusiform gyrus and left frontal cortex, the data were most consistent with the Synchrony model, with increased coupling between right temporoparietal and anterior cingulate cortex for repeated objects that correlated with priming magnitude across participants. Increased coupling and repetition suppression varied independently, each explaining unique variance in priming and requiring modifications of all current models.

Examination of Driver Visual and Cognitive Responses to Billboard Elicited Passive Distraction Using Eye-Fixation Related Potential

Distractions external to a vehicle contribute to visual attention diversion that may cause traffic accidents. As a low-cost and efficient advertising solution, billboards are widely installed on side of the road, especially the motorway. However, the effect of billboards on driver distraction, eye gaze, and cognition has not been fully investigated. This study utilises a customised driving simulator and synchronised electroencephalography (EEG) and eye tracking system to investigate the cognitive processes relating to the processing of driver visual information. A distinction is made between eye gaze fixations relating to stimuli that assist driving and others that may be a source of distraction. The study compares the driver’s cognitive responses to fixations on billboards with fixations on the vehicle dashboard. The measured eye-fixation related potential (EFRP) shows that the P1 components are similar; however, the subsequent N1 and P2 components differ. In addition, an EEG motor response is observed when the driver makes an adjustment of driving speed when prompted by speed limit signs. The experimental results demonstrate that the proposed measurement system is a valid tool in assessing driver cognition and suggests the cognitive level of engagement to the billboard is likely to be a precursor to driver distraction. The experimental results are compared with the human information processing model found in the literature.

Perceptual Representations and the Vividness of Stimulus-Triggered and Stimulus-Independent Experiences

In recent years, researchers from independent subfields have begun to engage with the idea that the same cortical regions that contribute to on-line perception are recruited during and underlie off-line activities such as information maintenance in working memory, mental imagery, hallucinations, dreaming, and mind wandering. Accumulating evidence suggests that in all of these cases the activity of posterior brain regions provides the contents of experiences. This article is intended to move one step further by exploring specific links between the vividness of experiences, which is a characteristic feature of consciousness regardless of its actual content, and certain properties of the content-specific neural-activity patterns. Investigating the mechanisms that underlie mental imagery and its relation to working memory and the processes responsible for mind wandering and its similarities to dreaming form two clusters of research that are in the forefront of the recent scientific study of mental phenomena, yet communication between these two clusters has been surprisingly sparse. Here our aim is to foster such information exchange by articulating a hypothesis about the fine-grained phenomenological structure determining subjective vividness and its possible neural basis that allows us to shed new light on these mental phenomena by bringing them under a common framework.

Hard criteria for empirical theories of consciousness

Consciousness is now a well-established field of empirical research. A large body of experimental results has been accumulated and is steadily growing. In parallel, many Theories of Consciousness (ToCs) have been proposed. These theories are diverse in nature, ranging from computational to neurophysiological and quantum theoretical approaches. This contrasts with other fields of natural science, which host a smaller number of competing theories. We suggest that one reason for this abundance of extremely different theories may be the lack of stringent criteria specifying how empirical data constrains ToCs. First, we argue that consciousness is a well-defined topic from an empirical point of view and motivate a purely empirical stance on the quest for consciousness. Second, we present a checklist of criteria that, we propose, empirical ToCs need to cope with. Third, we review 13 of the most influential ToCs and subject them to the criteria. Our analysis helps to situate these different ToCs in the theoretical landscapeand sheds light on their strengths and weaknesses from a strictly empirical point of view.

Changes in human brain dynamics during behavioral priming and repetition suppression

Behavioral responses to a perceptual stimulus are typically faster with repeated exposure to the stimulus (behavioral priming). This implicit learning mechanism is critical for survival but impaired in a variety of neurological disorders, including Alzheimer's disease. Many studies of the neural bases for behavioral priming have encountered an interesting paradox: in spite of faster behavioral responses, repeated stimuli usually elicit weaker neural responses (repetition suppression). Several neurophysiological models have been proposed to resolve this paradox, but noninvasive techniques for human studies have had insufficient spatial-temporal precision for testing their predictions. Here, we used the unparalleled precision of electrocorticography (ECoG) to analyze the timing and magnitude of task-related changes in neural activation and propagation while patients named novel vs repeated visual objects. Stimulus repetition was associated with faster verbal responses and decreased neural activation (repetition suppression) in ventral occipito-temporal cortex (VOTC) and left prefrontal cortex (LPFC). Interestingly, we also observed increased neural activation (repetition enhancement) in LPFC and other recording sites. Moreover, with analysis of high gamma propagation we observed increased top-down propagation from LPFC into VOTC, preceding repetition suppression. The latter results indicate that repetition suppression and behavioral priming are associated with strengthening of top-down network influences on perceptual processing, consistent with predictive coding models of repetition suppression, and they support a central role for changes in large-scale cortical dynamics in achieving more efficient and rapid behavioral responses.

Effects of Repetition Suppression on Sound Induced Flash Illusion With Aging

The sound-induced flash illusion (SiFI) is a classical auditory-dominated multisensory integration phenomenon in which the observer misperceives the number of visual flashes due to the simultaneous presentation of a different number of auditory beeps. Although the SiFI has been documented to correlate with perceptual sensitivity, to date there is no consensus as to how it corresponds to sensitivity with aging. The present study was based on the SiFI paradigm (Shams et al., 2000), adding repeated auditory stimuli prior to the appearance of audiovisual stimuli to investigate the effects of repetition suppression (RS) on the SiFI with aging. The repeated auditory stimuli consisted of one or two of the same auditory stimuli presented twice in succession, which were then followed by the audiovisual stimuli. By comparing the illusions in old and young adults, we aimed to explore the influence of aging on the RS of auditory stimuli on the SiFI. The results showed that both age groups showed SiFI effects, however, the RS performance of the two age groups had different effects on the fusion and fission illusions. The illusion effect in old adults was weaker than in young adults. Specifically, RS only affected fission illusions in the old adults but both fission and fusion illusions in young adults. Thus, the present study indicated that the decreased perceptual sensitivity based on auditory RS could weaken the SiFI effect in multisensory integration and that old adults are more susceptible to RS, showing that old adults perceived the SiFI effect weakly under auditory RS.

White dreams are made of colours: What studying contentless dreams can teach about the neural basis of dreaming and conscious experiences

Reports of white dreams, the feeling of having had a dream experience without being able to specify this experience any further, make up almost one third of all dream reports, yet this phenomenon-until very recently-had not yet been in the focus of targeted investigations. White dreams are typically interpreted as forgotten dreams, and are sidelined as not being particularly informative with regard to the nature of dreaming. In this review article, we propose a paradigm shift with respect to the status of white dreams arguing that focusing on this phenomenon can reveal fundamental insights about the neural processes that occur in the dreaming brain. As part of this paradigm shift, we propose a novel interpretation of what white dreams are. This new interpretation is made possible by recent advancements in three different though interrelated fields focusing on dreaming, mental imagery, and wakeful perception. In this paper, we bring these different threads together to show how the latest findings from these fields fit together and point towards a general framework regarding the neural underpinnings of conscious experiences that might turn out to be highly relevant not just for dream research but for all aspects of studying consciousness.

Incremental learning of perceptual and conceptual representations and the puzzle of neural repetition suppression

Incremental learning models of long-term perceptual and conceptual knowledge hold that neural representations are gradually acquired over many individual experiences via Hebbian-like activity-dependent synaptic plasticity across cortical connections of the brain. In such models, variation in task relevance of information, anatomic constraints, and the statistics of sensory inputs and motor outputs lead to qualitative alterations in the nature of representations that are acquired. Here, the proposal that behavioral repetition priming and neural repetition suppression effects are empirical markers of incremental learning in the cortex is discussed, and research results that both support and challenge this position are reviewed. Discussion is focused on a recent fMRI-adaptation study from our laboratory that shows decoupling of experience-dependent changes in neural tuning, priming, and repetition suppression, with representational changes that appear to work counter to the explicit task demands. Finally, critical experiments that may help to clarify and resolve current challenges are outlined.

Repetition suppression: a means to index neural representations using BOLD?

Understanding how the human brain gives rise to complex cognitive processes remains one of the biggest challenges of contemporary neuroscience. While invasive recording in animal models can provide insight into neural processes that are conserved across species, our understanding of cognition more broadly relies upon investigation of the human brain itself. There is therefore an imperative to establish non-invasive tools that allow human brain activity to be measured at high spatial and temporal resolution. In recent years, various attempts have been made to refine the coarse signal available in functional magnetic resonance imaging (fMRI), providing a means to investigate neural activity at the meso-scale, i.e. at the level of neural populations. The most widely used techniques include repetition suppression and multivariate pattern analysis. Human neuroscience can now use these techniques to investigate how representations are encoded across neural populations and transformed by relevant computations. Here, we review the physiological basis, applications and limitations of fMRI repetition suppression with a brief comparison to multivariate techniques. By doing so, we show how fMRI repetition suppression holds promise as a tool to reveal complex neural mechanisms that underlie human cognitive function.

This article is part of the themed issue ‘Interpreting BOLD: a dialogue between cognitive and cellular neuroscience’.

Epicenters of dynamic connectivity in the adaptation of the ventral visual system

OBJECTIVES AND DESIGN

Neuronal responses adapt to familiar and repeated sensory stimuli. Enhanced synchrony across wide brain systems has been postulated as a potential mechanism for this adaptation phenomenon. Here, we used recently developed graph theory methods to investigate hidden connectivity features of dynamic synchrony changes during a visual repetition paradigm. Particularly, we focused on strength connectivity changes occurring at local and distant brain neighborhoods.

PRINCIPAL OBSERVATIONS

We found that connectivity reorganization in visual modal cortex-such as local suppressed connectivity in primary visual areas and distant suppressed connectivity in fusiform areas-is accompanied by enhanced local and distant connectivity in higher cognitive processing areas in multimodal and association cortex. Moreover, we found a shift of the dynamic functional connections from primary-visual-fusiform to primary-multimodal/association cortex.

CONCLUSIONS

These findings suggest that repetition-suppression is made possible by reorganization of functional connectivity that enables communication between low- and high-order areas. Hum Brain Mapp 38:1965-1976, 2017. © 2017 Wiley Periodicals, Inc.

Cultural differences in visual object recognition in 3-year-old children

Recent research indicates that culture penetrates fundamental processes of perception and cognition. Here, we provide evidence that these influences begin early and influence how preschool children recognize common objects. The three tasks (N=128) examined the degree to which nonface object recognition by 3-year-olds was based on individual diagnostic features versus more configural and holistic processing. Task 1 used a 6-alternative forced choice task in which children were asked to find a named category in arrays of masked objects where only three diagnostic features were visible for each object. U.S. children outperformed age-matched Japanese children. Task 2 presented pictures of objects to children piece by piece. U.S. children recognized the objects given fewer pieces than Japanese children, and the likelihood of recognition increased for U.S. children, but not Japanese children, when the piece added was rated by both U.S. and Japanese adults as highly defining. Task 3 used a standard measure of configural progressing, asking the degree to which recognition of matching pictures was disrupted by the rotation of one picture. Japanese children's recognition was more disrupted by inversion than was that of U.S. children, indicating more configural processing by Japanese than U.S. children. The pattern suggests early cross-cultural differences in visual processing; findings that raise important questions about how visual experiences differ across cultures and about universal patterns of cognitive development.

Distinct Visual Processing of Real Objects and Pictures of Those Objects in 7- to 9-month-old Infants

The present study examined 7- and 9-month-old infants' visual habituation to real objects and pictures of the same objects and their preferences between real and pictorial versions of the same objects following habituation. Different hypotheses would predict that infants may habituate faster to pictures than real objects (based on proposed theoretical links between behavioral habituation in infants and neuroimaging adaptation in adults) or to real objects vs. pictures (based on past infant electrophysiology data). Sixty-one 7-month-old infants and fifty-nine 9-month-old infants were habituated to either a real object or a picture of the same object and afterward preference tested with the habituation object paired with either the novel real object or its picture counterpart. Infants of both age groups showed basic information-processing advantages for real objects. Specifically, during the initial presentations, 9-month-old infants looked longer at stimuli in both formats than the 7-month olds but more importantly both age groups looked longer at real objects than pictures, though with repeated presentations, they habituated faster for real objects such that at the end of habituation, they looked equally at both types of stimuli. Surprisingly, even after habituation, infants preferred to look at the real objects, regardless of whether they had habituated to photos or real objects. Our findings suggest that from as early as 7-months of age, infants show strong preferences for real objects, perhaps because real objects are visually richer and/or enable the potential for genuine interactions.

Age-related differences in the neural basis of the subjective vividness of memories: evidence from multivoxel pattern classification

Although older adults often show reduced episodic memory accuracy, their ratings of the subjective vividness of their memories often equal or even exceed those of young adults. Such findings suggest that young and older adults may differentially access and/or weight different kinds of information in making vividness judgments. We examined this idea using multivoxel pattern classification of fMRI data to measure category representations while participants saw and remembered pictures of objects and scenes. Consistent with our hypothesis, there were age-related differences in how category representations related to the subjective sense of vividness. During remembering, older adults' vividness ratings were more related, relative to young adults', to category representations in prefrontal cortex. In contrast, young adults' vividness ratings were more related, relative to older adults, to category representations in parietal cortex. In addition, category representations were more correlated among posterior regions in young than in older adults, whereas correlations between PFC and posterior regions did not differ between the 2 groups. Together, these results are consistent with the idea that young and older adults differentially weight different types of information in assessing subjective vividness of their memories.

Repetition suppression and its contextual determinants in predictive coding

This paper presents a review of theoretical and empirical work on repetition suppression in the context of predictive coding. Predictive coding is a neurobiologically plausible scheme explaining how biological systems might perform perceptual inference and learning. From this perspective, repetition suppression is a manifestation of minimising prediction error through adaptive changes in predictions about the content and precision of sensory inputs. Simulations of artificial neural hierarchies provide a principled way of understanding how repetition suppression – at different time scales – can be explained in terms of inference and learning implemented under predictive coding. This formulation of repetition suppression is supported by results of numerous empirical studies of repetition suppression and its contextual determinants.

Lateral and Medial Ventral Occipitotemporal Regions Interact During the Recognition of Images Revealed from Noise

Several studies suggest different functional roles for the medial and the lateral sections of the ventral visual cortex in object recognition. Texture and surface information is processed in medial sections, while shape information is processed in lateral sections. This begs the question whether and how these functionally specialized sections interact with each other and with early visual cortex to facilitate object recognition. In the current research, we set out to answer this question. In an fMRI study, 13 subjects viewed and recognized images of objects and animals that were gradually revealed from noise while their brains were being scanned. We applied dynamic causal modeling (DCM)-a method to characterize network interactions-to determine the modulatory effect of object recognition on a network comprising the primary visual cortex (V1), the lingual gyrus (LG) in medial ventral cortex and the lateral occipital cortex (LO). We found that object recognition modulated the bilateral connectivity between LG and LO. Moreover, the feed-forward connectivity from V1 to LG and LO was modulated, while there was no evidence for feedback from these regions to V1 during object recognition. In particular, the interaction between medial and lateral areas supports a framework in which visual recognition of objects is achieved by networked regions that integrate information on image statistics, scene content and shape-rather than by a single categorically specialized region-within the ventral visual cortex.

Selecting appropriate designs and comparison conditions in repetition paradigms

The studies described by Vogels (this issue) demonstrate the complexity of repetition effects in the visual processing stream. In addition to signal suppression, findings of inherited effects from earlier processing, and discrepancies between the stimulus selectivity of cells before and after repetition, have informed the inferences that can be drawn from measures over larger scales such as functional magnetic resonance imaging (fMRI) or electroencephalography (EEG). This work also demonstrates that integration of evidence across recording methods is vital for understanding repetition effects in the brain. It is however difficult to integrate evidence across different recording methods and repetition paradigms. At the crux of this difficulty is the selection of comparison or unrepeated stimulus conditions within paradigms, which influence the observed strength, selectivity and even direction of repetition effects. This viewpoint highlights prevalent methodological issues with regard to repeated-unrepeated stimulus comparisons: inherited adaptation, stimulus specific expectations, concurrent memory retrieval, stimulus novelty and familiarity, attention, and changes in neuronal selectivity with repetition. The extent to which current conflicting and uncertain findings are due to selection of unrepeated stimulus conditions is unknown, but needs to be addressed to develop models of repetition spanning recording methods and repetition paradigms.

Involvement of the TPJ Area in Processing of Novel Global Forms

The neuropsychological syndrome “simultanagnosia” is characterized by the inability to integrate local elements into a global entity. This deficit in Gestalt perception is mainly apparent for novel global structures administered in clinical tests or unfamiliar visual scenes. Recognition of familiar complex objects or well-known visual scenes is often unaffected. Recent neuroimaging studies and reports from simultanagnosia patients suggest a crucial involvement of temporoparietal brain areas in processing of hierarchically organized visual material. In this study, we investigated the specific role of the TPJ in Gestalt perception. On the basis of perceptual characteristics known from simultanagnosia, we hypothesized that TPJ is dominantly involved in processing of novel object arrangements. To answer this question, we performed a learning study with hierarchical stimuli and tested behavioral and neuronal characteristics of Gestalt perception pre- and posttraining. The study included 16 psychophysical training sessions and two neuroimaging sessions. Participants improved their behavioral performance for trained global stimuli and showed limited transfer to untrained global material. We found significant training dependent neuronal signal modulations in anterior right hemispheric TPJ regions. These activation changes were specific to trained global stimuli, whereas no systematic neuronal response changes were observed for recognition of untrained global stimuli, local elements and regular objects that served as control stimuli. In line with perceptual characteristics in simultanagnosia, the results argue for an involvement of TPJ in processing of novel global structures. We discuss the signal modulations in the context of a more efficient or different neuronal strategy to process familiar global stimuli.

Content-specific evidence accumulation in inferior temporal cortex during perceptual decision-making

During a perceptual decision, neuronal activity can change as a function of time-integrated evidence. Such neurons may serve as decision variables, signaling a choice when activity reaches a boundary. Because the signals occur on a millisecond timescale, translating to human decision-making using functional neuroimaging has been challenging. Previous neuroimaging work in humans has identified patterns of neural activity consistent with an accumulation account. However, the degree to which the accumulating neuroimaging signals reflect specific sources of perceptual evidence is unknown. Using an extended face/house discrimination task in conjunction with cognitive modeling, we tested whether accumulation signals, as measured using functional magnetic resonance imaging (fMRI), are stimulus-specific. Accumulation signals were defined as a change in the slope of the rising edge of activation corresponding with response time (RT), with higher slopes associated with faster RTs. Consistent with an accumulation account, fMRI activity in face- and house-selective regions in the inferior temporal cortex increased at a rate proportional to decision time in favor of the preferred stimulus. This finding indicates that stimulus-specific regions perform an evidence integrative function during goal-directed behavior and that different sources of evidence accumulate separately. We also assessed the decision-related function of other regions throughout the brain and found that several regions were consistent with classifications from prior work, suggesting a degree of domain generality in decision processing. Taken together, these results provide support for an integration-to-boundary decision mechanism and highlight possible roles of both domain-specific and domain-general regions in decision evidence evaluation.

Stochastic process underlying emergent recognition of visual objects hidden in degraded images

When a degraded two-tone image such as a “Mooney” image is seen for the first time, it is unrecognizable in the initial seconds. The recognition of such an image is facilitated by giving prior information on the object, which is known as top-down facilitation and has been intensively studied. Even in the absence of any prior information, however, we experience sudden perception of the emergence of a salient object after continued observation of the image, whose processes remain poorly understood. This emergent recognition is characterized by a comparatively long reaction time ranging from seconds to tens of seconds. In this study, to explore this time-consuming process of emergent recognition, we investigated the properties of the reaction times for recognition of degraded images of various objects. The results show that the time-consuming component of the reaction times follows a specific exponential function related to levels of image degradation and subject's capability. Because generally an exponential time is required for multiple stochastic events to co-occur, we constructed a descriptive mathematical model inspired by the neurophysiological idea of combination coding of visual objects. Our model assumed that the coincidence of stochastic events complement the information loss of a degraded image leading to the recognition of its hidden object, which could successfully explain the experimental results. Furthermore, to see whether the present results are specific to the task of emergent recognition, we also conducted a comparison experiment with the task of perceptual decision making of degraded images, which is well known to be modeled by the stochastic diffusion process. The results indicate that the exponential dependence on the level of image degradation is specific to emergent recognition. The present study suggests that emergent recognition is caused by the underlying stochastic process which is based on the coincidence of multiple stochastic events.

The strength of gradually accruing probabilistic evidence modulates brain activity during a categorical decision

The evolution of neural activity during a perceptual decision is well characterized by the evidence parameter in sequential sampling models. However, it is not known whether accumulating signals in human neuroimaging are related to the integration of evidence. Our aim was to determine whether activity accumulates in a nonperceptual task by identifying brain regions tracking the strength of probabilistic evidence. fMRI was used to measure whole-brain activity as choices were informed by integrating a series of learned prior probabilities. Participants first learned the predictive relationship between a set of shape stimuli and one of two choices. During scanned testing, they made binary choices informed by the sum of the predictive strengths of individual shapes. Sequences of shapes adhered to three distinct rates of evidence (RoEs): rapid, gradual, and switch. We predicted that activity in regions informing the decision would modulate as a function of RoE prior to the choice. Activity in some regions, including premotor areas, changed as a function of RoE and response hand, indicating a role in forming an intention to respond. Regions in occipital, temporal, and parietal lobes modulated as a function of RoE only, suggesting a preresponse stage of evidence processing. In all of these regions, activity was greatest on rapid trials and least on switch trials, which is consistent with an accumulation-to-boundary account. In contrast, activity in a set of frontal and parietal regions was greatest on switch and least on rapid trials, which is consistent with an effort or time-on-task account.

Multisensory convergence of visual and haptic object preference across development

Visuohaptic inputs offer redundant and complementary information regarding an object׳s geometrical structure. The integration of these inputs facilitates object recognition in adults. While the ability to recognize objects in the environment both visually and haptically develops early on, the development of the neural mechanisms for integrating visual and haptic object shape information remains unknown. In the present study, we used functional Magnetic Resonance Imaging (fMRI) in three groups of participants, 4 to 5.5 year olds, 7 to 8.5 year olds, and adults. Participants were tested in a block design involving visual exploration of two-dimensional images of common objects and real textures, and haptic exploration of their three-dimensional counterparts. As in previous studies, object preference was defined as a greater BOLD response for objects than textures. The analyses specifically target two sites of known visuohaptic convergence in adults: the lateral occipital tactile-visual region (LOtv) and intraparietal sulcus (IPS). Results indicated that the LOtv is involved in visuohaptic object recognition early on. More importantly, object preference in the LOtv became increasingly visually dominant with development. Despite previous reports that the lateral occipital complex (LOC) is adult-like by 8 years, these findings indicate that at least part of the LOC is not. Whole-brain maps showed overlap between adults and both groups of children in the LOC. However, the overlap did not build incrementally from the younger to the older group, suggesting that visuohaptic object preference does not develop in an additive manner. Taken together, the results show that the development of neural substrates for visuohaptic recognition is protracted compared to substrates that are primarily visual or haptic.

Single-trial analysis of neuroimaging data: inferring neural networks underlying perceptual decision-making in the human brain

Advances in neural signal and image acquisition as well as in multivariate signal processing and machine learning are enabling a richer and more rigorous understanding of the neural basis of human decision-making. Decision-making is essentially characterized behaviorally by the variability of the decision across individual trials--e.g., error and response time distributions. To infer the neural processes that govern decision-making requires identifying neural correlates of such trial-to-trial behavioral variability. In this paper, we review efforts that utilize signal processing and machine learning to enable single-trial analysis of neural signals acquired while subjects perform simple decision-making tasks. Our focus is on neuroimaging data collected noninvasively via electroencephalograpy (EEG) and functional magnetic resonance imaging (fMRI). We review the specific framework for extracting decision-relevant neural components from the neuroimaging data, the goal being to analyze the trial-to-trial variability of the neural signal along these component directions and to relate them to elements of the decision-making process. We review results for perceptual decision-making and discrimination tasks, including paradigms in which EEG variability is used to inform an fMRI analysis. We discuss how single-trial analysis reveals aspects of the underlying decision-making networks that are unobservable using traditional trial-averaging methods.

Imaging prior information in the brain

In making sense of the visual world, the brain's processing is driven by two factors: the physical information provided by the eyes ("bottom-up" data) and the expectancies driven by past experience ("top-down" influences). We use degraded stimuli to tease apart the effects of bottom-up and top-down processes because they are easier to recognize with prior knowledge of undegraded images. Using machine learning algorithms, we quantify the amount of information that brain regions contain about stimuli as the subject learns the coherent images. Our results show that several distinct regions, including high-level visual areas and the retinotopic cortex, contain more information about degraded stimuli with prior knowledge. Critically, these regions are separate from those that exhibit classical priming, indicating that top-down influences are more than feature-based attention. Together, our results show how the neural processing of complex imagery is rapidly influenced by fleeting experiences.

Repetition Priming and Repetition Suppression: A Case for Enhanced Efficiency Through Neural Synchronization

Stimulus repetition in identification tasks leads to improved behavioral performance ("repetition priming") but attenuated neural responses ("repetition suppression") throughout task-engaged cortical regions. While it's clear that this pervasive brain-behavior relationship reflects some form of improved processing efficiency, the exact form that it takes remains elusive. In this Discussion Paper, we review four different theoretical proposals that have the potential to link repetition suppression and priming, with a particular focus on a proposal that stimulus repetition affects improved efficiency through enhanced neural synchronization. We argue that despite exciting recent work on the role of neural synchronization in cognitive processes such as attention and perception, similar studies in the domain of learning and memory - and priming, in particular - have been lacking. We emphasize the need for new studies with adequate spatiotemporal resolution, formulate several novel predictions, and discuss our ongoing efforts to disentangle the current proposals.

Repetition priming and repetition suppression: Multiple mechanisms in need of testing

In our Discussion Paper, we reviewed four theoretical proposals that have the potential to link the neural and behavioral phenomena of Repetition Suppression and Repetition Priming. We argued that among these proposals, the Synchrony and Bayesian Explaining Away models appear to be the most promising in addressing existing data, and we articulated a series of predictions to distinguish between them. The commentaries have helped to clarify some of these predictions, have highlighted additional evidence supporting the Facilitation and Sharpening models, and have emphasized dissociations by repetition lag and brain location. Our reply addresses these issues in turn, and we argue that progress will require the testing of Repetition Suppression, changes in neural tuning, and changes in synchronization throughout the brain and over a variety of lags and task contexts.

Predictive Context Influences Perceptual Selection during Binocular Rivalry

Prediction may be a fundamental principle of sensory processing: it has been proposed that the brain continuously generates predictions about forthcoming sensory information. However, little is known about how prediction contributes to the selection of a conscious percept from among competing alternatives. Here, we used binocular rivalry to investigate the effects of prediction on perceptual selection. In binocular rivalry, incompatible images presented to the two eyes result in a perceptual alternation between the images, even though the visual stimuli remain constant. If predictive signals influence the competition between neural representations of rivalrous images, this influence should generate a bias in perceptual selection that depends on predictive context. To manipulate predictive context, we developed a novel binocular rivalry paradigm in which rivalrous test images were immediately preceded by a sequence of context images presented identically to the two eyes. One of the test images was consistent with the preceding image sequence (it was the expected next image in the series), and the other was inconsistent (non-predicted). We found that human observers were more likely to perceive the consistent image at the onset of rivalry, suggesting that predictive context biased selection in favor of the predicted percept. This prediction effect was distinct from the effects of adaptation to stimuli presented before the binocular rivalry test. In addition, perceptual reports were speeded for predicted percepts relative to non-predicted percepts. These results suggest that predictive signals related to visual stimulus history exist at neural sites that can bias conscious perception during binocular rivalry. Our paradigm provides a new way to study how prior information and incoming sensory information combine to generate visual percepts.

Face categorization in visual scenes may start in a higher order area of the right fusiform gyrus: evidence from dynamic visual stimulation in neuroimaging

How a visual stimulus is initially categorized as a face by the cortical face-processing network remains largely unclear. In this study we used functional MRI to study the dynamics of face detection in visual scenes by using a paradigm in which scenes containing faces or cars are revealed progressively as they emerge from visual noise. Participants were asked to respond as soon as they detected a face or car during the noise sequence. Among the face-sensitive regions identified based on a standard localizer, a high-level face-sensitive area, the right fusiform face area (FFA), showed the earliest difference between face and car activation. Critically, differential activation in FFA was observed before differential activation in the more posteriorly located occipital face area (OFA). A whole brain analysis confirmed these findings, with a face-sensitive cluster in the right fusiform gyrus being the only cluster showing face preference before successful behavioral detection. Overall, these findings indicate that following generic low-level visual analysis, a face stimulus presented in a gradually revealed visual scene is first detected in the right middle fusiform gyrus, only after which further processing spreads to a network of cortical and subcortical face-sensitive areas (including the posteriorly located OFA). These results provide further evidence for a nonhierarchical organization of the cortical face-processing network.

Neural representations for object perception: structure, category, and adaptive coding

Object perception is one of the most remarkable capacities of the primate brain. Owing to the large and indeterminate dimensionality of object space, the neural basis of object perception has been difficult to study and remains controversial. Recent work has provided a more precise picture of how 2D and 3D object structure is encoded in intermediate and higher-level visual cortices. Yet, other studies suggest that higher-level visual cortex represents categorical identity rather than structure. Furthermore, object responses are surprisingly adaptive to changes in environmental statistics, implying that learning through evolution, development, and also shorter-term experience during adulthood may optimize the object code. Future progress in reconciling these findings will depend on more effective sampling of the object domain and direct comparison of these competing hypotheses.

The functional neuroanatomy of object agnosia: a case study

Cortical reorganization of visual and object representations following neural injury was examined using fMRI and behavioral investigations. We probed the visual responsivity of the ventral visual cortex of an agnosic patient who was impaired at object recognition following a lesion to the right lateral fusiform gyrus. In both hemispheres, retinotopic mapping revealed typical topographic organization and visual activation of early visual cortex. However, visual responses, object-related, and -selective responses were reduced in regions immediately surrounding the lesion in the right hemisphere, and also, surprisingly, in corresponding locations in the structurally intact left hemisphere. In contrast, hV4 of the right hemisphere showed expanded response properties. These findings indicate that the right lateral fusiform gyrus is critically involved in object recognition and that an impairment to this region has widespread consequences for remote parts of cortex. Finally, functional neural plasticity is possible even when a cortical lesion is sustained in adulthood.

Expectations change the signatures and timing of electrophysiological correlates of perceptual awareness

Previous experience allows the brain to predict what comes next. How these expectations affect conscious experience is poorly understood. In particular, it is unknown whether and when expectations interact with sensory evidence in granting access to conscious perception, and how this is reflected electrophysiologically. Here, we parametrically manipulate sensory evidence and expectations while measuring event-related potentials in human subjects to assess the time course of evoked responses that correlate with subjective visibility, the properties of the stimuli, and/or perceptual expectations. We found that expectations lower the threshold of conscious perception and reduce the latency of neuronal signatures differentiating seen and unseen stimuli. Without expectations, this differentiation occurs ∼300 ms and with expectations ∼200 ms after stimulus in occipitoparietal sensors. The amplitude of this differentiating response component (P2) decreases as visibility increases, regardless of whether this increase is attributable to enhanced sensory evidence and/or the gradual buildup of perceptual expectations. Importantly, at matched performance levels, responses to seen and unseen stimuli differed regardless of the physical stimulus properties. These findings indicate that the latency of the neuronal correlates of access to consciousness depend on whether access is driven by stimulus saliency or by a combination of expectations and sensory evidence.

Brain dynamics of upstream perceptual processes leading to visual object recognition: a high density ERP topographic mapping study

Recent studies suggest that visual object recognition is a proactive process through which perceptual evidence accumulates over time before a decision can be made about the object. However, the exact electrophysiological correlates and time-course of this complex process remain unclear. In addition, the potential influence of emotion on this process has not been investigated yet. We recorded high density EEG in healthy adult participants performing a novel perceptual recognition task. For each trial, an initial blurred visual scene was first shown, before the actual content of the stimulus was gradually revealed by progressively adding diagnostic high spatial frequency information. Participants were asked to stop this stimulus sequence as soon as they could correctly perform an animacy judgment task. Behavioral results showed that participants reliably gathered perceptual evidence before recognition. Furthermore, prolonged exploration times were observed for pleasant, relative to either neutral or unpleasant scenes. ERP results showed distinct effects starting at 280 ms post-stimulus onset in distant brain regions during stimulus processing, mainly characterized by: (i) a monotonic accumulation of evidence, involving regions of the posterior cingulate cortex/parahippocampal gyrus, and (ii) true categorical recognition effects in medial frontal regions, including the dorsal anterior cingulate cortex. These findings provide evidence for the early involvement, following stimulus onset, of non-overlapping brain networks during proactive processes eventually leading to visual object recognition.

Characterizing the spatio-temporal dynamics of the neural events occurring prior to and up to overt recognition of famous faces

Although it is generally acknowledged that familiar face recognition is fast, mandatory, and proceeds outside conscious control, it is still unclear whether processes leading to familiar face recognition occur in a linear (i.e., gradual) or a nonlinear (i.e., all-or-none) manner. To test these two alternative accounts, we recorded scalp ERPs while participants indicated whether they recognize as familiar the faces of famous and unfamiliar persons gradually revealed in a descending sequence of frames, from the noisier to the least noisy. This presentation procedure allowed us to characterize the changes in scalp ERP responses occurring prior to and up to overt recognition. Our main finding is that gradual and all-or-none processes are possibly involved during overt recognition of familiar faces. Although the N170 and the N250 face-sensitive responses displayed an abrupt activity change at the moment of overt recognition of famous faces, later ERPs encompassing the N400 and late positive component exhibited an incremental increase in amplitude as the point of recognition approached. In addition, famous faces that were not overtly recognized at one trial before recognition elicited larger ERP potentials than unfamiliar faces, probably reflecting a covert recognition process. Overall, these findings present evidence that recognition of familiar faces implicates spatio-temporally complex neural processes exhibiting differential pattern activity changes as a function of recognition state.

Observing Learned Object-specific Functional Grasps Preferentially Activates the Ventral Stream

In one popular account of the human visual system, two streams are distinguished, a ventral stream specialized for perception and a dorsal stream specialized for action. The skillful use of familiar tools, however, is likely to involve the cooperation of both streams. Using functional magnetic resonance imaging, we scanned individuals while they viewed short movies of familiar tools being grasped in ways that were either consistent or inconsistent with how tools are typically grasped during use. Typical-for-use actions were predicted to preferentially activate parietal areas important for tool use. Instead, our results revealed several areas within the ventral stream, as well as the left posterior middle temporal gyrus, as preferentially active for our typical-for-use actions. We believe these findings reflect sensitivity to learned semantic associations and suggest a special role for these areas in representing object-specific actions. We hypothesize that during actual tool use a complex interplay between the two streams must take place, with ventral stream areas providing critical input as to how an object should be engaged in accordance with stored semantic knowledge.

When seeing depends on knowing: adults with Autism Spectrum Conditions show diminished top-down processes in the visual perception of degraded faces but not degraded objects

Behavioural, neuroimaging and neurophysiological approaches emphasise the active and constructive nature of visual perception, determined not solely by the environmental input, but modulated top-down by prior knowledge. For example, degraded images, which at first appear as meaningless 'blobs', can easily be recognized as, say, a face, after having seen the same image un-degraded. This conscious perception of the fragmented stimuli relies on top-down priming influences from systems involved in attention and mental imagery on the processing of stimulus attributes, and feature-binding [Dolan, R. J., Fink, G. R., Rolls, E., Booth, M., Holmes, A., Frackowiak, R. S. J., et al. (1997). How the brain learns to see objects and faces in an impoverished context. Nature, 389, 596-599]. In Autism Spectrum Conditions (ASC), face processing abnormalities are well-established, but top-down anomalies in various domains have also been shown. Thus, we tested two alternative hypotheses: (i) that people with ASC show overall reduced top-down modulation in visual perception, or (ii) that top-down anomalies affect specifically the perception of faces. Participants were presented with sets of three consecutive images: degraded images (of faces or objects), corresponding or non-corresponding grey-scale photographs, and the same degraded images again. In a passive viewing sequence we compared gaze times (an index of focal attention) on faces/objects vs. background before and after viewers had seen the undegraded photographs. In an active viewing sequence, we compared how many faces/objects were identified pre- and post-exposure. Behavioural and gaze tracking data showed significantly reduced effects of prior knowledge on the conscious perception of degraded faces, but not objects in the ASC group. Implications for future work on the underlying mechanisms, at the cognitive and neurofunctional levels, are discussed.