How thoughts arise from sights: inferotemporal and prefrontal contributions to vision

What does decoding from the PFC reveal about consciousness?

Disputes between rival theories of consciousness have often centered on whether perceptual contents can be decoded from the prefrontal cortex (PFC). Failures to decode from the PFC are taken to challenge 'cognitive' theories of consciousness such as the global workspace theory and higher-order monitoring theories, and decoding successes have been taken to confirm these theories. However, PFC decoding shows both too much and too little. Too much because cognitive theories of consciousness do not need PFC rerepresentation of perceptual contents since pointers to perceptual representations suffice. Too little because there is evidence that PFC decoding of perceptual content reflects postperceptual cognitive representation, such as thoughts that have those perceptual contents rather than conscious percepts.

Long-range cortical synchronization supports abrupt visual learning

Visual plasticity declines sharply after the critical period, yet we easily learn to recognize new faces and places, even as adults. Such learning is often characterized by a "moment of insight," an abrupt and dramatic improvement in recognition. The mechanisms that support abrupt learning are unknown, but one hypothesis is that they involve changes in synchronization between brain regions. To test this hypothesis, we used a behavioral task in which non-human primates rapidly learned to recognize novel images and to associate them with specific responses. Simultaneous recordings from inferotemporal and prefrontal cortices revealed a transient synchronization of neural activity between these areas that peaked around the moment of insight. Synchronization was strongest between inferotemporal sites that encoded images and reward-sensitive prefrontal sites. Moreover, its magnitude intensified gradually over image exposures, suggesting that abrupt learning is the culmination of a search for informative signals within a circuit linking sensory information to task demands.

Neural network models for DMT-induced visual hallucinations

The regulatory role of the serotonergic system on conscious perception can be investigated perturbatorily with psychedelic drugs such as N,N-Dimethyltryptamine. There is increasing evidence that the serotonergic system gates prior (endogenous) and sensory (exogenous) information in the construction of a conscious experience. Using two generative deep neural networks as examples, we discuss how such models have the potential to be, firstly, an important medium to illustrate phenomenological visual effects of psychedelics-besides paintings, verbal reports and psychometric testing-and, secondly, their utility to conceptualize biological mechanisms of gating the influence of exogenous and endogenous information on visual perception.

Dynamic cognitive processes of text-picture integration revealed by event-related potentials

The integration of text and picture is the core of multimedia information processing. Relevant theories suggest that text and picture are processed through different channels in the early stage, and integrated in the late stage of processing. Based on these theories, the current study adopted measures of event-related potentials to examine the cognitive and neural processes of text-picture integration. The results showed that in the early stage of text-picture integration, picture processing evoked a more negative N1 over the occipital area and a N300 over the prefrontal area, which might reflect the discrimination process of visual stimuli and the imagery representation of the picture, respectively; in the late stage, the text-picture induced a N400 in the central area and an LPC over the central, parietal and temporal areas, which might be associated with the semantic activation and integration of text and picture, respectively. These results not only provide support for existing theories, but also further elucidate the dynamic neural processing of text-picture integration in terms of its temporal and spatial characteristics.

Representation of abstract semantic knowledge in populations of human single neurons in the medial temporal lobe

Sensory experience elicits complex activity patterns throughout the neocortex. Projections from the neocortex converge onto the medial temporal lobe (MTL), in which distributed neocortical firing patterns are distilled into sparse representations. The precise nature of these neuronal representations is still unknown. Here, we show that population activity patterns in the MTL are governed by high levels of semantic abstraction. We recorded human single-unit activity in the MTL (4,917 units, 25 patients) while subjects viewed 100 images grouped into 10 semantic categories of 10 exemplars each. High levels of semantic abstraction were indicated by representational similarity analyses (RSAs) of patterns elicited by individual stimuli. Moreover, pattern classifiers trained to decode semantic categories generalised successfully to unseen exemplars, and classifiers trained to decode exemplar identity more often confused exemplars of the same versus different categories. Semantic abstraction and generalisation may thus be key to efficiently distill the essence of an experience into sparse representations in the human MTL. Although semantic abstraction is efficient and may facilitate generalisation of knowledge to novel situations, it comes at the cost of a loss of detail and may be central to the generation of false memories.

Single-neuron representations of stimuli in the human medial temporal lobe at the population level are governed by highly abstract semantic principles, but the attendant efficiency and potential for generalization comes at the cost of confusion between related stimuli.

What is the neuronal code for sensory experience in the human medial temporal lobe (MTL)? Single-cell electrophysiology in the awake human brain during chronic, invasive epilepsy monitoring has previously revealed the existence of so-called concept cells. These cells have been found to increase their firing rate in response to, for example, the famous tennis player ‘Roger Federer’, whether his name is spoken by a computer voice or a picture of him is presented on a computer screen. These neurons thus seem to encode the semantic content of a stimulus, regardless of the sensory modality through which it is delivered. Previous work has predominantly focused on individual neurons that were selected based on their strong response to a particular stimulus using rather conservative statistical criteria. Those studies stressed that concept cells encode a single, concrete concept in an all-or-nothing fashion. Here, we analysed the neuronal code on the level of the entire population of neurons without any preselection. We conducted representational similarity analyses (RSAs) and pattern classification analyses of firing patterns evoked by visual stimuli (for example, a picture of an apple) that could be grouped into semantic categories on multiple levels of abstraction (‘fruit’, ‘food’, ‘natural things’). We found that neuronal activation patterns contain information on higher levels of categorical abstraction rather than just the level of individual exemplars. On the one hand, the neuronal code in the human MTL thus seems well suited to generalise semantic knowledge to new situations; on the other hand, it could also be responsible for the generation of false memories.

Neural mechanisms of attending to items in working memory

Working memory, the ability to keep recently accessed information available for immediate manipulation, has been proposed to rely on two mechanisms that appear difficult to reconcile: self-sustained neural firing, or the opposite-activity-silent synaptic traces. Here we review and contrast models of these two mechanisms, and then show that both phenomena can co-exist within a unified system in which neurons hold information in both activity and synapses. Rapid plasticity in flexibly-coding neurons allows features to be bound together into objects, with an important emergent property being the focus of attention. One memory item is held by persistent activity in an attended or "focused" state, and is thus remembered better than other items. Other, previously attended items can remain in memory but in the background, encoded in activity-silent synaptic traces. This dual functional architecture provides a unified common mechanism accounting for a diversity of perplexing attention and memory effects that have been hitherto difficult to explain in a single theoretical framework.

State-dependent encoding of sound and behavioral meaning in a tertiary region of the ferret auditory cortex

In higher sensory cortices, there is a gradual transformation from sensation to perception and action. In the auditory system, this transformation is revealed by responses in the rostral ventral posterior auditory field (VPr), a tertiary area in the ferret auditory cortex, which shows long-term learning in trained compared to naïve animals, arising from selectively enhanced responses to behaviorally relevant target stimuli. This enhanced representation is further amplified during active performance of spectral or temporal auditory discrimination tasks. VPr also shows sustained short-term memory activity after target stimulus offset, correlated with task response timing and action. These task-related changes in auditory filter properties enable VPr neurons to quickly and nimbly switch between different responses to the same acoustic stimuli, reflecting either spectrotemporal properties, timing, or behavioral meaning of the sound. Furthermore, they demonstrate an interaction between the dynamics of short-term attention and long-term learning, as incoming sound is selectively attended, recognized, and translated into action.

How biological attention mechanisms improve task performance in a large-scale visual system model

How does attentional modulation of neural activity enhance performance? Here we use a deep convolutional neural network as a large-scale model of the visual system to address this question. We model the feature similarity gain model of attention, in which attentional modulation is applied according to neural stimulus tuning. Using a variety of visual tasks, we show that neural modulations of the kind and magnitude observed experimentally lead to performance changes of the kind and magnitude observed experimentally. We find that, at earlier layers, attention applied according to tuning does not successfully propagate through the network, and has a weaker impact on performance than attention applied according to values computed for optimally modulating higher areas. This raises the question of whether biological attention might be applied at least in part to optimize function rather than strictly according to tuning. We suggest a simple experiment to distinguish these alternatives.