Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study

Combined Whole Eye and Face Transplant: Microsurgical Strategy and 1-Year Clinical Course

Importance

Catastrophic facial injury with globe loss remains a formidable clinical problem with no previous reports of reconstruction by whole eye or combined whole eye and facial transplant.

Objective

To develop a microsurgical strategy for combined whole eye and facial transplant and describe the clinical findings during the first year following transplant.

Design, Setting, and Participant

A 46-year-old man who sustained a high-voltage electrical injury with catastrophic tissue loss to his face and left globe underwent combined whole eye and face transplant using personalized surgical devices and a novel microsurgical strategy at a specialized center for vascularized composite allotransplantation.

Main Outcomes and Measures

Reperfusion and viability of the whole eye and facial allografts, retinal function, and incidence of acute rejection.

Results

The patient underwent a combined whole eye and face transplant from an immunologically compatible donor with primary optic nerve coaptation and conventional postoperative immunosuppression. Globe and retinal perfusion were maintained throughout the immediate postoperative period, evidenced by fluorescein angiography. Optical coherence tomography demonstrated atrophy of inner retinal layers and attenuation and disruption of the ellipsoid zone. Serial electroretinography confirmed retinal responses to light in the transplanted eye. Using structural and functional magnetic resonance imaging, the integrity of the transplanted visual pathways and potential occipital cortical response to light stimulation of the transplanted eye was demonstrated. At 1 year post transplant (postoperative day 366), there was no perception of light in the transplanted eye.

Conclusions and Relevance

This is the first report of whole eye transplant combined with facial transplant, demonstrating allograft survival including rejection-free graft survival and electroretinographic measurements indicating retinal response to light stimuli. These data highlight the potential for clinical allotransplantation for globe loss.

Hierarchical syntax model of music predicts theta power during music listening

Linguistic research showed that the depth of syntactic embedding is reflected in brain theta power. Here, we test whether this also extends to non-linguistic stimuli, specifically music. We used a hierarchical model of musical syntax to continuously quantify two types of expert-annotated harmonic dependencies throughout a piece of Western classical music: prolongation and preparation. Prolongations can roughly be understood as a musical analogue to linguistic coordination between constituents that share the same function (e.g., 'pizza' and 'pasta' in 'I ate pizza and pasta'). Preparation refers to the dependency between two harmonies whereby the first implies a resolution towards the second (e.g., dominant towards tonic; similar to how the adjective implies the presence of a noun in 'I like spicy … '). Source reconstructed MEG data of sixty-five participants listening to the musical piece was then analysed. We used Bayesian Mixed Effects models to predict theta envelope in the brain, using the number of open prolongation and preparation dependencies as predictors whilst controlling for audio envelope. We observed that prolongation and preparation both carry independent and distinguishable predictive value for theta band fluctuation in key linguistic areas such as the Angular, Superior Temporal, and Heschl's Gyri, or their right-lateralised homologues, with preparation showing additional predictive value for areas associated with the reward system and prediction. Musical expertise further mediated these effects in language-related brain areas. Results show that predictions of precisely formalised music-theoretical models are reflected in the brain activity of listeners which furthers our understanding of the perception and cognition of musical structure.

Antipsychotic drugs selectively decorrelate long-range interactions in deep cortical layers

Psychosis is characterized by a diminished ability of the brain to distinguish externally driven activity patterns from self-generated activity patterns. Antipsychotic drugs are a class of small molecules with relatively broad binding affinity for a variety of neuromodulator receptors that, in humans, can prevent or ameliorate psychosis. How these drugs influence the function of cortical circuits, and in particular their ability to distinguish between externally and self-generated activity patterns, is still largely unclear. To have experimental control over self-generated sensory feedback, we used a virtual reality environment in which the coupling between movement and visual feedback can be altered. We then used widefield calcium imaging to determine the cell type-specific functional effects of antipsychotic drugs in mouse dorsal cortex under different conditions of visuomotor coupling. By comparing cell type-specific activation patterns between locomotion onsets that were experimentally coupled to self-generated visual feedback and locomotion onsets that were not coupled, we show that deep cortical layers were differentially activated in these two conditions. We then show that the antipsychotic drug clozapine disrupted visuomotor integration at locomotion onsets also primarily in deep cortical layers. Given that one of the key components of visuomotor integration in cortex is long-range cortico-cortical connections, we tested whether the effect of clozapine was detectable in the correlation structure of activity patterns across dorsal cortex. We found that clozapine as well as two other antipsychotic drugs, aripiprazole and haloperidol, resulted in a strong reduction in correlations of layer 5 activity between cortical areas and impaired the spread of visuomotor prediction errors generated in visual cortex. Our results are consistent with the interpretation that a major functional effect of antipsychotic drugs is a selective alteration of long-range layer 5-mediated communication.

Similarity in evoked responses does not imply similarity in macroscopic network states

It is commonplace in neuroscience to assume that if two tasks activate the same brain areas in the same way, then they are recruiting the same underlying networks. Yet computational theory has shown that the same pattern of activity can emerge from many different underlying network representations. Here we evaluated whether similarity in activation necessarily implies similarity in network architecture by comparing region-wise activation patterns and functional correlation profiles from a large sample of healthy subjects (N = 242). Participants performed two executive control tasks known to recruit nearly identical brain areas, the color-word Stroop task and the Multi-Source Interference Task (MSIT). Using a measure of instantaneous functional correlations, based on edge time series, we estimated the task-related networks that differed between incongruent and congruent conditions. We found that the two tasks were much more different in their network profiles than in their evoked activity patterns at different analytical levels, as well as for a wide range of methodological pipelines. Our results reject the notion that having the same activation patterns means two tasks engage the same underlying representations, suggesting that task representations should be independently evaluated at both node and edge (connectivity) levels.

The locus of recognition memory signals in human cortex depends on the complexity of the memory representations

According to a "Swiss Army Knife" model of the brain, cognitive functions such as episodic memory and face perception map onto distinct neural substrates. In contrast, representational accounts propose that each brain region is best explained not by which specialized function it performs, but by the type of information it represents with its neural firing. In a functional magnetic resonance imaging study, we asked whether the neural signals supporting recognition memory fall mandatorily within the medial temporal lobes (MTL), traditionally thought the seat of declarative memory, or whether these signals shift within cortex according to the content of the memory. Participants studied objects and scenes that were unique conjunctions of pre-defined visual features. Next, we tested recognition memory in a task that required mnemonic discrimination of both simple features and complex conjunctions. Feature memory signals were strongest in posterior visual regions, declining with anterior progression toward the MTL, while conjunction memory signals followed the opposite pattern. Moreover, feature memory signals correlated with feature memory discrimination performance most strongly in posterior visual regions, whereas conjunction memory signals correlated with conjunction memory discrimination most strongly in anterior sites. Thus, recognition memory signals shifted with changes in memory content, in line with representational accounts.

What do you have in mind? ERP markers of visual and auditory imagery

This study aimed to investigate the psychophysiological markers of imagery processes through EEG/ERP recordings. Visual and auditory stimuli representing 10 different semantic categories were shown to 30 healthy participants. After a given interval and prompted by a light signal, participants were asked to activate a mental image corresponding to the semantic category for recording synchronized electrical potentials. Unprecedented electrophysiological markers of imagination were recorded in the absence of sensory stimulation. The following peaks were identified at specific scalp sites and latencies, during imagination of infants (centroparietal positivity, CPP, and late CPP), human faces (anterior negativity, AN), animals (anterior positivity, AP), music (P300-like), speech (N400-like), affective vocalizations (P2-like) and sensory (visual vs auditory) modality (PN300). Overall, perception and imagery conditions shared some common electro/cortical markers, but during imagery the category-dependent modulation of ERPs was long latency and more anterior, with respect to the perceptual condition. These ERP markers might be precious tools for BCI systems (pattern recognition, classification, or A.I. algorithms) applied to patients affected by consciousness disorders (e.g., in a vegetative or comatose state) or locked-in-patients (e.g., spinal or SLA patients).

The neurobiological basis of affect is consistent with psychological construction theory and shares a common neural basis across emotional categories

Affective experience colours everyday perception and cognition, yet its fundamental and neurobiological basis is poorly understood. The current debate essentially centers around the communalities and specificities across individuals, events, and emotional categories like anger, sadness, and happiness. Using fMRI during the experience of these emotions, we critically compare the two dominant conflicting theories on human affect. Basic emotion theory posits emotions as discrete universal entities generated by dedicated emotion category-specific neural circuits, while psychological construction theory claims emotional events as unique, idiosyncratic, and constructed by psychological primitives like core affect and conceptualization, which underlie each emotional event and operate in a predictive framework. Based on the findings of 8 a priori-defined model-specific prediction tests on the neural response amplitudes and patterns, we conclude that the neurobiological basis of affect is primarily characterized by idiosyncratic mechanisms and a common neural basis shared across emotion categories, consistent with psychological construction theory. The findings provide further insight into the organizational principles of the neural basis of affect and brain function in general. Future studies in clinical populations with affective symptoms may reveal the corresponding underlying neural changes from a psychological construction perspective.

Experimental evidence for involvement of monocular channels in mental rotation

According to the prevailing view, cognitive processes of mental rotation are carried out by visuospatial perceptual circuits located primarily in high cortical areas. Here, we examined the functional involvement of (mostly subcortical) monocular channels in mental rotation tasks. Images of two rotated objects (0°, 50°, 100°, or 150°; identical or mirrored) were presented either to one eye (monocular) or segregated between the eyes (interocular). The results indicated a causal role for low monocular visual channels in mental rotation: Response times for identical ("same") objects at high angular disparities (100°, 150°) were shorter when both objects were presented to a single eye than when each object was presented to a different eye. We suggest that mental rotation processes rely on cortico-subcortical loops that support visuospatial perception. More generally, the findings highlight the potential contribution of lower-level mechanisms to what are typically considered to be high-level cognitive functions, such as mental representation.

Improving Motor Imagery EEG Classification Based on Channel Selection Using a Deep Learning Architecture

Recently, motor imagery EEG signals have been widely applied in Brain–Computer Interfaces (BCI). These signals are typically observed in the first motor cortex of the brain, resulting from the imagination of body limb movements. For non-invasive BCI systems, it is not apparent how to locate the electrodes, optimizing the accuracy for a given task. This study proposes a comparative analysis of channel signals exploiting the Deep Learning (DL) technique and a public dataset to locate the most discriminant channels. EEG channels are usually selected based on the function and nomenclature of electrode location from international standards. Instead, the most suitable configuration for a given paradigm must be determined by analyzing the proper selection of the channels. Therefore, an EEGNet network was implemented to classify signals from different channel location using the accuracy metric. Achieved results were then contrasted with results from the state-of-the-art. As a result, the proposed method improved BCI classification accuracy.

Altered linear coupling between stimulus-evoked blood flow and oxygen metabolism in the aging human brain

Neural-vascular coupling (NVC) is the process by which oxygen and nutrients are delivered to metabolically active neurons by blood vessels. Murine models of NVC disruption have revealed its critical role in healthy neural function. We hypothesized that, in humans, aging exerts detrimental effects upon the integrity of the neural-glial-vascular system that underlies NVC. To test this hypothesis, calibrated functional magnetic resonance imaging (cfMRI) was used to characterize age-related changes in cerebral blood flow (CBF) and oxygen metabolism during visual cortex stimulation. Thirty-three younger and 27 older participants underwent cfMRI scanning during both an attention-controlled visual stimulation task and a hypercapnia paradigm used to calibrate the blood-oxygen-level-dependent signal. Measurement of stimulus-evoked blood flow and oxygen metabolism permitted calculation of the NVC ratio to assess the integrity of neural-vascular communication. Consistent with our hypothesis, we observed monotonic NVC ratio increases with increasing visual stimulation frequency in younger adults but not in older adults. Age-related changes in stimulus-evoked cerebrovascular and neurometabolic signal could not fully explain this disruption; increases in stimulus-evoked neurometabolic activity elicited corresponding increases in stimulus-evoked CBF in younger but not in older adults. These results implicate age-related, demand-dependent failures of the neural-glial-vascular structures that comprise the NVC system.

Relative precision of top-down attentional modulations is lower in early visual cortex compared to mid- and high-level visual areas

Top-down spatial attention enhances cortical representations of behaviorally relevant visual information and increases the precision of perceptual reports. However, little is known about the relative precision of top-down attentional modulations in different visual areas, especially compared with the highly precise stimulus-driven responses that are observed in early visual cortex. For example, the precision of attentional modulations in early visual areas may be limited by the relatively coarse spatial selectivity and the anatomical connectivity of the areas in prefrontal cortex that generate and relay the top-down signals. Here, we used functional MRI (fMRI) and human participants to assess the precision of bottom-up spatial representations evoked by high-contrast stimuli across the visual hierarchy. Then, we examined the relative precision of top-down attentional modulations in the absence of spatially specific bottom-up drive. Whereas V1 showed the largest relative difference between the precision of top-down attentional modulations and the precision of bottom-up modulations, midlevel areas such as V4 showed relatively smaller differences between the precision of top-down and bottom-up modulations. Overall, this interaction between visual areas (e.g., V1 vs. V4) and the relative precision of top-down and bottom-up modulations suggests that the precision of top-down attentional modulations is limited by the representational fidelity of areas that generate and relay top-down feedback signals.NEW & NOTEWORTHY When the relative precision of purely top-down and bottom-up signals were compared across visual areas, early visual areas like V1 showed higher bottom-up precision compared with top-down precision. In contrast, midlevel areas showed similar levels of top-down and bottom-up precision. This result suggests that the precision of top-down attentional modulations may be limited by the relatively coarse spatial selectivity and the anatomical connectivity of the areas generating and relaying the signals.

An investigation of the neural association between auditory imagery and perception of complex sounds

Neuroimaging studies have demonstrated that mental imagery and perception share similar neural substrates, however, there are still ambiguities according to different auditory imagery content. In addition, there is still a lack of information regarding the underlying neural correlation between the two modalities. In the present study, we adopted functional magnetic resonance imaging to explore the neural representation during imagery and perception of actual sounds in our surroundings. Univariate analysis was used to assess the differences between the modalities of average activation intensity, and stronger imagery activation was found in sensorimotor regions but weaker activation in auditory association cortices. Additionally, multi-voxel pattern analysis with a support vector machine classifier was implemented to decode environmental sounds within- or cross-modality. Significant above-chance accuracies were found in all overlapping regions in the classification of within-modality, while successful cross-modality classification only was found in sensorimotor regions. Both univariate and multivariate analyses found distinct representation between auditory imagery and perception in the overlapping regions, including superior temporal gyrus and inferior frontal sulcus as well as the precentral cortex and pre-supplementary motor area. Our results confirm the overlapping activation regions between auditory imagery and perception reported by previous studies and suggest that activation regions showed dissociable representation pattern in imagery and perception of sound categories.

Animal awareness

The problem of demonstrating that animals are consciously aware of their environment or themselves in a similar manner to ourselves has proved to be a relatively intractable one since we have mainly been constrained to rely upon observation of an animal’s behaviour as an index of its awareness. It can usually be argued that even complex behaviours need not necessarily involve an animal being aware of either environmental cues or its response to them. Indeed, simple organisms or computers can perform complex responses to environmental cues when they are clearly not aware of them in the sense that we are ourselves. What can particularly distinguish conscious awareness from simple stimulus-response guided behaviour is the ability of an animal to experience some form of emotional response to an environmental cue or to its own actions. In this review I will summarize some of our experiments on sheep which show that the way their brains process visual and olfactory social and non-social cues from the environment is integrally related to and organized by the emotional significance of what is perceived. I will also argue that the way their brains process complex social stimuli from the environment in order to recognize individuals is very similar to ourselves although these processes are very strongly influenced by changes in their motivational state. From these observations I will maintain that sheep are almost certainly capable of conscious awareness of their environment although perhaps this awareness is mainly restricted to present events and needs with a limited capacity to reflect on the past or future.

No Evidence of Narrowly Defined Cognitive Penetrability in Unambiguous Vision

The classical notion of cognitive impenetrability suggests that perceptual processing is an automatic modular system and not under conscious control. Near consensus is now emerging that this classical notion is untenable. However, as recently pointed out by Firestone and Scholl, this consensus is built on quicksand. In most studies claiming perception is cognitively penetrable, it remains unclear which actual process has been affected (perception, memory, imagery, input selection or judgment). In fact, the only available "proofs" for cognitive penetrability are proxies for perception, such as behavioral responses and neural correlates. We suggest that one can interpret cognitive penetrability in two different ways, a broad sense and a narrow sense. In the broad sense, attention and memory are not considered as "just" pre- and post-perceptual systems but as part of the mechanisms by which top-down processes influence the actual percept. Although many studies have proven top-down influences in this broader sense, it is still debatable whether cognitive penetrability remains tenable in a narrow sense. The narrow sense states that cognitive penetrability only occurs when top-down factors are flexible and cause a clear illusion from a first person perspective. So far, there is no strong evidence from a first person perspective that visual illusions can indeed be driven by high-level flexible factors. One cannot be cognitively trained to see and unsee visual illusions. We argue that this lack of convincing proof for cognitive penetrability in the narrow sense can be explained by the fact that most research focuses on foveal vision only. This type of perception may be too unambiguous for transient high-level factors to control perception. Therefore, illusions in more ambiguous perception, such as peripheral vision, can offer a unique insight into the matter. They produce a clear subjective percept based on unclear, degraded visual input: the optimal basis to study narrowly defined cognitive penetrability.

A functional magnetic resonance imaging investigation of visual hallucinations in the human striate cortex

Purpose

Human beings frequently experience fear, phobia, migraine and hallucinations, however, the cerebral mechanisms underpinning these conditions remain poorly understood. Towards this goal, in this work, we aim to correlate the human ocular perceptions with visual hallucinations, and map them to their cerebral origins.

Methods

An fMRI study was performed to examine the visual cortical areas including the striate, parastriate and peristriate cortex in the occipital lobe of the human brain. 24 healthy subjects were enrolled and four visual patterns including hallucination circle (HCC), hallucination fan (HCF), retinotopy circle (RTC) and retinotopy cross (RTX) were used towards registering their impact in the aforementioned visual related areas. One-way analysis of variance was used to evaluate the significance of difference between induced activations. Multinomial regression and and K-means were used to cluster activation patterns in visual areas of the brain.

Results

Significant activations were observed in the visual cortex as a result of stimulus presentation. The responses induced by visual stimuli were resolved to Brodmann areas 17, 18 and 19. Activation data clustered into independent and mutually exclusive clusters with HCC registering higher activations as compared to HCF, RTC and RTX.

Conclusions

We conclude that small circular objects, in rotation, tend to leave greater hallucinating impressions in the visual region. The similarity between observed activation patterns and those reported in conditions such as epilepsy and visual hallucinations can help elucidate the cortical mechanisms underlying these conditions.

Trial Registration 1121_GWJUNG

Higher Intelligence Is Associated with Less Task-Related Brain Network Reconfiguration

The human brain is able to exceed modern computers on multiple computational demands (e.g., language, planning) using a small fraction of the energy. The mystery of how the brain can be so efficient is compounded by recent evidence that all brain regions are constantly active as they interact in so-called resting-state networks (RSNs). To investigate the brain's ability to process complex cognitive demands efficiently, we compared functional connectivity (FC) during rest and multiple highly distinct tasks. We found previously that RSNs are present during a wide variety of tasks and that tasks only minimally modify FC patterns throughout the brain. Here, we tested the hypothesis that, although subtle, these task-evoked FC updates from rest nonetheless contribute strongly to behavioral performance. One might expect that larger changes in FC reflect optimization of networks for the task at hand, improving behavioral performance. Alternatively, smaller changes in FC could reflect optimization for efficient (i.e., small) network updates, reducing processing demands to improve behavioral performance. We found across three task domains that high-performing individuals exhibited more efficient brain connectivity updates in the form of smaller changes in functional network architecture between rest and task. These smaller changes suggest that individuals with an optimized intrinsic network configuration for domain-general task performance experience more efficient network updates generally. Confirming this, network update efficiency correlated with general intelligence. The brain's reconfiguration efficiency therefore appears to be a key feature contributing to both its network dynamics and general cognitive ability.

SIGNIFICANCE STATEMENT

The brain's network configuration varies based on current task demands. For example, functional brain connections are organized in one way when one is resting quietly but in another way if one is asked to make a decision. We found that the efficiency of these updates in brain network organization is positively related to general intelligence, the ability to perform a wide variety of cognitively challenging tasks well. Specifically, we found that brain network configuration at rest was already closer to a wide variety of task configurations in intelligent individuals. This suggests that the ability to modify network connectivity efficiently when task demands change is a hallmark of high intelligence.

Review : Functional Neuroimaging of Memory

Cognitive neuroscientists have acquired powerful new tools for studying the functional neuroanatomy of the human brain. Traditional methods such as lesion analysis have been supplemented with electrical and mag netic field recording techniques that can measure the informational transactions of the brain in the scale of milliseconds and neuroimaging techniques that can provide structural details of the brain to a fraction of a millimeter. The most powerful new methods have been functional imaging techniques in which brain activity engendered by a sensory, motor, or cognitive task causes an increase in local blood flow and metabolism that can be imaged with high resolution. This article reviews recent progress in functional neuroimaging, with special emphasis on understanding the neural substrates of memory. The Neuroscientist 1:155-163, 1995

Review : Mapping Brain Pathophysiology and Higher Cortical Function with Magnetic Resonance Imaging

Advances in magnetic resonance imaging (MRI) have moved the technology beyond its application solely as a diagnostic test to become a tool for addressing questions of in vivo pathophysiology and higher cortical function in humans. Diffusion-weighted MRI measures the apparent rate of translational movement of water molecules through brain parenchyma. This measurement can be used to determine axonal orientation within white matter, to define regions of tissue edema, and to permit early identification of ischemic neuronal injury related to impairment of Na+-K +-ATPase activity in experimental and human stroke. Changes in various aspects of cerebral perfusion—blood volume, blood flow, and hemoglobin oxygen saturation—can be mea sured with MRI, and altered cerebrovascular circulation and regional brain activation can thereby be inves tigated. Echo planar imaging is a method of ultrafast data acquisition with MRI—individual images are ac quired on the order of 100 msec. Echo planar imaging makes diffusion and perfusion measurements more practicable for diverse applications and allows for the study of temporal characteristics of regional brain responses to stimuli. Diffusion and perfusion MRI, generally termed functional MRI, are tools for studying in vivo brain physiology with MRI and are being applied to a broad range of questions in neuroscience. The Neuroscientist 1:221-235, 1995

Mental Imagery: Functional Mechanisms and Clinical Applications

Mental imagery research has weathered both disbelief of the phenomenon and inherent methodological limitations. Here we review recent behavioral, brain imaging, and clinical research that has reshaped our understanding of mental imagery. Research supports the claim that visual mental imagery is a depictive internal representation that functions like a weak form of perception. Brain imaging work has demonstrated that neural representations of mental and perceptual images resemble one another as early as the primary visual cortex (V1). Activity patterns in V1 encode mental images and perceptual images via a common set of low-level depictive visual features. Recent translational and clinical research reveals the pivotal role that imagery plays in many mental disorders and suggests how clinicians can utilize imagery in treatment.

Neural correlates of spatial working memory manipulation in a sequential Vernier discrimination task

Visuospatial working memory refers to the short-term storage and manipulation of visuospatial information. To study the neural bases of these processes, 17 participants took part in a modified sequential Vernier task while they were being scanned using an event-related functional MRI protocol. During each trial, participants retained the spatial position of a line during a delay period to later evaluate if it was presented aligned to a second line. This design allowed testing the manipulation of the spatial information from memory. During encoding, there was a larger parietal and cingulate activation under the experimental condition, whereas the opposite was true for the occipital cortex. Throughout the delay period of the experimental condition there was significant bilateral activation in the caudal superior frontal sulcus/middle frontal gyrus, as well as the insular and superior parietal lobes, which confirms the findings from previous studies. During manipulation of spatial memory, the analysis showed higher activation in the lingual gyrus. This increase of activity in visual areas during the manipulation phase fits with the hypothesis that information stored in sensory cortices becomes reactivated once the information is needed to be utilized.

Cognition does not affect perception: Evaluating the evidence for “top-down” effects

What determines what we see? In contrast to the traditional “modular” understanding of perception, according to which visual processing is encapsulated from higher-level cognition, a tidal wave of recent research alleges that states such as beliefs, desires, emotions, motivations, intentions, and linguistic representations exert direct, top-down influences on what we see. There is a growing consensus that such effects are ubiquitous, and that the distinction between perception and cognition may itself be unsustainable. We argue otherwise: None of these hundreds of studies – either individually or collectively – provides compelling evidence for true top-down effects on perception, or “cognitive penetrability.” In particular, and despite their variety, we suggest that these studies all fall prey to only a handful of pitfalls. And whereas abstract theoretical challenges have failed to resolve this debate in the past, our presentation of these pitfalls is empirically anchored: In each case, we show not only how certain studies could be susceptible to the pitfall (in principle), but also how several alleged top-down effects actually are explained by the pitfall (in practice). Moreover, these pitfalls are perfectly general, with each applying to dozens of other top-down effects. We conclude by extracting the lessons provided by these pitfalls into a checklist that future work could use to convincingly demonstrate top-down effects on visual perception. The discovery of substantive top-down effects of cognition on perception would revolutionize our understanding of how the mind is organized; but without addressing these pitfalls, no such empirical report will license such exciting conclusions.

A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes

Recent multi-voxel pattern classification (MVPC) studies have shown that in early visual cortex patterns of brain activity generated during mental imagery are similar to patterns of activity generated during perception. This finding implies that low-level visual features (e.g., space, spatial frequency, and orientation) are encoded during mental imagery. However, the specific hypothesis that low-level visual features are encoded during mental imagery is difficult to directly test using MVPC. The difficulty is especially acute when considering the representation of complex, multi-object scenes that can evoke multiple sources of variation that are distinct from low-level visual features. Therefore, we used a voxel-wise modeling and decoding approach to directly test the hypothesis that low-level visual features are encoded in activity generated during mental imagery of complex scenes. Using fMRI measurements of cortical activity evoked by viewing photographs, we constructed voxel-wise encoding models of tuning to low-level visual features. We also measured activity as subjects imagined previously memorized works of art. We then used the encoding models to determine if putative low-level visual features encoded in this activity could pick out the imagined artwork from among thousands of other randomly selected images. We show that mental images can be accurately identified in this way; moreover, mental image identification accuracy depends upon the degree of tuning to low-level visual features in the voxels selected for decoding. These results directly confirm the hypothesis that low-level visual features are encoded during mental imagery of complex scenes. Our work also points to novel forms of brain-machine interaction: we provide a proof-of-concept demonstration of an internet image search guided by mental imagery.

Individual differences in solving arithmetic word problems

BACKGROUND

With the present functional magnetic resonance imaging (fMRI) study at 3 T, we investigated the neural correlates of visualization and verbalization during arithmetic word problem solving. In the domain of arithmetic, visualization might mean to visualize numbers and (intermediate) results while calculating, and verbalization might mean that numbers and (intermediate) results are verbally repeated during calculation. If the brain areas involved in number processing are domain-specific as assumed, that is, that the left angular gyrus (AG) shows an affinity to the verbal domain, and that the left and right intraparietal sulcus (IPS) shows an affinity to the visual domain, the activation of these areas should show a dependency on an individual's cognitive style.

METHODS

36 healthy young adults participated in the fMRI study. The participants habitual use of visualization and verbalization during solving arithmetic word problems was assessed with a short self-report assessment. During the fMRI measurement, arithmetic word problems that had to be solved by the participants were presented in an event-related design.

RESULTS

We found that visualizers showed greater brain activation in brain areas involved in visual processing, and that verbalizers showed greater brain activation within the left angular gyrus.

CONCLUSIONS

Our results indicate that cognitive styles or preferences play an important role in understanding brain activation. Our results confirm, that strong visualizers use mental imagery more strongly than weak visualizers during calculation. Moreover, our results suggest that the left AG shows a specific affinity to the verbal domain and subserves number processing in a modality-specific way.

A review of cell assemblies

Since the cell assembly (CA) was hypothesised, it has gained substantial support and is believed to be the neural basis of psychological concepts. A CA is a relatively small set of connected neurons, that through neural firing can sustain activation without stimulus from outside the CA, and is formed by learning. Extensive evidence from multiple single unit recording and other techniques provides support for the existence of CAs that have these properties, and that their neurons also spike with some degree of synchrony. Since the evidence is so broad and deep, the review concludes that CAs are all but certain. A model of CAs is introduced that is informal, but is broad enough to include, e.g. synfire chains, without including, e.g. holographic reduced representation. CAs are found in most cortical areas and in some sub-cortical areas, they are involved in psychological tasks including categorisation, short-term memory and long-term memory, and are central to other tasks including working memory. There is currently insufficient evidence to conclude that CAs are the neural basis of all concepts. A range of models have been used to simulate CA behaviour including associative memory and more process- oriented tasks such as natural language parsing. Questions involving CAs, e.g. memory persistence, CAs' complex interactions with brain waves and learning, remain unanswered. CA research involves a wide range of disciplines including biology and psychology, and this paper reviews literature directly related to the CA, providing a basis of discussion for this interdisciplinary community on this important topic. Hopefully, this discussion will lead to more formal and accurate models of CAs that are better linked to neuropsychological data.

A Cross-Modal Perspective on the Relationships between Imagery and Working Memory

Mapping the distinctions and interrelationships between imagery and working memory (WM) remains challenging. Although each of these major cognitive constructs is defined and treated in various ways across studies, most accept that both imagery and WM involve a form of internal representation available to our awareness. In WM, there is a further emphasis on goal-oriented, active maintenance, and use of this conscious representation to guide voluntary action. Multicomponent WM models incorporate representational buffers, such as the visuo-spatial sketchpad, plus central executive functions. If there is a visuo-spatial "sketchpad" for WM, does imagery involve the same representational buffer? Alternatively, does WM employ an imagery-specific representational mechanism to occupy our awareness? Or do both constructs utilize a more generic "projection screen" of an amodal nature? To address these issues, in a cross-modal fMRI study, I introduce a novel Drawing-Based Memory Paradigm, and conceptualize drawing as a complex behavior that is readily adaptable from the visual to non-visual modalities (such as the tactile modality), which opens intriguing possibilities for investigating cross-modal learning and plasticity. Blindfolded participants were trained through our Cognitive-Kinesthetic Method (Likova, 2010a, 2012) to draw complex objects guided purely by the memory of felt tactile images. If this WM task had been mediated by transfer of the felt spatial configuration to the visual imagery mechanism, the response-profile in visual cortex would be predicted to have the "top-down" signature of propagation of the imagery signal downward through the visual hierarchy. Remarkably, the pattern of cross-modal occipital activation generated by the non-visual memory drawing was essentially the inverse of this typical imagery signature. The sole visual hierarchy activation was isolated to the primary visual area (V1), and accompanied by deactivation of the entire extrastriate cortex, thus 'cutting-off' any signal propagation from/to V1 through the visual hierarchy. The implications of these findings for the debate on the interrelationships between the core cognitive constructs of WM and imagery and the nature of internal representations are evaluated.

The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge

Concepts are the elementary units of reason and linguistic meaning. They are conventional and relatively stable. As such, they must somehow be the result of neural activity in the brain. The questions are: Where? and How? A common philosophical position is that all concepts-even concepts about action and perception-are symbolic and abstract, and therefore must be implemented outside the brain's sensory-motor system. We will argue against this position using (1) neuroscientific evidence; (2) results from neural computation; and (3) results about the nature of concepts from cognitive linguistics. We will propose that the sensory-motor system has the right kind of structure to characterise both sensory-motor and more abstract concepts. Central to this picture are the neural theory of language and the theory of cogs, according to which, brain structures in the sensory-motor regions are exploited to characterise the so-called "abstract" concepts that constitute the meanings of grammatical constructions and general inference patterns.

Mental images and the Brain

One theory of visual mental imagery posits that early visual cortex is also used to support representations during imagery. This claim is important because it bears on the "imagery debate": Early visual cortex supports depictive representations during perception, not descriptive ones. Thus, if such cortex also plays a functional role in imagery, this is strong evidence that imagery does not rely exclusively on the same sorts of representations that underlie language. The present article first outlines the nature of a processing system in which such a dual use of early visual cortex (in perception and in imagery) makes sense. Following this, literature bearing on the claim that early visual cortex is used in visual mental imagery is reviewed, and key issues are discussed.

Spontaneously reactivated patterns in frontal and temporal lobe predict semantic clustering during memory search

Although it is well established that remembering an item will bring to mind memories of other semantically related items (Bousfield, 1953), the neural basis of this phenomenon is poorly understood. We studied how the similarity relations among items influence their retrieval by analyzing electrocorticographic recordings taken as 46 human neurosurgical patients studied and freely recalled lists of words. We first identified semantic components of neural activity that varied systematically with the meanings of each studied word, as defined by latent semantic analysis (Landauer and Dumais, 1997). We then examined the dynamics of these semantic components as participants attempted to recall the previously studied words. Our analyses revealed that the semantic components of neural activity were spontaneously reactivated during memory search, just before recall of the studied words. Further, the degree to which neural activity correlated with semantic similarity during recall predicted participants' tendencies to organize the sequences of their responses on the basis of semantic similarity. Thus, our work shows that differences in the neural correlates of semantic information, and how they are reactivated before recall, reveal how individuals organize and retrieve memories of words.

A linear/nonlinear characterization of resting state brain networks in FMRI time series

Resting state functional connectivity studies in fMRI have been used to demonstrate that the human brain is organized into inherent functional networks in the absence of stimuli. The basis for this activity is based on the spontaneous fluctuations observed during rest. In the present study, the time series generated from these fluctuations were characterized as either being linear or nonlinear based on the Delay Vector Variance method, applied through an examination of the local predictability of the signal. It was found that the default mode resting state network is composed of relatively more linear signals compared to the visual, task positive visuospatial, motor, and auditory resting state network time series. Also, it was shown that the visual cortex resting state network is more nonlinear relative to these aforementioned networks. Furthermore, using a histogram map of the nonlinearly characterized voxels for all the subjects, the histogram map was able to retrieve the peak intensity in four out of six resting state networks. Thus, the findings may provide the basis for a novel way to explore spontaneous fluctuations in the resting state brain.

Hemispheric Differences within the Fronto-Parietal Network Dynamics Underlying Spatial Imagery

Spatial imagery refers to the inspection and evaluation of spatial features (e.g., distance, relative position, configuration) and/or the spatial manipulation (e.g., rotation, shifting, reorienting) of mentally generated visual images. In the past few decades, psychophysical as well as functional brain imaging studies have indicated that any such processing of spatially coded information and/or manipulation based on mental images (i) is subject to similar behavioral demands and limitations as in the case of spatial processing based on real visual images, and (ii) consistently activates several nodes of widely distributed cortical networks in the brain. These nodes include areas within both, the dorsal fronto-parietal as well as ventral occipito-temporal visual processing pathway, representing the “what” versus “where” aspects of spatial imagery. We here describe evidence from functional brain imaging and brain interference studies indicating systematic hemispheric differences within the dorsal fronto-parietal networks during the execution of spatial imagery. Importantly, such hemispheric differences and functional lateralization principles are also found in the effective brain network connectivity within and across these networks, with a direction of information flow from anterior frontal/premotor regions to posterior parietal cortices. In an attempt to integrate these findings of hemispheric lateralization and fronto-to-parietal interactions, we argue that spatial imagery constitutes a multifaceted cognitive construct that can be segregated in several distinct mental sub processes, each associated with activity within specific lateralized fronto-parietal (sub) networks, forming the basis of the here proposed dynamic network model of spatial imagery.

On the recall of vestibular sensations

Functional neuroimaging studies on the recall or imagination of a distinctive task in the motor network or of sensations in sensory systems (visual, acoustic, nociceptive, gustatory, and olfactory) demonstrated that the respective primary cortex is often involved in the mental imagery process. Our aim was to examine this phenomenon in the vestibular system using fMRI. Sixteen healthy subjects were asked to remember the feeling of a rotatory chair procedure in contrast to an identical situation at rest. Shortly afterwards they were asked to recall the vestibular experience in a 1.5-T scanner. The resulting activations were then compared with the responses of a galvanic vestibular control experiment and a rest condition. The vestibular recall showed significant bihemispheric activations in the inferior frontal gyri, the anterior operculum, the middle cingulate, the putamen, the globus pallidus, the premotor motor cortex, and the anterior insula. We found activations in regions known to play a role in spatial referencing, motor programs, and attention in the recall of vestibular sensations. But important known relay stations for the cortical processing of vestibular information showed neither relevant activations nor deactivations.

SPM: a history

Karl Friston began the SPM project around 1991. The rest is history.

Orientation-specific aftereffects to mentally generated lines

After staring at a pattern of tilted lines, subsequent lines appear to be tilted in the opposite direction (direct tilt aftereffect, TAE). In a previous fMRI study we have demonstrated a direct TAE solely induced by the mental imagination accompanied by adaptation of orientation-selective neurons located in the extrastriate cortex, supporting the assumption of a perception-like coding of mental images. In this study we enlarge and specify the evidence for a perception-like coding of orientation-imagination. First, we replicated the previously detected direct TAE induced by line imagination with altered design-variations to control possible perceptual task confounds. Second, we tried to induce two other orientation-specific aftereffects: indirect TAE and contrast-threshold elevation aftereffect by mental imagery. The results replicate a robust direct TAE by mental imagery and by visual stimulation, with no influence of attentional resource allocation or perceptual task confounds. We could not induce an indirect TAE, but observed a perception bias in the opposite direction of the indirect TAE. The mental imagery of lines induced no orientation-selective contrast-threshold elevation aftereffect. In general, mental imagery seems to influence visual perception, indicating that perceptual resources are used by mental imagery. However, the utilisation of visual resources seems to be somewhat different from utilisation by perception.

The NIH experience in first advancing fMRI

The introduction of functional MRI at NIH in 1992 was the outcome of research goals first formulated by Turner in 1983. Between 1988 and 1990, Turner worked at NIH on actively-shielded gradient coils and the implementation of EPI-based techniques, especially diffusion-weighted EPI. His work on hypoxia in cat brain in 1990 directly inspired Ken Kwong's demonstration of BOLD contrast in humans at MGH in May 1991. Turner collaborated actively with this MGH team, the first group to map entirely noninvasively human brain activity due to visual stimulation. He introduced BOLD fMRI at NIH in February 1992. This paper reviews the steps that led up to BOLD EPI, and Turner's initial applications of BOLD fMRI at NIH.

The current status of the simulation theory of cognition

It is proposed that thinking is simulated interaction with the environment. Three assumptions underlie this 'simulation' theory of cognitive function. Firstly, behaviour can be simulated in the sense that we can activate motor structures, as during a normal overt action, but suppress its execution. Secondly, perception can be simulated by internal activation of sensory cortex in a way that resembles its normal activation during perception of external stimuli. The third assumption ('anticipation') is that both overt and simulated actions can elicit perceptual simulation of their most probable consequences. A large body of evidence, mainly from neuroimaging studies, that supports these assumptions, is reviewed briefly. The theory is ontologically parsimonious and does not rely on standard cognitivist constructs such as internal models or representations. It is argued that the simulation approach can explain the relations between motor, sensory and cognitive functions and the appearance of an inner world. It also unifies and explains important features of a wide variety of cognitive phenomena such as memory and cognitive maps. Novel findings from recent developments in memory research on the similarity of imaging and memory and on the role of both prefrontal cortex and sensory cortex in declarative memory and working memory are predicted by the theory and provide striking support for it. This article is part of a Special Issue entitled "The Cognitive Neuroscience".

Seeing touch is correlated with content-specific activity in primary somatosensory cortex

There is increasing evidence to suggest that primary sensory cortices can become active in the absence of external stimulation in their respective modalities. This occurs, for example, when stimuli processed via one sensory modality imply features characteristic of a different modality; for instance, visual stimuli that imply touch have been observed to activate the primary somatosensory cortex (SI). In the present study, we addressed the question of whether such cross-modal activations are content specific. To this end, we investigated neural activity in the primary somatosensory cortex of subjects who observed human hands engaged in the haptic exploration of different everyday objects. Using multivariate pattern analysis of functional magnetic resonance imaging data, we were able to predict, based exclusively on the activity pattern in SI, which of several objects a subject saw being explored. Along with previous studies that found similar evidence for other modalities, our results suggest that primary sensory cortices represent information relevant for their modality even when this information enters the brain via a different sensory system.

Seeing with the mind's eye: top-down, bottom-up, and conscious awareness

With the advent of functional brain imaging techniques and recent developments in the analysis of cortical connectivity, the focus of mental imagery studies has shifted from a semi-modular approach to an integrated cortical network perspective. Functional magnetic resonance imaging studies of visual imagery of faces and objects show that activation of content-specific representations stored in the ventral visual stream is top-down-modulated by parietal and frontal regions. Recent findings in patients with conscious awareness disorders reveal that mental imagery can be used to map patterns of residual cognitive function in their brain and to provide diagnostic and prognostic indicators.

Role of layer 6 of V2 visual cortex in object-recognition memory

Cellular responses in the V2 secondary visual cortex to simple as well as complex visual stimuli have been well studied. However, the role of area V2 in visual memory remains unexplored. We found that layer 6 neurons of V2 are crucial for the processing of object-recognition memory (ORM). Using the protein regulator of G protein signaling-14 (RGS-14) as a tool, we found that the expression of this protein into layer 6 neurons of rat-brain area V2 promoted the conversion of a normal short-term ORM that normally lasts for 45 minutes into long-term memory detectable even after many months. Furthermore, elimination of the same-layer neurons by means of injection of a selective cytotoxin resulted in the complete loss of normal as well as protein-mediated enhanced ORM.

The role of early visual cortex in visual short-term memory and visual attention

We measured cortical activity with functional magnetic resonance imaging to probe the involvement of early visual cortex in visual short-term memory and visual attention. In four experimental tasks, human subjects viewed two visual stimuli separated by a variable delay period. The tasks placed differential demands on short-term memory and attention, but the stimuli were visually identical until after the delay period. Early visual cortex exhibited sustained responses throughout the delay when subjects performed attention-demanding tasks, but delay-period activity was not distinguishable from zero when subjects performed a task that required short-term memory. This dissociation reveals different computational mechanisms underlying the two processes.