Representing spatial relationships in posterior parietal cortex: single neurons code object-referenced position

Position- and scale-invariant object-centered spatial localization in monkey frontoparietal cortex dynamically adapts to cognitive demand

Egocentric encoding is a well-known property of brain areas along the dorsal pathway. Different to previous experiments, which typically only demanded egocentric spatial processing during movement preparation, we designed a task where two male rhesus monkeys memorized an on-the-object target position and then planned a reach to this position after the object re-occurred at variable location with potentially different size. We found allocentric (in addition to egocentric) encoding in the dorsal stream reach planning areas, parietal reach region and dorsal premotor cortex, which is invariant with respect to the position, and, remarkably, also the size of the object. The dynamic adjustment from predominantly allocentric encoding during visual memory to predominantly egocentric during reach planning in the same brain areas and often the same neurons, suggests that the prevailing frame of reference is less a question of brain area or processing stream, but more of the cognitive demands.

Cortico-spinal modularity in the parieto-frontal system: A new perspective on action control

Classical neurophysiology suggests that the motor cortex (MI) has a unique role in action control. In contrast, this review presents evidence for multiple parieto-frontal spinal command modules that can bypass MI. Five observations support this modular perspective: (i) the statistics of cortical connectivity demonstrate functionally-related clusters of cortical areas, defining functional modules in the premotor, cingulate, and parietal cortices; (ii) different corticospinal pathways originate from the above areas, each with a distinct range of conduction velocities; (iii) the activation time of each module varies depending on task, and different modules can be activated simultaneously; (iv) a modular architecture with direct motor output is faster and less metabolically expensive than an architecture that relies on MI, given the slow connections between MI and other cortical areas; (v) lesions of the areas composing parieto-frontal modules have different effects from lesions of MI. Here we provide examples of six cortico-spinal modules and functions they subserve: module 1) arm reaching, tool use and object construction; module 2) spatial navigation and locomotion; module 3) grasping and observation of hand and mouth actions; module 4) action initiation, motor sequences, time encoding; module 5) conditional motor association and learning, action plan switching and action inhibition; module 6) planning defensive actions. These modules can serve as a library of tools to be recombined when faced with novel tasks, and MI might serve as a recombinatory hub. In conclusion, the availability of locally-stored information and multiple outflow paths supports the physiological plausibility of the proposed modular perspective.

Generative replay underlies compositional inference in the hippocampal-prefrontal circuit

Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.

Shared Neural Activity But Distinct Neural Dynamics for Cognitive Control in Monkey Prefrontal and Parietal Cortex

To better understand how prefrontal networks mediate forms of cognitive control disrupted in schizophrenia, we translated a variant of the AX continuous performance task that measures specific deficits in the human disease to 2 male monkeys and recorded neurons in PFC and parietal cortex during task performance. In the task, contextual information instructed by cue stimuli determines the response required to a subsequent probe stimulus. We found parietal neurons encoding the behavioral context instructed by cues that exhibited nearly identical activity to their prefrontal counterparts (Blackman et al., 2016). This neural population switched their preference for stimuli over the course of the trial depending on whether the stimuli signaled the need to engage cognitive control to override a prepotent response. Cues evoked visual responses that appeared in parietal neurons first, whereas population activity encoding contextual information instructed by cues was stronger and more persistent in PFC. Increasing cognitive control demand biased the representation of contextual information toward the PFC and augmented the temporal correlation of task-defined information encoded by neurons in the two areas. Oscillatory dynamics in local field potentials differed between cortical areas and carried as much information about task conditions as spike rates. We found that, at the single-neuron level, patterns of activity evoked by the task were nearly identical between the two cortical areas. Nonetheless, distinct population dynamics in PFC and parietal cortex were evident. suggesting differential contributions to cognitive control.SIGNIFICANCE STATEMENT We recorded neural activity in PFC and parietal cortex of monkeys performing a task that measures cognitive control deficits in schizophrenia. This allowed us to characterize computations performed by neurons in the two areas to support forms of cognitive control disrupted in the disease. Subpopulations of neurons in the two areas exhibited parallel modulations in firing rate; and as a result, all patterns of task-evoked activity were distributed between PFC and parietal cortex. This included the presence in both cortical areas of neurons reflecting proactive and reactive cognitive control dissociated from stimuli or responses in the task. However, differences in the timing, strength, synchrony, and correlation of information encoded by neural activity were evident, indicating differential contributions to cognitive control.

The parietal lobe evolution and the emergence of material culture in the human genus

Traditional and new disciplines converge in suggesting that the parietal lobe underwent a considerable expansion during human evolution. Through the study of endocasts and shape analysis, paleoneurology has shown an increased globularity of the braincase and bulging of the parietal region in modern humans, as compared to other human species, including Neandertals. Cortical complexity increased in both the superior and inferior parietal lobules. Emerging fields bridging archaeology and neuroscience supply further evidence of the involvement of the parietal cortex in human-specific behaviors related to visuospatial capacity, technological integration, self-awareness, numerosity, mathematical reasoning and language. Here, we complement these inferences on the parietal lobe evolution, with results from more classical neuroscience disciplines, such as behavioral neurophysiology, functional neuroimaging, and brain lesions; and apply these to define the neural substrates and the role of the parietal lobes in the emergence of functions at the core of material culture, such as tool-making, tool use and constructional abilities.

Posterior parietal cortex predicts upcoming movement in dynamic sensorimotor control

SignificanceMost studies in sensorimotor neurophysiology have utilized reactive movements to stationary goals pre-defined by sensory cues, but this approach is fundamentally incapable of determining whether the observed neural activity reflects current sensory stimuli or predicts future movements. In the present study, we recorded single-neuron activity from behaving monkeys engaged in a dynamic, flexible, stimulus-response contingency task that enabled us to distinguish activity co-varying with sensory inflow from that co-varying with motor outflow in the posterior parietal cortex.

Functional and Structural Architectures of Allocentric and Egocentric Spatial Coding in Aging: A Combined DTI and fMRI Study

Background

Aging disrupts the optimal balance between neural nodes underlying orienting and attention control functions. Previous studies have suggested that age-related changes in cognitive process are associated to the changes in the myelinated fiber bundles, which affected the speed and actions of the signal propagation across different neural networks. However, whether the age-related difference in allocentric and egocentric spatial coding is accounted by the difference in white-matter integrity is unclear. In this study, using diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI), we sought to elucidate whether age-related differences in white matter integrity accounts for the difference in nodes to the distributed spatial coding-relevant brain networks.

Material and Method

Older (n = 24) and younger (n = 27) participants completed the structural DTI and fMRI scans during which they engaged in a cue-to-target task to elicit allocentric or egocentric processes.

Results and Conclusion

Efficient modulation of both allocentric and egocentric spatial coding in fronto-parietal attention network (FPAN) requires structure–function interaction. Allocentric task-modulated connectivity of the fronto-parietal network (FPN) and dorsal attention network (DAN) with the temporal lobe was influenced by the aging differences of the white-matter tracts of the posterior and superior corona radiata (PCR and SCR), respectively. On the other hand, aging difference of the superior longitudinal fasciculus mainly influenced the egocentric-task-modulated connections of the DAN and FPN with frontal regions and posterior cingulate cortex. This study suggested that functional connections of the FPAN with near and far task-relevant nodes vary significantly with age and conditions.

Psychological and physiological evidence for an initial 'Rough Sketch' calculation of personal space

Personal space has been defined as “the area individuals maintain around themselves into which others cannot intrude without arousing discomfort”. However, the precise relationship between discomfort (or arousal) responses as a function of distance from an observer remains incompletely understood. Also the mechanisms involved in recognizing conspecifics and distinguishing them from other objects within personal space have not been identified. Accordingly, here we measured personal space preferences in response to real humans and human-like avatars (in virtual reality), using well-validated “stop distance” procedures. Based on threshold measurements of personal space, we examined within-subject variations in discomfort-related responses across multiple distances (spanning inside and outside each individual’s personal space boundary), as reflected by psychological (ratings) and physiological (skin conductance) responses to both humans and avatars. We found that the discomfort-by-distance functions for both humans and avatars were closely fit by a power law. These results suggest that the brain computation of visually-defined personal space begins with a ‘rough sketch’ stage, which generates responses to a broad range of human-like stimuli, in addition to humans. Analogous processing mechanisms may underlie other brain functions which respond similarly to both real and simulated human body parts.

Category learning in a transitive inference paradigm

The implied order of a ranked set of visual images can be learned without reliance on information that explicitly signals their order. Such learning is difficult to explain by associative mechanisms, but can be accounted for by cognitive representations and processes such as transitive inference. Our study sought to determine if those processes also apply to learning categories of images. We asked whether participants can (a) infer that stimulus images belonged to familiar categories, even when the images for each trial were unique, and (b) sort those categories into an ordering that obeys transitivity. Participants received minimal verbal instruction and a single session of training. Despite this, they learned the implied order of lists of fixed stimuli and lists of ordered categories, using trial-unique exemplars. We trained two groups, one for which stimuli were constant throughout training and testing (n = 60), and one for which exemplars of each category were trial-unique (n = 50). Our findings suggest that differing cognitive processes may underpin serial learning when learning about specific stimuli as opposed to stimulus categories.

Common and distinct neural trends of allocentric and egocentric spatial coding: An ALE meta-analysis

The uniqueness of neural processes between allocentric and egocentric spatial coding has been controversial. The distinctive paradigms used in previous studies for manipulating spatial coding could have attributed for the inconsistent results. This study was aimed to generate converging evidence from previous functional brain imaging experiments for collating neural substrates associated with these two types of spatial coding. An additional aim was to test whether test-taking processes would have influenced the results. We obtained coordinate-based functional neuroimaging data for 447 subjects and performed activation likelihood estimation (ALE) meta-analysis. Among the 28 experiments, the results indicate two common clusters of convergence. They were the right precuneus and the right superior frontal gyrus as parts of the parieto-frontal circuit. Between-type differences were in the parieto-occipital circuit, with allocentric showing convergence in the superior occipital gyrus (SOG) cluster compared with egocentric showing convergence in the middle occipital gyrus (MOG) cluster. Task-specific influences were only found in allocentric spatial coding. Spatial judgment-oriented tasks seem to increase the demands on manipulating spatial relationships among the visual objects, while spatial navigation tasks seem to increase the demands on maintaining object representations. Our findings address the theoretical controversies on spatial coding that both the allocentric and egocentric types are common in their processes mediated by the parieto-frontal network, while unique and additional processes in the allocentric type are mediated by the parieto-occipital network. The positive results on possible task-specific confound offer insights into the future design of spatial tasks for eliciting spatial coding processes.

Functional specialization within the inferior parietal lobes across cognitive domains

The inferior parietal lobe (IPL) is a key neural substrate underlying diverse mental processes, from basic attention to language and social cognition, that define human interactions. Its putative domain-global role appears to tie into poorly understood differences between cognitive domains in both hemispheres. Across attentional, semantic, and social cognitive tasks, our study explored functional specialization within the IPL. The task specificity of IPL subregion activity was substantiated by distinct predictive signatures identified by multivariate pattern-learning algorithms. Moreover, the left and right IPL exerted domain-specific modulation of effective connectivity among their subregions. Task-evoked functional interactions of the anterior and posterior IPL subregions involved recruitment of distributed cortical partners. While anterior IPL subregions were engaged in strongly lateralized coupling links, both posterior subregions showed more symmetric coupling patterns across hemispheres. Our collective results shed light on how under-appreciated hemispheric specialization in the IPL supports some of the most distinctive human mental capacities.

Learned Representation of Implied Serial Order in Posterior Parietal Cortex

Monkeys can learn the implied ranking of pairs of images drawn from an ordered set, despite never seeing all of the images simultaneously and without explicit spatial or temporal cues. We recorded the activity of posterior parietal cortex (including lateral intraparietal area LIP) neurons while monkeys learned 7-item transitive inference (TI) lists with 2 items presented on each trial. Behavior and neuronal activity were significantly influenced by the ordinal relationship of the stimulus pairs, specifically symbolic distance (the difference in rank) and joint rank (the sum of the ranks). Symbolic distance strongly predicted decision accuracy and learning rate. An effect of joint rank on performance was found nested within the symbolic distance effect. Across the population of neurons, there was significant modulation of firing correlated with the relative ranks of the two stimuli presented on each trial. Neurons exhibited selectivity for stimulus rank during learning, but not before or after. The observed behavior is poorly explained by associative or reward mechanisms, and appears more consistent with a mental workspace model in which implied serial order is mapped within a spatial framework. The neural data suggest that posterior parietal cortex supports serial learning by representing information about the ordinal relationship of the stimuli presented during a given trial.

A Brief History of the Encoding of Hand Position by the Cerebral Cortex: Implications for Motor Control and Cognition

Encoding hand position by the cerebral cortex is essential not only for the neural representation of the body image but also for different actions based on eye-hand coordination. These include reaching for visual objects as well as complex movement sequences, such as tea-making, tool use, and object construction, among many others. All these functions depend on a continuous refreshing of the hand position representation, relying on both predictive signaling and afferent information. The hand position influence on neural activity in the parietofrontal system, together with eye position signals, are the basic elements of an eye-hand matrix from which all the above functions can emerge and could be regarded as key features of a network with several entry points, command nodes and outflow pathways, as confirmed by the discovery of a direct parietospinal projection for the control of hand action. The integrity of this system is crucial for daily life, as testified by the consequences of cortical lesions, spanning from severe paralysis to complex forms of apraxia. In this review, I will sketch my personal understanding of the scientific and conceptual trajectory of a line of investigation with many unexpected influences on cortical function and disease, from motor behavior to cognition.

Cerebellar-hippocampal processing in passive perception of visuospatial change: An ego- and allocentric axis?

In addition to its role in visuospatial navigation and the generation of spatial representations, in recent years, the hippocampus has been proposed to support perceptual processes. This is especially the case where high‐resolution details, in the form of fine‐grained relationships between features such as angles between components of a visual scene, are involved. An unresolved question is how, in the visual domain, perspective‐changes are differentiated from allocentric changes to these perceived feature relationships, both of which may be argued to involve the hippocampus. We conducted functional magnetic resonance imaging of the brain response (corroborated through separate event‐related potential source‐localization) in a passive visuospatial oddball‐paradigm to examine to what extent the hippocampus and other brain regions process changes in perspective, or configuration of abstract, three‐dimensional structures. We observed activation of the left superior parietal cortex during perspective shifts, and right anterior hippocampus in configuration‐changes. Strikingly, we also found the cerebellum to differentiate between the two, in a way that appeared tightly coupled to hippocampal processing. These results point toward a relationship between the cerebellum and the hippocampus that occurs during perception of changes in visuospatial information that has previously only been reported with regard to visuospatial navigation.

Structure learning and the posterior parietal cortex

We propose a theory of structure learning in the primate brain. We argue that the parietal cortex is critical for learning about relations among the objects and categories that populate a visual scene. We suggest that current deep learning models exhibit poor global scene understanding because they fail to perform the relational inferences that occur in the primate dorsal stream. We review studies of neural coding in primate posterior parietal cortex (PPC), drawing the conclusion that neurons in this brain area represent potentially high-dimensional inputs on a low-dimensional manifold that encodes the relative position of objects or features in physical space, and relations among entities in abstract conceptual space. We argue that this low-dimensional code supports generalisation of relational information, even in nonspatial domains. Finally, we propose that structure learning is grounded in the actions that primates take when they reach for objects or fixate them with their eyes. We sketch a model of how this might occur in neural circuits.

Storing Object-Dependent Sparse Codes in a Willshaw Associative Network

Willshaw networks are single-layered neural networks that store associations between binary vectors. Using only binary weights, these networks can be implemented efficiently to store large numbers of patterns and allow for fault-tolerant recovery of those patterns from noisy cues. However, this is only the case when the involved codes are sparse and randomly generated. In this letter, we use a recently proposed approach that maps visual patterns into informative binary features. By doing so, we manage to transform MNIST handwritten digits into well-distributed codes that we then store in a Willshaw network in autoassociation. We perform experiments with both noisy and noiseless cues and verify a tenuous impact on the recovered pattern's relevant information. More specifically, we were able to perform retrieval after filling the memory to several factors of its number of units while preserving the information of the class to which the pattern belongs.

Corticocortical Systems Underlying High-Order Motor Control

Cortical networks are characterized by the origin, destination, and reciprocity of their connections, as well as by the diameter, conduction velocity, and synaptic efficacy of their axons. The network formed by parietal and frontal areas lies at the core of cognitive-motor control because the outflow of parietofrontal signaling is conveyed to the subcortical centers and spinal cord through different parallel pathways, whose orchestration determines, not only when and how movements will be generated, but also the nature of forthcoming actions. Despite intensive studies over the last 50 years, the role of corticocortical connections in motor control and the principles whereby selected cortical networks are recruited by different task demands remain elusive. Furthermore, the synaptic integration of different cortical signals, their modulation by transthalamic loops, and the effects of conduction delays remain challenging questions that must be tackled to understand the dynamical aspects of parietofrontal operations. In this article, we evaluate results from nonhuman primate and selected rodent experiments to offer a viewpoint on how corticocortical systems contribute to learning and producing skilled actions. Addressing this subject is not only of scientific interest but also essential for interpreting the devastating consequences for motor control of lesions at different nodes of this integrated circuit. In humans, the study of corticocortical motor networks is currently based on MRI-related methods, such as resting-state connectivity and diffusion tract-tracing, which both need to be contrasted with histological studies in nonhuman primates.

Attention Inspired Network: Steep learning curve in an invariant pattern recognition model

Hubel and Wiesel's study about low areas of the visual cortex (VC) inspired deep models for invariant pattern recognition. In such models, simple and complex layers alternate local feature extraction with subsampling to add invariance to distortion or transformations. However, it was shown that to tolerate large changes between examples of the same category, the subsampling operation has to discard so much information that the model loses the capability to discriminate between categories. So, in practice, small changes are tolerated by these layers and, afterwards, a powerful classifier is introduced to do the rest. By incorporating insights from higher areas of the VC, we add to the already used retinotopic step an object-centered step which increases invariance capabilities without losing so much information. By doing so, we reduce the need for a powerful, data hungry classification layer and, thus, are able to introduce a simple classification mechanism which is based on selective attention. The resulting model is tested with an invariant pattern recognition task in the MNIST and ETL-1 datasets. We verify that the model is able to achieve better accuracies with less training examples. More specifically, on the MNIST test set, the model achieves a 100% accuracy when trained with little more than 10% of the training set.

Space and Time: The Hippocampus as a Sequence Generator

Neural computations are often compared to instrument-measured distance or duration, and such relationships are interpreted by a human observer. However, neural circuits do not depend on human-made instruments but perform computations relative to an internally defined rate-of-change. While neuronal correlations with external measures, such as distance or duration, can be observed in spike rates or other measures of neuronal activity, what matters for the brain is how such activity patterns are utilized by downstream neural observers. We suggest that hippocampal operations can be described by the sequential activity of neuronal assemblies and their internally defined rate of change without resorting to the concept of space or time.

Modulating Spatial Processes and Navigation via Transcranial Electrical Stimulation: A Mini Review

Transcranial electrical stimulation (tES) uses low intensity current to alter neuronal activity in superficial cortical regions, and has gained popularity as a tool for modulating several aspects of perception and cognition. This mini-review article provides an overview of tES and its potential for modulating spatial processes underlying successful navigation, including spatial attention, spatial perception, mental rotation and visualization. Also considered are recent advances in empirical research and computational modeling elucidating several stable cortical-subcortical networks with dynamic involvement in spatial processing and navigation. Leveraging these advances may prove valuable for using tES, particularly transcranial direct and alternating current stimulation (tDCS/tACS), to indirectly target subcortical brain regions by altering neuronal activity in distant yet functionally connected cortical areas. We propose future research directions to leverage these advances in human neuroscience.

Constructional apraxia

Since the classic papers of Kleist, Mayer Gross, and Critchley, constructional apraxia (CA) has been considered to be a typical sign of a parietal lobe lesion, and as a precious tool to appreciate the spatial abilities subserved by this lobe. However, the development of more sophisticated neuropsychologic models and methods of investigation has revealed several problematic aspects. It has become increasingly clear that CA is a heterogeneous construct that can be examined with very different tasks, that are only mildly interconnected, and tap various kinds of visuospatial, perceptual, attentional, planning, and motor mechanisms. On the basis of these considerations, the relationships between parietal lobe functions and constructional activities must be considered, taking into account on the one hand the heterogeneity of the tasks and of the cognitive functions requested by different kinds of constructional activities and, on the other hand, the plurality of functions and of processing streams linking different parts of the parietal lobes to the occipital and frontal lobes.

Abnormal cortical neural synchrony during working memory in schizophrenia

OBJECTIVE

To better understand the origins of working memory (WM) impairment in schizophrenia we investigated cortical oscillatory activity in people with schizophrenia (PSZ) while they performed a WM task requiring encoding, maintenance, and retrieval/manipulation processes of spatial information.

METHODS

We examined time-frequency synchronous energy of cortical source signals that were derived from magnetoencephalography (MEG) localized to cortical regions using WM-related hemodynamic responses and individualized structural head-models.

RESULTS

Compared to thirteen healthy controls (HC), twelve PSZ showed performance deficits regardless of WM-load or duration. During encoding, PSZ had early theta and delta event-related synchrony (ERS) deficits in prefrontal and visual cortices which worsened with greater memory load and predicted WM performance. During prolonged maintenance of material, PSZ showed deficient beta event-related desynchrony (ERD) in dorsolateral prefrontal, posterior parietal, and visual cortices. In retrieval, PSZ showed reduced delta/theta ERS in the anterior prefrontal and ventral visual cortices and diminished gamma ERS in the premotor and posterior parietal cortices.

CONCLUSIONS

Although beta/gamma cortical neural oscillatory deficits for maintenance/retrieval are evident during WM, the abnormal prefrontal theta-frequency ERS for encoding is most predictive of poor WM in schizophrenia.

SIGNIFICANCE

Time-frequency-spatial analysis identified process- and frequency-specific neural synchrony abnormalities underlying WM deficits in schizophrenia.

The role of alpha oscillations in deriving and maintaining spatial relations in working memory

Previous research has demonstrated distinct neural correlates for maintenance of abstract, relational versus concrete, sensory information in working memory (WM). Storage of spatial relations in WM results in suppression of posterior sensory regions, which suggests that sensory information is task-irrelevant when relational representations are maintained in WM. However, the neural mechanisms by which abstract representations are derived from sensory information remain unclear. Here, using electroencephalography, we investigated the role of alpha oscillations in deriving spatial relations from a sensory stimulus and maintaining them in WM. Participants encoded two locations into WM, then after an initial maintenance period, a cue indicated whether to convert the spatial information to another sensory representation or to a relational representation. Results revealed that alpha power increased over posterior electrodes when sensory information was converted to a relational representation, but not when the information was converted to another sensory representation. Further, alpha phase synchrony between posterior and frontal regions increased for relational compared to sensory trials during the maintenance period. These results demonstrate that maintaining spatial relations and locations in WM rely on distinct neural oscillatory patterns.

Computational Architecture of the Parieto-Frontal Network Underlying Cognitive-Motor Control in Monkeys

The statistical structure of intrinsic parietal and parieto-frontal connectivity in monkeys was studied through hierarchical cluster analysis. Based on their inputs, parietal and frontal areas were grouped into different clusters, including a variable number of areas that in most instances occupied contiguous architectonic fields. Connectivity tended to be stronger locally: that is, within areas of the same cluster. Distant frontal and parietal areas were targeted through connections that in most instances were reciprocal and often of different strength. These connections linked parietal and frontal clusters formed by areas sharing basic functional properties. This led to five different medio-laterally oriented pillar domains spanning the entire extent of the parieto-frontal system, in the posterior parietal, anterior parietal, cingulate, frontal, and prefrontal cortex. Different information processing streams could be identified thanks to inter-domain connectivity. These streams encode fast hand reaching and its control, complex visuomotor action spaces, hand grasping, action/intention recognition, oculomotor intention and visual attention, behavioral goals and strategies, and reward and decision value outcome. Most of these streams converge on the cingulate domain, the main hub of the system. All of them are embedded within a larger eye–hand coordination network, from which they can be selectively set in motion by task demands.

Short-latency allocentric control of saccadic eye movements

It is generally accepted that the neural circuits that are implicated in saccade control use retinotopically coded target locations. However, several studies have revealed that nonretinotopic representation is also used. This idea raises a question about whether nonretinotopic coding is egocentric (head or body centered) or allocentric (environment centered). In the current study, we hypothesized that allocentric coding may play a crucial role in immediate saccade control. To test this hypothesis, we used an immediate double-step saccade task toward two sequentially flashed targets with a frame in the background, and we examined whether the end point of the second saccade was affected by a transient shift of the background that participants were told to ignore. When the background was shifted transiently upward (or downward) during the flash of the second target, the second saccade generally erred the target downward (or upward), which was in the direction opposite to the shift of the background. The effect on the second saccade became significant within 150 ms after the frame was presented for decoding and was built up for 200 ms thereafter. When the second saccade was not adjusted, a small, corrective saccade followed within 300 ms. The effect scaled linearly with the shift size up to 3° for a noncorrective second saccade and up to 6° for a corrective saccade. The present results show that an allocentric location of a target is rapidly represented by the brain and used for controlling saccades.

NEW & NOTEWORTHY

We found that the saccade end point was shifted from the actual target position toward the direction expected from allocentric coding when a large frame in the background was transiently shifted during the period of target presentation. The effect occurred within 150 ms. The present study provides direct evidence that the brain rapidly uses allocentric coding of a target to control immediate saccades.

Recording extracellular neural activity in the behaving monkey using a semichronic and high-density electrode system

We describe a technique to semichronically record the cortical extracellular neural activity in the behaving monkey employing commercial high-density electrodes. After the design and construction of low cost microdrives that allow varying the depth of the recording locations after the implantation surgery, we recorded the extracellular unit activity from pools of neurons at different depths in the presupplementary motor cortex (pre-SMA) of a rhesus monkey trained in a tapping task. The collected data were processed to classify cells as putative pyramidal cells or interneurons on the basis of their waveform features. We also demonstrate that short time cross-correlogram occasionally yields unit pairs with high short latency (<5 ms), narrow bin (<3 ms) peaks, indicative of monosynaptic spike transmission from pre- to postsynaptic neurons. These methods have been verified extensively in rodents. Finally, we observed that the pattern of population activity was repetitive over distinct trials of the tapping task. These results show that the semichronic technique is a viable option for the large-scale parallel recording of local circuit activity at different depths in the cortex of the macaque monkey and other large species.

Toward a Unified Sub-symbolic Computational Theory of Cognition

This paper proposes how various disciplinary theories of cognition may be combined into a unifying, sub-symbolic, computational theory of cognition. The following theories are considered for integration: psychological theories, including the theory of event coding, event segmentation theory, the theory of anticipatory behavioral control, and concept development; artificial intelligence and machine learning theories, including reinforcement learning and generative artificial neural networks; and theories from theoretical and computational neuroscience, including predictive coding and free energy-based inference. In the light of such a potential unification, it is discussed how abstract cognitive, conceptualized knowledge and understanding may be learned from actively gathered sensorimotor experiences. The unification rests on the free energy-based inference principle, which essentially implies that the brain builds a predictive, generative model of its environment. Neural activity-oriented inference causes the continuous adaptation of the currently active predictive encodings. Neural structure-oriented inference causes the longer term adaptation of the developing generative model as a whole. Finally, active inference strives for maintaining internal homeostasis, causing goal-directed motor behavior. To learn abstract, hierarchical encodings, however, it is proposed that free energy-based inference needs to be enhanced with structural priors, which bias cognitive development toward the formation of particular, behaviorally suitable encoding structures. As a result, it is hypothesized how abstract concepts can develop from, and thus how they are structured by and grounded in, sensorimotor experiences. Moreover, it is sketched-out how symbol-like thought can be generated by a temporarily active set of predictive encodings, which constitute a distributed neural attractor in the form of an interactive free-energy minimum. The activated, interactive network attractor essentially characterizes the semantics of a concept or a concept composition, such as an actual or imagined situation in our environment. Temporal successions of attractors then encode unfolding semantics, which may be generated by a behavioral or mental interaction with an actual or imagined situation in our environment. Implications, further predictions, possible verification, and falsifications, as well as potential enhancements into a fully spelled-out unified theory of cognition are discussed at the end of the paper.

Are All Spatial Reference Frames Egocentric? Reinterpreting Evidence for Allocentric, Object-Centered, or World-Centered Reference Frames

The use and neural representation of egocentric spatial reference frames is well-documented. In contrast, whether the brain represents spatial relationships between objects in allocentric, object-centered, or world-centered coordinates is debated. Here, I review behavioral, neuropsychological, neurophysiological (neuronal recording), and neuroimaging evidence for and against allocentric, object-centered, or world-centered spatial reference frames. Based on theoretical considerations, simulations, and empirical findings from spatial navigation, spatial judgments, and goal-directed movements, I suggest that all spatial representations may in fact be dependent on egocentric reference frames.

Differential neural activity patterns for spatial relations in humans: a MEG study

Children learn the words for above-below relations earlier than for left-right relations, despite treating these equally well in a simple visual categorization task. Even as adults--conflicts in congruency, such as when a stimulus is depicted in a spatially incongruent manner with respect to salient global cues--can be challenging. Here we investigated the neural correlates of encoding and maintaining in working memory above-below and left-right relational planes in 12 adults using magnetoencephalography in order to discover whether above-below relations are represented by the brain differently than left-right relations. Adults performed perfectly on the task behaviorally, so any differences in neural activity were attributed to the stimuli's cognitive attributes. In comparing above-below to left-right relations during stimulus encoding, we found the greatest differences in neural activity in areas associated with space and movement. In comparing congruent to incongruent trials, we found the greatest differential activity in premotor areas. For both contrasts, brain areas involved in the encoding phase were also involved in the maintenance phase, which provides evidence that those brain areas are particularly important in representing the relational planes or congruency types throughout the trial. When comparing neural activity associated with the relational planes during working memory, additional right posterior areas were implicated, whereas the congruent-incongruent contrast implicated additional bilateral frontal and temporal areas. These findings are consistent with the hypothesis left-right relations are represented differently than above-below relations.

Organization and evolution of parieto-frontal processing streams in macaque monkeys and humans

The functional organization of the parieto-frontal system is crucial for understanding cognitive-motor behavior and provides the basis for interpreting the consequences of parietal lesions in humans from a neurobiological perspective. The parieto-frontal connectivity defines some main information streams that, rather than being devoted to restricted functions, underlie a rich behavioral repertoire. Surprisingly, from macaque to humans, evolution has added only a few, new functional streams, increasing however their complexity and encoding power. In fact, the characterization of the conduction times of parietal and frontal areas to different target structures has recently opened a new window on cortical dynamics, suggesting that evolution has amplified the probability of dynamic interactions between the nodes of the network, thanks to communication patterns based on temporally-dispersed conduction delays. This might allow the representation of sensory-motor signals within multiple neural assemblies and reference frames, as to optimize sensory-motor remapping within an action space characterized by different and more complex demands across evolution.

Automatic representation of a visual stimulus relative to a background in the right precuneus

Our brains represent the position of a visual stimulus egocentrically, in either retinal or craniotopic coordinates. In addition, recent behavioral studies have shown that the stimulus position is automatically represented allocentrically relative to a large frame in the background. Here, we investigated neural correlates of the ‘background coordinate’ using an fMRI adaptation technique. A red dot was presented at different locations on a screen, in combination with a rectangular frame that was also presented at different locations, while the participants looked at a fixation cross. When the red dot was presented repeatedly at the same location relative to the rectangular frame, the fMRI signals significantly decreased in the right precuneus. No adaptation was observed after repeated presentations relative to a small, but salient, landmark. These results suggest that the background coordinate is implemented in the right precuneus.

Fine-grained, local maps and coarse, global representations support human spatial working memory

While sensory processes are tuned to particular features, such as an object's specific location, color or orientation, visual working memory (vWM) is assumed to store information using representations, which generalize over a feature dimension. Additionally, current vWM models presume that different features or objects are stored independently. On the other hand, configurational effects, when observed, are supposed to mainly reflect encoding strategies. We show that the location of the target, relative to the display center and boundaries, and overall memory load influenced recall precision, indicating that, like sensory processes, capacity limited vWM resources are spatially tuned. When recalling one of three memory items the target distance from the display center was overestimated, similar to the error when only one item was memorized, but its distance from the memory items' average position was underestimated, showing that not only individual memory items' position, but also the global configuration of the memory array may be stored. Finally, presenting the non-target items at recall, consequently providing landmarks and configurational information, improved precision and accuracy of target recall. Similarly, when the non-target items were translated at recall, relative to their position in the initial display, a parallel displacement of the recalled target was observed. These findings suggest that fine-grained spatial information in vWM is represented in local maps whose resolution varies with distance from landmarks, such as the display center, while coarse representations are used to store the memory array configuration. Both these representations are updated at the time of recall.

Spatial task context makes short-latency reaches prone to induced Roelofs illusion

The perceptual localization of an object is often more prone to illusions than an immediate visuomotor action towards that object. The induced Roelofs effect (IRE) probes the illusory influence of task-irrelevant visual contextual stimuli on the processing of task-relevant visuospatial instructions during movement preparation. In the IRE, the position of a task-irrelevant visual object induces a shift in the localization of a visual target when subjects indicate the position of the target by verbal response, key-presses or delayed pointing to the target (“perception” tasks), but not when immediately pointing or reaching towards it without instructed delay (“action” tasks). This discrepancy was taken as evidence for the dual-visual-stream or perception-action hypothesis, but was later explained by a phasic distortion of the egocentric spatial reference frame which is centered on subjective straight-ahead (SSA) and used for reach planning. Both explanations critically depend on delayed movements to explain the IRE for action tasks. Here we ask: first, if the IRE can be observed for short-latency reaches; second, if the IRE in fact depends on a distorted egocentric frame of reference. Human subjects were tested in new versions of the IRE task in which the reach goal had to be localized with respect to another object, i.e., in an allocentric reference frame. First, we found an IRE even for immediate reaches in our allocentric task, but not for an otherwise similar egocentric control task. Second, the IRE depended on the position of the task-irrelevant frame relative to the reference object, not relative to SSA. We conclude that the IRE for reaching does not mandatorily depend on prolonged response delays, nor does it depend on motor planning in an egocentric reference frame. Instead, allocentric encoding of a movement goal is sufficient to make immediate reaches susceptible to IRE, underlining the context dependence of visuomotor illusions.

Vestibular pathways involved in cognition

Recent discoveries have emphasized the role of the vestibular system in cognitive processes such as memory, spatial navigation and bodily self-consciousness. A precise understanding of the vestibular pathways involved is essential to understand the consequences of vestibular diseases for cognition, as well as develop therapeutic strategies to facilitate recovery. The knowledge of the “vestibular cortical projection areas”, defined as the cortical areas activated by vestibular stimulation, has dramatically increased over the last several years from both anatomical and functional points of view. Four major pathways have been hypothesized to transmit vestibular information to the vestibular cortex: (1) the vestibulo-thalamo-cortical pathway, which probably transmits spatial information about the environment via the parietal, entorhinal and perirhinal cortices to the hippocampus and is associated with spatial representation and self-versus object motion distinctions; (2) the pathway from the dorsal tegmental nucleus via the lateral mammillary nucleus, the anterodorsal nucleus of the thalamus to the entorhinal cortex, which transmits information for estimations of head direction; (3) the pathway via the nucleus reticularis pontis oralis, the supramammillary nucleus and the medial septum to the hippocampus, which transmits information supporting hippocampal theta rhythm and memory; and (4) a possible pathway via the cerebellum, and the ventral lateral nucleus of the thalamus (perhaps to the parietal cortex), which transmits information for spatial learning. Finally a new pathway is hypothesized via the basal ganglia, potentially involved in spatial learning and spatial memory. From these pathways, progressively emerges the anatomical network of vestibular cognition.

Maintenance of relational information in working memory leads to suppression of the sensory cortex

Working memory (WM) for sensory-based information about individual objects and their locations appears to involve interactions between lateral prefrontal and sensory cortexes. The mechanisms and representations for maintenance of more abstract, nonsensory information in WM are unknown, particularly whether such actively maintained information can become independent of the sensory information from which it was derived. Previous studies of WM for individual visual items found increased electroencephalogram (EEG) alpha (8-13 Hz) power over posterior electrode sites, which appears to correspond to the suppression of cortical areas that represent irrelevant sensory information. Here, we recorded EEG while participants performed a visual WM task that involved maintaining either concrete spatial coordinates or abstract relational information. Maintenance of relational information resulted in higher alpha power in posterior electrodes. Furthermore, lateralization of alpha power due to a covert shift of attention to one visual hemifield was marginally weaker during storage of relational information than during storage of concrete information. These results suggest that abstract relational information is maintained in WM differently from concrete, sensory representations and that during maintenance of abstract information, posterior sensory regions become task irrelevant and are thus suppressed.

Human fronto-parietal and parieto-hippocampal pathways represent behavioral priorities in multiple spatial reference frames

We represent behaviorally relevant information in different spatial reference frames in order to interact effectively with our environment. For example, we need an egocentric (e.g., body-centered) reference frame to specify limb movements and an allocentric (e.g., world-centered) reference frame to navigate from one location to another. Posterior parietal cortex (PPC) is vital for performing transformations between these different coordinate systems. Here, we review evidence for multiple pathways in the human brain, from PPC to motor, premotor, and supplementary motor areas, as well as to structures in the medial temporal lobe. These connections are important for transformations between egocentric reference frames to facilitate sensory-guided action, or from egocentric to allocentric reference frames to facilitate spatial navigation.

Spatiotopic perceptual learning mediated by retinotopic processing and attentional remapping

Visual processing takes place in both retinotopic and spatiotopic frames of reference. Whereas visual perceptual learning is usually specific to the trained retinotopic location, our recent study has shown spatiotopic specificity of learning in motion direction discrimination. To explore the mechanisms underlying spatiotopic processing and learning, and to examine whether similar mechanisms also exist in visual form processing, we trained human subjects to discriminate an orientation difference between two successively displayed stimuli, with a gaze shift in between to manipulate their positional relation in the spatiotopic frame of reference without changing their retinal locations. Training resulted in better orientation discriminability for the trained than for the untrained spatial relation of the two stimuli. This learning-induced spatiotopic preference was seen only at the trained retinal location and orientation, suggesting experience-dependent spatiotopic form processing directly based on a retinotopic map. Moreover, a similar but weaker learning-induced spatiotopic preference was still present even if the first stimulus was rendered irrelevant to the orientation discrimination task by having the subjects judge the orientation of the second stimulus relative to its mean orientation in a block of trials. However, if the first stimulus was absent, and thus no attention was captured before the gaze shift, the learning produced no significant spatiotopic preference, suggesting an important role of attentional remapping in spatiotopic processing and learning. Taken together, our results suggest that spatiotopic visual representation can be mediated by interactions between retinotopic processing and attentional remapping, and can be modified by perceptual training.

Functional and structural architecture of the human dorsal frontoparietal attention network

The dorsal frontoparietal attention network has been subdivided into at least eight areas in humans. However, the circuitry linking these areas and the functions of different circuit paths remain unclear. Using a combination of neuroimaging techniques to map spatial representations in frontoparietal areas, their functional interactions, and structural connections, we demonstrate different pathways across human dorsal frontoparietal cortex for the control of spatial attention. Our results are consistent with these pathways computing object-centered and/or viewer-centered representations of attentional priorities depending on task requirements. Our findings provide an organizing principle for the frontoparietal attention network, where distinct pathways between frontal and parietal regions contribute to multiple spatial representations, enabling flexible selection of behaviorally relevant information.

Transfer of a serial representation between two distinct tasks by rhesus macaques

Do animals form task-specific representations, or do those representations take a general form that can be applied to qualitatively different tasks? Rhesus monkeys (Macaca mulatta) learned the ordering of stimulus lists using two different serial tasks, in order to test whether prior experience in each task could be transfered to the other, enhancing performance. The simultaneous chaining paradigm delivered rewards only after subjects responded in the correct order to all stimuli displayed on a touch sensitive video monitor. The transitive inference paradigm presented pairs of items and delivered rewards when subjects selected the item with the lower ordinal rank. After learning a list in one paradigm, subjects' knowledge of that list was tested using the other paradigm. Performance was enhanced from the very start of transfer training. Transitive inference performance was characterized by 'symbolic distance effects,' whereby the ordinal distance between stimuli in the implied list ordering was strongly predictive of the probability of a correct response. The patterns of error displayed by subjects in both tasks were best explained by a spatially coded representation of list items, regardless of which task was used to learn the list. Our analysis permits properties of this representation to be investigated without the confound of verbal reasoning.

Physiologically inspired model for the visual recognition of transitive hand actions

The visual recognition of actions is an important visual function that is critical for motor learning and social communication. Action-selective neurons have been found in different cortical regions, including the superior temporal sulcus, parietal and premotor cortex. Among those are mirror neurons, which link visual and motor representations of body movements. While numerous theoretical models for the mirror neuron system have been proposed, the computational basis of the visual processing of goal-directed actions remains largely unclear. While most existing models focus on the possible role of motor representations in action recognition, we propose a model showing that many critical properties of action-selective visual neurons can be accounted for by well-established visual mechanisms. Our model accomplishes the recognition of hand actions from real video stimuli, exploiting exclusively mechanisms that can be implemented in a biologically plausible way by cortical neurons. We show that the model provides a unifying quantitatively consistent account of a variety of electrophysiological results from action-selective visual neurons. In addition, it makes a number of predictions, some of which could be confirmed in recent electrophysiological experiments.

A scalar neural code for categories in parietal cortex: representing cognitive variables as "more" or "less"

In this issue, Fitzgerald et al. (2013) show that LIP neurons in monkeys encode categorically distinct task conditions using a scalar code. Activity scales up or down to encode different categories, with neurons maintaining proportional levels of activity in relation to one another.

Thinking in spatial terms: decoupling spatial representation from sensorimotor control in monkey posterior parietal areas 7a and LIP

Perhaps the simplest and most complete description of the cerebral cortex is that it is a sensorimotor controller whose primary purpose is to represent stimuli and movements, and adaptively control the mapping between them. However, in order to think, the cerebral cortex has to generate patterns of neuronal activity that encode abstract, generalized information independently of ongoing sensorimotor events. A critical question confronting cognitive systems neuroscience at present therefore is how neural signals encoding abstract information emerge within the sensorimotor control networks of the brain. In this review, we approach that question in the context of the neural representation of space in posterior parietal cortex of non-human primates. We describe evidence indicating that parietal cortex generates a hierarchy of spatial representations with three basic levels: including (1) sensorimotor signals that are tightly coupled to stimuli or movements, (2) sensorimotor signals modified in strength or timing to mediate cognition (examples include attention, working memory, and decision-processing), as well as (3) signals that encode frankly abstract spatial information (such as spatial relationships or categories) generalizing across a wide diversity of specific stimulus conditions. Here we summarize the evidence for this hierarchy, and consider data showing that signals at higher levels derive from signals at lower levels. That in turn could help characterize neural mechanisms that derive a capacity for abstraction from sensorimotor experience.

Trial time warping to discriminate stimulus-related from movement-related neural activity

In tasks where different sensory, cognitive, and motor events are mixed in a sequence it is difficult to determine whether neural activity is related to any behavioral parameter. Here, we consider the case in which two alternative trial-alignment schemes correspond to two different neural representations, one stimulus-related and the other movement-related, using both simulations of neural activity and real recordings in the medial premotor areas during a multiple-interval tapping task called synchronization-continuation task (SCT). To discover whether neural responses are better aligned to sensory or motor events we introduce a family of trial-alignment time-warping functions indexed by a single parameter such that when the parameter takes the value 0 the trials are aligned to the stimulus and when the parameter takes the value 1 they are aligned to the movement. We then characterize neurons by the best-fitting alignment scheme (in the sense of maximum likelihood) under the assumption that the correct alignment would produce homogeneous trials without excess trial-to-trial variation. We use Bayes factors to determine the evidence in favor of sensory or motor neural alignments. The simulations revealed that the variability in neural responses and sequential motor outputs are key parameters to obtain appropriate warping results. In addition, the analysis on the activity of 500 neurons in the medial premotor areas of monkeys executing the SCT showed that most of the neural responses (54.2%) were aligned to the tapping movements instead of the stimuli used to drive the temporal behavior.

A neurobehavioral model of flexible spatial language behaviors

We propose a neural dynamic model that specifies how low-level visual processes can be integrated with higher level cognition to achieve flexible spatial language behaviors. This model uses real-word visual input that is linked to relational spatial descriptions through a neural mechanism for reference frame transformations. We demonstrate that the system can extract spatial relations from visual scenes, select items based on relational spatial descriptions, and perform reference object selection in a single unified architecture. We further show that the performance of the system is consistent with behavioral data in humans by simulating results from 2 independent empirical studies, 1 spatial term rating task and 1 study of reference object selection behavior. The architecture we present thereby achieves a high degree of task flexibility under realistic stimulus conditions. At the same time, it also provides a detailed neural grounding for complex behavioral and cognitive processes.