Representational geometry of perceptual decisions in the monkey parietal cortex

Irrational choices via a curvilinear representational geometry for value

We make decisions by comparing values, but it is not yet clear how value is represented in the brain. Many models assume, if only implicitly, that the representational geometry of value is linear. However, in part due to a historical focus on noisy single neurons, rather than neuronal populations, this hypothesis has not been rigorously tested. Here, we examine the representational geometry of value in the ventromedial prefrontal cortex (vmPFC), a part of the brain linked to economic decision-making, in two male rhesus macaques. We find that values are encoded along a curved manifold in vmPFC. This curvilinear geometry predicts a specific pattern of irrational decision-making: that decision-makers will make worse choices when an irrelevant, decoy option is worse in value, compared to when it is better. We observe this type of irrational choices in behavior. Together, these results not only suggest that the representational geometry of value is nonlinear, but that this nonlinearity could impose bounds on rational decision-making.

Coordinated Response Modulations Enable Flexible Use of Visual Information

We use sensory information in remarkably flexible ways. We can generalize by ignoring task-irrelevant features, report different features of a stimulus, and use different actions to report a perceptual judgment. These forms of flexible behavior are associated with small modulations of the responses of sensory neurons. While the existence of these response modulations is indisputable, efforts to understand their function have been largely relegated to theory, where they have been posited to change information coding or enable downstream neurons to read out different visual and cognitive information using flexible weights. Here, we tested these ideas using a rich, flexible behavioral paradigm, multi-neuron, multi-area recordings in primary visual cortex (V1) and mid-level visual area V4. We discovered that those response modulations in V4 (but not V1) contain the ingredients necessary to enable flexible behavior, but not via those previously hypothesized mechanisms. Instead, we demonstrated that these response modulations are precisely coordinated across the population such that downstream neurons have ready access to the correct information to flexibly guide behavior without making changes to information coding or synapses. Our results suggest a novel computational role for task-dependent response modulations: they enable flexible behavior by changing the information that gets out of a sensory area, not by changing information coding within it.

Computational role of structure in neural activity and connectivity

One major challenge of neuroscience is identifying structure in seemingly disorganized neural activity. Different types of structure have different computational implications that can help neuroscientists understand the functional role of a particular brain area. Here, we outline a unified approach to characterize structure by inspecting the representational geometry and the modularity properties of the recorded activity and show that a similar approach can also reveal structure in connectivity. We start by setting up a general framework for determining geometry and modularity in activity and connectivity and relating these properties with computations performed by the network. We then use this framework to review the types of structure found in recent studies of model networks performing three classes of computations.

A neurocomputational model of decision and confidence in object recognition task

How does the brain process natural visual stimuli to make a decision? Imagine driving through fog. An object looms ahead. What do you do? This decision requires not only identifying the object but also choosing an action based on your decision confidence. In this circumstance, confidence is making a bridge between seeing and believing. Our study unveils how the brain processes visual information to make such decisions with an assessment of confidence, using a model inspired by the visual cortex. To computationally model the process, this study uses a spiking neural network inspired by the hierarchy of the visual cortex in mammals to investigate the dynamics of feedforward object recognition and decision-making in the brain. The model consists of two modules: a temporal dynamic object representation module and an attractor neural network-based decision-making module. Unlike traditional models, ours captures the evolution of evidence within the visual cortex, mimicking how confidence forms in the brain. This offers a more biologically plausible approach to decision-making when encountering real-world stimuli. We conducted experiments using natural stimuli and measured accuracy, reaction time, and confidence. The model's estimated confidence aligns remarkably well with human-reported confidence. Furthermore, the model can simulate the human change-of-mind phenomenon, reflecting the ongoing evaluation of evidence in the brain. Also, this finding offers decision-making and confidence encoding share the same neural circuit.

Decision uncertainty as a context for motor memory

The current view of perceptual decision-making suggests that once a decision is made, only a single motor programme associated with the decision is carried out, irrespective of the uncertainty involved in decision making. In contrast, we show that multiple motor programmes can be acquired on the basis of the preceding uncertainty of the decision, indicating that decision uncertainty functions as a contextual cue for motor memory. The actions learned after making certain (uncertain) decisions are only partially transferred to uncertain (certain) decisions. Participants were able to form distinct motor memories for the same movement on the basis of the preceding decision uncertainty. Crucially, this contextual effect generalizes to novel stimuli with matched uncertainty levels, demonstrating that decision uncertainty is itself a contextual cue. These findings broaden the understanding of contextual inference in motor memory, emphasizing that it extends beyond direct motor control cues to encompass the decision-making process.

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes.

Nonlinear manifolds underlie neural population activity during behaviour

There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.

Recurrent Neural Circuits Overcome Partial Inactivation by Compensation and Re-learning

Technical advances in artificial manipulation of neural activity have precipitated a surge in studying the causal contribution of brain circuits to cognition and behavior. However, complexities of neural circuits challenge interpretation of experimental results, necessitating new theoretical frameworks for reasoning about causal effects. Here, we take a step in this direction, through the lens of recurrent neural networks trained to perform perceptual decisions. We show that understanding the dynamical system structure that underlies network solutions provides a precise account for the magnitude of behavioral effects due to perturbations. Our framework explains past empirical observations by clarifying the most sensitive features of behavior, and how complex circuits compensate and adapt to perturbations. In the process, we also identify strategies that can improve the interpretability of inactivation experiments.

What does the mean mean? A simple test for neuroscience

Trial-averaged metrics, e.g. tuning curves or population response vectors, are a ubiquitous way of characterizing neuronal activity. But how relevant are such trial-averaged responses to neuronal computation itself? Here we present a simple test to estimate whether average responses reflect aspects of neuronal activity that contribute to neuronal processing. The test probes two assumptions implicitly made whenever average metrics are treated as meaningful representations of neuronal activity: Reliability: Neuronal responses repeat consistently enough across trials that they convey a recognizable reflection of the average response to downstream regions.Behavioural relevance: If a single-trial response is more similar to the average template, it is more likely to evoke correct behavioural responses. We apply this test to two data sets: (1) Two-photon recordings in primary somatosensory cortices (S1 and S2) of mice trained to detect optogenetic stimulation in S1; and (2) Electrophysiological recordings from 71 brain areas in mice performing a contrast discrimination task. Under the highly controlled settings of Data set 1, both assumptions were largely fulfilled. In contrast, the less restrictive paradigm of Data set 2 met neither assumption. Simulations predict that the larger diversity of neuronal response preferences, rather than higher cross-trial reliability, drives the better performance of Data set 1. We conclude that when behaviour is less tightly restricted, average responses do not seem particularly relevant to neuronal computation, potentially because information is encoded more dynamically. Most importantly, we encourage researchers to apply this simple test of computational relevance whenever using trial-averaged neuronal metrics, in order to gauge how representative cross-trial averages are in a given context.

A fully spiking coupled model of a deep neural network and a recurrent attractor explains dynamics of decision making in an object recognition task

Objective.Object recognition and making a choice regarding the recognized object is pivotal for most animals. This process in the brain contains information representation and decision making steps which both take different amount of times for different objects. While dynamics of object recognition and decision making are usually ignored in object recognition models, here we proposed a fully spiking hierarchical model, explaining the process of object recognition from information representation to making decision.Approach.Coupling a deep neural network and a recurrent attractor based decision making model beside using spike time dependent plasticity learning rules in several convolutional and pooling layers, we proposed a model which can resemble brain behaviors during an object recognition task. We also measured human choices and reaction times in a psychophysical object recognition task and used it as a reference to evaluate the model.Main results.The proposed model explains not only the probability of making a correct decision but also the time that it takes to make a decision. Importantly, neural firing rates in both feature representation and decision making levels mimic the observed patterns in animal studies (number of spikes (p-value < 10-173) and the time of the peak response (p-value < 10-31) are significantly modulated with the strength of the stimulus). Moreover, the speed-accuracy trade-off as a well-known characteristic of decision making process in the brain is also observed in the model (changing the decision bound significantly affect the reaction time (p-value < 10-59) and accuracy (p-value < 10-165)).Significance.We proposed a fully spiking deep neural network which can explain dynamics of making decision about an object in both neural and behavioral level. Results showed that there is a strong and significant correlation (r= 0.57) between the reaction time of the model and of human participants in the psychophysical object recognition task.

Learning attentional templates for value-based decision-making

Attention filters sensory inputs to enhance task-relevant information. It is guided by an "attentional template" that represents the stimulus features that are currently relevant. To understand how the brain learns and uses templates, we trained monkeys to perform a visual search task that required them to repeatedly learn new attentional templates. Neural recordings found that templates were represented across the prefrontal and parietal cortex in a structured manner, such that perceptually neighboring templates had similar neural representations. When the task changed, a new attentional template was learned by incrementally shifting the template toward rewarded features. Finally, we found that attentional templates transformed stimulus features into a common value representation that allowed the same decision-making mechanisms to deploy attention, regardless of the identity of the template. Altogether, our results provide insight into the neural mechanisms by which the brain learns to control attention and how attention can be flexibly deployed across tasks.

Representation of Natural Contours by a Neural Population in Monkey V4

The cortical visual area, V4, has been considered to code contours that contribute to the intermediate-level representation of objects. The neural responses to the complex contour features intrinsic to natural contours are expected to clarify the essence of the representation. To approach the cortical coding of natural contours, we investigated the simultaneous coding of multiple contour features in monkey (Macaca fuscata) V4 neurons and their population-level representation. A substantial number of neurons showed significant tuning for two or more features such as curvature and closure, indicating that a substantial number of V4 neurons simultaneously code multiple contour features. A large portion of the neurons responded vigorously to acutely curved contours that surrounded the center of classical receptive field, suggesting that V4 neurons tend to code prominent features of object contours. The analysis of mutual information (MI) between the neural responses and each contour feature showed that most neurons exhibited similar magnitudes for each type of MI, indicating that many neurons showing the responses depended on multiple contour features. We next examined the population-level representation by using multidimensional scaling analysis. The neural preferences to the multiple contour features and that to natural stimuli compared with silhouette stimuli increased along with the primary and secondary axes, respectively, indicating the contribution of the multiple contour features and surface textures in the population responses. Our analyses suggested that V4 neurons simultaneously code multiple contour features in natural images and represent contour and surface properties in population.

The neuronal implementation of representational geometry in primate prefrontal cortex

Modern neuroscience has seen the rise of a population-doctrine that represents cognitive variables using geometrical structures in activity space. Representational geometry does not, however, account for how individual neurons implement these representations. Leveraging the principle of sparse coding, we present a framework to dissect representational geometry into biologically interpretable components that retain links to single neurons. Applied to extracellular recordings from the primate prefrontal cortex in a working memory task with interference, the identified components revealed disentangled and sequential memory representations including the recovery of memory content after distraction, signals hidden to conventional analyses. Each component was contributed by small subpopulations of neurons with distinct spiking properties and response dynamics. Modeling showed that such sparse implementations are supported by recurrently connected circuits as in prefrontal cortex. The perspective of neuronal implementation links representational geometries to their cellular constituents, providing mechanistic insights into how neural systems encode and process information.

Scalable, flexible carbon fiber electrode thread arrays for three-dimensional probing of neurochemical activity in deep brain structures of rodents

We developed a flexible "electrode-thread" array for recording dopamine neurochemicals from a lateral distribution of subcortical targets (up to 16) transverse to the axis of insertion. Ultrathin (∼10 μm diameter) carbon fiber (CF) electrode-threads (CFETs) are clustered into a tight bundle to introduce them into the brain from a single-entry point. The individual CFETs splay laterally in deep brain tissue during insertion due to their innate flexibility. This spatial redistribution allows navigation of the CFETs towards deep brain targets spreading horizontally from the axis of insertion. Commercial "linear" arrays provide single-entry insertion but only allow measurements along the axis of insertion. Horizontally configured arrays inflict separate penetrations for each individual channel. We tested functional performance of our CFET arrays in vivo for recording dopamine and for providing lateral spread to multiple distributed sites in the rat striatum. Spatial spread was further characterized in agar brain phantoms as a function of insertion depth. We also developed protocols to slice the embedded CFETs within fixed brain tissue using standard histology. This method allowed extraction of the precise spatial coordinates of the implanted CFETs and their recording sites as integrated with immunohistochemical staining for surrounding anatomical, cytological, and protein expression labels. Our CFET array has the potential to unlock a wide range of applications, from uncovering the role of neuromodulators in synaptic plasticity, to addressing critical safety barriers in clinical translation towards diagnostic and adaptive treatment in Parkinson's disease and major mood disorders.

Transitions in dynamical regime and neural mode underlie perceptual decision-making

Perceptual decision-making is the process by which an animal uses sensory stimuli to choose an action or mental proposition. This process is thought to be mediated by neurons organized as attractor networks 1,2 . However, whether attractor dynamics underlie decision behavior and the complex neuronal responses remains unclear. Here we use an unsupervised, deep learning-based method to discover decision-related dynamics from the simultaneous activity of neurons in frontal cortex and striatum of rats while they accumulate pulsatile auditory evidence. We show that contrary to prevailing hypotheses, attractors play a role only after a transition from a regime in the dynamics that is strongly driven by inputs to one dominated by the intrinsic dynamics. The initial regime mediates evidence accumulation, and the subsequent intrinsic-dominant regime subserves decision commitment. This regime transition is coupled to a rapid reorganization in the representation of the decision process in the neural population (a change in the "neural mode" along which the process develops). A simplified model approximating the coupled transition in the dynamics and neural mode allows inferring, from each trial's neural activity, the internal decision commitment time in that trial, and captures diverse and complex single-neuron temporal profiles, such as ramping and stepping 3-5 . It also captures trial-averaged curved trajectories 6-8 , and reveals distinctions between brain regions. Our results show that the formation of a perceptual choice involves a rapid, coordinated transition in both the dynamical regime and the neural mode of the decision process, and suggest pairing deep learning and parsimonious models as a promising approach for understanding complex data.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here we provide evidence that the energy landscape around attractor basins in population neural activity in the prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays to reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Exploring strategy differences between humans and monkeys with recurrent neural networks

Animal models are used to understand principles of human biology. Within cognitive neuroscience, non-human primates are considered the premier model for studying decision-making behaviors in which direct manipulation experiments are still possible. Some prominent studies have brought to light major discrepancies between monkey and human cognition, highlighting problems with unverified extrapolation from monkey to human. Here, we use a parallel model system-artificial neural networks (ANNs)-to investigate a well-established discrepancy identified between monkeys and humans with a working memory task, in which monkeys appear to use a recency-based strategy while humans use a target-selective strategy. We find that ANNs trained on the same task exhibit a progression of behavior from random behavior (untrained) to recency-like behavior (partially trained) and finally to selective behavior (further trained), suggesting monkeys and humans may occupy different points in the same overall learning progression. Surprisingly, what appears to be recency-like behavior in the ANN, is in fact an emergent non-recency-based property of the organization of the neural network's state space during its development through training. We find that explicit encouragement of recency behavior during training has a dual effect, not only causing an accentuated recency-like behavior, but also speeding up the learning process altogether, resulting in an efficient shaping mechanism to achieve the optimal strategy. Our results suggest a new explanation for the discrepency observed between monkeys and humans and reveal that what can appear to be a recency-based strategy in some cases may not be recency at all.

Initial conditions combine with sensory evidence to induce decision-related dynamics in premotor cortex

We used a dynamical systems perspective to understand decision-related neural activity, a fundamentally unresolved problem. This perspective posits that time-varying neural activity is described by a state equation with an initial condition and evolves in time by combining at each time step, recurrent activity and inputs. We hypothesized various dynamical mechanisms of decisions, simulated them in models to derive predictions, and evaluated these predictions by examining firing rates of neurons in the dorsal premotor cortex (PMd) of monkeys performing a perceptual decision-making task. Prestimulus neural activity (i.e., the initial condition) predicted poststimulus neural trajectories, covaried with RT and the outcome of the previous trial, but not with choice. Poststimulus dynamics depended on both the sensory evidence and initial condition, with easier stimuli and fast initial conditions leading to the fastest choice-related dynamics. Together, these results suggest that initial conditions combine with sensory evidence to induce decision-related dynamics in PMd.

Abstract perceptual choice signals during action-linked decisions in the human brain

Humans can make abstract choices independent of motor actions. However, in laboratory tasks, choices are typically reported with an associated action. Consequentially, knowledge about the neural representation of abstract choices is sparse, and choices are often thought to evolve as motor intentions. Here, we show that in the human brain, perceptual choices are represented in an abstract, motor-independent manner, even when they are directly linked to an action. We measured MEG signals while participants made choices with known or unknown motor response mapping. Using multivariate decoding, we quantified stimulus, perceptual choice, and motor response information with distinct cortical distributions. Choice representations were invariant to whether the response mapping was known during stimulus presentation, and they occupied a distinct representational space from motor signals. As expected from an internal decision variable, they were informed by the stimuli, and their strength predicted decision confidence and accuracy. Our results demonstrate abstract neural choice signals that generalize to action-linked decisions, suggesting a general role of an abstract choice stage in human decision-making.

High-dimensional topographic organization of visual features in the primate temporal lobe

The inferotemporal cortex supports our supreme object recognition ability. Numerous studies have been conducted to elucidate the functional organization of this brain area, but there are still important questions that remain unanswered, including how this organization differs between humans and non-human primates. Here, we use deep neural networks trained on object categorization to construct a 25-dimensional space of visual features, and systematically measure the spatial organization of feature preference in both male monkey brains and human brains using fMRI. These feature maps allow us to predict the selectivity of a previously unknown region in monkey brains, which is corroborated by additional fMRI and electrophysiology experiments. These maps also enable quantitative analyses of the topographic organization of the temporal lobe, demonstrating the existence of a pair of orthogonal gradients that differ in spatial scale and revealing significant differences in the functional organization of high-level visual areas between monkey and human brains.

Attractor dynamics reflect decision confidence in macaque prefrontal cortex

Decisions are made with different degrees of consistency, and this consistency can be linked to the confidence that the best choice has been made. Theoretical work suggests that attractor dynamics in networks can account for choice consistency, but how this is implemented in the brain remains unclear. Here, we provide evidence that the energy landscape around attractor basins in population neural activity in prefrontal cortex reflects choice consistency. We trained two rhesus monkeys to make accept/reject decisions based on pretrained visual cues that signaled reward offers with different magnitudes and delays-to-reward. Monkeys made consistent decisions for very good and very bad offers, but decisions were less consistent for intermediate offers. Analysis of neural data showed that the attractor basins around patterns of activity reflecting decisions had steeper landscapes for offers that led to consistent decisions. Therefore, we provide neural evidence that energy landscapes predict decision consistency, which reflects decision confidence.

Single-neuron and population measures of neuronal activity in working memory tasks

Information represented in working memory is reflected in the firing rate of neurons in the prefrontal cortex and brain areas connected to it. In recent years, there has been an increased realization that population measures capture more accurately neural correlates of cognitive functions. We examined how single neuron firing in the prefrontal and posterior parietal cortex of two male monkeys compared with population measures in spatial working memory tasks. Persistent activity was observed in the dorsolateral prefrontal and posterior parietal cortex and firing rate predicted working memory behavior, particularly in the prefrontal cortex. These findings had equivalents in population measures, including trajectories in state space that became less separated in error trials. We additionally observed rotations of stimulus representations in the neuronal state space for different task conditions, which were not obvious in firing rate measures. These results suggest that population measures provide a richer view of how neuronal activity is associated with behavior, largely confirming that persistent activity is the core phenomenon that maintains visual-spatial information in working memory.NEW & NOTEWORTHY Recordings from large numbers of neurons led to a reevaluation of neural correlates of cognitive functions, which traditionally were defined based on responses of single neurons or averages of firing rates. Analysis of neuronal recordings from the dorsolateral prefrontal and posterior parietal cortex revealed that properties of neuronal firing captured in classical studies of persistent activity can account for population representations, though some population characteristics did not have clear correlates in single neuron activity.

Representational geometry of incomplete faces in macaque face patches

The neural code of faces has been intensively studied in the macaque face patch system. Although the majority of previous studies used complete faces as stimuli, faces are often seen partially in daily life. Here, we investigated how face-selective cells represent two types of incomplete faces: face fragments and occluded faces, with the location of the fragment/occluder and the facial features systematically varied. Contrary to popular belief, we found that the preferred face regions identified with two stimulus types are dissociated in many face cells. This dissociation can be explained by the nonlinear integration of information from different face parts and is closely related to a curved representation of face completeness in the state space, which allows a clear discrimination between different stimulus types. Furthermore, identity-related facial features are represented in a subspace orthogonal to the nonlinear dimension of face completeness, supporting a condition-general code of facial identity.

Neural knowledge assembly in humans and neural networks

Human understanding of the world can change rapidly when new information comes to light, such as when a plot twist occurs in a work of fiction. This flexible "knowledge assembly" requires few-shot reorganization of neural codes for relations among objects and events. However, existing computational theories are largely silent about how this could occur. Here, participants learned a transitive ordering among novel objects within two distinct contexts before exposure to new knowledge that revealed how they were linked. Blood-oxygen-level-dependent (BOLD) signals in dorsal frontoparietal cortical areas revealed that objects were rapidly and dramatically rearranged on the neural manifold after minimal exposure to linking information. We then adapt online stochastic gradient descent to permit similar rapid knowledge assembly in a neural network model.

Comparing representations and computations in single neurons versus neural networks

Single-neuron-level explanations have been the gold standard in neuroscience for decades. Recently, however, neural-network-level explanations have become increasingly popular. This increase in popularity is driven by the fact that the analysis of neural networks can solve problems that cannot be addressed by analyzing neurons independently. In this opinion article, I argue that while both frameworks employ the same general logic to link physical and mental phenomena, in many cases the neural network framework provides better explanatory objects to understand representations and computations related to mental phenomena. I discuss what constitutes a mechanistic explanation in neural systems, provide examples, and conclude by highlighting a number of the challenges and considerations associated with the use of analyses of neural networks to study brain function.

Low-dimensional encoding of decisions in parietal cortex reflects long-term training history

Neurons in parietal cortex exhibit task-related activity during decision-making tasks. However, it remains unclear how long-term training to perform different tasks over months or even years shapes neural computations and representations. We examine lateral intraparietal area (LIP) responses during a visual motion delayed-match-to-category task. We consider two pairs of male macaque monkeys with different training histories: one trained only on the categorization task, and another first trained to perform fine motion-direction discrimination (i.e., pretrained). We introduce a novel analytical approach-generalized multilinear models-to quantify low-dimensional, task-relevant components in population activity. During the categorization task, we found stronger cosine-like motion-direction tuning in the pretrained monkeys than in the category-only monkeys, and that the pretrained monkeys' performance depended more heavily on fine discrimination between sample and test stimuli. These results suggest that sensory representations in LIP depend on the sequence of tasks that the animals have learned, underscoring the importance of considering training history in studies with complex behavioral tasks.

Neural Mechanisms That Make Perceptual Decisions Flexible

Neural mechanisms of perceptual decision making have been extensively studied in experimental settings that mimic stable environments with repeating stimuli, fixed rules, and payoffs. In contrast, we live in an ever-changing environment and have varying goals and behavioral demands. To accommodate variability, our brain flexibly adjusts decision-making processes depending on context. Here, we review a growing body of research that explores the neural mechanisms underlying this flexibility. We highlight diverse forms of context dependency in decision making implemented through a variety of neural computations. Context-dependent neural activity is observed in a distributed network of brain structures, including posterior parietal, sensory, motor, and subcortical regions, as well as the prefrontal areas classically implicated in cognitive control. We propose that investigating the distributed network underlying flexible decisions is key to advancing our understanding and discuss a path forward for experimental and theoretical investigations.

Transformation of acoustic information to sensory decision variables in the parietal cortex

The process by which sensory evidence contributes to perceptual choices requires an understanding of its transformation into decision variables. Here, we address this issue by evaluating the neural representation of acoustic information in the auditory cortex-recipient parietal cortex, while gerbils either performed a two-alternative forced-choice auditory discrimination task or while they passively listened to identical acoustic stimuli. During task engagement, stimulus identity decoding performance from simultaneously recorded parietal neurons significantly correlated with psychometric sensitivity. In contrast, decoding performance during passive listening was significantly reduced. Principal component and geometric analyses revealed the emergence of low-dimensional encoding of linearly separable manifolds with respect to stimulus identity and decision, but only during task engagement. These findings confirm that the parietal cortex mediates a transition of acoustic representations into decision-related variables. Finally, using a clustering analysis, we identified three functionally distinct subpopulations of neurons that each encoded task-relevant information during separate temporal segments of a trial. Taken together, our findings demonstrate how parietal cortex neurons integrate and transform encoded auditory information to guide sound-driven perceptual decisions.

Turning decisions into actions

Are selection and control of action serial processes of separate neural modules? A new study in PLOS Biology argues against this and in favor of an integrated process distributed across multiple brain regions, each contributing in a distinct way.

Integrated neural dynamics of sensorimotor decisions and actions

Recent theoretical models suggest that deciding about actions and executing them are not implemented by completely distinct neural mechanisms but are instead two modes of an integrated dynamical system. Here, we investigate this proposal by examining how neural activity unfolds during a dynamic decision-making task within the high-dimensional space defined by the activity of cells in monkey dorsal premotor (PMd), primary motor (M1), and dorsolateral prefrontal cortex (dlPFC) as well as the external and internal segments of the globus pallidus (GPe, GPi). Dimensionality reduction shows that the four strongest components of neural activity are functionally interpretable, reflecting a state transition between deliberation and commitment, the transformation of sensory evidence into a choice, and the baseline and slope of the rising urgency to decide. Analysis of the contribution of each population to these components shows meaningful differences between regions but no distinct clusters within each region, consistent with an integrated dynamical system. During deliberation, cortical activity unfolds on a two-dimensional "decision manifold" defined by sensory evidence and urgency and falls off this manifold at the moment of commitment into a choice-dependent trajectory leading to movement initiation. The structure of the manifold varies between regions: In PMd, it is curved; in M1, it is nearly perfectly flat; and in dlPFC, it is almost entirely confined to the sensory evidence dimension. In contrast, pallidal activity during deliberation is primarily defined by urgency. We suggest that these findings reveal the distinct functional contributions of different brain regions to an integrated dynamical system governing action selection and execution.

Motor cortical signals reflecting decision making and action preparation

Decision making often requires accumulating evidence in favour of a particular option. When choices are expressed with a motor response, these actions are preceded by reductions in the power of oscillations in the alpha and beta range in motor cortices. For unimanual movements, these reductions are greater over the hemisphere contralateral to the response side. Such lateralizations are hypothesized to be an online index of the neural state of decisions as they develop over time of processing. In contrast, the lateralized readiness potential (LRP) is considered to selectively activate a response and appears shortly before the motor output. We investigated to what extent these neural signals reflect integration of decision evidence or more motor-related action preparation. Using two different experiments, we found that lateralization of alpha and beta power (APL and BPL, respectively) rapidly emerged after stimulus presentation, even when making an overt response was not yet possible. In contrast, we show that even after prolonged stimulus presentation, no LRP was present. Instead, the LRP emerged only after an imperative cue, prompting participants to indicate their choice. Furthermore, we could show that variations in sensory evidence strength modulate APL and BPL onset times, suggesting that integration of evidence is represented in these motor cortical signals. We conclude that APL and BPL reflect higher cognitive processes rather than pure action preparation, whereas LRP is more closely tied to motor performance. APL and BPL potentially encode decision information in motor areas serving the later preparation of overt decision output.

Geometric basis of action potential of skeletal muscle cells and neurons

Although we know something about single-cell neuromuscular junctions, it is still unclear how multiple skeletal muscle cells coordinate to complete intricate spatial curve movement. Here, we hypothesize that skeletal muscle cell populations with action potentials are aligned according to curved manifolds in space (a curved shape in space). When a specific motor nerve impulse is transmitted, the skeletal muscle also moves according to the corresponding shape (manifolds). The action potential of motor nerve fibers has the characteristics of a time curve manifold, and this time-manifold curve of motor nerve fibers comes from the visual cortex in which spatial geometric manifolds are formed within the synaptic connection of neurons. This spatial geometric manifold of the synaptic connection of neurons originates from spatial geometric manifolds outside nature that are transmitted to the brain through the cone cells and ganglion cells of the retina. The essence of life is that life is an object that can move autonomously, and the essence of life's autonomous movement is the movement of proteins. Theoretically, because of the infinite diversity of geometric manifold shapes in nature, the arrangement and combination of 20 amino acids should have infinite diversity, and the geometric manifold formed by the protein three-dimensional spatial structure should also have infinite diversity.

Dynamic task-belief is an integral part of decision-making

Natural decisions involve two seemingly separable processes: inferring the relevant task (task-belief) and performing the believed-relevant task. The assumed separability has led to the traditional practice of studying task-switching and perceptual decision-making individually. Here, we used a novel paradigm to manipulate and measure macaque monkeys' task-belief and demonstrated inextricable neuronal links between flexible task-belief and perceptual decision-making. We showed that in animals, but not in artificial networks that performed as well or better than the animals, stronger task-belief is associated with better perception. Correspondingly, recordings from neuronal populations in cortical areas 7a and V1 revealed that stronger task-belief is associated with better discriminability of the believed-relevant, but not the believed-irrelevant, feature. Perception also impacts belief updating; noise fluctuations in V1 help explain how task-belief is updated. Our results demonstrate that complex tasks and multi-area recordings can reveal fundamentally new principles of how biology affects behavior in health and disease.

Evidence accumulation occurs locally in the parietal cortex

Decision making often entails evidence accumulation, a process that is represented by neural activities in a network of multiple brain areas. Yet, it has not been identified where exactly the accumulation originates. We reason that a candidate brain area should both represent evidence accumulation and information that is used to compute evidence. Therefore, we designed a two-stage probabilistic reasoning task in which the evidence for accumulation had to be first determined from sensory signals orthogonal to decisions. With a linear encoding model, we decomposed the responses of posterior parietal neurons to each stimulus into an early and a late component that represented two dissociable stages of decision making. The former reflected the transformation from sensory inputs to accumulable evidence, and the latter reflected the accumulation of evidence and the formation of decisions. The presence of both computational stages indicates that evidence accumulation signal in the parietal cortex is computed locally.

Mnemonic construction and representation of temporal structure in the hippocampal formation

The hippocampal-entorhinal region supports memory for episodic details, such as temporal relations of sequential events, and mnemonic constructions combining experiences for inferential reasoning. However, it is unclear whether hippocampal event memories reflect temporal relations derived from mnemonic constructions, event order, or elapsing time, and whether these sequence representations generalize temporal relations across similar sequences. Here, participants mnemonically constructed times of events from multiple sequences using infrequent cues and their experience of passing time. After learning, event representations in the anterior hippocampus reflected temporal relations based on constructed times. Temporal relations were generalized across sequences, revealing distinct representational formats for events from the same or different sequences. Structural knowledge about time patterns, abstracted from different sequences, biased the construction of specific event times. These findings demonstrate that mnemonic construction and the generalization of relational knowledge combine in the hippocampus, consistent with the simulation of scenarios from episodic details and structural knowledge.

Activity patterns in the hippocampus resemble temporal relations of learned event sequences. Here, the authors show that these relational memories arise through mnemonic construction and are generalized to reflect the temporal event structure.

Medial and orbital frontal cortex in decision-making and flexible behavior

The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.

Neural tuning and representational geometry

A central goal of neuroscience is to understand the representations formed by brain activity patterns and their connection to behaviour. The classic approach is to investigate how individual neurons encode stimuli and how their tuning determines the fidelity of the neural representation. Tuning analyses often use the Fisher information to characterize the sensitivity of neural responses to small changes of the stimulus. In recent decades, measurements of large populations of neurons have motivated a complementary approach, which focuses on the information available to linear decoders. The decodable information is captured by the geometry of the representational patterns in the multivariate response space. Here we review neural tuning and representational geometry with the goal of clarifying the relationship between them. The tuning induces the geometry, but different sets of tuned neurons can induce the same geometry. The geometry determines the Fisher information, the mutual information and the behavioural performance of an ideal observer in a range of psychophysical tasks. We argue that future studies can benefit from considering both tuning and geometry to understand neural codes and reveal the connections between stimuli, brain activity and behaviour.

Neural population geometry: An approach for understanding biological and artificial neural networks

Advances in experimental neuroscience have transformed our ability to explore the structure and function of neural circuits. At the same time, advances in machine learning have unleashed the remarkable computational power of artificial neural networks (ANNs). While these two fields have different tools and applications, they present a similar challenge: namely, understanding how information is embedded and processed through high-dimensional representations to solve complex tasks. One approach to addressing this challenge is to utilize mathematical and computational tools to analyze the geometry of these high-dimensional representations, i.e., neural population geometry. We review examples of geometrical approaches providing insight into the function of biological and artificial neural networks: representation untangling in perception, a geometric theory of classification capacity, disentanglement, and abstraction in cognitive systems, topological representations underlying cognitive maps, dynamic untangling in motor systems, and a dynamical approach to cognition. Together, these findings illustrate an exciting trend at the intersection of machine learning, neuroscience, and geometry, in which neural population geometry provides a useful population-level mechanistic descriptor underlying task implementation. Importantly, geometric descriptions are applicable across sensory modalities, brain regions, network architectures, and timescales. Thus, neural population geometry has the potential to unify our understanding of structure and function in biological and artificial neural networks, bridging the gap between single neurons, population activities, and behavior.

Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity

The ongoing exponential rise in recording capacity calls for new approaches for analysing and interpreting neural data. Effective dimensionality has emerged as an important property of neural activity across populations of neurons, yet different studies rely on different definitions and interpretations of this quantity. Here, we focus on intrinsic and embedding dimensionality, and discuss how they might reveal computational principles from data. Reviewing recent works, we propose that the intrinsic dimensionality reflects information about the latent variables encoded in collective activity while embedding dimensionality reveals the manner in which this information is processed. We conclude by highlighting the role of network models as an ideal substrate for testing more specifically various hypotheses on the computational principles reflected through intrinsic and embedding dimensionality.

Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

Neuronal identity and cognitive control dynamics in the PFC

Adaptive behavior is supported by context-dependent cognitive control that enables stable and flexible sensorimotor transformations. Impairments in this type of control are often attributed to dysfunction in the prefrontal cortex (PFC). However, the underlying circuit principles of PFC function that support cognitive control have remained elusive. While the complex, diverse responses of PFC neurons to cognitive variables have been studied both from the perspective of individual cell activity and overall population dynamics, these two levels have often been investigated separately. This review discusses two specific cell groups, context/brain state responsive interneuron subtypes and output decoder neurons, that might bridge conceptual frameworks derived from these two research approaches. I highlight the unique properties and functions of these cell groups and discuss how future studies leveraging their features are likely to provide a new understanding of PFC dynamics combining single-neuron and network perspectives.