Beyond simple reinforcement learning: the computational neurobiology of reward-learning and valuation

Association Between Aggression and Differential Functional Activity of Neural Regions Implicated in Retaliation

OBJECTIVE

The goal of the current study was to determine the extent to which atypical neural responsiveness during retaliation is associated with observed aggression in youth in residential care.

METHOD

This functional magnetic resonance imaging study involved 83 adolescents (56 male and 27 female; mean age, 16.18 years) in residential care performing a retaliation task. Of the 83 adolescents, 42 displayed aggressive behavior within the first 3 months of residential care, whereas 41 did not. During the retaliation task, participants were offered either fair or unfair divisions of $20 pots (allocation phase) and could either accept the offer or reject it, and, by spending $1, $2, or $3, punish the partner (retaliation phase).

RESULTS

The study's main findings were that aggressive adolescent showed the following: reduced down-regulation of activity within regions involved in representing the expected value of choice options (left ventromedial prefrontal cortex and left posterior cingulate cortex) as a function of offer unfairness and retaliation level; and reduced recruitment of regions implicated in response control (right inferior frontal gyrus and bilateral anterior insular cortex) and associated fronto-parietal regions as a function of retaliation level. The aggressive adolescents were also significantly more likely to have been aggressive prior to residential care and showed a strong trend for increased retaliation on the task.

CONCLUSION

We suggest that individuals with a greater propensity for aggression show reduced representation of the negative consequences of retaliation and associated reduced recruitment of regions potentially involved in over-ruling these negative consequences to engage in retaliation.

DIVERSITY & INCLUSION STATEMENT

We worked to ensure sex and gender balance in the recruitment of human participants. We worked to ensure that the study questionnaires were prepared in an inclusive way. We worked to ensure race, ethnic, and/or other types of diversity in the recruitment of human participants. We worked to ensure sex balance in the selection of non-human subjects. We actively worked to promote sex and gender balance in our author group. The author list of this paper includes contributors from the location and/or community where the research was conducted who participated in the data collection, design, analysis, and/or interpretation of the work. While citing references scientifically relevant for this work, we also actively worked to promote inclusion of historically underrepresented racial and/or ethnic groups in science in our reference list. While citing references scientifically relevant for this work, we also actively worked to promote sex and gender balance in our reference list. We actively worked to promote inclusion of historically underrepresented racial and/or ethnic groups in science in our author group.

The Situated Assessment Method (SAM2): Establishing individual differences in habitual behavior

From the perspectives of grounded, situated, and embodied cognition, we have developed a new approach for assessing individual differences. Because this approach is grounded in two dimensions of situatedness-situational experience and the Situated Action Cycle-we refer to it as the Situated Assessment Method (SAM2). Rather than abstracting over situations during assessment of a construct (as in traditional assessment instruments), SAM2 assesses a construct in situations where it occurs, simultaneously measuring factors from the Situated Action Cycle known to influence it. To demonstrate this framework, we developed the SAM2 Habitual Behavior Instrument (SAM2 HBI). Across three studies with a total of 442 participants, the SAM2 HBI produced a robust and replicable pattern of results at both the group and individual levels. Trait-level measures of habitual behavior exhibited large reliable individual differences in the regularity of performing positive versus negative habits. Situational assessments established large effects of situations and large situation by individual interactions. Several sources of evidence demonstrated construct and content validity for SAM2 measures of habitual behavior. At both the group and individual levels, these measures were associated with factors from the Situated Action Cycle known to influence habitual behavior in the literature (consistency, automaticity, immediate reward, long-term reward). Regressions explained approximately 65% of the variance at the group level and a median of approximately 75% at the individual level. SAM2 measures further exhibited well-established interactions with personality measures for self-control and neuroticism. Cognitive-affective processes from the Situated Action Cycle explained nearly all the variance in these interactions. Finally, a composite measure of habitualness established habitual behaviors at both the group and individual levels. Additionally, a composite measure of reward was positively related to the composite measure of habitualness, increasing with self-control and decreasing with neuroticism.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Challenges and Opportunities for Grounding Cognition

According to the grounded perspective, cognition emerges from the interaction of classic cognitive processes with the modalities, the body, and the environment. Rather than being an autonomous impenetrable module, cognition incorporates these other domains intrinsically into its operation. The Situated Action Cycle offers one way of understanding how the modalities, the body, and the environment become integrated to ground cognition. Seven challenges and opportunities are raised for this perspective: (1) How does cognition emerge from the Situated Action Cycle and in turn support it? (2) How can we move beyond simply equating embodiment with action, additionally establishing how embodiment arises in the autonomic, neuroendocrine, immune, cardiovascular, respiratory, digestive, and integumentary systems? (3) How can we better understand the mechanisms underlying multimodal simulation, its functions across the Situated Action Cycle, and its integration with other representational systems? (4) How can we develop and assess theoretical accounts of symbolic processing from the grounded perspective (perhaps using the construct of simulators)? (5) How can we move beyond the simplistic distinction between concrete and abstract concepts, instead addressing how concepts about the external and internal worlds pattern to support the Situated Action Cycle? (6) How do individual differences emerge from different populations of situational memories as the Situated Action Cycle manifests itself differently across individuals? (7) How can constructs from grounded cognition provide insight into the replication and generalization crises, perhaps from a quantum perspective on mechanisms (as exemplified by simulators).

Specific cortical and subcortical alterations for reactive and proactive aggression in children and adolescents with disruptive behavior

Maladaptive aggression, as present in conduct disorder (CD) and, to a lesser extent, oppositional defiant disorder (ODD), has been associated with structural alterations in various brain regions, such as ventromedial prefrontal cortex (vmPFC), anterior cingulate cortex (ACC), amygdala, insula and ventral striatum. Although aggression can be subdivided into reactive and proactive subtypes, no neuroimaging studies have yet investigated if any structural brain alterations are associated with either of the subtypes specifically. Here we investigated associations between aggression subtypes, CU traits and ADHD symptoms in predefined regions of interest.

T1-weighted magnetic resonance images were acquired from 158 children and adolescents with disruptive behavior (ODD/CD) and 96 controls in a multi-center study (aged 8–18). Aggression subtypes were assessed by questionnaires filled in by participants and their parents. Cortical volume and subcortical volumes and shape were determined using Freesurfer and the FMRIB integrated registration and segmentation tool. Associations between volumes and continuous measures of aggression were established using multilevel linear mixed effects models.

Proactive aggression was negatively associated with amygdala volume (b = -10.7, p = 0.02), while reactive aggression was negatively associated with insula volume (b = -21.7, p = 0.01). No associations were found with CU traits or ADHD symptomatology. Classical group comparison showed that children and adolescents with disruptive behavior had smaller volumes than controls in (bilateral) vmPFC (p = 0.003) with modest effect size and a reduced shape in the anterior part of the left ventral striatum (p = 0.005).

Our study showed negative associations between reactive aggression and volumes in a region involved in threat responsivity and between proactive aggression and a region linked to empathy. This provides evidence for aggression subtype-specific alterations in brain structure which may provide useful insights for clinical practice.

Traits of empathy and anger: implications for psychopathy and other disorders associated with aggression

Empathy and anger are two social emotions that modulate an individual's risk for aggression. Empathy is an emotional reaction to another individual's emotional state. Anger is an emotional reaction to threat, frustration or social provocation. Reduced empathy, seen in psychopathy, increases the risk for goal-directed aggression. Atypically increased anger (i.e. irritability), seen in conditions like disruptive mood dysregulation disorder and borderline personality disorder, increases the risk for reactive aggression. In this paper, I will outline core neurocognitive functions that correspond to empathy and which are compromised in individuals with psychopathic traits. In addition, I will outline neurocognitive functions involved in either the generation or regulation of anger and which are compromised in psychiatric conditions at increased risk for irritability/reactive aggression. It can be hoped that improved understanding of empathy and anger will lead to better assessment tools and improved interventions to reduce aggression risk.This article is part of the theme issue 'Diverse perspectives on diversity: multi-disciplinary approaches to taxonomies of individual differences'.

Active Confirmation Bias in the Evaluative Processing of Food Images

Predictive processing is fundamental to many aspects of the human mind, including perception and decision-making. It remains to be elucidated, however, in which way predictive information impacts on evaluative processing, particularly in tasks that employ bivalent stimulus sets. Various accounts, including framing, proactive interference, and cognitive control, appear to imply contradictory proposals on the relation between prediction and preference formation. To disambiguate whether predictive cues produce congruent biases versus opponent mechanisms in evaluative processing, we conducted two experiments in which participants were asked to rate individual food images. The image database included appetitive and aversive items. In each trial, a cue predicted, with varying degrees of reliability, the valence of the impending food image. In both experiments, we found that the ratings exhibited congruent biases as a function of the reliability of the predictive cue, with the highest evaluations following the most reliable positive-valence predictions. Eye prepositioning further showed a selective spatial bias suggestive of response preparation in line with the predictions. The response times also exhibited a pattern of results consistent with selective preparation, producing slow responses following invalid predictions. The data suggested an active form of evaluative processing, implementing a confirmation bias that aims to accommodate the prediction.

Emotion-based learning systems and the development of morality

In this paper it is proposed that important components of moral development and moral judgment rely on two forms of emotional learning: stimulus-reinforcement and response-outcome learning. Data in support of this position will be primarily drawn from work with individuals with the developmental condition of psychopathy as well as fMRI studies with healthy individuals. Individuals with psychopathy show impairment on moral judgment tasks and a pronounced increased risk for instrumental antisocial behavior. It will be argued that these impairments are developmental consequences of impaired stimulus-aversive conditioning on the basis of distress cue reinforcers and response-outcome learning in individuals with this disorder.

Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments

Little is known about the relationship between attention and learning during decision making. Using eye tracking and multivariate pattern analysis of fMRI data, we measured participants' dimensional attention as they performed a trial-and-error learning task in which only one of three stimulus dimensions was relevant for reward at any given time. Analysis of participants' choices revealed that attention biased both value computation during choice and value update during learning. Value signals in the ventromedial prefrontal cortex and prediction errors in the striatum were similarly biased by attention. In turn, participants' focus of attention was dynamically modulated by ongoing learning. Attentional switches across dimensions correlated with activity in a frontoparietal attention network, which showed enhanced connectivity with the ventromedial prefrontal cortex between switches. Our results suggest a bidirectional interaction between attention and learning: attention constrains learning to relevant dimensions of the environment, while we learn what to attend to via trial and error.

Reminiscing about positive memories buffers acute stress responses

Recalling happy memories elicits positive feelings and enhances one's wellbeing, suggesting a potential adaptive function in using this strategy for coping with stress. In two studies, we explored whether recalling autobiographical memories that have a positive content - i.e., remembering the good times - can dampen the hypothalamic-pituitary-adrenal (HPA) axis stress response. Participants underwent an acute stressor or control task followed by autobiographical memory recollection (of only positive or neutral valence). Across both studies, recalling positive, but not neutral, memories resulted in a dampened cortisol rise and reduced negative affect. Further, individuals with greater self-reported resiliency showed enhanced mood, despite stress exposure. During positive reminiscence, we observed engagement of corticostriatal circuits previously implicated in reward-processing and emotion regulation, and stronger connectivity between ventrolateral and dorsolateral prefrontal cortex as a function of positivity. These findings highlight the restorative and protective function of self-generated positive emotions via memory recall in the face of stress.

Abnormal decision-making in generalized anxiety disorder: Aversion of risk or stimulus-reinforcement impairment?

There is preliminary data indicating that patients with generalized anxiety disorder (GAD) show impairment on decision-making tasks requiring the appropriate representation of reinforcement value. The current study aimed to extend this literature using the passive avoidance (PA) learning task, where the participant has to learn to respond to stimuli that engender reward and avoid responding to stimuli that engender punishment. Six stimuli engendering reward and six engendering punishment are presented once per block for 10 blocks of trials. Thirty-nine medication-free patients with GAD and 29 age-, IQ and gender matched healthy comparison individuals performed the task. In addition, indexes of social functioning as assessed by the Global Assessment of Functioning (GAF) scale were obtained to allow for correlational analyzes of potential relations between cognitive and social impairments. The results revealed a Group-by-Error Type-by-Block interaction; patients with GAD committed significantly more commission (passive avoidance) errors than comparison individuals in the later blocks (blocks 7,8, and 9). In addition, the extent of impairment on these blocks was associated with their functional impairment as measured by the GAF scale. These results link GAD with anomalous decision-making and indicate that a potential problem in reinforcement representation may contribute to the severity of expression of their disorder.

The Neurobiology of Impulsive Aggression

This selective review provides a model of the neurobiology of impulsive aggression from a cognitive neuroscience perspective. It is argued that prototypical cases of impulsive aggression, those associated with anger, involve the recruitment of the acute threat response system structures; that is, the amygdala, hypothalamus, and periaqueductal gray. It is argued that whether the recruitment of these structures results in impulsive aggression or not reflects the functional roles of ventromedial frontal cortex and dorsomedial frontal and anterior insula cortex in response selection. It is also argued that impulsive aggression may occur because of impaired decision making. The aggression may not be accompanied by anger, but it will reflect disrupted evaluation of the rewards/benefits of the action.

A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker

Rate distortion theory describes how to communicate relevant information most efficiently over a channel with limited capacity. One of the many applications of rate distortion theory is bounded rational decision making, where decision makers are modeled as information channels that transform sensory input into motor output under the constraint that their channel capacity is limited. Such a bounded rational decision maker can be thought to optimize an objective function that trades off the decision maker's utility or cumulative reward against the information processing cost measured by the mutual information between sensory input and motor output. In this study, we interpret a spiking neuron as a bounded rational decision maker that aims to maximize its expected reward under the computational constraint that the mutual information between the neuron's input and output is upper bounded. This abstract computational constraint translates into a penalization of the deviation between the neuron's instantaneous and average firing behavior. We derive a synaptic weight update rule for such a rate distortion optimizing neuron and show in simulations that the neuron efficiently extracts reward-relevant information from the input by trading off its synaptic strengths against the collected reward.

Conduct disorder and callous-unemotional traits in youth

The term “conduct problems” refers to a pattern of repetitive rule-breaking behavior, aggression, and disregard for others. Such problems have received increased attention recently, owing to violent events perpetrated by youth and modifications in the Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria for conduct disorder, a syndrome involving recurrent conduct problems in children and adolescents. Youth conduct problems are predictive of an increased risk of substance abuse, criminal behavior, and educational disruption; they also incur a considerable societal burden from interpersonal suffering and financial costs. This review summarizes current data on youth conduct problems and highlights promising avenues for research. Prior reviews have summarized either the clinical literature on outcome, treatment, and familial aggregation or the neurocognitive literature on mechanisms and pathophysiology.^– The current review differs by more tightly integrating clinical and neurocognitive perspectives.

Savoring the past: positive memories evoke value representations in the striatum

Reminders of happy memories can bring back pleasant feelings tied to the original experience, suggesting an intrinsic value in reminiscing about the positive past. However, the neural circuitry underlying the rewarding aspects of autobiographical memory is poorly understood. Using fMRI, we observed enhanced activity during the recall of positive relative to neutral autobiographical memories in corticostriatal circuits that also responded to monetary reward. Enhanced activity in the striatum and medial prefrontal cortex was associated with increases in positive emotion during recall, and striatal engagement further correlated with individual measures of resiliency. Striatal response to the recall of positive memories was greater in individuals whose mood improved after the task. Notably, participants were willing to sacrifice a more tangible reward, money, in order to reminisce about positive past experiences. Our findings suggest that recalling positive autobiographical memories is intrinsically valuable, which may be adaptive for regulating positive emotion and promoting better well-being.

Psychopathic traits from an RDoC perspective

In this review an RDoC approach is applied to psychopathic traits. Two core neuro-cognitive systems relevant to the emergence of psychopathic traits are considered. These are the response to other individuals' emotional displays and reinforcement-based decision-making.

Psychopathy: cognitive and neural dysfunction

Psychopathy is a developmental disorder marked by emotional deficits and an increased risk for antisocial behavior. It is not equivalent to the diagnosis Antisocial Personality Disorder, which concentrates only on the increased risk for antisocial behavior and not a specific cause—ie, the reduced empathy and guilt that constitutes the emotional deficit. The current review considers data from adults with psychopathy with respect to the main cognitive accounts of the disorder that stress either a primary attention deficit or a primary emotion deficit. In addition, the current review considers data regarding the neurobiology of this disorder. Dysfunction within the amygdala's role in reinforcement learning and the role of ventromedial frontal cortex in the representation of reinforcement value is stressed. Data is also presented indicating potential difficulties within parts of temporal and posterior cingulate cortex. Suggestions are made with respect to why these deficits lead to the development of the disorder.

Deep and beautiful. The reward prediction error hypothesis of dopamine

According to the reward-prediction error hypothesis (RPEH) of dopamine, the phasic activity of dopaminergic neurons in the midbrain signals a discrepancy between the predicted and currently experienced reward of a particular event. It can be claimed that this hypothesis is deep, elegant and beautiful, representing one of the largest successes of computational neuroscience. This paper examines this claim, making two contributions to existing literature. First, it draws a comprehensive historical account of the main steps that led to the formulation and subsequent success of the RPEH. Second, in light of this historical account, it explains in which sense the RPEH is explanatory and under which conditions it can be justifiably deemed deeper than the incentive salience hypothesis of dopamine, which is arguably the most prominent contemporary alternative to the RPEH.

How self-determined choice facilitates performance: a key role of the ventromedial prefrontal cortex

Recent studies have documented that self-determined choice does indeed enhance performance. However, the precise neural mechanisms underlying this effect are not well understood. We examined the neural correlates of the facilitative effects of self-determined choice using functional magnetic resonance imaging (fMRI). Participants played a game-like task involving a stopwatch with either a stopwatch they selected (self-determined-choice condition) or one they were assigned without choice (forced-choice condition). Our results showed that self-determined choice enhanced performance on the stopwatch task, despite the fact that the choices were clearly irrelevant to task difficulty. Neuroimaging results showed that failure feedback, compared with success feedback, elicited a drop in the vmPFC activation in the forced-choice condition, but not in the self-determined-choice condition, indicating that negative reward value associated with the failure feedback vanished in the self-determined-choice condition. Moreover, the vmPFC resilience to failure in the self-determined-choice condition was significantly correlated with the increased performance. Striatal responses to failure and success feedback were not modulated by the choice condition, indicating the dissociation between the vmPFC and striatal activation pattern. These findings suggest that the vmPFC plays a unique and critical role in the facilitative effects of self-determined choice on performance.

The neurobiology of psychopathic traits in youths

Conduct disorder is a childhood behaviour disorder that is characterized by persistent aggressive or antisocial behaviour that disrupts the child's environment and impairs his or her functioning. A proportion of children with conduct disorder have psychopathic traits. Psychopathic traits consist of a callous-unemotional component and an impulsive-antisocial component, which are associated with two core impairments. The first is a reduced empathic response to the distress of other individuals, which primarily reflects reduced amygdala responsiveness to distress cues; the second is deficits in decision making and in reinforcement learning, which reflects dysfunction in the ventromedial prefrontal cortex and striatum. Genetic and prenatal factors contribute to the abnormal development of these neural systems, and social-environmental variables that affect motivation influence the probability that antisocial behaviour will be subsequently displayed.

The mixed instrumental controller: using value of information to combine habitual choice and mental simulation

Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available "cached" value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated "Value of Information" exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus - ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation.

Neurons in the cingulate motor area signal context-based and outcome-based volitional selection of action

Volitional selection of action is subject to continuous adjustment under the influence of information obtained by monitoring behavioral consequences and by exploiting behavioral context based on prior knowledge about the environment. The rostral cingulate motor area (CMAr) is thought to be responsible for adjusting behavior by monitoring its consequences. To investigate whether the CMAr also plays a role in exploitation of behavioral context in action selection, we recorded neuronal activities from the CMAr while monkeys performed a reward-based motor selection task that required them to switch from one action to the other based on the amount of reward. We examined both the behavior of monkeys and the activity of CMA neurons quantitatively by constructing a hybrid reinforcement learning model incorporating context-based and outcome-based action values into a new action value. We found that CMAr neurons encoded the context-based action value by increasing or decreasing their firing rates gradually with the number of repetitions of the same movement (i.e., behavioral context). We also found that CMAr neurons encoded the context-based and outcome-based action values in two distinct time windows at single neuron and population levels. Our findings indicate that the CMAr is involved in behavioral adjustment of action selection by exploiting the behavioral context and not merely by monitoring reward outcome.

Activity in inferior parietal and medial prefrontal cortex signals the accumulation of evidence in a probability learning task

In an uncertain environment, probabilities are key to predicting future events and making adaptive choices. However, little is known about how humans learn such probabilities and where and how they are encoded in the brain, especially when they concern more than two outcomes. During functional magnetic resonance imaging (fMRI), young adults learned the probabilities of uncertain stimuli through repetitive sampling. Stimuli represented payoffs and participants had to predict their occurrence to maximize their earnings. Choices indicated loss and risk aversion but unbiased estimation of probabilities. BOLD response in medial prefrontal cortex and angular gyri increased linearly with the probability of the currently observed stimulus, untainted by its value. Connectivity analyses during rest and task revealed that these regions belonged to the default mode network. The activation of past outcomes in memory is evoked as a possible mechanism to explain the engagement of the default mode network in probability learning. A BOLD response relating to value was detected only at decision time, mainly in striatum. It is concluded that activity in inferior parietal and medial prefrontal cortex reflects the amount of evidence accumulated in favor of competing and uncertain outcomes.

Don'T let me do that! - models of precommitment

Precommitment, or taking away a future choice from oneself, is a mechanism for overcoming impulsivity. Here we review recent work suggesting that precommitment can be best explained through a distributed decision-making system with multiple discounting rates. This model makes specific predictions about precommitment behavior and is especially interesting in light of the emerging multiple-systems view of decision-making, in which functional systems with distinct neural substrates use different computational strategies to optimize decisions. Given the growing consensus that impulsivity constitutes a common point of breakdown in decision-making processes, with common neural and computational mechanisms across multiple psychiatric disorders, it is useful to translate precommitment into the common language of temporal difference reinforcement learning that unites many of these behavioral and neural data.