An exploration of error-driven learning in simple two-layer networks from a discriminative learning perspective

Error-driven learning algorithms, which iteratively adjust expectations based on prediction error, are the basis for a vast array of computational models in the brain and cognitive sciences that often differ widely in their precise form and application: they range from simple models in psychology and cybernetics to current complex deep learning models dominating discussions in machine learning and artificial intelligence. However, despite the ubiquity of this mechanism, detailed analyses of its basic workings uninfluenced by existing theories or specific research goals are rare in the literature. To address this, we present an exposition of error-driven learning - focusing on its simplest form for clarity - and relate this to the historical development of error-driven learning models in the cognitive sciences. Although historically error-driven models have been thought of as associative, such that learning is thought to combine preexisting elemental representations, our analysis will highlight the discriminative nature of learning in these models and the implications of this for the way how learning is conceptualized. We complement our theoretical introduction to error-driven learning with a practical guide to the application of simple error-driven learning models in which we discuss a number of example simulations, that are also presented in detail in an accompanying tutorial.

Topographical Visualization of the Reciprocal Projection between the Medial Septum and the Hippocampus in the 5XFAD Mouse Model of Alzheimer's Disease

It is widely known that the degeneration of neural circuits is prominent in the brains of Alzheimer’s disease (AD) patients. The reciprocal connectivity of the medial septum (MS) and hippocampus, which constitutes the septo-hippocampo-septal (SHS) loop, is known to be associated with learning and memory. Despite the importance of the reciprocal projections between the MS and hippocampus in AD, the alteration of bidirectional connectivity between two structures has not yet been investigated at the mesoscale level. In this study, we adopted AD animal model, five familial AD mutations (5XFAD) mice, and anterograde and retrograde tracers, BDA and DiI, respectively, to visualize the pathology-related changes in topographical connectivity of the SHS loop in the 5XFAD brain. By comparing 4.5-month-old and 14-month-old 5XFAD mice, we successfully identified key circuit components of the SHS loop altered in 5XFAD brains. Remarkably, the SHS loop began to degenerate in 4.5-month-old 5XFAD mice before the onset of neuronal loss. The impairment of connectivity between the MS and hippocampus was accelerated in 14-month-old 5XFAD mice. These results demonstrate, for the first time, topographical evidence for the degradation of the interconnection between the MS and hippocampus at the mesoscale level in a mouse model of AD. Our results provide structural and functional insights into the interconnectivity of the MS and hippocampus, which will inform the use and development of various therapeutic approaches that target neural circuits for the treatment of AD.

Cognitive appraisal of environmental stimuli induces emotion-like states in fish

The occurrence of emotions in non-human animals has been the focus of debate over the years. Recently, an interest in expanding this debate to non-tetrapod vertebrates and to invertebrates has emerged. Within vertebrates, the study of emotion in teleosts is particularly interesting since they represent a divergent evolutionary radiation from that of tetrapods, and thus they provide an insight into the evolution of the biological mechanisms of emotion. We report that Sea Bream exposed to stimuli that vary according to valence (positive, negative) and salience (predictable, unpredictable) exhibit different behavioural, physiological and neuromolecular states. Since according to the dimensional theory of emotion valence and salience define a two-dimensional affective space, our data can be interpreted as evidence for the occurrence of distinctive affective states in fish corresponding to each the four quadrants of the core affective space. Moreover, the fact that the same stimuli presented in a predictable vs. unpredictable way elicited different behavioural, physiological and neuromolecular states, suggests that stimulus appraisal by the individual, rather than an intrinsic characteristic of the stimulus, has triggered the observed responses. Therefore, our data supports the occurrence of emotion-like states in fish that are regulated by the individual's perception of environmental stimuli.

Developmental Changes in Hippocampal CA1 Single Neuron Firing and Theta Activity during Associative Learning

Hippocampal development is thought to play a crucial role in the emergence of many forms of learning and memory, but ontogenetic changes in hippocampal activity during learning have not been examined thoroughly. We examined the ontogeny of hippocampal function by recording theta and single neuron activity from the dorsal hippocampal CA1 area while rat pups were trained in associative learning. Three different age groups [postnatal days (P)17-19, P21-23, and P24-26] were trained over six sessions using a tone conditioned stimulus (CS) and a periorbital stimulation unconditioned stimulus (US). Learning increased as a function of age, with the P21-23 and P24-26 groups learning faster than the P17-19 group. Age- and learning-related changes in both theta and single neuron activity were observed. CA1 pyramidal cells in the older age groups showed greater task-related activity than the P17-19 group during CS-US paired sessions. The proportion of trials with a significant theta (4-10 Hz) power change, the theta/delta ratio, and theta peak frequency also increased in an age-dependent manner. Finally, spike/theta phase-locking during the CS showed an age-related increase. The findings indicate substantial developmental changes in dorsal hippocampal function that may play a role in the ontogeny of learning and memory.

Ontogeny of septohippocampal modulation of delay eyeblink conditioning

The current study investigated the effects of disrupting the septohippocampal theta system on the developmental emergence of delay eyeblink conditioning. Theta oscillations are defined as electroencephalographic (EEG) waveforms with a frequency between 3-8 Hz. Hippocampal theta oscillations are generated by inputs from the entorhinal cortex and the medial septum. Theta activity has been shown to facilitate learning in a variety of paradigms, including delay eyeblink conditioning. Lesions of the medial septum disrupt theta activity and slow the rate at which delay eyeblink conditioning is learned (Berry & Thompson, [1979] Science 200:1298-1300). The role of the septohippocampal theta system in the ontogeny of eyeblink conditioning has not been examined. In the current study, infant rats received an electrolytic lesion of the medial septum on postnatal day (P) 12. Rats were later given eyeblink conditioning for 6 sessions with an auditory conditioned stimulus on P17-19, P21-23, or P24-26. Lesions impaired eyeblink conditioning on P21-23 and P24-26 but not on P17-19. The results suggest that the septohippocampal system comes online to facilitate acquisition of eyeblink conditioning between P19 and P21. Developmental changes in septohippocampal modulation of the cerebellum may play a significant role in the ontogeny of eyeblink conditioning.

The expression of contextual fear conditioning involves activation of a NMDA receptor-nitric oxide-cGMP pathway in the dorsal hippocampus of rats

The dorsal portion of the hippocampus is a limbic structure that is involved in fear conditioning modulation in rats. Moreover, evidence shows that the local dorsal hippocampus glutamatergic system, nitric oxide (NO) and cGMP modulate behavioral responses during aversive situations. Therefore, the present study investigated the involvement of dorsal hippocampus NMDA receptors and the NO/cGMP pathway in contextual fear conditioning expression. Male Wistar rats were submitted to an aversive contextual conditioning session and 48 h later they were re-exposed to the aversive context in which freezing, cardiovascular responses (increase of both arterial pressure and heart rate) and decrease of tail temperature were recorded. The intra-dorsal hippocampus administration of the NMDA receptor antagonist AP7, prior to the re-exposure to the aversive context, attenuated fear-conditioned responses. The re-exposure to the context evoked an increase in NO concentration in the dorsal hippocampus of conditioned animals. Similar to AP7 administration, we observed a reduction of contextual fear conditioning after dorsal hippocampus administration of either the neuronal NO synthase inhibitor N-propyl-L-arginine, the NO scavenger c-PTIO or the guanylate cyclase inhibitor ODQ. Therefore, the present findings suggest the possible existence of a dorsal hippocampus NMDA/NO/cGMP pathway modulating the expression of contextual fear conditioning in rats.

The GABAergic septohippocampal pathway is directly involved in internal processes related to operant reward learning

We studied the role of γ-aminobutyric acid (GABA)ergic septohippocampal projections in medial septum (MS) self-stimulation of behaving mice. Self-stimulation was evoked in wild-type (WT) mice using instrumental conditioning procedures and in J20 mutant mice, a type of mouse with a significant deficit in GABAergic septohippocampal projections. J20 mice showed a significant modification in hippocampal activities, including a different response for input/output curves and the paired-pulse test, a larger long-term potentiation (LTP), and a delayed acquisition and lower performance in the MS self-stimulation task. LTP evoked at the CA3-CA1 synapse further decreased self-stimulation performance in J20, but not in WT, mice. MS self-stimulation evoked a decrease in the amplitude of field excitatory postsynaptic potentials (fEPSPs) at the CA3-CA1 synapse in WT, but not in J20, mice. This self-stimulation-dependent decrease in the amplitude of fEPSPs was also observed in the presence of another positive reinforcer (food collected during an operant task) and was canceled by the local administration of an antibody-inhibiting glutamate decarboxylase 65 (GAD65). LTP evoked in the GAD65Ab-treated group was also larger than in controls. The hippocampus has a different susceptibility to septal GABAergic inputs depending on ongoing cognitive processes, and the GABAergic septohippocampal pathway is involved in consummatory processes related to operant rewards.

Beyond novelty detection: incongruent events, when general and specific classifiers disagree

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of- Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.

A neural model of hippocampal-striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients

Building on our previous neurocomputational models of basal ganglia and hippocampal region function (and their modulation by dopamine and acetylcholine, respectively), we show here how an integration of these models can inform our understanding of the interaction between the basal ganglia and hippocampal region in associative learning and transfer generalization across various patient populations. As a common test bed for exploring interactions between these brain regions and neuromodulators, we focus on the acquired equivalence task, an associative learning paradigm in which stimuli that have been associated with the same outcome acquire a functional similarity such that subsequent generalization between these stimuli increases. This task has been used to test cognitive dysfunction in various patient populations with damages to the hippocampal region and basal ganglia, including studies of patients with Parkinson's disease (PD), schizophrenia, basal forebrain amnesia, and hippocampal atrophy. Simulation results show that damage to the hippocampal region-as in patients with hippocampal atrophy (HA), hypoxia, mild Alzheimer's (AD), or schizophrenia-leads to intact associative learning but impaired transfer generalization performance. Moreover, the model demonstrates how PD and anterior communicating artery (ACoA) aneurysm-two very different brain disorders that affect different neural mechanisms-can have similar effects on acquired equivalence performance. In particular, the model shows that simulating a loss of dopamine function in the basal ganglia module (as in PD) leads to slow acquisition learning but intact transfer generalization. Similarly, the model shows that simulating the loss of acetylcholine in the hippocampal region (as in ACoA aneurysm) also results in slower acquisition learning. We argue from this that changes in associative learning of stimulus-action pathways (in the basal ganglia) or changes in the learning of stimulus representations (in the hippocampal region) can have similar functional effects.

Enhanced c-Fos expression in superior colliculus, paraventricular thalamus and septum during learning of cue-reward association

Reward-mediated associative learning is important for recognizing the significance of environmental cues. Such learning involves convergence of multimodal sensory inputs with circuits involved in affective and memory processes. Dopamine-dependent plasticity in the striatum plays a pivotal role, but the wider circuits engaged in cue-reward association are poorly understood. To identify candidate structures that may be of particular interest for further detailed electrophysiological and functional analysis, we quantified c-Fos expression in a selection of brain structures. c-Fos is a well-known marker of cell activation with additional potential importance for synaptic plasticity. We compared c-Fos expression between animals exposed to 100 pairings of a novel conditioned stimulus with a subsequent reward, and control animals exposed to the same number of cues and rewards, but where the cues and rewards occurred at random with respect to each other. We found significant increases in c-Fos expression in the superior colliculus in the group exposed to cue-reward pairing. This is consistent with previous recordings in conscious animals, showing modulation of phasic visual responses of single collicular neurons depending on their association with reward. Further, the data also suggest the possibility that the thalamic paraventricular nucleus and septal nuclei may be selectively activated during cue-reward association learning. Little is known of the neurophysiological responses in these structures during such tasks, so the present results suggest they would be targets of interest for future single-neuron recording experiments, designed to confirm whether the neurons show learning-specific modulation.

Histaminergic receptors of medial septum and conditioned place preference: D1 dopamine receptor mechanism

In the present study, the effects of intra-medial septum injections of histamine and/or the histamine H1 or H2 receptor antagonists on the acquisition of conditioned place preference (CPP) in male Wistar rats have been investigated. Our data showed that the conditioning treatments with intra-medial septum injection of different doses of histamine (0.5-15 microg/rat) induced a significant CPP for the drug-associated place. Using a 3-day schedule of conditioning, it was found that the histamine H1 receptor antagonist, pyrilamine (10 and 15 microg/rat, intra-medial septum) also induced a significant place preference. In addition, pyrilamine inhibited the histamine (7.5 microg/rat)-induced place preference. Intra-medial septum administration of the histamine H2 receptor antagonist, ranitidine (5-15 microg/rat) alone or in combination with histamine did not produce a significant place preference or place aversion. On the other hand, intra-medial septum administration of the dopamine D1 receptor antagonist, SCH 233390 (0.5, 0.75 and 1 microg/rat) inhibited the histamine (7.5 microg/rat) or pyrilamine (15 microg/rat)-induced place preference in a dose-dependent manner, but no effect was observed for the dopamine D2 receptor antagonist, sulpiride on the histamine or pyrilamine response. The administration of histamine (2.5-15 microg/rat) or pyrilamine (10 and 15 microg/rat) during acquisition increased locomotor activity of the animals on the testing days. The results suggest that histaminergic receptors of the medial septum may be involved in CPP and thus it is postulated that dopamine D1 receptors may play an important role in this effect.

Conditional discrimination and reversal in amnesia subsequent to hypoxic brain injury or anterior communicating artery aneurysm rupture

Human anterograde amnesia can develop following bilateral damage to the hippocampus and medial temporal lobes, as in hypoxic brain injury, or following damage to the basal forebrain, as following anterior communicating artery (ACoA) aneurysm rupture. In both cases, the mnestic deficit may be similar when assessed by standard neuropsychological measures. However, animal and computational models suggest that there are qualitative differences in the pattern of impaired and spared memory abilities following damage to hippocampus versus basal forebrain. Here, we show such a dissociation in human amnesia using a single two-stage task, involving conditional discrimination and reversal. Consistent with a prior study, 10 individuals with anterograde amnesia subsequent to hypoxic brain injury were spared on acquisition but impaired at reversal. However, 10 individuals with amnesia subsequent to ACoA aneurysm showed the opposite pattern of impaired acquisition but spared reversal. The differences between groups cannot be easily ascribed to severity of mnestic or cognitive deficit, since the two amnesic groups performed similarly on neuropsychological tests of memory, intelligence and attention. The results illustrate qualitative differences in memory impairments in hypoxic and ACoA amnesics and highlight the importance of considering etiology in evaluating mnemonic deficits in amnesic populations.

Effects of Pavlovian fear conditioning on septohippocampal metabolism in rats

The effects of classical fear conditioning in different regions of the limbic system were analysed using cytochrome oxidase (CO) histochemistry. Wistar rats were submitted to different conditions. Rats in the group Paired received tone-shock pairing, to elicit conditioned suppression of lever pressing (i.e., tone will evoke conditioned fear responses). The group Unpaired underwent random presentations of these stimuli and developed no conditioned fear. Untrained animals were also included as a control group. A significant decrease in CO activity was found in the medial septal area and the dorsal hippocampus (CA3 subfield and dentate gyrus) in the group Paired as compared with the group Unpaired. Furthermore there was greater metabolic activity in the control group as compared with the other two groups. No differences in CO labelling of the basolateral amygdala were detected among all groups. These findings suggest that the septohippocampal system plays an important role in controlling conditioned fear behaviour.

Integrating Incremental Learning and Episodic Memory Models of the Hippocampal Region

By integrating previous computational models of corticohippocampal function, the authors develop and test a unified theory of the neural substrates of familiarity, recollection, and classical conditioning. This approach integrates models from 2 traditions of hippocampal modeling, those of episodic memory and incremental learning, by drawing on an earlier mathematical model of conditioning, SOP (A. Wagner, 1981). The model describes how a familiarity signal may arise from parahippocampal cortices, giving a novel explanation for the finding that the neural response to a stimulus in these regions decreases with increasing stimulus familiarity. Recollection is ascribed to the hippocampus proper. It is shown how the properties of episodic representations in the neocortex, parahippocampal gyrus, and hippocampus proper may explain phenomena in classical conditioning. The model reproduces the effects of hippocampal, septal, and broad hippocampal region lesions on contextual modulation of classical conditioning, blocking, learned irrelevance, and latent inhibition.

Effects of Medial Septal Lesions on Action-Outcome Associations in Rats Under Conditions of Delayed Reinforcement

In operant tasks, control rats maintain high response rates under positive contingencies, when the probability of reinforcement is greater following a response (contingent reinforcement) than during the absence of that response. However, as contingencies approach zero, response rates decrease. In this experiment, under immediate contingent reinforcement, rats with medial septal lesions reduced their response rates, just like controls, when contingencies were shifted from positive toward zero. However, the septal rats were less sensitive to this contingency shift, compared with controls, when there was a 5-s delay between lever presses and contingent reinforcements. This lesion effect appeared to be due to a failure of voluntary response memory, which impaired sensitivity to operant contingencies when there was a delay between action and outcome.