Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory?

Multifunctional In-Memory Logics Based on a Dual-Gate Antiambipolar Transistor toward Non-von Neumann Computing Architecture

In-memory computing may make it possible to realize non-von Neumann computing because the logic circuits are unified in the memory units. We investigated two types of in-memory logic operations, namely, two-input logic circuits and multifunctional artificial synapses. These were realized in a dual-gate antiambipolar transistor (AAT) with a ReS2/WSe2 heterojunction, in which polystyrene with a zinc phthalocyanine core (ZnPc-PS4) was incorporated as a memory layer. Here, reconfigurability is a key concept for both types of device operations and was achieved by merging the Λ-shaped transfer curve of the AAT and the nonvolatile memory effect of ZnPc-PS4. First, we achieved electrically reconfigurable two-input logic circuits. Versatile logic circuits such as AND, OR, NAND, NOR, and XOR circuits were demonstrated by taking advantage of the Λ-shaped transfer curve of the dual-gate AAT. Importantly, the nonvolatile memory function provided the electrical switching of the individual circuits between AND/OR, NAND/NOR, and XOR/NAND circuits with constant input signals. Second, the memory effect was applied to multifunctional artificial synapses. The inhibitory/excitatory and long-term potentiation/depression synaptic operations were electrically reconfigured simply by controlling one parameter (readout voltage), making three distinct responses possible even with the same presynaptic signals. These findings provide hints that may lead to the realization of new in-memory computing architectures beyond the current von Neumann computers.

Computational Study on Interlocked-Ferroelectricity-Contributed High-Performance Memristors Based on Two-Dimensional van der Waals Ferroelectric Semiconductors

In order to realize the prevailing artificial intelligence technology, memristor-implemented in-memory or neuromorphic computing is highly expected to break the bottleneck of von Neumann computers. Although high-performance memristors have been vigorously developed in labs or in industry, systematic computational investigations on memristors are seldom. Hence, it is urgent to provide theoretical or computational support for the exploration of memristor operating mechanisms or the screening of memristor materials. Here, a computational method based on the main input parameters learned from the first-principles calculations was developed to measure resistance switching of two-terminal memristors with sandwiched metal/ferroelectric semiconductor/metal architectures, which strikingly agrees with the experimental measurements. Based on our developed method, the diverse multiterminal memristors were designed to fully exploit the application of interlocked ferroelectricity of a ferroelectric semiconductor and realize their heterosynaptic plasticity, and their heterosynaptic behaviors can still be well described. Our developed method can provide a paradigm for the emulation of ferroelectric memristors and inspire subsequent computational exploration. Furthermore, our study also supplies a device optimization strategy based on the interlocked ferroelectricity and easy processing of two-dimensional van der Waals ferroelectric semiconductors, and our proposed heterosynaptic memristors still await further experimental exploration.

Plasticity of Response Properties of Mouse Visual Cortex Neurons Induced by Optogenetic Tetanization In Vivo

Heterosynaptic plasticity, along with Hebbian homosynaptic plasticity, is an important mechanism ensuring the stable operation of learning neuronal networks. However, whether heterosynaptic plasticity occurs in the whole brain in vivo, and what role(s) in brain function in vivo it could play, remains unclear. Here, we used an optogenetics approach to apply a model of intracellular tetanization, which was established and employed to study heterosynaptic plasticity in brain slices, to study the plasticity of response properties of neurons in the mouse visual cortex in vivo. We show that optogenetically evoked high-frequency bursts of action potentials (optogenetic tetanization) in the principal neurons of the visual cortex induce long-term changes in the responses to visual stimuli. Optogenetic tetanization had distinct effects on responses to different stimuli, as follows: responses to optimal and orthogonal orientations decreased, responses to null direction did not change, and responses to oblique orientations increased. As a result, direction selectivity of the neurons decreased and orientation tuning became broader. Since optogenetic tetanization was a postsynaptic protocol, applied in the absence of sensory stimulation, and, thus, without association of presynaptic activity with bursts of action potentials, the observed changes were mediated by mechanisms of heterosynaptic plasticity. We conclude that heterosynaptic plasticity can be induced in vivo and propose that it may play important homeostatic roles in operation of neural networks by helping to prevent runaway dynamics of responses to visual stimuli and to keep the tuning of neuronal responses within the range optimized for the encoding of multiple features in population activity.

Ionic Diffusive Nanomemristors with Dendritic Competition and Cooperation Functions for Ultralow Voltage Neuromorphic Computing

Realization of dendric signal processing in the human brain is of great significance for spatiotemporal neuromorphic engineering. Here, we proposed an ionic dendrite device with multichannel communication, which could realize synaptic behaviors even under an ultralow action potential of 80 mV. The device not only could simulate one-to-one information transfer of axons but also achieve a many-to-one modulation mode of dendrites. By the adjustment of two presynapses, Pavlov's dog conditioning experiment was learned successfully. Furthermore, the device also could emulate the biological synaptic competition and synaptic cooperation phenomenon through the comodulation of three presynapses, which are crucial for artificial neural network (ANN) implementation. Finally, an ANN was further constructed to realize highly efficient and anti-interference recognition of fashion patterns. By introducing the cooperative device, synaptic weight updates could be improved for higher linearity and larger dynamic regulation range in neuromorphic computing, resulting in higher recognition accuracy and efficiency. Such an artificial dendric device has great application prospects in the processing of more complex information and the construction of an ANN system with more functions.

Heterosynaptic plasticity-induced modulation of synapses

Plasticity is a common feature of synapses that is stated in different ways and occurs through several mechanisms. The regular action of the brain needs to be balanced in several neuronal and synaptic features, one of which is synaptic plasticity. The different homeostatic processes, including the balance between excitation/inhibition or homeostasis of synaptic weights at the single-neuron level, may obtain this. Homosynaptic Hebbian-type plasticity causes associative alterations of synapses. Both homosynaptic and heterosynaptic plasticity characterize the corresponding aspects of adjustable synapses, and both are essential for the regular action of neural systems and their plastic synapses.In this review, we will compare homo- and heterosynaptic plasticity and the main factors affecting the direction of plastic changes. This review paper will also discuss the diverse functions of the different kinds of heterosynaptic plasticity and their properties. We argue that a complementary system of heterosynaptic plasticity demonstrates an essential cellular constituent for homeostatic modulation of synaptic weights and neuronal activity.

Quantum imaging of the reconfigurable VO2 synaptic electronics for neuromorphic computing

Neuromorphic computing has shown remarkable capabilities in silicon-based artificial intelligence, which can be optimized by using Mott materials for functional synaptic connections. However, the research efforts focus on two-terminal artificial synapses and envisioned the networks controlled by silicon-based circuits, which is difficult to develop and integrate. Here, we propose a dynamic network with laser-controlled conducting filaments based on electric field-induced local insulator-metal transition of vanadium dioxide. Quantum sensing is used to realize conductivity-sensitive imaging of conducting filament. We find that the location of filament formation is manipulated by focused laser, which is applicable to simulate the dynamical synaptic connections between the neurons. The ability to process signals with both long-term and short-term potentiation is further demonstrated with ~60 times on/off ratio while switching the pathways. This study opens the door to the development of dynamic network structures depending on easily controlled conduction pathways, mimicking the biological nervous systems.

Heterosynaptic MoS2 Memtransistors Emulating Biological Neuromodulation for Energy-Efficient Neuromorphic Electronics

Heterosynaptic neuromodulation is a key enabler for energy-efficient and high-level biological neural processing. However, such manifold synaptic modulation cannot be emulated using conventional memristors and synaptic transistors. Thus, reported herein is a three-terminal heterosynaptic memtransistor using an intentional-defect-generated molybdenum disulfide channel. Particularly, the defect-mediated space-charge-limited conduction in the ultrathin channel results in memristive switching characteristics between the source and drain terminals, which are further modulated using a gate terminal according to the gate-tuned filling of trap states. The device acts as an artificial synapse controlled by sub-femtojoule impulses from both the source and gate terminals, consuming lower energy than its biological counterpart. In particular, electrostatic gate modulation, corresponding to biological neuromodulation, additionally regulates the dynamic range and tuning rate of the synaptic weight, independent of the programming (source) impulses. Notably, this heterosynaptic modulation not only improves the learning accuracy and efficiency but also reduces energy consumption in the pattern recognition. Thus, the study presents a new route leading toward the realization of highly networked and energy-efficient neuromorphic electronics.

A survey and perspective on neuromorphic continual learning systems

With the advent of low-power neuromorphic computing systems, new possibilities have emerged for deployment in various sectors, like healthcare and transport, that require intelligent autonomous applications. These applications require reliable low-power solutions for sequentially adapting to new relevant data without loss of learning. Neuromorphic systems are inherently inspired by biological neural networks that have the potential to offer an efficient solution toward the feat of continual learning. With increasing attention in this area, we present a first comprehensive review of state-of-the-art neuromorphic continual learning (NCL) paradigms. The significance of our study is multi-fold. We summarize the recent progress and propose a plausible roadmap for developing end-to-end NCL systems. We also attempt to identify the gap between research and the real-world deployment of NCL systems in multiple applications. We do so by assessing the recent contributions in neuromorphic continual learning at multiple levels-applications, algorithms, architectures, and hardware. We discuss the relevance of NCL systems and draw out application-specific requisites. We analyze the biological underpinnings that are used for acquiring high-level performance. At the hardware level, we assess the ability of the current neuromorphic platforms and emerging nano-device-based architectures to support these algorithms in the presence of several constraints. Further, we propose refinements to continual learning metrics for applying them to NCL systems. Finally, the review identifies gaps and possible solutions that are not yet focused upon for deploying application-specific NCL systems in real-life scenarios.

Invertebrates as models of learning and memory: investigating neural and molecular mechanisms

In this Commentary, we shed light on the use of invertebrates as model organisms for understanding the causal and conserved mechanisms of learning and memory. We provide a condensed chronicle of the contribution offered by mollusks to the studies on how and where the nervous system encodes and stores memory and describe the rich cognitive capabilities of some insect species, including attention and concept learning. We also discuss the use of planarians for investigating the dynamics of memory during brain regeneration and highlight the role of stressful stimuli in forming memories. Furthermore, we focus on the increasing evidence that invertebrates display some forms of emotions, which provides new opportunities for unveiling the neural and molecular mechanisms underlying the complex interaction between stress, emotions and cognition. In doing so, we highlight experimental challenges and suggest future directions that we expect the field to take in the coming years, particularly regarding what we, as humans, need to know for preventing and/or delaying memory loss. This article has an associated ECR Spotlight interview with Veronica Rivi.

Emerging electrolyte-gated transistors for neuromorphic perception

With the rapid development of intelligent robotics, the Internet of Things, and smart sensor technologies, great enthusiasm has been devoted to developing next-generation intelligent systems for the emulation of advanced perception functions of humans. Neuromorphic devices, capable of emulating the learning, memory, analysis, and recognition functions of biological neural systems, offer solutions to intelligently process sensory information. As one of the most important neuromorphic devices, Electrolyte-gated transistors (EGTs) have shown great promise in implementing various vital neural functions and good compatibility with sensors. This review introduces the materials, operating principle, and performances of EGTs, followed by discussing the recent progress of EGTs for synapse and neuron emulation. Integrating EGTs with sensors that faithfully emulate diverse perception functions of humans such as tactile and visual perception is discussed. The challenges of EGTs for further development are given.

Progress and Challenges for Memtransistors in Neuromorphic Circuits and Systems

Due to the increasing importance of artificial intelligence (AI), significant recent effort has been devoted to the development of neuromorphic circuits that seek to emulate the energy-efficient information processing of the brain. While non-volatile memory (NVM) based on resistive switches, phase-change memory, and magnetic tunnel junctions has shown potential for implementing neural networks, additional multi-terminal device concepts are required for more sophisticated bio-realistic functions. Of particular interest are memtransistors based on low-dimensional nanomaterials, which are capable of electrostatically tuning memory and learning behavior at the device level. Herein, a conceptual overview of the memtransistor is provided in the context of neuromorphic circuits. Recent progress is surveyed for memtransistors and related multi-terminal NVM devices including dual-gated floating-gate memories, dual-gated ferroelectric transistors, and dual-gated van der Waals heterojunctions. The different materials systems and device architectures are classified based on the degree of control and relative tunability of synaptic behavior, with an emphasis on device concepts that harness the reduced dimensionality, weak electrostatic screening, and phase-changes properties of nanomaterials. Finally, strategies for achieving wafer-scale integration of memtransistors and multi-terminal NVM devices are delineated, with specific attention given to the materials challenges for practical neuromorphic circuits.

Linking the Amyloid, Tau, and Mitochondrial Hypotheses of Alzheimer's Disease and Identifying Promising Drug Targets

Damage or loss of brain cells and impaired neurochemistry, neurogenesis, and synaptic and nonsynaptic plasticity of the brain lead to dementia in neurodegenerative diseases, such as Alzheimer's disease (AD). Injury to synapses and neurons and accumulation of extracellular amyloid plaques and intracellular neurofibrillary tangles are considered the main morphological and neuropathological features of AD. Age, genetic and epigenetic factors, environmental stressors, and lifestyle contribute to the risk of AD onset and progression. These risk factors are associated with structural and functional changes in the brain, leading to cognitive decline. Biomarkers of AD reflect or cause specific changes in brain function, especially changes in pathways associated with neurotransmission, neuroinflammation, bioenergetics, apoptosis, and oxidative and nitrosative stress. Even in the initial stages, AD is associated with Aβ neurotoxicity, mitochondrial dysfunction, and tau neurotoxicity. The integrative amyloid-tau-mitochondrial hypothesis assumes that the primary cause of AD is the neurotoxicity of Aβ oligomers and tau oligomers, mitochondrial dysfunction, and their mutual synergy. For the development of new efficient AD drugs, targeting the elimination of neurotoxicity, mutual potentiation of effects, and unwanted protein interactions of risk factors and biomarkers (mainly Aβ oligomers, tau oligomers, and mitochondrial dysfunction) in the early stage of the disease seems promising.

Motor constellation theory: A model of infants' phonological development

Every normally developing human infant solves the difficult problem of mapping their native-language phonology, but the neural mechanisms underpinning this behavior remain poorly understood. Here, motor constellation theory, an integrative neurophonological model, is presented, with the goal of explicating this issue. It is assumed that infants' motor-auditory phonological mapping takes place through infants' orosensory "reaching" for phonological elements observed in the language-specific ambient phonology, via reference to kinesthetic feedback from motor systems (e.g., articulators), and auditory feedback from resulting speech and speech-like sounds. Attempts are regulated by basal ganglion-cerebellar speech neural circuitry, and successful attempts at reproduction are enforced through dopaminergic signaling. Early in life, the pace of anatomical development constrains mapping such that complete language-specific phonological mapping is prohibited by infants' undeveloped supralaryngeal vocal tract and undescended larynx; constraints gradually dissolve with age, enabling adult phonology. Where appropriate, reference is made to findings from animal and clinical models. Some implications for future modeling and simulation efforts, as well as clinical settings, are also discussed.

Heterosynaptic plasticity in biomembrane memristors controlled by pH

Abstract

In biology, heterosynaptic plasticity maintains homeostasis in synaptic inputs during associative learning and memory, and initiates long-term changes in synaptic strengths that nonspecifically modulate different synapse types. In bioinspired neuromorphic circuits, heterosynaptic plasticity may be used to extend the functionality of two-terminal, biomimetic memristors. In this article, we explore how changes in the pH of droplet interface bilayer aqueous solutions modulate the memristive responses of a lipid bilayer membrane in the pH range 4.97-7.40. Surprisingly, we did not find conclusive evidence for pH-dependent shifts in the voltage thresholds (V*) needed for alamethicin ion channel formation in the membrane. However, we did observe a clear modulation in the dynamics of pore formation with pH in time-dependent, pulsed voltage experiments. Moreover, at the same voltage, lowering the pH resulted in higher steady-state currents because of increased numbers of conductive peptide ion channels in the membrane. This was due to increased partitioning of alamethicin monomers into the membrane at pH 4.97, which is below the pKa (~5.3-5.7) of carboxylate groups on the glutamate residues of the peptide, making the monomers more hydrophobic. Neutralization of the negative charges on these residues, under acidic conditions, increased the concentration of peptide monomers in the membrane, shifting the equilibrium concentrations of peptide aggregate assemblies in the membrane to favor greater numbers of larger, increasingly more conductive pores. It also increased the relaxation time constants for pore formation and decay, and enhanced short-term facilitation and depression of the switching characteristics of the device. Modulating these thresholds globally and independently of alamethicin concentration and applied voltage will enable the assembly of neuromorphic computational circuitry with enhanced functionality.

Impact statement

We describe how to use pH as a modulatory "interneuron" that changes the voltage-dependent memristance of alamethicin ion channels in lipid bilayers by changing the structure and dynamical properties of the bilayer. Having the ability to independently control the threshold levels for pore conduction from voltage or ion channel concentration enables additional levels of programmability in a neuromorphic system. In this article, we note that barriers to conduction from membrane-bound ion channels can be lowered by reducing solution pH, resulting in higher currents, and enhanced short-term learning behavior in the form of paired-pulse facilitation. Tuning threshold values with environmental variables, such as pH, provide additional training and learning algorithms that can be used to elicit complex functionality within spiking neural networks.

Graphical abstract

Supplementary information

The online version contains supplementary material available at 10.1557/s43577-022-00344-z.

Neural Information Processing and Computations of Two-Input Synapses

Information processing in artificial neural networks is largely dependent on the nature of neuron models. While commonly used models are designed for linear integration of synaptic inputs, accumulating experimental evidence suggests that biological neurons are capable of nonlinear computations for many converging synaptic inputs via homo- and heterosynaptic mechanisms. This nonlinear neuronal computation may play an important role in complex information processing at the neural circuit level. Here we characterize the dynamics and coding properties of neuron models on synaptic transmissions delivered from two hidden states. The neuronal information processing is influenced by the cooperative and competitive interactions among synapses and the coherence of the hidden states. Furthermore, we demonstrate that neuronal information processing under two-input synaptic transmission can be mapped to linearly nonseparable XOR as well as basic AND/OR operations. In particular, the mixtures of linear and nonlinear neuron models outperform the fashion-MNIST test compared to the neural networks consisting of only one type. This study provides a computational framework for assessing information processing of neuron and synapse models that may be beneficial for the design of brain-inspired artificial intelligence algorithms and neuromorphic systems.

Dissociable use-dependent processes for volitional goal-directed reaching

Repetition of specific movement biases subsequent actions towards the practiced movement, a phenomenon known as use-dependent learning (UDL). Recent experiments that impose strict constraints on planning time have revealed two sources of use-dependent biases, one arising from dynamic changes occurring during motor planning and another reflecting a stable shift in motor execution. Here, we used a distributional analysis to examine the contribution of these biases in reaching. To create the conditions for UDL, the target appeared at a designated 'frequent' location on most trials, and at one of six 'rare' locations on other trials. Strikingly, the heading angles were bimodally distributed, with peaks at both frequent and rare target locations. Despite having no constraints on planning time, participants exhibited a robust bias towards the frequent target when movements were self-initiated quickly, the signature of a planning bias; notably, the peak near the rare target was shifted in the frequently practiced direction, the signature of an execution bias. Furthermore, these execution biases were not only replicated in a delayed-response task but were also insensitive to reward. Taken together, these results extend our understanding of how volitional movements are influenced by recent experience.

Introducing principles of synaptic integration in the optimization of deep neural networks

Plasticity circuits in the brain are known to be influenced by the distribution of the synaptic weights through the mechanisms of synaptic integration and local regulation of synaptic strength. However, the complex interplay of stimulation-dependent plasticity with local learning signals is disregarded by most of the artificial neural network training algorithms devised so far. Here, we propose a novel biologically inspired optimizer for artificial and spiking neural networks that incorporates key principles of synaptic plasticity observed in cortical dendrites: GRAPES (Group Responsibility for Adjusting the Propagation of Error Signals). GRAPES implements a weight-distribution-dependent modulation of the error signal at each node of the network. We show that this biologically inspired mechanism leads to a substantial improvement of the performance of artificial and spiking networks with feedforward, convolutional, and recurrent architectures, it mitigates catastrophic forgetting, and it is optimally suited for dedicated hardware implementations. Overall, our work indicates that reconciling neurophysiology insights with machine intelligence is key to boosting the performance of neural networks.

Tasks involving continual learning and adaptation to real-time scenarios remain challenging for artificial neural networks in contrast to real brain. The authors propose here a brain-inspired optimizer based on mechanisms of synaptic integration and strength regulation for improved performance of both artificial and spiking neural networks.

A Conditioned Place Preference for Heroin Is Signaled by Increased Dopamine and Direct Pathway Activity and Decreased Indirect Pathway Activity in the Nucleus Accumbens

Repeated pairing of a drug with a neutral stimulus, such as a cue or context, leads to the attribution of the drug's reinforcing properties to that stimulus, and exposure to that stimulus in the absence of the drug can elicit drug-seeking. A principal role for the NAc in the response to drug-associated stimuli has been well documented. Direct and indirect pathway medium spiny neurons (dMSNs and iMSNs) have been shown to bidirectionally regulate cue-induced heroin-seeking in rats expressing addiction-like phenotypes, and a shift in NAc activity toward the direct pathway has been shown in mice following cocaine conditioned place preference (CPP). However, how NAc signaling guides heroin CPP, and whether heroin alters the balance of signaling between dMSNs and iMSNs, remains unknown. Moreover, the role of NAc dopamine signaling in heroin reinforcement is unclear. Here, we integrate fiber photometry for in vivo monitoring of dopamine and dMSN/iMSN calcium activity with a heroin CPP procedure in rats to begin to address these questions. We identify a sensitization-like response to heroin in the NAc, with prominent iMSN activity during initial heroin exposure and prominent dMSN activity following repeated heroin exposure. We demonstrate a ramp in dopamine activity, dMSN activation, and iMSN inactivation preceding entry into a heroin-paired context, and a decrease in dopamine activity, dMSN inactivation, and iMSN activation preceding exit from a heroin-paired context. Finally, we show that buprenorphine is sufficient to prevent the development of heroin CPP and reduce Fos activation in the NAc after conditioning. Together, these data support the hypothesis that an imbalance in NAc activity contributes to the development of drug-cue associations that can drive addiction processes.SIGNIFICANCE STATEMENT The attribution of the reinforcing effects of drugs to neutral stimuli (e.g., cues and contexts) contributes to the long-standing nature of addiction, as re-exposure to drug-associated stimuli can reinstate drug-seeking and -taking even after long periods of abstinence. The NAc has an established role in encoding the value of drug-associated stimuli, and dopamine release into the NAc is known to modulate the reinforcing effects of drugs, including heroin. Using fiber photometry, we show that entering a heroin-paired context is driven by dopamine signaling and NAc direct pathway activation, whereas exiting a heroin-paired context is driven by NAc indirect pathway activation. This study provides further insight into the role of NAc microcircuitry in encoding the reinforcing properties of heroin.

Learning as filtering: Implications for spike-based plasticity

Most normative models in computational neuroscience describe the task of learning as the optimisation of a cost function with respect to a set of parameters. However, learning as optimisation fails to account for a time-varying environment during the learning process and the resulting point estimate in parameter space does not account for uncertainty. Here, we frame learning as filtering, i.e., a principled method for including time and parameter uncertainty. We derive the filtering-based learning rule for a spiking neuronal network—the Synaptic Filter—and show its computational and biological relevance. For the computational relevance, we show that filtering improves the weight estimation performance compared to a gradient learning rule with optimal learning rate. The dynamics of the mean of the Synaptic Filter is consistent with spike-timing dependent plasticity (STDP) while the dynamics of the variance makes novel predictions regarding spike-timing dependent changes of EPSP variability. Moreover, the Synaptic Filter explains experimentally observed negative correlations between homo- and heterosynaptic plasticity.

The task of learning is commonly framed as parameter optimisation. Here, we adopt the framework of learning as filtering where the task is to continuously estimate the uncertainty about the parameters to be learned. We apply this framework to synaptic plasticity in a spiking neuronal network. Filtering includes a time-varying environment and parameter uncertainty on the level of the learning task. We show that learning as filtering can qualitatively explain two biological experiments on synaptic plasticity that cannot be explained by learning as optimisation. Moreover, we make a new prediction and improve performance with respect to a gradient learning rule. Thus, learning as filtering is a promising candidate for learning models.

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.

How do humans flexibly adapt action sequences? For instance, musicians can learn a song and quickly speed up or slow down the tempo, or even play the song following a completely different rhythm (e.g., a rock song using a bossa nova rhythm). In this work, we build a biologically plausible network of cortico-basal ganglia interactions that explains how this temporal flexibility may emerge in the brain. Crucially, our model factorizes sequence order and action timing, respectively represented in cortical and basal ganglia dynamics. This factorization allows full temporal flexibility, i.e. the timing of a learned action sequence can be recomposed without interfering with the order of the sequence. As such, our model is capable of learning asynchronous action sequences, and flexibly shift, rescale, and recompose them, while accounting for biological data.

Brain-inspired global-local learning incorporated with neuromorphic computing

There are two principle approaches for learning in artificial intelligence: error-driven global learning and neuroscience-oriented local learning. Integrating them into one network may provide complementary learning capabilities for versatile learning scenarios. At the same time, neuromorphic computing holds great promise, but still needs plenty of useful algorithms and algorithm-hardware co-designs to fully exploit its advantages. Here, we present a neuromorphic global-local synergic learning model by introducing a brain-inspired meta-learning paradigm and a differentiable spiking model incorporating neuronal dynamics and synaptic plasticity. It can meta-learn local plasticity and receive top-down supervision information for multiscale learning. We demonstrate the advantages of this model in multiple different tasks, including few-shot learning, continual learning, and fault-tolerance learning in neuromorphic vision sensors. It achieves significantly higher performance than single-learning methods. We further implement the model in the Tianjic neuromorphic platform by exploiting algorithm-hardware co-designs and prove that the model can fully utilize neuromorphic many-core architecture to develop hybrid computation paradigm.

Dendritic Organic Electrochemical Transistors Grown by Electropolymerization for 3D Neuromorphic Engineering

One of the major limitations of standard top-down technologies used in today's neuromorphic engineering is their inability to map the 3D nature of biological brains. Here, it is shown how bipolar electropolymerization can be used to engineer 3D networks of PEDOT:PSS dendritic fibers. By controlling the growth conditions of the electropolymerized material, it is investigated how dendritic fibers can reproduce structural plasticity by creating structures of controllable shape. Gradual topologies evolution is demonstrated in a multielectrode configuration. A detailed electrical characterization of the PEDOT:PSS dendrites is conducted through DC and impedance spectroscopy measurements and it is shown how organic electrochemical transistors (OECT) can be realized with these structures. These measurements reveal that quasi-static and transient response of OECTs can be adjusted by controlling dendrites' morphologies. The unique properties of organic dendrites are used to demonstrate short-term, long-term, and structural plasticity, which are essential features required for future neuromorphic hardware development.

Multi-Terminal Memristive Devices Enabling Tunable Synaptic Plasticity in Neuromorphic Hardware: A Mini-Review

Neuromorphic computing based on spiking neural networks has the potential to significantly improve on-line learning capabilities and energy efficiency of artificial intelligence, specially for edge computing. Recent progress in computational neuroscience have demonstrated the importance of heterosynaptic plasticity for network activity regulation and memorization. Implementing heterosynaptic plasticity in hardware is thus highly desirable, but important materials and engineering challenges remain, calling for breakthroughs in neuromorphic devices. In this mini-review, we propose an overview of the latest advances in multi-terminal memristive devices on silicon with tunable synaptic plasticity, enabling heterosynaptic plasticity in hardware. The scalability and compatibility of the devices with industrial complementary metal oxide semiconductor (CMOS) technologies are discussed.

Brain is modulated by neuronal plasticity during postnatal development

Neuroplasticity is referred to the ability of the nervous system to change its structure or functions as a result of former stimuli. It is a plausible mechanism underlying a dynamic brain through adaptation processes of neural structure and activity patterns. Nevertheless, it is still unclear how the plastic neural systems achieve and maintain their equilibrium. Additionally, the alterations of balanced brain dynamics under different plasticity rules have not been explored either. Therefore, the present article primarily aims to review recent research studies regarding homosynaptic and heterosynaptic neuroplasticity characterized by the manipulation of excitatory and inhibitory synaptic inputs. Moreover, it attempts to understand different mechanisms related to the main forms of synaptic plasticity at the excitatory and inhibitory synapses during the brain development processes. Hence, this study comprised surveying those articles published since 1988 and available through PubMed, Google Scholar and science direct databases on a keyword-based search paradigm. All in all, the study results presented extensive and corroborative pieces of evidence for the main types of plasticity, including the long-term potentiation (LTP) and long-term depression (LTD) of the excitatory and inhibitory postsynaptic potentials (EPSPs and IPSPs).

Artificial Synapse Based on a 2D-SnO2 Memtransistor with Dynamically Tunable Analog Switching for Neuromorphic Computing

A new type of two-dimensional (2D) SnO2 semiconductor-based gate-tunable memristor, that is, a memtransistor, an integrated device of a memristor and a transistor, was demonstrated to advance next-generation neuromorphic computing technology. The polycrystalline 2D-SnO2 memristors derived from a low-temperature and vacuum-free liquid metal process offer several interesting resistive switching properties such as excellent digital/analog resistive switching, multistate storage, and gate-tunability function of resistance switching states. Significantly, the gate tunability function that is not achievable in conventional two-terminal memristors provides the capability to implement heterosynaptic analog switching by regulating gate bias for enabling complex neuromorphic learning. We successfully demonstrated that the gate-tunable synaptic device dynamically modulated the analog switching behavior with good linearity and an improved conductance change ratio for high recognition accuracy learning. The presented gate-tunable 2D-oxide memtransistor will advance neuromorphic device technology and open up new opportunities to design learning schemes with an extra degree of freedom.

Reconfigurable 2D WSe2 -Based Memtransistor for Mimicking Homosynaptic and Heterosynaptic Plasticity

The mimicking of both homosynaptic and heterosynaptic plasticity using a high-performance synaptic device is important for developing human-brain-like neuromorphic computing systems to overcome the ever-increasing challenges caused by the conventional von Neumann architecture. However, the commonly used synaptic devices (e.g., memristors and transistors) require an extra modulate terminal to mimic heterosynaptic plasticity, and their capability of synaptic plasticity simulation is limited by the low weight adjustability. In this study, a WSe₂ -based memtransistor for mimicking both homosynaptic and heterosynaptic plasticity is fabricated. By applying spikes on either the drain or gate terminal, the memtransistor can mimic common homosynaptic plasticity, including spiking rate dependent plasticity, paired pulse facilitation/depression, synaptic potentiation/depression, and filtering. Benefitting from the multi-terminal input and high adjustability, the resistance state number and linearity of the memtransistor can be improved by optimizing the conditions of the two inputs. Moreover, the device can successfully mimic heterosynaptic plasticity without introducing an extra terminal and can simultaneously offer versatile reconfigurability of excitatory and inhibitory plasticity. These highly adjustable and reconfigurable characteristics offer memtransistors more freedom of choice for tuning synaptic weight, optimizing circuit design, and building artificial neuromorphic computing systems.

Constrained plasticity reserve as a natural way to control frequency and weights in spiking neural networks

Biological neurons have adaptive nature and perform complex computations involving the filtering of redundant information. However, most common neural cell models, including biologically plausible, such as Hodgkin-Huxley or Izhikevich, do not possess predictive dynamics on a single-cell level. Moreover, the modern rules of synaptic plasticity or interconnections weights adaptation also do not provide grounding for the ability of neurons to adapt to the ever-changing input signal intensity. While natural neuron synaptic growth is precisely controlled and restricted by protein supply and recycling, weight correction rules such as widely used STDP are efficiently unlimited in change rate and scale. The present article introduces new mechanics of interconnection between neuron firing rate homeostasis and weight change through STDP growth bounded by abstract protein reserve, controlled by the intracellular optimization algorithm. We show how these cellular dynamics help neurons filter out the intense noise signals to help neurons keep a stable firing rate. We also examine that such filtering does not affect the ability of neurons to recognize the correlated inputs in unsupervised mode. Such an approach might be used in the machine learning domain to improve the robustness of AI systems.

Giant Ferroelectric Resistance Switching Controlled by a Modulatory Terminal for Low-Power Neuromorphic In-Memory Computing

Ferroelectrics have been demonstrated as excellent building blocks for high-performance nonvolatile memories, including memristors, which play critical roles in the hardware implementation of artificial synapses and in-memory computing. Here, it is reported that the emerging van der Waals ferroelectric α-In₂ Se₃ can be used to successfully implement heterosynaptic plasticity (a fundamental but rarely emulated synaptic form) and achieve a resistance-switching ratio of heterosynaptic memristors above 10³ , which is two orders of magnitude larger than that in other similar devices. The polarization change of ferroelectric α-In₂ Se₃ channel is responsible for the resistance switching at various paired terminals. The third terminal of α-In₂ Se₃ memristors exhibits nonvolatile control over channel current at a picoampere level, endowing the devices with picojoule read-energy consumption to emulate the associative heterosynaptic learning. The simulation proves that both supervised and unsupervised learning manners can be implemented in α-In₂ Se₃ neutral networks with high image recognition accuracy. Moreover, these heterosynaptic devices can naturally realize Boolean logic without an additional circuit component. The results suggest that van der Waals ferroelectrics hold great potential for applications in complex, energy-efficient, brain-inspired computing systems and logic-in-memory computers.

Dopamine: The Neuromodulator of Long-Term Synaptic Plasticity, Reward and Movement Control

Dopamine (DA) is a key neurotransmitter involved in multiple physiological functions including motor control, modulation of affective and emotional states, reward mechanisms, reinforcement of behavior, and selected higher cognitive functions. Dysfunction in dopaminergic transmission is recognized as a core alteration in several devastating neurological and psychiatric disorders, including Parkinson's disease (PD), schizophrenia, bipolar disorder, attention deficit hyperactivity disorder (ADHD) and addiction. Here we will discuss the current insights on the role of DA in motor control and reward learning mechanisms and its involvement in the modulation of synaptic dynamics through different pathways. In particular, we will consider the role of DA as neuromodulator of two forms of synaptic plasticity, known as long-term potentiation (LTP) and long-term depression (LTD) in several cortical and subcortical areas. Finally, we will delineate how the effect of DA on dendritic spines places this molecule at the interface between the motor and the cognitive systems. Specifically, we will be focusing on PD, vascular dementia, and schizophrenia.

AgRP neurons trigger long-term potentiation and facilitate food seeking

Sufficient feeding is essential for animals' survival, which requires a cognitive capability to facilitate food seeking, but the neurobiological processes regulating food seeking are not fully understood. Here we show that stimulation of agouti-related peptide-expressing (AgRP) neurons triggers a long-term depression (LTD) of spontaneous excitatory post-synaptic current (sEPSC) in adjacent pro-opiomelanocortin (POMC) neurons and in most of their distant synaptic targets, including neurons in the paraventricular nucleus of the thalamus (PVT). The AgRP-induced sEPCS LTD can be enhanced by fasting but blunted by satiety signals, e.g. leptin and insulin. Mice subjected to food-seeking tasks develop similar neural plasticity in AgRP-innervated PVT neurons. Further, ablation of the majority of AgRP neurons, or only a subset of AgRP neurons that project to the PVT, impairs animals' ability to associate spatial and contextual cues with food availability during food seeking. A similar impairment can be also induced by optogenetic inhibition of the AgRP→PVT projections. Together, these results indicate that the AgRP→PVT circuit is necessary for food seeking.

Neuromorphic Engineering: From Biological to Spike-Based Hardware Nervous Systems

The human brain is a sophisticated, high-performance biocomputer that processes multiple complex tasks in parallel with high efficiency and remarkably low power consumption. Scientists have long been pursuing an artificial intelligence (AI) that can rival the human brain. Spiking neural networks based on neuromorphic computing platforms simulate the architecture and information processing of the intelligent brain, providing new insights for building AIs. The rapid development of materials engineering, device physics, chip integration, and neuroscience has led to exciting progress in neuromorphic computing with the goal of overcoming the von Neumann bottleneck. Herein, fundamental knowledge related to the structures and working principles of neurons and synapses of the biological nervous system is reviewed. An overview is then provided on the development of neuromorphic hardware systems, from artificial synapses and neurons to spike-based neuromorphic computing platforms. It is hoped that this review will shed new light on the evolution of brain-like computing.

Graphene memristive synapses for high precision neuromorphic computing

Memristive crossbar architectures are evolving as powerful in-memory computing engines for artificial neural networks. However, the limited number of non-volatile conductance states offered by state-of-the-art memristors is a concern for their hardware implementation since trained weights must be rounded to the nearest conductance states, introducing error which can significantly limit inference accuracy. Moreover, the incapability of precise weight updates can lead to convergence problems and slowdown of on-chip training. In this article, we circumvent these challenges by introducing graphene-based multi-level (>16) and non-volatile memristive synapses with arbitrarily programmable conductance states. We also show desirable retention and programming endurance. Finally, we demonstrate that graphene memristors enable weight assignment based on k-means clustering, which offers greater computing accuracy when compared with uniform weight quantization for vector matrix multiplication, an essential component for any artificial neural network.

A model for the transfer of control from the brain to the spinal cord through synaptic learning

The spinal cord is essential to the control of locomotion in legged animals and humans. However, the actual circuitry of the spinal controller remains only vaguely understood. Here we approach this problem from the viewpoint of learning. More precisely, we assume the circuitry evolves through the transfer of control from the brain to the spinal cord, propose a specific learning mechanism for this transfer based on the error between the cord and brain contributions to muscle control, and study the resulting structure of the spinal controller in a simplified neuromuscular model of human locomotion. The model focuses on the leg rebound behavior in stance and represents the spinal circuitry with 150 muscle reflexes. We find that after learning a spinal controller has evolved that produces leg rebound motions in the absence of a central brain input with only three structural reflex groups. These groups contain individual reflexes well known from physiological experiments but thought to serve separate purposes in the control of human locomotion. Our results suggest a more holistic interpretation of the role of individual sensory projections in spinal networks than is common. In addition, we discuss potential neural correlates for the proposed learning mechanism that may be probed in experiments. Together with such experiments, neuromuscular models of spinal learning likely will become effective tools for uncovering the structure and development of the spinal control circuitry.

Heterosynaptic Plasticity Emulated by Liquid Crystal-Carbon Nanotube Composites with Modulatory Interneurons

The aim of the neuromorphic computing is to emulate energy-efficient and smart data-processing ability of the biological brain, which is achieved by massively interconnected neurons and synapses. The strength of a connection between two neurons is modified by homosynaptic and heterosynaptic plasticity. As current research in the neuromorphic device is mainly focused on emulating homosynaptic plasticity, complex biological functions are not easy to mimic because they require both homosynaptic and heterosynaptic plasticity. We demonstrate the use of a liquid crystal-carbon nanotube (LC-CNT) composite as a resistive switching material that can emulate both the homosynaptic and heterosynaptic functions of biological neurons. The LC-CNT composite undergoes resistance change by CNT alignment and aggregated wire formation subjected to an applied electric field. A two-terminal device that exploits this mechanism achieves analog switching and homosynaptic potentiation. In a multiterminal device structure, the modulatory interneuron could tune the synaptic properties to perform heterosynaptic functions such as heterosynaptic potentiation, heterosynaptic facilitation, and synaptic weight normalization to emulate complex biological functions of a brain. Artificial synapses that exploit this multifunctionality of the LC-CNT composite have uses in next-generation neuromorphic devices.

Ultralow Power Wearable Heterosynapse with Photoelectric Synergistic Modulation

Although the energy consumption of reported neuromorphic computing devices inspired by biological systems has become lower than traditional memory, it still remains greater than bio-synapses (≈10 fJ per spike). Herein, a flexible MoS2-based heterosynapse is designed with two modulation modes, an electronic mode and a photoexcited mode. A one-step mechanical exfoliation method on flexible substrate and low-temperature atomic layer deposition process compatible with flexible electronics are developed for fabricating wearable heterosynapses. With a pre-spike of 100 ns, the synaptic device exhibits ultralow energy consumption of 18.3 aJ per spike in long-term potentiation and 28.9 aJ per spike in long-term depression. The ultrafast speed and ultralow power consumption provide a path for a neuromorphic computing system owning more excellent processing ability than the human brain. By adding optical modulation, a modulatory synapse is constructed to dynamically control correlations between pre- and post-synapses and realize complex global neuromodulations. The novel wearable heterosynapse expands the accessible range of synaptic weights (ratio of facilitation ≈228%), providing an insight into the application of wearable 2D highly efficient neuromorphic computing architectures.

Artificial Synapse Based on van der Waals Heterostructures with Tunable Synaptic Functions for Neuromorphic Computing

Two-dimensional (2D) materials and van der Waals heterostructures have attracted tremendous attention because of their appealing electronic, mechanical, and optoelectronic properties, which offer the possibility to extend the range of functionalities for diverse potential applications. Here, we fabricate a novel multiterminal device with dual-gate based on 2D material van der Waals heterostructures. Such a multiterminal device exhibited excellent nonvolatile multilevel resistance switching performance controlled by the source-drain voltage and back-gate voltage. Based on these features, heterosynaptic plasticity, in which the synaptic weight can be tuned by another modulatory interneuron, has been mimicked. A tunable analogue weight update (both on/off ratio and update nonlinearity) of synapse with high speed (50 ns) and low energy (∼7.3 fJ) programming has been achieved. These results demonstrate the great potential of the artificial synapse based on van der Waals heterostructures for neuromorphic computing.

Bidirectional short-term plasticity during single-trial learning of cerebellar-driven eyelid movements in mice

The brain is constantly monitoring its own performance, using error signals to trigger mechanisms of plasticity that help improve future behavior. Indeed, adaptive changes in behavior have been observed after a single error trial in many learning tasks, including cerebellum-dependent eyeblink conditioning. Here, we demonstrate that the plasticity underlying single-trial learning during eyeblink conditioning in mice is bidirectionally regulated by positive and negative prediction errors, has an ephemeral effect on behavior (decays in <1 min), and can be triggered in the absence of errors in performance. We suggest that these three properties of single-trial learning may be particularly useful for driving mechanisms of motor adaptation that can achieve optimal performance in the face of environmental disturbances with a fast timescale.

Effects of L-DOPA on quadripulse magnetic stimulation-induced long-term potentiation in older adults

Reduced cortical plasticity has been previously reported in older adult as compared with young adults. However, the effects of dopamine on this plasticity reduction remain unknown. Here, we assessed the effects of high-dose (200 mg) and medium-dose (100 mg) L-3,4-dihydroxyphenylalanine (L-DOPA) intake on the long-term potentiation (LTP)-like effect induced by quadripulse magnetic stimulation (QPS) in older adults (aged ∼65 years). The subjects were 32 (200 mg) and 20 (100 mg) healthy older adult volunteers. This study was designed as a double-blind, crossover and placebo-controlled trial on one dose of L-dopa. Two hours after taking L-DOPA or placebo-drug, QPS was applied over the motor cortex. Motor evoked potentials were recorded to evaluate the motor cortical excitability changes. We found that both doses of L-DOPA enhanced LTP after QPS in older adults as one group. We classified subjects into QPS responders and QPS nonresponders. Both L-DOPA doses produced significant LTP enhancement in QPS nonresponders, whereas either of doses did not produce significant LTP enhancement in QPS responders. Collectively, our findings suggest that the neural plasticity reductions observed in older adults could be partly improved by dopamine.

Modeling the Influence of Synaptic Plasticity on After-effects

While circadian rhythms in physiology and behavior demonstrate remarkable day-to-day precision, they are also able to exhibit plasticity in a variety of parameters and under a variety of conditions. After-effects are one type of plasticity in which exposure to non-24-h light-dark cycles (T-cycles) will alter the animal's free-running rhythm in subsequent constant conditions. We use a mathematical model to explore whether the concept of synaptic plasticity can explain the observation of after-effects. In this model, the SCN is composed of a set of individual oscillators randomly selected from a normally distributed population. Each cell receives input from a defined set of oscillators, and the overall period of a cell is a weighted average of its own period and that of its inputs. The influence that an input has on its target's period is determined by the proximity of the input cell's period to the imposed T-cycle period, such that cells with periods near T will have greater influence. Such an arrangement is able to duplicate the phenomenon of after-effects, with relatively few inputs per cell (~4-5) being required. When the variability of periods between oscillators is low, the system is quite robust and results in minimal after-effects, while systems with greater between-cell variability exhibit greater magnitude after-effects. T-cycles that produce maximal after-effects have periods within ~2.5 to 3 h of the population period. Overall, this model demonstrates that synaptic plasticity in the SCN network could contribute to plasticity of the circadian period.

Ionic modulation and ionic coupling effects in MoS2 devices for neuromorphic computing

Coupled ionic-electronic effects present intriguing opportunities for device and circuit development. In particular, layered two-dimensional materials such as MoS₂ offer highly anisotropic ionic transport properties, facilitating controlled ion migration and efficient ionic coupling among devices. Here, we report reversible modulation of MoS₂ films that is consistent with local 2H-1T' phase transitions by controlling the migration of Li⁺ ions with an electric field, where an increase/decrease in the local Li⁺ ion concentration leads to the transition between the 2H (semiconductor) and 1T' (metal) phases. The resulting devices show excellent memristive behaviour and can be directly coupled with each other through local ionic exchange, naturally leading to synaptic competition and synaptic cooperation effects observed in biology. These results demonstrate the potential of direct modulation of two-dimensional materials through field-driven ionic processes, and can lead to future electronic and energy devices based on coupled ionic-electronic effects and biorealistic implementation of artificial neural networks.

Convergence of monosynaptic and polysynaptic sensory paths onto common motor outputs in a Drosophila feeding connectome

We reconstructed, from a whole CNS EM volume, the synaptic map of input and output neurons that underlie food intake behavior of Drosophila larvae. Input neurons originate from enteric, pharyngeal and external sensory organs and converge onto seven distinct sensory synaptic compartments within the CNS. Output neurons consist of feeding motor, serotonergic modulatory and neuroendocrine neurons. Monosynaptic connections from a set of sensory synaptic compartments cover the motor, modulatory and neuroendocrine targets in overlapping domains. Polysynaptic routes are superimposed on top of monosynaptic connections, resulting in divergent sensory paths that converge on common outputs. A completely different set of sensory compartments is connected to the mushroom body calyx. The mushroom body output neurons are connected to interneurons that directly target the feeding output neurons. Our results illustrate a circuit architecture in which monosynaptic and multisynaptic connections from sensory inputs traverse onto output neurons via a series of converging paths.

Heterosynaptic modulation in the paraventricular nucleus of the hypothalamus

The stress response-originally described by Hans Selye as "the nonspecific response of the body to any demand made upon it"-is chiefly mediated by the hypothalamic-pituitary-adrenal (HPA) axis and is activated by diverse sensory stimuli that inform threats to homeostasis. The diversity of signals regulating the HPA axis is partly achieved by the complexity of afferent inputs that converge at the apex of the HPA axis: this apex is formed by a group of neurosecretory neurons that synthesize corticotropin-releasing hormone (CRH) in the paraventricular nucleus of the hypothalamus (PVN). The afferent synaptic inputs onto these PVN-CRH neurons originate from a number of brain areas, and PVN-CRH neurons respond to a long list of neurotransmitters/neuropeptides. Considering this complexity, an important question is how these diverse afferent signals independently and/or in concert influence the excitability of PVN-CRH neurons. While many of these inputs directly act on the postsynaptic PVN-CRH neurons for the summation of signals, accumulating data indicates that they also modulate each other's transmission in the PVN. This mode of transmission, termed heterosynaptic modulation, points to mechanisms through which the activity of a specific modulatory input (conveying a specific sensory signal) can up- or down-regulate the efficacy of other afferent synapses (mediating other stress modalities) depending on receptor expression for and spatial proximity to the heterosynaptic signals. Here, we review examples of heterosynaptic modulation in the PVN and discuss its potential role in the regulation of PVN-CRH neurons' excitability and resulting HPA axis activity. This article is part of the Special Issue entitled 'Hypothalamic Control of Homeostasis'.

Clock-Generated Temporal Codes Determine Synaptic Plasticity to Control Sleep

Neurons use two main schemes to encode information: rate coding (frequency of firing) and temporal coding (timing or pattern of firing). While the importance of rate coding is well established, it remains controversial whether temporal codes alone are sufficient for controlling behavior. Moreover, the molecular mechanisms underlying the generation of specific temporal codes are enigmatic. Here, we show in Drosophila clock neurons that distinct temporal spike patterns, dissociated from changes in firing rate, encode time-dependent arousal and regulate sleep. From a large-scale genetic screen, we identify the molecular pathways mediating the circadian-dependent changes in ionic flux and spike morphology that rhythmically modulate spike timing. Remarkably, the daytime spiking pattern alone is sufficient to drive plasticity in downstream arousal neurons, leading to increased firing of these cells. These findings demonstrate a causal role for temporal coding in behavior and define a form of synaptic plasticity triggered solely by temporal spike patterns.

Competition for synaptic building blocks shapes synaptic plasticity

Changes in the efficacies of synapses are thought to be the neurobiological basis of learning and memory. The efficacy of a synapse depends on its current number of neurotransmitter receptors. Recent experiments have shown that these receptors are highly dynamic, moving back and forth between synapses on time scales of seconds and minutes. This suggests spontaneous fluctuations in synaptic efficacies and a competition of nearby synapses for available receptors. Here we propose a mathematical model of this competition of synapses for neurotransmitter receptors from a local dendritic pool. Using minimal assumptions, the model produces a fast multiplicative scaling behavior of synapses. Furthermore, the model explains a transient form of heterosynaptic plasticity and predicts that its amount is inversely related to the size of the local receptor pool. Overall, our model reveals logistical tradeoffs during the induction of synaptic plasticity due to the rapid exchange of neurotransmitter receptors between synapses.

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifelong learning. The interplay of these elements leads to the emergence of biological intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) employ simulated evolution in-silico to breed plastic neural networks with the aim to autonomously design and create learning systems. EPANN experiments evolve networks that include both innate properties and the ability to change and learn in response to experiences in different environments and problem domains. EPANNs' aims include autonomously creating learning systems, bootstrapping learning from scratch, recovering performance in unseen conditions, testing the computational advantages of particular neural components, and deriving hypotheses on the emergence of biological learning. Thus, EPANNs may include a large variety of different neuron types and dynamics, network architectures, plasticity rules, and other factors. While EPANNs have seen considerable progress over the last two decades, current scientific and technological advances in artificial neural networks are setting the conditions for radically new approaches and results. Exploiting the increased availability of computational resources and of simulation environments, the often challenging task of hand-designing learning neural networks could be replaced by more autonomous and creative processes. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and possible developments are presented.

Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of NeoHebbian Three-Factor Learning Rules

Most elementary behaviors such as moving the arm to grasp an object or walking into the next room to explore a museum evolve on the time scale of seconds; in contrast, neuronal action potentials occur on the time scale of a few milliseconds. Learning rules of the brain must therefore bridge the gap between these two different time scales. Modern theories of synaptic plasticity have postulated that the co-activation of pre- and postsynaptic neurons sets a flag at the synapse, called an eligibility trace, that leads to a weight change only if an additional factor is present while the flag is set. This third factor, signaling reward, punishment, surprise, or novelty, could be implemented by the phasic activity of neuromodulators or specific neuronal inputs signaling special events. While the theoretical framework has been developed over the last decades, experimental evidence in support of eligibility traces on the time scale of seconds has been collected only during the last few years. Here we review, in the context of three-factor rules of synaptic plasticity, four key experiments that support the role of synaptic eligibility traces in combination with a third factor as a biological implementation of neoHebbian three-factor learning rules.

Hippocampal 5-HT Input Regulates Memory Formation and Schaffer Collateral Excitation

The efficacy and duration of memory storage is regulated by neuromodulatory transmitter actions. While the modulatory transmitter serotonin (5-HT) plays an important role in implicit forms of memory in the invertebrate Aplysia, its function in explicit memory mediated by the mammalian hippocampus is less clear. Specifically, the consequences elicited by the spatio-temporal gradient of endogenous 5-HT release are not known. Here we applied optogenetic techniques in mice to gain insight into this fundamental biological process. We find that activation of serotonergic terminals in the hippocampal CA1 region both potentiates excitatory transmission at CA3-to-CA1 synapses and enhances spatial memory. Conversely, optogenetic silencing of CA1 5-HT terminals inhibits spatial memory. We furthermore find that synaptic potentiation is mediated by 5-HT4 receptors and that systemic modulation of 5-HT4 receptor function can bidirectionally impact memory formation. Collectively, these data reveal powerful modulatory influence of serotonergic synaptic input on hippocampal function and memory formation.

Alkaline pH Promotes NADPH Oxidase-Independent Neutrophil Extracellular Trap Formation: A Matter of Mitochondrial Reactive Oxygen Species Generation and Citrullination and Cleavage of Histone

pH is highly variable in different tissues and affects many enzymatic reactions in neutrophils. In response to calcium ionophores such as A23187 and ionomycin, neutrophils undergo nicotinamide adenine dinucleotide phosphate oxidase (NOX)-independent neutrophil extracellular trap (NET) formation (NETosis). However, how pH influences calcium-dependent Nox-independent NET formation is not well understood. We hypothesized that increasing pH promotes Nox-independent NET formation by promoting calcium influx, mitochondrial reactive oxygen species (mROS) generation, histone citrullination, and histone cleavage. Here, we show that stimulating human neutrophils isolated from peripheral blood with calcium ionophore A23187 or ionomycin in the media with increasing extracellular pH (6.6, 6.8, 7.0, 7.2, 7.4, 7.8) drastically increases intracellular pH within in 10-20 min. These intracellular pH values are much higher compared to unstimulated cells placed in the media with corresponding pH values. Raising pH slightly drastically increases intracellular calcium concentration in resting and stimulated neutrophils, respectively. Like calcium, mROS generation also increases with increasing pH. An mROS scavenger, MitoTempo, significantly suppresses calcium ionophore-mediated NET formation with a greater effect at higher pH, indicating that mROS production is at least partly responsible for pH-dependent suppression of Nox-independent NETosis. In addition, raising pH increases PAD4 activity as determined by the citrullination of histone (CitH3) and histone cleavage determined by Western blots. The pH-dependent histone cleavage is reproducibly very high during ionomycin-induced NETosis compared to A23187-induced NETosis. Little or no histone cleavage was noted in unstimulated cells, at any pH. Both CitH3 and cleavage of histones facilitate DNA decondensation. Therefore, alkaline pH promotes intracellular calcium influx, mROS generation, PAD4-mediated CitH3 formation, histone 4 cleavage and eventually NET formation. Calcium-mediated NET formation and CitH3 formation are often related to sterile inflammation. Hence, understanding these important mechanistic steps helps to explain how pH regulates NOX-independent NET formation, and modifying pH may help to regulate NET formation during sterile inflammation or potential damage caused by compounds such as ionomycin, secreted by Streptomyces, a group of Gram-positive bacteria well known for producing antibiotics.