Computational models in the age of large datasets

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Flexible computation is a hallmark of intelligent behavior. However, little is known about how neural networks contextually reconfigure for different computations. In the present work, we identified an algorithmic neural substrate for modular computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses revealed learned computational strategies mirroring the modular subtask structure of the training task set. Dynamical motifs, which are recurring patterns of neural activity that implement specific computations through dynamics, such as attractors, decision boundaries and rotations, were reused across tasks. For example, tasks requiring memory of a continuous circular variable repurposed the same ring attractor. We showed that dynamical motifs were implemented by clusters of units when the unit activation function was restricted to be positive. Cluster lesions caused modular performance deficits. Motifs were reconfigured for fast transfer learning after an initial phase of learning. This work establishes dynamical motifs as a fundamental unit of compositional computation, intermediate between neuron and network. As whole-brain studies simultaneously record activity from multiple specialized systems, the dynamical motif framework will guide questions about specialization and generalization.

Prediction of neural activity in connectome-constrained recurrent networks

We develop a theory of connectome-constrained neural networks in which a "student" network is trained to reproduce the activity of a ground-truth "teacher," representing a neural system for which a connectome is available. Unlike standard paradigms with unconstrained connectivity, here the two networks have the same connectivity but different biophysical parameters, reflecting uncertainty in neuronal and synaptic properties. We find that a connectome is often insufficient to constrain the dynamics of networks that perform a specific task, illustrating the difficulty of inferring function from connectivity alone. However, recordings from a small subset of neurons can remove this degeneracy, producing dynamics in the student that agree with the teacher. Our theory can also prioritize which neurons to record from to most efficiently predict unmeasured network activity. Our analysis shows that the solution spaces of connectome-constrained and unconstrained models are qualitatively different and provides a framework to determine when such models yield consistent dynamics.

Single shot detection of alterations across multiple ionic currents from assimilation of cell membrane dynamics

The dysfunction of ion channels is a causative factor in a variety of neurological diseases, thereby defining the implicated channels as key drug targets. The detection of functional changes in multiple specific ionic currents currently presents a challenge, particularly when the neurological causes are either a priori unknown, or are unexpected. Traditional patch clamp electrophysiology is a powerful tool in this regard but is low throughput. Here, we introduce a single-shot method for detecting alterations amongst a range of ion channel types from subtle changes in membrane voltage in response to a short chaotically driven current clamp protocol. We used data assimilation to estimate the parameters of individual ion channels and from these we reconstructed ionic currents which exhibit significantly lower error than the parameter estimates. Such reconstructed currents thereby become sensitive predictors of functional alterations in biological ion channels. The technique correctly predicted which ionic current was altered, and by approximately how much, following pharmacological blockade of BK, SK, A-type K+ and HCN channels in hippocampal CA1 neurons. We anticipate this assay technique could aid in the detection of functional changes in specific ionic currents during drug screening, as well as in research targeting ion channel dysfunction.

Heterogeneous off-target impact of ion-channel deletion on intrinsic properties of hippocampal model neurons that self-regulate calcium

How do neurons that implement cell-autonomous self-regulation of calcium react to knockout of individual ion-channel conductances? To address this question, we used a heterogeneous population of 78 conductance-based models of hippocampal pyramidal neurons that maintained cell-autonomous calcium homeostasis while receiving theta-frequency inputs. At calcium steady-state, we individually deleted each of the 11 active ion-channel conductances from each model. We measured the acute impact of deleting each conductance (one at a time) by comparing intrinsic electrophysiological properties before and immediately after channel deletion. The acute impact of deleting individual conductances on physiological properties (including calcium homeostasis) was heterogeneous, depending on the property, the specific model, and the deleted channel. The underlying many-to-many mapping between ion channels and properties pointed to ion-channel degeneracy. Next, we allowed the other conductances (barring the deleted conductance) to evolve towards achieving calcium homeostasis during theta-frequency activity. When calcium homeostasis was perturbed by ion-channel deletion, post-knockout plasticity in other conductances ensured resilience of calcium homeostasis to ion-channel deletion. These results demonstrate degeneracy in calcium homeostasis, as calcium homeostasis in knockout models was implemented in the absence of a channel that was earlier involved in the homeostatic process. Importantly, in reacquiring homeostasis, ion-channel conductances and physiological properties underwent heterogenous plasticity (dependent on the model, the property, and the deleted channel), even introducing changes in properties that were not directly connected to the deleted channel. Together, post-knockout plasticity geared towards maintaining homeostasis introduced heterogenous off-target effects on several channels and properties, suggesting that extreme caution be exercised in interpreting experimental outcomes involving channel knockouts.

A Fundamental Inequality Governing the Rate Coding Response of Sensory Neurons

A fundamental inequality governing the spike activity of peripheral neurons is derived and tested against auditory data. This inequality states that the steady-state firing rate must lie between the arithmetic and geometric means of the spontaneous and peak activities during adaptation. Implications towards the development of auditory mechanistic models are explored.

On the scope of scientific hypotheses

Hypotheses are frequently the starting point when undertaking the empirical portion of the scientific process. They state something that the scientific process will attempt to evaluate, corroborate, verify or falsify. Their purpose is to guide the types of data we collect, analyses we conduct, and inferences we would like to make. Over the last decade, metascience has advocated for hypotheses being in preregistrations or registered reports, but how to formulate these hypotheses has received less attention. Here, we argue that hypotheses can vary in specificity along at least three independent dimensions: the relationship, the variables, and the pipeline. Together, these dimensions form the scope of the hypothesis. We demonstrate how narrowing the scope of a hypothesis in any of these three ways reduces the hypothesis space and that this reduction is a type of novelty. Finally, we discuss how this formulation of hypotheses can guide researchers to formulate the appropriate scope for their hypotheses and should aim for neither too broad nor too narrow a scope. This framework can guide hypothesis-makers when formulating their hypotheses by helping clarify what is being tested, chaining results to previous known findings, and demarcating what is explicitly tested in the hypothesis.

Uncovering the organization of neural circuits with Generalized Phase Locking Analysis

Despite the considerable progress of in vivo neural recording techniques, inferring the biophysical mechanisms underlying large scale coordination of brain activity from neural data remains challenging. One obstacle is the difficulty to link high dimensional functional connectivity measures to mechanistic models of network activity. We address this issue by investigating spike-field coupling (SFC) measurements, which quantify the synchronization between, on the one hand, the action potentials produced by neurons, and on the other hand mesoscopic "field" signals, reflecting subthreshold activities at possibly multiple recording sites. As the number of recording sites gets large, the amount of pairwise SFC measurements becomes overwhelmingly challenging to interpret. We develop Generalized Phase Locking Analysis (GPLA) as an interpretable dimensionality reduction of this multivariate SFC. GPLA describes the dominant coupling between field activity and neural ensembles across space and frequencies. We show that GPLA features are biophysically interpretable when used in conjunction with appropriate network models, such that we can identify the influence of underlying circuit properties on these features. We demonstrate the statistical benefits and interpretability of this approach in various computational models and Utah array recordings. The results suggest that GPLA, used jointly with biophysical modeling, can help uncover the contribution of recurrent microcircuits to the spatio-temporal dynamics observed in multi-channel experimental recordings.

On the Role of Theory and Modeling in Neuroscience

In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena that cross a wide range of scales and often require consideration at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative models and theories each play a distinct role in defining and bridging levels of abstraction, will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.

A general pattern of non-spiking neuron dynamics under the effect of potassium and calcium channel modifications

Electrical activity of excitable cells results from ion exchanges through cell membranes, so that genetic or epigenetic changes in genes encoding ion channels are likely to affect neuronal electrical signaling throughout the brain. There is a large literature on the effect of variations in ion channels on the dynamics of spiking neurons that represent the main type of neurons found in the vertebrate nervous systems. Nevertheless, non-spiking neurons are also ubiquitous in many nervous tissues and play a critical role in the processing of some sensory systems. To our knowledge, however, how conductance variations affect the dynamics of non-spiking neurons has never been assessed. Based on experimental observations reported in the biological literature and on mathematical considerations, we first propose a phenotypic classification of non-spiking neurons. Then, we determine a general pattern of the phenotypic evolution of non-spiking neurons as a function of changes in calcium and potassium conductances. Furthermore, we study the homeostatic compensatory mechanisms of ion channels in a well-posed non-spiking retinal cone model. We show that there is a restricted range of ion conductance values for which the behavior and phenotype of the neuron are maintained. Finally, we discuss the implications of the phenotypic changes of individual cells at the level of neuronal network functioning of the C. elegans worm and the retina, which are two non-spiking nervous tissues composed of neurons with various phenotypes.

Information geometry for multiparameter models: new perspectives on the origin of simplicity

Complex models in physics, biology, economics, and engineering are oftensloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades without significant changes in the predictions. This review uses information geometry to explore sloppiness and its deep relation to emergent theories. We introduce themodel manifoldof predictions, whose coordinates are the model parameters. Itshyperribbonstructure explains why only a few parameter combinations matter for the behavior. We review recent rigorous results that connect the hierarchy of hyperribbon widths to approximation theory, and to the smoothness of model predictions under changes of the control variables. We discuss recent geodesic methods to find simpler models on nearby boundaries of the model manifold-emergent theories with fewer parameters that explain the behavior equally well. We discuss a Bayesian prior which optimizes the mutual information between model parameters and experimental data, naturally favoring points on the emergent boundary theories and thus simpler models. We introduce a 'projected maximum likelihood' prior that efficiently approximates this optimal prior, and contrast both to the poor behavior of the traditional Jeffreys prior. We discuss the way the renormalization group coarse-graining in statistical mechanics introduces a flow of the model manifold, and connect stiff and sloppy directions along the model manifold with relevant and irrelevant eigendirections of the renormalization group. Finally, we discuss recently developed 'intensive' embedding methods, allowing one to visualize the predictions of arbitrary probabilistic models as low-dimensional projections of an isometric embedding, and illustrate our method by generating the model manifold of the Ising model.

Energy-efficient network activity from disparate circuit parameters

Neural circuits can produce similar activity patterns from vastly different combinations of channel and synaptic conductances. These conductances are tuned for specific activity patterns but might also reflect additional constraints, such as metabolic cost or robustness to perturbations. How do such constraints influence the range of permissible conductances? Here we investigate how metabolic cost affects the parameters of neural circuits with similar activity in a model of the pyloric network of the crab Cancer borealis . We present a machine learning method that can identify a range of network models that generate activity patterns matching experimental data and find that neural circuits can consume largely different amounts of energy despite similar circuit activity. Furthermore, a reduced but still significant range of circuit parameters gives rise to energy-efficient circuits. We then examine the space of parameters of energy-efficient circuits and identify potential tuning strategies for low metabolic cost. Finally, we investigate the interaction between metabolic cost and temperature robustness. We show that metabolic cost can vary across temperatures but that robustness to temperature changes does not necessarily incur an increased metabolic cost. Our analyses show that despite metabolic efficiency and temperature robustness constraining circuit parameters, neural systems can generate functional, efficient, and robust network activity with widely disparate sets of conductances.

Geometric framework to predict structure from function in neural networks

Neural computation in biological and artificial networks relies on the nonlinear summation of many inputs. The structural connectivity matrix of synaptic weights between neurons is a critical determinant of overall network function, but quantitative links between neural network structure and function are complex and subtle. For example, many networks can give rise to similar functional responses, and the same network can function differently depending on context. Whether certain patterns of synaptic connectivity are required to generate specific network-level computations is largely unknown. Here we introduce a geometric framework for identifying synaptic connections required by steady-state responses in recurrent networks of threshold-linear neurons. Assuming that the number of specified response patterns does not exceed the number of input synapses, we analytically calculate the solution space of all feedforward and recurrent connectivity matrices that can generate the specified responses from the network inputs. A generalization accounting for noise further reveals that the solution space geometry can undergo topological transitions as the allowed error increases, which could provide insight into both neuroscience and machine learning. We ultimately use this geometric characterization to derive certainty conditions guaranteeing a nonzero synapse between neurons. Our theoretical framework could thus be applied to neural activity data to make rigorous anatomical predictions that follow generally from the model architecture.

Noise-activated barrier crossing in multiattractor dissipative neural networks

Noise-activated transitions between coexisting attractors are investigated in a chaotic spiking network. At low noise level, attractor hopping consists of discrete bifurcation events that conserve the memory of initial conditions. When the escape probability becomes comparable to the intrabasin hopping probability, the lifetime of attractors is given by a detailed balance where the less coherent attractors act as a sink for the more coherent ones. In this regime, the escape probability follows an activation law allowing us to assign pseudoactivation energies to limit cycle attractors. These pseudoenergies introduce a useful metric for evaluating the resilience of biological rhythms to perturbations.

Approaches to Parameter Estimation from Model Neurons and Biological Neurons

Model optimization in neuroscience has focused on inferring intracellular parameters from time series observations of the membrane voltage and calcium concentrations. These parameters constitute the fingerprints of ion channel subtypes and may identify ion channel mutations from observed changes in electrical activity. A central question in neuroscience is whether computational methods may obtain ion channel parameters with sufficient consistency and accuracy to provide new information on the underlying biology. Finding single-valued solutions in particular, remains an outstanding theoretical challenge. This note reviews recent progress in the field. It first covers well-posed problems and describes the conditions that the model and data need to meet to warrant the recovery of all the original parameters—even in the presence of noise. The main challenge is model error, which reflects our lack of knowledge of exact equations. We report on strategies that have been partially successful at inferring the parameters of rodent and songbird neurons, when model error is sufficiently small for accurate predictions to be made irrespective of stimulation.

Multiscale Computer Model of the Spinal Dorsal Horn Reveals Changes in Network Processing Associated with Chronic Pain

Pain-related sensory input is processed in the spinal dorsal horn (SDH) before being relayed to the brain. That processing profoundly influences whether stimuli are correctly or incorrectly perceived as painful. Significant advances have been made in identifying the types of excitatory and inhibitory neurons that comprise the SDH, and there is some information about how neuron types are connected, but it remains unclear how the overall circuit processes sensory input or how that processing is disrupted under chronic pain conditions. To explore SDH function, we developed a computational model of the circuit that is tightly constrained by experimental data. Our model comprises conductance-based neuron models that reproduce the characteristic firing patterns of spinal neurons. Excitatory and inhibitory neuron populations, defined by their expression of genetic markers, spiking pattern, or morphology, were synaptically connected according to available qualitative data. Using a genetic algorithm, synaptic weights were tuned to reproduce projection neuron firing rates (model output) based on primary afferent firing rates (model input) across a range of mechanical stimulus intensities. Disparate synaptic weight combinations could produce equivalent circuit function, revealing degeneracy that may underlie heterogeneous responses of different circuits to perturbations or pathologic insults. To validate our model, we verified that it responded to the reduction of inhibition (i.e., disinhibition) and ablation of specific neuron types in a manner consistent with experiments. Thus validated, our model offers a valuable resource for interpreting experimental results and testing hypotheses in silico to plan experiments for examining normal and pathologic SDH circuit function.SIGNIFICANCE STATEMENT We developed a multiscale computer model of the posterior part of spinal cord gray matter (spinal dorsal horn), which is involved in perceiving touch and pain. The model reproduces several experimental observations and makes predictions about how specific types of spinal neurons and synapses influence projection neurons that send information to the brain. Misfiring of these projection neurons can produce anomalous sensations associated with chronic pain. Our computer model will not only assist in planning future experiments, but will also be useful for developing new pharmacotherapy for chronic pain disorders, connecting the effect of drugs acting at the molecular scale with emergent properties of neurons and circuits that shape the pain experience.

Reinforcement Learning for Central Pattern Generation in Dynamical Recurrent Neural Networks

Lifetime learning, or the change (or acquisition) of behaviors during a lifetime, based on experience, is a hallmark of living organisms. Multiple mechanisms may be involved, but biological neural circuits have repeatedly demonstrated a vital role in the learning process. These neural circuits are recurrent, dynamic, and non-linear and models of neural circuits employed in neuroscience and neuroethology tend to involve, accordingly, continuous-time, non-linear, and recurrently interconnected components. Currently, the main approach for finding configurations of dynamical recurrent neural networks that demonstrate behaviors of interest is using stochastic search techniques, such as evolutionary algorithms. In an evolutionary algorithm, these dynamic recurrent neural networks are evolved to perform the behavior over multiple generations, through selection, inheritance, and mutation, across a population of solutions. Although, these systems can be evolved to exhibit lifetime learning behavior, there are no explicit rules built into these dynamic recurrent neural networks that facilitate learning during their lifetime (e.g., reward signals). In this work, we examine a biologically plausible lifetime learning mechanism for dynamical recurrent neural networks. We focus on a recently proposed reinforcement learning mechanism inspired by neuromodulatory reward signals and ongoing fluctuations in synaptic strengths. Specifically, we extend one of the best-studied and most-commonly used dynamic recurrent neural networks to incorporate the reinforcement learning mechanism. First, we demonstrate that this extended dynamical system (model and learning mechanism) can autonomously learn to perform a central pattern generation task. Second, we compare the robustness and efficiency of the reinforcement learning rules in relation to two baseline models, a random walk and a hill-climbing walk through parameter space. Third, we systematically study the effect of the different meta-parameters of the learning mechanism on the behavioral learning performance. Finally, we report on preliminary results exploring the generality and scalability of this learning mechanism for dynamical neural networks as well as directions for future work.

Learning to represent continuous variables in heterogeneous neural networks

Animals must monitor continuous variables such as position or head direction. Manifold attractor networks-which enable a continuum of persistent neuronal states-provide a key framework to explain this monitoring ability. Neural networks with symmetric synaptic connectivity dominate this framework but are inconsistent with the diverse synaptic connectivity and neuronal representations observed in experiments. Here, we developed a theory for manifold attractors in trained neural networks, which approximates a continuum of persistent states, without assuming unrealistic symmetry. We exploit the theory to predict how asymmetries in the representation and heterogeneity in the connectivity affect the formation of the manifold via training, shape network response to stimulus, and govern mechanisms that possibly lead to destabilization of the manifold. Our work suggests that the functional properties of manifold attractors in the brain can be inferred from the overlooked asymmetries in connectivity and in the low-dimensional representation of the encoded variable.

Behavioral Neuroscience in the Era of Genomics: Tools and Lessons for Analyzing High-Dimensional Datasets

Behavioral neuroscience underwent a technology-driven revolution with the emergence of machine-vision and machine-learning technologies. These technological advances facilitated the generation of high-resolution, high-throughput capture and analysis of complex behaviors. Therefore, behavioral neuroscience is becoming a data-rich field. While behavioral researchers use advanced computational tools to analyze the resulting datasets, the search for robust and standardized analysis tools is still ongoing. At the same time, the field of genomics exploded with a plethora of technologies which enabled the generation of massive datasets. This growth of genomics data drove the emergence of powerful computational approaches to analyze these data. Here, we discuss the composition of a large behavioral dataset, and the differences and similarities between behavioral and genomics data. We then give examples of genomics-related tools that might be of use for behavioral analysis and discuss concepts that might emerge when considering the two fields together.

Interrogating theoretical models of neural computation with emergent property inference

A cornerstone of theoretical neuroscience is the circuit model: a system of equations that captures a hypothesized neural mechanism. Such models are valuable when they give rise to an experimentally observed phenomenon -- whether behavioral or a pattern of neural activity -- and thus can offer insights into neural computation. The operation of these circuits, like all models, critically depends on the choice of model parameters. A key step is then to identify the model parameters consistent with observed phenomena: to solve the inverse problem. In this work, we present a novel technique, emergent property inference (EPI), that brings the modern probabilistic modeling toolkit to theoretical neuroscience. When theorizing circuit models, theoreticians predominantly focus on reproducing computational properties rather than a particular dataset. Our method uses deep neural networks to learn parameter distributions with these computational properties. This methodology is introduced through a motivational example of parameter inference in the stomatogastric ganglion. EPI is then shown to allow precise control over the behavior of inferred parameters and to scale in parameter dimension better than alternative techniques. In the remainder of this work, we present novel theoretical findings in models of primary visual cortex and superior colliculus, which were gained through the examination of complex parametric structure captured by EPI. Beyond its scientific contribution, this work illustrates the variety of analyses possible once deep learning is harnessed towards solving theoretical inverse problems.

From Univariate to Multivariate Coupling Between Continuous Signals and Point Processes: A Mathematical Framework

Time series data sets often contain heterogeneous signals, composed of both continuously changing quantities and discretely occurring events. The coupling between these measurements may provide insights into key underlying mechanisms of the systems under study. To better extract this information, we investigate the asymptotic statistical properties of coupling measures between continuous signals and point processes. We first introduce martingale stochastic integration theory as a mathematical model for a family of statistical quantities that include the phase locking value, a classical coupling measure to characterize complex dynamics. Based on the martingale central limit theorem, we can then derive the asymptotic gaussian distribution of estimates of such coupling measure that can be exploited for statistical testing. Second, based on multivariate extensions of this result and random matrix theory, we establish a principled way to analyze the low-rank coupling between a large number of point processes and continuous signals. For a null hypothesis of no coupling, we establish sufficient conditions for the empirical distribution of squared singular values of the matrix to converge, as the number of measured signals increases, to the well-known Marchenko-Pastur (MP) law, and the largest squared singular value converges to the upper end of the MP support. This justifies a simple thresholding approach to assess the significance of multivariate coupling. Finally, we illustrate with simulations the relevance of our univariate and multivariate results in the context of neural time series, addressing how to reliably quantify the interplay between multichannel local field potential signals and the spiking activity of a large population of neurons.

Theoretical principles for illuminating sensorimotor processing with brain-wide neuronal recordings

Modern recording techniques now permit brain-wide sensorimotor circuits to be observed at single neuron resolution in small animals. Extracting theoretical understanding from these recordings requires principles that organize findings and guide future experiments. Here we review theoretical principles that shed light onto brain-wide sensorimotor processing. We begin with an analogy that conceptualizes principles as streetlamps that illuminate the empirical terrain, and we illustrate the analogy by showing how two familiar principles apply in new ways to brain-wide phenomena. We then focus the bulk of the review on describing three more principles that have wide utility for mapping brain-wide neural activity, making testable predictions from highly parameterized mechanistic models, and investigating the computational determinants of neuronal response patterns across the brain.

Co-expression of calcium and hERG potassium channels reduces the incidence of proarrhythmic events

AIMS

Cardiac electrical activity is extraordinarily robust. However, when it goes wrong it can have fatal consequences. Electrical activity in the heart is controlled by the carefully orchestrated activity of more than a dozen different ion conductances. While there is considerable variability in cardiac ion channel expression levels between individuals, studies in rodents have indicated that there are modules of ion channels whose expression co-vary. The aim of this study was to investigate whether meta-analytic co-expression analysis of large-scale gene expression datasets could identify modules of co-expressed cardiac ion channel genes in human hearts that are of functional importance.

METHODS AND RESULTS

Meta-analysis of 3653 public human RNA-seq datasets identified a strong correlation between expression of CACNA1C (L-type calcium current, ICaL) and KCNH2 (rapid delayed rectifier K+ current, IKr), which was also observed in human adult heart tissue samples. In silico modelling suggested that co-expression of CACNA1C and KCNH2 would limit the variability in action potential duration seen with variations in expression of ion channel genes and reduce susceptibility to early afterdepolarizations, a surrogate marker for proarrhythmia. We also found that levels of KCNH2 and CACNA1C expression are correlated in human-induced pluripotent stem cell-derived cardiac myocytes and the levels of CACNA1C and KCNH2 expression were inversely correlated with the magnitude of changes in repolarization duration following inhibition of IKr.

CONCLUSION

Meta-analytic approaches of multiple independent human gene expression datasets can be used to identify gene modules that are important for regulating heart function. Specifically, we have verified that there is co-expression of CACNA1C and KCNH2 ion channel genes in human heart tissue, and in silico analyses suggest that CACNA1C-KCNH2 co-expression increases the robustness of cardiac electrical activity.

Integration of Within-Cell Experimental Data With Multi-Compartmental Modeling Predicts H-Channel Densities and Distributions in Hippocampal OLM Cells

Determining biophysical details of spatially extended neurons is a challenge that needs to be overcome if we are to understand the dynamics of brain function from cellular perspectives. Moreover, we now know that we should not average across recordings from many cells of a given cell type to obtain quantitative measures such as conductance since measures can vary multiple-fold for a given cell type. In this work we examine whether a tight combination of experimental and computational work can address this challenge. The oriens-lacunosum/moleculare (OLM) interneuron operates as a “gate” that controls incoming sensory and ongoing contextual information in the CA1 of the hippocampus, making it essential to understand how its biophysical properties contribute to memory function. OLM cells fire phase-locked to the prominent hippocampal theta rhythms, and we previously used computational models to show that OLM cells exhibit high or low theta spiking resonance frequencies that depend respectively on whether their dendrites have hyperpolarization-activated cation channels (h-channels) or not. However, whether OLM cells actually possess dendritic h-channels is unknown at present. We performed a set of whole-cell recordings of OLM cells from mouse hippocampus and constructed three multi-compartment models using morphological and electrophysiological parameters extracted from the same OLM cell, including per-cell pharmacologically isolated h-channel currents. We found that the models best matched experiments when h-channels were present in the dendrites of each of the three model cells created. This strongly suggests that h-channels must be present in OLM cell dendrites and are not localized to their somata. Importantly, this work shows that a tight integration of model and experiment can help tackle the challenge of characterizing biophysical details and distributions in spatially extended neurons. Full spiking models were built for two of the OLM cells, matching their current clamp cell-specific electrophysiological recordings. Overall, our work presents a technical advancement in modeling OLM cells. Our models are available to the community to use to gain insight into cellular dynamics underlying hippocampal function.

Training deep neural density estimators to identify mechanistic models of neural dynamics

Mechanistic modeling in neuroscience aims to explain observed phenomena in terms of underlying causes. However, determining which model parameters agree with complex and stochastic neural data presents a significant challenge. We address this challenge with a machine learning tool which uses deep neural density estimators-trained using model simulations-to carry out Bayesian inference and retrieve the full space of parameters compatible with raw data or selected data features. Our method is scalable in parameters and data features and can rapidly analyze new data after initial training. We demonstrate the power and flexibility of our approach on receptive fields, ion channels, and Hodgkin-Huxley models. We also characterize the space of circuit configurations giving rise to rhythmic activity in the crustacean stomatogastric ganglion, and use these results to derive hypotheses for underlying compensation mechanisms. Our approach will help close the gap between data-driven and theory-driven models of neural dynamics.

Estimation of neuron parameters from imperfect observations

The estimation of parameters controlling the electrical properties of biological neurons is essential to determine their complement of ion channels and to understand the function of biological circuits. By synchronizing conductance models to time series observations of the membrane voltage, one may construct models capable of predicting neuronal dynamics. However, identifying the actual set of parameters of biological ion channels remains a formidable theoretical challenge. Here, we present a regularization method that improves convergence towards this optimal solution when data are noisy and the model is unknown. Our method relies on the existence of an offset in parameter space arising from the interplay between model nonlinearity and experimental error. By tuning this offset, we induce saddle-node bifurcations from sub-optimal to optimal solutions. This regularization method increases the probability of finding the optimal set of parameters from 67% to 94.3%. We also reduce parameter correlations by implementing adaptive sampling and stimulation protocols compatible with parameter identifiability requirements. Our results show that the optimal model parameters may be inferred from imperfect observations provided the conditions of observability and identifiability are fulfilled.

Neuronal population model of globular bushy cells covering unit-to-unit variability

Computations of acoustic information along the central auditory pathways start in the cochlear nucleus. Bushy cells in the anteroventral cochlear nucleus, which innervate monaural and binaural stations in the superior olivary complex, process and transfer temporal cues relevant for sound localization. These cells are categorized into two groups: spherical and globular bushy cells (SBCs/GBCs). Spontaneous rates of GBCs innervated by multiple auditory nerve (AN) fibers are generally lower than those of SBCs that receive a small number of large AN synapses. In response to low-frequency tonal stimulation, both types of bushy cells show improved phase-locking and entrainment compared to AN fibers. When driven by high-frequency tones, GBCs show primary-like-with-notch or onset-L peristimulus time histograms and relatively irregular spiking. However, previous in vivo physiological studies of bushy cells also found considerable unit-to-unit variability in these response patterns. Here we present a population of models that can simulate the observed variation in GBCs. We used a simple coincidence detection model with an adaptive threshold and systematically varied its six parameters. Out of 567000 parameter combinations tested, 7520 primary-like-with-notch models and 4094 onset-L models were selected that satisfied a set of physiological criteria for a GBC unit. Analyses of the model parameters and output measures revealed that the parameters of the accepted model population are weakly correlated with each other to retain major GBC properties, and that the output spiking patterns of the model are affected by a combination of multiple parameters. Simulations of frequency-dependent temporal properties of the model GBCs showed a reasonable fit to empirical data, supporting the validity of our population modeling. The computational simplicity and efficiency of the model structure makes our approach suitable for future large-scale simulations of binaural information processing that may involve thousands of GBC units.

In the auditory system, specialized neuronal circuits process various types of acoustic information. A group of neurons, called globular bushy cells (GBCs), faithfully transfer timing information of acoustic signals to their downstream neurons responsible for the perception of sound location. Previous physiological studies found representative activity patterns of GBCs, but with substantial individual variations among them. In this study, we present a population of models, instead of creating one best model, to account for the observed variations of GBCs. We varied all six parameters of a simple auditory neuron model and selected the combinations of parameters that led to acceptable activity patterns of GBCs. In total, we tested more than half a million combinations and accepted ~11600 GBC models. Temporal spiking patterns of real GBCs depend on the sound frequency, and our model population was able to replicate this trend. The model used here is computationally efficient and can thus serve as a building block for future large-scale simulations of auditory information processing.

A Model of Brain Folding Based on Strong Local and Weak Long-Range Connectivity Requirements

Throughout the animal kingdom, the structure of the central nervous system varies widely from distributed ganglia in worms to compact brains with varying degrees of folding in mammals. The differences in structure may indicate a fundamentally different circuit organization. However, the folded brain most likely is a direct result of mechanical forces when considering that a larger surface area of cortex packs into the restricted volume provided by the skull. Here, we introduce a computational model that instead of modeling mechanical forces relies on dimension reduction methods to place neurons according to specific connectivity requirements. For a simplified connectivity with strong local and weak long-range connections, our model predicts a transition from separate ganglia through smooth brain structures to heavily folded brains as the number of cortical columns increases. The model reproduces experimentally determined relationships between metrics of cortical folding and its pathological phenotypes in lissencephaly, polymicrogyria, microcephaly, autism, and schizophrenia. This suggests that mechanical forces that are known to lead to cortical folding may synergistically contribute to arrangements that reduce wiring. Our model provides a unified conceptual understanding of gyrification linking cellular connectivity and macroscopic structures in large-scale neural network models of the brain.

Optimal solid state neurons

Bioelectronic medicine is driving the need for neuromorphic microcircuits that integrate raw nervous stimuli and respond identically to biological neurons. However, designing such circuits remains a challenge. Here we estimate the parameters of highly nonlinear conductance models and derive the ab initio equations of intracellular currents and membrane voltages embodied in analog solid-state electronics. By configuring individual ion channels of solid-state neurons with parameters estimated from large-scale assimilation of electrophysiological recordings, we successfully transfer the complete dynamics of hippocampal and respiratory neurons in silico. The solid-state neurons are found to respond nearly identically to biological neurons under stimulation by a wide range of current injection protocols. The optimization of nonlinear models demonstrates a powerful method for programming analog electronic circuits. This approach offers a route for repairing diseased biocircuits and emulating their function with biomedical implants that can adapt to biofeedback.

Brian 2, an intuitive and efficient neural simulator

Brian 2 allows scientists to simply and efficiently simulate spiking neural network models. These models can feature novel dynamical equations, their interactions with the environment, and experimental protocols. To preserve high performance when defining new models, most simulators offer two options: low-level programming or description languages. The first option requires expertise, is prone to errors, and is problematic for reproducibility. The second option cannot describe all aspects of a computational experiment, such as the potentially complex logic of a stimulation protocol. Brian addresses these issues using runtime code generation. Scientists write code with simple and concise high-level descriptions, and Brian transforms them into efficient low-level code that can run interleaved with their code. We illustrate this with several challenging examples: a plastic model of the pyloric network, a closed-loop sensorimotor model, a programmatic exploration of a neuron model, and an auditory model with real-time input.

Parameter estimation and identifiability in a neural population model for electro-cortical activity

Electroencephalography (EEG) provides a non-invasive measure of brain electrical activity. Neural population models, where large numbers of interacting neurons are considered collectively as a macroscopic system, have long been used to understand features in EEG signals. By tuning dozens of input parameters describing the excitatory and inhibitory neuron populations, these models can reproduce prominent features of the EEG such as the alpha-rhythm. However, the inverse problem, of directly estimating the parameters from fits to EEG data, remains unsolved. Solving this multi-parameter non-linear fitting problem will potentially provide a real-time method for characterizing average neuronal properties in human subjects. Here we perform unbiased fits of a 22-parameter neural population model to EEG data from 82 individuals, using both particle swarm optimization and Markov chain Monte Carlo sampling. We estimate how much is learned about individual parameters by computing Kullback-Leibler divergences between posterior and prior distributions for each parameter. Results indicate that only a single parameter, that determining the dynamics of inhibitory synaptic activity, is directly identifiable, while other parameters have large, though correlated, uncertainties. We show that the eigenvalues of the Fisher information matrix are roughly uniformly spaced over a log scale, indicating that the model is sloppy, like many of the regulatory network models in systems biology. These eigenvalues indicate that the system can be modeled with a low effective dimensionality, with inhibitory synaptic activity being prominent in driving system behavior.

The roles of supervised machine learning in systems neuroscience

Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four primary roles of ML within neuroscience: (1) creating solutions to engineering problems, (2) identifying predictive variables, (3) setting benchmarks for simple models of the brain, and (4) serving itself as a model for the brain. The breadth and ease of its applicability suggests that machine learning should be in the toolbox of most systems neuroscientists.

A Proposed Mechanism for Spontaneous Transitions between Interictal and Ictal Activity

Epileptic networks are characterized by two outputs: brief interictal spikes and rarer, more prolonged seizures. Although either output state is readily modeled in silico and induced experimentally, the transition mechanisms are unknown, in part because no models exhibit both output states spontaneously. In silico small-world neural networks were built using single-compartment neurons whose physiological parameters were derived from dual whole-cell recordings of pyramidal cells in organotypic hippocampal slice cultures that were generating spontaneous seizure-like activity. In silico, neurons were connected by abundant local synapses and rare long-distance synapses. Activity-dependent synaptic depression and gradual recovery delimited synchronous activity. Full synaptic recovery engendered interictal population spikes that spread via long-distance synapses. When synaptic recovery was incomplete, postsynaptic neurons required coincident activation of multiple presynaptic terminals to reach firing threshold. Only local connections were sufficiently dense to spread activity under these conditions. This coalesced network activity into traveling waves whose velocity varied with synaptic recovery. Seizures were comprised of sustained traveling waves that were similar to those recorded during experimental and human neocortical seizures. Sustained traveling waves occurred only when wave velocity, network dimensions, and the rate of synaptic recovery enabled wave reentry into previously depressed areas at precisely ictogenic levels of synaptic recovery. Wide-field, cellular-resolution GCamP7b calcium imaging demonstrated similar initial patterns of activation in the hippocampus, although the anatomical distribution of traveling waves of synaptic activation was altered by the pattern of synaptic connectivity in the organotypic hippocampal cultures.SIGNIFICANCE STATEMENT When computerized distributed neural network models are required to generate both features of epileptic networks (i.e., spontaneous interictal population spikes and seizures), the network structure is substantially constrained. These constraints provide important new hypotheses regarding the nature of epileptic networks and mechanisms of seizure onset.

Cellular function given parametric variation in the Hodgkin and Huxley model of excitability

How is reliable physiological function maintained in cells despite considerable variability in the values of key parameters of multiple interacting processes that govern that function? Here, we use the classic Hodgkin-Huxley formulation of the squid giant axon action potential to propose a possible approach to this problem. Although the full Hodgkin-Huxley model is very sensitive to fluctuations that independently occur in its many parameters, the outcome is in fact determined by simple combinations of these parameters along two physiological dimensions: structural and kinetic (denoted S and K, respectively). Structural parameters describe the properties of the cell, including its capacitance and the densities of its ion channels. Kinetic parameters are those that describe the opening and closing of the voltage-dependent conductances. The impacts of parametric fluctuations on the dynamics of the system-seemingly complex in the high-dimensional representation of the Hodgkin-Huxley model-are tractable when examined within the S-K plane. We demonstrate that slow inactivation, a ubiquitous activity-dependent feature of ionic channels, is a powerful local homeostatic control mechanism that stabilizes excitability amid changes in structural and kinetic parameters.

Maximizing the information learned from finite data selects a simple model

We use the language of uninformative Bayesian prior choice to study the selection of appropriately simple effective models. We advocate for the prior which maximizes the mutual information between parameters and predictions, learning as much as possible from limited data. When many parameters are poorly constrained by the available data, we find that this prior puts weight only on boundaries of the parameter space. Thus, it selects a lower-dimensional effective theory in a principled way, ignoring irrelevant parameter directions. In the limit where there are sufficient data to tightly constrain any number of parameters, this reduces to the Jeffreys prior. However, we argue that this limit is pathological when applied to the hyperribbon parameter manifolds generic in science, because it leads to dramatic dependence on effects invisible to experiment.

Beyond excitation/inhibition imbalance in multidimensional models of neural circuit changes in brain disorders

A leading theory holds that neurodevelopmental brain disorders arise from imbalances in excitatory and inhibitory (E/I) brain circuitry. However, it is unclear whether this one-dimensional model is rich enough to capture the multiple neural circuit alterations underlying brain disorders. Here, we combined computational simulations with analysis of in vivo two-photon Ca²⁺ imaging data from somatosensory cortex of Fmr1 knock-out (KO) mice, a model of Fragile-X Syndrome, to test the E/I imbalance theory. We found that: (1) The E/I imbalance model cannot account for joint alterations in the observed neural firing rates and correlations; (2) Neural circuit function is vastly more sensitive to changes in some cellular components over others; (3) The direction of circuit alterations in Fmr1 KO mice changes across development. These findings suggest that the basic E/I imbalance model should be updated to higher dimensional models that can better capture the multidimensional computational functions of neural circuits.

In many brain disorders, from autism to schizophrenia, the anatomy of the brain appears remarkably unchanged. This implies that the problem may reside in how neurons communicate with one another. Unfortunately, neuroscientists know little about how brain activity might differ from normal in these disorders, or how specific changes in activity give rise to symptoms. One leading theory, first proposed over a decade ago, is that these disorders reflect an imbalance in the activity of excitatory and inhibitory neurons. Excitatory neurons activate their targets, whereas inhibitory neurons suppress or silence them. While studies in mice have lent support to this theory, they have not yet culminated in new treatments for brain disorders.

One limitation of the excitation-inhibition imbalance theory is that it is one-dimensional. It assumes that there is an optimal balance of excitation and inhibition, and that brain disorders can be arranged in an imaginary line on either side of this optimum. Disorders to the right of the optimum, such as epilepsy and some forms of autism, feature too much excitation. Disorders to the left, such as the developmental disorder Rett syndrome, feature too much inhibition. But can diverse brain disorders really be classified on the basis of a single property, or do scientists need to consider other factors?

To find out, O’Donnell et al. analyzed recordings of brain activity from genetically modified mice with the mutation that causes fragile X syndrome, the most common form of inherited learning disability and autism. The mice showed changes in their overall brain activity compared to control animals. Their neurons also tended to fire in a more synchronized manner. A computer simulation revealed that an imbalance in excitation and inhibition alone could not explain these changes. Yet, a more complex simulation incorporating extra properties of neural circuits did a better job of explaining the altered neural activity seen in the mice.

O’Donnell et al. propose that this more advanced multi-dimensional model of changes in neural circuits could be used to screen candidate drugs before testing them in patients. In principle, the model could even help with designing drugs or other interventions by making it easier for researchers to target more precisely the changes in neural circuits that occur in brain disorders.

Learning from large-scale neural simulations

Large-scale neural simulations have the marks of a distinct methodology which can be fruitfully deployed in neuroscience. I distinguish between two types of applications of the simulation methodology in neuroscientific research. Model-oriented applications aim to use the simulation outputs to derive new hypotheses about brain organization and functioning and thus to extend current theoretical knowledge and understanding in the field. Data-oriented applications of the simulation methodology target the collection and analysis of data relevant for neuroscientific research that is inaccessible via more traditional experimental methods. I argue for a two-stage evaluation schema which helps clarify the differences and similarities between three current large-scale simulation projects pursued in neuroscience.

Disentangling neural cell diversity using single-cell transcriptomics

Cellular specialization is particularly prominent in mammalian nervous systems, which are composed of millions to billions of neurons that appear in thousands of different 'flavors' and contribute to a variety of functions. Even in a single brain region, individual neurons differ greatly in their morphology, connectivity and electrophysiological properties. Systematic classification of all mammalian neurons is a key goal towards deconstructing the nervous system into its basic components. With the recent advances in single-cell gene expression profiling technologies, it is now possible to undertake the enormous task of disentangling neuronal heterogeneity. High-throughput single-cell RNA sequencing and multiplexed quantitative RT-PCR have become more accessible, and these technologies enable systematic categorization of individual neurons into groups with similar molecular properties. Here we provide a conceptual and practical guide to classification of neural cell types using single-cell gene expression profiling technologies.

From Whole-Brain Data to Functional Circuit Models: The Zebrafish Optomotor Response

Detailed descriptions of brain-scale sensorimotor circuits underlying vertebrate behavior remain elusive. Recent advances in zebrafish neuroscience offer new opportunities to dissect such circuits via whole-brain imaging, behavioral analysis, functional perturbations, and network modeling. Here, we harness these tools to generate a brain-scale circuit model of the optomotor response, an orienting behavior evoked by visual motion. We show that such motion is processed by diverse neural response types distributed across multiple brain regions. To transform sensory input into action, these regions sequentially integrate eye- and direction-specific sensory streams, refine representations via interhemispheric inhibition, and demix locomotor instructions to independently drive turning and forward swimming. While experiments revealed many neural response types throughout the brain, modeling identified the dimensions of functional connectivity most critical for the behavior. We thus reveal how distributed neurons collaborate to generate behavior and illustrate a paradigm for distilling functional circuit models from whole-brain data.

Neural Population Dynamics during Reaching Are Better Explained by a Dynamical System than Representational Tuning

Recent models of movement generation in motor cortex have sought to explain neural activity not as a function of movement parameters, known as representational models, but as a dynamical system acting at the level of the population. Despite evidence supporting this framework, the evaluation of representational models and their integration with dynamical systems is incomplete in the literature. Using a representational velocity-tuning based simulation of center-out reaching, we show that incorporating variable latency offsets between neural activity and kinematics is sufficient to generate rotational dynamics at the level of neural populations, a phenomenon observed in motor cortex. However, we developed a covariance-matched permutation test (CMPT) that reassigns neural data between task conditions independently for each neuron while maintaining overall neuron-to-neuron relationships, revealing that rotations based on the representational model did not uniquely depend on the underlying condition structure. In contrast, rotations based on either a dynamical model or motor cortex data depend on this relationship, providing evidence that the dynamical model more readily explains motor cortex activity. Importantly, implementing a recurrent neural network we demonstrate that both representational tuning properties and rotational dynamics emerge, providing evidence that a dynamical system can reproduce previous findings of representational tuning. Finally, using motor cortex data in combination with the CMPT, we show that results based on small numbers of neurons or conditions should be interpreted cautiously, potentially informing future experimental design. Together, our findings reinforce the view that representational models lack the explanatory power to describe complex aspects of single neuron and population level activity.

The question of how the brain generates movement has been extensively studied, yet multiple competing models exist. Representational approaches relate the activity of single neurons to movement parameters such as velocity and position, approaches useful for the decoding of movement intentions, while the dynamical systems approach predicts that neural activity should evolve in a predictable way based on population activity. Existing representational models cannot reproduce the recent finding in monkeys that predictable rotational patterns underlie motor cortex activity during reach initiation, a finding predicted by a dynamical model in which muscle activity is a direct combination of neural population rotations. However, previous simulations did not consider an essential aspect of representational models: variable time offsets between neurons and kinematics. Whereas these offsets reveal rotational patterns in the model, these rotations are statistically different from those observed in the brain and predicted by a dynamical model. Importantly, a simple recurrent neural network model also showed rotational patterns statistically similar to those observed in the brain, supporting the idea that dynamical systems-based approaches may provide a powerful explanation of motor cortex function.

Automatic Construction of Predictive Neuron Models through Large Scale Assimilation of Electrophysiological Data

We report on the construction of neuron models by assimilating electrophysiological data with large-scale constrained nonlinear optimization. The method implements interior point line parameter search to determine parameters from the responses to intracellular current injections of zebra finch HVC neurons. We incorporated these parameters into a nine ionic channel conductance model to obtain completed models which we then use to predict the state of the neuron under arbitrary current stimulation. Each model was validated by successfully predicting the dynamics of the membrane potential induced by 20–50 different current protocols. The dispersion of parameters extracted from different assimilation windows was studied. Differences in constraints from current protocols, stochastic variability in neuron output, and noise behave as a residual temperature which broadens the global minimum of the objective function to an ellipsoid domain whose principal axes follow an exponentially decaying distribution. The maximum likelihood expectation of extracted parameters was found to provide an excellent approximation of the global minimum and yields highly consistent kinetics for both neurons studied. Large scale assimilation absorbs the intrinsic variability of electrophysiological data over wide assimilation windows. It builds models in an automatic manner treating all data as equal quantities and requiring minimal additional insight.

Emergence and maintenance of excitability: kinetics over structure

The capacity to generate action potentials in neurons and other excitable cells requires tuning of both ionic channel expression and kinetics in a large parameter space. Alongside studies that extend traditional focus on control-based regulation of structural parameters (channel densities), there is a budding interest in self-organization of kinetic parameters. In this picture, ionic channels are continually forced by activity in-and-out of a pool of states not available for the mechanism of excitability. The process, acting on expressed structure, provides a bed for generation of a spectrum of excitability modes. Driven by microscopic fluctuations over a broad range of temporal scales, self-organization of kinetic parameters enriches the concepts and tools used in the study of development of excitability.

Genetic variants in Alzheimer disease - molecular and brain network approaches

Genetic studies in late-onset Alzheimer disease (LOAD) are aimed at identifying core disease mechanisms and providing potential biomarkers and drug candidates to improve clinical care of AD. However, owing to the complexity of LOAD, including pathological heterogeneity and disease polygenicity, extraction of actionable guidance from LOAD genetics has been challenging. Past attempts to summarize the effects of LOAD-associated genetic variants have used pathway analysis and collections of small-scale experiments to hypothesize functional convergence across several variants. In this Review, we discuss how the study of molecular, cellular and brain networks provides additional information on the effects of LOAD-associated genetic variants. We then discuss emerging combinations of these omic data sets into multiscale models, which provide a more comprehensive representation of the effects of LOAD-associated genetic variants at multiple biophysical scales. Furthermore, we highlight the clinical potential of mechanistically coupling genetic variants and disease phenotypes with multiscale brain models.

The Development and Analysis of Integrated Neuroscience Data

There is a strong emphasis on developing novel neuroscience technologies, in particular on recording from more neurons. There has thus been increasing discussion about how to analyze the resulting big datasets. What has received less attention is that over the last 30 years, papers in neuroscience have progressively integrated more approaches, such as electrophysiology, anatomy, and genetics. As such, there has been little discussion on how to combine and analyze this multimodal data. Here, we describe the growth of multimodal approaches, and discuss the needed analysis advancements to make sense of this data.