Grounding neuroscience in behavioral changes using artificial neural networks

Connecting neural activity to function is a common aim in neuroscience. How to define and conceptualize function, however, can vary. Here I focus on grounding this goal in the specific question of how a given change in behavior is produced by a change in neural circuits or activity. Artificial neural network models offer a particularly fruitful format for tackling such questions because they use neural mechanisms to perform complex transformations and produce appropriate behavior. Therefore, they can be a means of causally testing the extent to which a neural change can be responsible for an experimentally observed behavioral change. Furthermore, because the field of interpretability in artificial intelligence has similar aims, neuroscientists can look to interpretability methods for new ways of identifying neural features that drive performance and behaviors.

Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

The neuroconnectionist research programme

Artificial neural networks (ANNs) inspired by biology are beginning to be widely used to model behavioural and neural data, an approach we call 'neuroconnectionism'. ANNs have been not only lauded as the current best models of information processing in the brain but also criticized for failing to account for basic cognitive functions. In this Perspective article, we propose that arguing about the successes and failures of a restricted set of current ANNs is the wrong approach to assess the promise of neuroconnectionism for brain science. Instead, we take inspiration from the philosophy of science, and in particular from Lakatos, who showed that the core of a scientific research programme is often not directly falsifiable but should be assessed by its capacity to generate novel insights. Following this view, we present neuroconnectionism as a general research programme centred around ANNs as a computational language for expressing falsifiable theories about brain computation. We describe the core of the programme, the underlying computational framework and its tools for testing specific neuroscientific hypotheses and deriving novel understanding. Taking a longitudinal view, we review past and present neuroconnectionist projects and their responses to challenges and argue that the research programme is highly progressive, generating new and otherwise unreachable insights into the workings of the brain.

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNs in vision research beyond basic object recognition.

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNs in vision research beyond basic object recognition.

Corrigendum: Attention in Psychology, Neuroscience, and Machine Learning

[This corrects the article DOI: 10.3389/fncom.2020.00029.].

Attention in Psychology, Neuroscience, and Machine Learning

Attention is the important ability to flexibly control limited computational resources. It has been studied in conjunction with many other topics in neuroscience and psychology including awareness, vigilance, saliency, executive control, and learning. It has also recently been applied in several domains in machine learning. The relationship between the study of biological attention and its use as a tool to enhance artificial neural networks is not always clear. This review starts by providing an overview of how attention is conceptualized in the neuroscience and psychology literature. It then covers several use cases of attention in machine learning, indicating their biological counterparts where they exist. Finally, the ways in which artificial attention can be further inspired by biology for the production of complex and integrative systems is explored.

A deep learning framework for neuroscience

Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, cognitive and motor tasks. Conversely, artificial intelligence attempts to design computational systems based on the tasks they will have to solve. In artificial neural networks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.

How biological attention mechanisms improve task performance in a large-scale visual system model

How does attentional modulation of neural activity enhance performance? Here we use a deep convolutional neural network as a large-scale model of the visual system to address this question. We model the feature similarity gain model of attention, in which attentional modulation is applied according to neural stimulus tuning. Using a variety of visual tasks, we show that neural modulations of the kind and magnitude observed experimentally lead to performance changes of the kind and magnitude observed experimentally. We find that, at earlier layers, attention applied according to tuning does not successfully propagate through the network, and has a weaker impact on performance than attention applied according to values computed for optimally modulating higher areas. This raises the question of whether biological attention might be applied at least in part to optimize function rather than strictly according to tuning. We suggest a simple experiment to distinguish these alternatives.

Enhanced ganglionic responses to substance P in spontaneously hypertensive rats

Intravenous injection of substance P (SP) increases blood pressure in normotensive rats by stimulating sympathetic ganglia. This study compared the effects of SP to increase renal nerve firing and blood pressure in normotensive and hypertensive rats treated with chlorisondamine. The increase in renal nerve firing was greatest in spontaneously hypertensive rats (SHR), intermediate in Wistar rats, and least in Wistar-Kyoto (WKY) rats. Blood pressure was increased more in SHR than in Wistar rats. Blood pressure was not increased in WKY rats. Responses to the ganglionic stimulant 1,1-dimethyl-4-phenylpiperazinium were the same in the three strains. These results suggest that there is a selective increase in the action of SP on sympathetic ganglia of SHR and that ganglion responsiveness to SP is correlated with its effect on blood pressure.

cAMP and in vitro inotropic actions of secretin and VIP in rat papillary muscle

Secretin and VIP stimulate cardiac adenylyl cyclase activity and exert a positive inotropic action in several mammalian species. This study examined positive inotropic activity and cAMP levels in rat papillary muscle. Isoproterenol and secretin increased contractions by 150+/-31% and 129+/-27%, respectively. VIP increased contraction by 30+/-21% only at 10 microM. Isoproterenol significantly increased cAMP levels by 82%, whereas increases by secretin (58%) and VIP (56%) were not significant. These results are consistent with reports that secretin and VIP stimulate cardiac adenylyl cyclase in the rat, but suggest that cAMP tissue levels cannot totally explain the positive inotropic responses to secretin and VIP.

Pressor and tachycardic responses to intravenous substance P in anesthetized rats

Intravenous injection of 3-33 nmol/kg of substance P (SP) caused pressor and tachycardic responses in anesthetized rats. The responses were not blocked by a ganglion nicotinic receptor antagonist or by pithing. Pretreatment with reserpine blocked both responses. beta-Adrenoceptor blockade attenuated only the tachycardic response, and alpha-adrenoceptor blockade attenuated only the pressor response. These findings indicated that the effects of SP to increase blood pressure and heart rate are due to sympathetic ganglion stimulation. Studies with adrenalectomized rats showed that stimulation of the adrenals by SP contributes to both responses but makes a greater contribution to the tachycardic response. These observations raise the possibility that the tachykinin innervation of sympathetic ganglia and the adrenal medulla may be involved in the local regulation of blood pressure and heart rate.