Modeling signal transduction leading to synaptic plasticity: evaluation and comparison of five models

Comparison of rule- and ordinary differential equation-based dynamic model of DARPP-32 signalling network

Dynamic modelling has considerably improved our understanding of complex molecular mechanisms. Ordinary differential equations (ODEs) are the most detailed and popular approach to modelling the dynamics of molecular systems. However, their application in signalling networks, characterised by multi-state molecular complexes, can be prohibitive. Contemporary modelling methods, such as rule- based (RB) modelling, have addressed these issues. The advantages of RB modelling over ODEs have been presented and discussed in numerous reviews. In this study, we conduct a direct comparison of the time courses of a molecular system founded on the same reaction network but encoded in the two frameworks. To make such a comparison, a set of reactions that underlie an ODE model was manually encoded in the Kappa language, one of the RB implementations. A comparison of the models was performed at the level of model specification and dynamics, acquired through model simulations. In line with previous reports, we confirm that the Kappa model recapitulates the general dynamics of its ODE counterpart with minor differences. These occur when molecules have multiple sites binding the same interactor. Furthermore, activation of these molecules in the RB model is slower than in the ODE one. As reported for other molecular systems, we find that, also for the DARPP-32 reaction network, the RB representation offers a more expressive and flexible syntax that facilitates access to fine details of the model, easing model reuse. In parallel with these analyses, we report a refactored model of the DARPP-32 interaction network that can serve as a canvas for the development of more complex dynamic models to study this important molecular system.

Replicability or reproducibility? On the replication crisis in computational neuroscience and sharing only relevant detail

Replicability and reproducibility of computational models has been somewhat understudied by "the replication movement." In this paper, we draw on methodological studies into the replicability of psychological experiments and on the mechanistic account of explanation to analyze the functions of model replications and model reproductions in computational neuroscience. We contend that model replicability, or independent researchers' ability to obtain the same output using original code and data, and model reproducibility, or independent researchers' ability to recreate a model without original code, serve different functions and fail for different reasons. This means that measures designed to improve model replicability may not enhance (and, in some cases, may actually damage) model reproducibility. We claim that although both are undesirable, low model reproducibility poses more of a threat to long-term scientific progress than low model replicability. In our opinion, low model reproducibility stems mostly from authors' omitting to provide crucial information in scientific papers and we stress that sharing all computer code and data is not a solution. Reports of computational studies should remain selective and include all and only relevant bits of code.

Challenges in Reproducibility, Replicability, and Comparability of Computational Models and Tools for Neuronal and Glial Networks, Cells, and Subcellular Structures

The possibility to replicate and reproduce published research results is one of the biggest challenges in all areas of science. In computational neuroscience, there are thousands of models available. However, it is rarely possible to reimplement the models based on the information in the original publication, let alone rerun the models just because the model implementations have not been made publicly available. We evaluate and discuss the comparability of a versatile choice of simulation tools: tools for biochemical reactions and spiking neuronal networks, and relatively new tools for growth in cell cultures. The replicability and reproducibility issues are considered for computational models that are equally diverse, including the models for intracellular signal transduction of neurons and glial cells, in addition to single glial cells, neuron-glia interactions, and selected examples of spiking neuronal networks. We also address the comparability of the simulation results with one another to comprehend if the studied models can be used to answer similar research questions. In addition to presenting the challenges in reproducibility and replicability of published results in computational neuroscience, we highlight the need for developing recommendations and good practices for publishing simulation tools and computational models. Model validation and flexible model description must be an integral part of the tool used to simulate and develop computational models. Constant improvement on experimental techniques and recording protocols leads to increasing knowledge about the biophysical mechanisms in neural systems. This poses new challenges for computational neuroscience: extended or completely new computational methods and models may be required. Careful evaluation and categorization of the existing models and tools provide a foundation for these future needs, for constructing multiscale models or extending the models to incorporate additional or more detailed biophysical mechanisms. Improving the quality of publications in computational neuroscience, enabling progressive building of advanced computational models and tools, can be achieved only through adopting publishing standards which underline replicability and reproducibility of research results.

Multi-scale modeling of altered synaptic plasticity related to Amyloid β effects

As suggested by Palop and Mucke (2010) pathologically elevated β-amyloid (Aβ) impairs long term potentiation (LTP) and enhances long term depression (LTD) possible underlying mechanisms in Alzheimer's Disease (AD). In the present paper we adopt and further elaborate a phenomenological computational model of bidirectional plasticity based on the calcium control hypothesis of Shouval et al. (2002). First, to account for Aβ effects the activation function Ω was modified assuming competition between LTP and LTD, and parameter sets were identified that well describe both normal and pathological synaptic plasticity processes. Second, a biophysically plausible kinetic model of bidirectional synaptic plasticity by D'Alcantara et al. (2003) was used to support findings of the phenomenological model and to further explain underlying kinetic processes. Model fitting pointed out molecular contributors, particularly calcineurin and type 1 protein phosphatase that might contribute to observed physiological disturbances in AD.

Reproducibility and Comparability of Computational Models for Astrocyte Calcium Excitability

The scientific community across all disciplines faces the same challenges of ensuring accessibility, reproducibility, and efficient comparability of scientific results. Computational neuroscience is a rapidly developing field, where reproducibility and comparability of research results have gained increasing interest over the past years. As the number of computational models of brain functions is increasing, we chose to address reproducibility using four previously published computational models of astrocyte excitability as an example. Although not conventionally taken into account when modeling neuronal systems, astrocytes have been shown to take part in a variety of in vitro and in vivo phenomena including synaptic transmission. Two of the selected astrocyte models describe spontaneous calcium excitability, and the other two neurotransmitter-evoked calcium excitability. We specifically addressed how well the original simulation results can be reproduced with a reimplementation of the models. Additionally, we studied how well the selected models can be reused and whether they are comparable in other stimulation conditions and research settings. Unexpectedly, we found out that three of the model publications did not give all the necessary information required to reimplement the models. In addition, we were able to reproduce the original results of only one of the models completely based on the information given in the original publications and in the errata. We actually found errors in the equations provided by two of the model publications; after modifying the equations accordingly, the original results were reproduced more accurately. Even though the selected models were developed to describe the same biological event, namely astrocyte calcium excitability, the models behaved quite differently compared to one another. Our findings on a specific set of published astrocyte models stress the importance of proper validation of the models against experimental wet-lab data from astrocytes as well as the careful review process of models. A variety of aspects of model development could be improved, including the presentation of models in publications and databases. Specifically, all necessary mathematical equations, as well as parameter values, initial values of variables, and stimuli used should be given precisely for successful reproduction of scientific results.