Reproducible Research Using Biomodels

ENCORE: a practical implementation to improve reproducibility and transparency of computational research

Reproducibility of computational research is often challenging despite established guidelines and best practices. Translating these guidelines into practical applications remains difficult. Here, we present ENCORE, an approach to enhance transparency and reproducibility by guiding researchers in how to structure and document a computational project. ENCORE builds on previous efforts in computational reproducibility and integrates all project components into a standardized file system structure. It utilizes pre-defined files as documentation templates, leverages GitHub for software versioning, and includes an HTML-based navigator. ENCORE is designed to be agnostic to the type of computational project, data, programming language, and ICT infrastructure, and does not rely on specific software tools. We also share our group's experience using ENCORE, highlighting that the most significant challenge to the routine adoption of approaches like ours is the lack of incentives to motivate researchers to dedicate sufficient time and effort to ensure reproducibility.

A Kinetic Approach to Oxygen Radical Absorbance Capacity (ORAC): Restoring Order to the Antioxidant Activity of Hydroxycinnamic Acids and Fruit Juices

This study introduces a kinetic model that significantly improves the interpretation of the oxygen radical absorbance capacity (ORAC) assay. Our model accurately simulates and fits the bleaching kinetics of fluorescein in the presence of various antioxidants, achieving high correlation values (R2 > 0.99) with the experimental data. The fit to the experimental data is achieved by optimizing two rate constants, k5 and k6. The k5 value reflects the reactivity of antioxidants toward scavenging peroxyl radicals, whereas k6 measures the ability of antioxidants to regenerate oxidized fluorescein. These parameters (1) allow the detailed classification of cinnamic acids based on their structure-activity relationships, (2) provide insights into the interaction of alkoxyl radicals with fluorescein, and (3) account for the regeneration of fluorescein radicals by antioxidants. The application of the model to different antioxidants and fruit extracts reveals significant deviations from the results of traditional ORAC tests based on the area under the curve (AUC) approach. For example, lemon juice, rich in 'fast' antioxidants such as ascorbic acid, shows a high k5 value, in contrast to its low AUC values. This finding underscores the limitations of the AUC approach and highlights the advantages of our kinetic model in understanding antioxidative dynamics in food systems. This study presents a comprehensive, quantitative, mechanism-oriented approach to assessing antioxidant reactivity, demonstrating a significant improvement in ORAC assay applications.

Reproducibility and FAIR principles: the case of a segment polarity network model

The issue of reproducibility of computational models and the related FAIR principles (findable, accessible, interoperable, and reusable) are examined in a specific test case. I analyze a computational model of the segment polarity network in Drosophila embryos published in 2000. Despite the high number of citations to this publication, 23 years later the model is barely accessible, and consequently not interoperable. Following the text of the original publication allowed successfully encoding the model for the open source software COPASI. Subsequently saving the model in the SBML format allowed it to be reused in other open source software packages. Submission of this SBML encoding of the model to the BioModels database enables its findability and accessibility. This demonstrates how the FAIR principles can be successfully enabled by using open source software, widely adopted standards, and public repositories, facilitating reproducibility and reuse of computational cell biology models that will outlive the specific software used.

Modeling the iron storage protein ferritin reveals how residual ferrihydrite iron determines initial ferritin iron sequestration kinetics

Computational models can be created more efficiently by composing them from smaller, well-defined sub-models that represent specific cellular structures that appear often in different contexts. Cellular iron metabolism is a prime example of this as multiple cell types tend to rely on a similar set of components (proteins and regulatory mechanisms) to ensure iron balance. One recurrent component, ferritin, is the primary iron storage protein in mammalian cells and is necessary for cellular iron homeostasis. Its ability to sequester iron protects cells from rising concentrations of ferrous iron limiting oxidative cell damage. The focus of the present work is establishing a model that tractably represents the ferritin iron sequestration kinetics such that it can be incorporated into larger cell models, in addition to contributing to the understanding of general ferritin iron sequestration dynamics within cells. The model's parameter values were determined from published kinetic and binding experiments and the model was validated against independent data not used in its construction. Simulation results indicate that FT concentration is the most impactful on overall sequestration dynamics, while the FT iron saturation (number of iron atoms sequestered per FT cage) fine tunes the initial rates. Finally, because this model has a small number of reactions and species, was built to represent important details of FT kinetics, and has flexibility to include subtle changes in subunit composition, we propose it to be used as a building block in a variety of specific cell type models of iron metabolism.

VeVaPy, a Python Platform for Efficient Verification and Validation of Systems Biology Models with Demonstrations Using Hypothalamic-Pituitary-Adrenal Axis Models

In order for mathematical models to make credible contributions, it is essential for them to be verified and validated. Currently, verification and validation (V&V) of these models does not meet the expectations of the system biology and systems pharmacology communities. Partially as a result of this shortfall, systemic V&V of existing models currently requires a lot of time and effort. In order to facilitate systemic V&V of chosen hypothalamic-pituitary-adrenal (HPA) axis models, we have developed a computational framework named VeVaPy-taking care to follow the recommended best practices regarding the development of mathematical models. VeVaPy includes four functional modules coded in Python, and the source code is publicly available. We demonstrate that VeVaPy can help us efficiently verify and validate the five HPA axis models we have chosen. Supplied with new and independent data, VeVaPy outputs objective V&V benchmarks for each model. We believe that VeVaPy will help future researchers with basic modeling and programming experience to efficiently verify and validate mathematical models from the fields of systems biology and systems pharmacology.

Dynamic publication media with the COPASI R Connector (CoRC)

In this article we show how dynamic publication media and the COPASI R Connector (CoRC) can be combined in a natural and synergistic way to communicate (biochemical) models. Dynamic publication media are becoming a popular tool for authors to effectively compose and publish their work. They are built from templates and the final documents are created dynamically. In addition, they can also be interactive. Working with dynamic publication media is made easy with the programming environment R via its integration with tools such as R Markdown, Jupyter and Shiny. Additionally, the COmplex PAthway SImulator COPASI (http://www.copasi.org), a widely used biochemical modelling toolkit, is available in R through the use of the COPASI R Connector (CoRC, https://jpahle.github.io/CoRC). Models are a common tool in the mathematical biosciences, in particular kinetic models of biochemical networks in (computational) systems biology. We focus on three application areas of dynamic publication media and CoRC: Documentation (reproducible workflows), Teaching (creating self-paced lessons) and Science Communication (immersive and engaging presentation). To illustrate these, we created six dynamic document examples in the form of R Markdown and Jupyter notebooks, hosted on the platforms GitHub, shinyapps.io, Google Colaboratory. Having code and output in one place, creating documents in template-form and the option of interactivity make the combination of dynamic documents and CoRC a versatile tool. All our example documents are freely available at https://jpahle.github.io/DynamiCoRC under the Creative Commons BY 4.0 licence.

Computational models in systems biology: standards, dissemination, and best practices

Mathematical and computational models are a key technology in systems biology. Progress in the field depends on the replicability and reproducibility of their properties and behavior. For this, an essential requirement is a set of clear standards for model specification and dissemination. This review covers existing standards, and it highlights the most important areas where further work is required. This includes the specification of agent-based models, an increasingly common modeling approach.

Validation Through Collaboration: Encouraging Team Efforts to Ensure Internal and External Validity of Computational Models of Biochemical Pathways

Computational modelling of biochemical reaction pathways is an increasingly important part of neuroscience research. In order to be useful, computational models need to be valid in two senses: First, they need to be consistent with experimental data and able to make testable predictions (external validity). Second, they need to be internally consistent and independently reproducible (internal validity). Here, we discuss both types of validity and provide a brief overview of tools and technologies used to ensure they are met. We also suggest the introduction of new collaborative technologies to ensure model validity: an incentivised experimental database for external validity and reproducibility audits for internal validity. Both rely on FAIR principles and on collaborative science practices.

Agent Based Models of Polymicrobial Biofilms and the Microbiome-A Review

The human microbiome has been a focus of intense study in recent years. Most of the living organisms comprising the microbiome exist in the form of biofilms on mucosal surfaces lining our digestive, respiratory, and genito-urinary tracts. While health-associated microbiota contribute to digestion, provide essential nutrients, and protect us from pathogens, disturbances due to illness or medical interventions contribute to infections, some that can be fatal. Myriad biological processes influence the make-up of the microbiota, for example: growth, division, death, and production of extracellular polymers (EPS), and metabolites. Inter-species interactions include competition, inhibition, and symbiosis. Computational models are becoming widely used to better understand these interactions. Agent-based modeling is a particularly useful computational approach to implement the various complex interactions in microbial communities when appropriately combined with an experimental approach. In these models, each cell is represented as an autonomous agent with its own set of rules, with different rules for each species. In this review, we will discuss innovations in agent-based modeling of biofilms and the microbiota in the past five years from the biological and mathematical perspectives and discuss how agent-based models can be further utilized to enhance our comprehension of the complex world of polymicrobial biofilms and the microbiome.

Reproducibility in systems biology modelling

Reproducibility of scientific results is a key element of science and credibility. The lack of reproducibility across many scientific fields has emerged as an important concern. In this piece, we assess mathematical model reproducibility and propose a scorecard for improving reproducibility in this field.

Practical steps to digital organism models, from laboratory model species to 'Crops in silico

A recent initiative named 'Crops in silico' proposes that multi-scale models 'have the potential to fill in missing mechanistic details and generate new hypotheses to prioritize directed engineering efforts' in plant science, particularly directed to crop species. To that end, the group called for 'a paradigm shift in plant modelling, from largely isolated efforts to a connected community'. 'Wet' (experimental) research has been especially productive in plant science, since the adoption of Arabidopsis thaliana as a laboratory model species allowed the emergence of an Arabidopsis research community. Parts of this community invested in 'dry' (theoretical) research, under the rubric of Systems Biology. Our past research combined concepts from Systems Biology and crop modelling. Here we outline the approaches that seem most relevant to connected, 'digital organism' initiatives. We illustrate the scale of experimental research required, by collecting the kinetic parameter values that are required for a quantitative, dynamic model of a gene regulatory network. By comparison with the Systems Biology Markup Language (SBML) community, we note computational resources and community structures that will help to realize the potential for plant Systems Biology to connect with a broader crop science community.