Examining the controllability of sepsis using genetic algorithms on an agent-based model of systemic inflammation

A design specification for Critical Illness Digital Twins (CIDTs) to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report "Foundational Research Gaps and Future Directions for Digital Twins"

On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: "Foundational Research Gaps and Future Directions for Digital Twins." The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using "fit-for-purpose" guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.

Digital twin mathematical models suggest individualized hemorrhagic shock resuscitation strategies

BACKGROUND

Optimizing resuscitation to reduce inflammation and organ dysfunction following human trauma-associated hemorrhagic shock is a major clinical hurdle. This is limited by the short duration of pre-clinical studies and the sparsity of early data in the clinical setting.

METHODS

We sought to bridge this gap by linking preclinical data in a porcine model with clinical data from patients from the Prospective, Observational, Multicenter, Major Trauma Transfusion (PROMMTT) study via a three-compartment ordinary differential equation model of inflammation and coagulation.

RESULTS

The mathematical model accurately predicts physiologic, inflammatory, and laboratory measures in both the porcine model and patients, as well as the outcome and time of death in the PROMMTT cohort. Model simulation suggests that resuscitation with plasma and red blood cells outperformed resuscitation with crystalloid or plasma alone, and that earlier plasma resuscitation reduced injury severity and increased survival time.

CONCLUSIONS

This workflow may serve as a translational bridge from pre-clinical to clinical studies in trauma-associated hemorrhagic shock and other complex disease settings.

The Wound Environment Agent-based Model (WEABM): a digital twin platform for characterization and complex therapeutic discovery for volumetric muscle loss

Volumetric Muscle Loss (VML) injuries are characterized by significant loss of muscle mass, usually due to trauma or surgical resection, often with a residual open wound in clinical settings and subsequent loss of limb function due to the replacement of the lost muscle mass with non-functional scar. Being able to regrow functional muscle in VML injuries is a complex control problem that needs to override robust, evolutionarily conserved healing processes aimed at rapidly closing the defect in lieu of restoration of function. We propose that discovering and implementing this complex control can be accomplished by the development of a Medical Digital Twin of VML. Digital Twins (DTs) are the subject of a recent report from the National Academies of Science, Engineering and Medicine (NASEM), which provides guidance as to the definition, capabilities and research challenges associated with the development and implementation of DTs. Specifically, DTs are defined as dynamic computational models that can be personalized to an individual real world "twin" and are connected to that twin via an ongoing data link. DTs can be used to provide control on the real-world twin that is, by the ongoing data connection, adaptive. We have developed an anatomic scale cell-level agent-based model of VML termed the Wound Environment Agent Based Model (WEABM) that can serve as the computational specification for a DT of VML. Simulations of the WEABM provided fundamental insights into the biology of VML, and we used the WEABM in our previously developed pipeline for simulation-based Deep Reinforcement Learning (DRL) to train an artificial intelligence (AI) to implement a robust generalizable control policy aimed at increasing the healing of VML with functional muscle. The insights into VML obtained include: 1) a competition between fibrosis and myogenesis due to spatial constraints on available edges of intact myofibrils to initiate the myoblast differentiation process, 2) the need to biologically "close" the wound from atmospheric/environmental exposure, which represents an ongoing inflammatory stimulus that promotes fibrosis and 3) that selective, multimodal and adaptive local mediator-level control can shift the trajectory of healing away from a highly evolutionarily beneficial imperative to close the wound via fibrosis. Control discovery with the WEABM identified the following design principles: 1) multimodal adaptive tissue-level mediator control to mitigate pro-inflammation as well as the pro-fibrotic aspects of compensatory anti-inflammation, 2) tissue-level mediator manipulation to promote myogenesis, 3) the use of an engineered extracellular matrix (ECM) to functionally close the wound and 4) the administration of an anti-fibrotic agent focused on the collagen-producing function of fibroblasts and myofibroblasts. The WEABM-trained DRL AI integrates these control modalities and provides design specifications for a potential device that can implement the required wound sensing and intervention delivery capabilities needed. The proposed cyber-physical system integrates the control AI with a physical sense-and-actuate device that meets the tenets of DTs put forth in the NASEM report and can serve as an example schema for the future development of Medical DTs.

Towards systems immunology of critical illness at scale: from single cell 'omics to digital twins

Single-cell 'omics methodology has yielded unprecedented insights based largely on data-centric informatics for reducing, and thus interpreting, massive datasets. In parallel, parsimonious mathematical modeling based on abstractions of pathobiology has also yielded major insights into inflammation and immunity, with these models being extended to describe multi-organ disease pathophysiology as the basis of 'digital twins' and in silico clinical trials. The integration of these distinct methods at scale can drive both basic and translational advances, especially in the context of critical illness, including diseases such as COVID-19. Here, I explore achievements and argue the challenges that are inherent to the integration of data-driven and mechanistic modeling approaches, highlighting the potential of modeling-based strategies for rational immune system reprogramming.

A survey on agents applications in healthcare: Opportunities, challenges and trends

BACKGROUND AND OBJECTIVE

The agent abstraction is a powerful one, developed decades ago to represent crucial aspects of artificial intelligence research. The meaning has transformed over the years and now there are different nuances across research communities. At its core, an agent is an autonomous computational entity capable of sensing, acting, and capturing interactions with other agents and its environment. This review examines how agent-based techniques have been implemented and evaluated in a specific and very important domain, i.e. healthcare research.

METHODS

We survey key areas of agent-based research in healthcare, e.g. individual and collective behaviours, communicable and non-communicable diseases, and social epidemiology. We propose a systematic search and critical review of relevant recent works, introduced by an exploratory network analysis.

RESULTS

Network analysis enables to devise out 5 main research clusters, the most active authors, and 4 main research topics.

CONCLUSIONS

Our findings support discussion of some future directions for increasing the value of agent-based approaches in healthcare.

Exploring the constituent mechanisms of hepatitis: a dynamical systems approach

Hepatitis is the term used to describe inflammation in the liver. It is associated with a high rate of mortality, but the underlying disease mechanisms are not completely understood and treatment options are limited. We present a mathematical model of hepatitis that captures the complex interactions between hepatocytes (liver cells), hepatic stellate cells (cells in the liver that produce hepatitis-associated fibrosis) and the immune components that mediate inflammation. The model is in the form of a system of ordinary differential equations. We use numerical techniques and bifurcation analysis to characterize and elucidate the physiological mechanisms that dominate liver injury and its outcome to a healthy or unhealthy, chronic state. This study reveals the complex interactions between the multiple cell types and mediators involved in this complex disease and highlights potential problems in targeting inflammation in the liver therapeutically.

Embracing complexity in sepsis

Sepsis involves the dynamic interplay between a pathogen, the host response, the failure of organ systems, medical interventions and a myriad of other factors. This together results in a complex, dynamic and dysregulated state that has remained ungovernable thus far. While it is generally accepted that sepsis is very complex indeed, the concepts, approaches and methods that are necessary to understand this complexity remain underappreciated. In this perspective we view sepsis through the lens of complexity theory. We describe the concepts that support viewing sepsis as a state of a highly complex, non-linear and spatio-dynamic system. We argue that methods from the field of complex systems are pivotal for a fuller understanding of sepsis, and we highlight the progress that has been made over the last decades in this respect. Still, despite these considerable advancements, methods like computational modelling and network-based analyses continue to fly under the general scientific radar. We discuss what barriers contribute to this disconnect, and what we can do to embrace complexity with regards to measurements, research approaches and clinical applications. Specifically, we advocate a focus on longitudinal, more continuous biological data collection in sepsis. Understanding the complexity of sepsis will require a huge multidisciplinary effort, in which computational approaches derived from complex systems science must be supported by, and integrated with, biological data. Such integration could finetune computational models, guide validation experiments, and identify key pathways that could be targeted to modulate the system to the benefit of the host. We offer an example for immunological predictive modelling, which may inform agile trials that could be adjusted throughout the trajectory of disease. Overall, we argue that we should expand our current mental frameworks of sepsis, and embrace nonlinear, system-based thinking in order to move the field forward.

Preparing for the next pandemic: Simulation-based deep reinforcement learning to discover and test multimodal control of systemic inflammation using repurposed immunomodulatory agents

Background

Preparation to address the critical gap in a future pandemic between non-pharmacological measures and the deployment of new drugs/vaccines requires addressing two factors: 1) finding virus/pathogen-agnostic pathophysiological targets to mitigate disease severity and 2) finding a more rational approach to repurposing existing drugs. It is increasingly recognized that acute viral disease severity is heavily driven by the immune response to the infection ("cytokine storm" or "cytokine release syndrome"). There exist numerous clinically available biologics that suppress various pro-inflammatory cytokines/mediators, but it is extremely difficult to identify clinically effective treatment regimens with these agents. We propose that this is a complex control problem that resists standard methods of developing treatment regimens and accomplishing this goal requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training successful game-playing artificial intelligences (AIs). This proof-of-concept study determines if simulated sepsis (e.g. infection-driven cytokine storm) can be controlled in the absence of effective antimicrobial agents by targeting cytokines for which FDA-approved biologics currently exist.

Methods

We use a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. DRL training used a Deep Deterministic Policy Gradient (DDPG) approach with a clinically plausible control interval of 6 hours with manipulation of six cytokines for which there are existing drugs: Tumor Necrosis Factor (TNF), Interleukin-1 (IL-1), Interleukin-4 (IL-4), Interleukin-8 (IL-8), Interleukin-12 (IL-12) and Interferon-γ(IFNg).

Results

DRL trained an AI policy that could improve outcomes from a baseline Recovered Rate of 61% to one with a Recovered Rate of 90% over ~21 days simulated time. This DRL policy was then tested on four different parameterizations not seen in training representing a range of host and microbe characteristics, demonstrating a range of improvement in Recovered Rate by +33% to +56.

Discussion

The current proof-of-concept study demonstrates that significant disease severity mitigation can potentially be accomplished with existing anti-mediator drugs, but only through a multi-modal, adaptive treatment policy requiring implementation with an AI. While the actual clinical implementation of this approach is a projection for the future, the current goal of this work is to inspire the development of a research ecosystem that marries what is needed to improve the simulation models with the development of the sensing/assay technologies to collect the data needed to iteratively refine those models.

An immune dysfunction score for stratification of patients with acute infection based on whole-blood gene expression

Dysregulated host responses to infection can lead to organ dysfunction and sepsis, causing millions of global deaths each year. To alleviate this burden, improved prognostication and biomarkers of response are urgently needed. We investigated the use of whole-blood transcriptomics for stratification of patients with severe infection by integrating data from 3149 samples from patients with sepsis due to community-acquired pneumonia or fecal peritonitis admitted to intensive care and healthy individuals into a gene expression reference map. We used this map to derive a quantitative sepsis response signature (SRSq) score reflective of immune dysfunction and predictive of clinical outcomes, which can be estimated using a 7- or 12-gene signature. Last, we built a machine learning framework, SepstratifieR, to deploy SRSq in adult and pediatric bacterial and viral sepsis, H1N1 influenza, and COVID-19, demonstrating clinically relevant stratification across diseases and revealing some of the physiological alterations linking immune dysregulation to mortality. Our method enables early identification of individuals with dysfunctional immune profiles, bringing us closer to precision medicine in infection.

Preparing for the next COVID: Deep Reinforcement Learning trained Artificial Intelligence discovery of multi-modal immunomodulatory control of systemic inflammation in the absence of effective anti-microbials

Background

Despite a great deal of interest in the application of artificial intelligence (AI) to sepsis/critical illness, most current approaches are limited in their potential impact: prediction models do not (and cannot) address the lack of effective therapeutics and current approaches to enhancing the treatment of sepsis focus on optimizing the application of existing interventions, and thus cannot address the development of new treatment options/modalities. The inability to test new therapeutic applications was highlighted by the generally unsatisfactory results from drug repurposing efforts in COVID-19.

Hypothesis

Addressing this challenge requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training the game-playing AIs. We have previously demonstrated the potential of this method in the context of bacterial sepsis in which the microbial infection is responsive to antibiotic therapy. The current work addresses the control problem of multi-modal, adaptive immunomodulation in the circumstance where there is no effective anti-pathogen therapy (e.g., in a novel viral pandemic or in the face of resistant microbes).

Methods

This is a proof-of-concept study that determines the controllability of sepsis without the ability to pharmacologically suppress the pathogen. We use as a surrogate system a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. The DRL algorithm 'trains' an AI on simulations of infection where both the control and observation spaces are limited to operating upon the defined immune mediators included in the IIRABM (a total of 11). Policies were learned using the Deep Deterministic Policy Gradient approach, with the objective function being a return to baseline system health.

Results

DRL trained an AI policy that improved system mortality from 85% to 10.4%. Control actions affected every one of the 11 targetable cytokines and could be divided into those with static/unchanging controls and those with variable/adaptive controls. Adaptive controls primarily targeted 3 different aspects of the immune response: 2nd order pro-inflammation governing TH1/TH2 balance, primary anti-inflammation, and inflammatory cell proliferation.

Discussion

The current treatment of sepsis is hampered by limitations in therapeutic options able to affect the biology of sepsis. This is heightened in circumstances where no effective antimicrobials exist, as was the case for COVID-19. Current AI methods are intrinsically unable to address this problem; doing so requires training AIs in contexts that fully represent the counterfactual space of potential treatments. The synthetic data needed for this task is only possible through the use of high-resolution, mechanism-based simulations. Finally, being able to treat sepsis will require a reorientation as to the sensing and actuating requirements needed to develop these simulations and bring them to the bedside.

Divergent COVID-19 Disease Trajectories Predicted by a DAMP-Centered Immune Network Model

COVID-19 presentations range from mild to moderate through severe disease but also manifest with persistent illness or viral recrudescence. We hypothesized that the spectrum of COVID-19 disease manifestations was a consequence of SARS-CoV-2-mediated delay in the pathogen-associated molecular pattern (PAMP) response, including dampened type I interferon signaling, thereby shifting the balance of the immune response to be dominated by damage-associated molecular pattern (DAMP) signaling. To test the hypothesis, we constructed a parsimonious mechanistic mathematical model. After calibration of the model for initial viral load and then by varying a few key parameters, we show that the core model generates four distinct viral load, immune response and associated disease trajectories termed “patient archetypes”, whose temporal dynamics are reflected in clinical data from hospitalized COVID-19 patients. The model also accounts for responses to corticosteroid therapy and predicts that vaccine-induced neutralizing antibodies and cellular memory will be protective, including from severe COVID-19 disease. This generalizable modeling framework could be used to analyze protective and pathogenic immune responses to diverse viral infections.

Iterative community-driven development of a SARS-CoV-2 tissue simulator

The 2019 novel coronavirus, SARS-CoV-2, is a pathogen of critical significance to international public health. Knowledge of the interplay between molecular-scale virus-receptor interactions, single-cell viral replication, intracellular-scale viral transport, and emergent tissue-scale viral propagation is limited. Moreover, little is known about immune system-virus-tissue interactions and how these can result in low-level (asymptomatic) infections in some cases and acute respiratory distress syndrome (ARDS) in others, particularly with respect to presentation in different age groups or pre-existing inflammatory risk factors. Given the nonlinear interactions within and among each of these processes, multiscale simulation models can shed light on the emergent dynamics that lead to divergent outcomes, identify actionable "choke points" for pharmacologic interventions, screen potential therapies, and identify potential biomarkers that differentiate patient outcomes. Given the complexity of the problem and the acute need for an actionable model to guide therapy discovery and optimization, we introduce and iteratively refine a prototype of a multiscale model of SARS-CoV-2 dynamics in lung tissue. The first prototype model was built and shared internationally as open source code and an online interactive model in under 12 hours, and community domain expertise is driving regular refinements. In a sustained community effort, this consortium is integrating data and expertise across virology, immunology, mathematical biology, quantitative systems physiology, cloud and high performance computing, and other domains to accelerate our response to this critical threat to international health. More broadly, this effort is creating a reusable, modular framework for studying viral replication and immune response in tissues, which can also potentially be adapted to related problems in immunology and immunotherapy.

The effects of early restrictive fluid resuscitation on the clinical outcomes in sepsis patients

OBJECTIVE

To investigate the effects of early restrictive fluid resuscitation (RFR) on the clinical outcomes in sepsis patients.

METHODS

A total of 122 sepsis patients admitted to our hospital were recruited for this study and divided into a study group (the SG, n=56) and a control group (the CG, n=66) according to the treatment method each patient was administered. The SG was administered early RFR, and the CG was administered adequate fluid resuscitation. The clinical data were analyzed retrospectively in both groups. The total infusion volumes, the hemorrhage amounts, the urine outputs, and the Acute Physiology and Chronic Health Evaluation (APACHE II) scores were compared between the two groups. In addition, the heart rates, the mean arterial pressure levels, the central venous pressure levels, and the cardiac function indices were compared between the two groups at 1-7 days after the procedures. The survival and the complication incidence rates were followed up.

RESULTS

The SG showed significantly lower heart rates and mean arterial pressure levels and higher central venous pressure levels than the CG at 1-7 days after the procedures (P<0.05). The cardiac troponin, N-terminal brain pro-natriuretic peptide, and C-reactive protein levels at 3-7 days after the procedures in the SG were significantly lower than the levels in the CG (P<0.05). The cardiac output, stroke volume, and left ventricular ejection fraction scores in the SG were significantly higher than they were in the CG (P<0.05). The survival rate in the SG was significantly higher than it was in the CG at 16, 32, and 64 days after the procedures (P<0.05). The incidence of complications in the SG was lower than it was in the CG (P<0.05).

CONCLUSION

Early RFR can remarkably improve the clinical outcomes, the myocardial injury and survival rates, and the multiple complications incidence rate in sepsis patients.

The Use of Artificial Neural Networks to Forecast the Behavior of Agent-Based Models of Pathophysiology: An Example Utilizing an Agent-Based Model of Sepsis

Introduction: Disease states are being characterized at finer and finer levels of resolution via biomarker or gene expression profiles, while at the same time. Machine learning (ML) is increasingly used to analyze and potentially classify or predict the behavior of biological systems based on such characterization. As ML applications are extremely data-intensive, given the relative sparsity of biomedical data sets ML training of artificial neural networks (ANNs) often require the use of synthetic training data. Agent-based models (ABMs) that incorporate known biological mechanisms and their associated stochastic properties are a potential means of generating synthetic data. Herein we present an example of ML used to train an artificial neural network (ANN) as a surrogate system used to predict the time evolution of an ABM focusing on the clinical condition of sepsis. Methods: The disease trajectories for clinical sepsis, in terms of temporal cytokine and phenotypic dynamics, can be interpreted as a random dynamical system. The Innate Immune Response Agent-based Model (IIRABM) is a well-established model that utilizes known cellular and molecular rules to simulate disease trajectories corresponding to clinical sepsis. We have utilized two distinct neural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take a time sequence of five measurements of eleven IIRABM simulated serum cytokine concentrations as input and to return both the future cytokine trajectories as well as an aggregate metric representing the patient's state of health. Results: The ANNs predicted model trajectories with the expected amount of error, due to stochasticity in the simulation, and recognizing that the mapping from a specific cytokine profile to a state-of-health is not unique. The Multi-Layer Perceptron neural network, generated predictions with a more accurate forecasted trajectory cone. Discussion: This work serves as a proof-of-concept for the use of ANNs to predict disease progression in sepsis as represented by an ABM. The findings demonstrate that multicellular systems with intrinsic stochasticity can be approximated with an ANN, but that forecasting a specific trajectory of the system requires sequential updating of the system state to provide a rolling forecast horizon.

An Agent-based Decision Support for a Vaccination Campaign

We explore the Covid-19 diffusion with an agent-based model of an Italian region with a population on a scale of 1:1000. We also simulate different vaccination strategies. From a decision support system perspective, we investigate the adoption of artificial intelligence techniques to provide suggestions about more effective policies. We adopt the widely used multi-agent programmable modeling environment NetLogo, adding genetic algorithms to evolve the best vaccination criteria. The results suggest a promising methodology for defining vaccine rates by population types over time. The results are encouraging towards a more extensive application of agent-oriented methods in public healthcare policies.

Comparative Computational Modeling of the Bat and Human Immune Response to Viral Infection with the Comparative Biology Immune Agent Based Model

Given the impact of pandemics due to viruses of bat origin, there is increasing interest in comparative investigation into the differences between bat and human immune responses. The practice of comparative biology can be enhanced by computational methods used for dynamic knowledge representation to visualize and interrogate the putative differences between the two systems. We present an agent based model that encompasses and bridges differences between bat and human responses to viral infection: the comparative biology immune agent based model, or CBIABM. The CBIABM examines differences in innate immune mechanisms between bats and humans, specifically regarding inflammasome activity and type 1 interferon dynamics, in terms of tolerance to viral infection. Simulation experiments with the CBIABM demonstrate the efficacy of bat-related features in conferring viral tolerance and also suggest a crucial role for endothelial inflammasome activity as a mechanism for bat systemic viral tolerance and affecting the severity of disease in human viral infections. We hope that this initial study will inspire additional comparative modeling projects to link, compare, and contrast immunological functions shared across different species, and in so doing, provide insight and aid in preparation for future viral pandemics of zoonotic origin.

Step-by-step comparison of ordinary differential equation and agent-based approaches to pharmacokinetic-pharmacodynamic models

Mathematical models in oncology aid in the design of drugs and understanding of their mechanisms of action by simulation of drug biodistribution, drug effects, and interaction between tumor and healthy cells. The traditional approach in pharmacometrics is to develop and validate ordinary differential equation models to quantify trends at the population level. In this approach, time‐course of biological measurements is modeled continuously, assuming a homogenous population. Another approach, agent‐based models, focuses on the behavior and fate of biological entities at the individual level, which subsequently could be summarized to reflect the population level. Heterogeneous cell populations and discrete events are simulated, and spatial distribution can be incorporated. In this tutorial, an agent‐based model is presented and compared to an ordinary differential equation model for a tumor efficacy model inhibiting the pERK pathway. We highlight strengths, weaknesses, and opportunities of each approach.

Utilizing the Heterogeneity of Clinical Data for Model Refinement and Rule Discovery Through the Application of Genetic Algorithms to Calibrate a High-Dimensional Agent-Based Model of Systemic Inflammation

Introduction: Accounting for biological heterogeneity represents one of the greatest challenges in biomedical research. Dynamic computational and mathematical models can be used to enhance the study and understanding of biological systems, but traditional methods for calibration and validation commonly do not account for the heterogeneity of biological data, which may result in overfitting and brittleness of these models. Herein we propose a machine learning approach that utilizes genetic algorithms (GAs) to calibrate and refine an agent-based model (ABM) of acute systemic inflammation, with a focus on accounting for the heterogeneity seen in a clinical data set, thereby avoiding overfitting and increasing the robustness and potential generalizability of the underlying simulation model. Methods: Agent-based modeling is a frequently used modeling method for multi-scale mechanistic modeling. However, the same properties that make ABMs well suited to representing biological systems also present significant challenges with respect to their construction and calibration, particularly with respect to the selection of potential mechanistic rules and the large number of associated free parameters. We have proposed that machine learning approaches (such as GAs) can be used to more effectively and efficiently deal with rule selection and parameter space characterization; the current work applies GAs to the challenge of calibrating a complex ABM to a specific data set, while preserving biological heterogeneity reflected in the range and variance of the data. This project uses a GA to augment the rule-set for a previously validated ABM of acute systemic inflammation, the Innate Immune Response ABM (IIRABM) to clinical time series data of systemic cytokine levels from a population of burn patients. The genome for the GA is a vector generated from the IIRABM's Model Rule Matrix (MRM), which is a matrix representation of not only the constants/parameters associated with the IIRABM's cytokine interaction rules, but also the existence of rules themselves. Capturing heterogeneity is accomplished by a fitness function that incorporates the sample value range ("error bars") of the clinical data. Results: The GA-enabled parameter space exploration resulted in a set of putative MRM rules and associated parameterizations which closely match the cytokine time course data used to design the fitness function. The number of non-zero elements in the MRM increases significantly as the model parameterizations evolve toward a fitness function minimum, transitioning from a sparse to a dense matrix. This results in a model structure that more closely resembles (at a superficial level) the structure of data generated by a standard differential gene expression experimental study. Conclusion: We present an HPC-enabled machine learning/evolutionary computing approach to calibrate a complex ABM to complex clinical data while preserving biological heterogeneity. The integration of machine learning, HPC, and multi-scale mechanistic modeling provides a pathway forward to more effectively representing the heterogeneity of clinical populations and their data.

Nested active learning for efficient model contextualization and parameterization: pathway to generating simulated populations using multi-scale computational models

There is increasing interest in the use of mechanism-based multi-scale computational models (such as agent-based models (ABMs)) to generate simulated clinical populations in order to discover and evaluate potential diagnostic and therapeutic modalities. The description of the environment in which a biomedical simulation operates (model context) and parameterization of internal model rules (model content) requires the optimization of a large number of free parameters. In this work, we utilize a nested active learning (AL) workflow to efficiently parameterize and contextualize an ABM of systemic inflammation used to examine sepsis. Contextual parameter space was examined using four parameters external to the model's rule set. The model's internal parameterization, which represents gene expression and associated cellular behaviors, was explored through the augmentation or inhibition of signaling pathways for 12 signaling mediators associated with inflammation and wound healing. We have implemented a nested AL approach in which the clinically relevant (CR) model environment space for a given internal model parameterization is mapped using a small Artificial Neural Network (ANN). The outer AL level workflow is a larger ANN that uses AL to efficiently regress the volume and centroid location of the CR space given by a single internal parameterization. We have reduced the number of simulations required to efficiently map the CR parameter space of this model by approximately 99%. In addition, we have shown that more complex models with a larger number of variables may expect further improvements in efficiency.

Spatial considerations in the resolution of inflammation: Elucidating leukocyte interactions via an experimentally-calibrated agent-based model

Many common medical conditions (such as cancer, arthritis, chronic obstructive pulmonary disease (COPD), and others) are associated with inflammation, and even more so when combined with the effects of ageing and multimorbidity. While the inflammatory response varies in different tissue types, under disease and in response to therapeutic interventions, it has common interactions that occur between immune cells and inflammatory mediators. Understanding these underlying inflammatory mechanisms is key in progressing treatments and therapies for numerous inflammatory conditions. It is now considered that constituent mechanisms of the inflammatory response can be actively manipulated in order to drive resolution of inflammatory damage; particularly, those mechanisms related to the pro-inflammatory role of neutrophils and the anti-inflammatory role of macrophages. In this article, we describe the assembly of a hybrid mathematical model in which the spatial spread of inflammatory mediators is described through partial differential equations, and immune cells (neutrophils and macrophages) are described individually via an agent-based modelling approach. We pay close attention to how immune cells chemotax toward pro-inflammatory mediators, presenting a model for cell chemotaxis that is calibrated against experimentally observed cell trajectories in healthy and COPD-affected scenarios. We illustrate how variations in key model parameters can drive the switch from resolution of inflammation to chronic outcomes, and show that aberrant neutrophil chemotaxis can move an otherwise healthy outcome to one of chronicity. Finally, we reflect on our results in the context of the on-going hunt for new therapeutic interventions.

Inflammation is the body’s primary defence to harmful stimuli such as infections, toxins and tissue strain but also underlies a much broader range of conditions, including asthma, arthritis and cancer. The inflammatory response is key in resolving injury to facilitate recovery, and involves a range of interactions between immune cells (leukocytes, neutrophils and macrophages in particular) and inflammatory mediators. Immune cells are recruited from the blood stream in response to injury. Once in tissue, neutrophils release toxins to kill invading agents and resolve damage; however, if not carefully managed by other immune cells (mainly macrophages), their responses can increase inflammation instead of helping to resolve it. We model these interactions in response to damage using a spatial model, examining how a healthy response can prevent localised inflammation from spreading. We pay close attention to how cells migrate toward the damaged area, as many inflammatory conditions are associated with impairment of this process. We calibrate our model against experimentally-observed cell trajectories from healthy patients and patients with chronic obstructive pulmonary disease. We illustrate that a healthy outcome depends strongly upon efficient cell migration and a delicate balance between the pro- and anti-inflammatory effects of neutrophils and macrophages.

Biomaterial-Driven Immunomodulation: Cell Biology-Based Strategies to Mitigate Severe Inflammation and Sepsis

Inflammation is an essential component of a wide variety of disease processes and oftentimes can increase the deleterious effects of a disease. Finding ways to modulate this essential immune process is the basis for many therapeutics under development and is a burgeoning area of research for both basic and translational immunology. In addition to developing therapeutics for cellular and molecular targets, the use of biomaterials to modify innate and adaptive immune responses is an area that has recently sparked significant interest. In particular, immunomodulatory activity can be engineered into biomaterials to elicit heightened or dampened immune responses for use in vaccines, immune tolerance, or anti-inflammatory applications. Importantly, the inherent physicochemical properties of the biomaterials play a significant role in determining the observed effects. Properties including composition, molecular weight, size, surface charge, and others affect interactions with immune cells (i.e., nano-bio interactions) and allow for differential biological responses such as activation or inhibition of inflammatory signaling pathways, surface molecule expression, and antigen presentation to be encoded. Numerous opportunities to open new avenues of research to understand the ways in which immune cells interact with and integrate information from their environment may provide critical solutions needed to treat a variety of disorders and diseases where immune dysregulation is a key inciting event. However, to elicit predictable immune responses there is a great need for a thorough understanding of how the biomaterial properties can be tuned to harness a designed immunological outcome. This review aims to systematically describe the biological effects of nanoparticle properties-separate from additional small molecule or biologic delivery-on modulating innate immune cell responses in the context of severe inflammation and sepsis. We propose that nanoparticles represent a potential polypharmacological strategy to simultaneously modify multiple aspects of dysregulated immune responses where single target therapies have fallen short for these applications. This review intends to serve as a resource for immunology labs and other associated fields that would like to apply the growing field of rationally designed biomaterials into their work.

Complex Stability and an Irrevertible Transition Reverted by Peptide and Fibroblasts in a Dynamic Model of Innate Immunity

We here apply a control analysis and various types of stability analysis to an in silico model of innate immunity that addresses the management of inflammation by a therapeutic peptide. Motivation is the observation, both in silico and in experiments, that this therapy is not robust. Our modeling results demonstrate how (1) the biological phenomena of acute and chronic modes of inflammation may reflect an inherently complex bistability with an irrevertible flip between the two modes, (2) the chronic mode of the model has stable, sometimes unique, steady states, while its acute-mode steady states are stable but not unique, (3) as witnessed by TNF levels, acute inflammation is controlled by multiple processes, whereas its chronic-mode inflammation is only controlled by TNF synthesis and washout, (4) only when the antigen load is close to the acute mode's flipping point, many processes impact very strongly on cells and cytokines, (5) there is no antigen exposure level below which reduction of the antigen load alone initiates a flip back to the acute mode, and (6) adding healthy fibroblasts makes the transition from acute to chronic inflammation revertible, although (7) there is a window of antigen load where such a therapy cannot be effective. This suggests that triple therapies may be essential to overcome chronic inflammation. These may comprise (1) anti-immunoglobulin light chain peptides, (2) a temporarily reduced antigen load, and (3a) fibroblast repopulation or (3b) stem cell strategies.

Agent-based models of inflammation in translational systems biology: A decade later

Agent-based modeling is a rule-based, discrete-event, and spatially explicit computational modeling method that employs computational objects that instantiate the rules and interactions among the individual components ("agents") of system. Agent-based modeling is well suited to translating into a computational model the knowledge generated from basic science research, particularly with respect to translating across scales the mechanisms of cellular behavior into aggregated cell population dynamics manifesting at the tissue and organ level. This capacity has made agent-based modeling an integral method in translational systems biology (TSB), an approach that uses multiscale dynamic computational modeling to explicitly represent disease processes in a clinically relevant fashion. The initial work in the early 2000s using agent-based models (ABMs) in TSB focused on examining acute inflammation and its intersection with wound healing; the decade since has seen vast growth in both the application of agent-based modeling to a wide array of disease processes as well as methodological advancements in the use and analysis of ABM. This report presents an update on an earlier review of ABMs in TSB and presents examples of exciting progress in the modeling of various organs and diseases that involve inflammation. This review also describes developments that integrate the use of ABMs with cutting-edge technologies such as high-performance computing, machine learning, and artificial intelligence, with a view toward the future integration of these methodologies. This article is categorized under: Translational, Genomic, and Systems Medicine > Translational Medicine Models of Systems Properties and Processes > Mechanistic Models Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models Models of Systems Properties and Processes > Organismal Models.

Computational Health Engineering Applied to Model Infectious Diseases and Antimicrobial Resistance Spread

Infectious diseases are the primary cause of mortality worldwide. The dangers of infectious disease are compounded with antimicrobial resistance, which remains the greatest concern for human health. Although novel approaches are under investigation, the World Health Organization predicts that by 2050, septicaemia caused by antimicrobial resistant bacteria could result in 10 million deaths per year. One of the main challenges in medical microbiology is to develop novel experimental approaches, which enable a better understanding of bacterial infections and antimicrobial resistance. After the introduction of whole genome sequencing, there was a great improvement in bacterial detection and identification, which also enabled the characterization of virulence factors and antimicrobial resistance genes. Today, the use of in silico experiments jointly with computational and machine learning offer an in depth understanding of systems biology, allowing us to use this knowledge for the prevention, prediction, and control of infectious disease. Herein, the aim of this review is to discuss the latest advances in human health engineering and their applicability in the control of infectious diseases. An in-depth knowledge of host–pathogen–protein interactions, combined with a better understanding of a host’s immune response and bacterial fitness, are key determinants for halting infectious diseases and antimicrobial resistance dissemination.

Alleviation of exhaustion-induced immunosuppression and sepsis by immune checkpoint blockers sequentially administered with antibiotics-analysis of a new mathematical model

Background

Sepsis-associated immune dysregulation, involving hyper-inflammation and immunosuppression, is common in intensive care patients, often leading to multiple organ dysfunction and death. The aim of this study was to identify the main driving force underlying immunosuppression in sepsis, and to suggest new therapeutic avenues for controlling this immune impairment and alleviating excessive pathogen load.

Methods

We developed two minimalistic (skeletal) mathematical models of pathogen-associated inflammation, which focus on the dynamics of myeloid, lymphocyte, and pathogen numbers in blood. Both models rely on the assumption that the presence of the pathogen causes a bias in hematopoietic stem cell differentiation toward the myeloid developmental line. Also in one of the models, we assumed that continuous exposure to pathogens induces lymphocyte exhaustion. In addition, we also created therapy models, both by antibiotics and by immunotherapy with PD-1/PD-L1 checkpoint inhibitors. Assuming realistic parameter ranges, we simulated the pathogen-associated inflammation models in silico with or without various antibiotic and immunotherapy schedules.

Results

Computer simulations of the two models show that the assumption of lymphocyte exhaustion is a prerequisite for attaining sepsis-associated immunosuppression, and that the ability of the innate and adaptive immune systems to control infections depends on the pathogen’s replication rate. Simulation results further show that combining antibiotics with immune checkpoint blockers can suffice for defeating even an aggressive pathogen within a relatively short period. This is so as long as the drugs are administered soon after diagnosis. In contrast, when applied as monotherapies, antibiotics or immune checkpoint blockers fall short of eliminating aggressive pathogens in reasonable time.

Conclusions

Our results suggest that lymphocyte exhaustion crucially drives immunosuppression in sepsis, and that one can efficiently resolve both immunosuppression and pathogenesis by timely coupling of antibiotics with an immune checkpoint blocker, but not by either one of these two treatment modalities alone. Following experimental validation, our model can be adapted to explore the potential of other therapeutic options in this field.

Electronic supplementary material

The online version of this article (10.1186/s40635-019-0260-3) contains supplementary material, which is available to authorized users.

Deep Reinforcement Learning and Simulation as a Path Toward Precision Medicine

Traditionally, precision medicine involves classifying patients to identify subpopulations that respond favorably to specific therapeutics. We pose precision medicine as a dynamic feedback control problem, where treatment administered to a patient is guided by measurements taken during the course of treatment. We consider sepsis, a life-threatening condition in which dysregulation of the immune system causes tissue damage. We leverage an existing simulation of the innate immune response to infection and apply deep reinforcement learning (DRL) to discover an adaptive personalized treatment policy that specifies effective multicytokine therapy to simulated sepsis patients based on systemic measurements. The learned policy achieves a dramatic reduction in mortality rate over a set of 500 simulated patients relative to standalone antibiotic therapy. Advantages of our approach are threefold: (1) the use of simulation allows exploring therapeutic strategies beyond clinical practice and available data, (2) advances in DRL accommodate learning complex therapeutic strategies for complex biological systems, and (3) optimized treatments respond to a patient's individual disease progression over time, therefore, capturing both differences across patients and the inherent randomness of disease progression within a single patient. We hope that this work motivates both considering adaptive personalized multicytokine mediation therapy for sepsis and exploiting simulation with DRL for precision medicine more broadly.

Optimization of Dose Schedules for Chemotherapy of Early Colon Cancer Determined by High-Performance Computer Simulations

Cancer chemotherapy dose schedules are conventionally applied intermittently, with dose duration of the order of hours, intervals between doses of days or weeks, and cycles repeated for weeks. The large number of possible combinations of values of duration, interval, and lethality has been an impediment to empirically determine the optimal set of treatment conditions. The purpose of this project was to determine the set of parameters for duration, interval, and lethality that would be most effective for treating early colon cancer. An agent-based computer model that simulated cell proliferation kinetics in normal human colon crypts was calibrated with measurements of human biopsy specimens. Mutant cells were simulated as proliferating and forming an adenoma, or dying if treated with cytotoxic chemotherapy. Using a high-performance computer, a total of 28 800 different parameter sets of duration, interval, and lethality were simulated. The effect of each parameter set on the stability of colon crypts, the time to cure a crypt of mutant cells, and the accumulated dose was determined. Of the 28 800 parameter sets, 434 parameter sets were effective in curing the crypts of mutant cells before they could form an adenoma and allowed the crypt normal cell dynamics to recover to pretreatment levels. A group of 14 similar parameter sets produced a minimal time to cure mutant cells. A different group of nine similar parameter sets produced the least accumulated dose. These parameter sets may be considered as candidate dose schedules to guide clinical trials for early colon cancer.

Inflammation and Disease: Modelling and Modulation of the Inflammatory Response to Alleviate Critical Illness

Critical illness, a constellation of interrelated inflammatory and physiological derangements occurring subsequent to severe infection or injury, affects a large number of individuals in both developed and developing countries. The prototypical complex system embodied in critical illness has largely defied therapy beyond supportive care. We have focused on the utility of data-driven and mechanistic computational modelling to help address the complexity of critical illness and provide pathways towards discovering potential therapeutic options and combinations. Herein, we review recent progress in this field, with a focus on both animal and computational models of critical illness. We suggest that therapy for critical illness can be posed as a model-based dynamic control problem, and discuss novel theoretical and experimental approaches involving biohybrid devices aimed at reprogramming inflammation dynamically. Together, these advances offer the potential for Model-based Precision Medicine for critical illness.

The Crisis of Reproducibility, the Denominator Problem and the Scientific Role of Multi-scale Modeling

The “Crisis of Reproducibility” has received considerable attention both within the scientific community and without. While factors associated with scientific culture and practical practice are most often invoked, I propose that the Crisis of Reproducibility is ultimately a failure of generalization with a fundamental scientific basis in the methods used for biomedical research. The Denominator Problem describes how limitations intrinsic to the two primary approaches of biomedical research, clinical studies and preclinical experimental biology, lead to an inability to effectively characterize the full extent of biological heterogeneity, which compromises the task of generalizing acquired knowledge. Drawing on the example of the unifying role of theory in the physical sciences, I propose that multi-scale mathematical and dynamic computational models, when mapped to the modular structure of biological systems, can serve a unifying role as formal representations of what is conserved and similar from one biological context to another. This ability to explicitly describe the generation of heterogeneity from similarity addresses the Denominator Problem and provides a scientific response to the Crisis of Reproducibility.