Utilizing the Heterogeneity of Clinical Data for Model Refinement and Rule Discovery Through the Application of Genetic Algorithms to Calibrate a High-Dimensional Agent-Based Model of Systemic Inflammation

A design specification for Critical Illness Digital Twins (CIDTs) to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report "Foundational Research Gaps and Future Directions for Digital Twins"

On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: "Foundational Research Gaps and Future Directions for Digital Twins." The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using "fit-for-purpose" guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.

The Wound Environment Agent-based Model (WEABM): a digital twin platform for characterization and complex therapeutic discovery for volumetric muscle loss

Volumetric Muscle Loss (VML) injuries are characterized by significant loss of muscle mass, usually due to trauma or surgical resection, often with a residual open wound in clinical settings and subsequent loss of limb function due to the replacement of the lost muscle mass with non-functional scar. Being able to regrow functional muscle in VML injuries is a complex control problem that needs to override robust, evolutionarily conserved healing processes aimed at rapidly closing the defect in lieu of restoration of function. We propose that discovering and implementing this complex control can be accomplished by the development of a Medical Digital Twin of VML. Digital Twins (DTs) are the subject of a recent report from the National Academies of Science, Engineering and Medicine (NASEM), which provides guidance as to the definition, capabilities and research challenges associated with the development and implementation of DTs. Specifically, DTs are defined as dynamic computational models that can be personalized to an individual real world "twin" and are connected to that twin via an ongoing data link. DTs can be used to provide control on the real-world twin that is, by the ongoing data connection, adaptive. We have developed an anatomic scale cell-level agent-based model of VML termed the Wound Environment Agent Based Model (WEABM) that can serve as the computational specification for a DT of VML. Simulations of the WEABM provided fundamental insights into the biology of VML, and we used the WEABM in our previously developed pipeline for simulation-based Deep Reinforcement Learning (DRL) to train an artificial intelligence (AI) to implement a robust generalizable control policy aimed at increasing the healing of VML with functional muscle. The insights into VML obtained include: 1) a competition between fibrosis and myogenesis due to spatial constraints on available edges of intact myofibrils to initiate the myoblast differentiation process, 2) the need to biologically "close" the wound from atmospheric/environmental exposure, which represents an ongoing inflammatory stimulus that promotes fibrosis and 3) that selective, multimodal and adaptive local mediator-level control can shift the trajectory of healing away from a highly evolutionarily beneficial imperative to close the wound via fibrosis. Control discovery with the WEABM identified the following design principles: 1) multimodal adaptive tissue-level mediator control to mitigate pro-inflammation as well as the pro-fibrotic aspects of compensatory anti-inflammation, 2) tissue-level mediator manipulation to promote myogenesis, 3) the use of an engineered extracellular matrix (ECM) to functionally close the wound and 4) the administration of an anti-fibrotic agent focused on the collagen-producing function of fibroblasts and myofibroblasts. The WEABM-trained DRL AI integrates these control modalities and provides design specifications for a potential device that can implement the required wound sensing and intervention delivery capabilities needed. The proposed cyber-physical system integrates the control AI with a physical sense-and-actuate device that meets the tenets of DTs put forth in the NASEM report and can serve as an example schema for the future development of Medical DTs.

Creating a Roadmap to Quantitative Systems Pharmacology-Informed Rare Disease Drug Development: A Workshop Report

One of the goals of the Accelerating Rare Disease Cures (ARC) program in the Center for Drug Evaluation and Research (CDER) at the US Food and Drug Administration (FDA) is the development and use of regulatory and scientific tools, including drug/disease modeling, dose selection, and translational medicine tools. To facilitate achieving this goal, the FDA in collaboration with the University of Maryland Center of Excellence in Regulatory Science and Innovation (M-CERSI) hosted a virtual public workshop on May 11, 2023, entitled "Creating a Roadmap to Quantitative Systems Pharmacology-Informed Rare Disease Drug Development." This workshop engaged scientists from pharmaceutical companies, academic institutes, and the FDA to discuss the potential utility of quantitative systems pharmacology (QSP) in rare disease drug development and identify potential challenges and solutions to facilitate its use. Here, we report the main findings from this workshop, highlight the key takeaways, and propose a roadmap to facilitate the use of QSP in rare disease drug development.

Characterizing the Crosstalk Between Programmed Cell Death Pathways in Cytokine Storm With an Agent-Based Model

Background: There is increasing recognition of extensive crosstalk between programmed cell death pathways (PCDPs), such as apoptosis, pyroptosis, and necroptosis, resulting in a highly redundant system responsive to a breadth of potential pathogens. However, because pyroptosis and necroptosis propagate inflammation, these redundancies also present challenges for therapeutic control of dysregulated hyperinflammation seen in cytokine storm (CS) generated organ dysfunction. Hypothesis: We hypothesize that the conversion of existing knowledge regarding apoptosis, pyroptosis, and necroptosis into a computational model can enhance our understanding of the crosstalk between PCDPs via simulation experiments of microbe interactions and experimental interventions. Materials and Methods: Literature regarding apoptosis, pyroptosis, and necroptosis was reviewed and transposed into an agent-based model, the programmed cell death agent-based model (PCDABM). Computational experiments were performed to simulate the activation of various PCDPs as seen by differing microbes, specifically: influenza A virus (IAV), enteropathic Escherichia coli (EPEC), and Salmonella enterica (SE). The potential protective value of PCDP crosstalk was evaluated by silencing either pyroptosis, necroptosis, or both. Computational experiments were also performed simulating the effect of potential therapies blocking tumor necrosis factor (TNF) and interleukin (IL)-1. Results: The PCDABM was implemented in the agent-based modeling toolkit NetLogo. Computational experiments of infection with IAV, EPEC, and SE reproduced cross-activation of PCDPs with effective microbial clearance. Simulations of anti-TNF and anti-IL-1 did not reduce the aggregated amount of inflammation-generated system damage, the surrogate for CS-generated tissue damage. Conclusions: Redundancies have evolved in host PCDPs to maintain protection against a wide range of pathogens. However, these redundancies also challenge attempts at dampening the pathogenic hyperinflammatory state of CS using therapeutic immunomodulation. Integrative simulation models such as the PCDABM can aid in identifying potentially targetable inflection points to mitigate CS while maintaining effective host defense.

Real World Data Studies of Antineoplastic Drugs: How Can They Be Improved to Steer Everyday Use in the Clinic?

There is a growing interest in real world evidence when developing antineoplastic drugs owing to the shorter length of time and low costs compared to randomised controlled trials. External validity of studies in the regulatory phase can be enhanced by complementing randomised controlled trials with real world evidence. Furthermore, the use of real world evidence ensures the inclusion of patients often excluded from randomised controlled trials such as the elderly, certain ethnicities or those from certain geographical areas. This review explores approaches in which real world data may be integrated with randomised controlled trials. One approach is by using big data, especially when investigating drugs in the antineoplastic setting. This can even inform artificial intelligence thus ensuring faster and more precise diagnosis and treatment decisions. Pragmatic trials also offer an approach to examine the effectiveness of novel antineoplastic drugs without evading the benefits of randomised controlled trials. A well-designed pragmatic trial would yield results with high external validity by employing a simple study design with a large sample size and diverse settings. Although randomised controlled trials can determine efficacy of antineoplastic drugs, effectiveness in the real world may differ. The need for pragmatic trials to help guide healthcare decision-making led to the development of trials within cohorts (TWICs). TWICs make use of cohorts to conduct multiple randomised controlled trials while maintaining characteristics of real world data in routine clinical practice. Although real world data is often affected by incomplete data and biases such as selection and unmeasured biases, the use of big data and pragmatic approaches can improve the use of real world data in the development of antineoplastic drugs that can in turn steer decision-making in clinical practice.

Preparing for the next pandemic: Simulation-based deep reinforcement learning to discover and test multimodal control of systemic inflammation using repurposed immunomodulatory agents

Background

Preparation to address the critical gap in a future pandemic between non-pharmacological measures and the deployment of new drugs/vaccines requires addressing two factors: 1) finding virus/pathogen-agnostic pathophysiological targets to mitigate disease severity and 2) finding a more rational approach to repurposing existing drugs. It is increasingly recognized that acute viral disease severity is heavily driven by the immune response to the infection ("cytokine storm" or "cytokine release syndrome"). There exist numerous clinically available biologics that suppress various pro-inflammatory cytokines/mediators, but it is extremely difficult to identify clinically effective treatment regimens with these agents. We propose that this is a complex control problem that resists standard methods of developing treatment regimens and accomplishing this goal requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training successful game-playing artificial intelligences (AIs). This proof-of-concept study determines if simulated sepsis (e.g. infection-driven cytokine storm) can be controlled in the absence of effective antimicrobial agents by targeting cytokines for which FDA-approved biologics currently exist.

Methods

We use a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. DRL training used a Deep Deterministic Policy Gradient (DDPG) approach with a clinically plausible control interval of 6 hours with manipulation of six cytokines for which there are existing drugs: Tumor Necrosis Factor (TNF), Interleukin-1 (IL-1), Interleukin-4 (IL-4), Interleukin-8 (IL-8), Interleukin-12 (IL-12) and Interferon-γ(IFNg).

Results

DRL trained an AI policy that could improve outcomes from a baseline Recovered Rate of 61% to one with a Recovered Rate of 90% over ~21 days simulated time. This DRL policy was then tested on four different parameterizations not seen in training representing a range of host and microbe characteristics, demonstrating a range of improvement in Recovered Rate by +33% to +56.

Discussion

The current proof-of-concept study demonstrates that significant disease severity mitigation can potentially be accomplished with existing anti-mediator drugs, but only through a multi-modal, adaptive treatment policy requiring implementation with an AI. While the actual clinical implementation of this approach is a projection for the future, the current goal of this work is to inspire the development of a research ecosystem that marries what is needed to improve the simulation models with the development of the sensing/assay technologies to collect the data needed to iteratively refine those models.

Drug Development Digital Twins for Drug Discovery, Testing and Repurposing: A Schema for Requirements and Development

There has been a great deal of interest in the concept, development and implementation of medical digital twins. This interest has led to wide ranging perceptions of what constitutes a medical digital twin. This Perspectives article will provide 1) a description of fundamental features of industrial digital twins, the source of the digital twin concept, 2) aspects of biology that challenge the implementation of medical digital twins, 3) a schematic program of how a specific medical digital twin project could be defined, and 4) an example description within that schematic program for a specific type of medical digital twin intended for drug discovery, testing and repurposing, the Drug Development Digital Twin (DDDT).

Preparing for the next COVID: Deep Reinforcement Learning trained Artificial Intelligence discovery of multi-modal immunomodulatory control of systemic inflammation in the absence of effective anti-microbials

Background

Despite a great deal of interest in the application of artificial intelligence (AI) to sepsis/critical illness, most current approaches are limited in their potential impact: prediction models do not (and cannot) address the lack of effective therapeutics and current approaches to enhancing the treatment of sepsis focus on optimizing the application of existing interventions, and thus cannot address the development of new treatment options/modalities. The inability to test new therapeutic applications was highlighted by the generally unsatisfactory results from drug repurposing efforts in COVID-19.

Hypothesis

Addressing this challenge requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training the game-playing AIs. We have previously demonstrated the potential of this method in the context of bacterial sepsis in which the microbial infection is responsive to antibiotic therapy. The current work addresses the control problem of multi-modal, adaptive immunomodulation in the circumstance where there is no effective anti-pathogen therapy (e.g., in a novel viral pandemic or in the face of resistant microbes).

Methods

This is a proof-of-concept study that determines the controllability of sepsis without the ability to pharmacologically suppress the pathogen. We use as a surrogate system a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. The DRL algorithm 'trains' an AI on simulations of infection where both the control and observation spaces are limited to operating upon the defined immune mediators included in the IIRABM (a total of 11). Policies were learned using the Deep Deterministic Policy Gradient approach, with the objective function being a return to baseline system health.

Results

DRL trained an AI policy that improved system mortality from 85% to 10.4%. Control actions affected every one of the 11 targetable cytokines and could be divided into those with static/unchanging controls and those with variable/adaptive controls. Adaptive controls primarily targeted 3 different aspects of the immune response: 2nd order pro-inflammation governing TH1/TH2 balance, primary anti-inflammation, and inflammatory cell proliferation.

Discussion

The current treatment of sepsis is hampered by limitations in therapeutic options able to affect the biology of sepsis. This is heightened in circumstances where no effective antimicrobials exist, as was the case for COVID-19. Current AI methods are intrinsically unable to address this problem; doing so requires training AIs in contexts that fully represent the counterfactual space of potential treatments. The synthetic data needed for this task is only possible through the use of high-resolution, mechanism-based simulations. Finally, being able to treat sepsis will require a reorientation as to the sensing and actuating requirements needed to develop these simulations and bring them to the bedside.

The Use of Artificial Neural Networks to Forecast the Behavior of Agent-Based Models of Pathophysiology: An Example Utilizing an Agent-Based Model of Sepsis

Introduction: Disease states are being characterized at finer and finer levels of resolution via biomarker or gene expression profiles, while at the same time. Machine learning (ML) is increasingly used to analyze and potentially classify or predict the behavior of biological systems based on such characterization. As ML applications are extremely data-intensive, given the relative sparsity of biomedical data sets ML training of artificial neural networks (ANNs) often require the use of synthetic training data. Agent-based models (ABMs) that incorporate known biological mechanisms and their associated stochastic properties are a potential means of generating synthetic data. Herein we present an example of ML used to train an artificial neural network (ANN) as a surrogate system used to predict the time evolution of an ABM focusing on the clinical condition of sepsis. Methods: The disease trajectories for clinical sepsis, in terms of temporal cytokine and phenotypic dynamics, can be interpreted as a random dynamical system. The Innate Immune Response Agent-based Model (IIRABM) is a well-established model that utilizes known cellular and molecular rules to simulate disease trajectories corresponding to clinical sepsis. We have utilized two distinct neural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take a time sequence of five measurements of eleven IIRABM simulated serum cytokine concentrations as input and to return both the future cytokine trajectories as well as an aggregate metric representing the patient's state of health. Results: The ANNs predicted model trajectories with the expected amount of error, due to stochasticity in the simulation, and recognizing that the mapping from a specific cytokine profile to a state-of-health is not unique. The Multi-Layer Perceptron neural network, generated predictions with a more accurate forecasted trajectory cone. Discussion: This work serves as a proof-of-concept for the use of ANNs to predict disease progression in sepsis as represented by an ABM. The findings demonstrate that multicellular systems with intrinsic stochasticity can be approximated with an ANN, but that forecasting a specific trajectory of the system requires sequential updating of the system state to provide a rolling forecast horizon.

Comparative Computational Modeling of the Bat and Human Immune Response to Viral Infection with the Comparative Biology Immune Agent Based Model

Given the impact of pandemics due to viruses of bat origin, there is increasing interest in comparative investigation into the differences between bat and human immune responses. The practice of comparative biology can be enhanced by computational methods used for dynamic knowledge representation to visualize and interrogate the putative differences between the two systems. We present an agent based model that encompasses and bridges differences between bat and human responses to viral infection: the comparative biology immune agent based model, or CBIABM. The CBIABM examines differences in innate immune mechanisms between bats and humans, specifically regarding inflammasome activity and type 1 interferon dynamics, in terms of tolerance to viral infection. Simulation experiments with the CBIABM demonstrate the efficacy of bat-related features in conferring viral tolerance and also suggest a crucial role for endothelial inflammasome activity as a mechanism for bat systemic viral tolerance and affecting the severity of disease in human viral infections. We hope that this initial study will inspire additional comparative modeling projects to link, compare, and contrast immunological functions shared across different species, and in so doing, provide insight and aid in preparation for future viral pandemics of zoonotic origin.

Nested active learning for efficient model contextualization and parameterization: pathway to generating simulated populations using multi-scale computational models

There is increasing interest in the use of mechanism-based multi-scale computational models (such as agent-based models (ABMs)) to generate simulated clinical populations in order to discover and evaluate potential diagnostic and therapeutic modalities. The description of the environment in which a biomedical simulation operates (model context) and parameterization of internal model rules (model content) requires the optimization of a large number of free parameters. In this work, we utilize a nested active learning (AL) workflow to efficiently parameterize and contextualize an ABM of systemic inflammation used to examine sepsis. Contextual parameter space was examined using four parameters external to the model's rule set. The model's internal parameterization, which represents gene expression and associated cellular behaviors, was explored through the augmentation or inhibition of signaling pathways for 12 signaling mediators associated with inflammation and wound healing. We have implemented a nested AL approach in which the clinically relevant (CR) model environment space for a given internal model parameterization is mapped using a small Artificial Neural Network (ANN). The outer AL level workflow is a larger ANN that uses AL to efficiently regress the volume and centroid location of the CR space given by a single internal parameterization. We have reduced the number of simulations required to efficiently map the CR parameter space of this model by approximately 99%. In addition, we have shown that more complex models with a larger number of variables may expect further improvements in efficiency.