1
|
Ghazizadeh N, Taghvaei S, Haghpanah SA. Modeling and control of COVID-19 disease using deep reinforcement learning method. Med Biol Eng Comput 2024:10.1007/s11517-024-03153-5. [PMID: 38940880 DOI: 10.1007/s11517-024-03153-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 06/10/2024] [Indexed: 06/29/2024]
Abstract
The prevalence of epidemics has been studied by researchers in various fields. In the last 2 years, the outbreak of COVID-19 has affected the health, economy, and industry of communities around the world and has caused the death of millions of people. Therefore, many researchers have tried to model and control the prevalence of this disease. In this article, the new SQEIAR model for the spread of the COVID-19 disease is provided, which, compared to previous models, explores the effects of additional interventions on the outbreak and incorporates a wider range of variables and parameters to enhance its accuracy and alignment with reality. These modifications in the model lead to a more rapid eradication and control of the disease. This model includes six variables of the group of susceptible, quarantined, exposed, symptomatic, asymptomatic, and recovered individuals and includes three control inputs such as quarantine of susceptible, vaccination, and treatments. In order to minimize symptomatic infectious individuals and susceptible individuals and also to reduce treatment, vaccination, and quarantine costs, an optimal control approach using the Deep Deterministic Policy Gradient (DDPG) method has been applied to the system. This algorithm is applied to the model in different cases of control inputs, and for each case, optimal control inputs are obtained. In the following, the number of deaths due to the disease and the total number of symptomatic infectious individuals for each of these optimal control cases has been calculated. The results of the implemented control structure demonstrated a reduction of 60% in the number of deaths and 74% in the number of symptomatically infected individuals compared to the uncontrolled model. Finally, to test the performance of the control system, noise was applied to the system in various ways, including three methods: applying noise to observer variables, applying noise to control inputs, and applying uncertainty to model parameters. Therefore, we found that this control system was robust and performed well in different conditions despite the disturbance.
Collapse
Affiliation(s)
- Nazanin Ghazizadeh
- School of Mechanical Engineering, Shiraz Univeristy, P.O.B. 7134851154, Shiraz, Iran
| | - Sajjad Taghvaei
- School of Mechanical Engineering, Shiraz Univeristy, P.O.B. 7134851154, Shiraz, Iran
| | | |
Collapse
|
2
|
An G, Cockrell C. A design specification for Critical Illness Digital Twins (CIDTs) to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report "Foundational Research Gaps and Future Directions for Digital Twins". ARXIV 2024:arXiv:2405.05301v2. [PMID: 38764598 PMCID: PMC11100920] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/21/2024]
Abstract
On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: "Foundational Research Gaps and Future Directions for Digital Twins." The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using "fit-for-purpose" guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.
Collapse
Affiliation(s)
- Gary An
- Department of Surgery, University of Vermont Larner College of Medicine
| | - Chase Cockrell
- Department of Surgery, University of Vermont Larner College of Medicine
| |
Collapse
|
3
|
Cockrell C, Vodovotz Y, Zamora R, An G. The Wound Environment Agent-based Model (WEABM): a digital twin platform for characterization and complex therapeutic discovery for volumetric muscle loss. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.04.595972. [PMID: 38895374 PMCID: PMC11185759 DOI: 10.1101/2024.06.04.595972] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
Volumetric Muscle Loss (VML) injuries are characterized by significant loss of muscle mass, usually due to trauma or surgical resection, often with a residual open wound in clinical settings and subsequent loss of limb function due to the replacement of the lost muscle mass with non-functional scar. Being able to regrow functional muscle in VML injuries is a complex control problem that needs to override robust, evolutionarily conserved healing processes aimed at rapidly closing the defect in lieu of restoration of function. We propose that discovering and implementing this complex control can be accomplished by the development of a Medical Digital Twin of VML. Digital Twins (DTs) are the subject of a recent report from the National Academies of Science, Engineering and Medicine (NASEM), which provides guidance as to the definition, capabilities and research challenges associated with the development and implementation of DTs. Specifically, DTs are defined as dynamic computational models that can be personalized to an individual real world "twin" and are connected to that twin via an ongoing data link. DTs can be used to provide control on the real-world twin that is, by the ongoing data connection, adaptive. We have developed an anatomic scale cell-level agent-based model of VML termed the Wound Environment Agent Based Model (WEABM) that can serve as the computational specification for a DT of VML. Simulations of the WEABM provided fundamental insights into the biology of VML, and we used the WEABM in our previously developed pipeline for simulation-based Deep Reinforcement Learning (DRL) to train an artificial intelligence (AI) to implement a robust generalizable control policy aimed at increasing the healing of VML with functional muscle. The insights into VML obtained include: 1) a competition between fibrosis and myogenesis due to spatial constraints on available edges of intact myofibrils to initiate the myoblast differentiation process, 2) the need to biologically "close" the wound from atmospheric/environmental exposure, which represents an ongoing inflammatory stimulus that promotes fibrosis and 3) that selective, multimodal and adaptive local mediator-level control can shift the trajectory of healing away from a highly evolutionarily beneficial imperative to close the wound via fibrosis. Control discovery with the WEABM identified the following design principles: 1) multimodal adaptive tissue-level mediator control to mitigate pro-inflammation as well as the pro-fibrotic aspects of compensatory anti-inflammation, 2) tissue-level mediator manipulation to promote myogenesis, 3) the use of an engineered extracellular matrix (ECM) to functionally close the wound and 4) the administration of an anti-fibrotic agent focused on the collagen-producing function of fibroblasts and myofibroblasts. The WEABM-trained DRL AI integrates these control modalities and provides design specifications for a potential device that can implement the required wound sensing and intervention delivery capabilities needed. The proposed cyber-physical system integrates the control AI with a physical sense-and-actuate device that meets the tenets of DTs put forth in the NASEM report and can serve as an example schema for the future development of Medical DTs.
Collapse
Affiliation(s)
- Chase Cockrell
- Department of Surgery, University of Vermont Larner College of Medicine
| | - Yoram Vodovotz
- Department of Surgery, University of Pittsburgh
- McGowan Institute of Regenerative Medicine, University of Pittsburgh
| | | | - Gary An
- Department of Surgery, University of Vermont Larner College of Medicine
| |
Collapse
|
4
|
Weaver DT, King ES, Maltas J, Scott JG. Reinforcement learning informs optimal treatment strategies to limit antibiotic resistance. Proc Natl Acad Sci U S A 2024; 121:e2303165121. [PMID: 38607932 PMCID: PMC11032439 DOI: 10.1073/pnas.2303165121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 02/23/2024] [Indexed: 04/14/2024] Open
Abstract
Antimicrobial resistance was estimated to be associated with 4.95 million deaths worldwide in 2019. It is possible to frame the antimicrobial resistance problem as a feedback-control problem. If we could optimize this feedback-control problem and translate our findings to the clinic, we could slow, prevent, or reverse the development of high-level drug resistance. Prior work on this topic has relied on systems where the exact dynamics and parameters were known a priori. In this study, we extend this work using a reinforcement learning (RL) approach capable of learning effective drug cycling policies in a system defined by empirically measured fitness landscapes. Crucially, we show that it is possible to learn effective drug cycling policies despite the problems of noisy, limited, or delayed measurement. Given access to a panel of 15 [Formula: see text]-lactam antibiotics with which to treat the simulated Escherichia coli population, we demonstrate that RL agents outperform two naive treatment paradigms at minimizing the population fitness over time. We also show that RL agents approach the performance of the optimal drug cycling policy. Even when stochastic noise is introduced to the measurements of population fitness, we show that RL agents are capable of maintaining evolving populations at lower growth rates compared to controls. We further tested our approach in arbitrary fitness landscapes of up to 1,024 genotypes. We show that minimization of population fitness using drug cycles is not limited by increasing genome size. Our work represents a proof-of-concept for using AI to control complex evolutionary processes.
Collapse
Affiliation(s)
- Davis T. Weaver
- Case Western Reserve University School of Medicine, Cleveland, OH44106
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland, OH44106
| | - Eshan S. King
- Case Western Reserve University School of Medicine, Cleveland, OH44106
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland, OH44106
| | - Jeff Maltas
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland, OH44106
| | - Jacob G. Scott
- Case Western Reserve University School of Medicine, Cleveland, OH44106
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland, OH44106
- Department of Physics, Case Western Reserve University, Cleveland, OH44106
| |
Collapse
|
5
|
Naik K, Goyal RK, Foschini L, Chak CW, Thielscher C, Zhu H, Lu J, Lehár J, Pacanoswki MA, Terranova N, Mehta N, Korsbo N, Fakhouri T, Liu Q, Gobburu J. Current Status and Future Directions: The Application of Artificial Intelligence/Machine Learning for Precision Medicine. Clin Pharmacol Ther 2024; 115:673-686. [PMID: 38103204 DOI: 10.1002/cpt.3152] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 11/28/2023] [Indexed: 12/18/2023]
Abstract
Technological innovations, such as artificial intelligence (AI) and machine learning (ML), have the potential to expedite the goal of precision medicine, especially when combined with increased capacity for voluminous data from multiple sources and expanded therapeutic modalities; however, they also present several challenges. In this communication, we first discuss the goals of precision medicine, and contextualize the use of AI in precision medicine by showcasing innovative applications (e.g., prediction of tumor growth and overall survival, biomarker identification using biomedical images, and identification of patient population for clinical practice) which were presented during the February 2023 virtual public workshop entitled "Application of Artificial Intelligence and Machine Learning for Precision Medicine," hosted by the US Food and Drug Administration (FDA) and University of Maryland Center of Excellence in Regulatory Science and Innovation (M-CERSI). Next, we put forward challenges brought about by the multidisciplinary nature of AI, particularly highlighting the need for AI to be trustworthy. To address such challenges, we subsequently note practical approaches, viz., differential privacy, synthetic data generation, and federated learning. The proposed strategies - some of which are highlighted presentations from the workshop - are for the protection of personal information and intellectual property. In addition, methods such as the risk-based management approach and the need for an agile regulatory ecosystem are discussed. Finally, we lay out a call for action that includes sharing of data and algorithms, development of regulatory guidance documents, and pooling of expertise from a broad-spectrum of stakeholders to enhance the application of AI in precision medicine.
Collapse
Affiliation(s)
- Kunal Naik
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Rahul K Goyal
- Center for Translational Medicine, University of Maryland School of Pharmacy, Baltimore, Maryland, USA
| | | | | | | | - Hao Zhu
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - James Lu
- Modeling & Simulation/Clinical Pharmacology, Genentech Inc., South San Francisco, California, USA
| | | | - Michael A Pacanoswki
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Nadia Terranova
- Quantitative Pharmacology, Ares Trading S.A. (an affiliate of Merck KGaA, Darmstadt, Germany), Lausanne, Switzerland
| | - Neha Mehta
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | | | - Tala Fakhouri
- Office of Medical Policy, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Qi Liu
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Jogarao Gobburu
- Center for Translational Medicine, University of Maryland School of Pharmacy, Baltimore, Maryland, USA
| |
Collapse
|
6
|
Lu Y, Chu Q, Li Z, Wang M, Gatenby R, Zhang Q. Deep reinforcement learning identifies personalized intermittent androgen deprivation therapy for prostate cancer. Brief Bioinform 2024; 25:bbae071. [PMID: 38493345 PMCID: PMC11174533 DOI: 10.1093/bib/bbae071] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Revised: 01/11/2024] [Accepted: 02/03/2024] [Indexed: 03/18/2024] Open
Abstract
The evolution of drug resistance leads to treatment failure and tumor progression. Intermittent androgen deprivation therapy (IADT) helps responsive cancer cells compete with resistant cancer cells in intratumoral competition. However, conventional IADT is population-based, ignoring the heterogeneity of patients and cancer. Additionally, existing IADT relies on pre-determined thresholds of prostate-specific antigen to pause and resume treatment, which is not optimized for individual patients. To address these challenges, we framed a data-driven method in two steps. First, we developed a time-varied, mixed-effect and generative Lotka-Volterra (tM-GLV) model to account for the heterogeneity of the evolution mechanism and the pharmacokinetics of two ADT drugs Cyproterone acetate and Leuprolide acetate for individual patients. Then, we proposed a reinforcement-learning-enabled individualized IADT framework, namely, I$^{2}$ADT, to learn the patient-specific tumor dynamics and derive the optimal drug administration policy. Experiments with clinical trial data demonstrated that the proposed I$^{2}$ADT can significantly prolong the time to progression of prostate cancer patients with reduced cumulative drug dosage. We further validated the efficacy of the proposed methods with a recent pilot clinical trial data. Moreover, the adaptability of I$^{2}$ADT makes it a promising tool for other cancers with the availability of clinical data, where treatment regimens might need to be individualized based on patient characteristics and disease dynamics. Our research elucidates the application of deep reinforcement learning to identify personalized adaptive cancer therapy.
Collapse
Affiliation(s)
- Yitao Lu
- School of Data Science, City University of Hong Kong, Hong Kong SAR, China
| | - Qian Chu
- Department of Thoracic Oncology, Tongji Hospital, Huazhong University of Science and Technology, 430030, Wuhan, China
| | - Zhen Li
- Department of Radiology, Tongji Hospital, Huazhong University of Science and Technology, 430030, Wuhan, China
| | - Mengdi Wang
- Department of Electrical and Computer Engineering and the Center for Statistics and Machine Learning, Princeton University, 08544, NJ, U.S.A
| | - Robert Gatenby
- Department of Integrated Mathematical Oncology and the Cancer Biology and Evolution Program, H. Lee Moffitt Cancer Center and Research Institute, 33612, FL, USA
| | - Qingpeng Zhang
- Musketeers Foundation Institute of Data Science and the Department of Pharmacology and Pharmacy, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| |
Collapse
|
7
|
Weaver DT, King ES, Maltas J, Scott JG. Reinforcement Learning informs optimal treatment strategies to limit antibiotic resistance. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.12.523765. [PMID: 36711676 PMCID: PMC9882109 DOI: 10.1101/2023.01.12.523765] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 01/15/2023]
Abstract
Antimicrobial resistance was estimated to be associated with 4.95 million deaths worldwide in 2019. It is possible to frame the antimicrobial resistance problem as a feedback-control problem. If we could optimize this feedback-control problem and translate our findings to the clinic, we could slow, prevent or reverse the development of high-level drug resistance. Prior work on this topic has relied on systems where the exact dynamics and parameters were known a priori. In this study, we extend this work using a reinforcement learning (RL) approach capable of learning effective drug cycling policies in a system defined by empirically measured fitness landscapes. Crucially, we show that is possible to learn effective drug cycling policies despite the problems of noisy, limited, or delayed measurement. Given access to a panel of 15 β-lactam antibiotics with which to treat the simulated E. coli population, we demonstrate that RL agents outperform two naive treatment paradigms at minimizing the population fitness over time. We also show that RL agents approach the performance of the optimal drug cycling policy. Even when stochastic noise is introduced to the measurements of population fitness, we show that RL agents are capable of maintaining evolving populations at lower growth rates compared to controls. We further tested our approach in arbitrary fitness landscapes of up to 1024 genotypes. We show that minimization of population fitness using drug cycles is not limited by increasing genome size. Our work represents a proof-of-concept for using AI to control complex evolutionary processes.
Collapse
Affiliation(s)
- Davis T. Weaver
- Case Western Reserve University School of Medicine, Cleveland, OH, 44106, USA
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland OH, 44106, USA
| | - Eshan S. King
- Case Western Reserve University School of Medicine, Cleveland, OH, 44106, USA
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland OH, 44106, USA
| | - Jeff Maltas
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland OH, 44106, USA
| | - Jacob G. Scott
- Case Western Reserve University School of Medicine, Cleveland, OH, 44106, USA
- Translational Hematology Oncology Research, Cleveland Clinic, Cleveland OH, 44106, USA
- Department of Physics, Case Western Reserve University, Cleveland, OH, 44106, USA
| |
Collapse
|
8
|
Feuerwerker S, Cockrell RC, An G. Characterizing the Crosstalk Between Programmed Cell Death Pathways in Cytokine Storm With an Agent-Based Model. Surg Infect (Larchmt) 2023; 24:725-733. [PMID: 37824803 PMCID: PMC10615089 DOI: 10.1089/sur.2023.115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2023] Open
Abstract
Background: There is increasing recognition of extensive crosstalk between programmed cell death pathways (PCDPs), such as apoptosis, pyroptosis, and necroptosis, resulting in a highly redundant system responsive to a breadth of potential pathogens. However, because pyroptosis and necroptosis propagate inflammation, these redundancies also present challenges for therapeutic control of dysregulated hyperinflammation seen in cytokine storm (CS) generated organ dysfunction. Hypothesis: We hypothesize that the conversion of existing knowledge regarding apoptosis, pyroptosis, and necroptosis into a computational model can enhance our understanding of the crosstalk between PCDPs via simulation experiments of microbe interactions and experimental interventions. Materials and Methods: Literature regarding apoptosis, pyroptosis, and necroptosis was reviewed and transposed into an agent-based model, the programmed cell death agent-based model (PCDABM). Computational experiments were performed to simulate the activation of various PCDPs as seen by differing microbes, specifically: influenza A virus (IAV), enteropathic Escherichia coli (EPEC), and Salmonella enterica (SE). The potential protective value of PCDP crosstalk was evaluated by silencing either pyroptosis, necroptosis, or both. Computational experiments were also performed simulating the effect of potential therapies blocking tumor necrosis factor (TNF) and interleukin (IL)-1. Results: The PCDABM was implemented in the agent-based modeling toolkit NetLogo. Computational experiments of infection with IAV, EPEC, and SE reproduced cross-activation of PCDPs with effective microbial clearance. Simulations of anti-TNF and anti-IL-1 did not reduce the aggregated amount of inflammation-generated system damage, the surrogate for CS-generated tissue damage. Conclusions: Redundancies have evolved in host PCDPs to maintain protection against a wide range of pathogens. However, these redundancies also challenge attempts at dampening the pathogenic hyperinflammatory state of CS using therapeutic immunomodulation. Integrative simulation models such as the PCDABM can aid in identifying potentially targetable inflection points to mitigate CS while maintaining effective host defense.
Collapse
Affiliation(s)
- Solomon Feuerwerker
- Department of Surgery, University of Vermont Larner College of Medicine, Burlington, Vermont, USA
| | - R. Chase Cockrell
- Department of Surgical Research, University of Vermont Larner College of Medicine, Burlington, Vermont, USA
| | - Gary An
- Department of Surgery, University of Vermont Larner College of Medicine, Burlington, Vermont, USA
| |
Collapse
|
9
|
Schuurman AR, Sloot PMA, Wiersinga WJ, van der Poll T. Embracing complexity in sepsis. Crit Care 2023; 27:102. [PMID: 36906606 PMCID: PMC10007743 DOI: 10.1186/s13054-023-04374-0] [Citation(s) in RCA: 19] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Accepted: 02/19/2023] [Indexed: 03/13/2023] Open
Abstract
Sepsis involves the dynamic interplay between a pathogen, the host response, the failure of organ systems, medical interventions and a myriad of other factors. This together results in a complex, dynamic and dysregulated state that has remained ungovernable thus far. While it is generally accepted that sepsis is very complex indeed, the concepts, approaches and methods that are necessary to understand this complexity remain underappreciated. In this perspective we view sepsis through the lens of complexity theory. We describe the concepts that support viewing sepsis as a state of a highly complex, non-linear and spatio-dynamic system. We argue that methods from the field of complex systems are pivotal for a fuller understanding of sepsis, and we highlight the progress that has been made over the last decades in this respect. Still, despite these considerable advancements, methods like computational modelling and network-based analyses continue to fly under the general scientific radar. We discuss what barriers contribute to this disconnect, and what we can do to embrace complexity with regards to measurements, research approaches and clinical applications. Specifically, we advocate a focus on longitudinal, more continuous biological data collection in sepsis. Understanding the complexity of sepsis will require a huge multidisciplinary effort, in which computational approaches derived from complex systems science must be supported by, and integrated with, biological data. Such integration could finetune computational models, guide validation experiments, and identify key pathways that could be targeted to modulate the system to the benefit of the host. We offer an example for immunological predictive modelling, which may inform agile trials that could be adjusted throughout the trajectory of disease. Overall, we argue that we should expand our current mental frameworks of sepsis, and embrace nonlinear, system-based thinking in order to move the field forward.
Collapse
Affiliation(s)
- Alex R Schuurman
- Centre for Experimental and Molecular Medicine (CEMM), Amsterdam University Medical Centres - Location AMC, University of Amsterdam, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands
| | - Peter M A Sloot
- Institute for Advanced Study, University of Amsterdam, Amsterdam, The Netherlands
| | - W Joost Wiersinga
- Centre for Experimental and Molecular Medicine (CEMM), Amsterdam University Medical Centres - Location AMC, University of Amsterdam, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands
- Division of Infectious Diseases, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands
| | - Tom van der Poll
- Centre for Experimental and Molecular Medicine (CEMM), Amsterdam University Medical Centres - Location AMC, University of Amsterdam, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands.
- Division of Infectious Diseases, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands.
| |
Collapse
|
10
|
Cockrell C, Larie D, An G. Preparing for the next pandemic: Simulation-based deep reinforcement learning to discover and test multimodal control of systemic inflammation using repurposed immunomodulatory agents. Front Immunol 2022; 13:995395. [PMID: 36479109 PMCID: PMC9720328 DOI: 10.3389/fimmu.2022.995395] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Accepted: 11/08/2022] [Indexed: 11/22/2022] Open
Abstract
Background Preparation to address the critical gap in a future pandemic between non-pharmacological measures and the deployment of new drugs/vaccines requires addressing two factors: 1) finding virus/pathogen-agnostic pathophysiological targets to mitigate disease severity and 2) finding a more rational approach to repurposing existing drugs. It is increasingly recognized that acute viral disease severity is heavily driven by the immune response to the infection ("cytokine storm" or "cytokine release syndrome"). There exist numerous clinically available biologics that suppress various pro-inflammatory cytokines/mediators, but it is extremely difficult to identify clinically effective treatment regimens with these agents. We propose that this is a complex control problem that resists standard methods of developing treatment regimens and accomplishing this goal requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training successful game-playing artificial intelligences (AIs). This proof-of-concept study determines if simulated sepsis (e.g. infection-driven cytokine storm) can be controlled in the absence of effective antimicrobial agents by targeting cytokines for which FDA-approved biologics currently exist. Methods We use a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. DRL training used a Deep Deterministic Policy Gradient (DDPG) approach with a clinically plausible control interval of 6 hours with manipulation of six cytokines for which there are existing drugs: Tumor Necrosis Factor (TNF), Interleukin-1 (IL-1), Interleukin-4 (IL-4), Interleukin-8 (IL-8), Interleukin-12 (IL-12) and Interferon-γ(IFNg). Results DRL trained an AI policy that could improve outcomes from a baseline Recovered Rate of 61% to one with a Recovered Rate of 90% over ~21 days simulated time. This DRL policy was then tested on four different parameterizations not seen in training representing a range of host and microbe characteristics, demonstrating a range of improvement in Recovered Rate by +33% to +56. Discussion The current proof-of-concept study demonstrates that significant disease severity mitigation can potentially be accomplished with existing anti-mediator drugs, but only through a multi-modal, adaptive treatment policy requiring implementation with an AI. While the actual clinical implementation of this approach is a projection for the future, the current goal of this work is to inspire the development of a research ecosystem that marries what is needed to improve the simulation models with the development of the sensing/assay technologies to collect the data needed to iteratively refine those models.
Collapse
|
11
|
Martinez-Millana A, Saez-Saez A, Tornero-Costa R, Azzopardi-Muscat N, Traver V, Novillo-Ortiz D. Artificial intelligence and its impact on the domains of universal health coverage, health emergencies and health promotion: An overview of systematic reviews. Int J Med Inform 2022; 166:104855. [PMID: 35998421 PMCID: PMC9551134 DOI: 10.1016/j.ijmedinf.2022.104855] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Revised: 08/01/2022] [Accepted: 08/11/2022] [Indexed: 12/04/2022]
Abstract
BACKGROUND Artificial intelligence is fueling a new revolution in medicine and in the healthcare sector. Despite the growing evidence on the benefits of artificial intelligence there are several aspects that limit the measure of its impact in people's health. It is necessary to assess the current status on the application of AI towards the improvement of people's health in the domains defined by WHO's Thirteenth General Programme of Work (GPW13) and the European Programme of Work (EPW), to inform about trends, gaps, opportunities, and challenges. OBJECTIVE To perform a systematic overview of systematic reviews on the application of artificial intelligence in the people's health domains as defined in the GPW13 and provide a comprehensive and updated map on the application specialties of artificial intelligence in terms of methodologies, algorithms, data sources, outcomes, predictors, performance, and methodological quality. METHODS A systematic search in MEDLINE, EMBASE, Cochrane and IEEEXplore was conducted between January 2015 and June 2021 to collect systematic reviews using a combination of keywords related to the domains of universal health coverage, health emergencies protection, and better health and wellbeing as defined by the WHO's PGW13 and EPW. Eligibility criteria was based on methodological quality and the inclusion of practical implementation of artificial intelligence. Records were classified and labeled using ICD-11 categories into the domains of the GPW13. Descriptors related to the area of implementation, type of modeling, data entities, outcomes and implementation on care delivery were extracted using a structured form and methodological aspects of the included reviews studies was assessed using the AMSTAR checklist. RESULTS The search strategy resulted in the screening of 815 systematic reviews from which 203 were assessed for eligibility and 129 were included in the review. The most predominant domain for artificial intelligence applications was Universal Health Coverage (N = 98) followed by Health Emergencies (N = 16) and Better Health and Wellbeing (N = 15). Neoplasms area on Universal Health Coverage was the disease area featuring most of the applications (21.7 %, N = 28). The reviews featured analytics primarily over both public and private data sources (67.44 %, N = 87). The most used type of data was medical imaging (31.8 %, N = 41) and predictors based on regions of interest and clinical data. The most prominent subdomain of Artificial Intelligence was Machine Learning (43.4 %, N = 56), in which Support Vector Machine method was predominant (20.9 %, N = 27). Regarding the purpose, the application of Artificial Intelligence I is focused on the prediction of the diseases (36.4 %, N = 47). With respect to the validation, more than a half of the reviews (54.3 %, N = 70) did not report a validation procedure and, whenever available, the main performance indicator was the accuracy (28.7 %, N = 37). According to the methodological quality assessment, a third of the reviews (34.9 %, N = 45) implemented methods for analysis the risk of bias and the overall AMSTAR score below was 5 (4.01 ± 1.93) on all the included systematic reviews. CONCLUSION Artificial intelligence is being used for disease modelling, diagnose, classification and prediction in the three domains of GPW13. However, the evidence is often limited to laboratory and the level of adoption is largely unbalanced between ICD-11 categoriesand diseases. Data availability is a determinant factor on the developmental stage of artificial intelligence applications. Most of the reviewed studies show a poor methodological quality and are at high risk of bias, which limits the reproducibility of the results and the reliability of translating these applications to real clinical scenarios. The analyzed papers show results only in laboratory and testing scenarios and not in clinical trials nor case studies, limiting the supporting evidence to transfer artificial intelligence to actual care delivery.
Collapse
Affiliation(s)
- Antonio Martinez-Millana
- Instituto Universitario de Investigación de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera S/N, Valencia 46022, Spain
| | - Aida Saez-Saez
- Instituto Universitario de Investigación de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera S/N, Valencia 46022, Spain
| | - Roberto Tornero-Costa
- Instituto Universitario de Investigación de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera S/N, Valencia 46022, Spain
| | - Natasha Azzopardi-Muscat
- Division of Country Health Policies and Systems, World Health Organization, Regional Office for Europe, Copenhagen, Denmark
| | - Vicente Traver
- Instituto Universitario de Investigación de Aplicaciones de las Tecnologías de la Información y de las Comunicaciones Avanzadas (ITACA), Universitat Politècnica de València, Camino de Vera S/N, Valencia 46022, Spain
| | - David Novillo-Ortiz
- Division of Country Health Policies and Systems, World Health Organization, Regional Office for Europe, Copenhagen, Denmark.
| |
Collapse
|
12
|
An G, Cockrell C. Drug Development Digital Twins for Drug Discovery, Testing and Repurposing: A Schema for Requirements and Development. FRONTIERS IN SYSTEMS BIOLOGY 2022; 2:928387. [PMID: 35935475 PMCID: PMC9351294 DOI: 10.3389/fsysb.2022.928387] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
There has been a great deal of interest in the concept, development and implementation of medical digital twins. This interest has led to wide ranging perceptions of what constitutes a medical digital twin. This Perspectives article will provide 1) a description of fundamental features of industrial digital twins, the source of the digital twin concept, 2) aspects of biology that challenge the implementation of medical digital twins, 3) a schematic program of how a specific medical digital twin project could be defined, and 4) an example description within that schematic program for a specific type of medical digital twin intended for drug discovery, testing and repurposing, the Drug Development Digital Twin (DDDT).
Collapse
Affiliation(s)
- Gary An
- Department of Surgery, University of Vermont Larner College of Medicine, Burlington, VT, United States
| | - Chase Cockrell
- Department of Surgery, University of Vermont Larner College of Medicine, Burlington, VT, United States
| |
Collapse
|
13
|
Prybutok AN, Cain JY, Leonard JN, Bagheri N. Fighting fire with fire: deploying complexity in computational modeling to effectively characterize complex biological systems. Curr Opin Biotechnol 2022; 75:102704. [DOI: 10.1016/j.copbio.2022.102704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Revised: 01/27/2022] [Accepted: 02/06/2022] [Indexed: 11/03/2022]
|
14
|
Dong Y, Wang G, Zhang J, Zhang S, Chen X, Guo Q, Qu F, Shou F. Robotic laser position versus freehand in CT-guided percutaneous microwave ablation for single hepatocellular carcinoma (diameter < 3 cm): a preliminary study. Int J Hyperthermia 2022; 39:725-732. [PMID: 35584811 DOI: 10.1080/02656736.2022.2072526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022] Open
Abstract
PURPOSE To compare the accuracy and safety of robotic laser position (RLP) versus freehand for antenna CT-guided microwave ablation (MWA) of single hepatocellular carcinoma (HCC) (diameter < 3 cm). MATERIALS AND METHODS This retrospective study was conducted between May 2020 and June 2021. A total of 40 patients with early HCC who underwent CT-guided MWA were divided into two groups: a freehand group (n = 20) and a RLP group (n = 20). Based on in-plane and out-of-plane data, the actual puncture point error (APPE), number of repositioning procedures, and operative duration were compared using the Mann-Whitney U test. Ablation-related complications were compared using the Chi-squared test. RESULTS The mean diameter of HCC patients who received MWA was 2.4 ± 0.5 cm. For in-plane APPE, APPE was comparable between the two groups (p = 0.299). However, for the out-of-plane position, the APPE in the freehand group was higher than that in the RLP group (p = 0.027). The number of repositioning procedures was 0 (range, 0-0) for RLP-guided procedures and 3 (range, 2-5) for freehand procedures, showing a statistically significant difference between the two groups (p < 0.001). The mean operative duration for freehand procedures was 39 min, compared with 26 min for RLP-guided procedures, showing a significant difference (p = 0.013). No deaths or major complications were directly related to MWA. Minor complications in the freehand group were comparable with those in the RLP group (p = 0.313). CONCLUSION RLP guidance significantly reduces the number of antenna repositioning procedures in MWA and improves puncture accuracy for target HCC out-of-plane. In addition, the operative duration of robotic guidance was shorter than that of freehand guidance.
Collapse
Affiliation(s)
- Yuru Dong
- Department of Radiology, The Third Medical Center of PLA General Hospital, Beijing, China
| | - Guisheng Wang
- Department of Radiology, The Third Medical Center of PLA General Hospital, Beijing, China
| | - Jingjun Zhang
- Department of Oncology, The People's Hospital of Jianyang City, Chengdu, China
| | - Shu Zhang
- Research Center for Birth Defects Prevention and Control Technology, Chinese PLA General Hospital, Beijing, China
| | - Xiaoxia Chen
- Department of Radiology, The Third Medical Center of PLA General Hospital, Beijing, China
| | - Qing Guo
- Department of Research, Characteristic Medical Center of Chinese People's Armed Police Force, Tianjin, China
| | - Feihuan Qu
- Readitec Medical Systems (Chengdu) Co., Ltd, Chengdu, China
| | - Feng Shou
- Department of Oncology, The People's Hospital of Jianyang City, Chengdu, China
| |
Collapse
|
15
|
Madamanchi A, Thomas M, Magana A, Heiland R, Macklin P. Supporting Computational Apprenticeship Through Educational and Software Infrastructure: A Case Study in a Mathematical Oncology Research Lab. PRIMUS 2022; 32:446-467. [PMID: 35197716 PMCID: PMC8863170 DOI: 10.1080/10511970.2021.1881849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
There is growing awareness of the need for mathematics and computing to quantitatively understand the complex dynamics and feedbacks in the life sciences. Although several institutions and research groups are conducting pioneering multidisciplinary research, communication and education across fields remain a bottleneck. The opportunity is ripe for using education research-supported mechanisms of cross-disciplinary training at the intersection of mathematics, computation, and biology. This case study uses the computational apprenticeship theoretical framework to describe the efforts of a computational biology lab to rapidly prototype, test, and refine a mentorship infrastructure for undergraduate research experiences. We describe the challenges, benefits, and lessons learned, as well as the utility of the computational apprenticeship framework in supporting computational/math students learning and contributing to biology, and biologists in learning computational methods. We also explore implications for undergraduate classroom instruction and cross-disciplinary scientific communication.
Collapse
|
16
|
High-dimensional role of AI and machine learning in cancer research. Br J Cancer 2022; 126:523-532. [PMID: 35013580 PMCID: PMC8854697 DOI: 10.1038/s41416-021-01689-z] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 11/23/2021] [Accepted: 12/23/2021] [Indexed: 01/12/2023] Open
Abstract
The role of Artificial Intelligence and Machine Learning in cancer research offers several advantages, primarily scaling up the information processing and increasing the accuracy of the clinical decision-making. The key enabling tools currently in use in Precision, Digital and Translational Medicine, here named as 'Intelligent Systems' (IS), leverage unprecedented data volumes and aim to model their underlying heterogeneous influences and variables correlated with patients' outcomes. As functionality and performance of IS are associated with complex diagnosis and therapy decisions, a rich spectrum of patterns and features detected in high-dimensional data may be critical for inference purposes. Many challenges are also present in such discovery task. First, the generation of interpretable model results from a mix of structured and unstructured input information. Second, the design, and implementation of automated clinical decision processes for drawing disease trajectories and patient profiles. Ultimately, the clinical impacts depend on the data effectively subjected to steps such as harmonisation, integration, validation, etc. The aim of this work is to discuss the transformative value of IS applied to multimodal data acquired through various interrelated cancer domains (high-throughput genomics, experimental biology, medical image processing, radiomics, patient electronic records, etc.).
Collapse
|
17
|
Larie D, An G, Cockrell C. Preparing for the next COVID: Deep Reinforcement Learning trained Artificial Intelligence discovery of multi-modal immunomodulatory control of systemic inflammation in the absence of effective anti-microbials. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2022:2022.02.17.480940. [PMID: 35194613 PMCID: PMC8863155 DOI: 10.1101/2022.02.17.480940] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Background Despite a great deal of interest in the application of artificial intelligence (AI) to sepsis/critical illness, most current approaches are limited in their potential impact: prediction models do not (and cannot) address the lack of effective therapeutics and current approaches to enhancing the treatment of sepsis focus on optimizing the application of existing interventions, and thus cannot address the development of new treatment options/modalities. The inability to test new therapeutic applications was highlighted by the generally unsatisfactory results from drug repurposing efforts in COVID-19. Hypothesis Addressing this challenge requires the application of simulation-based, model-free deep reinforcement learning (DRL) in a fashion akin to training the game-playing AIs. We have previously demonstrated the potential of this method in the context of bacterial sepsis in which the microbial infection is responsive to antibiotic therapy. The current work addresses the control problem of multi-modal, adaptive immunomodulation in the circumstance where there is no effective anti-pathogen therapy (e.g., in a novel viral pandemic or in the face of resistant microbes). Methods This is a proof-of-concept study that determines the controllability of sepsis without the ability to pharmacologically suppress the pathogen. We use as a surrogate system a previously validated agent-based model, the Innate Immune Response Agent-based Model (IIRABM), for control discovery using DRL. The DRL algorithm 'trains' an AI on simulations of infection where both the control and observation spaces are limited to operating upon the defined immune mediators included in the IIRABM (a total of 11). Policies were learned using the Deep Deterministic Policy Gradient approach, with the objective function being a return to baseline system health. Results DRL trained an AI policy that improved system mortality from 85% to 10.4%. Control actions affected every one of the 11 targetable cytokines and could be divided into those with static/unchanging controls and those with variable/adaptive controls. Adaptive controls primarily targeted 3 different aspects of the immune response: 2nd order pro-inflammation governing TH1/TH2 balance, primary anti-inflammation, and inflammatory cell proliferation. Discussion The current treatment of sepsis is hampered by limitations in therapeutic options able to affect the biology of sepsis. This is heightened in circumstances where no effective antimicrobials exist, as was the case for COVID-19. Current AI methods are intrinsically unable to address this problem; doing so requires training AIs in contexts that fully represent the counterfactual space of potential treatments. The synthetic data needed for this task is only possible through the use of high-resolution, mechanism-based simulations. Finally, being able to treat sepsis will require a reorientation as to the sensing and actuating requirements needed to develop these simulations and bring them to the bedside.
Collapse
Affiliation(s)
- Dale Larie
- Department of Surgery, University of Vermont Larner College of Medicine
| | - Gary An
- Department of Surgery, University of Vermont Larner College of Medicine
| | - Chase Cockrell
- Department of Surgery, University of Vermont Larner College of Medicine
| |
Collapse
|
18
|
Jovel J, Greiner R. An Introduction to Machine Learning Approaches for Biomedical Research. Front Med (Lausanne) 2021; 8:771607. [PMID: 34977072 PMCID: PMC8716730 DOI: 10.3389/fmed.2021.771607] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2021] [Accepted: 11/18/2021] [Indexed: 11/15/2022] Open
Abstract
Machine learning (ML) approaches are a collection of algorithms that attempt to extract patterns from data and to associate such patterns with discrete classes of samples in the data—e.g., given a series of features describing persons, a ML model predicts whether a person is diseased or healthy, or given features of animals, it predicts weather an animal is treated or control, or whether molecules have the potential to interact or not, etc. ML approaches can also find such patterns in an agnostic manner, i.e., without having information about the classes. Respectively, those methods are referred to as supervised and unsupervised ML. A third type of ML is reinforcement learning, which attempts to find a sequence of actions that contribute to achieving a specific goal. All of these methods are becoming increasingly popular in biomedical research in quite diverse areas including drug design, stratification of patients, medical images analysis, molecular interactions, prediction of therapy outcomes and many more. We describe several supervised and unsupervised ML techniques, and illustrate a series of prototypical examples using state-of-the-art computational approaches. Given the complexity of reinforcement learning, it is not discussed in detail here, instead, interested readers are referred to excellent reviews on that topic. We focus on concepts rather than procedures, as our goal is to attract the attention of researchers in biomedicine toward the plethora of powerful ML methods and their potential to leverage basic and applied research programs.
Collapse
Affiliation(s)
- Juan Jovel
- The Metabolomics Innovation Centre, University of Alberta, Edmonton, AB, Canada
- *Correspondence: Juan Jovel
| | - Russell Greiner
- Faculty of Science-Computing Science, University of Alberta, Edmonton, AB, Canada
- Russell Greiner
| |
Collapse
|
19
|
Getz M, Wang Y, An G, Asthana M, Becker A, Cockrell C, Collier N, Craig M, Davis CL, Faeder JR, Ford Versypt AN, Mapder T, Gianlupi JF, Glazier JA, Hamis S, Heiland R, Hillen T, Hou D, Islam MA, Jenner AL, Kurtoglu F, Larkin CI, Liu B, Macfarlane F, Maygrundter P, Morel PA, Narayanan A, Ozik J, Pienaar E, Rangamani P, Saglam AS, Shoemaker JE, Smith AM, Weaver JJA, Macklin P. Iterative community-driven development of a SARS-CoV-2 tissue simulator. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2021:2020.04.02.019075. [PMID: 32511322 PMCID: PMC7239052 DOI: 10.1101/2020.04.02.019075] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
The 2019 novel coronavirus, SARS-CoV-2, is a pathogen of critical significance to international public health. Knowledge of the interplay between molecular-scale virus-receptor interactions, single-cell viral replication, intracellular-scale viral transport, and emergent tissue-scale viral propagation is limited. Moreover, little is known about immune system-virus-tissue interactions and how these can result in low-level (asymptomatic) infections in some cases and acute respiratory distress syndrome (ARDS) in others, particularly with respect to presentation in different age groups or pre-existing inflammatory risk factors. Given the nonlinear interactions within and among each of these processes, multiscale simulation models can shed light on the emergent dynamics that lead to divergent outcomes, identify actionable "choke points" for pharmacologic interventions, screen potential therapies, and identify potential biomarkers that differentiate patient outcomes. Given the complexity of the problem and the acute need for an actionable model to guide therapy discovery and optimization, we introduce and iteratively refine a prototype of a multiscale model of SARS-CoV-2 dynamics in lung tissue. The first prototype model was built and shared internationally as open source code and an online interactive model in under 12 hours, and community domain expertise is driving regular refinements. In a sustained community effort, this consortium is integrating data and expertise across virology, immunology, mathematical biology, quantitative systems physiology, cloud and high performance computing, and other domains to accelerate our response to this critical threat to international health. More broadly, this effort is creating a reusable, modular framework for studying viral replication and immune response in tissues, which can also potentially be adapted to related problems in immunology and immunotherapy.
Collapse
|
20
|
Larie D, An G, Cockrell RC. The Use of Artificial Neural Networks to Forecast the Behavior of Agent-Based Models of Pathophysiology: An Example Utilizing an Agent-Based Model of Sepsis. Front Physiol 2021; 12:716434. [PMID: 34721057 PMCID: PMC8552109 DOI: 10.3389/fphys.2021.716434] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2021] [Accepted: 09/24/2021] [Indexed: 11/13/2022] Open
Abstract
Introduction: Disease states are being characterized at finer and finer levels of resolution via biomarker or gene expression profiles, while at the same time. Machine learning (ML) is increasingly used to analyze and potentially classify or predict the behavior of biological systems based on such characterization. As ML applications are extremely data-intensive, given the relative sparsity of biomedical data sets ML training of artificial neural networks (ANNs) often require the use of synthetic training data. Agent-based models (ABMs) that incorporate known biological mechanisms and their associated stochastic properties are a potential means of generating synthetic data. Herein we present an example of ML used to train an artificial neural network (ANN) as a surrogate system used to predict the time evolution of an ABM focusing on the clinical condition of sepsis. Methods: The disease trajectories for clinical sepsis, in terms of temporal cytokine and phenotypic dynamics, can be interpreted as a random dynamical system. The Innate Immune Response Agent-based Model (IIRABM) is a well-established model that utilizes known cellular and molecular rules to simulate disease trajectories corresponding to clinical sepsis. We have utilized two distinct neural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take a time sequence of five measurements of eleven IIRABM simulated serum cytokine concentrations as input and to return both the future cytokine trajectories as well as an aggregate metric representing the patient's state of health. Results: The ANNs predicted model trajectories with the expected amount of error, due to stochasticity in the simulation, and recognizing that the mapping from a specific cytokine profile to a state-of-health is not unique. The Multi-Layer Perceptron neural network, generated predictions with a more accurate forecasted trajectory cone. Discussion: This work serves as a proof-of-concept for the use of ANNs to predict disease progression in sepsis as represented by an ABM. The findings demonstrate that multicellular systems with intrinsic stochasticity can be approximated with an ANN, but that forecasting a specific trajectory of the system requires sequential updating of the system state to provide a rolling forecast horizon.
Collapse
Affiliation(s)
| | | | - R. Chase Cockrell
- Department of Surgery, Larner College of Medicine, University of Vermont, Burlington, VT, United States
| |
Collapse
|
21
|
Comparative Computational Modeling of the Bat and Human Immune Response to Viral Infection with the Comparative Biology Immune Agent Based Model. Viruses 2021; 13:v13081620. [PMID: 34452484 PMCID: PMC8402910 DOI: 10.3390/v13081620] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2021] [Revised: 08/05/2021] [Accepted: 08/13/2021] [Indexed: 12/22/2022] Open
Abstract
Given the impact of pandemics due to viruses of bat origin, there is increasing interest in comparative investigation into the differences between bat and human immune responses. The practice of comparative biology can be enhanced by computational methods used for dynamic knowledge representation to visualize and interrogate the putative differences between the two systems. We present an agent based model that encompasses and bridges differences between bat and human responses to viral infection: the comparative biology immune agent based model, or CBIABM. The CBIABM examines differences in innate immune mechanisms between bats and humans, specifically regarding inflammasome activity and type 1 interferon dynamics, in terms of tolerance to viral infection. Simulation experiments with the CBIABM demonstrate the efficacy of bat-related features in conferring viral tolerance and also suggest a crucial role for endothelial inflammasome activity as a mechanism for bat systemic viral tolerance and affecting the severity of disease in human viral infections. We hope that this initial study will inspire additional comparative modeling projects to link, compare, and contrast immunological functions shared across different species, and in so doing, provide insight and aid in preparation for future viral pandemics of zoonotic origin.
Collapse
|
22
|
Byerly S, Maurer LR, Mantero A, Naar L, An G, Kaafarani HMA. Machine Learning and Artificial Intelligence for Surgical Decision Making. Surg Infect (Larchmt) 2021; 22:626-634. [PMID: 34270361 DOI: 10.1089/sur.2021.007] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022] Open
Abstract
Background: The use of machine learning (ML) and artificial intelligence (AI) in medical research continues to grow as the amount and availability of clinical data expands. These techniques allow complex interpretation of data and capture non-linear relations not immediately apparent by classic statistical techniques. Methods: This review of the ML/AI literature provides a brief overview for practicing surgeons and clinicians of the current and future roles these methods will have within surgical infection research. Results: A conceptual overview of the techniques is provided along with concrete examples in the surgical infections literature. Further examples of ML/AI techniques in clinical decision support as well as therapy discovery with model-based deep reinforcement learning are illustrated. Conclusions: Artificial intelligence and ML are important and increasingly utilized techniques within the expanding body of surgical infection research. This article provides a minimal baseline literacy in ML/AI to be able to view such projects in an appropriately critical fashion.
Collapse
Affiliation(s)
- Saskya Byerly
- Department of Surgery, University of Tennessee Health Science Center, Memphis, Tennessee, USA
| | - Lydia R Maurer
- Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Alejandro Mantero
- Division of Biostatistics, Department of Public Health Sciences, University of Miami, Miami, Florida, USA
| | - Leon Naar
- Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| | - Gary An
- Department of Surgery, University of Vermont, Burlington, Vermont, USA
| | - Haytham M A Kaafarani
- Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
23
|
Cockrell C, An G. Utilizing the Heterogeneity of Clinical Data for Model Refinement and Rule Discovery Through the Application of Genetic Algorithms to Calibrate a High-Dimensional Agent-Based Model of Systemic Inflammation. Front Physiol 2021; 12:662845. [PMID: 34093225 PMCID: PMC8172123 DOI: 10.3389/fphys.2021.662845] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Accepted: 04/27/2021] [Indexed: 12/31/2022] Open
Abstract
Introduction: Accounting for biological heterogeneity represents one of the greatest challenges in biomedical research. Dynamic computational and mathematical models can be used to enhance the study and understanding of biological systems, but traditional methods for calibration and validation commonly do not account for the heterogeneity of biological data, which may result in overfitting and brittleness of these models. Herein we propose a machine learning approach that utilizes genetic algorithms (GAs) to calibrate and refine an agent-based model (ABM) of acute systemic inflammation, with a focus on accounting for the heterogeneity seen in a clinical data set, thereby avoiding overfitting and increasing the robustness and potential generalizability of the underlying simulation model. Methods: Agent-based modeling is a frequently used modeling method for multi-scale mechanistic modeling. However, the same properties that make ABMs well suited to representing biological systems also present significant challenges with respect to their construction and calibration, particularly with respect to the selection of potential mechanistic rules and the large number of associated free parameters. We have proposed that machine learning approaches (such as GAs) can be used to more effectively and efficiently deal with rule selection and parameter space characterization; the current work applies GAs to the challenge of calibrating a complex ABM to a specific data set, while preserving biological heterogeneity reflected in the range and variance of the data. This project uses a GA to augment the rule-set for a previously validated ABM of acute systemic inflammation, the Innate Immune Response ABM (IIRABM) to clinical time series data of systemic cytokine levels from a population of burn patients. The genome for the GA is a vector generated from the IIRABM's Model Rule Matrix (MRM), which is a matrix representation of not only the constants/parameters associated with the IIRABM's cytokine interaction rules, but also the existence of rules themselves. Capturing heterogeneity is accomplished by a fitness function that incorporates the sample value range ("error bars") of the clinical data. Results: The GA-enabled parameter space exploration resulted in a set of putative MRM rules and associated parameterizations which closely match the cytokine time course data used to design the fitness function. The number of non-zero elements in the MRM increases significantly as the model parameterizations evolve toward a fitness function minimum, transitioning from a sparse to a dense matrix. This results in a model structure that more closely resembles (at a superficial level) the structure of data generated by a standard differential gene expression experimental study. Conclusion: We present an HPC-enabled machine learning/evolutionary computing approach to calibrate a complex ABM to complex clinical data while preserving biological heterogeneity. The integration of machine learning, HPC, and multi-scale mechanistic modeling provides a pathway forward to more effectively representing the heterogeneity of clinical populations and their data.
Collapse
Affiliation(s)
- Chase Cockrell
- Departmen of Surgery, Larner College of Medicine, The University of Vermont, Burlington, VT, United States
| | | |
Collapse
|
24
|
Yang R, Wang Z, Chen J. An Integrated Approach of Mechanistic-Modeling and Machine-Learning for Thickness Optimization of Frozen Microwaveable Foods. Foods 2021; 10:foods10040763. [PMID: 33916660 PMCID: PMC8066635 DOI: 10.3390/foods10040763] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2021] [Revised: 03/25/2021] [Accepted: 04/01/2021] [Indexed: 11/16/2022] Open
Abstract
Mechanistic-modeling has been a useful tool to help food scientists in understanding complicated microwave-food interactions, but it cannot be directly used by the food developers for food design due to its resource-intensive characteristic. This study developed and validated an integrated approach that coupled mechanistic-modeling and machine-learning to achieve efficient food product design (thickness optimization) with better heating uniformity. The mechanistic-modeling that incorporated electromagnetics and heat transfer was previously developed and validated extensively and was used directly in this study. A Bayesian optimization machine-learning algorithm was developed and integrated with the mechanistic-modeling. The integrated approach was validated by comparing the optimization performance with a parametric sweep approach, which is solely based on mechanistic-modeling. The results showed that the integrated approach had the capability and robustness to optimize the thickness of different-shape products using different initial training datasets with higher efficiency (45.9% to 62.1% improvement) than the parametric sweep approach. Three rectangular-shape trays with one optimized thickness (1.56 cm) and two non-optimized thicknesses (1.20 and 2.00 cm) were 3-D printed and used in microwave heating experiments, which confirmed the feasibility of the integrated approach in thickness optimization. The integrated approach can be further developed and extended as a platform to efficiently design complicated microwavable foods with multiple-parameter optimization.
Collapse
Affiliation(s)
- Ran Yang
- Department of Food Science, University of Tennessee, Knoxville, TN 37996, USA;
| | - Zhenbo Wang
- Department of Mechanical, Aerospace and Biomedical Engineering, University of Tennessee, Knoxville, TN 37996, USA;
| | - Jiajia Chen
- Department of Food Science, University of Tennessee, Knoxville, TN 37996, USA;
- Correspondence: ; Tel.: +1-865-974-8226
| |
Collapse
|
25
|
Digital Twins for Tissue Culture Techniques—Concepts, Expectations, and State of the Art. Processes (Basel) 2021. [DOI: 10.3390/pr9030447] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022] Open
Abstract
Techniques to provide in vitro tissue culture have undergone significant changes during the last decades, and current applications involve interactions of cells and organoids, three-dimensional cell co-cultures, and organ/body-on-chip tools. Efficient computer-aided and mathematical model-based methods are required for efficient and knowledge-driven characterization, optimization, and routine manufacturing of tissue culture systems. As an alternative to purely experimental-driven research, the usage of comprehensive mathematical models as a virtual in silico representation of the tissue culture, namely a digital twin, can be advantageous. Digital twins include the mechanistic of the biological system in the form of diverse mathematical models, which describe the interaction between tissue culture techniques and cell growth, metabolism, and the quality of the tissue. In this review, current concepts, expectations, and the state of the art of digital twins for tissue culture concepts will be highlighted. In general, DT’s can be applied along the full process chain and along the product life cycle. Due to the complexity, the focus of this review will be especially on the design, characterization, and operation of the tissue culture techniques.
Collapse
|
26
|
|
27
|
Hayama Nishida CE, Costa Bianchi RA, Reali Costa AH. A framework to shift basins of attraction of gene regulatory networks through batch reinforcement learning. Artif Intell Med 2020; 107:101853. [PMID: 32828434 DOI: 10.1016/j.artmed.2020.101853] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2019] [Revised: 03/23/2020] [Accepted: 03/31/2020] [Indexed: 11/25/2022]
Abstract
A major challenge in gene regulatory networks (GRN) of biological systems is to discover when and what interventions should be applied to shift them to healthy phenotypes. A set of gene activity profiles, called basin of attraction (BOA), takes this network to a specific phenotype; therefore, a healthy BOA leads the GRN to a healthy phenotype. However, without the complete observability of the genes, it is not possible to identify whether the current BOA is healthy. In this article we investigate external interventions in GRN with partial observability aiming to bring it to healthy BOAs. We propose a new batch reinforcement learning method (BRL), called mSFQI, to define intervention strategies based on the probabilities of the gene activity profiles being in healthy BOAs, which are calculated from a set of previous observed experiences. BRL uses approximation functions and repeated applications of previous experiences to accelerate learning. Results demonstrate that our proposal can quickly shift a partially observable GRN to healthy BOAs, while reducing the number of interventions. In addition, when observability is poor, mSFQI produces better results when the probabilities for a greater amount of previous observations are available.
Collapse
|
28
|
Abudukelimu A, Barberis M, Redegeld F, Sahin N, Sharma RP, Westerhoff HV. Complex Stability and an Irrevertible Transition Reverted by Peptide and Fibroblasts in a Dynamic Model of Innate Immunity. Front Immunol 2020; 10:3091. [PMID: 32117197 PMCID: PMC7033641 DOI: 10.3389/fimmu.2019.03091] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2019] [Accepted: 12/17/2019] [Indexed: 12/12/2022] Open
Abstract
We here apply a control analysis and various types of stability analysis to an in silico model of innate immunity that addresses the management of inflammation by a therapeutic peptide. Motivation is the observation, both in silico and in experiments, that this therapy is not robust. Our modeling results demonstrate how (1) the biological phenomena of acute and chronic modes of inflammation may reflect an inherently complex bistability with an irrevertible flip between the two modes, (2) the chronic mode of the model has stable, sometimes unique, steady states, while its acute-mode steady states are stable but not unique, (3) as witnessed by TNF levels, acute inflammation is controlled by multiple processes, whereas its chronic-mode inflammation is only controlled by TNF synthesis and washout, (4) only when the antigen load is close to the acute mode's flipping point, many processes impact very strongly on cells and cytokines, (5) there is no antigen exposure level below which reduction of the antigen load alone initiates a flip back to the acute mode, and (6) adding healthy fibroblasts makes the transition from acute to chronic inflammation revertible, although (7) there is a window of antigen load where such a therapy cannot be effective. This suggests that triple therapies may be essential to overcome chronic inflammation. These may comprise (1) anti-immunoglobulin light chain peptides, (2) a temporarily reduced antigen load, and (3a) fibroblast repopulation or (3b) stem cell strategies.
Collapse
Affiliation(s)
- Abulikemu Abudukelimu
- Synthetic Systems Biology and Nuclear Organization, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, Netherlands.,Molecular Cell Physiology, VU University Amsterdam, Amsterdam, Netherlands
| | - Matteo Barberis
- Synthetic Systems Biology and Nuclear Organization, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, Netherlands.,Systems Biology, School of Biosciences and Medicine, Faculty of Health and Medical Sciences, University of Surrey, Guildford, United Kingdom.,Centre for Mathematical and Computational Biology, CMCB, University of Surrey, Guildford, United Kingdom
| | - Frank Redegeld
- Division of Pharmacology, Department of Pharmaceutical Sciences, Faculty of Science, Utrecht University, Utrecht, Netherlands
| | - Nilgun Sahin
- Molecular Cell Physiology, VU University Amsterdam, Amsterdam, Netherlands
| | - Raju P Sharma
- Molecular Cell Physiology, VU University Amsterdam, Amsterdam, Netherlands
| | - Hans V Westerhoff
- Synthetic Systems Biology and Nuclear Organization, Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, Netherlands.,Molecular Cell Physiology, VU University Amsterdam, Amsterdam, Netherlands.,School for Chemical Engineering and Analytical Science, University of Manchester, Manchester, United Kingdom.,Systems Biology Amsterdam, VU University Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
29
|
Vodovotz Y, An G. Agent-based models of inflammation in translational systems biology: A decade later. WILEY INTERDISCIPLINARY REVIEWS. SYSTEMS BIOLOGY AND MEDICINE 2019; 11:e1460. [PMID: 31260168 PMCID: PMC8140858 DOI: 10.1002/wsbm.1460] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/26/2019] [Revised: 06/14/2019] [Accepted: 06/15/2019] [Indexed: 12/11/2022]
Abstract
Agent-based modeling is a rule-based, discrete-event, and spatially explicit computational modeling method that employs computational objects that instantiate the rules and interactions among the individual components ("agents") of system. Agent-based modeling is well suited to translating into a computational model the knowledge generated from basic science research, particularly with respect to translating across scales the mechanisms of cellular behavior into aggregated cell population dynamics manifesting at the tissue and organ level. This capacity has made agent-based modeling an integral method in translational systems biology (TSB), an approach that uses multiscale dynamic computational modeling to explicitly represent disease processes in a clinically relevant fashion. The initial work in the early 2000s using agent-based models (ABMs) in TSB focused on examining acute inflammation and its intersection with wound healing; the decade since has seen vast growth in both the application of agent-based modeling to a wide array of disease processes as well as methodological advancements in the use and analysis of ABM. This report presents an update on an earlier review of ABMs in TSB and presents examples of exciting progress in the modeling of various organs and diseases that involve inflammation. This review also describes developments that integrate the use of ABMs with cutting-edge technologies such as high-performance computing, machine learning, and artificial intelligence, with a view toward the future integration of these methodologies. This article is categorized under: Translational, Genomic, and Systems Medicine > Translational Medicine Models of Systems Properties and Processes > Mechanistic Models Models of Systems Properties and Processes > Organ, Tissue, and Physiological Models Models of Systems Properties and Processes > Organismal Models.
Collapse
Affiliation(s)
- Yoram Vodovotz
- Department of Surgery, Immunology, Computational & Systems Biology, University of Pittsburgh, Pittsburgh, Pennsylvania
- Department of Bioengineering, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Gary An
- Department of Surgery, University of Vermont, Burlington, Vermont
| |
Collapse
|