The prisoner's dilemma as a cancer model

Pediatric Leukemia Roadmaps Are a Guide for Positive Metastatic Bone Sarcoma Trials

ALL cures require many MRD therapies. This strategy should drive experiments and trials in metastatic bone sarcomas.

Impact of Resistance on Therapeutic Design: A Moran Model of Cancer Growth

Resistance of cancers to treatments, such as chemotherapy, largely arise due to cell mutations. These mutations allow cells to resist apoptosis and inevitably lead to recurrence and often progression to more aggressive cancer forms. Sustained-low dose therapies are being considered as an alternative over maximum tolerated dose treatments, whereby a smaller drug dosage is given over a longer period of time. However, understanding the impact that the presence of treatment-resistant clones may have on these new treatment modalities is crucial to validating them as a therapeutic avenue. In this study, a Moran process is used to capture stochastic mutations arising in cancer cells, inferring treatment resistance. The model is used to predict the probability of cancer recurrence given varying treatment modalities. The simulations predict that sustained-low dose therapies would be virtually ineffective for a cancer with a non-negligible probability of developing a sub-clone with resistance tendencies. Furthermore, calibrating the model to in vivo measurements for breast cancer treatment with Herceptin, the model suggests that standard treatment regimens are ineffective in this mouse model. Using a simple Moran model, it is possible to explore the likelihood of treatment success given a non-negligible probability of treatment resistant mutations and suggest more robust therapeutic schedules.

Stochastic competitive release and adaptive chemotherapy

We develop a finite-cell model of tumor natural selection dynamics to investigate the stochastic fluctuations associated with multiple rounds of adaptive chemotherapy. The adaptive cycles are designed to avoid chemoresistance in the tumor by managing the ecological mechanism of competitive release of a resistant subpopulation. Our model is based on a three-component evolutionary game played among healthy (H), sensitive (S), and resistant (R) populations of N cells, with a chemotherapy control parameter, C(t), which we use to dynamically impose selection pressure on the sensitive subpopulation to slow tumor growth and manage competitive release of the resistant population. The adaptive chemoschedule is designed based on the deterministic (N→∞) adjusted replicator dynamical system, then implemented using the finite-cell stochastic frequency dependent Moran process model (N=10K-50K) to ascertain the cumulative effect of the stochastic fluctuations on the efficacy of the adaptive schedules over multiple rounds. We quantify the stochastic fixation probability regions of the R and S populations in the HSR trilinear phase plane as a function of the control parameter C∈[0,1], showing that the size of the R region increases with increasing C. We then implement an adaptive time-dependent schedule C(t) for the stochastic model and quantify the variances (using principal component coordinates) associated with the evolutionary cycles over multiple rounds of adaptive therapy. The variances increase subquadratically through several rounds before the evolutionary cycle begins to break down. Despite this, we show the stochastic adaptive schedules are more effective at delaying resistance than standard maximum tolerated dose and low-dose metronomic schedules. The simplified low-dimensional model provides some insights on how well multiple rounds of adaptive therapies are likely to perform over a range of tumor sizes (i.e., different values of N) if the goal is to maintain a sustained balance among competing subpopulations of cells to avoid chemoresistance via competitive release in a stochastic environment.

Effects of Heterogeneity on Cancer: A Game Theory Perspective

In this study, we explore interactions between cancer cells by using the hawk-dove game. We analyze the heterogeneity of tumors by considering games with populations composed of 2 or 3 types of cell. We determine what strategies are evolutionarily stable in the 2-type and 3-type population games and what the corresponding expected payoffs are. Our results show that the payoff of the best-off cell in the 2-type population game is higher than that of the best-off cell in the 3-type population game. When these mathematical findings are transferred to the field of oncology they suggest that a tumor with low intratumor heterogeneity pursues a more aggressive course than one with high intratumor heterogeneity. Some histological and genomic data on clear cell renal cell carcinomas is consistent with these results. We underline the importance of identifying intratumor heterogeneity in routine practice and suggest that therapeutic strategies that preserve heterogeneity may be promising as they may slow down cancer growth.

COVID-19 vaccine incentive scheduling using an optimally controlled reinforcement learning model

We model Covid-19 vaccine uptake as a reinforcement learning dynamic between two populations: the vaccine adopters, and the vaccine hesitant. Using data available from the Center for Disease Control (CDC), we estimate the payoff matrix governing the interaction between these two groups over time and show they are playing a Hawk-Dove evolutionary game with an internal evolutionarily stable Nash equilibrium (the asymptotic percentage of vaccinated in the population). We then ask whether vaccine adoption can be improved by implementing dynamic incentive schedules that reward/punish the vaccine hesitant, and if so, what schedules are optimal and how effective are they likely to be? When is the optimal time to start an incentive program, how large should the incentives be, and is there a point of diminishing returns? By using a tailored replicator dynamic reinforcement learning model together with optimal control theory, we show that well designed and timed incentive programs can improve vaccine uptake by shifting the Nash equilibrium upward in large populations, but only so much, and incentive sizes above a certain threshold show diminishing returns.

Artificial Intelligence and Advanced Melanoma: Treatment Management Implications

Artificial intelligence (AI), a field of research in which computers are applied to mimic humans, is continuously expanding and influencing many aspects of our lives. From electric cars to search motors, AI helps us manage our daily lives by simplifying functions and activities that would be more complex otherwise. Even in the medical field, and specifically in oncology, many studies in recent years have highlighted the possible helping role that AI could play in clinical and therapeutic patient management. In specific contexts, clinical decisions are supported by "intelligent" machines and the development of specific softwares that assist the specialist in the management of the oncology patient. Melanoma, a highly heterogeneous disease influenced by several genetic and environmental factors, to date is still difficult to manage clinically in its advanced stages. Therapies often fail, due to the establishment of intrinsic or secondary resistance, making clinical decisions complex. In this sense, although much work still needs to be conducted, numerous evidence shows that AI (through the processing of large available data) could positively influence the management of the patient with advanced melanoma, helping the clinician in the most favorable therapeutic choice and avoiding unnecessary treatments that are sure to fail. In this review, the most recent applications of AI in melanoma will be described, focusing especially on the possible finding of this field in the management of drug treatments.

The role of evolutionary game theory in spatial and non-spatial models of the survival of cooperation in cancer: a review

Evolutionary game theory (EGT) is a branch of mathematics which considers populations of individuals interacting with each other to receive pay-offs. An individual’s pay-off is dependent on the strategy of its opponent(s) as well as on its own, and the higher its pay-off, the higher its reproductive fitness. Its offspring generally inherit its interaction strategy, subject to random mutation. Over time, the composition of the population shifts as different strategies spread or are driven extinct. In the last 25 years there has been a flood of interest in applying EGT to cancer modelling, with the aim of explaining how cancerous mutations spread through healthy tissue and how intercellular cooperation persists in tumour-cell populations. This review traces this body of work from theoretical analyses of well-mixed infinite populations through to more realistic spatial models of the development of cooperation between epithelial cells. We also consider work in which EGT has been used to make experimental predictions about the evolution of cancer, and discuss work that remains to be done before EGT can make large-scale contributions to clinical treatment and patient outcomes.

Coordination games in cancer

We propose a model of cancer initiation and progression where tumor growth is modulated by an evolutionary coordination game. Evolutionary games of cancer are widely used to model frequency-dependent cell interactions with the most studied games being the Prisoner's Dilemma and public goods games. Coordination games, by their more obscure and less evocative nature, are left understudied, despite the fact that, as we argue, they offer great potential in understanding and treating cancer. In this paper we present the conditions under which coordination games between cancer cells evolve, we propose aspects of cancer that can be modeled as results of coordination games, and explore the ways through which coordination games of cancer can be exploited for therapy.

Optimal dynamic incentive scheduling for Hawk-Dove evolutionary games

The Hawk-Dove evolutionary game offers a paradigm of the trade-offs associated with aggressive and passive behaviors. When two (or more) populations of players compete, their success or failure is measured by their frequency in the population, and the system is governed by the replicator dynamics. We develop a time-dependent optimal-adaptive control theory for this dynamical system in which the entries of the payoff matrix are dynamically altered to produce control schedules that minimize and maximize the aggressive population through a finite-time cycle. These schedules provide upper and lower bounds on the outcomes for all possible strategies since they represent two extremizers of the cost function. We then adaptively extend the optimal control schedules over multiple cycles to produce absolute maximizers and minimizers for the system.

Coordination games in cancer

We propose a model of cancer initiation and progression where tumor growth is modulated by an evolutionary coordination game. Evolutionary games of cancer are widely used to model frequency-dependent cell interactions with the most studied games being the Prisoner's Dilemma and public goods games. Coordination games, by their more obscure and less evocative nature, are left understudied, despite the fact that, as we argue, they offer great potential in understanding and treating cancer. In this paper we present the conditions under which coordination games between cancer cells evolve, we propose aspects of cancer that can be modeled as results of coordination games, and explore the ways through which coordination games of cancer can be exploited for therapy.

Are Adaptive Chemotherapy Schedules Robust? A Three-Strategy Stochastic Evolutionary Game Theory Model

We investigate the robustness of adaptive chemotherapy schedules over repeated cycles and a wide range of tumor sizes. Using a non-stationary stochastic three-component fitness-dependent Moran process model (to track frequencies), we quantify the variance of the response to treatment associated with multidrug adaptive schedules that are designed to mitigate chemotherapeutic resistance in an idealized (well-mixed) setting. The finite cell (N tumor cells) stochastic process consists of populations of chemosensitive cells, chemoresistant cells to drug 1, and chemoresistant cells to drug 2, and the drug interactions can be synergistic, additive, or antagonistic. Tumor growth rates in this model are proportional to the average fitness of the tumor as measured by the three populations of cancer cells compared to a background microenvironment average value. An adaptive chemoschedule is determined by using the N→∞ limit of the finite-cell process (i.e., the adjusted replicator equations) which is constructed by finding closed treatment response loops (which we call evolutionary cycles) in the three component phase-space. The schedules that give rise to these cycles are designed to manage chemoresistance by avoiding competitive release of the resistant cell populations. To address the question of how these cycles perform in practice over large patient populations with tumors across a range of sizes, we consider the variances associated with the approximate stochastic cycles for finite N, repeating the idealized adaptive schedule over multiple periods. For finite cell populations, the distributions remain approximately multi-Gaussian in the principal component coordinates through the first three cycles, with variances increasing exponentially with each cycle. As the number of cycles increases, the multi-Gaussian nature of the distribution breaks down due to the fact that one of the three sub-populations typically saturates the tumor (competitive release) resulting in treatment failure. This suggests that to design an effective and repeatable adaptive chemoschedule in practice will require a highly accurate tumor model and accurate measurements of the sub-population frequencies or the errors will quickly (exponentially) degrade its effectiveness, particularly when the drug interactions are synergistic. Possible ways to extend the efficacy of the stochastic cycles in light of the computational simulations are discussed.

Role of synergy and antagonism in designing multidrug adaptive chemotherapy schedules

Chemotherapeutic resistance via the mechanism of competitive release of resistant tumor cell subpopulations is a major problem associated with cancer treatments and one of the main causes of tumor recurrence. Often, chemoresistance is mitigated by using multidrug schedules (two or more combination therapies) that can act synergistically, additively, or antagonistically on the heterogeneous population of cells as they evolve. In this paper, we develop a three-component evolutionary game theory model to design two-drug adaptive schedules that mitigate chemoresistance and delay tumor recurrence in an evolving collection of tumor cells with two resistant subpopulations and one chemosensitive population that has a higher baseline fitness but is not resistant to either drug. Using the nonlinear replicator dynamical system with a payoff matrix of Prisoner's Dilemma (PD) type (enforcing a cost to resistance), we investigate the nonlinear dynamics of this three-component system along with an additional tumor growth model whose growth rate is a function of the fitness landscape of the tumor cell populations. A key parameter determines whether the two drugs interact synergistically, additively, or antagonistically. We show that antagonistic drug interactions generally result in slower rates of adaptation of the resistant cells than synergistic ones, making them more effective in combating the evolution of resistance. We then design evolutionary cycles (closed loops) in the three-component phase space by shaping the fitness landscape of the cell populations (i.e., altering the evolutionary stable states of the game) using appropriately designed time-dependent schedules (adaptive therapy), altering the dosages and timing of the two drugs. We describe two key bifurcations associated with our drug interaction parameter which help explain why antagonistic interactions are more effective at controlling competitive release of the resistant population than synergistic interactions in the context of an evolving tumor.

Maximizing cooperation in the prisoner's dilemma evolutionary game via optimal control

The prisoner's dilemma (PD) game offers a simple paradigm of competition between two players who can either cooperate or defect. Since defection is a strict Nash equilibrium, it is an asymptotically stable state of the replicator dynamical system that uses the PD payoff matrix to define the fitness landscape of two interacting evolving populations. The dilemma arises from the fact that the average payoff of this asymptotically stable state is suboptimal. Coaxing the players to cooperate would result in a higher payoff for both. Here we develop an optimal control theory for the prisoner's dilemma evolutionary game in order to maximize cooperation (minimize the defector population) over a given cycle time T, subject to constraints. Our two time-dependent controllers are applied to the off-diagonal elements of the payoff matrix in a bang-bang sequence that dynamically changes the game being played by dynamically adjusting the payoffs, with optimal timing that depends on the initial population distributions. Over multiple cycles nT (n>1), the method is adaptive as it uses the defector population at the end of the nth cycle to calculate the optimal schedule over the n+1st cycle. The control method, based on Pontryagin's maximum principle, can be viewed as determining the optimal way to dynamically alter incentives and penalties in order to maximize the probability of cooperation in settings that track dynamic changes in the frequency of strategists, with potential applications in evolutionary biology, economics, theoretical ecology, social sciences, reinforcement learning, and other fields where the replicator system is used.

IsoMaTrix: a framework to visualize the isoclines of matrix games and quantify uncertainty in structured populations

 

Evolutionary game theory describes frequency-dependent selection for fixed, heritable strategies in a population of competing individuals using a payoff matrix. We present a software package to aid in the construction, analysis, and visualization of three-strategy matrix games. The IsoMaTrix package computes the isoclines (lines of zero growth) of matrix games, and facilitates direct comparison of well-mixed dynamics to structured populations on a lattice grid. IsoMaTrix computes fixed points, phase flow, trajectories, (sub)velocities, and uncertainty quantification for stochastic effects in spatial matrix games. We describe a result obtained via IsoMaTrix's spatial games functionality, which shows that the timing of competitive release in a cancer model (under continuous treatment) critically depends on the initial spatial configuration of the tumor.

AVAILABILITY

The code is available at: https://github.com/mathonco/isomatrix.

SUPPLEMENTARY INFORMATION

Supplementary data are available at Bioinformatics online.

Optimizing adaptive cancer therapy: dynamic programming and evolutionary game theory

Recent clinical trials have shown that adaptive drug therapies can be more efficient than a standard cancer treatment based on a continuous use of maximum tolerated doses (MTD). The adaptive therapy paradigm is not based on a preset schedule; instead, the doses are administered based on the current state of tumour. But the adaptive treatment policies examined so far have been largely ad hoc. We propose a method for systematically optimizing adaptive policies based on an evolutionary game theory model of cancer dynamics. Given a set of treatment objectives, we use the framework of dynamic programming to find the optimal treatment strategies. In particular, we optimize the total drug usage and time to recovery by solving a Hamilton-Jacobi-Bellman equation. We compare MTD-based treatment strategy with optimal adaptive treatment policies and show that the latter can significantly decrease the total amount of drugs prescribed while also increasing the fraction of initial tumour states from which the recovery is possible. We conclude that the use of optimal control theory to improve adaptive policies is a promising concept in cancer treatment and should be integrated into clinical trial design.

Nonlinear adaptive control of competitive release and chemotherapeutic resistance

We use a three-component replicator system with healthy cells, sensitive cells, and resistant cells, with a prisoner's dilemma payoff matrix from evolutionary game theory, to model and control the nonlinear dynamical system governing the ecological mechanism of competitive release by which tumors develop chemotherapeutic resistance. The control method we describe is based on nonlinear trajectory design and energy transfer methods first introduced in the orbital mechanics literature for Hamiltonian systems. For continuous therapy, the basin boundaries of attraction associated with the chemo-sensitive population and the chemo-resistant population for increasing values of chemo-concentrations have an intertwined spiral structure with extreme sensitivity to changes in chemo-concentration level as well as sensitivity with respect to resistant mutations. For time-dependent therapies, we introduce an orbit transfer method to construct continuous families of periodic (closed) orbits by switching the chemo-dose at carefully chosen times and appropriate levels to design schedules that are superior to both maximum tolerated dose (MTD) schedules and low-dose metronomic (LDM) schedules, both of which ultimately lead to fixation of sensitive cells or resistant cells. By keeping the three subpopulations of cells in competition with each other indefinitely, we avoid fixation of the cancer cell population and regrowth of a resistant tumor. The method can be viewed as a way to dynamically shape the average population fitness landscape of a tumor to steer the chemotherapeutic response curve. We show that the method is remarkably insensitive to initial conditions and small changes in chemo-dosages, an important criterion for turning the method into an actionable strategy.

Evolution reinforces cooperation with the emergence of self-recognition mechanisms: An empirical study of strategies in the Moran process for the iterated prisoner's dilemma

We present insights and empirical results from an extensive numerical study of the evolutionary dynamics of the iterated prisoner's dilemma. Fixation probabilities for Moran processes are obtained for all pairs of 164 different strategies including classics such as TitForTat, zero determinant strategies, and many more sophisticated strategies. Players with long memories and sophisticated behaviours outperform many strategies that perform well in a two player setting. Moreover we introduce several strategies trained with evolutionary algorithms to excel at the Moran process. These strategies are excellent invaders and resistors of invasion and in some cases naturally evolve handshaking mechanisms to resist invasion. The best invaders were those trained to maximize total payoff while the best resistors invoke handshake mechanisms. This suggests that while maximizing individual payoff can lead to the evolution of cooperation through invasion, the relatively weak invasion resistance of payoff maximizing strategies are not as evolutionarily stable as strategies employing handshake mechanisms.

Capitalizing on competition: An evolutionary model of competitive release in metastatic castration resistant prostate cancer treatment

The development of chemotherapeutic resistance resulting in tumor relapse is largely the consequence of the mechanism of competitive release of pre-existing resistant tumor cells selected for regrowth after chemotherapeutic agents attack the previously dominant chemo-sensitive population. We introduce a prisoner's dilemma game theoretic mathematical model based on the replicator of three competing cell populations: healthy (cooperators), sensitive (defectors), and resistant (defectors) cells. The model is shown to recapitulate prostate-specific antigen measurement data from three clinical trials for metastatic castration-resistant prostate cancer patients treated with 1) prednisone, 2) mitoxantrone and prednisone and 3) docetaxel and prednisone. Continuous maximum tolerated dose schedules reduce the sensitive cell population, initially shrinking tumor burden, but subsequently "release" the resistant cells from competition to re-populate and re-grow the tumor in a resistant form. The evolutionary model allows us to quantify responses to conventional (continuous) therapeutic strategies as well as to design adaptive strategies.These novel adaptive strategies are robust to small perturbations in timing and extend simulated time to relapse from continuous therapy administration.

Weight of fitness deviation governs strict physical chaos in replicator dynamics

Replicator equation-a paradigm equation in evolutionary game dynamics-mathematizes the frequency dependent selection of competing strategies vying to enhance their fitness (quantified by the average payoffs) with respect to the average fitnesses of the evolving population under consideration. In this paper, we deal with two discrete versions of the replicator equation employed to study evolution in a population where any two players' interaction is modelled by a two-strategy symmetric normal-form game. There are twelve distinct classes of such games, each typified by a particular ordinal relationship among the elements of the corresponding payoff matrix. Here, we find the sufficient conditions for the existence of asymptotic solutions of the replicator equations such that the solutions-fixed points, periodic orbits, and chaotic trajectories-are all strictly physical, meaning that the frequency of any strategy lies inside the closed interval zero to one at all times. Thus, we elaborate on which of the twelve types of games are capable of showing meaningful physical solutions and for which of the two types of replicator equation. Subsequently, we introduce the concept of the weight of fitness deviation that is the scaling factor in a positive affine transformation connecting two payoff matrices such that the corresponding one-shot games have exactly same Nash equilibria and evolutionary stable states. The weight also quantifies how much the excess of fitness of a strategy over the average fitness of the population affects the per capita change in the frequency of the strategy. Intriguingly, the weight's variation is capable of making the Nash equilibria and the evolutionary stable states, useless by introducing strict physical chaos in the replicator dynamics based on the normal-form game.

Chemotherapeutic Dose Scheduling Based on Tumor Growth Rates Provides a Case for Low-Dose Metronomic High-Entropy Therapies

We extended the classical tumor regression models such as Skipper's laws and the Norton-Simon hypothesis from instantaneous regression rates to the cumulative effect over repeated cycles of chemotherapy. To achieve this end, we used a stochastic Moran process model of tumor cell kinetics coupled with a prisoner's dilemma game-theoretic cell-cell interaction model to design chemotherapeutic strategies tailored to different tumor growth characteristics. Using the Shannon entropy as a novel tool to quantify the success of dosing strategies, we contrasted MTD strategies as compared with low-dose, high-density metronomic strategies (LDM) for tumors with different growth rates. Our results show that LDM strategies outperformed MTD strategies in total tumor cell reduction. This advantage was magnified for fast-growing tumors that thrive on long periods of unhindered growth without chemotherapy drugs present and was not evident after a single cycle of chemotherapy but grew after each subsequent cycle of repeated chemotherapy. The evolutionary growth/regression model introduced in this article agrees well with murine models. Overall, this model supports the concept of designing different chemotherapeutic schedules for tumors with different growth rates and develops quantitative tools to optimize these schedules for maintaining low-volume tumors. Cancer Res; 77(23); 6717-28. ©2017 AACR.

AN EVOLUTIONARY MODEL OF TUMOR CELL KINETICS AND THE EMERGENCE OF MOLECULAR HETEROGENEITY DRIVING GOMPERTZIAN GROWTH

We describe a cell-molecular based evolutionary mathematical model of tumor development driven by a stochastic Moran birth-death process. The cells in the tumor carry molecular information in the form of a numerical genome which we represent as a four-digit binary string used to differentiate cells into 16 molecular types. The binary string is able to undergo stochastic point mutations that are passed to a daughter cell after each birth event. The value of the binary string determines the cell fitness, with lower fit cells (e.g. 0000) defined as healthy phenotypes, and higher fit cells (e.g. 1111) defined as malignant phenotypes. At each step of the birth-death process, the two phenotypic sub-populations compete in a prisoner's dilemma evolutionary game with the healthy cells playing the role of cooperators, and the cancer cells playing the role of defectors. Fitness, birth-death rates of the cell populations, and overall tumor fitness are defined via the prisoner's dilemma payoff matrix. Mutation parameters include passenger mutations (mutations conferring no fitness advantage) and driver mutations (mutations which increase cell fitness). The model is used to explore key emergent features associated with tumor development, including tumor growth rates as it relates to intratumor molecular heterogeneity. The tumor growth equation states that the growth rate is proportional to the logarithm of cellular diversity/heterogeneity. The Shannon entropy from information theory is used as a quantitative measure of heterogeneity and tumor complexity based on the distribution of the 4-digit binary sequences produced by the cell population. To track the development of heterogeneity from an initial population of healthy cells (0000), we use dynamic phylogenetic trees which show clonal and sub-clonal expansions of cancer cell sub-populations from an initial malignant cell. We show tumor growth rates are not constant throughout tumor development, and are generally much higher in the subclinical range than in later stages of development, which leads to a Gompertzian growth curve. We explain the early exponential growth of the tumor and the later saturation associated with the Gompertzian curve which results from our evolutionary simulations using simple statistical mechanics principles related to the degree of functional coupling of the cell states. We then compare dosing strategies at early stage development, mid-stage (clinical stage), and late stage development of the tumor. If used early during tumor development in the subclinical stage, well before the cancer cell population is selected for growth, therapy is most effective at disrupting key emergent features of tumor development.