1
|
Tosca EM, De Carlo A, Ronchi D, Magni P. Model-Informed Reinforcement Learning for Enabling Precision Dosing Via Adaptive Dosing. Clin Pharmacol Ther 2024; 116:619-636. [PMID: 38989560 DOI: 10.1002/cpt.3356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2024] [Accepted: 06/08/2024] [Indexed: 07/12/2024]
Abstract
Precision dosing, the tailoring of drug doses to optimize therapeutic benefits and minimize risks in each patient, is essential for drugs with a narrow therapeutic window and severe adverse effects. Adaptive dosing strategies extend the precision dosing concept to time-varying treatments which require sequential dose adjustments based on evolving patient conditions. Reinforcement learning (RL) naturally fits this paradigm: it perfectly mimics the sequential decision-making process where clinicians adapt dose administration based on patient response and evolution monitoring. This paper aims to investigate the potentiality of coupling RL with population PK/PD models to develop precision dosing algorithms, reviewing the most relevant works in the field. Case studies in which PK/PD models were integrated within RL algorithms as simulation engine to predict consequences of any dosing action have been considered and discussed. They mainly concern propofol-induced anesthesia, anticoagulant therapy with warfarin and a variety of anticancer treatments differing for administered agents and/or monitored biomarkers. The resulted picture highlights a certain heterogeneity in terms of precision dosing approaches, applied methodologies, and degree of adherence to the clinical domain. In addition, a tutorial on how a precision dosing problem should be formulated in terms of the key elements composing the RL framework (i.e., system state, agent actions and reward function), and on how PK/PD models could enhance RL approaches is proposed for readers interested in delving in this field. Overall, the integration of PK/PD models into a RL-framework holds great promise for precision dosing, but further investigations and advancements are still needed to address current limitations and extend the applicability of this methodology to drugs requiring adaptive dosing strategies.
Collapse
Affiliation(s)
- Elena Maria Tosca
- Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Alessandro De Carlo
- Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Davide Ronchi
- Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Paolo Magni
- Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| |
Collapse
|
2
|
Bassani D, Parrott NJ, Manevski N, Zhang JD. Another string to your bow: machine learning prediction of the pharmacokinetic properties of small molecules. Expert Opin Drug Discov 2024; 19:683-698. [PMID: 38727016 DOI: 10.1080/17460441.2024.2348157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 04/23/2024] [Indexed: 05/22/2024]
Abstract
INTRODUCTION Prediction of pharmacokinetic (PK) properties is crucial for drug discovery and development. Machine-learning (ML) models, which use statistical pattern recognition to learn correlations between input features (such as chemical structures) and target variables (such as PK parameters), are being increasingly used for this purpose. To embed ML models for PK prediction into workflows and to guide future development, a solid understanding of their applicability, advantages, limitations, and synergies with other approaches is necessary. AREAS COVERED This narrative review discusses the design and application of ML models to predict PK parameters of small molecules, especially in light of established approaches including in vitro-in vivo extrapolation (IVIVE) and physiologically based pharmacokinetic (PBPK) models. The authors illustrate scenarios in which the three approaches are used and emphasize how they enhance and complement each other. In particular, they highlight achievements, the state of the art and potentials of applying machine learning for PK prediction through a comphrehensive literature review. EXPERT OPINION ML models, when carefully crafted, regularly updated, and appropriately used, empower users to prioritize molecules with favorable PK properties. Informed practitioners can leverage these models to improve the efficiency of drug discovery and development process.
Collapse
Affiliation(s)
- Davide Bassani
- Pharmaceutical Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| | - Neil John Parrott
- Pharmaceutical Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| | - Nenad Manevski
- Pharmaceutical Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| | - Jitao David Zhang
- Pharmaceutical Research & Early Development, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd., Basel, Switzerland
| |
Collapse
|
3
|
De Carlo A, Tosca EM, Fantozzi M, Magni P. Reinforcement Learning and PK-PD Models Integration to Personalize the Adaptive Dosing Protocol of Erdafitinib in Patients with Metastatic Urothelial Carcinoma. Clin Pharmacol Ther 2024; 115:825-838. [PMID: 38339803 DOI: 10.1002/cpt.3176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 12/15/2023] [Indexed: 02/12/2024]
Abstract
The integration of pharmacokinetic-pharmacodynamic (PK-PD) modeling and simulations with artificial intelligence/machine learning algorithms is one of the most attractive areas of the pharmacometric research. These hybrid techniques are currently under investigation to perform several tasks, among which precision dosing. In this scenario, this paper presents and evaluates a new framework embedding PK-PD models into a reinforcement learning (RL) algorithm, Q-learning (QL), to personalize pharmacological treatment. Each patient is represented with a set of PK-PD parameters and has a personal QL agent which optimizes the individual treatment. In the training phase, leveraging PK-PD simulations, the QL agent assesses different actions, defined consistently with the clinical knowledge to consider only plausible dose-adjustments, in order to find the optimal rules. The proposed framework was evaluated to optimize the erdafitinib treatment in patients with metastatic urothelial carcinoma. This drug was approved by the US Food and Drug Administration (FDA) with a dose-adaptive protocol based on monitoring the levels of serum phosphate, which represent a biomarker of both treatment efficacy and toxicity. To evaluate the flexibility of the methodology, a heterogeneous virtual population of 141 patients was generated using an erdafitinib population PK (PopPK)-PD literature model. For each patient, treatment response was simulated by using both QL-optimized protocol and the clinical one. QL agents outperform the approved dose-adaptive rules, increasing more than 10% the efficacy and the safety of treatment at each end point. Results confirm the great potentialities of the integration of PopPK-PD models and RL algorithms to optimize precision dosing tasks.
Collapse
Affiliation(s)
- Alessandro De Carlo
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Elena Maria Tosca
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Martina Fantozzi
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Paolo Magni
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| |
Collapse
|
4
|
Shahin MH, Barth A, Podichetty JT, Liu Q, Goyal N, Jin JY, Ouellet D. Artificial Intelligence: From Buzzword to Useful Tool in Clinical Pharmacology. Clin Pharmacol Ther 2024; 115:698-709. [PMID: 37881133 DOI: 10.1002/cpt.3083] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2023] [Accepted: 10/06/2023] [Indexed: 10/27/2023]
Abstract
The advent of artificial intelligence (AI) in clinical pharmacology and drug development is akin to the dawning of a new era. Previously dismissed as merely technological hype, these approaches have emerged as promising tools in different domains, including health care, demonstrating their potential to empower clinical pharmacology decision making, revolutionize the drug development landscape, and advance patient care. Although challenges remain, the remarkable progress already made signals that the leap from hype to reality is well underway, and AI promises to offer clinical pharmacology new tools and possibilities for optimizing patient care is gradually coming to fruition. This review dives into the burgeoning world of AI and machine learning (ML), showcasing different applications of AI in clinical pharmacology and the impact of successful AI/ML implementation on drug development and/or regulatory decisions. This review also highlights recommendations for areas of opportunity in clinical pharmacology, including data analysis (e.g., handling large data sets, screening to identify important covariates, and optimizing patient population) and efficiencies (e.g., automation, translation, literature curation, and training). Realizing the benefits of AI in drug development and understanding its value will lead to the successful integration of AI tools in our clinical pharmacology and pharmacometrics armamentarium.
Collapse
Affiliation(s)
- Mohamed H Shahin
- Clinical Pharmacology and Bioanalytics, Pfizer Inc., Groton, Connecticut, USA
| | - Aline Barth
- Clinical Pharmacology and Bioanalytics, Pfizer Inc., Groton, Connecticut, USA
| | | | - Qi Liu
- Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, Maryland, USA
| | - Navin Goyal
- Clinical Pharmacology and Pharmacometrics, Janssen Research and Development, LLC., Spring House, Pennsylvania, USA
| | - Jin Y Jin
- Department of Clinical Pharmacology, Genentech, South San Francisco, California, USA
| | - Daniele Ouellet
- Clinical Pharmacology and Pharmacometrics, Janssen Research and Development, LLC., Spring House, Pennsylvania, USA
| |
Collapse
|
5
|
Curth A, Peck RW, McKinney E, Weatherall J, van der Schaar M. Using Machine Learning to Individualize Treatment Effect Estimation: Challenges and Opportunities. Clin Pharmacol Ther 2024; 115:710-719. [PMID: 38124482 DOI: 10.1002/cpt.3159] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 12/07/2023] [Indexed: 12/23/2023]
Abstract
The use of data from randomized clinical trials to justify treatment decisions for real-world patients is the current state of the art. It relies on the assumption that average treatment effects from the trial can be extrapolated to patients with personal and/or disease characteristics different from those treated in the trial. Yet, because of heterogeneity of treatment effects between patients and between the trial population and real-world patients, this assumption may not be correct for many patients. Using machine learning to estimate the expected conditional average treatment effect (CATE) in individual patients from observational data offers the potential for more accurate estimation of the expected treatment effects in each patient based on their observed characteristics. In this review, we discuss some of the challenges and opportunities for machine learning to estimate CATE, including ensuring identification assumptions are met, managing covariate shift, and learning without access to the true label of interest. We also discuss the potential applications as well as future work and collaborations needed to further improve identification and utilization of CATE estimates to increase patient benefit.
Collapse
Affiliation(s)
- Alicia Curth
- Department of Applied Mathematics & Theoretical Physics, University of Cambridge, Cambridge, UK
| | - Richard W Peck
- Department of Pharmacology & Therapeutics, University of Liverpool, Liverpool, UK
- Roche Pharma Research & Early Development (pRED), Roche Innovation Center, Basel, Switzerland
| | - Eoin McKinney
- Cambridge Institute for Immunotherapy & Infectious Disease, Jeffrey Cheah Biomedical Center, Cambridge Biomedical Campus, Addenbrooke's Hospital, Cambridge, UK
- Cambridge Centre for AI in Medicine, Cambridge, UK
| | - James Weatherall
- AstraZeneca R&D Data Science and Artificial Intelligence, Cambridge, UK
| | - Mihaela van der Schaar
- Department of Applied Mathematics & Theoretical Physics, University of Cambridge, Cambridge, UK
- Cambridge Centre for AI in Medicine, Cambridge, UK
- The Alan Turing Institute, London, UK
| |
Collapse
|
6
|
Augustin D, Lambert B, Robinson M, Wang K, Gavaghan D. Simulating clinical trials for model-informed precision dosing: using warfarin treatment as a use case. Front Pharmacol 2023; 14:1270443. [PMID: 37927586 PMCID: PMC10621790 DOI: 10.3389/fphar.2023.1270443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 10/05/2023] [Indexed: 11/07/2023] Open
Abstract
Treatment response variability across patients is a common phenomenon in clinical practice. For many drugs this inter-individual variability does not require much (if any) individualisation of dosing strategies. However, for some drugs, including chemotherapies and some monoclonal antibody treatments, individualisation of dosages are needed to avoid harmful adverse events. Model-informed precision dosing (MIPD) is an emerging approach to guide the individualisation of dosing regimens of otherwise difficult-to-administer drugs. Several MIPD approaches have been suggested to predict dosing strategies, including regression, reinforcement learning (RL) and pharmacokinetic and pharmacodynamic (PKPD) modelling. A unified framework to study the strengths and limitations of these approaches is missing. We develop a framework to simulate clinical MIPD trials, providing a cost and time efficient way to test different MIPD approaches. Central for our framework is a clinical trial model that emulates the complexities in clinical practice that challenge successful treatment individualisation. We demonstrate this framework using warfarin treatment as a use case and investigate three popular MIPD methods: 1. Neural network regression; 2. Deep RL; and 3. PKPD modelling. We find that the PKPD model individualises warfarin dosing regimens with the highest success rate and the highest efficiency: 75.1% of the individuals display INRs inside the therapeutic range at the end of the simulated trial; and the median time in the therapeutic range (TTR) is 74%. In comparison, the regression model and the deep RL model have success rates of 47.0% and 65.8%, and median TTRs of 45% and 68%. We also find that the MIPD models can attain different degrees of individualisation: the Regression model individualises dosing regimens up to variability explained by covariates; the Deep RL model and the PKPD model individualise dosing regimens accounting also for additional variation using monitoring data. However, the Deep RL model focusses on control of the treatment response, while the PKPD model uses the data also to further the individualisation of predictions.
Collapse
Affiliation(s)
- David Augustin
- Department of Computer Science, University of Oxford, Oxford, United Kingdom
| | - Ben Lambert
- College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter, United Kingdom
| | - Martin Robinson
- Department of Computer Science, University of Oxford, Oxford, United Kingdom
| | - Ken Wang
- Research and Early Development, F. Hoffmann-La Roche AG, Basel, Switzerland
| | - David Gavaghan
- Department of Computer Science, University of Oxford, Oxford, United Kingdom
| |
Collapse
|