Reinforcement learning as an innovative model-based approach: Examples from precision dosing, digital health and computational psychiatry

Reinforcement Learning and PK-PD Models Integration to Personalize the Adaptive Dosing Protocol of Erdafitinib in Patients with Metastatic Urothelial Carcinoma

The integration of pharmacokinetic-pharmacodynamic (PK-PD) modeling and simulations with artificial intelligence/machine learning algorithms is one of the most attractive areas of the pharmacometric research. These hybrid techniques are currently under investigation to perform several tasks, among which precision dosing. In this scenario, this paper presents and evaluates a new framework embedding PK-PD models into a reinforcement learning (RL) algorithm, Q-learning (QL), to personalize pharmacological treatment. Each patient is represented with a set of PK-PD parameters and has a personal QL agent which optimizes the individual treatment. In the training phase, leveraging PK-PD simulations, the QL agent assesses different actions, defined consistently with the clinical knowledge to consider only plausible dose-adjustments, in order to find the optimal rules. The proposed framework was evaluated to optimize the erdafitinib treatment in patients with metastatic urothelial carcinoma. This drug was approved by the US Food and Drug Administration (FDA) with a dose-adaptive protocol based on monitoring the levels of serum phosphate, which represent a biomarker of both treatment efficacy and toxicity. To evaluate the flexibility of the methodology, a heterogeneous virtual population of 141 patients was generated using an erdafitinib population PK (PopPK)-PD literature model. For each patient, treatment response was simulated by using both QL-optimized protocol and the clinical one. QL agents outperform the approved dose-adaptive rules, increasing more than 10% the efficacy and the safety of treatment at each end point. Results confirm the great potentialities of the integration of PopPK-PD models and RL algorithms to optimize precision dosing tasks.

Digital Therapeutics as a New Therapeutic Modality: A Review from the Perspective of Clinical Pharmacology

The promise of transforming digital technologies into treatments is what drives the development of digital therapeutics (DTx), generally known as software applications embedded within accessible technologies-such as smartphones-to treat, manage, or prevent a pathological condition. Whereas DTx solutions that successfully demonstrate effectiveness and safety could drastically improve the life of patients in multiple therapeutic areas, there is a general consensus that generating therapeutic evidence for DTx presents challenges and open questions. We believe there are three main areas where the application of clinical pharmacology principles from the drug development field could benefit DTx development: the characterization of the mechanism of action, the optimization of the intervention, and, finally, its dosing. We reviewed DTx studies to explore how the field is approaching these topics and to better characterize the challenges associated with them. This leads us to emphasize the role that the application of clinical pharmacology principles could play in the development of DTx and to advocate for a development approach that merges such principles from development of traditional therapeutics with important considerations from the highly attractive and fast-paced world of digital solutions.