Solving the Evidence Interpretability Crisis in Health Technology Assessment: A Role for Mechanistic Models?

A quantitative systems pharmacology workflow toward optimal design and biomarker stratification of atopic dermatitis clinical trials

BACKGROUND

The development of atopic dermatitis (AD) drugs is challenged by many disease phenotypes and trial design options, which are hard to explore experimentally.

OBJECTIVE

We aimed to optimize AD trial design using simulations.

METHODS

We constructed a quantitative systems pharmacology model of AD and standard of care (SoC) treatments and generated a phenotypically diverse virtual population whose parameter distribution was derived from known relationships between AD biomarkers and disease severity and calibrated using disease severity evolution under SoC regimens.

RESULTS

We applied this workflow to the immunomodulator OM-85, currently being investigated for its potential use in AD, and calibrated the investigational treatment model with the efficacy profile of an existing trial (thereby enriching it with plausible marker levels and dynamics). We assessed the sensitivity of trial outcomes to trial protocol and found that for this particular example the choice of end point is more important than the choice of dosing regimen and patient selection by model-based responder enrichment could increase the expected effect size. A global sensitivity analysis revealed that only a limited subset of baseline biomarkers is needed to predict the drug response of the full virtual population.

CONCLUSIONS

This AD quantitative systems pharmacology workflow built around knowledge of marker-severity relationships as well as SoC efficacy can be tailored to specific development cases to optimize several trial protocol parameters and biomarker stratification and therefore has promise to become a powerful model-informed AD drug development and personalized medicine tool.

In Silico Clinical Trials: Is It Possible?

Modeling and simulation (M&S), including in silico (clinical) trials, helps accelerate drug research and development and reduce costs and have coined the term "model-informed drug development (MIDD)." Data-driven, inferential approaches are now becoming increasingly complemented by emerging complex physiologically and knowledge-based disease (and drug) models, but differ in setup, bottlenecks, data requirements, and applications (also reminiscent of the different scientific communities they arose from). At the same time, and within the MIDD landscape, regulators and drug developers start to embrace in silico trials as a potential tool to refine, reduce, and ultimately replace clinical trials. Effectively, silos between the historically distinct modeling approaches start to break down. Widespread adoption of in silico trials still needs more collaboration between different stakeholders and established precedence use cases in key applications, which is currently impeded by a shattered collection of tools and practices. In order to address these key challenges, efforts to establish best practice workflows need to be undertaken and new collaborative M&S tools devised, and an attempt to provide a coherent set of solutions is provided in this chapter. First, a dedicated workflow for in silico clinical trial (development) life cycle is provided, which takes up general ideas from the systems biology and quantitative systems pharmacology space and which implements specific steps toward regulatory qualification. Then, key characteristics of an in silico trial software platform implementation are given on the example of jinkō.ai (nova's end-to-end in silico clinical trial platform). Considering these enabling scientific and technological advances, future applications of in silico trials to refine, reduce, and replace clinical research are indicated, ranging from synthetic control strategies and digital twins, which overall shows promise to begin a new era of more efficient drug development.

Empirical methods for the validation of time-to-event mathematical models taking into account uncertainty and variability: application to EGFR + lung adenocarcinoma

BACKGROUND

Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for mechanistic models. To our knowledge, in the latter case, there are no consensus standards, for instance for the validation of predictions against real-world data given the variability and uncertainty of the data. In this work, we focus on the prediction of time-to-event curves using as an application example a mechanistic model of non-small cell lung cancer. We designed four empirical methods to assess both model performance and reliability of predictions: two methods based on bootstrapped versions of parametric statistical tests: log-rank and combined weighted log-ranks (MaxCombo); and two methods based on bootstrapped prediction intervals, referred to here as raw coverage and the juncture metric. We also introduced the notion of observation time uncertainty to take into consideration the real life delay between the moment when an event happens, and the moment when it is observed and reported.

RESULTS

We highlight the advantages and disadvantages of these methods according to their application context. We have shown that the context of use of the model has an impact on the model validation process. Thanks to the use of several validation metrics we have highlighted the limit of the model to predict the evolution of the disease in the whole population of mutations at the same time, and that it was more efficient with specific predictions in the target mutation populations. The choice and use of a single metric could have led to an erroneous validation of the model and its context of use.

CONCLUSIONS

With this work, we stress the importance of making judicious choices for a metric, and how using a combination of metrics could be more relevant, with the objective of validating a given model and its predictions within a specific context of use. We also show how the reliability of the results depends both on the metric and on the statistical comparisons, and that the conditions of application and the type of available information need to be taken into account to choose the best validation strategy.