Predictive validation of modeled health technology assessment claims: lessons from NICE

Validation of modeled pharmacoeconomic claims in formulary submissions

Modeled or simulated claims for costs and outcomes are a key element in formulary submissions and comparative assessments of drug products and devices; however, all too often these claims are presented in a form that is either unverifiable or potentially verifiable but in a time frame that is of no practical use to formulary committees and others who may be committed to ongoing disease-area and therapeutic-class reviews. On the assumption that formulary committees are interested in testable predictions for product performance in target populations and ongoing disease area and therapeutic reviews, the methodological standards that should be applied are those that are accepted in the natural sciences. Claims should be presented in a form that is amenable to falsification. If not, they have no scientific standing. Certainly one can follow ISPOR-SMDM standards for validating the assumptions underpinning a model or simulation. There is clearly an important role for simulations as an input to policy initiatives and developing claims for healthcare interventions and testable hypotheses; however, one would not evaluate such claims on the realism or otherwise of the model. The only standard is one of the model's ability to predict outcomes successfully in a time frame that is practical and useful. No other standard is acceptable. This sets the stage for an active research agenda.