Purposes and duties in scientific modelling

Improving Transparency of Decision Models Through the Application of Decision Analytic Models with Omitted Objects Displayed (DAMWOOD)

The coronavirus disease 2019 (COVID-19) pandemic has increased public awareness of the influence of epidemiological and economic decision models on public policy decisions. Alongside this is an increased scrutiny on the development, analysis, reporting and utilisation of decision models for public policy making. Therefore, it is important that model developers can clearly explain and justify to all stakeholders what is included and excluded from a model developed to support decision-making, to both improve transparency and trust in decision-making. Our aim is to provide tools for improving communication between modellers and decision-makers, leading to improved transparency in decision-making. To do so, we extend the recently described directed acyclic graphs with omitted objects displayed (DAGWOOD) approach from Haber et al. (Ann Epidemiol 68:64-71, 2022) to decision analytic models, giving the decision analytic models with omitted objects displayed (DAMWOOD) approach. DAMWOOD is a framework for the identification of objects omitted from a decision model, as well as for consideration of the effects of omissions on model outcomes. Objects omitted from a decision model are classed as either an exclusion (known and unknown confounders), misdirection (alternative model pathways) or structure (e.g. model type, methods for estimating relationships between objects). DAMWOOD requires model developers to use explicit statements and provide illustration of included and omitted objects, supporting communication with model users and stakeholders, allowing them to provide input and feedback to modellers about which objects to include or omit in a model. In developing DAMWOOD, we considered two challenges we encountered in modelling for pandemic policy response. First, the scope of the decision problem is not always made sufficiently explicit by decision-makers, requiring modellers to intuit which policy options should be considered, and/or which outcomes should be considered in their evaluation. Second, there is rarely sufficient transparency to ensure stakeholders can see what is included in models and why. This limits stakeholders' ability to advocate to decision-makers for the prioritisation of specific outcomes and challenge the model results. To illustrate the application of DAMWOOD, we apply it to a previously published COVID-19 vaccine allocation optimisation model. The DAMWOOD diagrams illustrate the ways in which it is possible to improve the communication of model assumptions. The diagrams make explicit which outcomes are omitted and provide information on the expected impact of the omissions on model results. We discuss the usefulness of DAMWOOD for framing the decision problem, communicating the model structure and results and engaging with those making and affected by the decisions the model is developed to inform.

Patient and Public Involvement in Health Economics Modelling Raises the Need for Normative Guidance

Patient and public involvement in health economics research and health technology assessment has been increasing for some time; however, patient and public involvement in health economics modelling is a more recent development. One reason to advance this type of involvement is to help appropriately manage the social and ethical value judgements that are required throughout model development and interpretation. At the same time, patient and public involvement in health economics modelling raises numerous practical and philosophical issues that invite discussion and debate. Recently, we attended an engagement event which invited patients, members of the public, researchers and decision-makers to discuss some of these issues. One priority that emerged in the discussion was to develop normative guidance for patient and public involvement in health economics modelling. In this article, we reflect on this goal from our own perspective, focusing on why normative guidance is needed and what questions that guidance should answer.

Contextualist model evaluation: models in financial economics and index funds

Philosophers of science typically focus on the epistemic performance of scientific models when evaluating them. Analysing the effects that models may have on the world has typically been the purview of sociologists of science. We argue that the reactive (or "performative") effects of models should also figure in model evaluations by philosophers of science. We provide a detailed analysis of how models in financial economics created the impetus for the growing importance of the phenomenon of "passive investing" in financial markets. Considering this case motivates the position that we call contextualism about model evaluation, or model contextualism for short. Model contextualism encompasses standard analyses of the epistemic performance of the model, but also includes their reactive aspects. It entails identifying the epistemic and contextual import of the model, the ways in which a model can engender change in the world (which we call the channels of transmission), and the interactions between the epistemic and reactive import of a model.