Five ways to ensure that models serve society: a manifesto

Quantifying regulatory limits for multiple stressors in an open and transparent way

Biodiversity is confronted globally by multiple stressors. Environmental policies must regulate these stressors to achieve targets, but how should that be done when the outcomes of limits on one stressor are contingent on other stressors, about which there is imperfect knowledge? Deriving regulatory frameworks that incorporate these contingencies is an emerging challenge at the science-policy interface. To be fit for implementation, these frameworks need to facilitate the inherently sociopolitical process of policy implementation and account transparently for uncertainty, such that practitioners and other stakeholders can more realistically anticipate the range of potential outcomes to policy. We developed an approach to quantify stressor limits that explicitly accounts for multistressor contingencies. Using an invertebrate data set collected over 30 years throughout New Zealand, we combined ecological and ecotoxicological models to predict biodiversity loss as a function of one stressor, treating multistressor contingencies as a form of uncertainty about the outcomes of limits on that stressor. We transparently accounted for that uncertainty by presenting regulatory limits as bands bounded between optimistic and pessimistic views that practitioners may have about the local context within which limits are applied. In addition to transparently accounting for uncertainties, our framework also leaves room for practitioners to build stakeholder consensus when refining limits to suit different local contexts. A criticism of this open, transparent approach is that it creates too much scope for choosing limits that are lenient on polluters, paralyzing on-the-ground management of multiple stressors, but we demonstrate that this is not necessarily the case.

Aquatic connectivity: challenges and solutions in a changing climate

The challenge of managing aquatic connectivity in a changing climate is exacerbated in the presence of additional anthropogenic stressors, social factors, and economic drivers. Here we discuss these issues in the context of structural and functional connectivity for aquatic biodiversity, specifically fish, in both the freshwater and marine realms. We posit that adaptive management strategies that consider shifting baselines and the socio-ecological implications of climate change will be required to achieve management objectives. The role of renewable energy expansion, particularly hydropower, is critically examined for its impact on connectivity. We advocate for strategic spatial planning that incorporates nature-positive solutions, ensuring climate mitigation efforts are harmonized with biodiversity conservation. We underscore the urgency of integrating robust scientific modelling with stakeholder values to define clear, adaptive management objectives. Finally, we call for innovative monitoring and predictive decision-making tools to navigate the uncertainties inherent in a changing climate, with the goal of ensuring the resilience and sustainability of aquatic ecosystems.

Numbers and emotions in the governance of the Covid-19 datademic

There is a rich body of literature on numbers as tools of governance. But the attention of the corpus in question is almost entirely on the rational properties of quantification. This article shows that government by numbers is also, and inseparably, a government by feelings. The Covid-19 pandemic was also a datademic in the sense that numbers populated and spread through the public sphere. We focus on three cases. Death tolls were associated with fear, immunization rates were linked to hope, and the threshold of 100,000 deaths was credited with symbolic significance. This article, based on the French case, examines how data like these, frequently perceived as objective evidence, can at the same time be a source of emotional engagement and, as such, be used to inform modes of public governance in times of crises.

Empirical validation of integrated stock assessment models to ensuring risk equivalence: A pathway to resilient fisheries management

The Precautionary Approach to Fisheries Management requires an assessment of the impact of uncertainty on the risk of achieving management objectives. However, the main quantities, such as spawning stock biomass (SSB) and fish mortality (F), used in management metrics cannot be directly observed. This requires the use of models to provide guidance, for which there are three paradigms: the best assessment, model ensemble, and Management Strategy Evaluation (MSE). It is important to validate the models used to provide advice. In this study, we demonstrate how stock assessment models can be validated using a diagnostic toolbox, with a specific focus on prediction skill. Prediction skill measures the precision of a predicted value, which is unknown to the model, in relation to its observed value. By evaluating the accuracy of model predictions against observed data, prediction skill establishes an objective framework for accepting or rejecting model hypotheses, as well as for assigning weights to models within an ensemble. Our analysis uncovers the limitations of traditional stock assessment methods. Through the quantification of uncertainties and the integration of multiple models, our objective is to improve the reliability of management advice considering the complex interplay of factors that influence the dynamics of fish stocks.

Inference on spatiotemporal dynamics for coupled biological populations

Mathematical models in ecology and epidemiology must be consistent with observed data in order to generate reliable knowledge and evidence-based policy. Metapopulation systems, which consist of a network of connected sub-populations, pose technical challenges in statistical inference owing to nonlinear, stochastic interactions. Numerical difficulties encountered in conducting inference can obstruct the core scientific questions concerning the link between the mathematical models and the data. Recently, an algorithm has been proposed that enables computationally tractable likelihood-based inference for high-dimensional partially observed stochastic dynamic models of metapopulation systems. We use this algorithm to build a statistically principled data analysis workflow for metapopulation systems. Via a case study of COVID-19, we show how this workflow addresses the limitations of previous approaches. The COVID-19 pandemic provides a situation where mathematical models and their policy implications are widely visible, and we revisit an influential metapopulation model used to inform basic epidemiological understanding early in the pandemic. Our methods support self-critical data analysis, enabling us to identify and address model weaknesses, leading to a new model with substantially improved statistical fit and parameter identifiability. Our results suggest that the lockdown initiated on 23 January 2020 in China was more effective than previously thought.

Identification of risk factors on rabies vaccine efficacy from censored data: Pre-travel tests for dogs and cats from Yaoundé (2005-2015)

Little research is available on acquired immunity to rabies in dogs and cats from Central Africa, particularly regarding the legal movements of pets. Movement of domestic animals from rabies-endemic countries like Cameroon to rabies free areas poses one of the main risks for rabies introduction into rabies-free areas. Thus, the aim of this study was to assess the effect of various risk factors on rabies vaccine efficacy in Cameroonian. Since the dependent variable, rabies neutralizing titres, were censored from above (right-censoring), Generalized Additive Model for Location, Scale and Shape (GAMLSS) was used in the analysis. Overall, 85.7% of dogs and 100% of cats had titres greater than or equal to 0.5 IU/mL, which is considered protective. Additionally, compared to cats, the value of the rabies-neutralizing serum titres in dogs was on average smaller by 2.3 IU/mL. For each additional year of age, the value of the rabies-neutralizing serum titre, on average, increased by approximately 0.14 IU/mL. Finally, for each 30 additional days between the date of the last rabies vaccination and the date of the sampling, the value the rabies neutralizing titre, on average, decreased by approximately 0.10 IU/mL, given the species and age at sampling were equivalent. These results are useful for assessing risk and improving surveillance to prevent the introduction of rabies into a country via the international movement of animals.

The effect of including a mobile arch, toe joint, and joint coupling on predictive neuromuscular simulations of human walking

Predictive neuromuscular simulations are a powerful tool for studying the biomechanics of human walking, and deriving design criteria for technical devices like prostheses or biorobots. Good agreement between simulation and human data is essential for transferability to the real world. The human foot is often modeled with a single rigid element, but knowledge of how the foot model affects gait prediction is limited. Standardized procedures for selecting appropriate foot models are lacking. We performed 2D predictive neuromuscular simulations with six different foot models of increasing complexity to answer two questions: What is the effect of a mobile arch, a toe joint, and the coupling of toe and arch motion through the plantar fascia on gait prediction? and How much of the foot's anatomy do we need to model to predict sagittal plane walking kinematics and kinetics in good agreement with human data? We found that the foot model had a significant impact on ankle kinematics during terminal stance, push-off, and toe and arch kinematics. When focusing only on hip and knee kinematics, rigid foot models are sufficient. We hope our findings will help guide the community in modeling the human foot according to specific research goals and improve neuromuscular simulation accuracy.

A pandemic of metrics

The COVID-19 pandemic prompted critical attention to the performative power of metrics. We suggest that the existential capacities of metrics as a means of pandemic living warrant further consideration. We describe how the COVID-19 pandemic that came into existence as a public health and political event could only have occurred because of the anticipatory metrical practices that were used to transform SARS-COV-2 into a matter of global health concern. By exploring the affective potencies of COVID-19 metrics we show their abilities to engage the public in ways that cannot be contained; in detailing the narrative arcs created through metrics we show their opportunities, misdirections, and erasures. A pandemic way of life persists: a pandemic of metrics.

Informing policy via dynamic models: Cholera in Haiti

Public health decisions must be made about when and how to implement interventions to control an infectious disease epidemic. These decisions should be informed by data on the epidemic as well as current understanding about the transmission dynamics. Such decisions can be posed as statistical questions about scientifically motivated dynamic models. Thus, we encounter the methodological task of building credible, data-informed decisions based on stochastic, partially observed, nonlinear dynamic models. This necessitates addressing the tradeoff between biological fidelity and model simplicity, and the reality of misspecification for models at all levels of complexity. We assess current methodological approaches to these issues via a case study of the 2010-2019 cholera epidemic in Haiti. We consider three dynamic models developed by expert teams to advise on vaccination policies. We evaluate previous methods used for fitting these models, and we demonstrate modified data analysis strategies leading to improved statistical fit. Specifically, we present approaches for diagnosing model misspecification and the consequent development of improved models. Additionally, we demonstrate the utility of recent advances in likelihood maximization for high-dimensional nonlinear dynamic models, enabling likelihood-based inference for spatiotemporal incidence data using this class of models. Our workflow is reproducible and extendable, facilitating future investigations of this disease system.

Reducing uncertainty in evidence-based health policy by integrating empirical and theoretical evidence: An EbM+theory approach

BACKGROUND

To reduce their decisional uncertainty, health policy decision-makers rely more often on experts or their intuition than on evidence-based knowledge, especially in times of urgency. However, this practice is unacceptable from an evidence-based medicine (EbM) perspective. Therefore, in fast-changing and complex situations, we need an approach that delivers recommendations that serve decision-makers' needs for urgent, sound and uncertainty-reducing decisions based on the principles of EbM.

AIMS

The aim of this paper is to propose an approach that serves this need by enriching EbM with theory.

MATERIALS AND METHODS

We call this the EbM+theory approach, which integrates empirical and theoretical evidence in a context-sensitive way to reduce intervention and implementation uncertainty.

RESULTS

Within this framework, we propose two distinct roadmaps to decrease intervention and implementation uncertainty: one for simple and the other for complex interventions. As part of the roadmap, we present a three-step approach: applying theory (step 1), conducting mechanistic studies (EbM+; step 2) and conducting experiments (EbM; step 3).

DISCUSSION

This paper is a plea for integrating empirical and theoretical knowledge by combining EbM, EbM+ and theoretical knowledge in a common procedural framework that allows flexibility even in dynamic times. A further aim is to stimulate a discussion on using theories in health sciences, health policy, and implementation.

CONCLUSION

The main implications are that scientists and health politicians - the two main target groups of this paper-should receive more training in theoretical thinking; moreover, regulatory agencies like NICE may think about the usefulness of integrating elements of the EbM+theory approach into their considerations.

Conceptualising the value of simulation modelling for public engagement with policy: a critical literature review

As we face complex and dynamically changing public health and environmental challenges, simulation modelling has come to occupy an increasingly central role in public engagements with policy. Shifts are occurring not only in terms of wider public understandings of modelling, but also in how the value of modelling is conceptualised within scientific modelling communities. We undertook a critical literature review to synthesise the underlying epistemic, theoretical and methodological assumptions about the role and value of simulation modelling within the literature across a range of fields (e.g., health, social science and environmental management) that engage with participatory modelling approaches. We identified four cross-cutting narrative conceptualisations of the value of modelling across different research traditions: (1) models simulate and help solve complex problems; (2) models as tools for community engagement; (3) models as tools for consensus building; (4) models as volatile technologies that generate social effects. Exploring how these ideas of 'value' overlap and what they offer one another has implications for how participatory simulation modelling approaches are designed, evaluated and communicated to diverse audiences. Deeper appreciation of the conditions under which simulation modelling can catalyse multiple social effects is recommended.

The responsibility turn Andrea Saltelli and Monica Di Fiore, Eds. Oxford University Press, 2023. 272 pp

Lessons from the COVID-19 pandemic inspire a guide to recognizing the politics of modeling.

A two-stages global sensitivity analysis by using the δ sensitivity index in presence of correlated inputs: application on a tumor growth inhibition model based on the dynamic energy budget theory

Global sensitivity analysis (GSA) evaluates the impact of variability and/or uncertainty of the model parameters on given model outputs. GSA is useful for assessing the quality of Pharmacometric model inference. Indeed, model parameters can be affected by high (estimation) uncertainty due to the sparsity of data. Independence between model parameters is a common assumption of GSA methods. However, ignoring (known) correlations between parameters may alter model predictions and, then, GSA results. To address this issue, a novel two-stages GSA technique based on the δ index, which is well-defined also in presence of correlated parameters, is here proposed. In the first step, statistical dependencies are neglected to identify parameters exerting causal effects. Correlations are introduced in the second step to consider the real distribution of the model output and investigate also the 'indirect' effects due to the correlation structure. The proposed two-stages GSA strategy was applied, as case study, to a preclinical tumor-in-host-growth inhibition model based on the Dynamic Energy Budget theory. The aim is to evaluate the impact of the model parameter estimate uncertainty (including correlations) on key model-derived metrics: the drug threshold concentration for tumor eradication, the tumor volume doubling time and a new index evaluating the drug efficacy-toxicity trade-off. This approach allowed to rank parameters according to their impact on the output, discerning whether a parameter mainly exerts a causal or 'indirect' effect. Thus, it was possible to identify uncertainties that should be necessarily reduced to obtain robust predictions for the outputs of interest.

Infectious Disease Modelling of HIV Prevention Interventions: A Systematic Review and Narrative Synthesis of Compartmental Models

BACKGROUND

The HIV epidemic remains a major public health problem. Critical to transmission control are HIV prevention strategies with new interventions continuing to be developed. Mathematical models are important for understanding the potential impact of these interventions and supporting policy decisions. This systematic review aims to answer the following question: when a new HIV prevention intervention is being considered or designed, what information regarding it is necessary to include in a compartmental model to provide useful insights to policy makers? The primary objective of this review is therefore to assess suitability of current compartmental HIV prevention models for informing policy development.

METHODS

Articles published in EMBASE, Medline, Econlit, and Global Health were screened. Included studies were identified using permutations of (i) HIV, (ii) pre-exposure prophylaxis (PrEP), circumcision (both voluntary male circumcision [VMMC] and early-infant male circumcision [EIMC]), and vaccination, and (iii) modelling. Data extraction focused on study design, model structure, and intervention incorporation into models. Article quality was assessed using the TRACE (TRAnsparent and Comprehensive Ecological modelling documentation) criteria for mathematical models.

RESULTS

Of 837 articles screened, 48 articles were included in the review, with 32 unique mathematical models identified. The substantial majority of studies included PrEP (83%), whilst fewer modelled circumcision (54%), and only a few focussed on vaccination (10%). Data evaluation, implementation verification, and model output corroboration were identified as areas of poorer model quality. Parameters commonly included in the mathematical models were intervention uptake and effectiveness, with additional intervention-specific common parameters identified. We identified key modelling gaps; critically, models insufficiently incorporate multiple interventions acting simultaneously. Additionally, population subgroups were generally poorly represented-with future models requiring improved incorporation of ethnicity and sexual risk group stratification-and many models contained inappropriate data in parameterisation which will affect output accuracy.

CONCLUSIONS

This review identified gaps in compartmental models to date and suggests areas of improvement for models focusing on new prevention interventions. Resolution of such gaps within future models will ensure greater robustness and transparency, and enable more accurate assessment of the impact that new interventions may have, thereby providing more meaningful guidance to policy makers.

mvLognCorrEst: an R package for sampling from multivariate lognormal distributions and estimating correlations from uncomplete correlation matrix

BACKGROUND AND OBJECTIVE

Pharmacometrics (PMX) is a quantitative discipline which supports decision-making processes in all stages of drug development. PMX leverages Modeling and Simulations (M&S), which represents a powerful tool to characterize and predict the behavior and the effect of a drug. M&S-based methods, such as Sensitivity Analysis (SA) and Global Sensitivity Analysis (GSA), are gaining interest in PMX as they allow the evaluation of model-informed inference quality. Simulations should be correctly designed to obtain reliable results. Neglecting correlations between model parameters can significantly alter the results of simulations. However, the introduction of a correlation structure between model parameters can cause some issues. Sampling from a multivariate lognormal distribution, which is the typically distribution assumed for PMX model parameters, is not straightforward when a correlation structure is introduced. Indeed, correlations need to respect some constraints which depend by the CVs (i.e., coefficients of variation) of lognormal variables. In addition, when correlation matrices have some unspecified values, they should be properly fixed preserving the positive semi-definiteness of the correlation structure. In this paper, we present mvLognCorrEst, an R package developed to address these issues.

METHODS

The proposed sampling strategy was based on reconducting the extraction from the multivariate lognormal distribution of interest to the underlying Normal distribution. However, with high lognormal CVs, a positive semi-definite Normal covariance matrix cannot be obtained due to the violation of some theoretical constraints. In these cases, the Normal covariance matrix was approximated to its nearest positive definite matrix using Frobenius norm as matrix distance. For the estimation of unknown correlations terms, the graph theory was used to represent the correlation structure as weighed undirected graph. Plausible value ranges for the unspecified correlations were derived considering the paths between variables. Then, their estimation was performed by solving a constrained optimization problem.

RESULTS

Package functions are presented and applied on a real case study, that is the GSA of a PMX model that has been recently developed to support preclinical oncological studies.

CONCLUSIONS

mvLognCorrEst package is an R tool to support simulation-based analysis for which sampling from multivariate lognormal distributions with correlated variables and/or estimation of partially defined correlation matrix are required.

Synergy of climate change with country success and city quality of life

Most people around the world have felt the effects of climate change on their quality of life. This study sought to achieve the maximum efficiency for climate change actions with the minimum negative impact on the well-being of countries and cities. The Climate Change and Country Success (C³S) and Climate Change and Cities' Quality of Life (C³QL) models and maps of the world created as part of this research showed that as economic, social, political, cultural, and environmental metrics of countries and cities improve, so do their climate change indicators. For the 14 climate change indicators, the C³S and C³QL models indicated 68.8% average dispersion dimensions in the case of countries and 52.8% in the case of cities. Our research showed that increases in the success of 169 countries saw improvements in 9 climate change indicators out of the 12 considered. Improvements in country success indicators were accompanied by a 71% improvement in climate change metrics.

What can mathematical modelling contribute to a sociology of quantification?

Sociology of quantification has spent relatively less energies investigating mathematical modelling than it has on other forms of quantification such as statistics, metrics, or algorithms based on artificial intelligence. Here we investigate whether concepts and approaches from mathematical modelling can provide sociology of quantification with nuanced tools to ensure the methodological soundness, normative adequacy and fairness of numbers. We suggest that methodological adequacy can be upheld by techniques in the field of sensitivity analysis, while normative adequacy and fairness are targeted by the different dimensions of sensitivity auditing. We also investigate in which ways modelling can inform other instances of quantification as to promote political agency.

Do We Learn From Errors? The Economic Impact of Differentiated Policy Restrictions in Italy

This paper investigates the economic impact of the three tiers risk framework implemented in Italy against the COVID-19 pandemic during the Autumn of 2020. Exploiting a large-scale dataset encompassing daily credit card transactions mediated by a large Italian bank, we estimate a set of panel event study models to disentangle the impact of restrictions with low, medium and high stringency levels in terms of consumption reduction. We show that space-time differentiated policies tend to produce stronger welfare losses for progressively more stringent restrictions in specific sectors targeted by these policies such as Retail and Restaurants. However, when we compare provinces implementing the same level of policy stringency, we show that territories with higher income per capita and larger concentration of manufacturing and service activities experience both significantly worse economic and epidemiological performances. Overall, our results suggest that policy makers should properly account for local socio-economic characteristics when designing tailored restrictions entailing an equal and homogeneous impact across territories.

Pursuitworthiness in urgent research: Lessons on well-ordered science from sustainability science

Urgent and complex sustainability problems are redefining the conditions for success of science and instigating novel methodologies and new roles of values in science. At the same time, sustainability research, mostly under the umbrella of sustainability science, is abundant with dubious methods and dubious aims of research, contributing to the already widespread crisis of the quality control mechanisms in science. This paper identifies some of these dubious methods (non-systemic thinking and specific contractual funding mechanisms) and dubious aims (unclear aims and undisclosed value assumptions), and argues that, with proper expert evaluation, it is possible to predict the type of content (and the scientific value) these forms of research will result in. Identifying the forms of research with predictably dubious results has practical implications for doing and evaluating research in sustainability science, on the one hand, while on the other, it contributes to the discussion on the ideal of well-ordered science by concretizing the ideal and developing a criterion for ordering science in the case of sustainability science. Finally, the paper provides a connection of sustainability science with the meta-science discussions on the crisis in the quality of science and on science organization, what at the same time more closely connects the philosophy of science with problems encountered in research practice tackling urgent, complex and value-laden problems.

Unintended Consequences of COVID-19 Non-Pharmaceutical Interventions (NPIs) for Population Health and Health Inequalities

Since the start of the COVID-19 pandemic in early 2020, governments around the world have adopted an array of measures intended to control the transmission of the SARS-CoV-2 virus, using both pharmaceutical and non-pharmaceutical interventions (NPIs). NPIs are public health interventions that do not rely on vaccines or medicines and include policies such as lockdowns, stay-at-home orders, school closures, and travel restrictions. Although the intention was to slow viral transmission, emerging research indicates that these NPIs have also had unintended consequences for other aspects of public health. Hence, we conducted a narrative review of studies investigating these unintended consequences of NPIs, with a particular emphasis on mental health and on lifestyle risk factors for non-communicable diseases (NCD): physical activity (PA), overweight and obesity, alcohol consumption, and tobacco smoking. We reviewed the scientific literature using combinations of search terms such as 'COVID-19', 'pandemic', 'lockdowns', 'mental health', 'physical activity', and 'obesity'. NPIs were found to have considerable adverse consequences for mental health, physical activity, and overweight and obesity. The impacts on alcohol and tobacco consumption varied greatly within and between studies. The variability in consequences for different groups implies increased health inequalities by age, sex/gender, socioeconomic status, pre-existing lifestyle, and place of residence. In conclusion, a proper assessment of the use of NPIs in attempts to control the spread of the pandemic should be weighed against the potential adverse impacts on other aspects of public health. Our findings should also be of relevance for future pandemic preparedness and pandemic response teams.

Integrated modeling framework for sustainable agricultural intensification

Agricultural scientists are pursuing sustainable intensification strategies to increase global food availability, but integration from research to impact at the local-level requires knowledge of demographic and human-environment to enhance the adaptive capacity of farmers cultivating <10 ha. Enhancing close collaboration among transdisciplinary teams and these smallholders is critical to co-elaborate policy solutions to ongoing food security crises that are likely to be attuned with local conditions. Human and socio-cultural aspects need to be considered to facilitate both adoption and dissemination of adapted management practices. Despite this well-known need to co-produce knowledge in human systems, we demonstrate the inequality of current agricultural research in smallholder farming systems with heavy focus on a few domains of the sustainable intensification agricultural framework (SIAF), ultimately reducing the overall impact of interventions due to the lack compatibility with prevailing social contexts. Here we propose to integrate agriculture and agronomic models with social and demographic modeling approaches to increase agricultural productivity and food system resilience, while addressing persistent issues in food security. Researchers should consider the scale of interventions, ensure attention is paid to equality and political processes, explore local change interactions, and improve connection of agriculture with nutrition and health outcomes, via nutrition-sensitive agricultural investments.

Epidemic dynamics in census-calibrated modular contact network

Network-based models are apt for understanding epidemic dynamics due to their inherent ability to model the heterogeneity of interactions in the contemporary world of intense human connectivity. We propose a framework to create a wire-frame that mimics the social contact network of the population in a geography by lacing it with demographic information. The framework results in a modular network with small-world topology that accommodates density variations and emulates human interactions in family, social, and work spaces. When loaded with suitable economic, social, and urban data shaping patterns of human connectance, the network emerges as a potent decision-making instrument for urban planners, demographers, and social scientists. We employ synthetic networks to experiment in a controlled environment and study the impact of zoning, density variations, and population mobility on the epidemic variables using a variant of the SEIR model. Our results reveal that these demographic factors have a characteristic influence on social contact patterns, manifesting as distinct epidemic dynamics. Subsequently, we present a real-world COVID-19 case study for three Indian states by creating corresponding surrogate social contact networks using available census data. The case study validates that the demography-laced modular contact network reduces errors in the estimates of epidemic variables.

Global sensitivity analysis in epidemiological modeling

Operations researchers worldwide rely extensively on quantitative simulations to model alternative aspects of the COVID-19 pandemic. Proper uncertainty quantification and sensitivity analysis are fundamental to enrich the modeling process and communicate correctly informed insights to decision-makers. We develop a methodology to obtain insights on key uncertainty drivers, trend analysis and interaction quantification through an innovative combination of probabilistic sensitivity techniques and machine learning tools. We illustrate the approach by applying it to a representative of the family of susceptible-infectious-recovered (SIR) models recently used in the context of the COVID-19 pandemic. We focus on data of the early pandemic progression in Italy and the United States (the U.S.). We perform the analysis for both cases of correlated and uncorrelated inputs. Results show that quarantine rate and intervention time are the key uncertainty drivers, have opposite effects on the number of total infected individuals and are involved in the most relevant interactions.

Applications of Smart Material Sensors and Soft Electronics in Healthcare Wearables for Better User Compliance

The need for innovation in the healthcare sector is essential to meet the demand of a rapidly growing population and the advent of progressive chronic ailments. Over the last decade, real-time monitoring of health conditions has been prioritized for accurate clinical diagnosis and access to accelerated treatment options. Therefore, the demand for wearable biosensing modules for preventive and monitoring purposes has been increasing over the last decade. Application of machine learning, big data analysis, neural networks, and artificial intelligence for precision and various power-saving approaches are used to increase the reliability and acceptance of smart wearables. However, user compliance and ergonomics are key areas that need focus to make the wearables mainstream. Much can be achieved through the incorporation of smart materials and soft electronics. Though skin-friendly wearable devices have been highlighted recently for their multifunctional abilities, a detailed discussion on the integration of smart materials for higher user compliance is still missing. In this review, we have discussed the principles and applications of sustainable smart material sensors and soft electronics for better ergonomics and increased user compliance in various healthcare devices. Moreover, the importance of nanomaterials and nanotechnology is discussed in the development of smart wearables.

The Challenge of Quantification: An Interdisciplinary Reading

The present work looks at what we call "the multiverse of quantification", where visible and invisible numbers permeate all aspects and venues of life. We review the contributions of different authors who focus on the roles of quantification in society, with the aim of capturing different and sometimes separate voices. Several scholars, including economists, jurists, philosophers, sociologists, communication and data scientists, express concerns or identify critical areas of our relationship with new technologies of 'numericization'. While mindful of the important specificities of the different families of quantification, we use our broad and holistic canvas to explore possible spaces for a more systematic investigation of incumbent and novel quantifications, as to increase communication among disciplinary communities, and among these and society, in the pursuit a democratic agency and self-defence.

Integrating multiple information sources for landslide hazard assessment: the case of Italy

Landslides are the most frequent and diffuse natural hazards in Italy causing the greatest number of fatalities and damage to urban areas. The integration of natural hazard information and social media data could improve warning systems to enhance the awareness of disaster managers and citizens about emergency events. The news about landslide events in newspapers or crowdsourcing platforms allows fast observation, surveying and classification. Currently, few studies have been produced on the combination of social media data and traditional sensors. This gap indicates that it is unclear how their integration can effectively provide emergency managers with appropriate knowledge. In this work, rainfall, human lives, and earmarked fund data sources were correlated to "landslide news". Analysis was applied to obtain information about temporal (2010-2019) and spatial (regional and warning hydrological zone scale) distribution. The temporal distribution of the data shows a continuous increase from 2015 until 2019 for both landslide and rainfall events. The number of people involved and the amount of earmarked funds do not exhibit any clear trend. The spatial distribution displays good correlation between "landslide news", traditional sensors (e.g., pluviometers) and possible effects in term of fatalities. In addition, the cost of soil protection, in monetary terms, indicates the effects of events.

Uncomfortable science: How mathematical models, and consensus, come to be in public policy

We explore messy translations of evidence in policy as a site of 'uncomfortable science'. Drawing on the work of John Law, we follow evidence as a 'fluid object' of its situation, also enacted in relation to a hinterland of practices. Working with the qualitative interview accounts of mathematical modellers and other scientists engaged in the UK COVID-19 response, we trace how models perform as evidence. Our point of departure is a moment of controversy in the public announcement of second national lockdown in the UK, and specifically, the projected daily deaths from COVID-19 presented in support of this policy decision. We reflect on this event to trace the messy translations of "scientific consensus" in the face of uncertainty. Efforts among scientists to realise evidence-based expectation and to manage the troubled translations of models in policy, including via "scientific consensus", can extend the dis-ease of uncomfortable science rather than clean it up or close it down. We argue that the project of evidence-based policy is not so much in need of technical management or repair, but that we need to be thinking altogether differently.

Pandemics, policy, and pluralism: A Feyerabend-inspired perspective on COVID-19

We analyse insufficient epistemic pluralism and associated problems in science-based policy advice during the COVID-19 pandemic drawing on specific arguments in Paul Feyerabend's philosophy. Our goal is twofold: to deepen our understanding of the epistemic shortcomings in science-based policy during the pandemic, and to assess the merits and problems of Feyerabend's arguments for epistemic pluralism as well as their relevance for policy-making. We discuss opportunities and challenges of integrating a plurality of viewpoints from within and outside science into policy advice thus contributing to discussions about normative issues concerning evidence and expertise in policy-making.

Models with higher effective dimensions tend to produce more uncertain estimates

Mathematical models are getting increasingly detailed to better predict phenomena or gain more accurate insights into the dynamics of a system of interest, even when there are no validation or training data available. Here, we show through ANOVA and statistical theory that this practice promotes fuzzier estimates because it generally increases the model's effective dimensions, i.e., the number of influential parameters and the weight of high-order interactions. By tracking the evolution of the effective dimensions and the output uncertainty at each model upgrade stage, modelers can better ponder whether the addition of detail truly matches the model's purpose and the quality of the data fed into it.

The role of models in the covid-19 pandemic

Mathematical and statistical models have played an important role in the analysis of data from COVID-19. They are important for tracking the progress of the pandemic, for understanding its spread in the population, and perhaps most significantly for forecasting the future course of the pandemic and evaluating potential policy options. This article describes the types of models that were used by research teams in Israel, presents their assumptions and basic elements, and illustrates how they were used, and how they influenced decisions. The article grew out of a "modelists' dialog" organized by the Israel National Institute for Health Policy Research with participation from some of the leaders in the local modeling effort.

An evaluation of North Carolina science advice on COVID-19 pandemic response

This qualitative case study contributes to the international research project EScAPE (Evaluating Scientific Advice in a Pandemic Emergency) and aims to understand how state leaders mobilized science advice in pandemic response during 2020 and into the early months of 2021. North Carolina, a state in the southeastern United States, mobilized much of its pandemic science advice through the state's Department of Health and Human Services. A fluid relationship between advisors and the governor-credited as a crucial component of a science driven, balanced pandemic response-created an opaque hub of advising and power. I analyze three advisory processes apparent during early stages of pandemic response noting strengths in mutual respect and trust between advisors and policymakers, data transparency, and commitment to equitable vaccine distribution. The interpersonal dynamics that provided these "good" science advice outcomes are a result of the individuals involved but the dynamic is not guaranteed in government over time. Also, while North Carolina provided data transparency it is unclear how data trends connected to decisions. There is a general lack of transparency around the breadth and content of advice. Transparency of advisory mechanisms is important to maintain public trust in government. Deep partisanship in the United States and distrust between leaders of opposing parties underscores the need for states to develop strong institutions for science advise to policymakers in an emergency. This article closes with several recommendations.

Challenges and recommendations for wearable devices in digital health: Data quality, interoperability, health equity, fairness

Wearable devices are increasingly present in the health context, as tools for biomedical research and clinical care. In this context, wearables are considered key tools for a more digital, personalised, preventive medicine. At the same time, wearables have also been associated with issues and risks, such as those connected to privacy and data sharing. Yet, discussions in the literature have mostly focused on either technical or ethical considerations, framing these as largely separate areas of discussion, and the contribution of wearables to the collection, development, application of biomedical knowledge has only partially been discussed. To fill in these gaps, in this article we provide an epistemic (knowledge-related) overview of the main functions of wearable technology for health: monitoring, screening, detection, and prediction. On this basis, we identify 4 areas of concern in the application of wearables for these functions: data quality, balanced estimations, health equity, and fairness. To move the field forward in an effective and beneficial direction, we present recommendations for the 4 areas: local standards of quality, interoperability, access, and representativity.

An external stability audit framework to test the validity of personality prediction in AI hiring

Automated hiring systems are among the fastest-developing of all high-stakes AI systems. Among these are algorithmic personality tests that use insights from psychometric testing, and promise to surface personality traits indicative of future success based on job seekers' resumes or social media profiles. We interrogate the validity of such systems using stability of the outputs they produce, noting that reliability is a necessary, but not a sufficient, condition for validity. Crucially, rather than challenging or affirming the assumptions made in psychometric testing - that personality is a meaningful and measurable construct, and that personality traits are indicative of future success on the job - we frame our audit methodology around testing the underlying assumptions made by the vendors of the algorithmic personality tests themselves. Our main contribution is the development of a socio-technical framework for auditing the stability of algorithmic systems. This contribution is supplemented with an open-source software library that implements the technical components of the audit, and can be used to conduct similar stability audits of algorithmic systems. We instantiate our framework with the audit of two real-world personality prediction systems, namely, Humantic AI and Crystal. The application of our audit framework demonstrates that both these systems show substantial instability with respect to key facets of measurement, and hence cannot be considered valid testing instruments.

Combining and comparing regional SARS-CoV-2 epidemic dynamics in Italy: Bayesian meta-analysis of compartmental models and global sensitivity analysis

During autumn 2020, Italy faced a second important SARS-CoV-2 epidemic wave. We explored the time pattern of the instantaneous reproductive number, R ₀(t), and estimated the prevalence of infections by region from August to December calibrating SIRD models on COVID-19-related deaths, fixing at values from literature Infection Fatality Rate (IFR) and average infection duration. A Global Sensitivity Analysis (GSA) was performed on the regional SIRD models. Then, we used Bayesian meta-analysis and meta-regression to combine and compare the regional results and investigate their heterogeneity. The meta-analytic R ₀(t) curves were similar in the Northern and Central regions, while a less peaked curve was estimated for the South. The maximum R ₀(t) ranged from 2.15 (South) to 2.61 (North) with an increase following school reopening and a decline at the end of October. The predictive performance of the regional models, assessed through cross validation, was good, with a Mean Absolute Percentage Error of 7.2% and 10.9% when considering prediction horizons of 7 and 14 days, respectively. Average temperature, urbanization, characteristics of family medicine and healthcare system, economic dynamism, and use of public transport could partly explain the regional heterogeneity. The GSA indicated the robustness of the regional R ₀(t) curves to different assumptions on IFR. The infectious period turned out to have a key role in determining the model results, but without compromising between-region comparisons.

Building the Culture of Public Health as a Positive Reflection from the COVID-19 Crisis

The goal of this study was to look at the impact of different components of COVID-19 on the development of a public health culture during the COVID-19 pandemic. Culture is largely acknowledged to have the greatest and most profound influence on many aspects of human and group behavior. Culture is the process of integration that distinguishes members of one group or category of people through others; in other words, culture is the operating system of the communities, cultural nuances among societies cause people to demonstrate different behavioral patterns in the face of events. However, the cultural differences of each community make it impossible to apply these metrics universally. Observing the impact of countries’ cultural traits in the fight against outbreaks is one area where social scientists have not put much attention. As a result, this study demonstrates how cultural shifts affect their fight against outbreaks and public health challenges. The human response to COVID-19 is likely to be influenced by country culture and how (and when) overcome will be determined by it. Cultural ideas and assumptions should be assessed as part of public health interventions. These interventions should indeed be addressed at the societal level to stimulate awareness and participation while also guaranteeing culturally relevant events. Healthcare practitioners should emphasize public health exertions on culturally relevant forms of instruction, prevention, behavior and follow-up for successful pandemic management, effective screening, and diagnosis in infected individuals. Public health professionals must recognize the significance of this and learn to use communication messaging and guidelines, which must be culturally acceptable and context-dependent rather than merely grounded on medical methodologies, as was the case in many countries with COVID-19 management.

Forecasting new diseases in low-data settings using transfer learning

Recent infectious disease outbreaks, such as the COVID-19 pandemic and the Zika epidemic in Brazil, have demonstrated both the importance and difficulty of accurately forecasting novel infectious diseases. When new diseases first emerge, we have little knowledge of the transmission process, the level and duration of immunity to reinfection, or other parameters required to build realistic epidemiological models. Time series forecasts and machine learning, while less reliant on assumptions about the disease, require large amounts of data that are also not available in early stages of an outbreak. In this study, we examine how knowledge of related diseases can help make predictions of new diseases in data-scarce environments using transfer learning. We implement both an empirical and a synthetic approach. Using data from Brazil, we compare how well different machine learning models transfer knowledge between two different dataset pairs: case counts of (i) dengue and Zika, and (ii) influenza and COVID-19. In the synthetic analysis, we generate data with an SIR model using different transmission and recovery rates, and then compare the effectiveness of different transfer learning methods. We find that transfer learning offers the potential to improve predictions, even beyond a model based on data from the target disease, though the appropriate source disease must be chosen carefully. While imperfect, these models offer an additional input for decision makers for pandemic response.

Improving the reliability of cohesion policy databases

In this contribution, we present an innovative data-driven model to reconstruct a reliable temporal pattern for time-lagged statistical monetary figures. Our research cuts across several domains regarding the production of robust economic inferences and the bridging of top-down aggregated information from central databases with disaggregated information obtained from local sources or national statistical offices. Our test bed case study is the European Regional Development Fund (ERDF). The application we discuss deals with the reported time lag between the local expenditures of ERDF by beneficiaries in Italian regions and the corresponding payments reported in the European Commission database. Our model reconstructs the timing of these local expenditures by back-dating the observed European Commission reimbursements. The inferred estimates are then validated against the expenditures reported from the Italian National Managing Authorities (NMAs) in terms of cumulative monetary difference. The lower cumulative yearly distance of our modelled expenditures compared to the official European Commission payments confirms the robustness of our model. Using sensitivity analysis, we also analyse the relative importance of the modelling parameters on the cumulative distance between the modelled and reported expenditures. The parameters with the greatest influence on the uncertainty of this distance are the following: first, how the non-clearly regionalised expenditures are attributed to individual regions; and second, the number of backward years that the residuals of the yearly payments are spread onto. In general, the distance between the modelled and reported expenditures can be further reduced by fixing these parameters. However, the gain is only marginal for some regions. The present study paves the way for modelling exercises that are aimed at more reliable estimates of the expenditures on the ground by the ultimate beneficiaries of European funds. Additionally, the output databases can contribute to enhancing the reliability of econometric studies on the effectiveness of European Union (EU) funds.

The role of statisticians in the response to COVID-19 in Israel: a holistic point of view

The COVID-19 pandemic cast a dramatic spotlight on the use of data as a fundamental component of good decision-making. Evaluating and comparing alternative policies required information on concurrent infection rates and insightful analysis to project them into the future. Statisticians in Israel were involved in these processes early in the pandemic in some silos as an ad-hoc unorganized effort. Informal discussions within the statistical community culminated in a roundtable, organized by three past presidents of the Israel Statistical Association, and hosted by the Samuel Neaman Institute in April 2021. The meeting was designed to provide a forum for exchange of views on the profession's role during the COVID-19 pandemic, and more generally, on its influence in promoting evidence-based public policy. This paper builds on the insights and discussions that emerged during the roundtable meeting and presents a general framework, with recommendations, for involving statisticians and statistics in decision-making.

Forecasting for COVID-19 has failed

Epidemic forecasting has a dubious track-record, and its failures became more prominent with COVID-19. Poor data input, wrong modeling assumptions, high sensitivity of estimates, lack of incorporation of epidemiological features, poor past evidence on effects of available interventions, lack of transparency, errors, lack of determinacy, consideration of only one or a few dimensions of the problem at hand, lack of expertise in crucial disciplines, groupthink and bandwagon effects, and selective reporting are some of the causes of these failures. Nevertheless, epidemic forecasting is unlikely to be abandoned. Some (but not all) of these problems can be fixed. Careful modeling of predictive distributions rather than focusing on point estimates, considering multiple dimensions of impact, and continuously reappraising models based on their validated performance may help. If extreme values are considered, extremes should be considered for the consequences of multiple dimensions of impact so as to continuously calibrate predictive insights and decision-making. When major decisions (e.g. draconian lockdowns) are based on forecasts, the harms (in terms of health, economy, and society at large) and the asymmetry of risks need to be approached in a holistic fashion, considering the totality of the evidence.

Opportunities and Barriers to the Development and Use of Open Source Health Economic Models: A Survey

OBJECTIVES

Health economic (HE) models are routinely used to support health policy and resource allocation decisions but are often considered "black boxes" that may be prone to error and bias. Open source models (OSMs) have been advocated to increase the transparency, credibility, and reuse of HE models. Previous studies have demonstrated interest in OSMs among the health economics and outcomes research community, but the number of OSMs remains low.

METHODS

We conducted an online survey of ISPOR (the leading professional society for health economics and outcomes research) members' perspectives on the usefulness of OSMs and barriers to their development and implementation.

RESULTS

Respondents (N = 230) included academics (27%), pharmaceutical (or related) industry representatives (23%), health research or consulting representatives (21%), governmental or nonprofit agency representatives (10%), and others (19%). Respondents were generally not familiar with barriers to the development and adoption of OSMs. Most agreed that OSMs would improve transparency (92%), efficiency (76%), and HE model reuse (86%) and promote confidence in using HE models (75%). The use of OSMs by health technology assessment authorities was considered a very important indicator of the usefulness of OSMs by 49% of respondents. Three-quarters of respondents perceived legal concerns and the ability to transfer data as important barriers to the development and use of OSMs.

CONCLUSIONS

Respondents believe that OSMs could increase the transparency, efficiency, and credibility of HE models, but that several barriers hamper their widespread adoption. Our results suggest that fundamental changes may be needed across the health economics and outcomes research community if OSMs are to become widely adopted.

How Meaningful is the Elite Quality Index Ranking?

The Elite Quality Index (EQx) attempts to measure the propensity of elites-on aggregate-to create value, rather than to rent seek. The index has attracted worldwide media and press attention. In their articles, journalists have based their analyses primarily on their own countries' position in the EQx ranking. But how meaningful is the EQx ranking? How do the uncertainties underlying some of the assumptions made in the index propagate to the country rankings? We conduct a global uncertainty and sensitivity analysis (UA and SA) of the EQx and compute Sobol' first and total order sensitivity indices using state of the art estimators, in order to scrutinise the implications of index assumptions and assess the reliability of the EQx ranking. The UA suggests that the EQx ranking of 2021 (EQx2021) is largely stable for the top 50 countries, but exhibits considerable uncertainties especially for middle and lower performing countries. The SA highlights the handling of missing data, the normalisation process and the weighting scheme as most important methodological choices, while the largest potential for improvement is observed in how raw missing indicator data is handled.

SAGE advice and political decision-making: 'Following the science' in times of epistemic uncertainty

This article presents a preliminary analysis of the advice provided by the UK government's Scientific Advisory Group for Emergencies (SAGE) held between 22 January and 23 March 2020 in response to the emerging coronavirus pandemic. Drawing on the published minutes of the group's meetings, the article examines what was known and not known, the assumptions and working practices that shaped their work, and how this knowledge was reflected in the decisions made by the government. In doing so, the article critically examines what it means for policy making to be 'led by the science' when the best available science is provisional and uncertain. Using ideas of 'externality' and 'evidential significance', the article argues that the apparent desire for high levels of certainty by both scientists and political decision-makers made early action impossible as the data needed were not, and could not be, available in time. This leads to an argument for changes to the institutions that provide scientific advice based on sociologically informed expectations of science in which expert judgement plays a more significant role.

Increasing Preparedness for Extreme Events using Plausibility-Based Scenario Planning: Lessons from COVID-19

A striking feature of COVID-19 is many countries' low level of preparedness for it, despite pandemics being a known threat. This raises a question as to the reasons for this underpreparedness. While preparedness should have better reflected pandemics' long-run inevitability and potentially catastrophic impact, government-planning horizons are short term, and the attentiveness of policymakers is bounded and subject to multiple demands. Preparedness is therefore affected by the fundamental uncertainty surrounding the exact nature, timing, and impact of a pandemic. While a subjective probability is attributable to such an event's occurrence, just like it is any other, if founded on scant knowledge and perceived as being low it may inhibit preparedness. Under such circumstances, preparedness may be better served by a focus on plausibility. Moreover, any tendency for policymakers to disregard highly uncertain, low-probability, yet highly impactful events of this type is exacerbated by their "fat-tailed" distribution, which obscures their potential extremity. This article considers how plausibility-based scenario planning can increase preparedness for extreme events like a global pandemic, thereby reducing overconfidence in continued business-as-usual in their face, and emphasizing precaution in their wake. In so doing, the article contributes to what in this journal has recently been called "type B," "generic and fundamental" risk science, which is concerned with identifying better ways to present and communicate uncertainties. In focusing on plausibility-based scenario planning, the article highlights a method seldom previously discussed in relation to risk science, yet one that can contribute much to this type B component of it.