Lack of practical identifiability may hamper reliable predictions in COVID-19 epidemic models

Mathematical modelling for pandemic preparedness in Canada: Learning from COVID-19

Background

The COVID-19 pandemic underlined the need for pandemic planning but also brought into focus the use of mathematical modelling to support public health decisions. The types of models needed (compartment, agent-based, importation) are described. Best practices regarding biological realism (including the need for multidisciplinary expert advisors to modellers), model complexity, consideration of uncertainty and communications to decision-makers and the public are outlined.

Methods

A narrative review was developed from the experiences of COVID-19 by members of the Public Health Agency of Canada External Modelling Network for Infectious Diseases (PHAC EMN-ID), a national community of practice on mathematical modelling of infectious diseases for public health.

Results

Modelling can best support pandemic preparedness in two ways: 1) by modelling to support decisions on resource needs for likely future pandemics by estimating numbers of infections, hospitalized cases and cases needing intensive care, associated with epidemics of "hypothetical-yet-plausible" pandemic pathogens in Canada; and 2) by having ready-to-go modelling methods that can be readily adapted to the features of an emerging pandemic pathogen and used for long-range forecasting of the epidemic in Canada, as well as to explore scenarios to support public health decisions on the use of interventions.

Conclusion

There is a need for modelling expertise within public health organizations in Canada, linked to modellers in academia in a community of practice, within which relationships built outside of times of crisis can be applied to enhance modelling during public health emergencies. Key challenges to modelling for pandemic preparedness include the availability of linked public health, hospital and genomic data in Canada.

Fitting Epidemic Models to Data: A Tutorial in Memory of Fred Brauer

Fred Brauer was an eminent mathematician who studied dynamical systems, especially differential equations. He made many contributions to mathematical epidemiology, a field that is strongly connected to data, but he always chose to avoid data analysis. Nevertheless, he recognized that fitting models to data is usually necessary when attempting to apply infectious disease transmission models to real public health problems. He was curious to know how one goes about fitting dynamical models to data, and why it can be hard. Initially in response to Fred's questions, we developed a user-friendly R package, fitode, that facilitates fitting ordinary differential equations to observed time series. Here, we use this package to provide a brief tutorial introduction to fitting compartmental epidemic models to a single observed time series. We assume that, like Fred, the reader is familiar with dynamical systems from a mathematical perspective, but has limited experience with statistical methodology or optimization techniques.

A data-driven analysis on the mediation effect of compartment models between control measures and COVID-19 epidemics

By collecting various control policies taken by 127 countries/territories during the first wave of COVID-19 pandemic until July 2nd, 2020, we evaluate their impacts on the epidemic dynamics quantitatively through a combination of the multiple linear regression, neural-network-based nonlinear regression and sensitivity analysis. Remarkable differences in the public health policies are observed across these countries, which affect the spreading rate and infected population size to a great extent. Several key dynamical features, like the normalized cumulative numbers of confirmed/cured/death cases on the 100th day and the half time, show statistically significant linear correlations with the control measures, which thereby confirms their dramatic impacts. Most importantly, we perform the mediation analysis on the SEIR-QD model, a representative of general compartment models, by using the structure equation modeling for multiple mediators operating in parallel. This, to the best of our knowledge, is the first of its kind in the field of epidemiology. The infection rate and the protection rate of the SEIR-QD model are confirmed to exhibit a statistically significant mediation effect between the control measures and dynamical features of epidemics. The mediation effect along the pathway from control measures in Category 2 to four dynamical features through the infection rate, highlights the crucial role of nucleic acid testing and suspected cases tracing in containing the spread of the epidemic. Our data-driven analysis offers a deeper insight into the inherent correlations between the effectiveness of public health policies and the dynamic features of COVID-19 epidemics.

Reconstructing higher-order interactions in coupled dynamical systems

Higher-order interactions play a key role for the operation and function of a complex system. However, how to identify them is still an open problem. Here, we propose a method to fully reconstruct the structural connectivity of a system of coupled dynamical units, identifying both pairwise and higher-order interactions from the system time evolution. Our method works for any dynamics, and allows the reconstruction of both hypergraphs and simplicial complexes, either undirected or directed, unweighted or weighted. With two concrete applications, we show how the method can help understanding the complexity of bacterial systems, or the microscopic mechanisms of interaction underlying coupled chaotic oscillators.

A novel within-host model of HIV and nutrition

In this paper we develop a four compartment within-host model of nutrition and HIV. We show that the model has two equilibria: an infection-free equilibrium and infection equilibrium. The infection free equilibrium is locally asymptotically stable when the basic reproduction number $ \mathcal{R}_0 < 1 $, and unstable when $ \mathcal{R}_0 > 1 $. The infection equilibrium is locally asymptotically stable if $ \mathcal{R}_0 > 1 $ and an additional condition holds. We show that the within-host model of HIV and nutrition is structured to reveal its parameters from the observations of viral load, CD4 cell count and total protein data. We then estimate the model parameters for these 3 data sets. We have also studied the practical identifiability of the model parameters by performing Monte Carlo simulations, and found that the rate of clearance of the virus by immunoglobulins is practically unidentifiable, and that the rest of the model parameters are only weakly identifiable given the experimental data. Furthermore, we have studied how the data frequency impacts the practical identifiability of model parameters.

How robust are estimates of key parameters in standard viral dynamic models?

Mathematical models of viral infection have been developed, fitted to data, and provide insight into disease pathogenesis for multiple agents that cause chronic infection, including HIV, hepatitis C, and B virus. However, for agents that cause acute infections or during the acute stage of agents that cause chronic infections, viral load data are often collected after symptoms develop, usually around or after the peak viral load. Consequently, we frequently lack data in the initial phase of viral growth, i.e., when pre-symptomatic transmission events occur. Missing data may make estimating the time of infection, the infectious period, and parameters in viral dynamic models, such as the cell infection rate, difficult. However, having extra information, such as the average time to peak viral load, may improve the robustness of the estimation. Here, we evaluated the robustness of estimates of key model parameters when viral load data prior to the viral load peak is missing, when we know the values of some parameters and/or the time from infection to peak viral load. Although estimates of the time of infection are sensitive to the quality and amount of available data, particularly pre-peak, other parameters important in understanding disease pathogenesis, such as the loss rate of infected cells, are less sensitive. Viral infectivity and the viral production rate are key parameters affecting the robustness of data fits. Fixing their values to literature values can help estimate the remaining model parameters when pre-peak data is missing or limited. We find a lack of data in the pre-peak growth phase underestimates the time to peak viral load by several days, leading to a shorter predicted growth phase. On the other hand, knowing the time of infection (e.g., from epidemiological data) and fixing it results in good estimates of dynamical parameters even in the absence of early data. While we provide ways to approximate model parameters in the absence of early viral load data, our results also suggest that these data, when available, are needed to estimate model parameters more precisely.

System identifiability in a time-evolving agent-based model

Mathematical models are a valuable tool for studying and predicting the spread of infectious agents. The accuracy of model simulations and predictions invariably depends on the specification of model parameters. Estimation of these parameters is therefore extremely important; however, while some parameters can be derived from observational studies, the values of others are difficult to measure. Instead, models can be coupled with inference algorithms (i.e., data assimilation methods, or statistical filters), which fit model simulations to existing observations and estimate unobserved model state variables and parameters. Ideally, these inference algorithms should find the best fitting solution for a given model and set of observations; however, as those estimated quantities are unobserved, it is typically uncertain whether the correct parameters have been identified. Further, it is unclear what 'correct' really means for abstract parameters defined based on specific model forms. In this work, we explored the problem of non-identifiability in a stochastic system which, when overlooked, can significantly impede model prediction. We used a network, agent-based model to simulate the transmission of Methicillin-resistant staphylococcus aureus (MRSA) within hospital settings and attempted to infer key model parameters using the Ensemble Adjustment Kalman Filter, an efficient Bayesian inference algorithm. We show that even though the inference method converged and that simulations using the estimated parameters produced an agreement with observations, the true parameters are not fully identifiable. While the model-inference system can exclude a substantial area of parameter space that is unlikely to contain the true parameters, the estimated parameter range still included multiple parameter combinations that can fit observations equally well. We show that analyzing synthetic trajectories can support or contradict claims of identifiability. While we perform this on a specific model system, this approach can be generalized for a variety of stochastic representations of partially observable systems. We also suggest data manipulations intended to improve identifiability that might be applicable in many systems of interest.

A computational approach to identifiability analysis for a model of the propagation and control of COVID-19 in Chile

A computational approach is adapted to analyze the parameter identifiability of a compartmental model. The model is intended to describe the progression of the COVID-19 pandemic in Chile during the initial phase in early 2020 when government declared quarantine measures. The computational approach to analyze the structural and practical identifiability is applied in two parts, one for synthetic data and another for some Chilean regional data. The first part defines the identifiable parameter sets when these recover the true parameters used to create the synthetic data. The second part compares the results derived from synthetic data, estimating the identifiable parameter sets from regional Chilean epidemic data. Experiments provide evidence of the loss of identifiability if some initial conditions are estimated, the period of time used to fit is before the peak, and if a significant proportion of the population is involved in quarantine periods.

Structural identifiability analysis of epidemic models based on differential equations: a tutorial-based primer

The successful application of epidemic models hinges on our ability to estimate model parameters from limited observations reliably. An often-overlooked step before estimating model parameters consists of ensuring that the model parameters are structurally identifiable from the observed states of the system. In this tutorial-based primer, intended for a diverse audience, including students training in dynamic systems, we review and provide detailed guidance for conducting structural identifiability analysis of differential equation epidemic models based on a differential algebra approach using differential algebra for identifiability of systems (DAISY) and Mathematica (Wolfram Research). This approach aims to uncover any existing parameter correlations that preclude their estimation from the observed variables. We demonstrate this approach through examples, including tutorial videos of compartmental epidemic models previously employed to study transmission dynamics and control. We show that the lack of structural identifiability may be remedied by incorporating additional observations from different model states, assuming that the system's initial conditions are known, using prior information to fix some parameters involved in parameter correlations, or modifying the model based on existing parameter correlations. We also underscore how the results of structural identifiability analysis can help enrich compartmental diagrams of differential-equation models by indicating the observed state variables and the results of the structural identifiability analysis.

Optimal age-specific vaccination control for COVID-19: An Irish case study

The outbreak of a novel coronavirus causing severe acute respiratory syndrome in December 2019 has escalated into a worldwide pandemic. In this work, we propose a compartmental model to describe the dynamics of transmission of infection and use it to obtain the optimal vaccination control. The model accounts for the various stages of the vaccination, and the optimisation is focused on minimising the infections to protect the population and relieve the healthcare system. As a case study, we selected the Republic of Ireland. We use data provided by Ireland's COVID-19 Data-Hub and simulate the evolution of the pandemic with and without the vaccination in place for two different scenarios, one representative of a national lockdown situation and the other indicating looser restrictions in place. One of the main findings of our work is that the optimal approach would involve a vaccination programme where the older population is vaccinated in larger numbers earlier while simultaneously part of the younger population also gets vaccinated to lower the risk of transmission between groups. We compare our simulated results with those of the vaccination policy taken by the Irish government to explore the advantages of our optimisation method. Our comparison suggests that a similar reduction in cases may have been possible even with a reduced set of vaccinations available for use.

On Parameter Identifiability in Network-Based Epidemic Models

Modelling epidemics on networks represents an important departure from classical compartmental models which assume random mixing. However, the resulting models are high-dimensional and their analysis is often out of reach. It turns out that mean-field models, low-dimensional systems of differential equations, whose variables are carefully chosen expected quantities from the exact model provide a good approximation and incorporate explicitly some network properties. Despite the emergence of such mean-field models, there has been limited work on investigating whether these can be used for inference purposes. In this paper, we consider network-based mean-field models and explore the problem of parameter identifiability when observations about an epidemic are available. Making use of the analytical tractability of most network-based mean-field models, e.g. explicit analytical expressions for leading eigenvalue and final epidemic size, we set up the parameter identifiability problem as finding the solution or solutions of a system of coupled equations. More precisely, subject to observing/measuring growth rate and final epidemic size, we seek to identify parameter values leading to these measurements. We are particularly concerned with disentangling transmission rate from the network density. To do this, we give a condition for practical identifiability and we find that except for the simplest model, parameters cannot be uniquely determined, that is, they are practically unidentifiable. This means that there exist multiple solutions (a manifold of infinite measure) which give rise to model output that is close to the data. Identifying, formalising and analytically describing this problem should lead to a better appreciation of the complexity involved in fitting models with many parameters to data.

The Structural Identifiability of a Humidity-Driven Epidemiological Model of Influenza Transmission

Influenza epidemics cause considerable morbidity and mortality every year worldwide. Climate-driven epidemiological models are mainstream tools to understand seasonal transmission dynamics and predict future trends of influenza activity, especially in temperate regions. Testing the structural identifiability of these models is a fundamental prerequisite for the model to be applied in practice, by assessing whether the unknown model parameters can be uniquely determined from epidemic data. In this study, we applied a scaling method to analyse the structural identifiability of four types of commonly used humidity-driven epidemiological models. Specifically, we investigated whether the key epidemiological parameters (i.e., infectious period, the average duration of immunity, the average latency period, and the maximum and minimum daily basic reproductive number) can be uniquely determined simultaneously when prevalence data is observable. We found that each model is identifiable when the prevalence of infection is observable. The structural identifiability of these models will lay the foundation for testing practical identifiability in the future using synthetic prevalence data when considering observation noise. In practice, epidemiological models should be examined with caution before using them to estimate model parameters from epidemic data.

A Modified PINN Approach for Identifiable Compartmental Models in Epidemiology with Application to COVID-19

Many approaches using compartmental models have been used to study the COVID-19 pandemic, with machine learning methods applied to these models having particularly notable success. We consider the Susceptible-Infected-Confirmed-Recovered-Deceased (SICRD) compartmental model, with the goal of estimating the unknown infected compartment I, and several unknown parameters. We apply a variation of a "Physics Informed Neural Network" (PINN), which uses knowledge of the system to aid learning. First, we ensure estimation is possible by verifying the model's identifiability. Then, we propose a wavelet transform to process data for the network training. Finally, our central result is a novel modification of the PINN's loss function to reduce the number of simultaneously considered unknowns. We find that our modified network is capable of stable, efficient, and accurate estimation, while the unmodified network consistently yields incorrect values. The modified network is also shown to be efficient enough to be applied to a model with time-varying parameters. We present an application of our model results for ranking states by their estimated relative testing efficiency. Our findings suggest the effectiveness of our modified PINN network, especially in the case of multiple unknown variables.

Why Controlling the Asymptomatic Infection Is Important: A Modelling Study with Stability and Sensitivity Analysis

The large proportion of asymptomatic patients is the major cause leading to the COVID-19 pandemic which is still a significant threat to the whole world. A six-dimensional ODE system (SEIAQR epidemical model) is established to study the dynamics of COVID-19 spreading considering infection by exposed, infected, and asymptomatic cases. The basic reproduction number derived from the model is more comprehensive including the contribution from the exposed, infected, and asymptomatic patients. For this more complex six-dimensional ODE system, we investigate the global and local stability of disease-free equilibrium, as well as the endemic equilibrium, whereas most studies overlooked asymptomatic infection or some other virus transmission features. In the sensitivity analysis, the parameters related to the asymptomatic play a significant role not only in the basic reproduction number R0. It is also found that the asymptomatic infection greatly affected the endemic equilibrium. Either in completely eradicating the disease or achieving a more realistic goal to reduce the COVID-19 cases in an endemic equilibrium, the importance of controlling the asymptomatic infection should be emphasized. The three-dimensional phase diagrams demonstrate the convergence point of the COVID-19 spreading under different initial conditions. In particular, massive infections will occur as shown in the phase diagram quantitatively in the case R0>1. Moreover, two four-dimensional contour maps of Rt are given varying with different parameters, which can offer better intuitive instructions on the control of the pandemic by adjusting policy-related parameters.