Characterising information gains and losses when collecting multiple epidemic model outputs

BACKGROUND

Collaborative comparisons and combinations of epidemic models are used as policy-relevant evidence during epidemic outbreaks. In the process of collecting multiple model projections, such collaborations may gain or lose relevant information. Typically, modellers contribute a probabilistic summary at each time-step. We compared this to directly collecting simulated trajectories. We aimed to explore information on key epidemic quantities; ensemble uncertainty; and performance against data, investigating potential to continuously gain information from a single cross-sectional collection of model results.

METHODS

We compared projections from the European COVID-19 Scenario Modelling Hub. Five teams modelled incidence in Belgium, the Netherlands, and Spain. We compared July 2022 projections by incidence, peaks, and cumulative totals. We created a probabilistic ensemble drawn from all trajectories, and compared to ensembles from a median across each model's quantiles, or a linear opinion pool. We measured the predictive accuracy of individual trajectories against observations, using this in a weighted ensemble. We repeated this sequentially against increasing weeks of observed data. We evaluated these ensembles to reflect performance with varying observed data.

RESULTS

By collecting modelled trajectories, we showed policy-relevant epidemic characteristics. Trajectories contained a right-skewed distribution well represented by an ensemble of trajectories or a linear opinion pool, but not models' quantile intervals. Ensembles weighted by performance typically retained the range of plausible incidence over time, and in some cases narrowed this by excluding some epidemic shapes.

CONCLUSIONS

We observed several information gains from collecting modelled trajectories rather than quantile distributions, including potential for continuously updated information from a single model collection. The value of information gains and losses may vary with each collaborative effort's aims, depending on the needs of projection users. Understanding the differing information potential of methods to collect model projections can support the accuracy, sustainability, and communication of collaborative infectious disease modelling efforts.

Transition from time-variant to static networks: Timescale separation in N-intertwined mean-field approximation of susceptible-infectious-susceptible epidemics

We extend the N-intertwined mean-field approximation (NIMFA) for the susceptible-infectious-susceptible (SIS) epidemiological process to time-varying networks. Processes on time-varying networks are often analyzed under the assumption that the process and network evolution happen on different timescales. This approximation is called timescale separation. We investigate timescale separation between disease spreading and topology updates of the network. We introduce the transition times [under T]̲(r) and T[over ¯](r) as the boundaries between the intermediate regime and the annealed (fast changing network) and quenched (static network) regimes, respectively, for a fixed accuracy tolerance r. By analyzing the convergence of static NIMFA processes, we analytically derive upper and lower bounds for T[over ¯](r). Our results provide insights and bounds on the time of convergence to the steady state of the static NIMFA SIS process. We show that, under our assumptions, the upper-transition time T[over ¯](r) is almost entirely determined by the basic reproduction number R_{0} of the network. The value of the upper-transition time T[over ¯](r) around the epidemic threshold is large, which agrees with the current understanding that some real-world epidemics cannot be approximated with the aforementioned timescale separation.

Predicting subnational incidence of COVID-19 cases and deaths in EU countries

BACKGROUND

Recurring COVID-19 waves highlight the need for tools able to quantify transmission risk, and identify geographical areas at risk of outbreaks. Local outbreak risk depends on complex immunity patterns resulting from previous infections, vaccination, waning and immune escape, alongside other factors (population density, social contact patterns). Immunity patterns are spatially and demographically heterogeneous, and are challenging to capture in country-level forecast models.

METHODS

We used a spatiotemporal regression model to forecast subnational case and death counts and applied it to three EU countries as test cases: France, Czechia, and Italy. Cases in local regions arise from importations or local transmission. Our model produces age-stratified forecasts given age-stratified data, and links reported case counts to routinely collected covariates (e.g. test number, vaccine coverage). We assessed the predictive performance of our model up to four weeks ahead using proper scoring rules and compared it to the European COVID-19 Forecast Hub ensemble model. Using simulations, we evaluated the impact of variations in transmission on the forecasts. We developed an open-source RShiny App to visualise the forecasts and scenarios.

RESULTS

At a national level, the median relative difference between our median weekly case forecasts and the data up to four weeks ahead was 25% (IQR: 12-50%) over the prediction period. The accuracy decreased as the forecast horizon increased (on average 24% increase in the median ranked probability score per added week), while the accuracy of death forecasts was more stable. Beyond two weeks, the model generated a narrow range of likely transmission dynamics. The median national case forecasts showed similar accuracy to forecasts from the European COVID-19 Forecast Hub ensemble model, but the prediction interval was narrower in our model. Generating forecasts under alternative transmission scenarios was therefore key to capturing the range of possible short-term transmission dynamics.

DISCUSSION

Our model captures changes in local COVID-19 outbreak dynamics, and enables quantification of short-term transmission risk at a subnational level. The outputs of the model improve our ability to identify areas where outbreaks are most likely, and are available to a wide range of public health professionals through the Shiny App we developed.

The influence of COVID-19 risk perception and vaccination status on the number of social contacts across Europe: insights from the CoMix study

BACKGROUND

The SARS-CoV-2 transmission dynamics have been greatly modulated by human contact behaviour. To curb the spread of the virus, global efforts focused on implementing both Non-Pharmaceutical Interventions (NPIs) and pharmaceutical interventions such as vaccination. This study was conducted to explore the influence of COVID-19 vaccination status and risk perceptions related to SARS-CoV-2 on the number of social contacts of individuals in 16 European countries.

METHODS

We used data from longitudinal surveys conducted in the 16 European countries to measure social contact behaviour in the course of the pandemic. The data consisted of representative panels of participants in terms of gender, age and region of residence in each country. The surveys were conducted in several rounds between December 2020 and September 2021 and comprised of 29,292 participants providing a total of 111,103 completed surveys. We employed a multilevel generalized linear mixed effects model to explore the influence of risk perceptions and COVID-19 vaccination status on the number of social contacts of individuals.

RESULTS

The results indicated that perceived severity played a significant role in social contact behaviour during the pandemic after controlling for other variables (p-value < 0.001). More specifically, participants who had low or neutral levels of perceived severity reported 1.25 (95% Confidence intervals (CI) 1.13 - 1.37) and 1.10 (95% CI 1.00 - 1.21) times more contacts compared to those who perceived COVID-19 to be a serious illness, respectively. Additionally, vaccination status was also a significant predictor of contacts (p-value < 0.001), with vaccinated individuals reporting 1.31 (95% CI 1.23 - 1.39) times higher number of contacts than the non-vaccinated. Furthermore, individual-level factors played a more substantial role in influencing contact behaviour than country-level factors.

CONCLUSION

Our multi-country study yields significant insights on the importance of risk perceptions and vaccination in behavioral changes during a pandemic emergency. The apparent increase in social contact behaviour following vaccination would require urgent intervention in the event of emergence of an immune escaping variant.

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Background: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts' predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022.

Methods: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance.

Results: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models.

Conclusions: Our results support combining independent models into an ensemble forecast to improve epidemiological predictions, and suggest that median averages yield better performance than methods based on means. We highlight that forecast consumers should place more weight on incident death forecasts than case forecasts at horizons greater than two weeks.

Funding: European Commission, Ministerio de Ciencia, Innovación y Universidades, FEDER; Agència de Qualitat i Avaluació Sanitàries de Catalunya; Netzwerk Universitätsmedizin; Health Protection Research Unit; Wellcome Trust; European Centre for Disease Prevention and Control; Ministry of Science and Higher Education of Poland; Federal Ministry of Education and Research; Los Alamos National Laboratory; German Free State of Saxony; NCBiR; FISR 2020 Covid-19 I Fase; Spanish Ministry of Health / REACT-UE (FEDER); National Institutes of General Medical Sciences; Ministerio de Sanidad/ISCIII; PERISCOPE European H2020; PERISCOPE European H2021; InPresa; National Institutes of Health, NSF, US Centers for Disease Control and Prevention, Google, University of Virginia, Defense Threat Reduction Agency.

Predicting time-resolved electrophysiological brain networks from structural eigenmodes

How temporal modulations in functional interactions are shaped by the underlying anatomical connections remains an open question. Here, we analyse the role of structural eigenmodes, in the formation and dissolution of temporally evolving functional brain networks using resting-state magnetoencephalography and diffusion magnetic resonance imaging data at the individual subject level. Our results show that even at short timescales, phase and amplitude connectivity can partly be expressed by structural eigenmodes, but hardly by direct structural connections. Albeit a stronger relationship was found between structural eigenmodes and time-resolved amplitude connectivity. Time-resolved connectivity for both phase and amplitude was mostly characterised by a stationary process, superimposed with very brief periods that showed deviations from this stationary process. For these brief periods, dynamic network states were extracted that showed different expressions of eigenmodes. Furthermore, the eigenmode expression was related to overall cognitive performance and co-occurred with fluctuations in community structure of functional networks. These results implicate that ongoing time-resolved resting-state networks, even at short timescales, can to some extent be understood in terms of activation and deactivation of structural eigenmodes and that these eigenmodes play a role in the dynamic integration and segregation of information across the cortex, subserving cognitive functions.

Analysis of continuous-time Markovian ɛ-SIS epidemics on networks

We analyze continuous-time Markovian ɛ-SIS epidemics with self-infections on the complete graph. The majority of the graphs are analytically intractable, but many physical features of the ɛ-SIS process observed in the complete graph can occur in any other graph. In this work, we illustrate that the timescales of the ɛ-SIS process are related to the eigenvalues of the tridiagonal matrix of the SIS Markov chain. We provide a detailed analysis of all eigenvalues and illustrate that the eigenvalues show staircases, which are caused by the nearly degenerate (but strictly distinct) pairs of eigenvalues. We also illustrate that the ratio between the second-largest and third-largest eigenvalue is a good indicator of metastability in the ɛ-SIS process. Additionally, we show that the epidemic threshold of the Markovian ɛ-SIS process can be accurately approximated by the effective infection rate for which the third-largest eigenvalue of the transition matrix is the smallest. Finally, we derive the exact mean-field solution for the ɛ-SIS process on the complete graph, and we show that the mean-field approximation does not correctly represent the metastable behavior of Markovian ɛ-SIS epidemics.

Public health considerations for transitioning beyond the acute phase of the COVID-19 pandemic in the EU/EEA

Many countries, including some within the EU/EEA, are in the process of transitioning from the acute pandemic phase. During this transition, it is crucial that countries’ strategies and activities remain guided by clear COVID-19 control objectives, which increasingly will focus on preventing and managing severe outcomes. Therefore, attention must be given to the groups that are particularly vulnerable to severe outcomes of SARS-CoV-2 infection, including individuals in congregate and healthcare settings. In this phase of pandemic management, a strong focus must remain on transitioning testing approaches and systems for targeted surveillance of COVID-19, capitalising on and strengthening existing systems for respiratory virus surveillance. Furthermore, it will be crucial to focus on lessons learned from the pandemic to enhance preparedness and to enact robust systems for the preparedness, detection, rapid investigation and assessment of new and emerging SARS-CoV-2 variants. Filling existing knowledge gaps, including behavioural insights, can help guide the response to future resurgences of SARS-CoV-2 and/or the emergence of other pandemics. Finally, ‘vaccine agility’ will be needed to respond to changes in people’s behaviours, changes in the virus, and changes in population immunity, all the while addressing issues of global health equity.

Comparing the accuracy of several network-based COVID-19 prediction algorithms

Researchers from various scientific disciplines have attempted to forecast the spread of coronavirus disease 2019 (COVID-19). The proposed epidemic prediction methods range from basic curve fitting methods and traffic interaction models to machine-learning approaches. If we combine all these approaches, we obtain the Network Inference-based Prediction Algorithm (NIPA). In this paper, we analyse a diverse set of COVID-19 forecast algorithms, including several modifications of NIPA. Among the algorithms that we evaluated, the original NIPA performed best at forecasting the spread of COVID-19 in Hubei, China and in the Netherlands. In particular, we show that network-based forecasting is superior to any other forecasting algorithm.

Accuracy of predicting epidemic outbreaks

During the outbreak of a virus, perhaps the greatest concern is the future evolution of the epidemic: How many people will be infected and which regions will be affected the most? The accurate prediction of an epidemic enables targeted disease countermeasures (e.g., allocating medical staff and quarantining). But when can we trust the prediction of an epidemic to be accurate? In this work we consider susceptible-infected-susceptible (SIS) and susceptible-infected-removed (SIR) epidemics on networks with time-invariant spreading parameters. (For time-varying spreading parameters, our results correspond to an optimistic scenario for the predictability of epidemics.) Our contribution is twofold. First, accurate long-term predictions of epidemics are possible only after the peak rate of new infections. Hence, before the peak, only short-term predictions are reliable. Second, we define an exponential growth metric, which quantifies the predictability of an epidemic. In particular, even without knowing the future evolution of the epidemic, the growth metric allows us to compare the predictability of an epidemic at different points in time. Our results are an important step towards understanding when and why epidemics can be predicted reliably.

Clustering for epidemics on networks: A geometric approach

Infectious diseases typically spread over a contact network with millions of individuals, whose sheer size is a tremendous challenge to analyzing and controlling an epidemic outbreak. For some contact networks, it is possible to group individuals into clusters. A high-level description of the epidemic between a few clusters is considerably simpler than on an individual level. However, to cluster individuals, most studies rely on equitable partitions, a rather restrictive structural property of the contact network. In this work, we focus on Susceptible-Infected-Susceptible (SIS) epidemics, and our contribution is threefold. First, we propose a geometric approach to specify all networks for which an epidemic outbreak simplifies to the interaction of only a few clusters. Second, for the complete graph and any initial viral state vectors, we derive the closed-form solution of the nonlinear differential equations of the N-intertwined mean-field approximation of the SIS process. Third, by relaxing the notion of equitable partitions, we derive low-complexity approximations and bounds for epidemics on arbitrary contact networks. Our results are an important step toward understanding and controlling epidemics on large networks.

Time-dependent solution of the NIMFA equations around the epidemic threshold

The majority of epidemic models are described by non-linear differential equations which do not have a closed-form solution. Due to the absence of a closed-form solution, the understanding of the precise dynamics of a virus is rather limited. We solve the differential equations of the N-intertwined mean-field approximation of the susceptible-infected-susceptible epidemic process with heterogeneous spreading parameters around the epidemic threshold for an arbitrary contact network, provided that the initial viral state vector is small or parallel to the steady-state vector. Numerical simulations demonstrate that the solution around the epidemic threshold is accurate, also above the epidemic threshold and for general initial viral states that are below the steady-state.

Network-inference-based prediction of the COVID-19 epidemic outbreak in the Chinese province Hubei

At the moment of writing, the future evolution of the COVID-19 epidemic is unclear. Predictions of the further course of the epidemic are decisive to deploy targeted disease control measures. We consider a network-based model to describe the COVID-19 epidemic in the Hubei province. The network is composed of the cities in Hubei and their interactions (e.g., traffic flow). However, the precise interactions between cities is unknown and must be inferred from observing the epidemic. We propose the Network-Inference-Based Prediction Algorithm (NIPA) to forecast the future prevalence of the COVID-19 epidemic in every city. Our results indicate that NIPA is beneficial for an accurate forecast of the epidemic outbreak.

Network-based prediction of COVID-19 epidemic spreading in Italy

Initially emerged in the Chinese city Wuhan and subsequently spread almost worldwide causing a pandemic, the SARS-CoV-2 virus follows reasonably well the Susceptible-Infectious-Recovered (SIR) epidemic model on contact networks in the Chinese case. In this paper, we investigate the prediction accuracy of the SIR model on networks also for Italy. Specifically, the Italian regions are a metapopulation represented by network nodes and the network links are the interactions between those regions. Then, we modify the network-based SIR model in order to take into account the different lockdown measures adopted by the Italian Government in the various phases of the spreading of the COVID-19. Our results indicate that the network-based model better predicts the daily cumulative infected individuals when time-varying lockdown protocols are incorporated in the classical SIR model.