Guarding Against Seven Common Threats to the Credible Estimation of COVID-19 Policy Effects

Where Do We Go From Here? A Framework for Using Susceptible-Infectious-Recovered Models for Policy Making in Emerging Infectious Diseases

OBJECTIVES

Throughout the coronavirus disease 2019 pandemic, susceptible-infectious-recovered (SIR) modeling has been the preeminent modeling method to inform policy making worldwide. Nevertheless, the usefulness of such models has been subject to controversy. An evolution in the epidemiological modeling field is urgently needed, beginning with an agreed-upon set of modeling standards for policy recommendations. The objective of this article is to propose a set of modeling standards to support policy decision making.

METHODS

We identify and describe 5 broad standards: transparency, heterogeneity, calibration and validation, cost-benefit analysis, and model obsolescence and recalibration. We give methodological recommendations and provide examples in the literature that employ these standards well. We also develop and demonstrate a modeling practices checklist using existing coronavirus disease 2019 literature that can be employed by readers, authors, and reviewers to evaluate and compare policy modeling literature along our formulated standards.

RESULTS

We graded 16 articles using our checklist. On average, the articles met 6.81 of our 19 categories (36.7%). No articles contained any cost-benefit analyses and few were adequately transparent.

CONCLUSIONS

There is significant room for improvement in modeling pandemic policy. Issues often arise from a lack of transparency, poor modeling assumptions, lack of a system-wide perspective in modeling, and lack of flexibility in the academic system to rapidly iterate modeling as new information becomes available. In anticipation of future challenges, we encourage the modeling community at large to contribute toward the refinement and consensus of a shared set of standards for infectious disease policy modeling.