1
|
Hao B, Hu Y, Adams WG, Assoumou SA, Hsu HE, Bhadelia N, Paschalidis IC. A GPT-based EHR modeling system for unsupervised novel disease detection. J Biomed Inform 2024; 157:104706. [PMID: 39121932 DOI: 10.1016/j.jbi.2024.104706] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2024] [Revised: 07/17/2024] [Accepted: 08/06/2024] [Indexed: 08/12/2024]
Abstract
OBJECTIVE To develop an Artificial Intelligence (AI)-based anomaly detection model as a complement of an "astute physician" in detecting novel disease cases in a hospital and preventing emerging outbreaks. METHODS Data included hospitalized patients (n = 120,714) at a safety-net hospital in Massachusetts. A novel Generative Pre-trained Transformer (GPT)-based clinical anomaly detection system was designed and further trained using Empirical Risk Minimization (ERM), which can model a hospitalized patient's Electronic Health Records (EHR) and detect atypical patients. Methods and performance metrics, similar to the ones behind the recent Large Language Models (LLMs), were leveraged to capture the dynamic evolution of the patient's clinical variables and compute an Out-Of-Distribution (OOD) anomaly score. RESULTS In a completely unsupervised setting, hospitalizations for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) infection could have been predicted by our GPT model at the beginning of the COVID-19 pandemic, with an Area Under the Receiver Operating Characteristic Curve (AUC) of 92.2 %, using 31 extracted clinical variables and a 3-day detection window. Our GPT achieves individual patient-level anomaly detection and mortality prediction AUC of 78.3 % and 94.7 %, outperforming traditional linear models by 6.6 % and 9 %, respectively. Different types of clinical trajectories of a SARS-CoV-2 infection are captured by our model to make interpretable detections, while a trend of over-pessimistic outcome prediction yields a more effective detection pathway. Furthermore, our comprehensive GPT model can potentially assist clinicians with forecasting patient clinical variables and developing personalized treatment plans. CONCLUSION This study demonstrates that an emerging outbreak can be accurately detected within a hospital, by using a GPT to model patient EHR time sequences and labeling them as anomalous when actual outcomes are not supported by the model. Such a GPT is also a comprehensive model with the functionality of generating future patient clinical variables, which can potentially assist clinicians in developing personalized treatment plans.
Collapse
Affiliation(s)
- Boran Hao
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - Yang Hu
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA
| | - William G Adams
- Department of Pediatrics, Boston Medical Center, Boston, MA, USA; Chobanian & Avedisian School of Medicine, Boston University, Boston, MA, USA
| | - Sabrina A Assoumou
- Department of Medicine, Boston Medical Center, Boston, MA, USA; Chobanian & Avedisian School of Medicine, Boston University, Boston, MA, USA
| | - Heather E Hsu
- Department of Pediatrics, Boston Medical Center, Boston, MA, USA; Chobanian & Avedisian School of Medicine, Boston University, Boston, MA, USA
| | - Nahid Bhadelia
- Chobanian & Avedisian School of Medicine, Boston University, Boston, MA, USA; Center for Emerging Infectious Diseases Policy and Research, Boston University, Boston, MA, USA
| | - Ioannis Ch Paschalidis
- Department of Electrical and Computer Engineering, Boston University, Boston, MA, USA; Department of Biomedical Engineering, Division of Systems Engineering, Faculty of Computing & Data Sciences, and Hariri Institute for Computing and Computational Science & Engineering, Boston University, Boston, MA, USA.
| |
Collapse
|
2
|
Gerlee P, Thoreén H, Joöud AS, Lundh T, Spreco A, Nordlund A, Brezicka T, Britton T, Kjellberg M, Kaöllberg H, Tegnell A, Brouwers L, Timpka T. Evaluation and communication of pandemic scenarios. Lancet Digit Health 2024; 6:e543-e544. [PMID: 39059885 DOI: 10.1016/s2589-7500(24)00144-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 04/15/2024] [Accepted: 06/25/2024] [Indexed: 07/28/2024]
Affiliation(s)
- Philip Gerlee
- Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, 41296 Gothenburg, Sweden.
| | | | - Anna Saxne Joöud
- Department of Laboratory Medicine, Faculty of Medicine, Lund University, Lund, Sweden; Department of Clinical Sciences Lund, Lund University, Lund, Sweden; Department of Research and Development, Skaåne University Hospital, Lund, Sweden
| | - Torbjoörn Lundh
- Mathematical Sciences, Chalmers University of Technology and University of Gothenburg, 41296 Gothenburg, Sweden
| | - Armin Spreco
- Department of Health, Medicine and Caring Sciences, Linkoöping University, Linkoöping, Sweden; Regional Management Office, Region Oöstergoötland, Linkoöping, Sweden
| | | | | | - Tom Britton
- Department of Mathematics, Stockholm University, Stockholm, Sweden
| | | | | | | | | | - Toomas Timpka
- Department of Health, Medicine and Caring Sciences, Linkoöping University, Linkoöping, Sweden; Regional Management Office, Region Oöstergoötland, Linkoöping, Sweden
| |
Collapse
|
3
|
Mathis SM, Webber AE, León TM, Murray EL, Sun M, White LA, Brooks LC, Green A, Hu AJ, Rosenfeld R, Shemetov D, Tibshirani RJ, McDonald DJ, Kandula S, Pei S, Yaari R, Yamana TK, Shaman J, Agarwal P, Balusu S, Gururajan G, Kamarthi H, Prakash BA, Raman R, Zhao Z, Rodríguez A, Meiyappan A, Omar S, Baccam P, Gurung HL, Suchoski BT, Stage SA, Ajelli M, Kummer AG, Litvinova M, Ventura PC, Wadsworth S, Niemi J, Carcelen E, Hill AL, Loo SL, McKee CD, Sato K, Smith C, Truelove S, Jung SM, Lemaitre JC, Lessler J, McAndrew T, Ye W, Bosse N, Hlavacek WS, Lin YT, Mallela A, Gibson GC, Chen Y, Lamm SM, Lee J, Posner RG, Perofsky AC, Viboud C, Clemente L, Lu F, Meyer AG, Santillana M, Chinazzi M, Davis JT, Mu K, Pastore Y Piontti A, Vespignani A, Xiong X, Ben-Nun M, Riley P, Turtle J, Hulme-Lowe C, Jessa S, Nagraj VP, Turner SD, Williams D, Basu A, Drake JM, Fox SJ, Suez E, Cojocaru MG, Thommes EW, Cramer EY, Gerding A, Stark A, Ray EL, Reich NG, Shandross L, Wattanachit N, Wang Y, Zorn MW, Aawar MA, Srivastava A, Meyers LA, Adiga A, Hurt B, Kaur G, Lewis BL, Marathe M, Venkatramanan S, Butler P, Farabow A, Ramakrishnan N, Muralidhar N, Reed C, Biggerstaff M, Borchering RK. Title evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations. Nat Commun 2024; 15:6289. [PMID: 39060259 PMCID: PMC11282251 DOI: 10.1038/s41467-024-50601-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 07/16/2024] [Indexed: 07/28/2024] Open
Abstract
Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021-22 and 2022-23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.
Collapse
Affiliation(s)
| | | | - Tomás M León
- California Department of Public Health, Richmond, CA, USA
| | - Erin L Murray
- California Department of Public Health, Richmond, CA, USA
| | - Monica Sun
- California Department of Public Health, Richmond, CA, USA
| | - Lauren A White
- California Department of Public Health, Richmond, CA, USA
| | - Logan C Brooks
- Carnegie Mellon University, Pittsburgh, PA, USA
- University of California, Berkeley, Berkeley, CA, USA
| | - Alden Green
- Carnegie Mellon University, Pittsburgh, PA, USA
| | | | | | | | - Ryan J Tibshirani
- Carnegie Mellon University, Pittsburgh, PA, USA
- University of California, Berkeley, Berkeley, CA, USA
| | | | | | - Sen Pei
- Columbia University, New York, NY, USA
| | | | | | - Jeffrey Shaman
- Columbia University, New York, NY, USA
- Columbia University School of Climate, New York, NY, USA
| | | | | | | | | | | | - Rishi Raman
- Georgia Institute of Technology, Atlanta, GA, USA
| | - Zhiyuan Zhao
- Georgia Institute of Technology, Atlanta, GA, USA
| | | | | | - Shalina Omar
- Guidehouse Advisory and Consulting Services, McClean, VA, USA
| | | | | | | | | | - Marco Ajelli
- Indiana University School of Public Health, Bloomington, IN, USA
| | | | - Maria Litvinova
- Indiana University School of Public Health, Bloomington, IN, USA
| | - Paulo C Ventura
- Indiana University School of Public Health, Bloomington, IN, USA
| | | | | | | | | | - Sara L Loo
- Johns Hopkins University, Baltimore, MD, USA
| | | | - Koji Sato
- Johns Hopkins University, Baltimore, MD, USA
| | | | | | - Sung-Mok Jung
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | - Justin Lessler
- University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | | | - Nikos Bosse
- London School of Health and Tropical Medicine, London, UK
| | | | - Yen Ting Lin
- Los Alamos National Laboratory, Los Alamos, NM, USA
| | | | | | - Ye Chen
- Northern Arizona University, Flagstaff, AZ, USA
| | | | | | | | - Amanda C Perofsky
- Fogarty International Center, National Institutes of Health, Bethesda, MD, USA
| | - Cécile Viboud
- Fogarty International Center, National Institutes of Health, Bethesda, MD, USA
| | | | - Fred Lu
- Northeastern University, Boston, MA, USA
| | | | | | | | | | - Kunpeng Mu
- Northeastern University, Boston, MA, USA
| | | | | | | | | | - Pete Riley
- Predictive Science Inc, San Diego, CA, USA
| | | | | | | | - V P Nagraj
- Signature Science, LLC, Charlottesville, VA, USA
| | | | | | | | | | | | | | | | - Edward W Thommes
- University of Guelph, Guelph, ON, Canada
- Sanofi, Toronto, ON, USA
| | | | - Aaron Gerding
- University of Massachusetts Amherst, Amherst, MA, USA
| | - Ariane Stark
- University of Massachusetts Amherst, Amherst, MA, USA
| | - Evan L Ray
- University of Massachusetts Amherst, Amherst, MA, USA
| | | | - Li Shandross
- University of Massachusetts Amherst, Amherst, MA, USA
| | | | - Yijin Wang
- University of Massachusetts Amherst, Amherst, MA, USA
| | - Martha W Zorn
- University of Massachusetts Amherst, Amherst, MA, USA
| | - Majd Al Aawar
- University of Southern California, Los Angeles, CA, USA
| | | | | | | | | | | | | | | | | | | | | | | | | | - Carrie Reed
- Centers for Disease Control and Prevention, Atlanta, GA, USA
| | | | | |
Collapse
|
4
|
Brosky H, Prasek SM, Innes GK, Pepper IL, Miranda J, Brierley PE, Slinski SL, Polashenski L, Betancourt WQ, Gronbach K, Gomez D, Neupane R, Johnson J, Weiss J, Yaglom HD, Engelthaler DM, Hepp CM, Crank K, Gerrity D, Stewart JR, Schmitz BW. A framework for integrating wastewater-based epidemiology and public health. Front Public Health 2024; 12:1418681. [PMID: 39131575 PMCID: PMC11312382 DOI: 10.3389/fpubh.2024.1418681] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Accepted: 06/24/2024] [Indexed: 08/13/2024] Open
Abstract
Wastewater-based epidemiology (WBE) is an environmental approach to monitor community health through the analysis of sewage. The COVID-19 pandemic catalyzed scientists and public health professionals to revisit WBE as a tool to optimize resource allocation to mitigate disease spread and prevent outbreaks. Some studies have highlighted the value of WBE programs that coordinate with public health professionals; however, the details necessary for implementation are not well-characterized. To respond to this knowledge gap, this article documents the framework of a successful WBE program in Arizona, titled Wastewater Analysis for Tactical Epidemiological Response Systems (WATERS), detailing the developed structure and methods of communication that enabled public health preparedness and response actions. This communication illustrates how program operations were employed to reduce outbreak severity. The structure outlined here is customizable and may guide other programs in the implementation of WBE as a public health tool.
Collapse
Affiliation(s)
- Hanna Brosky
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
- Environmental Sciences and Engineering, University of North Carolina, Chapel Hill, NC, United States
| | - Sarah M. Prasek
- Water and Energy Sustainable Technology (WEST) Center, University of Arizona, Tucson, AZ, United States
| | - Gabriel K. Innes
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| | - Ian L. Pepper
- Water and Energy Sustainable Technology (WEST) Center, University of Arizona, Tucson, AZ, United States
| | - Jasmine Miranda
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| | - Paul E. Brierley
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| | - Stephanie L. Slinski
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| | - Lois Polashenski
- Water and Energy Sustainable Technology (WEST) Center, University of Arizona, Tucson, AZ, United States
| | - Walter Q. Betancourt
- Water and Energy Sustainable Technology (WEST) Center, University of Arizona, Tucson, AZ, United States
| | - Katie Gronbach
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| | - Diana Gomez
- Yuma County Public Health Services District, Yuma, AZ, United States
| | - Reshma Neupane
- Arizona Department of Health Services, Office of Infectious Disease Services, Phoenix, AZ, United States
| | - Jasmine Johnson
- Arizona Department of Health Services, Office of Infectious Disease Services, Phoenix, AZ, United States
| | - Joli Weiss
- Arizona Department of Health Services, Office of Infectious Disease Services, Phoenix, AZ, United States
| | - Hayley D. Yaglom
- Translational Genomics Research Institute, Pathogen and Microbiome Institute, Flagstaff, AZ, United States
| | - David M. Engelthaler
- Translational Genomics Research Institute, Pathogen and Microbiome Institute, Flagstaff, AZ, United States
| | - Crystal M. Hepp
- Translational Genomics Research Institute, Pathogen and Microbiome Institute, Flagstaff, AZ, United States
| | - Katherine Crank
- Applied Research and Development Center, Southern Nevada Water Authority, Las Vegas, NV, United States
| | - Daniel Gerrity
- Applied Research and Development Center, Southern Nevada Water Authority, Las Vegas, NV, United States
| | - Jill R. Stewart
- Environmental Sciences and Engineering, University of North Carolina, Chapel Hill, NC, United States
| | - Bradley W. Schmitz
- Yuma Center of Excellence for Desert Agriculture (YCEDA), University of Arizona, Tucson, AZ, United States
| |
Collapse
|
5
|
Lipsitch M, Bassett MT, Brownstein JS, Elliott P, Eyre D, Grabowski MK, Hay JA, Johansson MA, Kissler SM, Larremore DB, Layden JE, Lessler J, Lynfield R, MacCannell D, Madoff LC, Metcalf CJE, Meyers LA, Ofori SK, Quinn C, Bento AI, Reich NG, Riley S, Rosenfeld R, Samore MH, Sampath R, Slayton RB, Swerdlow DL, Truelove S, Varma JK, Grad YH. Infectious disease surveillance needs for the United States: lessons from Covid-19. Front Public Health 2024; 12:1408193. [PMID: 39076420 PMCID: PMC11285106 DOI: 10.3389/fpubh.2024.1408193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2024] [Accepted: 06/18/2024] [Indexed: 07/31/2024] Open
Abstract
The COVID-19 pandemic has highlighted the need to upgrade systems for infectious disease surveillance and forecasting and modeling of the spread of infection, both of which inform evidence-based public health guidance and policies. Here, we discuss requirements for an effective surveillance system to support decision making during a pandemic, drawing on the lessons of COVID-19 in the U.S., while looking to jurisdictions in the U.S. and beyond to learn lessons about the value of specific data types. In this report, we define the range of decisions for which surveillance data are required, the data elements needed to inform these decisions and to calibrate inputs and outputs of transmission-dynamic models, and the types of data needed to inform decisions by state, territorial, local, and tribal health authorities. We define actions needed to ensure that such data will be available and consider the contribution of such efforts to improving health equity.
Collapse
Affiliation(s)
- Marc Lipsitch
- Center for Forecasting and Outbreak Analytics, US Centers for Disease Control and Prevention, Atlanta, GA, United States
- Center for Communicable Disease Dynamics, Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, MA, United States
| | - Mary T. Bassett
- François-Xavier Bagnoud Center for Health and Human Rights, Department of Social and Behavioral Sciences, Harvard T. H. Chan School of Public Health, Boston, MA, United States
| | - John S. Brownstein
- Boston Children’s Hospital, Harvard Medical School, Boston, MA, United States
| | - Paul Elliott
- Department of Epidemiology and Public Health Medicine, Imperial College London, London, United Kingdom
| | - David Eyre
- Big Data Institute, Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom
| | - M. Kate Grabowski
- Department of Epidemiology, Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, United States
| | - James A. Hay
- Big Data Institute, Nuffield Department of Population Health, University of Oxford, Oxford, United Kingdom
| | - Michael A. Johansson
- Division of Vector-Borne Diseases, US Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Stephen M. Kissler
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
| | - Daniel B. Larremore
- Department of Computer Science, University of Colorado Boulder, Boulder, CO, United States
- BioFrontiers Institute, University of Colorado Boulder, Boulder, CO, United States
| | - Jennifer E. Layden
- Office of Public Health Data, Surveillance, and Technology, US Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Justin Lessler
- Department of Epidemiology, UNC Gillings School of Public Health, Chapel Hill, NC, United States
| | - Ruth Lynfield
- Minnesota Department of Health, Minneapolis, MN, United States
| | - Duncan MacCannell
- US Centers for Disease Control and Prevention, Office of Advanced Molecular Detection, Atlanta, GA, United States
| | | | - C. Jessica E. Metcalf
- Department of Ecology and Evolutionary Biology, Princeton University, Princeton, NJ, United States
| | - Lauren A. Meyers
- Department of Integrative Biology, University of Texas at Austin, Austin, TX, United States
| | - Sylvia K. Ofori
- Center for Communicable Disease Dynamics, Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, MA, United States
| | - Celia Quinn
- Division of Disease Control, New York City Department of Health and Mental Hygiene, New York City, NY, United States
| | - Ana I. Bento
- Department of Public and Ecosystem Health, College of Veterinary Medicine, Cornell University, Ithaca, NY, United States
| | - Nicholas G. Reich
- Departments of Biostatistics and Epidemiology, University of Massachusetts Amherst, Amherst, MA, United States
| | - Steven Riley
- United Kingdom Health Security Agency, London, United Kingdom
| | - Roni Rosenfeld
- Departments of Computer Science and Computational Biology, Carnegie Melon University, Pittsburgh, PA, United States
| | - Matthew H. Samore
- Division of Epidemiology, Department of Medicine, University of Utah, Salt Lake City, UT, United States
| | | | - Rachel B. Slayton
- Division of Healthcare Quality Promotion, US Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - David L. Swerdlow
- Center for Communicable Disease Dynamics, Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, MA, United States
| | - Shaun Truelove
- Department of Epidemiology, UNC Gillings School of Public Health, Chapel Hill, NC, United States
| | - Jay K. Varma
- SIGA Technologies, New York City, NY, United States
| | - Yonatan H. Grad
- Department of Immunology and Infectious Diseases, Harvard T. H. Chan School of Public Health, Boston, MA, United States
| |
Collapse
|
6
|
Sudhakar T, Bhansali A, Walkington J, Puelz D. The disutility of compartmental model forecasts during the COVID-19 pandemic. FRONTIERS IN EPIDEMIOLOGY 2024; 4:1389617. [PMID: 38966521 PMCID: PMC11222405 DOI: 10.3389/fepid.2024.1389617] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Accepted: 05/27/2024] [Indexed: 07/06/2024]
Abstract
During the COVID-19 pandemic, several forecasting models were released to predict the spread of the virus along variables vital for public health policymaking. Of these, the susceptible-infected-recovered (SIR) compartmental model was the most common. In this paper, we investigated the forecasting performance of The University of Texas COVID-19 Modeling Consortium SIR model. We considered the following daily outcomes: hospitalizations, ICU patients, and deaths. We evaluated the overall forecasting performance, highlighted some stark forecast biases, and considered forecast errors conditional on different pandemic regimes. We found that this model tends to overforecast over the longer horizons and when there is a surge in viral spread. We bolstered these findings by linking them to faults with the SIR framework itself.
Collapse
Affiliation(s)
| | | | | | - David Puelz
- Salem Center for Policy, Department of Finance, & Department of Information, Risk, and Operations Management, Austin, TX, United States
| |
Collapse
|
7
|
Porebski P, Venkatramanan S, Adiga A, Klahn B, Hurt B, Wilson ML, Chen J, Vullikanti A, Marathe M, Lewis B. Data-driven mechanistic framework with stratified immunity and effective transmissibility for COVID-19 scenario projections. Epidemics 2024; 47:100761. [PMID: 38555667 PMCID: PMC11205267 DOI: 10.1016/j.epidem.2024.100761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 01/30/2024] [Accepted: 03/07/2024] [Indexed: 04/02/2024] Open
Abstract
Scenario-based modeling frameworks have been widely used to support policy-making at state and federal levels in the United States during the COVID-19 response. While custom-built models can be used to support one-off studies, sustained updates to projections under changing pandemic conditions requires a robust, integrated, and adaptive framework. In this paper, we describe one such framework, UVA-adaptive, that was built to support the CDC-aligned Scenario Modeling Hub (SMH) across multiple rounds, as well as weekly/biweekly projections to Virginia Department of Health (VDH) and US Department of Defense during the COVID-19 response. Building upon an existing metapopulation framework, PatchSim, UVA-adaptive uses a calibration mechanism relying on adjustable effective transmissibility as a basis for scenario definition while also incorporating real-time datasets on case incidence, seroprevalence, variant characteristics, and vaccine uptake. Through the pandemic, our framework evolved by incorporating available data sources and was extended to capture complexities of multiple strains and heterogeneous immunity of the population. Here we present the version of the model that was used for the recent projections for SMH and VDH, describe the calibration and projection framework, and demonstrate that the calibrated transmissibility correlates with the evolution of the pathogen as well as associated societal dynamics.
Collapse
Affiliation(s)
- Przemyslaw Porebski
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA.
| | | | - Aniruddha Adiga
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| | - Brian Klahn
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| | - Benjamin Hurt
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| | - Mandy L Wilson
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| | - Jiangzhuo Chen
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| | - Anil Vullikanti
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA; Department of Computer Science, University of Virginia, Charlottesville, 22904, VA, USA
| | - Madhav Marathe
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA; Department of Computer Science, University of Virginia, Charlottesville, 22904, VA, USA
| | - Bryan Lewis
- Biocomplexity Institute & Initiative, University of Virginia, Charlottesville, 22911, VA, USA
| |
Collapse
|
8
|
Pasco R, Fox SJ, Lachmann M, Meyers LA. Effectiveness of interventions to reduce COVID-19 transmission in schools. Epidemics 2024; 47:100762. [PMID: 38489849 DOI: 10.1016/j.epidem.2024.100762] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 03/05/2024] [Accepted: 03/08/2024] [Indexed: 03/17/2024] Open
Abstract
School reopenings in 2021 and 2022 coincided with the rapid emergence of new SARS-CoV-2 variants in the United States. In-school mitigation efforts varied, depending on local COVID-19 mandates and resources. Using a stochastic age-stratified agent-based model of SARS-CoV-2 transmission, we estimate the impacts of multiple in-school strategies on both infection rates and absenteeism, relative to a baseline scenario in which only symptomatic cases are tested and positive tests trigger a 10-day isolation of the case and 10-day quarantine of their household and classroom. We find that monthly asymptomatic screening coupled with the 10-day isolation and quarantine period is expected to avert 55.4% of infections while increasing absenteeism by 104.3%. Replacing quarantine with test-to-stay would reduce absenteeism by 66.3% (while hardly impacting infection rates), but would require roughly 10-fold more testing resources. Alternatively, vaccination or mask wearing by 50% of the student body is expected to avert 54.1% or 43.1% of infections while decreasing absenteeism by 34.1% or 27.4%, respectively. Separating students into classrooms based on mask usage is expected to reduce infection risks among those who wear masks (by 23.1%), exacerbate risks among those who do not (by 27.8%), but have little impact on overall risk. A combined strategy of monthly screening, household and classroom quarantine, a 50% vaccination rate, and a 50% masking rate (in mixed classrooms) is expected to avert 81.7% of infections while increasing absenteeism by 90.6%. During future public health emergencies, such analyses can inform the rapid design of resource-constrained strategies that mitigate both public health and educational risks.
Collapse
Affiliation(s)
- Remy Pasco
- Integrative Biology, The University of Texas at Austin, Austin, TX,USA
| | - Spencer J Fox
- Department of Epidemiology & Biostatistics, University of Georgia, Athens, GA, USA
| | - Michael Lachmann
- Department of Epidemiology & Biostatistics, University of Georgia, Athens, GA, USA
| | - Lauren Ancel Meyers
- Integrative Biology, The University of Texas at Austin, Austin, TX,USA; Santa Fe Institute, Santa Fe, NM, USA.
| |
Collapse
|
9
|
Borchering RK, Biggerstaff M, Brammer L, Budd A, Garg S, Fry AM, Iuliano AD, Reed C. Responding to the Return of Influenza in the United States by Applying Centers for Disease Control and Prevention Surveillance, Analysis, and Modeling to Inform Understanding of Seasonal Influenza. JMIR Public Health Surveill 2024; 10:e54340. [PMID: 38587882 PMCID: PMC11036179 DOI: 10.2196/54340] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 02/12/2024] [Accepted: 02/13/2024] [Indexed: 04/09/2024] Open
Abstract
We reviewed the tools that have been developed to characterize and communicate seasonal influenza activity in the United States. Here we focus on systematic surveillance and applied analytics, including seasonal burden and disease severity estimation, short-term forecasting, and longer-term modeling efforts. For each set of activities, we describe the challenges and opportunities that have arisen because of the COVID-19 pandemic. In conclusion, we highlight how collaboration and communication have been and will continue to be key components of reliable and actionable influenza monitoring, forecasting, and modeling activities.
Collapse
Affiliation(s)
- Rebecca K Borchering
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Matthew Biggerstaff
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Lynnette Brammer
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Alicia Budd
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Shikha Garg
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Alicia M Fry
- Fulton County Board of Health, Atlanta, GA, United States
| | - A Danielle Iuliano
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| | - Carrie Reed
- National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, Atlanta, GA, United States
| |
Collapse
|
10
|
Mathis SM, Webber AE, León TM, Murray EL, Sun M, White LA, Brooks LC, Green A, Hu AJ, McDonald DJ, Rosenfeld R, Shemetov D, Tibshirani RJ, Kandula S, Pei S, Shaman J, Yaari R, Yamana TK, Agarwal P, Balusu S, Gururajan G, Kamarthi H, Prakash BA, Raman R, Rodríguez A, Zhao Z, Meiyappan A, Omar S, Baccam P, Gurung HL, Stage SA, Suchoski BT, Ajelli M, Kummer AG, Litvinova M, Ventura PC, Wadsworth S, Niemi J, Carcelen E, Hill AL, Jung SM, Lemaitre JC, Lessler J, Loo SL, McKee CD, Sato K, Smith C, Truelove S, McAndrew T, Ye W, Bosse N, Hlavacek WS, Lin YT, Mallela A, Chen Y, Lamm SM, Lee J, Posner RG, Perofsky AC, Viboud C, Clemente L, Lu F, Meyer AG, Santillana M, Chinazzi M, Davis JT, Mu K, Piontti APY, Vespignani A, Xiong X, Ben-Nun M, Riley P, Turtle J, Hulme-Lowe C, Jessa S, Nagraj VP, Turner SD, Williams D, Basu A, Drake JM, Fox SJ, Gibson GC, Suez E, Thommes EW, Cojocaru MG, Cramer EY, Gerding A, Stark A, Ray EL, Reich NG, Shandross L, Wattanachit N, Wang Y, Zorn MW, Al Aawar M, Srivastava A, Meyers LA, Adiga A, Hurt B, Kaur G, Lewis BL, Marathe M, Venkatramanan S, Butler P, Farabow A, Muralidhar N, Ramakrishnan N, Reed C, Biggerstaff M, Borchering RK. Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.12.08.23299726. [PMID: 38168429 PMCID: PMC10760285 DOI: 10.1101/2023.12.08.23299726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one through four weeks ahead for the 2021-22 and 2022-23 influenza seasons. Across both seasons, 26 teams submitted forecasts, with the submitting teams varying between seasons. Forecast skill was evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperformed the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble was the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degraded over longer forecast horizons and during periods of rapid change. Current influenza forecasting efforts help inform situational awareness, but research is needed to address limitations, including decreased performance during periods of changing epidemic dynamics.
Collapse
Affiliation(s)
- Sarabeth M Mathis
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | - Alexander E Webber
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | - Tomás M León
- California Department of Public Health, Richmond, CA, 95899
| | - Erin L Murray
- California Department of Public Health, Richmond, CA, 95899
| | - Monica Sun
- California Department of Public Health, Richmond, CA, 95899
| | - Lauren A White
- California Department of Public Health, Richmond, CA, 95899
| | - Logan C Brooks
- Carnegie Mellon University, Pittsburgh, PA, 15213
- University of California, Berkeley, Berkeley, CA 94720
| | - Alden Green
- Carnegie Mellon University, Pittsburgh, PA, 15213
| | - Addison J Hu
- Carnegie Mellon University, Pittsburgh, PA, 15213
| | | | | | | | - Ryan J Tibshirani
- Carnegie Mellon University, Pittsburgh, PA, 15213
- University of California, Berkeley, Berkeley, CA 94720
| | | | - Sen Pei
- Columbia University, New York, NY, 10032
| | - Jeffrey Shaman
- Columbia University, New York, NY, 10032
- Columbia University School of Climate, New York, NY 10025
| | - Rami Yaari
- Columbia University, New York, NY, 10032
| | | | | | | | | | | | | | - Rishi Raman
- Georgia Institute of Technology, Atlanta, GA, 30318
| | | | - Zhiyuan Zhao
- Georgia Institute of Technology, Atlanta, GA, 30318
| | | | - Shalina Omar
- Guidehouse Advisory and Consulting Services, McClean VA, 22102
| | | | | | | | | | - Marco Ajelli
- Indiana University School of Public Health, Bloomington, IN, 47405
| | | | - Maria Litvinova
- Indiana University School of Public Health, Bloomington, IN, 47405
| | - Paulo C Ventura
- Indiana University School of Public Health, Bloomington, IN, 47405
| | | | | | | | | | - Sung-Mok Jung
- University of North Carolina at Chapel Hill, Chapel Hill, NC
| | | | - Justin Lessler
- University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Sara L Loo
- Johns Hopkins University, Baltimore, MD, 21205
| | | | - Koji Sato
- Johns Hopkins University, Baltimore, MD, 21205
| | | | | | | | | | - Nikos Bosse
- London School of Health and Tropical Medicine, London, UK, WC1E 7HT
| | | | - Yen Ting Lin
- Los Alamos National Laboratory, Los Alamos, NM, 87545
| | | | - Ye Chen
- Northern Arizona University, Flagstaff, AZ, 86011
| | | | - Jaechoul Lee
- Northern Arizona University, Flagstaff, AZ, 86011
| | | | - Amanda C Perofsky
- Fogarty International Center, National Institutes of Health, Bethesda, MD, 20892
| | - Cécile Viboud
- Fogarty International Center, National Institutes of Health, Bethesda, MD, 20892
| | | | - Fred Lu
- Northeastern University, Boston, MA, 02115
| | | | | | | | | | - Kunpeng Mu
- Northeastern University, Boston, MA, 02115
| | | | | | | | | | - Pete Riley
- Predictive Science Inc, San Diego, CA 92121
| | | | | | | | - V P Nagraj
- Signature Science, LLC, Charlottesville, VA, 22911
| | | | | | | | | | | | | | - Ehsan Suez
- University of Georgia, Athens, GA, 30609
| | - Edward W Thommes
- University of Guelph, Guelph, ON N1G 2W1, Canada
- Sanofi, Toronto, ON, M2R 3T4
| | | | | | - Aaron Gerding
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Ariane Stark
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Evan L Ray
- University of Massachusetts Amherst, Amherst, MA, 01003
| | | | - Li Shandross
- University of Massachusetts Amherst, Amherst, MA, 01003
| | | | - Yijin Wang
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Martha W Zorn
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Majd Al Aawar
- University of Southern California, Los Angeles, CA, 90089
| | | | | | | | | | | | | | | | | | | | | | | | | | - Carrie Reed
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | | | | |
Collapse
|
11
|
Reich NG, Wang Y, Burns M, Ergas R, Cramer EY, Ray EL. Assessing the utility of COVID-19 case reports as a leading indicator for hospitalization forecasting in the United States. Epidemics 2023; 45:100728. [PMID: 37976681 PMCID: PMC10871058 DOI: 10.1016/j.epidem.2023.100728] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 09/29/2023] [Accepted: 11/06/2023] [Indexed: 11/19/2023] Open
Abstract
Identifying data streams that can consistently improve the accuracy of epidemiological forecasting models is challenging. Using models designed to predict daily state-level hospital admissions due to COVID-19 in California and Massachusetts, we investigated whether incorporating COVID-19 case data systematically improved forecast accuracy. Additionally, we considered whether using case data aggregated by date of test or by date of report from a surveillance system made a difference to the forecast accuracy. Evaluating forecast accuracy in a test period, after first having selected the best-performing methods in a validation period, we found that overall the difference in accuracy between approaches was small, especially at forecast horizons of less than two weeks. However, forecasts from models using cases aggregated by test date showed lower accuracy at longer horizons and at key moments in the pandemic, such as the peak of the Omicron wave in January 2022. Overall, these results highlight the challenge of finding a modeling approach that can generate accurate forecasts of outbreak trends both during periods of relative stability and during periods that show rapid growth or decay of transmission rates. While COVID-19 case counts seem to be a natural choice to help predict COVID-19 hospitalizations, in practice any benefits we observed were small and inconsistent.
Collapse
Affiliation(s)
- Nicholas G Reich
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA, United States of America.
| | - Yijin Wang
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA, United States of America
| | - Meagan Burns
- Massachusetts Department of Public Health, Boston, MA, United States of America
| | - Rosa Ergas
- Massachusetts Department of Public Health, Boston, MA, United States of America
| | - Estee Y Cramer
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA, United States of America
| | - Evan L Ray
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA, United States of America
| |
Collapse
|
12
|
Drake JM, Handel A, Marty É, O’Dea EB, O’Sullivan T, Righi G, Tredennick AT. A data-driven semi-parametric model of SARS-CoV-2 transmission in the United States. PLoS Comput Biol 2023; 19:e1011610. [PMID: 37939201 PMCID: PMC10659176 DOI: 10.1371/journal.pcbi.1011610] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2023] [Revised: 11/20/2023] [Accepted: 10/17/2023] [Indexed: 11/10/2023] Open
Abstract
To support decision-making and policy for managing epidemics of emerging pathogens, we present a model for inference and scenario analysis of SARS-CoV-2 transmission in the USA. The stochastic SEIR-type model includes compartments for latent, asymptomatic, detected and undetected symptomatic individuals, and hospitalized cases, and features realistic interval distributions for presymptomatic and symptomatic periods, time varying rates of case detection, diagnosis, and mortality. The model accounts for the effects on transmission of human mobility using anonymized mobility data collected from cellular devices, and of difficult to quantify environmental and behavioral factors using a latent process. The baseline transmission rate is the product of a human mobility metric obtained from data and this fitted latent process. We fit the model to incident case and death reports for each state in the USA and Washington D.C., using likelihood Maximization by Iterated particle Filtering (MIF). Observations (daily case and death reports) are modeled as arising from a negative binomial reporting process. We estimate time-varying transmission rate, parameters of a sigmoidal time-varying fraction of hospitalized cases that result in death, extra-demographic process noise, two dispersion parameters of the observation process, and the initial sizes of the latent, asymptomatic, and symptomatic classes. In a retrospective analysis covering March-December 2020, we show how mobility and transmission strength became decoupled across two distinct phases of the pandemic. The decoupling demonstrates the need for flexible, semi-parametric approaches for modeling infectious disease dynamics in real-time.
Collapse
Affiliation(s)
- John M. Drake
- Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
| | - Andreas Handel
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
- College of Public Health, University of Georgia, Athens, Georgia, United States of America
| | - Éric Marty
- Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
| | - Eamon B. O’Dea
- Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
| | - Tierney O’Sullivan
- Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
| | - Giovanni Righi
- Odum School of Ecology, University of Georgia, Athens, Georgia, United States of America
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
| | - Andrew T. Tredennick
- Center for the Ecology of Infectious Diseases, University of Georgia, Athens, Georgia, United States of America
- Western EcoSystems Technology, Inc., Laramie, Wyoming, United States of America
| |
Collapse
|
13
|
Sharmin M, Manivannan M, Woo D, Sorel O, Auclair JR, Gandhi M, Mujawar I. Cross-sectional Ct distributions from qPCR tests can provide an early warning signal for the spread of COVID-19 in communities. Front Public Health 2023; 11:1185720. [PMID: 37841738 PMCID: PMC10570742 DOI: 10.3389/fpubh.2023.1185720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2023] [Accepted: 09/07/2023] [Indexed: 10/17/2023] Open
Abstract
Background SARS-CoV-2 PCR testing data has been widely used for COVID-19 surveillance. Existing COVID-19 forecasting models mainly rely on case counts obtained from qPCR results, even though the binary PCR results provide a limited picture of the pandemic trajectory. Most forecasting models have failed to accurately predict the COVID-19 waves before they occur. Recently a model utilizing cross-sectional population cycle threshold (Ct-the number of cycles required for the fluorescent signal to cross the background threshold) values obtained from PCR tests (Ct-based model) was developed to overcome the limitations of using only binary PCR results. In this study, we aimed to improve on COVID-19 forecasting models using features derived from the Ct-based model, to detect epidemic waves earlier than case-based trajectories. Methods PCR data was collected weekly at Northeastern University (NU) between August 2020 and January 2022. Campus and county epidemic trajectories were generated from case counts. A novel forecasting approach was developed by enhancing a recent deep learning model with Ct-based features and applied in Suffolk County and NU campus. For this, cross-sectional Ct values from PCR data were used to generate Ct-based epidemic trajectories, including effective reproductive rate (Rt) and incidence. The improvement in forecasting performance was compared using absolute errors and residual squared errors with respect to actual observed cases at the 7-day and 14-day forecasting horizons. The model was also tested prospectively over the period January 2022 to April 2022. Results Rt curves estimated from the Ct-based model indicated epidemic waves 12 to 14 days earlier than Rt curves from NU campus and Suffolk County cases, with a correlation of 0.57. Enhancing the forecasting models with Ct-based information significantly decreased absolute error (decrease of 49.4 and 221.5 for the 7 and 14-day forecasting horizons) and residual squared error (40.6 and 217.1 for the 7 and 14-day forecasting horizons) compared to the original model without Ct features. Conclusion Ct-based epidemic trajectories can herald an earlier signal for impending epidemic waves in the community and forecast transmission peaks. Moreover, COVID-19 forecasting models can be enhanced using these Ct features to improve their forecasting accuracy. In this study, we make the case that public health agencies should publish Ct values along with the binary positive/negative PCR results. Early and accurate forecasting of epidemic waves can inform public health policies and countermeasures which can mitigate spread.
Collapse
Affiliation(s)
- Mahfuza Sharmin
- Thermo Fisher Scientific, South San Francisco, CA, United States
| | - Mani Manivannan
- Thermo Fisher Scientific, South San Francisco, CA, United States
| | - David Woo
- Thermo Fisher Scientific, South San Francisco, CA, United States
| | - Océane Sorel
- Thermo Fisher Scientific, South San Francisco, CA, United States
| | - Jared R. Auclair
- Department of Chemistry and Chemical Biology, Northeastern University, Burlington, MA, United States
| | - Manoj Gandhi
- Thermo Fisher Scientific, South San Francisco, CA, United States
| | - Imran Mujawar
- Thermo Fisher Scientific, South San Francisco, CA, United States
| |
Collapse
|
14
|
Chu AM, Chong ACY, Lai NHT, Tiwari A, So MKP. Enhancing the Predictive Power of Google Trends Data Through Network Analysis: Infodemiology Study of COVID-19. JMIR Public Health Surveill 2023; 9:e42446. [PMID: 37676701 PMCID: PMC10488898 DOI: 10.2196/42446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2022] [Revised: 06/01/2023] [Accepted: 06/29/2023] [Indexed: 09/08/2023] Open
Abstract
BACKGROUND The COVID-19 outbreak has revealed a high demand for timely surveillance of pandemic developments. Google Trends (GT), which provides freely available search volume data, has been proven to be a reliable forecast and nowcast measure for public health issues. Previous studies have tended to use relative search volumes from GT directly to analyze associations and predict the progression of pandemic. However, GT's normalization of the search volumes data and data retrieval restrictions affect the data resolution in reflecting the actual search behaviors, thus limiting the potential for using GT data to predict disease outbreaks. OBJECTIVE This study aimed to introduce a merged algorithm that helps recover the resolution and accuracy of the search volume data extracted from GT over long observation periods. In addition, this study also aimed to demonstrate the extended application of merged search volumes (MSVs) in combination of network analysis, via tracking the COVID-19 pandemic risk. METHODS We collected relative search volumes from GT and transformed them into MSVs using our proposed merged algorithm. The MSVs of the selected coronavirus-related keywords were compiled using the rolling window method. The correlations between the MSVs were calculated to form a dynamic network. The network statistics, including network density and the global clustering coefficients between the MSVs, were also calculated. RESULTS Our research findings suggested that although GT restricts the search data retrieval into weekly data points over a long period, our proposed approach could recover the daily search volume over the same investigation period to facilitate subsequent research analyses. In addition, the dynamic time warping diagrams show that the dynamic networks were capable of predicting the COVID-19 pandemic trends, in terms of the number of COVID-19 confirmed cases and severity risk scores. CONCLUSIONS The innovative method for handling GT search data and the application of MSVs and network analysis to broaden the potential for GT data are useful for predicting the pandemic risk. Further investigation of the GT dynamic network can focus on noncommunicable diseases, health-related behaviors, and misinformation on the internet.
Collapse
Affiliation(s)
- Amanda My Chu
- Department of Social Sciences and Policy Studies, The Education University of Hong Kong, Hong Kong, Hong Kong
| | - Andy C Y Chong
- School of Nursing, Tung Wah College, Hong Kong, Hong Kong
| | - Nick H T Lai
- Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
| | - Agnes Tiwari
- School of Nursing, Hong Kong Sanatorium & Hospital, Hong Kong, Hong Kong
- Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, Hong Kong
| | - Mike K P So
- Department of Information Systems, Business Statistics and Operations Management, The Hong Kong University of Science and Technology, Hong Kong, Hong Kong
| |
Collapse
|
15
|
Mellor J, Overton CE, Fyles M, Chawner L, Baxter J, Baird T, Ward T. Understanding the leading indicators of hospital admissions from COVID-19 across successive waves in the UK. Epidemiol Infect 2023; 151:e172. [PMID: 37664991 PMCID: PMC10600913 DOI: 10.1017/s0950268823001449] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 07/20/2023] [Accepted: 07/25/2023] [Indexed: 09/05/2023] Open
Abstract
Following the end of universal testing in the UK, hospital admissions are a key measure of COVID-19 pandemic pressure. Understanding leading indicators of admissions at the National Health Service (NHS) Trust, regional and national geographies help health services plan for ongoing pressures. We explored the spatio-temporal relationships of leading indicators of hospitalisations across SARS-CoV-2 waves in England. This analysis includes an evaluation of internet search volumes from Google Trends, NHS triage calls and online queries, the NHS COVID-19 app, lateral flow devices (LFDs), and the ZOE app. Data sources were analysed for their feasibility as leading indicators using Granger causality, cross-correlation, and dynamic time warping at fine spatial scales. Google Trends and NHS triages consistently temporally led admissions in most locations, with lead times ranging from 5 to 20 days, whereas an inconsistent relationship was found for the ZOE app, NHS COVID-19 app, and LFD testing, which diminished with spatial resolution, showing cross-correlation of leads between -7 and 7 days. The results indicate that novel surveillance sources can be used effectively to understand the expected healthcare burden within hospital administrative areas though the temporal and spatial heterogeneity of these relationships is a key determinant of their operational public health utility.
Collapse
Affiliation(s)
- Jonathon Mellor
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
| | - Christopher E Overton
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
- Department of Mathematical Sciences, University of Liverpool, Liverpool, UK
- Department of Mathematics, University of Manchester, Manchester, UK
| | - Martyn Fyles
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
- Department of Mathematics, University of Manchester, Manchester, UK
| | - Liam Chawner
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
| | - James Baxter
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
| | - Tarrion Baird
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
- Department of Pathology, University of Cambridge, Cambridge, UK
| | - Thomas Ward
- UK Health Security Agency, Data, Analytics and Surveillance, Nobel House, London, UK
| |
Collapse
|
16
|
Bilinski AM, Salomon JA, Hatfield LA. Adaptive metrics for an evolving pandemic: A dynamic approach to area-level COVID-19 risk designations. Proc Natl Acad Sci U S A 2023; 120:e2302528120. [PMID: 37527346 PMCID: PMC10410764 DOI: 10.1073/pnas.2302528120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 04/27/2023] [Indexed: 08/03/2023] Open
Abstract
Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and have often lacked transparency in terms of prioritization of false-positive versus false-negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false-negative versus false-positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a method to update risk thresholds in real time. We show that this adaptive approach to designating areas as "high risk" improves performance over static metrics in predicting 3-wk-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-wk-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context.
Collapse
Affiliation(s)
- Alyssa M. Bilinski
- Departments of Health Services, Policy and Practice & Biostatistics, Brown University, Providence, RI02912
| | | | - Laura A. Hatfield
- Department of Health Care Policy, Harvard Medical School, Boston, MA02115
| |
Collapse
|
17
|
Li X, Liu H, Gao L, Sherchan SP, Zhou T, Khan SJ, van Loosdrecht MCM, Wang Q. Wastewater-based epidemiology predicts COVID-19-induced weekly new hospital admissions in over 150 USA counties. Nat Commun 2023; 14:4548. [PMID: 37507407 PMCID: PMC10382499 DOI: 10.1038/s41467-023-40305-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Accepted: 07/19/2023] [Indexed: 07/30/2023] Open
Abstract
Although the coronavirus disease (COVID-19) emergency status is easing, the COVID-19 pandemic continues to affect healthcare systems globally. It is crucial to have a reliable and population-wide prediction tool for estimating COVID-19-induced hospital admissions. We evaluated the feasibility of using wastewater-based epidemiology (WBE) to predict COVID-19-induced weekly new hospitalizations in 159 counties across 45 states in the United States of America (USA), covering a population of nearly 100 million. Using county-level weekly wastewater surveillance data (over 20 months), WBE-based models were established through the random forest algorithm. WBE-based models accurately predicted the county-level weekly new admissions, allowing a preparation window of 1-4 weeks. In real applications, periodically updated WBE-based models showed good accuracy and transferability, with mean absolute error within 4-6 patients/100k population for upcoming weekly new hospitalization numbers. Our study demonstrated the potential of using WBE as an effective method to provide early warnings for healthcare systems.
Collapse
Affiliation(s)
- Xuan Li
- Centre for Technology in Water and Wastewater, School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW, 2007, Australia
| | - Huan Liu
- Centre for Technology in Water and Wastewater, School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW, 2007, Australia
| | - Li Gao
- South East Water, 101 Wells Street, Frankston, VIC, 3199, Australia
| | - Samendra P Sherchan
- Department of Biology, Morgan State University, Baltimore, MD, USA
- Department of Environmental Health Sciences, School of Public Health and Tropical Medicine, Tulane University, New Orleans, LA, USA
| | - Ting Zhou
- Centre for Technology in Water and Wastewater, School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW, 2007, Australia
| | - Stuart J Khan
- Water Research Centre, School of Civil and Environmental Engineering, University of New South Wales, Sydney, NSW, 2052, Australia
| | - Mark C M van Loosdrecht
- Department of Biotechnology, Delft University of Technology, Julianalaan 67, 2628 BC, Delft, the Netherlands
| | - Qilin Wang
- Centre for Technology in Water and Wastewater, School of Civil and Environmental Engineering, University of Technology Sydney, Ultimo, NSW, 2007, Australia.
| |
Collapse
|
18
|
Garcia-Carretero R, Vazquez-Gomez O, Ordoñez-Garcia M, Garrido-Peño N, Gil-Prieto R, Gil-de-Miguel A. Differences in Trends in Admissions and Outcomes among Patients from a Secondary Hospital in Madrid during the COVID-19 Pandemic: A Hospital-Based Epidemiological Analysis (2020-2022). Viruses 2023; 15:1616. [PMID: 37515302 PMCID: PMC10384448 DOI: 10.3390/v15071616] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Revised: 07/17/2023] [Accepted: 07/21/2023] [Indexed: 07/30/2023] Open
Abstract
Spain had some of Europe's highest incidence and mortality rates for coronavirus disease 2019 (COVID-19). This study highlights the impact of the COVID-19 pandemic on daily health care in terms of incidence, critical patients, and mortality. We describe the characteristics and clinical outcomes of patients, comparing variables over the different waves. We performed a descriptive, retrospective study using the historical records of patients hospitalized with COVID-19. We describe demographic characteristics, admissions, and occupancy. Time series allowed us to visualize and analyze trends and patterns, and identify several waves during the 27-month period. A total of 3315 patients had been hospitalized with confirmed COVID-19. One-third of these patients were hospitalized during the first weeks of the pandemic. We observed that 4.6% of all hospitalizations had been admitted to the intensive care unit, and we identified a mortality rate of 9.4% among hospitalized patients. Arithmetic- and semi-logarithmic-scale charts showed how admissions and deaths rose sharply during the first weeks, increasing by 10 every few days. We described a single hospital's response and experiences during the pandemic. This research highlights certain demographic profiles in a population and emphasizes the importance of identifying waves when performing research on COVID-19. Our results can extend the analysis of the impact of COVID-19 and can be applied in other contexts, and can be considered when further analyzing the clinical, epidemiological, or demographic characteristics of populations with COVID-19. Our findings suggest that the pandemic should be analyzed not as a whole but rather in different waves.
Collapse
Affiliation(s)
- Rafael Garcia-Carretero
- Department of Internal Medicine, Mostoles University Hospital, 28935 Móstoles, Madrid, Spain
| | - Oscar Vazquez-Gomez
- Department of Internal Medicine, Mostoles University Hospital, 28935 Móstoles, Madrid, Spain
| | - María Ordoñez-Garcia
- Department of Hematology, Mostoles University Hospital, 28935 Móstoles, Madrid, Spain
| | - Noelia Garrido-Peño
- Department of Pharmacy, Mostoles University Hospital, 28935 Móstoles, Madrid, Spain
| | - Ruth Gil-Prieto
- Department of Preventive Medicine and Public Health, Rey Juan Carlos University, 28922 Alcorcón, Madrid, Spain
| | - Angel Gil-de-Miguel
- Department of Preventive Medicine and Public Health, Rey Juan Carlos University, 28922 Alcorcón, Madrid, Spain
| |
Collapse
|
19
|
Chen X, Dong Y, Wu M. Medical capacity investment for epidemic disease: The effects of policymaker's confidence and public trust. RISK ANALYSIS : AN OFFICIAL PUBLICATION OF THE SOCIETY FOR RISK ANALYSIS 2023; 43:1187-1211. [PMID: 35822620 DOI: 10.1111/risa.13988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Due to the server bed shortage, which has raised ethical dilemmas in the earliest days of the COVID-19 crisis, medical capacity investment has become a vital decision-making issue in the attempt to contain the epidemic. Furthermore, economic strength has failed to explain the significant performance difference across countries in combatting COVID-19. Unlike common diseases, epidemic diseases add substantial unpredictability, complexity, and uncertainty to decision-making. Knowledge miscalibration on epidemiological uncertainties by policymaker's over- and underconfidence can seriously impact policymaking. Ineffective risk communication may lead to conflicting and incoherent information transmission. As a result, public reactions and attitudes could be influenced by policymakers' confidence due to the level of public trust, which eventually affects the degree to which an epidemic spreads. To uncover the impacts of policymakers' confidence and public trust on the medical capacity investment, we establish epidemic diffusion models to characterize how transmission evolves with (and without) vaccination and frame the capacity investment problem as a newsvendor problem. Our results show that if the public fully trusts the public health experts, the policymaker's behavioral bias is always harmful, but its effect on cost increment is marginal. If a policymaker's behavior induces public reactions due to public trust, both the spread of the epidemic and the overall performance will be significantly affected, but such impacts are not always harmful. Decision bias may be beneficial when policymakers are pessimistic or highly overconfident. Having an opportunity to amend initially biased decisions can debias a particular topic but has a limited cost-saving effect.
Collapse
Affiliation(s)
- Xin Chen
- School of Business Administration, South China University of Technology, Guangzhou, China
| | - Yucheng Dong
- Center for Network Big Data and Decision-Making, Business School, Sichuan University, Chengdu, China
| | - Meng Wu
- Business School, Sichuan University, Chengdu, China
| |
Collapse
|
20
|
Reich NG, Wang Y, Burns M, Ergas R, Cramer EY, Ray EL. Assessing the utility of COVID-19 case reports as a leading indicator for hospitalization forecasting in the United States. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.03.08.23286582. [PMID: 36945396 PMCID: PMC10029058 DOI: 10.1101/2023.03.08.23286582] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2023]
Abstract
Identifying data streams that can consistently improve the accuracy of epidemiological forecasting models is challenging. Using models designed to predict daily state-level hospital admissions due to COVID-19 in California and Massachusetts, we investigated whether incorporating COVID-19 case data systematically improved forecast accuracy. Additionally, we considered whether using case data aggregated by date of test or by date of report from a surveillance system made a difference to the forecast accuracy. Evaluating forecast accuracy in a test period, after first having selected the best-performing methods in a validation period, we found that overall the difference in accuracy between approaches was small, especially at forecast horizons of less than two weeks. However, forecasts from models using cases aggregated by test date showed lower accuracy at longer horizons and at key moments in the pandemic, such as the peak of the Omicron wave in January 2022. Overall, these results highlight the challenge of finding a modeling approach that can generate accurate forecasts of outbreak trends both during periods of relative stability and during periods that show rapid growth or decay of transmission rates. While COVID-19 case counts seem to be a natural choice to help predict COVID-19 hospitalizations, in practice any benefits we observed were small and inconsistent.
Collapse
Affiliation(s)
- Nicholas G Reich
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA
| | - Yijin Wang
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA
| | - Meagan Burns
- Massachusetts Department of Public Health, Boston, MA March 8, 2023
| | - Rosa Ergas
- Massachusetts Department of Public Health, Boston, MA March 8, 2023
| | - Estee Y Cramer
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA
| | - Evan L Ray
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA
| |
Collapse
|
21
|
Du H, Dong E, Badr HS, Petrone ME, Grubaugh ND, Gardner LM. Incorporating variant frequencies data into short-term forecasting for COVID-19 cases and deaths in the USA: a deep learning approach. EBioMedicine 2023; 89:104482. [PMID: 36821889 PMCID: PMC9943054 DOI: 10.1016/j.ebiom.2023.104482] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2022] [Revised: 01/30/2023] [Accepted: 02/02/2023] [Indexed: 02/24/2023] Open
Abstract
BACKGROUND Since the US reported its first COVID-19 case on January 21, 2020, the science community has been applying various techniques to forecast incident cases and deaths. To date, providing an accurate and robust forecast at a high spatial resolution has proved challenging, even in the short term. METHOD Here we present a novel multi-stage deep learning model to forecast the number of COVID-19 cases and deaths for each US state at a weekly level for a forecast horizon of 1-4 weeks. The model is heavily data driven, and relies on epidemiological, mobility, survey, climate, demographic, and SARS-CoV-2 variant frequencies data. We implement a rigorous and robust evaluation of our model-specifically we report on weekly performance over a one-year period based on multiple error metrics, and explicitly assess how our model performance varies over space, chronological time, and different outbreak phases. FINDINGS The proposed model is shown to consistently outperform the CDC ensemble model for all evaluation metrics in multiple spatiotemporal settings, especially for the longer-term (3 and 4 weeks ahead) forecast horizon. Our case study also highlights the potential value of variant frequencies data for use in short-term forecasting to identify forthcoming surges driven by new variants. INTERPRETATION Based on our findings, the proposed forecasting framework improves upon the available state-of-the-art forecasting tools currently used to support public health decision making with respect to COVID-19 risk. FUNDING This work was funded the NSF Rapid Response Research (RAPID) grant Award ID 2108526 and the CDC Contract #75D30120C09570.
Collapse
Affiliation(s)
- Hongru Du
- Center for Systems Science and Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Ensheng Dong
- Center for Systems Science and Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Hamada S Badr
- Center for Systems Science and Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Earth and Planetary Sciences, Johns Hopkins University, Baltimore, MD, 21218, USA
| | - Mary E Petrone
- Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, 06510, USA
| | - Nathan D Grubaugh
- Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, 06510, USA; Department of Ecology and Evolutionary Biology, Yale University, New Haven, CT, 06510, USA
| | - Lauren M Gardner
- Center for Systems Science and Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Civil and Systems Engineering, Johns Hopkins University, Baltimore, MD, 21218, USA; Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, 21205, USA.
| |
Collapse
|
22
|
Bilinski AM, Salomon JA, Hatfield LA. Adaptive metrics for an evolving pandemic A dynamic approach to area-level COVID-19 risk designations. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.02.15.23285969. [PMID: 36824769 PMCID: PMC9949193 DOI: 10.1101/2023.02.15.23285969] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and often lack transparency in terms of prioritization of false positive versus false negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses of risk metrics. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false negative versus false positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a novel method to update risk thresholds in real-time. We show that this adaptive approach to designating areas as "high risk" improves performance over static metrics in predicting 3-week-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-week-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context. Significance Statement In the rapidly-evolving COVID-19 pandemic, public health risk metrics often become less relevant over time. Risk metrics are designed to predict future severe disease and mortality based on currently-available surveillance data, such as cases and hospitalizations. However, the relationship between cases, hospitalizations, and mortality has varied considerably over the course of the pandemic, in the context of new variants and shifts in vaccine- and infection-induced immunity. We propose an adaptive approach that regularly updates metrics based on the relationship between surveillance inputs and future outcomes of policy interest. Our method captures changing pandemic dynamics, requires only hospitalization input data, and outperforms static risk metrics in predicting high-risk states and counties.
Collapse
Affiliation(s)
- Alyssa M. Bilinski
- Departments of Health Services, Policy and Practice & Biostatistics, Brown University, 121 S. Main St., Providence, RI 02912 USA
| | - Joshua A. Salomon
- Department of Health Policy, Stanford University, Stanford, CA 94305 USA
| | - Laura A. Hatfield
- Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave., Boston, MA 02115 USA
| |
Collapse
|
23
|
Forecasting hospital-level COVID-19 admissions using real-time mobility data. COMMUNICATIONS MEDICINE 2023; 3:25. [PMID: 36788347 PMCID: PMC9927044 DOI: 10.1038/s43856-023-00253-5] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Accepted: 01/31/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND For each of the COVID-19 pandemic waves, hospitals have had to plan for deploying surge capacity and resources to manage large but transient increases in COVID-19 admissions. While a lot of effort has gone into predicting regional trends in COVID-19 cases and hospitalizations, there are far fewer successful tools for creating accurate hospital-level forecasts. METHODS Large-scale, anonymized mobile phone data has been shown to correlate with regional case counts during the first two waves of the pandemic (spring 2020, and fall/winter 2021). Building off this success, we developed a multi-step, recursive forecasting model to predict individual hospital admissions; this model incorporates the following data: (i) hospital-level COVID-19 admissions, (ii) statewide test positivity data, and (iii) aggregate measures of large-scale human mobility, contact patterns, and commuting volume. RESULTS Incorporating large-scale, aggregate mobility data as exogenous variables in prediction models allows us to make hospital-specific COVID-19 admission forecasts 21 days ahead. We show this through highly accurate predictions of hospital admissions for five hospitals in Massachusetts during the first year of the COVID-19 pandemic. CONCLUSIONS The high predictive capability of the model was achieved by combining anonymized, aggregated mobile device data about users' contact patterns, commuting volume, and mobility range with COVID hospitalizations and test-positivity data. Mobility-informed forecasting models can increase the lead-time of accurate predictions for individual hospitals, giving managers valuable time to strategize how best to allocate resources to manage forthcoming surges.
Collapse
|
24
|
McWilliams C, Bothwell L, Yousey‐Hindes K, Hadler JL. Trends in disparities in COVID hospitalizations among community-dwelling residents of two counties in Connecticut, before and after vaccine introduction, March 2020-September 2021. Influenza Other Respir Viruses 2023; 17:e13082. [PMID: 36509459 PMCID: PMC9835416 DOI: 10.1111/irv.13082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 11/16/2022] [Accepted: 11/17/2022] [Indexed: 12/15/2022] Open
Abstract
BACKGROUND Prior to the introduction of vaccines, COVID-19 hospitalizations of non-institutionalized persons in Connecticut disproportionately affected communities of color and individuals of low socioeconomic status (SES). Whether the magnitude of these disparities changed 7-9 months after vaccine rollout during the Delta wave is not well documented. METHODS All initially hospitalized patients with laboratory-confirmed COVID-19 during July-September 2021 were obtained from the Connecticut COVID-19-Associated Hospitalization Surveillance Network database, including patients' geocoded residential addresses. Census tract measures of poverty and crowding were determined by linking geocoded residential addresses to the 2014-2018 American Community Survey. Age-adjusted incidence and relative rates of COVID-19 hospitalization were calculated and compared with those from July to December 2020. Vaccination levels by age and race/ethnicity at the beginning and end of the study period were obtained from Connecticut's COVID vaccine registry, and age-adjusted average values were determined. RESULTS There were 708 COVID-19 hospitalizations among community residents of the two counties, July-September 2021. Age-adjusted incidence was the highest among non-Hispanic Blacks and Hispanic/Latinx compared with non-Hispanic Whites (RR 4.10 [95% CI 3.41-4.94] and 3.47 [95% CI 2.89-4.16]). Although RR decreased significantly among Hispanic/Latinx and among the lowest SES groups, it increased among non-Hispanic Blacks (from RR 3.2 [95% CI 2.83-3.32] to RR 4.10). Average age-adjusted vaccination rates among those ≥12 years were the lowest among non-Hispanic Blacks compared with Hispanic/Latinx and non-Hispanic Whites (50.6% vs. 64.7% and 66.6%). CONCLUSIONS Although racial/ethnic and SES disparities in COVID-19 hospitalization have mostly decreased over time, disparities among non-Hispanic Blacks increased, possibly due to differences in vaccination rates.
Collapse
Affiliation(s)
- Caroline McWilliams
- Epidemiology of Microbial DiseasesYale School of Public HealthNew HavenConnecticutUSA
| | - Laura Bothwell
- Epidemiology of Microbial DiseasesYale School of Public HealthNew HavenConnecticutUSA
| | - Kimberly Yousey‐Hindes
- Connecticut Emerging Infections ProgramYale School of Public HealthNew HavenConnecticutUSA
| | - James L. Hadler
- Connecticut Emerging Infections ProgramYale School of Public HealthNew HavenConnecticutUSA
| |
Collapse
|
25
|
Damone A, Vainieri M, Brunetto MR, Bonino F, Nuti S, Ciuti G. Decision-making algorithm and predictive model to assess the impact of infectious disease epidemics on the healthcare system: the COVID-19 case study in Italy. IEEE J Biomed Health Inform 2022; 26:3661-3672. [PMID: 35544510 DOI: 10.1109/jbhi.2022.3174470] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Until SARS-CoV-2 vaccination consolidates immunity worldwide, infectious strains will continue to cause infection, causing critical pressures on the healthcare systems and lockdowns. To improve decision-making strategies and prediction based on epidemiological data, so far biased by highly-variable testing criteria, an algorithm using unbiased morbidity parameters, i.e. Intensive Care Units (ICU) and Ordinary Hospitalization (HO), is proposed. ICU/HO acceleration and velocities are mathematically modeled using official available data to yield two thresholds, alerting on 30% ICU and 40% HO of COVID-19 daily occupancy settled by the Italian Minister of Health, as a case of study. A predictive model is also proposed to estimate the daily occupancy of ICU and HO in hospitals for each region, using a Susceptible-Infected-Recovered-Death (SIRD) epidemic model to further extend occupancy prediction in each regional district. Computed data validated the proposed models in Italy after more than one-year of pandemic obtaining agreements with the Italian Presidential Decree, regardless of the different regional trends of epidemic waves. Therefore, the decision-making algorithm and prediction model resulted valuable tools, retrospectively, to be tested prospectively in sustainable strategies to curb the impact of COVID-19, or of any other pandemic threats with any aggregate of data, on the local healthcare systems.
Collapse
|
26
|
Her PH, Saeed S, Tram KH, Bhatnagar SR. Novel mobility index tracks COVID-19 transmission following stay-at-home orders. Sci Rep 2022; 12:7654. [PMID: 35538129 PMCID: PMC9088135 DOI: 10.1038/s41598-022-10941-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Accepted: 04/12/2022] [Indexed: 12/13/2022] Open
Abstract
Considering the emergence of SARS-CoV-2 variants and low vaccine access and uptake, minimizing human interactions remains an effective strategy to mitigate the spread of SARS-CoV-2. Using a functional principal component analysis, we created a multidimensional mobility index (MI) using six metrics compiled by SafeGraph from all counties in Illinois, Ohio, Michigan and Indiana between January 1 to December 8, 2020. Changes in mobility were defined as a time-updated 7-day rolling average. Associations between our MI and COVID-19 cases were estimated using a quasi-Poisson hierarchical generalized additive model adjusted for population density and the COVID-19 Community Vulnerability Index. Individual mobility metrics varied significantly by counties and by calendar time. More than 50% of the variability in the data was explained by the first principal component by each state, indicating good dimension reduction. While an individual metric of mobility was not associated with surges of COVID-19, our MI was independently associated with COVID-19 cases in all four states given varying time-lags. Following the expiration of stay-at-home orders, a single metric of mobility was not sensitive enough to capture the complexity of human interactions. Monitoring mobility can be an important public health tool, however, it should be modelled as a multidimensional construct.
Collapse
Affiliation(s)
- Peter Hyunwuk Her
- Department of Pharmacology and Therapeutics, McGill University, Montreal, Canada.,Department of Medical Biophysics, University of Toronto, Toronto, Canada
| | - Sahar Saeed
- Division of Infectious Diseases, Department of Medicine, Washington University School of Medicine, St. Louis, USA.,Department of Public Health Sciences, Queen's University, Ontario, Canada
| | - Khai Hoan Tram
- Division of Infectious Diseases, Department of Medicine, University of Washington, Seattle, USA
| | - Sahir R Bhatnagar
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Canada. .,Department of Diagnostic Radiology, McGill University, Montreal, Canada.
| |
Collapse
|
27
|
Reich NG, Ray EL. Collaborative modeling key to improving outbreak response. Proc Natl Acad Sci U S A 2022; 119:e2200703119. [PMID: 35320028 PMCID: PMC9169077 DOI: 10.1073/pnas.2200703119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022] Open
Affiliation(s)
- Nicholas G. Reich
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA 01003
| | - Evan L. Ray
- School of Public Health and Health Sciences, University of Massachusetts Amherst, Amherst, MA 01003
| |
Collapse
|