1
|
Mathis SM, Webber AE, León TM, Murray EL, Sun M, White LA, Brooks LC, Green A, Hu AJ, McDonald DJ, Rosenfeld R, Shemetov D, Tibshirani RJ, Kandula S, Pei S, Shaman J, Yaari R, Yamana TK, Agarwal P, Balusu S, Gururajan G, Kamarthi H, Prakash BA, Raman R, Rodríguez A, Zhao Z, Meiyappan A, Omar S, Baccam P, Gurung HL, Stage SA, Suchoski BT, Ajelli M, Kummer AG, Litvinova M, Ventura PC, Wadsworth S, Niemi J, Carcelen E, Hill AL, Jung SM, Lemaitre JC, Lessler J, Loo SL, McKee CD, Sato K, Smith C, Truelove S, McAndrew T, Ye W, Bosse N, Hlavacek WS, Lin YT, Mallela A, Chen Y, Lamm SM, Lee J, Posner RG, Perofsky AC, Viboud C, Clemente L, Lu F, Meyer AG, Santillana M, Chinazzi M, Davis JT, Mu K, Piontti APY, Vespignani A, Xiong X, Ben-Nun M, Riley P, Turtle J, Hulme-Lowe C, Jessa S, Nagraj VP, Turner SD, Williams D, Basu A, Drake JM, Fox SJ, Gibson GC, Suez E, Thommes EW, Cojocaru MG, Cramer EY, Gerding A, Stark A, Ray EL, Reich NG, Shandross L, Wattanachit N, Wang Y, Zorn MW, Al Aawar M, Srivastava A, Meyers LA, Adiga A, Hurt B, Kaur G, Lewis BL, Marathe M, Venkatramanan S, Butler P, Farabow A, Muralidhar N, Ramakrishnan N, Reed C, Biggerstaff M, Borchering RK. Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations. medRxiv 2023:2023.12.08.23299726. [PMID: 38168429 PMCID: PMC10760285 DOI: 10.1101/2023.12.08.23299726] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one through four weeks ahead for the 2021-22 and 2022-23 influenza seasons. Across both seasons, 26 teams submitted forecasts, with the submitting teams varying between seasons. Forecast skill was evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperformed the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble was the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degraded over longer forecast horizons and during periods of rapid change. Current influenza forecasting efforts help inform situational awareness, but research is needed to address limitations, including decreased performance during periods of changing epidemic dynamics.
Collapse
Affiliation(s)
- Sarabeth M Mathis
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | - Alexander E Webber
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | - Tomás M León
- California Department of Public Health, Richmond, CA, 95899
| | - Erin L Murray
- California Department of Public Health, Richmond, CA, 95899
| | - Monica Sun
- California Department of Public Health, Richmond, CA, 95899
| | - Lauren A White
- California Department of Public Health, Richmond, CA, 95899
| | - Logan C Brooks
- Carnegie Mellon University, Pittsburgh, PA, 15213
- University of California, Berkeley, Berkeley, CA 94720
| | - Alden Green
- Carnegie Mellon University, Pittsburgh, PA, 15213
| | - Addison J Hu
- Carnegie Mellon University, Pittsburgh, PA, 15213
| | | | | | | | - Ryan J Tibshirani
- Carnegie Mellon University, Pittsburgh, PA, 15213
- University of California, Berkeley, Berkeley, CA 94720
| | | | - Sen Pei
- Columbia University, New York, NY, 10032
| | - Jeffrey Shaman
- Columbia University, New York, NY, 10032
- Columbia University School of Climate, New York, NY 10025
| | - Rami Yaari
- Columbia University, New York, NY, 10032
| | | | | | | | | | | | | | - Rishi Raman
- Georgia Institute of Technology, Atlanta, GA, 30318
| | | | - Zhiyuan Zhao
- Georgia Institute of Technology, Atlanta, GA, 30318
| | | | - Shalina Omar
- Guidehouse Advisory and Consulting Services, McClean VA, 22102
| | | | | | | | | | - Marco Ajelli
- Indiana University School of Public Health, Bloomington, IN, 47405
| | | | - Maria Litvinova
- Indiana University School of Public Health, Bloomington, IN, 47405
| | - Paulo C Ventura
- Indiana University School of Public Health, Bloomington, IN, 47405
| | | | | | | | | | - Sung-Mok Jung
- University of North Carolina at Chapel Hill, Chapel Hill, NC
| | | | - Justin Lessler
- University of North Carolina at Chapel Hill, Chapel Hill, NC
| | - Sara L Loo
- Johns Hopkins University, Baltimore, MD, 21205
| | | | - Koji Sato
- Johns Hopkins University, Baltimore, MD, 21205
| | | | | | | | | | - Nikos Bosse
- London School of Health and Tropical Medicine, London, UK, WC1E 7HT
| | | | - Yen Ting Lin
- Los Alamos National Laboratory, Los Alamos, NM, 87545
| | | | - Ye Chen
- Northern Arizona University, Flagstaff, AZ, 86011
| | | | - Jaechoul Lee
- Northern Arizona University, Flagstaff, AZ, 86011
| | | | - Amanda C Perofsky
- Fogarty International Center, National Institutes of Health, Bethesda, MD, 20892
| | - Cécile Viboud
- Fogarty International Center, National Institutes of Health, Bethesda, MD, 20892
| | | | - Fred Lu
- Northeastern University, Boston, MA, 02115
| | | | | | | | | | - Kunpeng Mu
- Northeastern University, Boston, MA, 02115
| | | | | | | | | | - Pete Riley
- Predictive Science Inc, San Diego, CA 92121
| | | | | | | | - V P Nagraj
- Signature Science, LLC, Charlottesville, VA, 22911
| | | | | | | | | | | | | | - Ehsan Suez
- University of Georgia, Athens, GA, 30609
| | - Edward W Thommes
- University of Guelph, Guelph, ON N1G 2W1, Canada
- Sanofi, Toronto, ON, M2R 3T4
| | | | | | - Aaron Gerding
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Ariane Stark
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Evan L Ray
- University of Massachusetts Amherst, Amherst, MA, 01003
| | | | - Li Shandross
- University of Massachusetts Amherst, Amherst, MA, 01003
| | | | - Yijin Wang
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Martha W Zorn
- University of Massachusetts Amherst, Amherst, MA, 01003
| | - Majd Al Aawar
- University of Southern California, Los Angeles, CA, 90089
| | | | | | | | | | | | | | | | | | | | | | | | | | - Carrie Reed
- Centers for Disease Control and Prevention, Atlanta, Georgia, 30329, USA
| | | | | |
Collapse
|