Air Passenger Travel and International Surveillance Data Predict Spatiotemporal Variation in Measles Importations to the United States

Prediction of measles cases in US counties: A machine learning approach

BACKGROUND

Although measles was declared eliminated from the United States in 2000, the frequency of measles outbreaks has increased in recent years. The ability to predict the locations of future cases could aid efforts to prevent and contain measles in the United States.

METHODS

We estimated county-level measles risk using a machine learning model with 17 predictor variables, which was trained on 2014 and 2018 United States county-level measles case data and tested on data from 2019. We compared the predicted and actual locations of 2019 measles cases.

RESULTS

The model accurately predicted 95 % (specificity) of United States counties without measles cases and 72 % (sensitivity) of the United States counties that experienced ≥1 measles case in 2019, accounting for 94 % of all measles cases in 2019. Among the top 30 counties with the highest risk scores, the model accurately predicted 22 (73 %) counties with a measles case in 2019, corresponding to 72 % of all measles cases.

CONCLUSIONS

This machine learning model accurately predicted a majority of the United States counties at high risk for measles and could be used as a framework by state and national health agencies in their measles prevention and containment efforts.

Predicting Measles Outbreaks in the United States: Evaluation of Machine Learning Approaches (Preprint)

BACKGROUND

Measles, a highly contagious viral infection, is resurging in the United States, driven by international importation and declining domestic vaccination coverage. Despite this resurgence, measles outbreaks are still rare events that are difficult to predict. Improved methods to predict outbreaks at the county level would facilitate the optimal allocation of public health resources.

OBJECTIVE

We aimed to validate and compare extreme gradient boosting (XGBoost) and logistic regression, 2 supervised learning approaches, to predict the US counties most likely to experience measles cases. We also aimed to assess the performance of hybrid versions of these models that incorporated additional predictors generated by 2 clustering algorithms, hierarchical density-based spatial clustering of applications with noise (HDBSCAN) and unsupervised random forest (uRF).

METHODS

We constructed a supervised machine learning model based on XGBoost and unsupervised models based on HDBSCAN and uRF. The unsupervised models were used to investigate clustering patterns among counties with measles outbreaks; these clustering data were also incorporated into hybrid XGBoost models as additional input variables. The machine learning models were then compared to logistic regression models with and without input from the unsupervised models.

RESULTS

Both HDBSCAN and uRF identified clusters that included a high percentage of counties with measles outbreaks. XGBoost and XGBoost hybrid models outperformed logistic regression and logistic regression hybrid models, with the area under the receiver operating curve values of 0.920-0.926 versus 0.900-0.908, the area under the precision-recall curve values of 0.522-0.532 versus 0.485-0.513, and F₂ scores of 0.595-0.601 versus 0.385-0.426. Logistic regression or logistic regression hybrid models had higher sensitivity than XGBoost or XGBoost hybrid models (0.837-0.857 vs 0.704-0.735) but a lower positive predictive value (0.122-0.141 vs 0.340-0.367) and specificity (0.793-0.821 vs 0.952-0.958). The hybrid versions of the logistic regression and XGBoost models had slightly higher areas under the precision-recall curve, specificity, and positive predictive values than the respective models that did not include any unsupervised features.

CONCLUSIONS

XGBoost provided more accurate predictions of measles cases at the county level compared with logistic regression. The threshold of prediction in this model can be adjusted to align with each county's resources, priorities, and risk for measles. While clustering pattern data from unsupervised machine learning approaches improved some aspects of model performance in this imbalanced data set, the optimal approach for the integration of such approaches with supervised machine learning models requires further investigation.