Consistent comparison of symptom-based methods for COVID-19 infection detection

BACKGROUND

During the global pandemic crisis, various detection methods of COVID-19-positive cases based on self-reported information were introduced to provide quick diagnosis tools for effectively planning and managing healthcare resources. These methods typically identify positive cases based on a particular combination of symptoms, and they have been evaluated using different datasets.

PURPOSE

This paper presents a comprehensive comparison of various COVID-19 detection methods based on self-reported information using the University of Maryland Global COVID-19 Trends and Impact Survey (UMD-CTIS), a large health surveillance platform, which was launched in partnership with Facebook.

METHODS

Detection methods were implemented to identify COVID-19-positive cases among UMD-CTIS participants reporting at least one symptom and a recent antigen test result (positive or negative) for six countries and two periods. Multiple detection methods were implemented for three different categories: rule-based approaches, logistic regression techniques, and tree-based machine-learning models. These methods were evaluated using different metrics including F1-score, sensitivity, specificity, and precision. An explainability analysis has also been conducted to compare methods.

RESULTS

Fifteen methods were evaluated for six countries and two periods. We identify the best method for each category: rule-based methods (F1-score: 51.48% - 71.11%), logistic regression techniques (F1-score: 39.91% - 71.13%), and tree-based machine learning models (F1-score: 45.07% - 73.72%). According to the explainability analysis, the relevance of the reported symptoms in COVID-19 detection varies between countries and years. However, there are two variables consistently relevant across approaches: stuffy or runny nose, and aches or muscle pain.

CONCLUSIONS

Regarding the categories of detection methods, evaluating detection methods using homogeneous data across countries and years provides a solid and consistent comparison. An explainability analysis of a tree-based machine-learning model can assist in identifying infected individuals specifically based on their relevant symptoms. This study is limited by the self-report nature of data, which cannot replace clinical diagnosis.

AI for rapid identification of major butyrate-producing bacteria in rhesus macaques (Macaca mulatta)

BACKGROUND

The gut microbiome plays a crucial role in health and disease, influencing digestion, metabolism, and immune function. Traditional microbiome analysis methods are often expensive, time-consuming, and require specialized expertise, limiting their practical application in clinical settings. Evolving artificial intelligence (AI) technologies present opportunities for developing alternative methods. However, the lack of transparency in these technologies limits the ability of clinicians to incorporate AI-driven diagnostic tools into their healthcare systems. The aim of this study was to investigate an AI approach that rapidly predicts different bacterial genera and bacterial groups, specifically butyrate producers, from digital images of fecal smears of rhesus macaques (Macaca mulatta). In addition, to improve transparency, we employed explainability analysis to uncover the image features influencing the model's predictions.

RESULTS

By integrating fecal image data with corresponding metagenomic sequencing information, the deep learning (DL) and machine learning (ML) algorithms successfully predicted 16 individual bacterial genera (area under the curve (AUC) > 0.7) among the 50 most abundant genera in rhesus macaques (Macaca mulatta). The model was successful in predicting functional groups, major butyrate producers (AUC 0.75) and a mixed group including fermenters and short-chain fatty acid (SCFA) producers (AUC 0.81). For both models of butyrate producers and mixed fermenters, the explainability experiments revealed no decline in the AUC when random noise was added to the images. Increased blurring led to a gradual decline in the AUC. The model's performance was robust against the impact of fecal shape from smearing, with a stable AUC maintained until patch 4 for all groups, as assessed through scrambling. No significant correlation was detected between the prediction probabilities and the total fecal weight used in the smear; r = 0.30 ± 0.3 (p > 0.1) and r = 0.04 ± 0.36 (p > 0.8) for the butyrate producers and mixed fermenters, respectively.

CONCLUSION

Our approach demonstrated the ability to predict a wide range of clinically relevant microbial genera and microbial groups in the gut microbiome based on digital images from a fecal smear. The models proved to be robust to the smearing method, random noise and the amount of fecal matter. This study introduces a rapid, non-invasive, and cost-effective method for microbiome profiling, with potential applications in veterinary diagnostics.

Performance and explainability of feature selection-boosted tree-based classifiers for COVID-19 detection

In this paper, we evaluate the performance and analyze the explainability of machine learning models boosted by feature selection in predicting COVID-19-positive cases from self-reported information. In essence, this work describes a methodology to identify COVID-19 infections that considers the large amount of information collected by the University of Maryland Global COVID-19 Trends and Impact Survey (UMD-CTIS). More precisely, this methodology performs a feature selection stage based on the recursive feature elimination (RFE) method to reduce the number of input variables without compromising detection accuracy. A tree-based supervised machine learning model is then optimized with the selected features to detect COVID-19-active cases. In contrast to previous approaches that use a limited set of selected symptoms, the proposed approach builds the detection engine considering a broad range of features including self-reported symptoms, local community information, vaccination acceptance, and isolation measures, among others. To implement the methodology, three different supervised classifiers were used: random forests (RF), light gradient boosting (LGB), and extreme gradient boosting (XGB). Based on data collected from the UMD-CTIS, we evaluated the detection performance of the methodology for four countries (Brazil, Canada, Japan, and South Africa) and two periods (2020 and 2021). The proposed approach was assessed in terms of various quality metrics: F1-score, sensitivity, specificity, precision, receiver operating characteristic (ROC), and area under the ROC curve (AUC). This work also shows the normalized daily incidence curves obtained by the proposed approach for the four countries. Finally, we perform an explainability analysis using Shapley values and feature importance to determine the relevance of each feature and the corresponding contribution for each country and each country/year.