SuRF: A new method for sparse variable selection, with application in microbiome data analysis

Novel application of survival models for predicting microbial community transitions with variable selection for eDNA

Survival analysis is a prolific statistical tool in medicine for inferring risk and time to disease-related events. However, it is under-utilized in microbiome research to predict microbial community mediated events, partly due to the sparsity and high dimensional nature of the data. We advance the application of Cox proportional hazards (Cox PH) survival models to environmental DNA (eDNA) data with feature selection suitable for filtering irrelevant and redundant taxonomic variables. Selection methods are compared in terms of false positives, sensitivity, and survival estimation accuracy in simulation and in a real data setting to forecast harmful cyanobacterial blooms. A novel extension of a method for selecting microbial biomarkers with survival data (SuRFCox) reliably outperforms other methods. We determine Cox PH models with SuRFCox selected predictors are more robust to varied signal, noise, and data correlation structure. SuRFCox also yields the most accurate and consistent prediction of blooms according to cross-validated testing by year over eight different bloom seasons. Identification of common biomarkers among validated survival forecasts over changing conditions has clear biological significance. Survival models with such biomarkers inform risk assessment and provide insight into the causes of critical community transitions. Importance In this paper, we report on a novel approach of selecting microorganisms for model-based prediction of the time to critical microbially-modulated events (e.g., harmful algal blooms, clinical outcomes, community shifts, etc.). Our novel method for identifying biomarkers from large, dynamic communities of microbes has broad utility to environmental and ecological impact risk assessment and public health. Results will also promote theoretical and practical advancements relevant to the biology of specific organisms. To address the unique challenge posed by diverse environmental conditions and sparse microbes, we developed a novel method of selecting predictors for modelling time-to-event data. Competing methods for selecting predictors are rigorously compared to determine which is the most accurate and generalizable. Model forecasts are applied to show suitable predictors can precisely quantify the risk over time of biological events like harmful cyanobacterial blooms.

NFnetFu: A novel workflow for microbiome data fusion

Microbiome data analysis and its interpretation into meaningful biological insights remain very challenging for numerous reasons, perhaps most prominently, due to the need to account for multiple factors, including collinearity, sparsity (excessive zeros) and effect size, that the complex experimental workflow and subsequent downstream data analysis require. Moreover, a meaningful microbiome data analysis necessitates the development of interpretable models that incorporate inferences across available data as well as background biomedical knowledge. We developed a multimodal framework that considers sparsity (excessive zeros), lower effect size, intrinsically microbial correlations, i.e., collinearity, as well as background biomedical knowledge in the form of a cluster-infused enriched network architecture. Finally, our framework also provides a candidate taxa/Operational Taxonomic Unit (OTU) that can be targeted for future validation experiments. We have developed a tool, the term NFnetFU (Neuro Fuzzy network Fusion), that encompasses our framework and have made it freely available at https://github.com/VartikaBisht6197/NFnetFu.