Efficient and Accurate Inference of Mixed Microbial Population Trajectories from Longitudinal Count Data

Interpolation of microbiome composition in longitudinal data sets

The human gut microbiome significantly impacts health, prompting a rise in longitudinal studies that capture microbiome samples at multiple time points. Such studies allow researchers to characterize microbiome changes over time, but importantly, also present major analytical challenges due to incomplete or irregular sampling. To address this challenge, longitudinal microbiome studies often employ various interpolation methods, aiming to infer missing microbiome data. However, to date, a comprehensive assessment of such microbiome interpolation techniques, as well as best practice guidelines for interpolating microbiome data, is still lacking. This work aims to fill this gap, rigorously implementing and systematically evaluating a large array of interpolation methods, spanning several different categories, for longitudinal microbiome interpolation. To assess each method and its ability to accurately infer microbiome composition at missing time points, we used three longitudinal microbiome data sets that follow individuals over a long period of time and a leave-one-out approach. Overall, our analysis demonstrated that the K-nearest neighbors algorithm consistently outperforms other methods in interpolation accuracy, yet, accuracy varied widely across data sets, individuals, and time. Factors such as microbiome stability, sample size, and the time gap between interpolated and adjacent samples significantly influenced accuracy, allowing us to develop a model for predicting the expected interpolation accuracy at a missing time point. Our findings, combined, suggest that accurate interpolation in longitudinal microbiome data is feasible, especially in dense cohorts. Furthermore, using our predictive model, future studies can interpolate data only in time points where the expected interpolation accuracy is high.

IMPORTANCE

Since missing samples are common in longitudinal microbiome dataset due to inconsistent collection practices, it is important to evaluate and benchmark different interpolation methods for predicting microbiome composition in such samples and facilitate downstream analysis. Our study rigorously evaluated several such methods and identified the K-nearest neighbors approach as particularly effective for this task. The study also notes significant variability in interpolation accuracy among individuals, influenced by factors such as age, sample size, and sampling frequency. Furthermore, we developed a predictive model for estimating interpolation accuracy at a specific time point, enhancing the reliability of such analyses in future studies. Combined, our study, thus, provides critical insights and tools that enhance the accuracy and reliability of data interpolation methods in the growing field of longitudinal microbiome research.

A digital twin of the infant microbiome to predict neurodevelopmental deficits

Despite the recognized gut-brain axis link, natural variations in microbial profiles between patients hinder definition of normal abundance ranges, confounding the impact of dysbiosis on infant neurodevelopment. We infer a digital twin of the infant microbiome, forecasting ecosystem trajectories from a few initial observations. Using 16S ribosomal RNA profiles from 88 preterm infants (398 fecal samples and 32,942 abundance estimates for 91 microbial classes), the model (Q-net) predicts abundance dynamics with R2 = 0.69. Contrasting the fit to Q-nets of typical versus suboptimal development, we can reliably estimate individual deficit risk (Mδ) and identify infants achieving poor future head circumference growth with ≈76% area under the receiver operator characteristic curve, 95% ± 1.8% positive predictive value at 98% specificity at 30 weeks postmenstrual age. We find that early transplantation might mitigate risk for ≈45.2% of the cohort, with potentially negative effects from incorrect supplementation. Q-nets are generative artificial intelligence models for ecosystem dynamics, with broad potential applications.

Modeling Microbial Community Networks: Methods and Tools for Studying Microbial Interactions

Microbial interactions function as a fundamental unit in complex ecosystems. By characterizing the type of interaction (positive, negative, neutral) occurring in these dynamic systems, one can begin to unravel the role played by the microbial species. Towards this, various methods have been developed to decipher the function of the microbial communities. The current review focuses on the various qualitative and quantitative methods that currently exist to study microbial interactions. Qualitative methods such as co-culturing experiments are visualized using microscopy-based techniques and are combined with data obtained from multi-omics technologies (metagenomics, metabolomics, metatranscriptomics). Quantitative methods include the construction of networks and network inference, computational models, and development of synthetic microbial consortia. These methods provide a valuable clue on various roles played by interacting partners, as well as possible solutions to overcome pathogenic microbes that can cause life-threatening infections in susceptible hosts. Studying the microbial interactions will further our understanding of complex less-studied ecosystems and enable design of effective frameworks for treatment of infectious diseases.

Methodological Considerations in Longitudinal Analyses of Microbiome Data: A Comprehensive Review

Biological processes underlying health and disease are inherently dynamic and are best understood when characterized in a time-informed manner. In this comprehensive review, we discuss challenges inherent in time-series microbiome data analyses and compare available approaches and methods to overcome them. Appropriate handling of longitudinal microbiome data can shed light on important roles, functions, patterns, and potential interactions between large numbers of microbial taxa or genes in the context of health, disease, or interventions. We present a comprehensive review and comparison of existing microbiome time-series analysis methods, for both preprocessing and downstream analyses, including differential analysis, clustering, network inference, and trait classification. We posit that the careful selection and appropriate utilization of computational tools for longitudinal microbiome analyses can help advance our understanding of the dynamic host-microbiome relationships that underlie health-maintaining homeostases, progressions to disease-promoting dysbioses, as well as phases of physiologic development like those encountered in childhood.

EMBED: Essential MicroBiomE Dynamics, a dimensionality reduction approach for longitudinal microbiome studies

Dimensionality reduction offers unique insights into high-dimensional microbiome dynamics by leveraging collective abundance fluctuations of multiple bacteria driven by similar ecological perturbations. However, methods providing lower-dimensional representations of microbiome dynamics both at the community and individual taxa levels are not currently available. To that end, we present EMBED: Essential MicroBiomE Dynamics, a probabilistic nonlinear tensor factorization approach. Like normal mode analysis in structural biophysics, EMBED infers ecological normal modes (ECNs), which represent the unique orthogonal modes capturing the collective behavior of microbial communities. Using multiple real and synthetic datasets, we show that a very small number of ECNs can accurately approximate microbiome dynamics. Inferred ECNs reflect specific ecological behaviors, providing natural templates along which the dynamics of individual bacteria may be partitioned. Moreover, the multi-subject treatment in EMBED systematically identifies subject-specific and universal abundance dynamics that are not detected by traditional approaches. Collectively, these results highlight the utility of EMBED as a versatile dimensionality reduction tool for studies of microbiome dynamics.

Wavelet clustering analysis as a tool for characterizing community structure in the human microbiome

Human microbiome research is helped by the characterization of microbial networks, as these may reveal key microbes that can be targeted for beneficial health effects. Prevailing methods of microbial network characterization are based on measures of association, often applied to limited sampling points in time. Here, we demonstrate the potential of wavelet clustering, a technique that clusters time series based on similarities in their spectral characteristics. We illustrate this technique with synthetic time series and apply wavelet clustering to densely sampled human gut microbiome time series. We compare our results with hierarchical clustering based on temporal correlations in abundance, within and across individuals, and show that the cluster trees obtained by using either method are significantly different in terms of elements clustered together, branching structure and total branch length. By capitalizing on the dynamic nature of the human microbiome, wavelet clustering reveals community structures that remain obscured in correlation-based methods.

Universal gut microbial relationships in the gut microbiome of wild baboons

Ecological relationships between bacteria mediate the services that gut microbiomes provide to their hosts. Knowing the overall direction and strength of these relationships is essential to learn how ecology scales up to affect microbiome assembly, dynamics, and host health. However, whether bacterial relationships are generalizable across hosts or personalized to individual hosts is debated. Here, we apply a robust, multinomial logistic-normal modeling framework to extensive time series data (5534 samples from 56 baboon hosts over 13 years) to infer thousands of correlations in bacterial abundance in individual baboons and test the degree to which bacterial abundance correlations are 'universal'. We also compare these patterns to two human data sets. We find that, most bacterial correlations are weak, negative, and universal across hosts, such that shared correlation patterns dominate over host-specific correlations by almost twofold. Further, taxon pairs that had inconsistent correlation signs (either positive or negative) in different hosts always had weak correlations within hosts. From the host perspective, host pairs with the most similar bacterial correlation patterns also had similar microbiome taxonomic compositions and tended to be genetic relatives. Compared to humans, universality in baboons was similar to that in human infants, and stronger than one data set from human adults. Bacterial families that showed universal correlations in human infants were often universal in baboons. Together, our work contributes new tools for analyzing the universality of bacterial associations across hosts, with implications for microbiome personalization, community assembly, and stability, and for designing microbiome interventions to improve host health.

BIRDMAn: A Bayesian differential abundance framework that enables robust inference of host-microbe associations

Quantifying the differential abundance (DA) of specific taxa among experimental groups in microbiome studies is challenging due to data characteristics (e.g., compositionality, sparsity) and specific study designs (e.g., repeated measures, meta-analysis, cross-over). Here we present BIRDMAn (Bayesian Inferential Regression for Differential Microbiome Analysis), a flexible DA method that can account for microbiome data characteristics and diverse experimental designs. Simulations show that BIRDMAn models are robust to uneven sequencing depth and provide a >20-fold improvement in statistical power over existing methods. We then use BIRDMAn to identify antibiotic-mediated perturbations undetected by other DA methods due to subject-level heterogeneity. Finally, we demonstrate how BIRDMAn can construct state-of-the-art cancer-type classifiers using The Cancer Genome Atlas (TCGA) dataset, with substantial accuracy improvements over random forests and existing DA tools across multiple sequencing centers. Collectively, BIRDMAn extracts more informative biological signals while accounting for study-specific experimental conditions than existing approaches.

MDITRE: Scalable and Interpretable Machine Learning for Predicting Host Status from Temporal Microbiome Dynamics

Longitudinal microbiome data sets are being generated with increasing regularity, and there is broad recognition that these studies are critical for unlocking the mechanisms through which the microbiome impacts human health and disease. However, there is a dearth of computational tools for analyzing microbiome time-series data. To address this gap, we developed an open-source software package, Microbiome Differentiable Interpretable Temporal Rule Engine (MDITRE), which implements a new highly efficient method leveraging deep-learning technologies to derive human-interpretable rules that predict host status from longitudinal microbiome data. Using semi-synthetic and a large compendium of publicly available 16S rRNA amplicon and metagenomics sequencing data sets, we demonstrate that in almost all cases, MDITRE performs on par with or better than popular uninterpretable machine learning methods, and orders-of-magnitude faster than the prior interpretable technique. MDITRE also provides a graphical user interface, which we show through case studies can be used to derive biologically meaningful interpretations linking patterns of microbiome changes over time with host phenotypes. IMPORTANCE The human microbiome, or collection of microbes living on and within us, changes over time. Linking these changes to the status of the human host is crucial to understanding how the microbiome influences a variety of human diseases. Due to the large scale and complexity of microbiome data, computational methods are essential. Existing computational methods for linking changes in the microbiome to the status of the human host are either unable to scale to large and complex microbiome data sets or cannot produce human-interpretable outputs. We present a new computational method and software package that overcomes the limitations of previous methods, allowing researchers to analyze larger and more complex data sets while producing easily interpretable outputs. Our method has the potential to enable new insights into how changes in the microbiome over time maintain health or lead to disease in humans and facilitate the development of diagnostic tests based on the microbiome.

MiSDEED: a synthetic data engine for microbiome study power analysis and study design

Summary

MiSDEED (Microbial Synthetic Data Engine for Experimental Design) is a command-line tool for generating synthetic longitudinal multinode data from simulated microbial environments. It generates relative-abundance timecourses under perturbations for an arbitrary number of time points, samples, locations and data types. All simulation parameters are exposed to the user to facilitate rapid power analysis and aid in study design. Users who want additional flexibility may also use MiSDEED as a Python package.

Availability and implementation

MiSDEED is written in Python and is freely available at https://github.com/pchlenski/misdeed.