Benchmarking taxonomic classifiers with Illumina and Nanopore sequence data for clinical metagenomic diagnostic applications

Evaluating metagenomics and targeted approaches for diagnosis and surveillance of viruses

BACKGROUND

Metagenomics is a powerful approach for the detection of unknown and novel pathogens. Workflows based on Illumina short-read sequencing are becoming established in diagnostic laboratories. However, high sequencing depth requirements, long turnaround times, and limited sensitivity hinder broader adoption. We investigated whether we could overcome these limitations using protocols based on untargeted sequencing with Oxford Nanopore Technologies (ONT), which offers real-time data acquisition and analysis, or a targeted panel approach, which allows the selective sequencing of known pathogens and could improve sensitivity.

METHODS

We evaluated detection of viruses with readily available untargeted metagenomic workflows using Illumina and ONT, and an Illumina-based enrichment approach using the Twist Bioscience Comprehensive Viral Research Panel (CVRP), which targets 3153 viruses. We tested samples consisting of a dilution series of a six-virus mock community in a human DNA/RNA background, designed to resemble clinical specimens with low microbial abundance and high host content. Protocols were designed to retain the host transcriptome, since this could help confirm the absence of infectious agents. We further compared the performance of commonly used taxonomic classifiers.

RESULTS

Capture with the Twist CVRP increased sensitivity by at least 10-100-fold over untargeted sequencing, making it suitable for the detection of low viral loads (60 genome copies per ml (gc/ml)), but additional methods may be needed in a diagnostic setting to detect untargeted organisms. While untargeted ONT had good sensitivity at high viral loads (60,000 gc/ml), at lower viral loads (600-6000 gc/ml), longer and more costly sequencing runs would be required to achieve sensitivities comparable to the untargeted Illumina protocol. Untargeted ONT provided better specificity than untargeted Illumina sequencing. However, the application of robust thresholds standardized results between taxonomic classifiers. Host gene expression analysis is optimal with untargeted Illumina sequencing but possible with both the CVRP and ONT.

CONCLUSIONS

Metagenomics has the potential to become standard-of-care in diagnostics and is a powerful tool for the discovery of emerging pathogens. Untargeted Illumina and ONT metagenomics and capture with the Twist CVRP have different advantages with respect to sensitivity, specificity, turnaround time and cost, and the optimal method will depend on the clinical context.

Benchmarking bacterial taxonomic classification using nanopore metagenomics data of several mock communities

Taxonomic classification is crucial in identifying organisms within diverse microbial communities when using metagenomics shotgun sequencing. While second-generation Illumina sequencing still dominates, third-generation nanopore sequencing promises improved classification through longer reads. However, extensive benchmarking studies on nanopore data are lacking. We systematically evaluated performance of bacterial taxonomic classification for metagenomics nanopore sequencing data for several commonly used classifiers, using standardized reference sequence databases, on the largest collection of publicly available data for defined mock communities thus far (nine samples), representing different research domains and application scopes. Our results categorize classifiers into three categories: low precision/high recall; medium precision/medium recall, and high precision/medium recall. Most fall into the first group, although precision can be improved without excessively penalizing recall with suitable abundance filtering. No definitive 'best' classifier emerges, and classifier selection depends on application scope and practical requirements. Although few classifiers designed for long reads exist, they generally exhibit better performance. Our comprehensive benchmarking provides concrete recommendations, supported by publicly available code for reassessment and fine-tuning by other scientists.

An in-depth evaluation of metagenomic classifiers for soil microbiomes

BACKGROUND

Recent endeavours in metagenomics, exemplified by projects such as the human microbiome project and TARA Oceans, have illuminated the complexities of microbial biomes. A robust bioinformatic pipeline and meticulous evaluation of their methodology have contributed to the success of these projects. The soil environment, however, with its unique challenges, requires a specialized methodological exploration to maximize microbial insights. A notable limitation in soil microbiome studies is the dearth of soil-specific reference databases available to classifiers that emulate the complexity of soil communities. There is also a lack of in-vitro mock communities derived from soil strains that can be assessed for taxonomic classification accuracy.

RESULTS

In this study, we generated a custom in-silico mock community containing microbial genomes commonly observed in the soil microbiome. Using this mock community, we simulated shotgun sequencing data to evaluate the performance of three leading metagenomic classifiers: Kraken2 (supplemented with Bracken, using a custom database derived from GTDB-TK genomes along with its own default database), Kaiju, and MetaPhlAn, utilizing their respective default databases for a robust analysis. Our results highlight the importance of optimizing taxonomic classification parameters, database selection, as well as analysing trimmed reads and contigs. Our study showed that classifiers tailored to the specific taxa present in our samples led to fewer errors compared to broader databases including microbial eukaryotes, protozoa, or human genomes, highlighting the effectiveness of targeted taxonomic classification. Notably, an optimal classifier performance was achieved when applying a relative abundance threshold of 0.001% or 0.005%. The Kraken2 supplemented with bracken, with a custom database demonstrated superior precision, sensitivity, F1 score, and overall sequence classification. Using a custom database, this classifier classified 99% of in-silico reads and 58% of real-world soil shotgun reads, with the latter identifying previously overlooked phyla using a custom database.

CONCLUSION

This study underscores the potential advantages of in-silico methodological optimization in metagenomic analyses, especially when deciphering the complexities of soil microbiomes. We demonstrate that the choice of classifier and database significantly impacts microbial taxonomic profiling. Our findings suggest that employing Kraken2 with Bracken, coupled with a custom database of GTDB-TK genomes and fungal genomes at a relative abundance threshold of 0.001% provides optimal accuracy in soil shotgun metagenome analysis.

Interpretation of vaginal metagenomic characteristics in different types of vaginitis

Although vaginitis is closely related to vaginal microecology in females, the precise composition and functional potential of different types of vaginitis remain unclear. Here, metagenomic sequencing was applied to analyze the vaginal flora in patients with various forms of vaginitis, including cases with a clue cell proportion ranging from 1% to 20% (Clue1_20), bacterial vaginitis (BV), vulvovaginal candidiasis (VVC), and BV combined with VVC (VVC_BV). Our results identified Prevotella as an important biomarker between BV and Clue1_20. Moreover, a gradual decrease was observed in the relative abundance of shikimic acid metabolism associated with bacteria producing indole as well as a decline in the abundance of Gardnerella vaginalis in patients with BV, Clue1_20, and healthy women. Interestingly, the vaginal flora of patients in the VVC_BV group exhibited structural similarities to that of the VVC group, and its potentially functional characteristics resembled those of the BV and VVC groups. Finally, Lactobacillus crispatus was found in high abundance in healthy samples, greatly contributing to the stability of the vaginal environment. For the further study of L. crispatus, we isolated five strains of L. crispatus from healthy samples and evaluated their capacity to inhibit G. vaginalis biofilms and produce lactic acid in vitro to select the potential probiotic candidate for improving vaginitis in future clinical studies. Overall, we successfully identified bacterial biomarkers of different vaginitis and characterized the dynamic shifts in vaginal flora between patients with BV and healthy females. This research advances our understanding and holds great promise in enhancing clinical approaches for the treatment of vaginitis.

IMPORTANCE

Vaginitis is one of the most common gynecological diseases, mostly caused by infections of pathogens such as Candida albicans and Gardnerella vaginalis. In recent years, it has been found that the stability of the vaginal flora plays an important role in vaginitis. Furthermore, the abundant Lactobacillus-producing rich lactic acid in the vagina provides a healthy acidic environment such as Lactobacillus crispatus. The metabolites of Lactobacillus can inhibit the colonization of pathogens. Here, we collected the vaginal samples of patients with bacterial vaginitis (BV), vulvovaginal candidiasis (VVC), and BV combined with VVC to discover the differences and relationships among the different kinds of vaginitis by metagenomic sequencing. Furthermore, because of the importance of L. crispatus in promoting vaginal health, we isolated multiple strains from vaginal samples of healthy females and chose the most promising strain with potential probiotic benefits to provide clinical implications for treatment strategies.

Unveiling the microbiome during post-partum uterine infection: a deep shotgun sequencing approach to characterize the dairy cow uterine microbiome

BACKGROUND

The goal of this study was to assess the microbial ecology and diversity present in the uterus of post-partum dairy cows with and without metritis from 24 commercial California dairy farms using shotgun metagenomics. A set subset of 95 intrauterine swab samples, taken from a larger selection of 307 individual cow samples previously collected, were examined for α and β diversity and differential abundance associated with metritis. Cows within 21 days post-partum were categorized into one of three clinical groups during sample collection: control (CT, n = 32), defined as cows with either no vaginal discharge or a clear, non-purulent mucus vaginal discharge; metritis (MET, n = 33), defined as a cow with watery, red or brown colored, and fetid vaginal discharge; and purulent discharge cows (PUS, n = 31), defined as a non-fetid purulent or mucopurulent vaginal discharge.

RESULTS

All three clinical groups (CT, MET, and PUS) were highly diverse, with the top 12 most abundant genera accounting for 10.3%, 8.8%, and 10.1% of mean relative abundance, respectively. The α diversity indices revealed a lower diversity from samples collected from MET and PUS when compared to CT cows. PERMANOVA statistical testing revealed a significant difference (P adjusted < 0.01) in the diversity of genera between CT and MET samples (R2 = 0.112, P = 0.003) and a non-significant difference between MET and PUS samples (R2 = 0.036, P = 0.046). ANCOM-BC analysis revealed that from the top 12 most abundant genera, seven genera were increased in the natural log fold change (LFC) of abundance in MET when compared to CT samples: Bacteroides, Clostridium, Fusobacterium, Phocaeicola, Porphyromonas, Prevotella, and Streptococcus. Two genera, Dietzia and Microbacterium, were decreased in natural LFC of abundance when comparing MET (regardless of treatment) and CT, while no changes in natural LFC of abundance were observed for Escherichia, Histophilus, and Trueperella.

CONCLUSIONS

The results presented here, are the current deepest shotgun metagenomic analyses conducted on the bovine uterine microbiome to date (mean of 256,425 genus-level reads per sample). Our findings support that uterine samples from cows without metritis (CT) had increased α-diversity but decreased β-diversity when compared to metritis or PUS cows, characteristic of dysbiosis. In summary, our findings highlight that MET cows have an increased abundance of Bacteroides, Porphyromonas, and Fusobacterium when compared to CT and PUS, and support the need for further studies to better understand their potential causal role in metritis pathogenesis.

Rapid metagenomic sequencing for diagnosis and antimicrobial sensitivity prediction of canine bacterial infections

Antimicrobial resistance is a major threat to human and animal health. There is an urgent need to ensure that antimicrobials are used appropriately to limit the emergence and impact of resistance. In the human and veterinary healthcare setting, traditional culture and antimicrobial sensitivity testing typically requires 48-72 h to identify appropriate antibiotics for treatment. In the meantime, broad-spectrum antimicrobials are often used, which may be ineffective or impact non-target commensal bacteria. Here, we present a rapid, culture-free, diagnostics pipeline, involving metagenomic nanopore sequencing directly from clinical urine and skin samples of dogs. We have planned this pipeline to be versatile and easily implementable in a clinical setting, with the potential for future adaptation to different sample types and animals. Using our approach, we can identify the bacterial pathogen present within 5 h, in some cases detecting species which are difficult to culture. For urine samples, we can predict antibiotic sensitivity with up to 95 % accuracy. Skin swabs usually have lower bacterial abundance and higher host DNA, confounding antibiotic sensitivity prediction; an additional host depletion step will likely be required during the processing of these, and other types of samples with high levels of host cell contamination. In summary, our pipeline represents an important step towards the design of individually tailored veterinary treatment plans on the same day as presentation, facilitating the effective use of antibiotics and promoting better antimicrobial stewardship.