Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data

Predicting S. aureus antimicrobial resistance with interpretable genomic space maps

Increasing antimicrobial resistance (AMR) represents a global healthcare threat. To decrease the spread of AMR and associated mortality, methods for rapid selection of optimal antibiotic treatment are urgently needed. Machine learning (ML) models based on genomic data to predict resistant phenotypes can serve as a fast screening tool prior to phenotypic testing. Nonetheless, many existing ML methods lack interpretability. Therefore, we present a methodology for visualization of sequence space and AMR prediction based on the non-linear dimensionality reduction method - generative topographic mapping (GTM). This approach, applied to AMR data of >5000 S. aureus isolates retrieved from the PATRIC database, yielded GTM models with reasonable accuracy for all drugs (balanced accuracy values ≥0.75). The Generative Topographic Maps (GTMs) represent data in the form of illustrative maps of the genomic space and allow for antibiotic-wise comparison of resistant phenotypes. The maps were also found to be useful for the analysis of genetic determinants responsible for drug resistance. Overall, the GTM-based methodology is a useful tool for both the illustrative exploration of the genomic sequence space and AMR prediction.

Tackling the Antimicrobial Resistance "Pandemic" with Machine Learning Tools: A Summary of Available Evidence

Antimicrobial resistance is recognised as one of the top threats healthcare is bound to face in the future. There have been various attempts to preserve the efficacy of existing antimicrobials, develop new and efficient antimicrobials, manage infections with multi-drug resistant strains, and improve patient outcomes, resulting in a growing mass of routinely available data, including electronic health records and microbiological information that can be employed to develop individualised antimicrobial stewardship. Machine learning methods have been developed to predict antimicrobial resistance from whole-genome sequencing data, forecast medication susceptibility, recognise epidemic patterns for surveillance purposes, or propose new antibacterial treatments and accelerate scientific discovery. Unfortunately, there is an evident gap between the number of machine learning applications in science and the effective implementation of these systems. This narrative review highlights some of the outstanding opportunities that machine learning offers when applied in research related to antimicrobial resistance. In the future, machine learning tools may prove to be superbugs' kryptonite. This review aims to provide an overview of available publications to aid researchers that are looking to expand their work with new approaches and to acquaint them with the current application of machine learning techniques in this field.

Machine learning and phylogenetic analysis allow for predicting antibiotic resistance in M. tuberculosis

BACKGROUND

Antimicrobial resistance (AMR) poses a significant global health threat, and an accurate prediction of bacterial resistance patterns is critical for effective treatment and control strategies. In recent years, machine learning (ML) approaches have emerged as powerful tools for analyzing large-scale bacterial AMR data. However, ML methods often ignore evolutionary relationships among bacterial strains, which can greatly impact performance of the ML methods, especially if resistance-associated features are attempted to be detected. Genome-wide association studies (GWAS) methods like linear mixed models accounts for the evolutionary relationships in bacteria, but they uncover only highly significant variants which have already been reported in literature.

RESULTS

In this work, we introduce a novel phylogeny-related parallelism score (PRPS), which measures whether a certain feature is correlated with the population structure of a set of samples. We demonstrate that PRPS can be used, in combination with SVM- and random forest-based models, to reduce the number of features in the analysis, while simultaneously increasing models' performance. We applied our pipeline to publicly available AMR data from PATRIC database for Mycobacterium tuberculosis against six common antibiotics.

CONCLUSIONS

Using our pipeline, we re-discovered known resistance-associated mutations as well as new candidate mutations which can be related to resistance and not previously reported in the literature. We demonstrated that taking into account phylogenetic relationships not only improves the model performance, but also yields more biologically relevant predicted most contributing resistance markers.

Contemporary Considerations for Establishing Reference Methods for Antibacterial Susceptibility Testing

Antibacterial susceptibility testing (AST) is performed to guide therapy, perform resistance surveillance studies, and support development of new antibacterial agents. For 5 decades, broth microdilution (BMD) has served as the reference method to assess in vitro activity of antibacterial agents against which both novel agents and diagnostic tests have been measured. BMD relies on in vitro inhibition or killing of bacteria. It is associated with several limitations: it is a poor mimic of the in vivo milieu of bacterial infections, requires multiple days to perform, and is associated with subtle, difficult to control variability. In addition, new reference methods will soon be needed for novel agents whose activity cannot be evaluated by BMD (e.g., those that target virulence). Any new reference methods must be standardized, correlated with clinical efficacy and be recognized internationally by researchers, industry, and regulators. Herein, we describe current reference methods for in vitro assessment of antibacterial activity and highlight key considerations for the generation of novel reference methods.

Metagenomic Antimicrobial Susceptibility Testing from Simulated Native Patient Samples

Genomic antimicrobial susceptibility testing (AST) has been shown to be accurate for many pathogens and antimicrobials. However, these methods have not been systematically evaluated for clinical metagenomic data. We investigate the performance of in-silico AST from clinical metagenomes (MG-AST). Using isolate sequencing data from a multi-center study on antimicrobial resistance (AMR) as well as shotgun-sequenced septic urine samples, we simulate over 2000 complicated urinary tract infection (cUTI) metagenomes with known resistance phenotype to 5 antimicrobials. Applying rule-based and machine learning-based genomic AST classifiers, we explore the impact of sequencing depth and technology, metagenome complexity, and bioinformatics processing approaches on AST accuracy. By using an optimized metagenomics assembly and binning workflow, MG-AST achieved balanced accuracy within 5.1% of isolate-derived genomic AST. For poly-microbial infections, taxonomic sample complexity and relatedness of taxa in the sample is a key factor influencing metagenomic binning and downstream MG-AST accuracy. We show that the reassignment of putative plasmid contigs by their predicted host range and investigation of whole resistome capabilities improved MG-AST performance on poly-microbial samples. We further demonstrate that machine learning-based methods enable MG-AST with superior accuracy compared to rule-based approaches on simulated native patient samples.

Machine learning models for Neisseria gonorrhoeae antimicrobial susceptibility tests

Neisseria gonorrhoeae is an urgent public health threat due to the emergence of antibiotic resistance. As most isolates in the United States are susceptible to at least one antibiotic, rapid molecular antimicrobial susceptibility tests (ASTs) would offer the opportunity to tailor antibiotic therapy, thereby expanding treatment options. With genome sequence and antibiotic resistance phenotype data for nearly 20,000 clinical N. gonorrhoeae isolates now available, there is an opportunity to use statistical methods to develop sequence-based diagnostics that predict antibiotic susceptibility from genotype. N. gonorrhoeae, therefore, provides a useful example illustrating how to apply machine learning models to aid in the design of sequence-based ASTs. We present an overview of this framework, which begins with establishing the assay technology, the performance criteria, the population in which the diagnostic will be used, and the clinical goals, and extends to the choices that must be made to arrive at a set of features with the desired properties for predicting susceptibility phenotype from genotype. While we focus on the example of N. gonorrhoeae, the framework generalizes to other organisms for which large-scale genotype and antibiotic resistance data can be combined to aid in diagnostics development.

Validation and Application of Long-Read Whole-Genome Sequencing for Antimicrobial Resistance Gene Detection and Antimicrobial Susceptibility Testing

Next-generation sequencing applications are increasingly used for detection and characterization of antimicrobial-resistant pathogens in clinical settings. Oxford Nanopore Technologies (ONT) sequencing offers advantages for clinical use compared with other sequencing methodologies because it enables real-time basecalling, produces long sequencing reads that increase the ability to correctly assemble DNA fragments, provides short turnaround times, and requires relatively uncomplicated sample preparation. A drawback of ONT sequencing, however, is its lower per-read accuracy than short-read sequencing. We sought to identify best practices in ONT sequencing protocols. As some variability in sequencing results may be introduced by the DNA extraction methodology, we tested three DNA extraction kits across three independent laboratories using a representative set of six bacterial isolates to investigate accuracy and reproducibility of ONT technology. All DNA extraction techniques showed comparable performance; however, the DNeasy PowerSoil Pro kit had the highest sequencing yield. This kit was subsequently applied to 42 sequentially collected bacterial isolates from blood cultures to assess Ares Genetics's pipelines for predictive whole-genome sequencing antimicrobial susceptibility testing (WGS-AST) performance compared to phenotypic triplicate broth microdilution results. WGS-AST results ranged across the organisms and resulted in an overall categorical agreement of 95% for penicillins, 82.4% for cephalosporins, 76.7% for carbapenems, 86.9% for fluoroquinolones, and 96.2% for aminoglycosides. Very major errors/major errors were 0%/16.7% (penicillins), 11.7%/3.6% (cephalosporins), 0%/24.4% (carbapenems), 2.5%/7.7% (fluoroquinolones), and 0%/4.1% (aminoglycosides), respectively. This work showed that, although additional refinements are necessary, ONT sequencing demonstrates potential as a method to perform WGS-AST on cultured isolates for patient care.

Automated antimicrobial susceptibility testing and antimicrobial resistance genotyping using Illumina and Oxford Nanopore Technologies sequencing data among Enterobacteriaceae

Whole-genome sequencing (WGS) enables the molecular characterization of bacterial pathogens. We compared the accuracy of the Illumina and Oxford Nanopore Technologies (ONT) sequencing platforms for the determination of AMR classes and antimicrobial susceptibility testing (AST) among 181 clinical Enterobacteriaceae isolates. Sequencing reads for each isolate were uploaded to AREScloud (Ares Genetics) to determine the presence of AMR markers and the predicted WGS-AST profile. The profiles of both sequencing platforms were compared to broth microdilution (BMD) AST. Isolates were delineated by resistance to third-generation cephalosporins and carbapenems as well as the presence of AMR markers to determine clinically relevant AMR classes. The overall categorical agreement (CA) was 90% (Illumina) and 88% (ONT) across all antimicrobials, 96% for the prediction of resistance to third-generation cephalosporins for both platforms, and 94% (Illumina) and 91% (ONT) for the prediction of resistance to carbapenems. Carbapenem resistance was overestimated on ONT with a major error of 16%. Sensitivity for the detection of carbapenemases, extended-spectrum β-lactamases, and plasmid-mediated ampC genes was 98, 95, and 70% by ONT compared to the Illumina dataset as the reference. Our results highlight the potential of the ONT platform’s use in clinical microbiology laboratories. When combined with robust bioinformatics methods, WGS-AST predictions may be a future approach to guide effective antimicrobial decision-making.

The Relative Power of Structural Genomic Variation versus SNPs in Explaining the Quantitative Trait Growth in the Marine Teleost Chrysophrys auratus

Background: Genetic diversity provides the basic substrate for evolution. Genetic variation consists of changes ranging from single base pairs (single-nucleotide polymorphisms, or SNPs) to larger-scale structural variants, such as inversions, deletions, and duplications. SNPs have long been used as the general currency for investigations into how genetic diversity fuels evolution. However, structural variants can affect more base pairs in the genome than SNPs and can be responsible for adaptive phenotypes due to their impact on linkage and recombination. In this study, we investigate the first steps needed to explore the genetic basis of an economically important growth trait in the marine teleost finfish Chrysophrys auratus using both SNP and structural variant data. Specifically, we use feature selection methods in machine learning to explore the relative predictive power of both types of genetic variants in explaining growth and discuss the feature selection results of the evaluated methods. Methods: SNP and structural variant callers were used to generate catalogues of variant data from 32 individual fish at ages 1 and 3 years. Three feature selection algorithms (ReliefF, Chi-square, and a mutual-information-based method) were used to reduce the dataset by selecting the most informative features. Following this selection process, the subset of variants was used as features to classify fish into small, medium, or large size categories using KNN, naïve Bayes, random forest, and logistic regression. The top-scoring features in each feature selection method were subsequently mapped to annotated genomic regions in the zebrafish genome, and a permutation test was conducted to see if the number of mapped regions was greater than when random sampling was applied. Results: Without feature selection, the prediction accuracies ranged from 0 to 0.5 for both structural variants and SNPs. Following feature selection, the prediction accuracy increased only slightly to between 0 and 0.65 for structural variants and between 0 and 0.75 for SNPs. The highest prediction accuracy for the logistic regression was achieved for age 3 fish using SNPs, although generally predictions for age 1 and 3 fish were very similar (ranging from 0–0.65 for both SNPs and structural variants). The Chi-square feature selection of SNP data was the only method that had a significantly higher number of matches to annotated genomic regions of zebrafish than would be explained by chance alone. Conclusions: Predicting a complex polygenic trait such as growth using data collected from a low number of individuals remains challenging. While we demonstrate that both SNPs and structural variants provide important information to help understand the genetic basis of phenotypic traits such as fish growth, the full complexities that exist within a genome cannot be easily captured by classical machine learning techniques. When using high-dimensional data, feature selection shows some increase in the prediction accuracy of classification models and provides the potential to identify unknown genomic correlates with growth. Our results show that both SNPs and structural variants significantly impact growth, and we therefore recommend that researchers interested in the genotype–phenotype map should strive to go beyond SNPs and incorporate structural variants in their studies as well. We discuss how our machine learning models can be further expanded to serve as a test bed to inform evolutionary studies and the applied management of species.

Genome-Wide Mutation Scoring for Machine-Learning-Based Antimicrobial Resistance Prediction

The prediction of antimicrobial resistance (AMR) based on genomic information can improve patient outcomes. Genetic mechanisms have been shown to explain AMR with accuracies in line with standard microbiology laboratory testing. To translate genetic mechanisms into phenotypic AMR, machine learning has been successfully applied. AMR machine learning models typically use nucleotide k-mer counts to represent genomic sequences. While k-mer representation efficiently captures sequence variation, it also results in high-dimensional and sparse data. With limited training data available, achieving acceptable model performance or model interpretability is challenging. In this study, we explore the utility of feature engineering with several biologically relevant signals. We propose to predict the functional impact of observed mutations with PROVEAN to use the predicted impact as a new feature for each protein in an organism’s proteome. The addition of the new features was tested on a total of 19,521 isolates across nine clinically relevant pathogens and 30 different antibiotics. The new features significantly improved the predictive performance of trained AMR models for Pseudomonas aeruginosa, Citrobacter freundii, and Escherichia coli. The balanced accuracy of the respective models of those three pathogens improved by 6.0% on average.

A genomic data resource for predicting antimicrobial resistance from laboratory-derived antimicrobial susceptibility phenotypes

Antimicrobial resistance (AMR) is a major global health threat that affects millions of people each year. Funding agencies worldwide and the global research community have expended considerable capital and effort tracking the evolution and spread of AMR by isolating and sequencing bacterial strains and performing antimicrobial susceptibility testing (AST). For the last several years, we have been capturing these efforts by curating data from the literature and data resources and building a set of assembled bacterial genome sequences that are paired with laboratory-derived AST data. This collection currently contains AST data for over 67 000 genomes encompassing approximately 40 genera and over 100 species. In this paper, we describe the characteristics of this collection, highlighting areas where sampling is comparatively deep or shallow, and showing areas where attention is needed from the research community to improve sampling and tracking efforts. In addition to using the data to track the evolution and spread of AMR, it also serves as a useful starting point for building machine learning models for predicting AMR phenotypes. We demonstrate this by describing two machine learning models that are built from the entire dataset to show where the predictive power is comparatively high or low. This AMR metadata collection is freely available and maintained on the Bacterial and Viral Bioinformatics Center (BV-BRC) FTP site ftp://ftp.bvbrc.org/RELEASE_NOTES/PATRIC_genomes_AMR.txt.

Core Genome Multilocus Sequence Typing and Prediction of Antimicrobial Susceptibility Using Whole-Genome Sequences of Escherichia coli Bloodstream Infection Isolates

In total, 50 Escherichia coli bloodstream isolates from the clinical laboratory and 12 E. coli isolates referred for pulsed-field gel electrophoresis (PFGE) were sequenced, assessed for clonality using core genome multilocus sequence typing (cgMLST), and evaluated for genomic susceptibility predictions using ARESdb. Results of sequence typing using whole-genome sequencing (WGS)-based MLST and sequence type (ST)-specific PCR were identical. Overall categorical agreement between genotypic (ARESdb) and phenotypic susceptibility testing for 62 isolates and 11 antimicrobial agents was 91%. Among the referred isolates, high major error rates were found for ceftazidime, cefepime, and piperacillin-tazobactam.

Predictive Antibiotic Susceptibility Testing by Next-Generation Sequencing for Periprosthetic Joint Infections: Potential and Limitations

Joint replacement surgeries are one of the most frequent medical interventions globally. Infections of prosthetic joints are a major health challenge and typically require prolonged or even indefinite antibiotic treatment. As multidrug-resistant pathogens continue to rise globally, novel diagnostics are critical to ensure appropriate treatment and help with prosthetic joint infections (PJI) management. To this end, recent studies have shown the potential of molecular methods such as next-generation sequencing to complement established phenotypic, culture-based methods. Together with advanced bioinformatics approaches, next-generation sequencing can provide comprehensive information on pathogen identity as well as antimicrobial susceptibility, potentially enabling rapid diagnosis and targeted therapy of PJIs. In this review, we summarize current developments in next generation sequencing based predictive antibiotic susceptibility testing and discuss potential and limitations for common PJI pathogens.