Evaluation of Machine Learning and Rules-Based Approaches for Predicting Antimicrobial Resistance Profiles in Gram-negative Bacilli from Whole Genome Sequence Data

Keeping up with the pathogens: improved antimicrobial resistance detection and prediction from Pseudomonas aeruginosa genomes

BACKGROUND

Antimicrobial resistance (AMR) is an intensifying threat that requires urgent mitigation to avoid a post-antibiotic era. Pseudomonas aeruginosa represents one of the greatest AMR concerns due to increasing multi- and pan-drug resistance rates. Shotgun sequencing is gaining traction for in silico AMR profiling due to its unambiguity and transferability; however, accurate and comprehensive AMR prediction from P. aeruginosa genomes remains an unsolved problem.

METHODS

We first curated the most comprehensive database yet of known P. aeruginosa AMR variants. Next, we performed comparative genomics and microbial genome-wide association study analysis across a Global isolate Dataset (n = 1877) with paired antimicrobial phenotype and genomic data to identify novel AMR variants. Finally, the performance of our P. aeruginosa AMR database, implemented in our AMR detection and prediction tool, ARDaP, was compared with three previously published in silico AMR gene detection or phenotype prediction tools-abritAMR, AMRFinderPlus, ResFinder-across both the Global Dataset and an analysis-naïve Validation Dataset (n = 102).

RESULTS

Our AMR database comprises 3639 mobile AMR genes and 728 chromosomal variants, including 75 previously unreported chromosomal AMR variants, 10 variants associated with unusual antimicrobial susceptibility, and 281 chromosomal variants that we show are unlikely to confer AMR. Our pipeline achieved a genotype-phenotype balanced accuracy (bACC) of 85% and 81% across 10 clinically relevant antibiotics when tested against the Global and Validation Datasets, respectively, vs. just 56% and 54% with abritAMR, 58% and 54% with AMRFinderPlus, and 60% and 53% with ResFinder. ARDaP's superior performance was predominantly due to the inclusion of chromosomal AMR variants, which are generally not identified with most AMR identification tools.

CONCLUSIONS

Our ARDaP software and associated AMR variant database provides an accurate tool for predicting AMR phenotypes in P. aeruginosa, far surpassing the performance of current tools. Implementation of ARDaP for routine AMR prediction from P. aeruginosa genomes and metagenomes will improve AMR identification, addressing a critical facet in combatting this treatment-refractory pathogen. However, knowledge gaps remain in our understanding of the P. aeruginosa resistome, particularly the basis of colistin AMR.

Tackling the Antimicrobial Resistance "Pandemic" with Machine Learning Tools: A Summary of Available Evidence

Antimicrobial resistance is recognised as one of the top threats healthcare is bound to face in the future. There have been various attempts to preserve the efficacy of existing antimicrobials, develop new and efficient antimicrobials, manage infections with multi-drug resistant strains, and improve patient outcomes, resulting in a growing mass of routinely available data, including electronic health records and microbiological information that can be employed to develop individualised antimicrobial stewardship. Machine learning methods have been developed to predict antimicrobial resistance from whole-genome sequencing data, forecast medication susceptibility, recognise epidemic patterns for surveillance purposes, or propose new antibacterial treatments and accelerate scientific discovery. Unfortunately, there is an evident gap between the number of machine learning applications in science and the effective implementation of these systems. This narrative review highlights some of the outstanding opportunities that machine learning offers when applied in research related to antimicrobial resistance. In the future, machine learning tools may prove to be superbugs' kryptonite. This review aims to provide an overview of available publications to aid researchers that are looking to expand their work with new approaches and to acquaint them with the current application of machine learning techniques in this field.

Assessing computational predictions of antimicrobial resistance phenotypes from microbial genomes

The advent of rapid whole-genome sequencing has created new opportunities for computational prediction of antimicrobial resistance (AMR) phenotypes from genomic data. Both rule-based and machine learning (ML) approaches have been explored for this task, but systematic benchmarking is still needed. Here, we evaluated four state-of-the-art ML methods (Kover, PhenotypeSeeker, Seq2Geno2Pheno and Aytan-Aktug), an ML baseline and the rule-based ResFinder by training and testing each of them across 78 species-antibiotic datasets, using a rigorous benchmarking workflow that integrates three evaluation approaches, each paired with three distinct sample splitting methods. Our analysis revealed considerable variation in the performance across techniques and datasets. Whereas ML methods generally excelled for closely related strains, ResFinder excelled for handling divergent genomes. Overall, Kover most frequently ranked top among the ML approaches, followed by PhenotypeSeeker and Seq2Geno2Pheno. AMR phenotypes for antibiotic classes such as macrolides and sulfonamides were predicted with the highest accuracies. The quality of predictions varied substantially across species-antibiotic combinations, particularly for beta-lactams; across species, resistance phenotyping of the beta-lactams compound, aztreonam, amoxicillin/clavulanic acid, cefoxitin, ceftazidime and piperacillin/tazobactam, alongside tetracyclines demonstrated more variable performance than the other benchmarked antibiotics. By organism, Campylobacter jejuni and Enterococcus faecium phenotypes were more robustly predicted than those of Escherichia coli, Staphylococcus aureus, Salmonella enterica, Neisseria gonorrhoeae, Klebsiella pneumoniae, Pseudomonas aeruginosa, Acinetobacter baumannii, Streptococcus pneumoniae and Mycobacterium tuberculosis. In addition, our study provides software recommendations for each species-antibiotic combination. It furthermore highlights the need for optimization for robust clinical applications, particularly for strains that diverge substantially from those used for training.

Generalizability of machine learning in predicting antimicrobial resistance in E. coli: a multi-country case study in Africa

BACKGROUND

Antimicrobial resistance (AMR) remains a significant global health threat particularly impacting low- and middle-income countries (LMICs). These regions often grapple with limited healthcare resources and access to advanced diagnostic tools. Consequently, there is a pressing need for innovative approaches that can enhance AMR surveillance and management. Machine learning (ML) though underutilized in these settings, presents a promising avenue. This study leverages ML models trained on whole-genome sequencing data from England, where such data is more readily available, to predict AMR in E. coli, targeting key antibiotics such as ciprofloxacin, ampicillin, and cefotaxime. A crucial part of our work involved the validation of these models using an independent dataset from Africa, specifically from Uganda, Nigeria, and Tanzania, to ascertain their applicability and effectiveness in LMICs.

RESULTS

Model performance varied across antibiotics. The Support Vector Machine excelled in predicting ciprofloxacin resistance (87% accuracy, F1 Score: 0.57), Light Gradient Boosting Machine for cefotaxime (92% accuracy, F1 Score: 0.42), and Gradient Boosting for ampicillin (58% accuracy, F1 Score: 0.66). In validation with data from Africa, Logistic Regression showed high accuracy for ampicillin (94%, F1 Score: 0.97), while Random Forest and Light Gradient Boosting Machine were effective for ciprofloxacin (50% accuracy, F1 Score: 0.56) and cefotaxime (45% accuracy, F1 Score:0.54), respectively. Key mutations associated with AMR were identified for these antibiotics.

CONCLUSION

As the threat of AMR continues to rise, the successful application of these models, particularly on genomic datasets from LMICs, signals a promising avenue for improving AMR prediction to support large AMR surveillance programs. This work thus not only expands our current understanding of the genetic underpinnings of AMR but also provides a robust methodological framework that can guide future research and applications in the fight against AMR.

Machine learning and feature extraction for rapid antimicrobial resistance prediction of Acinetobacter baumannii from whole-genome sequencing data

Background

Whole-genome sequencing (WGS) has contributed significantly to advancements in machine learning methods for predicting antimicrobial resistance (AMR). However, the comparisons of different methods for AMR prediction without requiring prior knowledge of resistance remains to be conducted.

Methods

We aimed to predict the minimum inhibitory concentrations (MICs) of 13 antimicrobial agents against Acinetobacter baumannii using three machine learning algorithms (random forest, support vector machine, and XGBoost) combined with k-mer features extracted from WGS data.

Results

A cohort of 339 isolates was used for model construction. The average essential agreement and category agreement of the best models exceeded 90.90% (95%CI, 89.03-92.77%) and 95.29% (95%CI, 94.91-95.67%), respectively; the exceptions being levofloxacin, minocycline and imipenem. The very major error rates ranged from 0.0 to 5.71%. We applied feature selection pipelines to extract the top-ranked 11-mers to optimise training time and computing resources. This approach slightly improved the prediction performance and enabled us to obtain prediction results within 10 min. Notably, when employing these top-ranked 11-mers in an independent test dataset (120 isolates), we achieved an average accuracy of 0.96.

Conclusion

Our study is the first to demonstrate that AMR prediction for A. baumannii using machine learning methods based on k-mer features has competitive performance over traditional workflows; hence, sequence-based AMR prediction and its application could be further promoted. The k-mer-based workflow developed in this study demonstrated high recall/sensitivity and specificity, making it a dependable tool for MIC prediction in clinical settings.

Predicting Salmonella MIC and Deciphering Genomic Determinants of Antibiotic Resistance and Susceptibility

Salmonella spp., a leading cause of foodborne illness, is a formidable global menace due to escalating antimicrobial resistance (AMR). The evaluation of minimum inhibitory concentration (MIC) for antimicrobials is critical for characterizing AMR. The current whole genome sequencing (WGS)-based approaches for predicting MIC are hindered by both computational and feature identification constraints. We propose an innovative methodology called the "Genome Feature Extractor Pipeline" that integrates traditional machine learning (random forest, RF) with deep learning models (multilayer perceptron (MLP) and DeepLift) for WGS-based MIC prediction. We used a dataset from the National Antimicrobial Resistance Monitoring System (NARMS), comprising 4500 assembled genomes of nontyphoidal Salmonella, each annotated with MIC metadata for 15 antibiotics. Our pipeline involves the batch downloading of annotated genomes, the determination of feature importance using RF, Gini-index-based selection of crucial 10-mers, and their expansion to 20-mers. This is followed by an MLP network, with four hidden layers of 1024 neurons each, to predict MIC values. Using DeepLift, key 20-mers and associated genes influencing MIC are identified. The 10 most significant 20-mers for each antibiotic are listed, showcasing our ability to discern genomic features affecting Salmonella MIC prediction with enhanced precision. The methodology replaces binary indicators with k-mer counts, offering a more nuanced analysis. The combination of RF and MLP addresses the limitations of the existing WGS approach, providing a robust and efficient method for predicting MIC values in Salmonella that could potentially be applied to other pathogens.

Predictive modeling of antibiotic eradication therapy success for new-onset Pseudomonas aeruginosa pulmonary infections in children with cystic fibrosis

Chronic Pseudomonas aeruginosa (Pa) lung infections are the leading cause of mortality among cystic fibrosis (CF) patients; therefore, the eradication of new-onset Pa lung infections is an important therapeutic goal that can have long-term health benefits. The use of early antibiotic eradication therapy (AET) has been shown to clear the majority of new-onset Pa infections, and it is hoped that identifying the underlying basis for AET failure will further improve treatment outcomes. Here we generated machine learning models to predict AET outcomes based on pathogen genomic data. We used a nested cross validation design, population structure control, and recursive feature selection to improve model performance and showed that incorporating population structure control was crucial for improving model interpretation and generalizability. Our best model, controlling for population structure and using only 30 recursively selected features, had an area under the curve of 0.87 for a holdout test dataset. The top-ranked features were generally associated with motility, adhesion, and biofilm formation.

An accurate and interpretable model for antimicrobial resistance in pathogenic Escherichia coli from livestock and companion animal species

Understanding the microbial genomic contributors to antimicrobial resistance (AMR) is essential for early detection of emerging AMR infections, a pressing global health threat in human and veterinary medicine. Here we used whole genome sequencing and antibiotic susceptibility test data from 980 disease causing Escherichia coli isolated from companion and farm animals to model AMR genotypes and phenotypes for 24 antibiotics. We determined the strength of genotype-to-phenotype relationships for 197 AMR genes with elastic net logistic regression. Model predictors were designed to evaluate different potential modes of AMR genotype translation into resistance phenotypes. Our results show a model that considers the presence of individual AMR genes and total number of AMR genes present from a set of genes known to confer resistance was able to accurately predict isolate resistance on average (mean F1 score = 98.0%, SD = 2.3%, mean accuracy = 98.2%, SD = 2.7%). However, fitted models sometimes varied for antibiotics in the same class and for the same antibiotic across animal hosts, suggesting heterogeneity in the genetic determinants of AMR resistance. We conclude that an interpretable AMR prediction model can be used to accurately predict resistance phenotypes across multiple host species and reveal testable hypotheses about how the mechanism of resistance may vary across antibiotics within the same class and across animal hosts for the same antibiotic.

Predicting antimicrobial resistance of bacterial pathogens using time series analysis

Antimicrobial resistance (AMR) is arguably one of the major health and economic challenges in our society. A key aspect of tackling AMR is rapid and accurate detection of the emergence and spread of AMR in food animal production, which requires routine AMR surveillance. However, AMR detection can be expensive and time-consuming considering the growth rate of the bacteria and the most commonly used analytical procedures, such as Minimum Inhibitory Concentration (MIC) testing. To mitigate this issue, we utilized machine learning to predict the future AMR burden of bacterial pathogens. We collected pathogen and antimicrobial data from >600 farms in the United States from 2010 to 2021 to generate AMR time series data. Our prediction focused on five bacterial pathogens (Escherichia coli, Streptococcus suis, Salmonella sp., Pasteurella multocida, and Bordetella bronchiseptica). We found that Seasonal Auto-Regressive Integrated Moving Average (SARIMA) outperformed five baselines, including Auto-Regressive Moving Average (ARMA) and Auto-Regressive Integrated Moving Average (ARIMA). We hope this study provides valuable tools to predict the AMR burden not only of the pathogens assessed in this study but also of other bacterial pathogens.

eMIC-AntiKP: Estimating minimum inhibitory concentrations of antibiotics towards Klebsiella pneumoniae using deep learning

Nowadays, antibiotic resistance has become one of the most concerning problems that directly affects the recovery process of patients. For years, numerous efforts have been made to efficiently use antimicrobial drugs with appropriate doses not only to exterminate microbes but also stringently constrain any chances for bacterial evolution. However, choosing proper antibiotics is not a straightforward and time-effective process because well-defined drugs can only be given to patients after determining microbic taxonomy and evaluating minimum inhibitory concentrations (MICs). Besides conventional methods, numerous computer-aided frameworks have been recently developed using computational advances and public data sources of clinical antimicrobial resistance. In this study, we introduce eMIC-AntiKP, a computational framework specifically designed to predict the MIC values of 20 antibiotics towards Klebsiella pneumoniae. Our prediction models were constructed using convolutional neural networks and k-mer counting-based features. The model for cefepime has the most limited performance with a test 1-tier accuracy of 0.49, while the model for ampicillin has the highest performance with a test 1-tier accuracy of 1.00. Most models have satisfactory performance, with test accuracies ranging from about 0.70-0.90. The significance of eMIC-AntiKP is the effective utilization of computing resources to make it a compact and portable tool for most moderately configured computers. We provide users with two options, including an online web server for basic analysis and an offline package for deeper analysis and technical modification.

Paving the way for precise diagnostics of antimicrobial resistant bacteria

The antimicrobial resistance (AMR) crisis from bacterial pathogens is frequently emerging and rapidly disseminated during the sustained antimicrobial exposure in human-dominated communities, posing a compelling threat as one of the biggest challenges in humans. The frequent incidences of some common but untreatable infections unfold the public health catastrophe that antimicrobial-resistant pathogens have outpaced the available countermeasures, now explicitly amplified during the COVID-19 pandemic. Nowadays, biotechnology and machine learning advancements help create more fundamental knowledge of distinct spatiotemporal dynamics in AMR bacterial adaptation and evolutionary processes. Integrated with reliable diagnostic tools and powerful analytic approaches, a collaborative and systematic surveillance platform with high accuracy and predictability should be established and implemented, which is not just for an effective controlling strategy on AMR but also for protecting the longevity of valuable antimicrobials currently and in the future.

Towards a robust out-of-the-box neural network model for genomic data

Background

The accurate prediction of biological features from genomic data is paramount for precision medicine and sustainable agriculture. For decades, neural network models have been widely popular in fields like computer vision, astrophysics and targeted marketing given their prediction accuracy and their robust performance under big data settings. Yet neural network models have not made a successful transition into the medical and biological world due to the ubiquitous characteristics of biological data such as modest sample sizes, sparsity, and extreme heterogeneity.

Results

Here, we investigate the robustness, generalization potential and prediction accuracy of widely used convolutional neural network and natural language processing models with a variety of heterogeneous genomic datasets. Mainly, recurrent neural network models outperform convolutional neural network models in terms of prediction accuracy, overfitting and transferability across the datasets under study.

Conclusions

While the perspective of a robust out-of-the-box neural network model is out of reach, we identify certain model characteristics that translate well across datasets and could serve as a baseline model for translational researchers.

Towards routine employment of computational tools for antimicrobial resistance determination via high-throughput sequencing

Antimicrobial resistance (AMR) is a growing threat to public health and farming at large. In clinical and veterinary practice, timely characterization of the antibiotic susceptibility profile of bacterial infections is a crucial step in optimizing treatment. High-throughput sequencing is a promising option for clinical point-of-care and ecological surveillance, opening the opportunity to develop genotyping-based AMR determination as a possibly faster alternative to phenotypic testing. In the present work, we compare the performance of state-of-the-art methods for detection of AMR using high-throughput sequencing data from clinical settings. We consider five computational approaches based on alignment (AMRPlusPlus), deep learning (DeepARG), k-mer genomic signatures (KARGA, ResFinder) or hidden Markov models (Meta-MARC). We use an extensive collection of 585 isolates with available AMR resistance profiles determined by phenotypic tests across nine antibiotic classes. We show how the prediction landscape of AMR classifiers is highly heterogeneous, with balanced accuracy varying from 0.40 to 0.92. Although some algorithms-ResFinder, KARGA and AMRPlusPlus-exhibit overall better balanced accuracy than others, the high per-AMR-class variance and related findings suggest that: (1) all algorithms might be subject to sampling bias both in data repositories used for training and experimental/clinical settings; and (2) a portion of clinical samples might contain uncharacterized AMR genes that the algorithms-mostly trained on known AMR genes-fail to generalize upon. These results lead us to formulate practical advice for software configuration and application, and give suggestions for future study designs to further develop AMR prediction tools from proof-of-concept to bedside.

Designing of a multi-epitopes-based peptide vaccine against rift valley fever virus and its validation through integrated computational approaches

Since its discovery, the Rift Valley Fever virus (RVFV) has been the source of numerous outbreaks in the Arab Peninsulas and Africa, wreaking havoc on humans and animals. The lack of therapeutics or licensed human vaccines limits the options for controlling RVFV outbreaks. Therefore, RVFV has been prioritized for rapid research and innovation of prevention strategies to control and prevent its outbreaks. The purpose of this study was to design a multi-epitope-based peptide vaccine (MEBPV) against RVFV. Bioinformatics approaches were used to design a potent MEBPV that can potentially activate both CD8⁺ and CD4⁺ T-cell immune responses, and several computational tools were employed to investigate its biological activities. Three antigenic proteins (Nucleocapsid (N), Glycoprotein C (GC), and Glycoprotein N (GN)) from the RVFV were chosen and potential immunogenic T- and B -cell epitopes were predicted from them. Based on in silico analysis, a MEBPV based on highly scored T and B-cell epitopes (6 CTL, 5 HTL, and 4 LBL) combined with linkers and adjuvants was developed. The finest predicted model was used for docking studies with Toll-like receptors (TLR3 and TLR8) and MHC molecules (MHC I and MHC II) after predicting and analyzing the tertiary structure of MEBPV. The designed MEBPV was then tested for stability with TLR3 and TLR8 receptors using molecular dynamics (MD) simulation and MMGBSA analysis. The MEBPV -TLR3, MEBPV -TLR8, MEBPV-MHC I and MEBPV -MHC II docked models were found stable during simulation time in MD and MMGBSA studies. In silico analysis revealed that the constructed vaccine could elicit both cell-mediated and humoral immune responses simultaneously. The proposed MEBPV could be a strong candidate against RVFV, but it will need to be tested in the laboratory to guarantee its safety and immunogenicity.

Bacterial keratitis: identifying the areas of clinical uncertainty

Bacterial keratitis is a common corneal infection that is treated with topical antimicrobials. By the time of presentation there may already be severe visual loss from corneal ulceration and opacity, which may persist despite treatment. There are significant differences in the associated risk factors and the bacterial isolates between high income and low- or middle-income countries, so that general management guidelines may not be appropriate. Although the diagnosis of bacterial keratitis may seem intuitive there are multiple uncertainties about the criteria that are used, which impacts the interpretation of investigations and recruitment to clinical studies. Importantly, the concept that bacterial keratitis can only be confirmed by culture ignores the approximately 50% of cases clinically consistent with bacterial keratitis in which investigations are negative. The aetiology of these culture-negative cases is unknown. Currently, the estimation of bacterial susceptibility to antimicrobials is based on data from systemic administration and achievable serum or tissue concentrations, rather than relevant corneal concentrations and biological activity in the cornea. The provision to the clinician of minimum inhibitory concentrations of the antimicrobials for the isolated bacteria would be an important step forward. An increase in the prevalence of antimicrobial resistance is a concern, but the effect this has on disease outcomes is yet unclear. Virulence factors are not routinely assessed although they may affect the pathogenicity of bacteria within species and affect outcomes. New technologies have been developed to detect and kill bacteria, and their application to bacterial keratitis is discussed. In this review we present the multiple areas of clinical uncertainty that hamper research and the clinical management of bacterial keratitis, and we address some of the assumptions and dogma that have become established in the literature.

Genome-Wide Mutation Scoring for Machine-Learning-Based Antimicrobial Resistance Prediction

The prediction of antimicrobial resistance (AMR) based on genomic information can improve patient outcomes. Genetic mechanisms have been shown to explain AMR with accuracies in line with standard microbiology laboratory testing. To translate genetic mechanisms into phenotypic AMR, machine learning has been successfully applied. AMR machine learning models typically use nucleotide k-mer counts to represent genomic sequences. While k-mer representation efficiently captures sequence variation, it also results in high-dimensional and sparse data. With limited training data available, achieving acceptable model performance or model interpretability is challenging. In this study, we explore the utility of feature engineering with several biologically relevant signals. We propose to predict the functional impact of observed mutations with PROVEAN to use the predicted impact as a new feature for each protein in an organism’s proteome. The addition of the new features was tested on a total of 19,521 isolates across nine clinically relevant pathogens and 30 different antibiotics. The new features significantly improved the predictive performance of trained AMR models for Pseudomonas aeruginosa, Citrobacter freundii, and Escherichia coli. The balanced accuracy of the respective models of those three pathogens improved by 6.0% on average.

A review: antimicrobial resistance data mining models and prediction methods study for pathogenic bacteria

Antimicrobials have paved the way for medical and social development over the last century and are indispensable for treating infections in humans and animals. The dramatic spread and diversity of antibiotic-resistant pathogens have significantly reduced the efficacy of essentially all antibiotic classes and is a global problem affecting human and animal health. Antimicrobial resistance is influenced by complex factors such as resistance genes and dosing, which are highly nonlinear, time-lagged and multivariate coupled, and the amount of resistance data is large and redundant, making it difficult to predict and analyze. Based on machine learning methods and data mining techniques, this paper reviews (1) antimicrobial resistance data storage and analysis techniques, (2) antimicrobial resistance assessment methods and the associated risk assessment methods for antimicrobial resistance, and (3) antimicrobial resistance prediction methods. Finally, the current research results on antimicrobial resistance and the development trend are summarized to provide a systematic and comprehensive reference for the research on antimicrobial resistance.

Machine learning for identifying resistance features of Klebsiella pneumoniae using whole-genome sequence single nucleotide polymorphisms

Introduction. Klebsiella pneumoniae, a gram-negative bacterium, is a common pathogen causing nosocomial infection. The drug-resistance rate of K. pneumoniae is increasing year by year, posing a severe threat to public health worldwide. K. pneumoniae has been listed as one of the pathogens causing the global crisis of antimicrobial resistance in nosocomial infections. We need to explore the drug resistance of K. pneumoniae for clinical diagnosis. Single nucleotide polymorphisms (SNPs) are of high density and have rich genetic information in whole-genome sequencing (WGS), which can affect the structure or expression of proteins. SNPs can be used to explore mutation sites associated with bacterial resistance.Hypothesis/Gap Statement. Machine learning methods can detect genetic features associated with the drug resistance of K. pneumoniae from whole-genome SNP data.Aims. This work used Fast Feature Selection (FFS) and Codon Mutation Detection (CMD) machine learning methods to detect genetic features related to drug resistance of K. pneumoniae from whole-genome SNP data.Methods. WGS data on resistance of K. pneumoniae strains to four antibiotics (tetracycline, gentamicin, imipenem, amikacin) were downloaded from the European Nucleotide Archive (ENA). Sequence alignments were performed with MUMmer 3 to complete SNP calling using K. pneumoniae HS11286 chromosome as the reference genome. The FFS algorithm was applied to feature selection of the SNP dataset. The training set was constructed based on mutation sites with mutation frequency >0.995. Based on the original SNP training set, 70% of SNPs were randomly selected from each dataset as the test set to verify the accuracy of the training results. Finally, the resistance genes were obtained by the CMD algorithm and Venny.Results. The number of strains resistant to tetracycline, gentamicin, imipenem and amikacin was 931, 1048, 789 and 203, respectively. Machine learning algorithms were applied to the SNP training set and test set, and 28 and 23 resistance genes were predicted, respectively. The 28 resistance genes in the training set included 22 genes in the test set, which verified the accuracy of gene prediction. Among them, some genes (KPHS_35310, KPHS_18220, KPHS_35880, etc.) corresponded to known resistance genes (Eef2, lpxK, MdtC, etc). Logistic regression classifiers were established based on the identified SNPs in the training set. The area under the curves (AUCs) of the four antibiotics was 0.939, 0.950, 0.912 and 0.935, showing a strong ability to predict bacterial resistance.Conclusion. Machine learning methods can effectively be used to predict resistance genes and associated SNPs. The FFS and CMD algorithms have wide applicability. They can be used for the drug-resistance analysis of any microorganism with genomic variation and phenotypic data. This work lays a foundation for resistance research in clinical applications.

An end-to-end heterogeneous graph attention network for Mycobacterium tuberculosis drug-resistance prediction

Antimicrobial resistance (AMR) poses a threat to global public health. To mitigate the impacts of AMR, it is important to identify the molecular mechanisms of AMR and thereby determine optimal therapy as early as possible. Conventional machine learning-based drug-resistance analyses assume genetic variations to be homogeneous, thus not distinguishing between coding and intergenic sequences. In this study, we represent genetic data from Mycobacterium tuberculosis as a graph, and then adopt a deep graph learning method—heterogeneous graph attention network (‘HGAT–AMR’)—to predict anti-tuberculosis (TB) drug resistance. The HGAT–AMR model is able to accommodate incomplete phenotypic profiles, as well as provide ‘attention scores’ of genes and single nucleotide polymorphisms (SNPs) both at a population level and for individual samples. These scores encode the inputs, which the model is ‘paying attention to’ in making its drug resistance predictions. The results show that the proposed model generated the best area under the receiver operating characteristic (AUROC) for isoniazid and rifampicin (98.53 and 99.10%), the best sensitivity for three first-line drugs (94.91% for isoniazid, 96.60% for ethambutol and 90.63% for pyrazinamide), and maintained performance when the data were associated with incomplete phenotypes (i.e. for those isolates for which phenotypic data for some drugs were missing). We also demonstrate that the model successfully identifies genes and SNPs associated with drug resistance, mitigating the impact of resistance profile while considering particular drug resistance, which is consistent with domain knowledge.

Artificial Intelligence and Mapping a New Direction in Laboratory Medicine: A Review

BACKGROUND

Modern artificial intelligence (AI) and machine learning (ML) methods are now capable of completing tasks with performance characteristics that are comparable to those of expert human operators. As a result, many areas throughout healthcare are incorporating these technologies, including in vitro diagnostics and, more broadly, laboratory medicine. However, there are limited literature reviews of the landscape, likely future, and challenges of the application of AI/ML in laboratory medicine.

CONTENT

In this review, we begin with a brief introduction to AI and its subfield of ML. The ensuing sections describe ML systems that are currently in clinical laboratory practice or are being proposed for such use in recent literature, ML systems that use laboratory data outside the clinical laboratory, challenges to the adoption of ML, and future opportunities for ML in laboratory medicine.

SUMMARY

AI and ML have and will continue to influence the practice and scope of laboratory medicine dramatically. This has been made possible by advancements in modern computing and the widespread digitization of health information. These technologies are being rapidly developed and described, but in comparison, their implementation thus far has been modest. To spur the implementation of reliable and sophisticated ML-based technologies, we need to establish best practices further and improve our information system and communication infrastructure. The participation of the clinical laboratory community is essential to ensure that laboratory data are sufficiently available and incorporated conscientiously into robust, safe, and clinically effective ML-supported clinical diagnostics.

Aminoglycoside-resistance gene signatures are predictive of aminoglycoside MICs for carbapenem-resistant Klebsiella pneumoniae

BACKGROUND

Aminoglycoside-containing regimens may be an effective treatment option for infections caused by carbapenem-resistant Klebsiella pneumoniae (CR-Kp), but aminoglycoside-resistance genes are common in these strains. The relationship between the aminoglycoside-resistance genes and aminoglycoside MICs remains poorly defined.

OBJECTIVES

To identify genotypic signatures capable of predicting aminoglycoside MICs for CR-Kp.

METHODS

Clinical CR-Kp isolates (n = 158) underwent WGS to detect aminoglycoside-resistance genes. MICs of amikacin, gentamicin, plazomicin and tobramycin were determined by broth microdilution (BMD). Principal component analysis was used to initially separate isolates based on genotype. Multiple linear regression was then used to generate models that predict aminoglycoside MICs based on the aminoglycoside-resistance genes. Last, the performance of the predictive models was tested against a validation cohort of 29 CR-Kp isolates.

RESULTS

Among the original 158 CR-Kp isolates, 91.77% (145/158) had at least one clinically relevant aminoglycoside-resistance gene. As a group, 99.37%, 84.81%, 82.28% and 10.76% of the CR-Kp isolates were susceptible to plazomicin, amikacin, gentamicin and tobramycin, respectively. The first two principal components explained 72.23% of the total variance in aminoglycoside MICs and separated isolates into four groups with aac(6')-Ib, aac(6')-Ib', aac(6')-Ib+aac(6')-Ib' or no clinically relevant aminoglycoside-resistance genes. Regression models predicted aminoglycoside MICs with adjusted R2 values of 56%-99%. Within the validation cohort, the categorical agreement when comparing the observed BMD MICs with the predicated MICs was 96.55%, 89.66%, 86.21% and 82.76% for plazomicin, gentamicin, amikacin and tobramycin, respectively.

CONCLUSIONS

Susceptibility to each aminoglycoside varies in CR-Kp. Detection of aminoglycoside-resistance genes may be useful to predict aminoglycoside MICs for CR-Kp.

Prediction of Minimal Inhibitory Concentration of Meropenem Against Klebsiella pneumoniae Using Metagenomic Data

Minimal inhibitory concentration (MIC) is defined as the lowest concentration of an antimicrobial agent that can inhibit the visible growth of a particular microorganism after overnight incubation. Clinically, antibiotic doses for specific infections are determined according to the fraction of MIC. Therefore, credible assessment of MICs will provide a physician valuable information on the choice of therapeutic strategy. Early and precise usage of antibiotics is the key to an infection therapy. Compared with the traditional culture-based method, the approach of whole genome sequencing to identify MICs can shorten the experimental time, thereby improving clinical efficacy. Klebsiella pneumoniae is one of the most significant members of the genus Klebsiella in the Enterobacteriaceae family and also a common non-social pathogen. Meropenem is a broad-spectrum antibacterial agent of the carbapenem family, which can produce antibacterial effects of most Gram-positive and -negative bacteria. In this study, we used single-nucleotide polymorphism (SNP) information and nucleotide k-mers count based on metagenomic data to predict MICs of meropenem against K. pneumoniae. Then, features of 110 sequenced K. pneumoniae genome data were combined and modeled with XGBoost algorithm and deep neural network (DNN) algorithm to predict MICs. We first use the XGBoost classification model and the XGBoost regression model. After five runs, the average accuracy of the test set was calculated. The accuracy of using nucleotide k-mers to predict MICs of the XGBoost classification model and XGBoost regression model was 84.5 and 89.1%. The accuracy of SNP in predicting MIC was 80 and 81.8%, respectively. The results show that XGBoost regression is better than XGBoost classification in both nucleotide k-mers and SNPs to predict MICs. We further selected 40 nucleotide k-mers and 40 SNPs with the highest correlation with MIC values as features to retrain the XGBoost regression model and DNN regression model. After 100 and 1,000 runs, the results show that the accuracy of the two models was improved. The accuracy of the XGBoost regression model for k-mers, SNPs, and k-mers & SNPs was 91.1, 85.2, and 91.3%, respectively. The accuracy of the DNN regression model was 91.9, 87.1, and 91.8%, respectively. Through external verification, some of the selected features were found to be related to drug resistance.

Genomic Analysis of the Predominant Strains and Antimicrobial Resistance Determinants Within 1479 Neisseria gonorrhoeae Isolates From the US Gonococcal Isolate Surveillance Project in 2018

BACKGROUND

The prevalence of Neisseria gonorrhoeae (GC) isolates with elevated minimum inhibitory concentrations to various antibiotics continues to rise in the United States and globally. Genomic analysis provides a powerful tool for surveillance of circulating strains, antimicrobial resistance determinants, and understanding of transmission through a population.

METHODS

Neisseria gonorrhoeae isolates collected from the US Gonococcal Isolate Surveillance Project in 2018 (n = 1479) were sequenced and characterized. Whole-genome sequencing was used to identify sequence types, antimicrobial resistance profiles, and phylogenetic relationships across demographic and geographic populations.

RESULTS

Genetic characterization identified that (1) 80% of the GC isolates were represented in 33 multilocus sequence types, (2) isolates clustered in 23 major phylogenetic clusters with select phenotypic and demographic prevalence, and (3) common antimicrobial resistance determinants associated with low-level or high-level decreased susceptibility or resistance to relevant antibiotics.

CONCLUSIONS

Characterization of this 2018 Gonococcal Isolate Surveillance Project genomic data set, which is the largest US whole-genome sequence data set to date, sets the basis for future prospective studies, and establishes a genomic baseline of GC populations for local and national monitoring.

Predictive Antibiotic Susceptibility Testing by Next-Generation Sequencing for Periprosthetic Joint Infections: Potential and Limitations

Joint replacement surgeries are one of the most frequent medical interventions globally. Infections of prosthetic joints are a major health challenge and typically require prolonged or even indefinite antibiotic treatment. As multidrug-resistant pathogens continue to rise globally, novel diagnostics are critical to ensure appropriate treatment and help with prosthetic joint infections (PJI) management. To this end, recent studies have shown the potential of molecular methods such as next-generation sequencing to complement established phenotypic, culture-based methods. Together with advanced bioinformatics approaches, next-generation sequencing can provide comprehensive information on pathogen identity as well as antimicrobial susceptibility, potentially enabling rapid diagnosis and targeted therapy of PJIs. In this review, we summarize current developments in next generation sequencing based predictive antibiotic susceptibility testing and discuss potential and limitations for common PJI pathogens.

Large-scale assessment of antimicrobial resistance marker databases for genetic phenotype prediction: a systematic review

Background

Antimicrobial resistance (AMR) is a rising health threat with 10 million annual casualties estimated by 2050. Appropriate treatment of infectious diseases with the right antibiotics reduces the spread of antibiotic resistance. Today, clinical practice relies on molecular and PCR techniques for pathogen identification and culture-based antibiotic susceptibility testing (AST). Recently, WGS has started to transform clinical microbiology, enabling prediction of resistance phenotypes from genotypes and allowing for more informed treatment decisions. WGS-based AST (WGS-AST) depends on the detection of AMR markers in sequenced isolates and therefore requires AMR reference databases. The completeness and quality of these databases are material to increase WGS-AST performance.

Methods

We present a systematic evaluation of the performance of publicly available AMR marker databases for resistance prediction on clinical isolates. We used the public databases CARD and ResFinder with a final dataset of 2587 isolates across five clinically relevant pathogens from PATRIC and NDARO, public repositories of antibiotic-resistant bacterial isolates.

Results

CARD and ResFinder WGS-AST performance had an overall balanced accuracy of 0.52 (±0.12) and 0.66 (±0.18), respectively. Major error rates were higher in CARD (42.68%) than ResFinder (25.06%). However, CARD showed almost no very major errors (1.17%) compared with ResFinder (4.42%).

Conclusions

We show that AMR databases need further expansion, improved marker annotations per antibiotic rather than per antibiotic class and validated multivariate marker panels to achieve clinical utility, e.g. in order to meet performance requirements such as provided by the FDA for clinical microbiology diagnostic testing.

A Prediction Method for Animal-Derived Drug Resistance Trend Using a Grey-BP Neural Network Combination Model

There is an increasing drug resistance of animal-derived pathogens, seriously posing a huge threat to the health of animals and humans. Traditional drug resistance testing methods are expensive, have low efficiency, and are time-consuming, making it difficult to evaluate overall drug resistance. To develop a better approach to detect drug resistance, a small sample of Escherichia coli resistance data from 2003 to 2014 in Chengdu, Sichuan Province was used, and multiple regression interpolation was applied to impute missing data based on the time series. Next, cluster analysis was used to classify anti-E. coli drugs. According to the classification results, a GM(1,1)-BP model was selected to analyze the changes in the drug resistance of E. coli, and a drug resistance prediction system was constructed based on the GM(1,1)-BP Neural Network model. The GM(1,1)-BP Neural Network model showed a good prediction effect using a small sample of drug resistance data, with a determination coefficient R² of 0.7830 and an RMSE of only 0.0527. This model can be applied for the prediction of drug resistance trends of other animal-derived pathogenic bacteria, and provides the scientific and technical means for the effective assessment of bacterial resistance.

Recent advances in rapid antimicrobial susceptibility testing systems

INTRODUCTION

Until recently antimicrobial susceptibility testing (AST) methods based on the demonstration of phenotypic susceptibility in 16-24 h remained largely unchanged.

AREAS COVERED

Advances in rapid phenotypic and molecular-based AST systems.

EXPERT OPINION

AST has changed over the past decade, with many rapid phenotypic and molecular methods developed to demonstrate phenotypic or genotypic resistance, or biochemical markers of resistance such as β-lactamases associated with carbapenem resistance. Most methods still require isolation of bacteria from specimens before both legacy and newer methods can be used. Bacterial identification by MALDI-TOF mass spectroscopy is now widely used and is often key to the interpretation of rapid AST results. Several PCR arrays are available to detect the most frequent pathogens associated with bloodstream infections and their major antimicrobial resistance genes. Many advances in whole-genome sequencing of bacteria and fungi isolated by culture as well as directly from clinical specimens have been made but are not yet widely available. High cost and limited throughput are the major obstacles to uptake of rapid methods, but targeted use, continued development and decreasing costs are expected to result in more extensive use of these increasingly useful methods.

An explainable machine learning platform for pyrazinamide resistance prediction and genetic feature identification of Mycobacterium tuberculosis

OBJECTIVE

Tuberculosis is the leading cause of death from a single infectious agent. The emergence of antimicrobial resistant Mycobacterium tuberculosis strains makes the problem more severe. Pyrazinamide (PZA) is an important component for short-course treatment regimens and first- and second-line treatment regimens. This research aims for fast diagnosis of M. tuberculosis resistance to PZA and identification of genetic features causing resistance.

MATERIALS AND METHODS

We use clinically collected genomic data of M. tuberculosis that are resistant or susceptible to PZA. A machine learning platform is built to diagnose PZA resistance using the whole genome sequence data, and to identify resistance genes and mutations. The platform consists of a deep convolutional neural network (DCNN) model for resistance diagnosis and a support vector machine (SVM) model as a surrogate to identify resistance genes and mutations.

RESULTS

The DCNN model achieves a PZA resistance diagnosis accuracy of 93%. Each prediction takes less than a second. The SVM has revealed 2 novel genes, embB and gyrA, besides the well-known pncA gene, and 9 mutations that harbor PZA resistance.

DISCUSSION

The DCNN and SVM machine learning platform, if used together with the real-time genome sequencing machines, could allow for rapid PZA diagnosis, allowing for critical time to ensure good patient outcomes, and preventing outbreaks of deadly infections. Furthermore, identifying pertinent resistance genes and mutations will help researchers better understand the biological mechanisms behind resistance.

CONCLUSIONS

Machine learning can be used to achieve high-accuracy resistance prediction, and identify genes and mutations causing the resistance.

Emerging Applications of Machine Learning in Food Safety

Food safety continues to threaten public health. Machine learning holds potential in leveraging large, emerging data sets to improve the safety of the food supply and mitigate the impact of food safety incidents. Foodborne pathogen genomes and novel data streams, including text, transactional, and trade data, have seen emerging applications enabled by a machine learning approach, such as prediction of antibiotic resistance, source attribution of pathogens, and foodborne outbreak detection and risk assessment. In this article, we provide a gentle introduction to machine learning in the context of food safety and an overview of recent developments and applications. With many of these applications still in their nascence, general and domain-specific pitfalls and challenges associated with machine learning have begun to be recognized and addressed, which are critical to prospective use and future deployment of large data sets and their associated machine learning models for food safety applications.

Identifying novel β-lactamase substrate activity through in silico prediction of antimicrobial resistance

Diagnosing antimicrobial resistance (AMR) in the clinic is based on empirical evidence and current gold standard laboratory phenotypic methods. Genotypic methods have the potential advantages of being faster and cheaper, and having improved mechanistic resolution over phenotypic methods. We generated and applied rule-based and logistic regression models to predict the AMR phenotype from Escherichia coli and Pseudomonas aeruginosa multidrug-resistant clinical isolate genomes. By inspecting and evaluating these models, we identified previously unknown β-lactamase substrate activities. In total, 22 unknown β-lactamase substrate activities were experimentally validated using targeted gene expression studies. Our results demonstrate that generating and analysing predictive models can help guide researchers to the mechanisms driving resistance and improve annotation of AMR genes and phenotypic prediction, and suggest that we cannot solely rely on curated knowledge to predict resistance phenotypes.

Predicting antimicrobial resistance using conserved genes

A growing number of studies are using machine learning models to accurately predict antimicrobial resistance (AMR) phenotypes from bacterial sequence data. Although these studies are showing promise, the models are typically trained using features derived from comprehensive sets of AMR genes or whole genome sequences and may not be suitable for use when genomes are incomplete. In this study, we explore the possibility of predicting AMR phenotypes using incomplete genome sequence data. Models were built from small sets of randomly-selected core genes after removing the AMR genes. For Klebsiella pneumoniae, Mycobacterium tuberculosis, Salmonella enterica, and Staphylococcus aureus, we report that it is possible to classify susceptible and resistant phenotypes with average F1 scores ranging from 0.80-0.89 with as few as 100 conserved non-AMR genes, with very major error rates ranging from 0.11-0.23 and major error rates ranging from 0.10-0.20. Models built from core genes have predictive power in cases where the primary AMR mechanisms result from SNPs or horizontal gene transfer. By randomly sampling non-overlapping sets of core genes, we show that F1 scores and error rates are stable and have little variance between replicates. Although these small core gene models have lower accuracies and higher error rates than models built from the corresponding assembled genomes, the results suggest that sufficient variation exists in the core non-AMR genes of a species for predicting AMR phenotypes.

Predicting antimicrobial susceptibility from the bacterial genome: A new paradigm for one health resistance monitoring

The laboratory identification of antibacterial resistance is a cornerstone of infectious disease medicine. In vitro antimicrobial susceptibility testing has long been based on the growth response of organisms in pure culture to a defined concentration of antimicrobial agents. By comparing individual isolates to wild-type susceptibility patterns, strains with acquired resistance can be identified. Acquired resistance can also be detected genetically. After many decades of research, the inventory of genes underlying antimicrobial resistance is well known for several pathogenic genera including zoonotic enteric organisms such as Salmonella and Campylobacter and continues to grow substantially for others. With the decline in costs for large scale DNA sequencing, it is now practicable to characterize bacteria using whole genome sequencing, including the carriage of resistance genes in individual microorganisms and those present in complex biological samples. With genomics, we can generate comprehensive, detailed information on the bacterium, the mechanisms of antibiotic resistance, clues to its source, and the nature of mobile DNA elements by which resistance spreads. These developments point to a new paradigm for antimicrobial resistance detection and tracking for both clinical and public health purposes.

Genomic Prediction of Antimicrobial Resistance: Ready or Not, Here It Comes!

BACKGROUND

Next-generation sequencing (NGS) technologies are being used to predict antimicrobial resistance. The field is evolving rapidly and transitioning out of the research setting into clinical use. Clinical laboratories are evaluating the accuracy and utility of genomic resistance prediction, including methods for NGS, downstream bioinformatic pipeline components, and the clinical settings in which this type of testing should be offered.

CONTENT

We describe genomic sequencing as it pertains to predicting antimicrobial resistance in clinical isolates and samples. We elaborate on current methodologies and workflows to perform this testing and summarize the current state of genomic resistance prediction in clinical settings. To highlight this aspect, we include 3 medically relevant microorganism exemplars: Mycobacterium tuberculosis, Staphylococcus aureus, and Neisseria gonorrhoeae. Last, we discuss the future of genomic-based resistance detection in clinical microbiology laboratories.

SUMMARY

Antimicrobial resistance prediction by genomic approaches is in its infancy for routine patient care. Genomic approaches have already added value to the current diagnostic testing landscape in specific circumstances and will play an increasingly important role in diagnostic microbiology. Future advancements will shorten turnaround time, reduce costs, and improve our analysis and interpretation of clinically actionable results.

Understanding and predicting ciprofloxacin minimum inhibitory concentration in Escherichia coli with machine learning

It is important that antibiotics prescriptions are based on antimicrobial susceptibility data to ensure effective treatment outcomes. The increasing availability of next-generation sequencing, bacterial whole genome sequencing (WGS) can facilitate a more reliable and faster alternative to traditional phenotyping for the detection and surveillance of AMR. This work proposes a machine learning approach that can predict the minimum inhibitory concentration (MIC) for a given antibiotic, here ciprofloxacin, on the basis of both genome-wide mutation profiles and profiles of acquired antimicrobial resistance genes. We analysed 704 Escherichia coli genomes combined with their respective MIC measurements for ciprofloxacin originating from different countries. The four most important predictors found by the model, mutations in gyrA residues Ser83 and Asp87, a mutation in parC residue Ser80 and presence of the qnrS1 gene, have been experimentally validated before. Using only these four predictors in a linear regression model, 65% and 93% of the test samples’ MIC were correctly predicted within a two- and a four-fold dilution range, respectively. The presented work does not treat machine learning as a black box model concept, but also identifies the genomic features that determine susceptibility. The recent progress in WGS technology in combination with machine learning analysis approaches indicates that in the near future WGS of bacteria might become cheaper and faster than a MIC measurement.

Development of an NGS-Based Workflow for Improved Monitoring of Circulating Plasmids in Support of Risk Assessment of Antimicrobial Resistance Gene Dissemination

Antimicrobial resistance (AMR) is one of the most prominent public health threats. AMR genes localized on plasmids can be easily transferred between bacterial isolates by horizontal gene transfer, thereby contributing to the spread of AMR. Next-generation sequencing (NGS) technologies are ideal for the detection of AMR genes; however, reliable reconstruction of plasmids is still a challenge due to large repetitive regions. This study proposes a workflow to reconstruct plasmids with NGS data in view of AMR gene localization, i.e., chromosomal or on a plasmid. Whole-genome and plasmid DNA extraction methods were compared, as were assemblies consisting of short reads (Illumina MiSeq), long reads (Oxford Nanopore Technologies) and a combination of both (hybrid). Furthermore, the added value of conjugation of a plasmid to a known host was evaluated. As a case study, an isolate harboring a large, low-copy mcr-1-carrying plasmid (>200 kb) was used. Hybrid assemblies of NGS data obtained from whole-genome DNA extractions of the original isolates resulted in the most complete reconstruction of plasmids. The optimal workflow was successfully applied to multidrug-resistant Salmonella Kentucky isolates, where the transfer of an ESBL-gene-containing fragment from a plasmid to the chromosome was detected. This study highlights a strategy including wet and dry lab parameters that allows accurate plasmid reconstruction, which will contribute to an improved monitoring of circulating plasmids and the assessment of their risk of transfer.

Analytical Performance Validation of Next-Generation Sequencing Based Clinical Microbiology Assays Using a K-mer Analysis Workflow

Next-generation sequencing (NGS) enables clinical microbiology assays such as molecular typing of bacterial isolates which is now routinely applied for infection control and epidemiology. Additionally, feasibility for NGS-based identification of antimicrobial resistance (AMR) markers as well as genetic prediction of antibiotic susceptibility testing results has been demonstrated. Various bioinformatics approaches enabling NGS-based clinical microbiology assays exist, but standardized, computationally efficient and scalable sample-to-results workflows including validated quality control parameters are still lacking. Bioinformatics analysis workflows based on k-mers have been shown to allow for fast and efficient analysis of large genomics data sets as obtained from microbial sequencing applications. We here demonstrate applicability of k-mer based clinical microbiology assays for whole-genome sequencing (WGS) including variant calling, taxonomic identification, bacterial typing as well as AMR marker detection. The wet-lab and dry-lab workflows were developed and validated in line with Clinical Laboratory Improvement Act (CLIA) guidelines for laboratory-developed tests (LDTs) on multi-drug resistant ESKAPE pathogens. The developed k-mer based workflow demonstrated ≥99.39% repeatability, ≥99.09% reproducibility and ≥99.76% accuracy for variant calling and applied assays as determined by intra-day and inter-day triplicate measurements. The limit of detection (LOD) across assays was found to be at 20× sequencing depth and 15× for AMR marker detection. Thorough benchmarking of the k-mer based workflow revealed analytical performance criteria are comparable to state-of-the-art alignment based workflows across clinical microbiology assays. Diagnostic sensitivity and specificity for multilocus sequence typing (MLST) and phylogenetic analysis were 100% for both approaches. For AMR marker detection, sensitivity and specificity were 95.29 and 99.78% for the k-mer based workflow as compared to 95.17 and 99.77% for the alignment-based approach. Summarizing, results illustrate that k-mer based analysis workflows enable a broad range of clinical microbiology assays, potentially not only for WGS-based typing and AMR gene detection but also genetic prediction of antibiotic susceptibility testing results.

BacEffluxPred: A two-tier system to predict and categorize bacterial efflux mediated antibiotic resistance proteins

Efflux proteins are transport proteins, which are involved in transporting different substrates from the cell to the external environment, including antibiotics. The efflux mechanism and efflux pumps are a major reason underlying emerging rampant antibiotic resistance (AR) in microbes. To reduce the resources required and time of identification, characterization and classification of bacterial efflux proteins, we have developed a fast and accurate support vector machine based two-tier prediction system, BacEffluxPred, which can predict bacterial efflux proteins responsible for AR and identify their corresponding families. A leave-one-out cross-validation also called jackknife procedure was used for performance evaluation. The accuracy to discriminate bacterial AR efflux from non-AR efflux was obtained as 85.81% (at tier-I) while accuracies for prediction of efflux pump families like ABC, MFS, RND and MATE family were found 92.13%, 85.39%, 91.01% and 99.44%, respectively (at tier-II). Benchmarking on an independent dataset also showed that BacEffluxPred had comparable accuracy for prediction of bacterial AR efflux pumps and their families. This is the first in-silico tool for predicting bacterial AR efflux proteins and their families and is freely available as both web-server and standalone versions at http://proteininformatics.org/mkumar/baceffluxpred/.

Predicting Phenotypic Polymyxin Resistance in Klebsiella pneumoniae through Machine Learning Analysis of Genomic Data

Polymyxins are used as treatments of last resort for Gram-negative bacterial infections. Their increased use has led to concerns about emerging polymyxin resistance (PR). Phenotypic polymyxin susceptibility testing is resource intensive and difficult to perform accurately. The complex polygenic nature of PR and our incomplete understanding of its genetic basis make it difficult to predict PR using detection of resistance determinants. We therefore applied machine learning (ML) to whole-genome sequencing data from >600 Klebsiella pneumoniae clonal group 258 (CG258) genomes to predict phenotypic PR. Using a reference-based representation of genomic data with ML outperformed a rule-based approach that detected variants in known PR genes (area under receiver-operator curve [AUROC], 0.894 versus 0.791, P = 0.006). We noted modest increases in performance by using a bacterial genome-wide association study to filter relevant genomic features and by integrating clinical data in the form of prior polymyxin exposure. Conversely, reference-free representation of genomic data as k-mers was associated with decreased performance (AUROC, 0.692 versus 0.894, P = 0.015). When ML models were interpreted to extract genomic features, six of seven known PR genes were correctly identified by models without prior programming and several genes involved in stress responses and maintenance of the cell membrane were identified as potential novel determinants of PR. These findings are a proof of concept that whole-genome sequencing data can accurately predict PR in K. pneumoniae CG258 and may be applicable to other forms of complex antimicrobial resistance.IMPORTANCE Polymyxins are last-resort antibiotics used to treat highly resistant Gram-negative bacteria. There are increasing reports of polymyxin resistance emerging, raising concerns of a postantibiotic era. Polymyxin resistance is therefore a significant public health threat, but current phenotypic methods for detection are difficult and time-consuming to perform. There have been increasing efforts to use whole-genome sequencing for detection of antibiotic resistance, but this has been difficult to apply to polymyxin resistance because of its complex polygenic nature. The significance of our research is that we successfully applied machine learning methods to predict polymyxin resistance in Klebsiella pneumoniae clonal group 258, a common health care-associated and multidrug-resistant pathogen. Our findings highlight that machine learning can be successfully applied even in complex forms of antibiotic resistance and represent a significant contribution to the literature that could be used to predict resistance in other bacteria and to other antibiotics.

Evaluation of Machine Learning Models for Predicting Antimicrobial Resistance of Actinobacillus pleuropneumoniae From Whole Genome Sequences

Antimicrobial resistance (AMR) is becoming a huge problem in countries all over the world, and new approaches to identifying strains resistant or susceptible to certain antibiotics are essential in fighting against antibiotic-resistant pathogens. Genotype-based machine learning methods showed great promise as a diagnostic tool, due to the increasing availability of genomic datasets and AST phenotypes. In this article, Support Vector Machine (SVM) and Set Covering Machine (SCM) models were used to learn and predict the resistance of the five drugs (Tetracycline, Ampicillin, Sulfisoxazole, Trimethoprim, and Enrofloxacin). The SVM model used the number of co-occurring k-mers between the genome of the isolates and the reference genes to learn and predict the phenotypes of the bacteria to a specific antimicrobial, while the SCM model uses a greedy approach to construct conjunction or disjunction of Boolean functions to find the most concise set of k-mers that allows for accurate prediction of the phenotype. Five-fold cross-validation was performed on the training set of the SVM and SCM model to select the best hyperparameter values to avoid model overfitting. The training accuracy (mean cross-validation score) and the testing accuracy of SVM and SCM models of five drugs were above 90% regardless of the resistant mechanism of which were acquired resistant or point mutation in the chromosome. The results of correlation between the phenotype and the model predictions of the five drugs indicated that both SVM and SCM models could significantly classify the resistant isolates from the sensitive isolates of the bacteria (p < 0.01), and would be used as potential tools in antimicrobial resistance surveillance and clinical diagnosis in veterinary medicine.

Typing methods based on whole genome sequencing data

Whole genome sequencing (WGS) of foodborne pathogens has become an effective method for investigating the information contained in the genome sequence of bacterial pathogens. In addition, its highly discriminative power enables the comparison of genetic relatedness between bacteria even on a sub-species level. For this reason, WGS is being implemented worldwide and across sectors (human, veterinary, food, and environment) for the investigation of disease outbreaks, source attribution, and improved risk characterization models. In order to extract relevant information from the large quantity and complex data produced by WGS, a host of bioinformatics tools has been developed, allowing users to analyze and interpret sequencing data, starting from simple gene-searches to complex phylogenetic studies. Depending on the research question, the complexity of the dataset and their bioinformatics skill set, users can choose between a great variety of tools for the analysis of WGS data. In this review, we describe the relevant approaches for phylogenomic studies for outbreak studies and give an overview of selected tools for the characterization of foodborne pathogens based on WGS data. Despite the efforts of the last years, harmonization and standardization of typing tools are still urgently needed to allow for an easy comparison of data between laboratories, moving towards a one health worldwide surveillance system for foodborne pathogens.

Antimicrobial resistance genetic factor identification from whole-genome sequence data using deep feature selection

BACKGROUND

Antimicrobial resistance (AMR) is a major threat to global public health because it makes standard treatments ineffective and contributes to the spread of infections. It is important to understand AMR's biological mechanisms for the development of new drugs and more rapid and accurate clinical diagnostics. The increasing availability of whole-genome SNP (single nucleotide polymorphism) information, obtained from whole-genome sequence data, along with AMR profiles provides an opportunity to use feature selection in machine learning to find AMR-associated mutations. This work describes the use of a supervised feature selection approach using deep neural networks to detect AMR-associated genetic factors from whole-genome SNP data.

RESULTS

The proposed method, DNP-AAP (deep neural pursuit - average activation potential), was tested on a Neisseria gonorrhoeae dataset with paired whole-genome sequence data and resistance profiles to five commonly used antibiotics including penicillin, tetracycline, azithromycin, ciprofloxacin, and cefixime. The results show that DNP-AAP can effectively identify known AMR-associated genes in N. gonorrhoeae, and also provide a list of candidate genomic features (SNPs) that might lead to the discovery of novel AMR determinants. Logistic regression classifiers were built with the identified SNPs and the prediction AUCs (area under the curve) for penicillin, tetracycline, azithromycin, ciprofloxacin, and cefixime were 0.974, 0.969, 0.949, 0.994, and 0.976, respectively.

CONCLUSIONS

DNP-AAP can effectively identify known AMR-associated genes in N. gonorrhoeae. It also provides a list of candidate genes and intergenic regions that might lead to novel AMR factor discovery. More generally, DNP-AAP can be applied to AMR analysis of any bacterial species with genomic variants and phenotype data. It can serve as a useful screening tool for microbiologists to generate genetic candidates for further lab experiments.

Evaluation of parameters affecting performance and reliability of machine learning-based antibiotic susceptibility testing from whole genome sequencing data

Prediction of antibiotic resistance phenotypes from whole genome sequencing data by machine learning methods has been proposed as a promising platform for the development of sequence-based diagnostics. However, there has been no systematic evaluation of factors that may influence performance of such models, how they might apply to and vary across clinical populations, and what the implications might be in the clinical setting. Here, we performed a meta-analysis of seven large Neisseria gonorrhoeae datasets, as well as Klebsiella pneumoniae and Acinetobacter baumannii datasets, with whole genome sequence data and antibiotic susceptibility phenotypes using set covering machine classification, random forest classification, and random forest regression models to predict resistance phenotypes from genotype. We demonstrate how model performance varies by drug, dataset, resistance metric, and species, reflecting the complexities of generating clinically relevant conclusions from machine learning-derived models. Our findings underscore the importance of incorporating relevant biological and epidemiological knowledge into model design and assessment and suggest that doing so can inform tailored modeling for individual drugs, pathogens, and clinical populations. We further suggest that continued comprehensive sampling and incorporation of up-to-date whole genome sequence data, resistance phenotypes, and treatment outcome data into model training will be crucial to the clinical utility and sustainability of machine learning-based molecular diagnostics.

Identification of Primary Antimicrobial Resistance Drivers in Agricultural Nontyphoidal Salmonella enterica Serovars by Using Machine Learning

Nontyphoidal Salmonella (NTS) is a leading global cause of bacterial foodborne morbidity and mortality. Our ability to treat severe NTS infections has been impaired by increasing antimicrobial resistance (AMR). To understand and mitigate the global health crisis AMR represents, we need to link the observed resistance phenotypes with their underlying genomic mechanisms. Broiler chickens represent a key reservoir and vector for NTS infections, but isolates from this setting have been characterized in only very low numbers relative to clinical isolates. In this study, we sequenced and assembled 97 genomes encompassing 7 serotypes isolated from broiler chicken in farms in British Columbia between 2005 and 2008. Through application of machine learning (ML) models to predict the observed AMR phenotype from this genomic data, we were able to generate highly (0.92 to 0.99) precise logistic regression models using known AMR gene annotations as features for 7 antibiotics (amoxicillin-clavulanic acid, ampicillin, cefoxitin, ceftiofur, ceftriaxone, streptomycin, and tetracycline). Similarly, we also trained "reference-free" k-mer-based set-covering machine phenotypic prediction models (0.91 to 1.0 precision) for these antibiotics. By combining the inferred k-mers and logistic regression weights, we identified the primary drivers of AMR for the 7 studied antibiotics in these isolates. With our research representing one of the largest studies of a diverse set of NTS isolates from broiler chicken, we can thus confirm that the AmpC-like CMY-2 β-lactamase is a primary driver of β-lactam resistance and that the phosphotransferases APH(6)-Id and APH(3″-Ib) are the principal drivers of streptomycin resistance in this important ecosystem.IMPORTANCE Antimicrobial resistance (AMR) represents an existential threat to the function of modern medicine. Genomics and machine learning methods are being increasingly used to analyze and predict AMR. This type of surveillance is very important to try to reduce the impact of AMR. Machine learning models are typically trained using genomic data, but the aspects of the genomes that they use to make predictions are rarely analyzed. In this work, we showed how, by using different types of machine learning models and performing this analysis, it is possible to identify the key genes underlying AMR in nontyphoidal Salmonella (NTS). NTS is among the leading cause of foodborne illness globally; however, AMR in NTS has not been heavily studied within the food chain itself. Therefore, in this work we performed a broad-scale analysis of the AMR in NTS isolates from commercial chicken farms and identified some priority AMR genes for surveillance.

Interpretable genotype-to-phenotype classifiers with performance guarantees

Understanding the relationship between the genome of a cell and its phenotype is a central problem in precision medicine. Nonetheless, genotype-to-phenotype prediction comes with great challenges for machine learning algorithms that limit their use in this setting. The high dimensionality of the data tends to hinder generalization and challenges the scalability of most learning algorithms. Additionally, most algorithms produce models that are complex and difficult to interpret. We alleviate these limitations by proposing strong performance guarantees, based on sample compression theory, for rule-based learning algorithms that produce highly interpretable models. We show that these guarantees can be leveraged to accelerate learning and improve model interpretability. Our approach is validated through an application to the genomic prediction of antimicrobial resistance, an important public health concern. Highly accurate models were obtained for 12 species and 56 antibiotics, and their interpretation revealed known resistance mechanisms, as well as some potentially new ones. An open-source disk-based implementation that is both memory and computationally efficient is provided with this work. The implementation is turnkey, requires no prior knowledge of machine learning, and is complemented by comprehensive tutorials.

Genome-Based Prediction of Bacterial Antibiotic Resistance

Clinical microbiology has long relied on growing bacteria in culture to determine antimicrobial susceptibility profiles, but the use of whole-genome sequencing for antibiotic susceptibility testing (WGS-AST) is now a powerful alternative. This review discusses the technologies that made this possible and presents results from recent studies to predict resistance based on genome sequences. We examine differences between calling antibiotic resistance profiles by the simple presence or absence of previously known genes and single-nucleotide polymorphisms (SNPs) against approaches that deploy machine learning and statistical models. Often, the limitations to genome-based prediction arise from limitations of accuracy of culture-based AST in addition to an incomplete knowledge of the genetic basis of resistance. However, we need to maintain phenotypic testing even as genome-based prediction becomes more widespread to ensure that the results do not diverge over time. We argue that standardization of WGS-AST by challenge with consistently phenotyped strain sets of defined genetic diversity is necessary to compare the efficacy of methods of prediction of antibiotic resistance based on genome sequences.

Integrated analysis of population genomics, transcriptomics and virulence provides novel insights into Streptococcus pyogenes pathogenesis

Streptococcus pyogenes causes 700 million human infections annually worldwide, yet, despite a century of intensive effort, there is no licensed vaccine against this bacterium. Although a number of large-scale genomic studies of bacterial pathogens have been published, the relationships among the genome, transcriptome, and virulence in large bacterial populations remain poorly understood. We sequenced the genomes of 2,101 emm28 S. pyogenes invasive strains, from which we selected 492 phylogenetically diverse strains for transcriptome analysis and 50 strains for virulence assessment. Data integration provided a novel understanding of the virulence mechanisms of this model organism. Genome-wide association study, expression quantitative trait loci analysis, machine learning, and isogenic mutant strains identified and confirmed a one-nucleotide indel in an intergenic region that significantly alters global transcript profiles and ultimately virulence. The integrative strategy that we used is generally applicable to any microbe and may lead to new therapeutics for many human pathogens.

Using Machine Learning To Predict Antimicrobial MICs and Associated Genomic Features for Nontyphoidal Salmonella

Nontyphoidal Salmonella species are the leading bacterial cause of foodborne disease in the United States. Whole-genome sequences and paired antimicrobial susceptibility data are available for Salmonella strains because of surveillance efforts from public health agencies. In this study, a collection of 5,278 nontyphoidal Salmonella genomes, collected over 15 years in the United States, was used to generate extreme gradient boosting (XGBoost)-based machine learning models for predicting MICs for 15 antibiotics. The MIC prediction models had an overall average accuracy of 95% within ±1 2-fold dilution step (confidence interval, 95% to 95%), an average very major error rate of 2.7% (confidence interval, 2.4% to 3.0%), and an average major error rate of 0.1% (confidence interval, 0.1% to 0.2%). The model predicted MICs with no a priori information about the underlying gene content or resistance phenotypes of the strains. By selecting diverse genomes for the training sets, we show that highly accurate MIC prediction models can be generated with less than 500 genomes. We also show that our approach for predicting MICs is stable over time, despite annual fluctuations in antimicrobial resistance gene content in the sampled genomes. Finally, using feature selection, we explore the important genomic regions identified by the models for predicting MICs. To date, this is one of the largest MIC modeling studies to be published. Our strategy for developing whole-genome sequence-based models for surveillance and clinical diagnostics can be readily applied to other important human pathogens.

Prediction of antibiotic resistance in Escherichia coli from large-scale pan-genome data

The emergence of microbial antibiotic resistance is a global health threat. In clinical settings, the key to controlling spread of resistant strains is accurate and rapid detection. As traditional culture-based methods are time consuming, genetic approaches have recently been developed for this task. The detection of antibiotic resistance is typically made by measuring a few known determinants previously identified from genome sequencing, and thus requires the prior knowledge of its biological mechanisms. To overcome this limitation, we employed machine learning models to predict resistance to 11 compounds across four classes of antibiotics from existing and novel whole genome sequences of 1936 E. coli strains. We considered a range of methods, and examined population structure, isolation year, gene content, and polymorphism information as predictors. Gradient boosted decision trees consistently outperformed alternative models with an average accuracy of 0.91 on held-out data (range 0.81-0.97). While the best models most frequently employed gene content, an average accuracy score of 0.79 could be obtained using population structure information alone. Single nucleotide variation data were less useful, and significantly improved prediction only for two antibiotics, including ciprofloxacin. These results demonstrate that antibiotic resistance in E. coli can be accurately predicted from whole genome sequences without a priori knowledge of mechanisms, and that both genomic and epidemiological data can be informative. This paves way to integrating machine learning approaches into diagnostic tools in the clinic.