1
|
Tejeda MI, Fernández J, Valledor P, Almirall C, Barberán J, Romero-Brufau S. Retrospective validation study of a machine learning-based software for empirical and organism-targeted antibiotic therapy selection. Antimicrob Agents Chemother 2024; 68:e0077724. [PMID: 39194206 PMCID: PMC11460031 DOI: 10.1128/aac.00777-24] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2024] [Accepted: 07/16/2024] [Indexed: 08/29/2024] Open
Abstract
Errors in antibiotic prescriptions are frequent, often resulting from the inadequate coverage of the infection-causative microorganism. The efficacy of iAST, a machine-learning-based software offering empirical and organism-targeted antibiotic recommendations, was assessed. The study was conducted in a 12-hospital Spanish institution. After model fine-tuning with 27,531 historical antibiograms, 325 consecutive patients with acute infections were selected for retrospective validation. The primary endpoint was comparing each of the top three of iAST's antibiotic recommendations' success rates (confirmed by antibiogram results) with the antibiotic prescribed by the physicians. Secondary endpoints included examining the same hypothesis within specific study population subgroups and assessing antibiotic stewardship by comparing the percentage of antibiotics recommended that belonged to different World Health Organization AWaRe groups within each arm of the study. All of iAST first three recommendations were non-inferior to doctor prescription in the primary endpoint analysis population as well as the secondary endpoint. The overall success rate of doctors' empirical treatment was 68.93%, while that of the first three iAST options was 91.06% (P < 0.001), 90.63% (P < 0.001), and 91.06% (P < 001), respectively. For organism-targeted therapy, the doctor's overall success rate was 84.16%, and that of the first three ranked iAST options was 97.83% (P < 0.001), 94.09% (P < 0.001), and 91.30% (P < 0.001), respectively. In empirical therapy, compared to physician prescriptions, iAST demonstrated a greater propensity to recommend access antibiotics, fewer watch antibiotics, and higher reserve antibiotics. In organism-targeted therapy, iAST advised a higher utilization of access antibiotics. The present study demonstrates iAST accuracy in predicting antibiotic susceptibility, showcasing its potential to promote effective antibiotic stewardship. CLINICAL TRIALS This study is registered with ClinicalTrials.gov as NCT06174519.
Collapse
Affiliation(s)
- Maria Isabel Tejeda
- Infectious Diseases Unit, Hospital Universitario HM Montepríncipe, Madrid, Spain
| | - Javier Fernández
- Research and Innovation Department, Pragmatech AI Solutions, Oviedo, Spain
- Microbiology Department, Hospital Universitario Central de Asturias, Oviedo, Spain
- Microbiology and Infectious Pathology, ISPA, Oviedo, Spain
- Functional Biology Department, Universidad de Oviedo, Oviedo, Spain
| | - Pablo Valledor
- Research and Innovation Department, Pragmatech AI Solutions, Oviedo, Spain
| | | | - José Barberán
- Infectious Diseases Unit, Hospital Universitario HM Montepríncipe, Madrid, Spain
- HM Faculty of Health Sciences, University Camilo Jose Cela, Madrid, Spain
| | - Santiago Romero-Brufau
- Research and Innovation Department, Pragmatech AI Solutions, Oviedo, Spain
- Department of Otorhinolaryngology–Head & Neck Surgery, Mayo Clinic, Rochester, Minnesota, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Harvard University, Boston, Massachusetts, USA
| |
Collapse
|
2
|
Kherabi Y, Thy M, Bouzid D, Antcliffe DB, Rawson TM, Peiffer-Smadja N. Machine learning to predict antimicrobial resistance: future applications in clinical practice? Infect Dis Now 2024; 54:104864. [PMID: 38355048 DOI: 10.1016/j.idnow.2024.104864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2023] [Revised: 02/05/2024] [Accepted: 02/07/2024] [Indexed: 02/16/2024]
Abstract
INTRODUCTION Machine learning (ML) is increasingly being used to predict antimicrobial resistance (AMR). This review aims to provide physicians with an overview of the literature on ML as a means of AMR prediction. METHODS References for this review were identified through searches of MEDLINE/PubMed, EMBASE, Google Scholar, ACM Digital Library, and IEEE Xplore Digital Library up to December 2023. RESULTS Thirty-six studies were included in this review. Thirty-two studies (32/36, 89 %) were based on hospital data and four (4/36, 11 %) on outpatient data. The vast majority of them were conducted in high-resource settings (33/36, 92 %). Twenty-four (24/36, 67 %) studies developed systems to predict drug resistance in infected patients, eight (8/36, 22 %) tested the performances of ML-assisted antibiotic prescription, two (2/36, 6 %) assessed ML performances in predicting colonization with carbapenem-resistant bacteria and, finally, two assessed national and international AMR trends. The most common inputs were demographic characteristics (25/36, 70 %), previous antibiotic susceptibility testing (19/36, 53 %) and prior antibiotic exposure (15/36, 42 %). Thirty-three (92 %) studies targeted prediction of Gram-negative bacteria (GNB) resistance as an output (92 %). The studies included showed moderate to high performances, with AUROC ranging from 0.56 to 0.93. CONCLUSION ML can potentially provide valuable assistance in AMR prediction. Although the literature on this topic is growing, future studies are needed to design, implement, and evaluate the use and impact of ML decision support systems.
Collapse
Affiliation(s)
- Yousra Kherabi
- Infectious and Tropical Disease Department, Bichat-Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, Université Paris Cité, Paris, France; Université Paris Cité and Université Sorbonne Paris Nord, Inserm, IAME, Paris, France.
| | - Michaël Thy
- Medical and Infectious Diseases ICU (MI2) - Bichat-Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, Université Paris Cité, Paris, France; EA 7323 - Pharmacology and Therapeutic Evaluation in Children and Pregnant Women, Université Paris Cité, Paris, France
| | - Donia Bouzid
- Université Paris Cité and Université Sorbonne Paris Nord, Inserm, IAME, Paris, France; Emergency Department, Bichat Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, Paris, France
| | - David B Antcliffe
- Division of Anaesthetics, Pain Medicine and Intensive Care, Department of Surgery and Cancer, Imperial College London, London, UK; Department of Intensive Care Unit, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, UK
| | - Timothy Miles Rawson
- National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK; Centre for Antimicrobial Optimisation Imperial College London, London, UK
| | - Nathan Peiffer-Smadja
- Infectious and Tropical Disease Department, Bichat-Claude Bernard Hospital, Assistance Publique-Hôpitaux de Paris, Université Paris Cité, Paris, France; Université Paris Cité and Université Sorbonne Paris Nord, Inserm, IAME, Paris, France; National Institute for Health Research Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance, Imperial College London, London, UK
| |
Collapse
|
3
|
Lv G, Wang Y. Machine learning-based antibiotic resistance prediction models: An updated systematic review and meta-analysis. Technol Health Care 2024; 32:2865-2882. [PMID: 38875058 DOI: 10.3233/thc-240119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2024]
Abstract
BACKGROUND The widespread use of antibiotics has led to a gradual adaptation of bacteria to these drugs, diminishing the effectiveness of treatments. OBJECTIVE To comprehensively assess the research progress of antibiotic resistance prediction models based on machine learning (ML) algorithms, providing the latest quantitative analysis and methodological evaluation. METHODS Relevant literature was systematically retrieved from databases, including PubMed, Embase and the Cochrane Library, from inception up to December 2023. Studies meeting predefined criteria were selected for inclusion. The prediction model risk of bias assessment tool was employed for methodological quality assessment, and a random-effects model was utilised for meta-analysis. RESULTS The systematic review included a total of 22 studies with a combined sample size of 43,628; 10 studies were ultimately included in the meta-analysis. Commonly used ML algorithms included random forest, decision trees and neural networks. Frequently utilised predictive variables encompassed demographics, drug use history and underlying diseases. The overall sensitivity was 0.57 (95% CI: 0.42-0.70; p< 0.001; I2= 99.7%), the specificity was 0.95 (95% CI: 0.79-0.99; p< 0.001; I2 = 99.9%), the positive likelihood ratio was 10.7 (95% CI: 2.9-39.5), the negative likelihood ratio was 0.46 (95% CI: 0.34-0.61), the diagnostic odds ratio was 23 (95% CI: 7-81) and the area under the receiver operating characteristic curve was 0.78 (95% CI: 0.74-0.81; p< 0.001), indicating a good discriminative ability of ML models for antibiotic resistance. However, methodological assessment and funnel plots suggested a high risk of bias and publication bias in the included studies. CONCLUSION This meta-analysis provides a current and comprehensive evaluation of ML models for predicting antibiotic resistance, emphasising their potential application in clinical practice. Nevertheless, stringent research design and reporting are warranted to enhance the quality and credibility of future studies. Future research should focus on methodological innovation and incorporate more high-quality studies to further advance this field.
Collapse
Affiliation(s)
- Guodong Lv
- Department of STD and AIDS Prevention and Control, Langfang Center for Disease Prevention and Control, Langfang, Hebei, China
| | - Yuntao Wang
- Department of Pharmacy, Langfang Health Vocational College, Langfang, Hebei, China
| |
Collapse
|
4
|
Sakagianni A, Koufopoulou C, Feretzakis G, Kalles D, Verykios VS, Myrianthefs P, Fildisis G. Using Machine Learning to Predict Antimicrobial Resistance-A Literature Review. Antibiotics (Basel) 2023; 12:antibiotics12030452. [PMID: 36978319 PMCID: PMC10044642 DOI: 10.3390/antibiotics12030452] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Revised: 02/21/2023] [Accepted: 02/22/2023] [Indexed: 03/30/2023] Open
Abstract
Machine learning (ML) algorithms are increasingly applied in medical research and in healthcare, gradually improving clinical practice. Among various applications of these novel methods, their usage in the combat against antimicrobial resistance (AMR) is one of the most crucial areas of interest, as increasing resistance to antibiotics and management of difficult-to-treat multidrug-resistant infections are significant challenges for most countries worldwide, with life-threatening consequences. As antibiotic efficacy and treatment options decrease, the need for implementation of multimodal antibiotic stewardship programs is of utmost importance in order to restrict antibiotic misuse and prevent further aggravation of the AMR problem. Both supervised and unsupervised machine learning tools have been successfully used to predict early antibiotic resistance, and thus support clinicians in selecting appropriate therapy. In this paper, we reviewed the existing literature on machine learning and artificial intelligence (AI) in general in conjunction with antimicrobial resistance prediction. This is a narrative review, where we discuss the applications of ML methods in the field of AMR and their value as a complementary tool in the antibiotic stewardship practice, mainly from the clinician's point of view.
Collapse
Affiliation(s)
| | - Christina Koufopoulou
- 1st Anesthesiology Department, Aretaieio Hospital, National and Kapodistrian University of Athens Medical School, 11528 Athens, Greece
| | - Georgios Feretzakis
- School of Science and Technology, Hellenic Open University, 26335 Patras, Greece
- Department of Quality Control, Research and Continuing Education, Sismanogleio General Hospital, 15126 Marousi, Greece
| | - Dimitris Kalles
- School of Science and Technology, Hellenic Open University, 26335 Patras, Greece
| | - Vassilios S Verykios
- School of Science and Technology, Hellenic Open University, 26335 Patras, Greece
| | - Pavlos Myrianthefs
- Faculty of Nursing, School of Health Sciences, National and Kapodistrian University of Athens, 11527 Athens, Greece
| | - Georgios Fildisis
- Faculty of Nursing, School of Health Sciences, National and Kapodistrian University of Athens, 11527 Athens, Greece
| |
Collapse
|
5
|
Tang R, Luo R, Tang S, Song H, Chen X. Machine learning in predicting antimicrobial resistance: a systematic review and meta-analysis. Int J Antimicrob Agents 2022; 60:106684. [PMID: 36279973 DOI: 10.1016/j.ijantimicag.2022.106684] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 10/17/2022] [Indexed: 11/05/2022]
Abstract
INTRODUCTION Antimicrobial resistance (AMR) is a global health threat; rapid and timely identification of AMR improves patient prognosis and reduces inappropriate antibiotic use. METHODS Relevant literature in PubMed, Web of Science, Embase and Institute of Electrical and Electronics Engineers prior to 28 September 2021 was searched. Any study that deployed machine learning (ML) or a risk score as a tool to predict AMR was included in the final review; there were 25 studies that employed the ML algorithm to predict AMR. RESULTS Extended spectrum β-lactamases, methicillin-resistant Staphylococcus aureus (MRSA) and carbapenem resistance were the most common outcomes in studies with a specific resistance pattern. The most common algorithms in ML prediction were logistic regression (n = 14 studies), decision tree (n = 14) and random forest (n = 7). The area under the curve (AUC) range for ML prediction was 0.48-0.93. The pooled AUC for ML prediction was 0.82 (0.78-0.85). Compared with risk score, higher specificity [87% (82-91) vs. 37% (25-51)] was indicated for ML prediction, but not sensitivity [67% (62-72) vs. 73% (67-79)]. CONCLUSIONS Machine learning might be a potential technology for AMR prediction; however, retrospective methodology for model development, nonstandard data processing and scarcity of validation in a randomised controlled trial or real-world study limit the application of these models in clinical practice.
Collapse
Affiliation(s)
- Rui Tang
- Department of Pharmacy, West China Hospital, Sichuan University, Chengdu, China.
| | - Rui Luo
- Department of Pain Medicine, Hospital of Chengdu University of Traditional Chinese Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Shiwei Tang
- Department of Pharmacy, People's Hospital of Xinjin District, Chengdu, China
| | - Haoxin Song
- Department of Pharmacy, West China Second University Hospital, Sichuan University, Chengdu, China
| | - Xiujuan Chen
- Department of Medical Big Data Center, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences, Guangzhou, China
| |
Collapse
|
6
|
Comparison of the performance of a clinical classification tree versus clinical gestalt in predicting sepsis with extended-spectrum beta-lactamase–producing gram-negative rods. ANTIMICROBIAL STEWARDSHIP & HEALTHCARE EPIDEMIOLOGY 2022; 2:e35. [PMID: 36310800 PMCID: PMC9614789 DOI: 10.1017/ash.2021.253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Revised: 12/01/2021] [Accepted: 12/03/2021] [Indexed: 11/18/2022]
Abstract
A clinical decision tree was developed using point-of-care characteristics to identify patients with culture-proven sepsis due to extended-spectrum β-lactamase–producing Enterobacterales (ESBL-PE). We compared its performance with the clinical gestalt of emergency department (ED) clinicians and hospital-based clinicians. The developed tree outperformed ED-based clinicians but was comparable to inpatient-based clinicians.
Collapse
|
7
|
Deelen JWT, Rottier WC, Giron Ortega JA, Rodriguez-Baño J, Harbarth S, Tacconelli E, Jacobsson G, Zahar JR, van Werkhoven CH, Bonten MJM. An international prospective cohort study to validate two prediction rules for infections caused by 3rd-generation cephalosporin-resistant Enterobacterales. Clin Infect Dis 2020; 73:e4475-e4483. [PMID: 32640024 PMCID: PMC8849131 DOI: 10.1093/cid/ciaa950] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Accepted: 07/03/2020] [Indexed: 11/13/2022] Open
Abstract
Background The possibility of bloodstream infections caused by third-generation cephalosporin-resistant Enterobacterales (3GC-R-BSI) leads to a trade-off between empiric inappropriate treatment (IAT) and unnecessary carbapenem use (UCU). Accurately predicting 3GC-R-BSI could reduce IAT and UCU. We externally validate 2 previously derived prediction rules for community-onset (CO) and hospital-onset (HO) suspected bloodstream infections. Methods In 33 hospitals in 13 countries we prospectively enrolled 200 patients per hospital in whom blood cultures were obtained and intravenous antibiotics with coverage for Enterobacterales were empirically started. Cases were defined as 3GC-R-BSI or 3GC-R gram-negative infection (3GC-R-GNI) (analysis 2); all other outcomes served as a comparator. Model discrimination and calibration were assessed. Impact on carbapenem use was assessed at several cutoff points. Results 4650 CO infection episodes were included and the prevalence of 3GC-R-BSI was 2.1% (n = 97). IAT occurred in 69 of 97 (71.1%) 3GC-R-BSI and UCU in 398 of 4553 non–3GC-R-BSI patients (8.7%). Model calibration was good, and the AUC was .79 (95% CI, .75–.83) for 3GC-R-BSI. The prediction rule potentially reduced IAT to 62% (60/97) while keeping UCU comparable at 8.4% or could reduce UCU to 6.3% (287/4553) while keeping IAT equal. IAT and UCU in all 3GC-R-GNIs (analysis 2) improved at similar percentages. 1683 HO infection episodes were included and the prevalence of 3GC-R-BSI was 4.9% (n = 83). Here model calibration was insufficient. Conclusions A prediction rule for CO 3GC-R infection was validated in an international cohort and could improve empirical antibiotic use. Validation of the HO rule yielded suboptimal performance.
Collapse
Affiliation(s)
- J W Timotëus Deelen
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Wouter C Rottier
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
| | - José A Giron Ortega
- Unidad Clínica de Enfermedades Infecciosas, Microbiología y Medicina Preventiva, Hospital Universitario Virgen Macarena/Departamento de Medicina, Universidad de Sevilla/Instituto de Biomedicina de Sevilla (IBiS), Seville, Spain
| | - Jesús Rodriguez-Baño
- Unidad Clínica de Enfermedades Infecciosas, Microbiología y Medicina Preventiva, Hospital Universitario Virgen Macarena/Departamento de Medicina, Universidad de Sevilla/Instituto de Biomedicina de Sevilla (IBiS), Seville, Spain
| | - Stephan Harbarth
- Infection Control Program, Geneva University Hospitals and Faculty of Medicine, Geneva, Switzerland
| | - Evelina Tacconelli
- Division of Infectious Diseases, Department of Diagnostic and Public Health, University of Verona, Verona, Italy
| | - Gunnar Jacobsson
- Region Västra Götaland, Skaraborg Hospital, Department of Infectious Diseases, Skövde, Sweden
| | - Jean-Ralph Zahar
- IAME, UMR 1137, Université Paris 13, Sorbonne Paris Cité, France; Service de Microbiologie Clinique et Unité de Contrôle et de Prévention Du Risque Infectieux, Groupe Hospitalier Paris Seine Saint-Denis, AP-HP, Rue de Stalingrad, Bobigny, France
| | - Cornelis H van Werkhoven
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands
| | - Marc J M Bonten
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, the Netherlands.,Department of Medical Microbiology, University Medical Center Utrecht, Utrecht, the Netherlands
| | | |
Collapse
|