1
|
Patton MJ, Liu VX. Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges. Crit Care Clin 2023; 39:647-673. [PMID: 37704332 DOI: 10.1016/j.ccc.2023.02.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2023]
Abstract
The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.
Collapse
Affiliation(s)
- Michael J Patton
- Medical Scientist Training Program, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL, USA; Hugh Kaul Precision Medicine Institute at the University of Alabama at Birmingham, 720 20th Street South, Suite 202, Birmingham, Alabama, 35233, USA.
| | - Vincent X Liu
- Kaiser Permanente Division of Research, Oakland, CA, USA.
| |
Collapse
|
2
|
Rodic S, Hryciw BN, Selim S, Wang CQ, Lepage MF, Goyal V, Nguyen LH, Fergusson DA, van Walraven C. Concurrent external validation of bloodstream infection probability models. Clin Microbiol Infect 2023; 29:61-69. [PMID: 35872173 DOI: 10.1016/j.cmi.2022.07.011] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 06/15/2022] [Accepted: 07/12/2022] [Indexed: 01/26/2023]
Abstract
OBJECTIVE Accurately estimating the likelihood of bloodstream infection (BSI) can help clinicians make diagnostic and therapeutic decisions. Many multivariate models predicting BSI probability have been published. This study measured the performance of BSI probability models within the same patient sample. METHODS We retrieved validated BSI probability models included in a recently published systematic review that returned a patient-level BSI probability for adults. Model applicability, discrimination, and accuracy was measured in a simple random sample of 4485 admitted adults having blood cultures ordered in the emergency department or the initial 48 hours of hospitalization. RESULTS Ten models were included (publication years 1991-2015). Common methodological threats to model performance included overfitting and continuous variable categorization. Restrictive inclusion criteria caused seven models to apply to <15% of validation patients. Model discrimination was less than originally reported in derivation groups (median c-statistic 60%, range 48-69). The observed BSI risk frequently deviated from expected (median integrated calibration index 4.0%, range 0.8-12.4). Notable disagreement in expected BSI probabilities was seen between models (median (25th-75th percentile) relative difference between expected risks 68.0% (28.6-113.6%)). DISCUSSION In a large randomly selected external validation population, many published BSI probability models had restricted applicability, limited discrimination and calibration, and extensive inter-model disagreement. Direct comparison of model performance is hampered by dissimilarities between model-specific validation groups.
Collapse
Affiliation(s)
- Stefan Rodic
- Department of Medicine, University of Ottawa, Canada
| | | | - Shehab Selim
- Department of Medicine, University of Ottawa, Canada
| | - Chu Qi Wang
- Department of Medicine, University of Ottawa, Canada
| | | | - Vineet Goyal
- Department of Medicine, University of Ottawa, Canada
| | | | - Dean A Fergusson
- Department of Medicine, University of Ottawa, Canada; Department of Epidemiology & Community Medicine, University of Ottawa, Ottawa Hospital Research Institute, ICES (formerly Institute for Clinical Evaluative Sciences), Canada
| | - Carl van Walraven
- Department of Medicine, University of Ottawa, Canada; Department of Epidemiology & Community Medicine, University of Ottawa, Ottawa Hospital Research Institute, ICES (formerly Institute for Clinical Evaluative Sciences), Canada.
| |
Collapse
|
3
|
Prediction of Bacteremia Based on 12-Year Medical Data Using a Machine Learning Approach: Effect of Medical Data by Extraction Time. Diagnostics (Basel) 2022; 12:diagnostics12010102. [PMID: 35054269 PMCID: PMC8774637 DOI: 10.3390/diagnostics12010102] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 12/29/2021] [Accepted: 12/31/2021] [Indexed: 12/12/2022] Open
Abstract
Early detection of bacteremia is important to prevent antibiotic abuse. Therefore, we aimed to develop a clinically applicable bacteremia prediction model using machine learning technology. Data from two tertiary medical centers’ electronic medical records during a 12-year-period were extracted. Multi-layer perceptron (MLP), random forest, and gradient boosting algorithms were applied for machine learning analysis. Clinical data within 12 and 24 hours of blood culture were analyzed and compared. Out of 622,771 blood cultures, 38,752 episodes of bacteremia were identified. In MLP with 128 hidden layer nodes, the area under the receiver operating characteristic curve (AUROC) of the prediction performance in 12- and 24-h data models was 0.762 (95% confidence interval (CI); 0.7617–0.7623) and 0.753 (95% CI; 0.7520–0.7529), respectively. AUROC of causative-pathogen subgroup analysis predictive value for Acinetobacter baumannii bacteremia was the highest at 0.839 (95% CI; 0.8388–0.8394). Compared to primary bacteremia, AUROC of sepsis caused by pneumonia was highest. Predictive performance of bacteremia was superior in younger age groups. Bacteremia prediction using machine learning technology appeared possible for acute infectious diseases. This model was more suitable especially to pneumonia caused by Acinetobacter baumannii. From the 24-h blood culture data, bacteremia was predictable by substituting only the continuously variable values.
Collapse
|
4
|
Pai KC, Wang MS, Chen YF, Tseng CH, Liu PY, Chen LC, Sheu RK, Wu CL. An Artificial Intelligence Approach to Bloodstream Infections Prediction. J Clin Med 2021; 10:jcm10132901. [PMID: 34209759 PMCID: PMC8268222 DOI: 10.3390/jcm10132901] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 06/28/2021] [Accepted: 06/28/2021] [Indexed: 11/16/2022] Open
Abstract
This study aimed to develop an early prediction model for identifying patients with bloodstream infections. The data resource was taken from 2015 to 2019 at Taichung Veterans General Hospital, and a total of 1647 bloodstream infection episodes and 3552 non-bloodstream infection episodes in the intensive care unit (ICU) were included in the model development and evaluation. During the data analysis, 30 clinical variables were selected, including patients’ basic characteristics, vital signs, laboratory data, and clinical information. Five machine learning algorithms were applied to examine the prediction model performance. The findings indicated that the area under the receiver operating characteristic curve (AUROC) of the prediction performance of the XGBoost model was 0.825 for the validation dataset and 0.821 for the testing dataset. The random forest model also presented higher values for the AUROC on the validation dataset and testing dataset, which were 0.855 and 0.851, respectively. The tree-based ensemble learning model enabled high detection ability for patients with bloodstream infections in the ICU. Additionally, the analysis of importance of features revealed that alkaline phosphatase (ALKP) and the period of the central venous catheter are the most important predictors for bloodstream infections. We further explored the relationship between features and the risk of bloodstream infection by using the Shapley Additive exPlanations (SHAP) visualized method. The results showed that a higher prothrombin time is more prominent in a bloodstream infection. Additionally, the impact of a lower platelet count and albumin was more prominent in a bloodstream infection. Our results provide additional clinical information for cut-off laboratory values to assist clinical decision-making in bloodstream infection diagnostics.
Collapse
Affiliation(s)
- Kai-Chih Pai
- College of Engineering, Tunghai University, Taichung City 407224, Taiwan; (K.-C.P.); (L.-C.C.)
| | - Min-Shian Wang
- Department of Critical Care Medicine, Taichung Veterans General Hospital, Taichung City 40705, Taiwan;
| | - Yun-Feng Chen
- Center for Infection Control, Taichung Veterans General Hospital, Taichung City 40705, Taiwan;
| | - Chien-Hao Tseng
- Department of Infectious Diseases, Taichung Veterans General Hospital, Taichung City 40705, Taiwan; (C.-H.T.); (P.-Y.L.)
| | - Po-Yu Liu
- Department of Infectious Diseases, Taichung Veterans General Hospital, Taichung City 40705, Taiwan; (C.-H.T.); (P.-Y.L.)
| | - Lun-Chi Chen
- College of Engineering, Tunghai University, Taichung City 407224, Taiwan; (K.-C.P.); (L.-C.C.)
| | - Ruey-Kai Sheu
- Department of Computer Science, Tunghai University, Taichung City 407224, Taiwan;
| | - Chieh-Liang Wu
- Department of Critical Care Medicine, Taichung Veterans General Hospital, Taichung City 40705, Taiwan;
- Correspondence:
| |
Collapse
|
5
|
Møller JK, Sørensen M, Hardahl C. Prediction of risk of acquiring urinary tract infection during hospital stay based on machine-learning: A retrospective cohort study. PLoS One 2021; 16:e0248636. [PMID: 33788888 PMCID: PMC8011767 DOI: 10.1371/journal.pone.0248636] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/12/2020] [Accepted: 03/02/2021] [Indexed: 11/19/2022] Open
Abstract
Background Healthcare associated infections (HAI) are a major burden for the healthcare system and associated with prolonged hospital stay, increased morbidity, mortality and costs. Healthcare associated urinary tract infections (HA-UTI) accounts for about 20–30% of all HAI’s, and with the emergence of multi-resistant urinary tract pathogens, the total burden of HA-UTI will most likely increase. Objective The aim of the current study was to develop two predictive models, using data from the index admission as well as historic data on a patient, to predict the development of UTI at the time of entry to the hospital and after 48 hours of admission (HA-UTI). The ultimate goal is to predict the individual patient risk of acquiring HA-UTI before it occurs so that health care professionals may take proper actions to prevent it. Methods Retrospective cohort analysis of approx. 300 000 adult admissions in a Danish region was performed. We developed models for UTI prediction with five machine-learning algorithms using demographic information, laboratory results, data on antibiotic treatment, past medical history (ICD10 codes), and clinical data by transformation of unstructured narrative text in Electronic Medical Records to structured data by Natural Language Processing. Results The five machine-learning algorithms have been evaluated by the performance measures average squared error, cumulative lift, and area under the curve (ROC-index). The algorithms had an area under the curve (ROC-index) ranging from 0.82 to 0.84 for the entry model (T = 0 hours after admission) and from 0.71 to 0.77 for the HA-UTI model (T = 48 hours after admission). Conclusion The study is proof of concept that it is possible to create machine-learning models that can serve as early warning systems to predict patients at risk of acquiring urinary tract infections during admission. The entry model and the HA-UTI models perform with a high ROC-index indicating a sufficient sensitivity and specificity, which may make both models instrumental in individualized prevention of UTI in hospitalized patients. The favored machine-learning methodology is Decision Trees to ensure the most transparent results and to increase clinical understanding and implementation of the models.
Collapse
Affiliation(s)
- Jens Kjølseth Møller
- Department of Clinical Microbiology, Lillebaelt Hospital, University Hospital of Southern Denmark, Vejle, Denmark
- * E-mail:
| | | | | |
Collapse
|
6
|
Risk of bacteremia in patients presenting with shaking chills and vomiting - a prospective cohort study. Epidemiol Infect 2020; 148:e86. [PMID: 32228723 PMCID: PMC7189349 DOI: 10.1017/s0950268820000746] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
Chills and vomiting have traditionally been associated with severe bacterial infections and bacteremia. However, few modern studies have in a prospective way evaluated the association of these signs with bacteremia, which is the aim of this prospective, multicenter study. Patients presenting to the emergency department with at least one affected vital sign (increased respiratory rate, increased heart rate, altered mental status, decreased blood pressure or decreased oxygen saturation) were included. A total of 479 patients were prospectively enrolled. Blood cultures were obtained from 197 patients. Of the 32 patients with a positive blood culture 11 patients (34%) had experienced shaking chills compared with 23 (14%) of the 165 patients with a negative blood culture, P = 0.009. A logistic regression was fitted to show the estimated odds ratio (OR) for a positive blood culture according to shaking chills. In a univariate model shaking chills had an OR of 3.23 (95% CI 1.35–7.52) and in a multivariate model the OR was 5.9 (95% CI 2.05–17.17) for those without prior antibiotics adjusted for age, sex, and prior antibiotics. The presence of vomiting was also addressed, but neither a univariate nor a multivariate logistic regression showed any association between vomiting and bacteremia. In conclusion, among patients at the emergency department with at least one affected vital sign, shaking chills but not vomiting were associated with bacteremia.
Collapse
|
7
|
Lee KH, Dong JJ, Jeong SJ, Chae MH, Lee BS, Kim HJ, Ko SH, Song YG. Early Detection of Bacteraemia Using Ten Clinical Variables with an Artificial Neural Network Approach. J Clin Med 2019; 8:jcm8101592. [PMID: 31581716 PMCID: PMC6832527 DOI: 10.3390/jcm8101592] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2019] [Revised: 09/21/2019] [Accepted: 09/23/2019] [Indexed: 12/20/2022] Open
Abstract
An adequate model for predicting bacteraemia has not yet been developed. This study aimed to evaluate the performance of an artificial neural network (ANN)-based prediction model in comparison with previous statistical models. The performance of multi-layer perceptron (MLP), a representative ANN model, was verified via comparison with a non-neural network model. A total of 1260 bacteraemia episodes were identified in 13,402 patients. In MLP with 128 hidden layer nodes, the area under the receiver operating characteristic curve (AUC) of the prediction performance was 0.729 (95% confidence interval [CI]; 0.712–0.728), while in MLP with 256 hidden layer nodes, it was 0.727 (95% CI; 0.713–0.727). In a conventional Bayesian statistical method, the AUC was 0.7. The aforementioned two MLP models exhibited the highest sensitivity (0.810). The ranking of clinical variables was used to describe the influential power of the prediction. Serum alkaline phosphatase was one of the most influential clinical variables, and one-out search was the best ranking method for measuring the influence of the clinical variables. Furthermore, adding variables beyond the 10 top-ranking ones did not significantly affect the prediction of bacteraemia. The ANN model is not inferior to conventional statistical approaches. Bacteraemia can be predicted using only the top 10 clinical variables determined by a ranking method, and the model can be used in clinical practice by applying real-time monitoring.
Collapse
Affiliation(s)
- Kyoung Hwa Lee
- Division of Infectious Diseases, Department of Internal Medicine, Gangnam Severance Hospital, Yonsei University College of Medicine, 211 Eonju-ro, Gangnam-gu, Seoul 06273, Korea.
| | - Jae June Dong
- Department of Family Medicine, Gangnam Severance Hospital, Yonsei University College of Medicine, Seoul 06273, Korea.
| | - Su Jin Jeong
- Division of Infectious Diseases, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine, Seoul 03722, Korea.
| | - Myeong-Hun Chae
- Selvas Artificial Intelligence Incorporate, Seoul 08594, Korea.
| | - Byeong Soo Lee
- Selvas Artificial Intelligence Incorporate, Seoul 08594, Korea.
| | - Hong Jae Kim
- Department of Medical Information, Gangnam Severance Hospital, Seoul 06273, Korea.
| | - Sung Hun Ko
- Department of Medical Information, Gangnam Severance Hospital, Seoul 06273, Korea.
| | - Young Goo Song
- Division of Infectious Diseases, Department of Internal Medicine, Gangnam Severance Hospital, Yonsei University College of Medicine, 211 Eonju-ro, Gangnam-gu, Seoul 06273, Korea.
| |
Collapse
|
8
|
Ward L, Andreassen S, Astrup JJ, Rahmani Z, Fantini M, Sambri V. Clinical- vs. model-based selection of patients suspected of sepsis for direct-from-blood rapid diagnostics in the emergency department: a retrospective study. Eur J Clin Microbiol Infect Dis 2019; 38:1515-1522. [PMID: 31079313 DOI: 10.1007/s10096-019-03581-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2019] [Accepted: 05/02/2019] [Indexed: 12/11/2022]
Abstract
Selecting high-risk patients may improve the cost-effectiveness of rapid diagnostics. Our objective was to assess whether model-based selection or clinical selection is better for selecting high-risk patients with a high rate of bacteremia and/or DNAemia. This study involved a model-based, retrospective selection of patients from a cohort from which clinicians selected high-risk patients for rapid direct-from-blood diagnostic testing. Patients were included if they were suspected of sepsis and had blood cultures ordered at the emergency department. Patients were selected by the model by adding those with the highest probability of bacteremia until the number of high-risk patients selected by clinicians was reached. The primary outcome was bacteremia rate. Secondary outcomes were DNAemia rate, and 30-day mortality. Data were collected for 1395 blood cultures. Following exclusion, 1142 patients were included in the analysis. In each high-risk group, 220/1142 were selected, where 55 were selected both by clinicians and the model. For the remaining 165 in each group, the model selected for a higher bacteremia rate (74/165, 44.8% vs. 45/165, 27.3%, p = 0.001), and a higher 30-day mortality (49/165, 29.7% vs. 19/165, 11.5%, p = 0.00004) than the clinically selected group. The model outperformed clinicians in selecting patients with a high rate of bacteremia. Using such a model for risk stratification may contribute towards closing the gap in cost between rapid and culture-based diagnostics.
Collapse
Affiliation(s)
- Logan Ward
- Treat Systems ApS, Aalborg, Denmark. .,Department of Health Science and Technology, Aalborg University, Aalborg, Denmark.
| | - Steen Andreassen
- Treat Systems ApS, Aalborg, Denmark.,Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | | | - Zakia Rahmani
- Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
| | - Michela Fantini
- Unit of Microbiology, The Greater Romagna Area Hub Laboratory, Pievesestina, Italy
| | - Vittorio Sambri
- Unit of Microbiology, The Greater Romagna Area Hub Laboratory, Pievesestina, Italy.,DIMES, University of Bologna, Bologna, Italy
| |
Collapse
|
9
|
Machine learning for fast identification of bacteraemia in SIRS patients treated on standard care wards: a cohort study. Sci Rep 2018; 8:12233. [PMID: 30111827 PMCID: PMC6093921 DOI: 10.1038/s41598-018-30236-9] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2018] [Accepted: 07/16/2018] [Indexed: 01/09/2023] Open
Abstract
Bacteraemia is a life-threating condition requiring immediate diagnostic and therapeutic actions. Blood culture (BC) analyses often result in a low true positive result rate, indicating its improper usage. A predictive model might assist clinicians in deciding for whom to conduct or to avoid BC analysis in patients having a relevant bacteraemia risk. Predictive models were established by using linear and non-linear machine learning methods. To obtain proper data, a unique data set was collected prior to model estimation in a prospective cohort study, screening 3,370 standard care patients with suspected bacteraemia. Data from 466 patients fulfilling two or more systemic inflammatory response syndrome criteria (bacteraemia rate: 28.8%) were finally used. A 29 parameter panel of clinical data, cytokine expression levels and standard laboratory markers was used for model training. Model tuning was performed in a ten-fold cross validation and tuned models were validated in a test set (80:20 random split). The random forest strategy presented the best result in the test set validation (ROC-AUC: 0.729, 95%CI: 0.679–0.779). However, procalcitonin (PCT), as the best individual variable, yielded a similar ROC-AUC (0.729, 95%CI: 0.679–0.779). Thus, machine learning methods failed to improve the moderate diagnostic accuracy of PCT.
Collapse
|
10
|
Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc 2016; 24:198-208. [PMID: 27189013 DOI: 10.1093/jamia/ocw042] [Citation(s) in RCA: 424] [Impact Index Per Article: 53.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2015] [Revised: 01/25/2016] [Accepted: 02/20/2016] [Indexed: 12/23/2022] Open
Abstract
OBJECTIVE Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction, presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR based risk prediction modeling through a systematic review of clinical prediction studies using EHR data. METHODS We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation. RESULTS We identified 107 articles from 15 different countries. Studies were generally very large (median sample size = 26 100) and utilized a diverse array of predictors. Most used validation techniques (n = 94 of 107) and reported model coefficients for reproducibility (n = 83). However, studies did not fully leverage the breadth of EHR data, as they uncommonly used longitudinal information (n = 37) and employed relatively few predictor variables (median = 27 variables). Less than half of the studies were multicenter (n = 50) and only 26 performed validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71). CONCLUSIONS EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.
Collapse
Affiliation(s)
- Benjamin A Goldstein
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA .,Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA
| | - Ann Marie Navar
- Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA.,Division of Cardiology at Duke University Medical Center, Duhram, NC 27710, USA
| | - Michael J Pencina
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA.,Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA
| | - John P A Ioannidis
- Department of Medicine, Stanford University, Palo Alto, CA 94305, USA.,Department of Health Research and Policy, and Statistics and Meta-Research Innovation Center at Stanford, Stanford University, Palo Alto, CA 94305, USA
| |
Collapse
|
11
|
Tudela P, Giménez M, Mòdol JM, Prat C. Hemocultivos en los servicios de urgencias, ¿hacia un nuevo enfoque? Med Clin (Barc) 2016; 146:455-9. [DOI: 10.1016/j.medcli.2015.11.026] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2015] [Revised: 11/24/2015] [Accepted: 11/26/2015] [Indexed: 01/10/2023]
|
12
|
Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models--a systematic review. Clin Microbiol Infect 2015; 21:295-301. [PMID: 25677625 DOI: 10.1016/j.cmi.2015.01.023] [Citation(s) in RCA: 48] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2014] [Revised: 01/21/2015] [Accepted: 01/22/2015] [Indexed: 11/18/2022]
Abstract
Bacteraemia is associated with high mortality. Although many models for predicting bacteraemia have been developed, not all have been validated, and even when they were, the validation processes varied. We identified validated models that have been developed; asked whether they were successful in defining groups with a very low or high prevalence of bacteraemia; and whether they were used in clinical practice. Electronic databases were searched to identify studies that underwent validation on prediction of bacteraemia in adults. We included only studies that were able to define groups with low or high probabilities for bacteraemia (arbitrarily defined as below 3% or above 30%). Fifteen publications fulfilled inclusion criteria, including 59 276 patients. Eleven were prospective and four retrospective. Study populations and the parameters included in the different models were heterogeneous. Ten studies underwent internal validation; the model performed well in all of them. Twelve performed external validation. Of the latter, seven models were validated in a different hospital, using a new independent database. In five of these, the model performed well. After contacting authors, we found that none of the models was implemented in clinical practice. We conclude that heterogeneous studies have been conducted in different defined groups of patients with limited external validation. Significant savings to the system and the individual patient can be gained by refraining from performing blood cultures in groups of patients in which the probability of true bacteraemia is very low, while the probability of contamination is constant. Clinical trials of existing or new models should be done to examine whether models are helpful and safe in clinical use, preferably multicentre in order to secure utility and safety in diverse clinical settings.
Collapse
Affiliation(s)
- N Eliakim-Raz
- Unit of Infectious Diseases Rabin Medical Center, Beilinson Hospital, Petah-Tikva, Israel; Sackler Faculty of Medicine, Tel Aviv University, Ramat Aviv, Israel.
| | - D W Bates
- Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA; Department of Health Policy and Management, Harvard School of Public Health, Boston, MA, USA
| | - L Leibovici
- Department of Medicine E, Rabin Medical Center, Beilinson Hospital, Petah-Tikva, Israel; Sackler Faculty of Medicine, Tel Aviv University, Ramat Aviv, Israel
| |
Collapse
|
13
|
Ratzinger F, Dedeyan M, Rammerstorfer M, Perkmann T, Burgmann H, Makristathis A, Dorffner G, Lötsch F, Blacky A, Ramharter M. A risk prediction model for screening bacteremic patients: a cross sectional study. PLoS One 2014; 9:e106765. [PMID: 25184209 PMCID: PMC4153716 DOI: 10.1371/journal.pone.0106765] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2014] [Accepted: 08/08/2014] [Indexed: 12/01/2022] Open
Abstract
Background Bacteraemia is a frequent and severe condition with a high mortality rate. Despite profound knowledge about the pre-test probability of bacteraemia, blood culture analysis often results in low rates of pathogen detection and therefore increasing diagnostic costs. To improve the cost-effectiveness of blood culture sampling, we computed a risk prediction model based on highly standardizable variables, with the ultimate goal to identify via an automated decision support tool patients with very low risk for bacteraemia. Methods In this retrospective hospital-wide cohort study evaluating 15,985 patients with suspected bacteraemia, 51 variables were assessed for their diagnostic potency. A derivation cohort (n = 14.699) was used for feature and model selection as well as for cut-off specification. Models were established using the A2DE classifier, a supervised Bayesian classifier. Two internally validated models were further evaluated by a validation cohort (n = 1,286). Results The proportion of neutrophile leukocytes in differential blood count was the best individual variable to predict bacteraemia (ROC-AUC: 0.694). Applying the A2DE classifier, two models, model 1 (20 variables) and model 2 (10 variables) were established with an area under the receiver operating characteristic curve (ROC-AUC) of 0.767 and 0.759, respectively. In the validation cohort, ROC-AUCs of 0.800 and 0.786 were achieved. Using predefined cut-off points, 16% and 12% of patients were allocated to the low risk group with a negative predictive value of more than 98.8%. Conclusion Applying the proposed models, more than ten percent of patients with suspected blood stream infection were identified having minimal risk for bacteraemia. Based on these data the application of this model as an automated decision support tool for physicians is conceivable leading to a potential increase in the cost-effectiveness of blood culture sampling. External prospective validation of the model's generalizability is needed for further appreciation of the usefulness of this tool.
Collapse
Affiliation(s)
- Franz Ratzinger
- Department of Laboratory Medicine, Division of Medical and Chemical Laboratory Diagnostics, Medical University of Vienna, Vienna, Austria
| | - Michel Dedeyan
- Department of Medicine I, Division of Infectious Diseases and Tropical Medicine, Medical University Vienna, Vienna, Austria
| | - Matthias Rammerstorfer
- Department of Medicine I, Division of Infectious Diseases and Tropical Medicine, Medical University Vienna, Vienna, Austria
| | - Thomas Perkmann
- Department of Laboratory Medicine, Division of Medical and Chemical Laboratory Diagnostics, Medical University of Vienna, Vienna, Austria
| | - Heinz Burgmann
- Department of Medicine I, Division of Infectious Diseases and Tropical Medicine, Medical University Vienna, Vienna, Austria
| | - Athanasios Makristathis
- Department of Laboratory Medicine, Division of Clinical Microbiology, Medical University of Vienna, Vienna, Austria
| | - Georg Dorffner
- Section for Artificial Intelligence, Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria
| | - Felix Lötsch
- Department of Medicine I, Division of Infectious Diseases and Tropical Medicine, Medical University Vienna, Vienna, Austria
| | - Alexander Blacky
- Clinical Institute for Hospital Hygiene, Medical University of Vienna, Vienna, Austria
| | - Michael Ramharter
- Department of Medicine I, Division of Infectious Diseases and Tropical Medicine, Medical University Vienna, Vienna, Austria
- Institut für Tropenmedizin, Universität Tübingen, Tübingen, Germany
- * E-mail:
| |
Collapse
|