1
|
Li QY, Tang BH, Wu YE, Yao BF, Zhang W, Zheng Y, Zhou Y, van den Anker J, Hao GX, Zhao W. Machine Learning: A New Approach for Dose Individualization. Clin Pharmacol Ther 2024; 115:727-744. [PMID: 37713106 DOI: 10.1002/cpt.3049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Accepted: 09/03/2023] [Indexed: 09/16/2023]
Abstract
The application of machine learning (ML) has shown promising results in precision medicine due to its exceptional performance in dealing with complex multidimensional data. However, using ML for individualized dosing of medicines is still in its early stage, meriting further exploration. A systematic review of study designs and modeling details of using ML for individualized dosing of different drugs was performed. We have summarized the status of the study populations, predictive targets, and data sources for ML modeling, the selection of ML algorithms and features, and the evaluation and validation of their predictive performance. We also used the Prediction model Risk of Bias Assessment Tool (PROBAST) to assess the risk of bias of included studies. Currently, ML can be used for both a priori and a posteriori dose selection and optimization, and it can also assist the implementation of therapeutic drug monitoring. However, studies are mainly focused on drugs with narrow therapeutic windows, predominantly immunosuppressants (N = 23, 35.9%) and anti-infectives (N = 21, 32.8%), and there is currently only very limited attention for special populations, such as children (N = 22, 34.4%). Most studies showed poor methodological quality and a high risk of bias. The lack of external validation and clinical utility evaluation currently limits the further clinical implementation of ML for dose individualization. We therefore have proposed several ways to improve the clinical relevance of the studies and facilitate the translation of ML models into clinical practice.
Collapse
Affiliation(s)
- Qiu-Yue Li
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Bo-Hao Tang
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yue-E Wu
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Bu-Fan Yao
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Wei Zhang
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yi Zheng
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Yue Zhou
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - John van den Anker
- Division of Clinical Pharmacology, Children's National Hospital, Washington, DC, USA
- Departments of Pediatrics, Pharmacology & Physiology, Genomics & Precision Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC, USA
- Department of Pediatric Pharmacology and Pharmacometrics, University of Basel Children's Hospital, Basel, Switzerland
| | - Guo-Xiang Hao
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
| | - Wei Zhao
- Department of Clinical Pharmacy, Institute of Clinical Pharmacology, Key Laboratory of Chemical Biology (Ministry of Education),NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, School of Pharmaceutical Sciences, Cheeloo College of Medicine, Shandong University, Jinan, China
- NMPA Key Laboratory for Clinical Research and Evaluation of Innovative Drug, Qilu Hospital of Shandong University, Shandong University, Jinan, China
| |
Collapse
|
2
|
Fu R, Hao X, Yu J, Wang D, Zhang J, Yu Z, Gao F, Zhou C. Machine learning-based prediction of sertraline concentration in patients with depression through therapeutic drug monitoring. Front Pharmacol 2024; 15:1289673. [PMID: 38510645 PMCID: PMC10953499 DOI: 10.3389/fphar.2024.1289673] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Accepted: 02/21/2024] [Indexed: 03/22/2024] Open
Abstract
Background: Sertraline is a commonly employed antidepressant in clinical practice. In order to control the plasma concentration of sertraline within the therapeutic window to achieve the best effect and avoid adverse reactions, a personalized model to predict sertraline concentration is necessary. Aims: This study aimed to establish a personalized medication model for patients with depression receiving sertraline based on machine learning to provide a reference for clinicians to formulate drug regimens. Methods: A total of 415 patients with 496 samples of sertraline concentration from December 2019 to July 2022 at the First Hospital of Hebei Medical University were collected as the dataset. Nine different algorithms, namely, XGBoost, LightGBM, CatBoost, random forest, GBDT, SVM, lasso regression, ANN, and TabNet, were used for modeling to compare the model abilities to predict sertraline concentration. Results: XGBoost was chosen to establish the personalized medication model with the best performance (R 2 = 0.63). Five important variables, namely, sertraline dose, alanine transaminase, aspartate transaminase, uric acid, and sex, were shown to be correlated with sertraline concentration. The model prediction accuracy of sertraline concentration in the therapeutic window was 62.5%. Conclusion: In conclusion, the personalized medication model of sertraline for patients with depression based on XGBoost had good predictive ability, which provides guidance for clinicians in proposing an optimal medication regimen.
Collapse
Affiliation(s)
- Ran Fu
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Xin Hao
- Dalian Medicinovo Technology Co., Ltd, Dalian, China
| | - Jing Yu
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Donghan Wang
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Jinyuan Zhang
- Beijing Medicinovo Technology Co., Ltd, Beijing, China
| | - Ze Yu
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Fei Gao
- Beijing Medicinovo Technology Co., Ltd, Beijing, China
| | - Chunhua Zhou
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| |
Collapse
|
3
|
Ran P, Tan T, Li J, Yang H, Li J, Zhang J. Advanced gastrointestinal stromal tumor: reliable classification of imatinib plasma trough concentration via machine learning. BMC Cancer 2024; 24:264. [PMID: 38402382 PMCID: PMC10894477 DOI: 10.1186/s12885-024-11930-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 01/29/2024] [Indexed: 02/26/2024] Open
Abstract
AIM Patients with advanced gastrointestinal stromal tumors (GISTs) exhibiting an imatinib plasma trough concentration (IM Cmin) under 1100 ng/ml may show a reduced drug response rate, leading to the suggestion of monitoring for IM Cmin. Consequently, the objective of this research was to create a customized IM Cmin classification model for patients with advanced GISTs from China. METHODS Initial data and laboratory indicators from patients with advanced GISTs were gathered, and the above information was segmented into a training set, validation set, and testing set in a 6:2:2 ratio. Key variables associated with IM Cmin were identified to construct the classification model using the least absolute shrinkage and selection operator (LASSO) regression and forward stepwise binary logistic regression. Within the training and validation sets, nine ML classification models were constructed via the resampling method and underwent comparison through the Brier scores, the areas under the receiver-operating characteristic curve (AUROC), the decision curve, and the precision-recall (AUPR) curve to determine the most suitable model for this dataset. Two methods of internal validation were used to assess the most suitable model's classification performance: tenfold cross-validation and random split-sample validation (test set), and the value of the test set AUROC was used to evaluate the model's classification performance. RESULTS Six key variables (gender, daily IM dose, metastatic site, red blood cell count, platelet count, and percentage of neutrophils) were ultimately selected to construct the classification model. In the validation set, it is found by comparison that the Extreme Gradient Boosting (XGBoost) model has the largest AUROC, the lowest Brier score, the largest area under the decision curve, and the largest AUPR value. Furthermore, as evaluated via internal verification, it also performed well in the test set (AUROC = 0.725). CONCLUSION For patients with advanced GISTs who receive IM, initial data and laboratory indicators could be used to accurately estimate whether the IM Cmin is below 1100 ng/ml. The XGBoost model may stand a chance to assist clinicians in directing the administration of IM.
Collapse
Affiliation(s)
- Pan Ran
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China
| | - Tao Tan
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China
| | - Jinjin Li
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China
| | - Hao Yang
- Department of Internal Medicine, Chongqing Key Laboratory of Translational Research for Cancer Metastasis and Individualized Treatment, Chongqing University Cancer Hospital, Chongqing, 400030, China
| | - Juan Li
- Department of Pharmacy, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China.
| | - Jun Zhang
- Department of Gastrointestinal Surgery, the First Affiliated Hospital of Chongqing Medical University, Chongqing, 400016, China.
| |
Collapse
|
4
|
Nose D, Matsui T, Otsuka T, Matsuda Y, Arimura T, Yasumoto K, Sugimoto M, Miura SI. Development of Machine Learning-Based Web System for Estimating Pleural Effusion Using Multi-Frequency Bioelectrical Impedance Analyses. J Cardiovasc Dev Dis 2023; 10:291. [PMID: 37504547 PMCID: PMC10380905 DOI: 10.3390/jcdd10070291] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 07/04/2023] [Accepted: 07/05/2023] [Indexed: 07/29/2023] Open
Abstract
BACKGROUND Transthoracic impedance values have not been widely used to measure extravascular pulmonary water content due to accuracy and complexity concerns. Our aim was to develop a foundational model for a novel system aiming to non-invasively estimate the intrathoracic condition of heart failure patients. METHODS We employed multi-frequency bioelectrical impedance analysis to simultaneously measure multiple frequencies, collecting electrical, physical, and hematological data from 63 hospitalized heart failure patients and 82 healthy volunteers. Measurements were taken upon admission and after treatment, and longitudinal analysis was conducted. RESULTS Using a light gradient boosting machine, and a decision tree-based machine learning method, we developed an intrathoracic estimation model based on electrical measurements and clinical findings. Out of the 286 features collected, the model utilized 16 features. Notably, the developed model demonstrated high accuracy in discriminating patients with pleural effusion, achieving an area under the receiver characteristic curves (AUC) of 0.905 (95% CI: 0.870-0.940, p < 0.0001) in the cross-validation test. The accuracy significantly outperformed the conventional frequency-based method with an AUC of 0.740 (95% CI: 0.688-0.792, and p < 0.0001). CONCLUSIONS Our findings indicate the potential of machine learning and transthoracic impedance measurements for estimating pleural effusion. By incorporating noninvasive and easily obtainable clinical and laboratory findings, this approach offers an effective means of assessing intrathoracic conditions.
Collapse
Affiliation(s)
- Daisuke Nose
- Department of Cardiology, Fukuoka University Faculty of Medicine, Fukuoka 814-0180, Japan
- Department of Cardiology, Fukuoka Heartnet Hospital, Fukuoka 819-0002, Japan
- Research Institute for Advanced Medical Development for Heart Failure, Fukuoka University, Fukuoka 814-0180, Japan
| | - Tomokazu Matsui
- Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 690-0101, Japan
| | - Takuya Otsuka
- Technical Sales Department, Dialysis Division, Toray Medical Company Limited, Tokyo 103-0023, Japan
| | - Yuki Matsuda
- Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 690-0101, Japan
| | - Tadaaki Arimura
- Department of Cardiology, Fukuoka University Faculty of Medicine, Fukuoka 814-0180, Japan
| | - Keiichi Yasumoto
- Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 690-0101, Japan
| | - Masahiro Sugimoto
- Institute for Advanced Biosciences, Keio University, Tsuruoka 997-0035, Japan
- Institute of Medical Science, Tokyo Medical University, Tokyo 160-0023, Japan
| | - Shin-Ichiro Miura
- Department of Cardiology, Fukuoka University Faculty of Medicine, Fukuoka 814-0180, Japan
- Research Institute for Advanced Medical Development for Heart Failure, Fukuoka University, Fukuoka 814-0180, Japan
| |
Collapse
|
5
|
Yang L, Zhang J, Yu J, Yu Z, Hao X, Gao F, Zhou C. Predicting plasma concentration of quetiapine in patients with depression using machine learning techniques based on real-world evidence. Expert Rev Clin Pharmacol 2023; 16:741-750. [PMID: 37466101 DOI: 10.1080/17512433.2023.2238604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Revised: 06/19/2023] [Accepted: 07/13/2023] [Indexed: 07/20/2023]
Abstract
OBJECTIVES We develop a model for predicting quetiapine levels in patients with depression, using machine learning to support decisions on clinical regimens. METHODS Inpatients diagnosed with depression at the First Hospital of Hebei Medical University from 1 November 2019, to 31 August were enrolled. The ratio of training cohort to testing cohort was fixed at 80%:20% for the whole dataset. Univariate analysis was executed on all information to screen the important variables influencing quetiapine TDM. The prediction abilities of nine machine learning and deep learning algorithms were compared. The prediction model was created using an algorithm with better model performance, and the model's interpretation was done using the SHapley Additive exPlanation. RESULTS There were 333 individuals and 412 cases of quetiapine TDM included in the study. Six significant variables were selected to establish the individualized medication model. A quetiapine concentration prediction model was created through CatBoost. In the testing cohort, the projected TDM's accuracy was 61.45%. The prediction accuracy of quetiapine concentration within the effective range (200-750 ng/mL) was 75.47%. CONCLUSIONS This study predicts the plasma concentration of quetiapine in depression patients by machine learning, which is meaningful for the clinical medication guidance.
Collapse
Affiliation(s)
- Lin Yang
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Jinyuan Zhang
- Beijing Medicinovo Technology Co, Ltd, Beijing, China
| | - Jing Yu
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| | - Ze Yu
- Institute of Interdisciplinary Integrative Medicine Research, Shanghai University of Traditional Chinese Medicine, Shanghai, China
| | - Xin Hao
- Dalian Medicinovo Technology Co, Ltd, Dalian, China
| | - Fei Gao
- Beijing Medicinovo Technology Co, Ltd, Beijing, China
| | - Chunhua Zhou
- Department of Clinical Pharmacy, The First Hospital of Hebei Medical University, Shijiazhuang, China
- The Technology Innovation Center for Artificial Intelligence in Clinical Pharmacy of Hebei Province, The First Hospital of Hebei Medical University, Shijiazhuang, China
| |
Collapse
|
6
|
Kırboğa KK, Abbasi S, Küçüksille EU. Explainability and white box in drug discovery. Chem Biol Drug Des 2023; 102:217-233. [PMID: 37105727 DOI: 10.1111/cbdd.14262] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Revised: 03/24/2023] [Accepted: 04/12/2023] [Indexed: 04/29/2023]
Abstract
Recently, artificial intelligence (AI) techniques have been increasingly used to overcome the challenges in drug discovery. Although traditional AI techniques generally have high accuracy rates, there may be difficulties in explaining the decision process and patterns. This can create difficulties in understanding and making sense of the outputs of algorithms used in drug discovery. Therefore, using explainable AI (XAI) techniques, the causes and consequences of the decision process are better understood. This can help further improve the drug discovery process and make the right decisions. To address this issue, Explainable Artificial Intelligence (XAI) emerged as a process and method that securely captures the results and outputs of machine learning (ML) and deep learning (DL) algorithms. Using techniques such as SHAP (SHApley Additive ExPlanations) and LIME (Locally Interpretable Model-Independent Explanations) has made the drug targeting phase clearer and more understandable. XAI methods are expected to reduce time and cost in future computational drug discovery studies. This review provides a comprehensive overview of XAI-based drug discovery and development prediction. XAI mechanisms to increase confidence in AI and modeling methods. The limitations and future directions of XAI in drug discovery are also discussed.
Collapse
Affiliation(s)
- Kevser Kübra Kırboğa
- Bioengineering Department, Bilecik Seyh Edebali University, Bilecik, Turkey
- Informatics Institute, Istanbul Technical University, Maslak, Turkey
| | - Sumra Abbasi
- Department of Biological Sciences, National of Medical Sciences, Rawalpindi, Pakistan
| | - Ecir Uğur Küçüksille
- Department of Computer Engineering, Süleyman Demirel University, Isparta, Turkey
| |
Collapse
|
7
|
Zhu X, Zhang M, Wen Y, Shang D. Machine learning advances the integration of covariates in population pharmacokinetic models: Valproic acid as an example. Front Pharmacol 2022; 13:994665. [PMID: 36324679 PMCID: PMC9621318 DOI: 10.3389/fphar.2022.994665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2022] [Accepted: 10/03/2022] [Indexed: 11/24/2022] Open
Abstract
Background and Aim: Many studies associated with the combination of machine learning (ML) and pharmacometrics have appeared in recent years. ML can be used as an initial step for fast screening of covariates in population pharmacokinetic (popPK) models. The present study aimed to integrate covariates derived from different popPK models using ML. Methods: Two published popPK models of valproic acid (VPA) in Chinese epileptic patients were used, where the population parameters were influenced by some covariates. Based on the covariates and a one-compartment model that describes the pharmacokinetics of VPA, a dataset was constructed using Monte Carlo simulation, to develop an XGBoost model to estimate the steady-state concentrations (Css) of VPA. We utilized SHapley Additive exPlanation (SHAP) values to interpret the prediction model, and calculated estimates of VPA exposure in four assumed scenarios involving different combinations of CYP2C19 genotypes and co-administered antiepileptic drugs. To develop an easy-to-use model in the clinic, we built a simplified model by using CYP2C19 genotypes and some noninvasive clinical parameters, and omitting several features that were infrequently measured or whose clinically available values were inaccurate, and verified it on our independent external dataset. Results: After data preprocessing, the finally generated combined dataset was divided into a derivation cohort and a validation cohort (8:2). The XGBoost model was developed in the derivation cohort and yielded excellent performance in the validation cohort with a mean absolute error of 2.4 mg/L, root-mean-squared error of 3.3 mg/L, mean relative error of 0%, and percentages within ±20% of actual values of 98.85%. The SHAP analysis revealed that daily dose, time, CYP2C19*2 and/or *3 variants, albumin, body weight, single dose, and CYP2C19*1*1 genotype were the top seven confounding factors influencing the Css of VPA. Under the simulated dosage regimen of 500 mg/bid, the VPA exposure in patients who had CYP2C19*2 and/or *3 variants and no carbamazepine, phenytoin, or phenobarbital treatment, was approximately 1.74-fold compared to those with CYP2C19*1/*1 genotype and co-administered carbamazepine + phenytoin + phenobarbital. The feasibility of the simplified model was fully illustrated by its performance in our external dataset. Conclusion: This study highlighted the bridging role of ML in big data and pharmacometrics, by integrating covariates derived from different popPK models.
Collapse
Affiliation(s)
- Xiuqing Zhu
- Department of Pharmacy, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
| | - Ming Zhang
- Department of Pharmacy, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
| | - Yuguan Wen
- Department of Pharmacy, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
- *Correspondence: Yuguan Wen, ; Dewei Shang,
| | - Dewei Shang
- Department of Pharmacy, The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
- Guangdong Engineering Technology Research Center for Translational Medicine of Mental Disorders, Guangzhou, China
- *Correspondence: Yuguan Wen, ; Dewei Shang,
| |
Collapse
|
8
|
Verhaeghe J, Dhaese SAM, De Corte T, Vander Mijnsbrugge D, Aardema H, Zijlstra JG, Verstraete AG, Stove V, Colin P, Ongenae F, De Waele JJ, Van Hoecke S. Development and evaluation of uncertainty quantifying machine learning models to predict piperacillin plasma concentrations in critically ill patients. BMC Med Inform Decis Mak 2022; 22:224. [PMID: 36008808 PMCID: PMC9404625 DOI: 10.1186/s12911-022-01970-y] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Accepted: 08/10/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Beta-lactam antimicrobial concentrations are frequently suboptimal in critically ill patients. Population pharmacokinetic (PopPK) modeling is the golden standard to predict drug concentrations. However, currently available PopPK models often lack predictive accuracy, making them less suited to guide dosing regimen adaptations. Furthermore, many currently developed models for clinical applications often lack uncertainty quantification. We, therefore, aimed to develop machine learning (ML) models for the prediction of piperacillin plasma concentrations while also providing uncertainty quantification with the aim of clinical practice. METHODS Blood samples for piperacillin analysis were prospectively collected from critically ill patients receiving continuous infusion of piperacillin/tazobactam. Interpretable ML models for the prediction of piperacillin concentrations were designed using CatBoost and Gaussian processes. Distribution-based Uncertainty Quantification was added to the CatBoost model using a proposed Quantile Ensemble method, useable for any model optimizing a quantile function. These models are subsequently evaluated using the distribution coverage error, a proposed interpretable uncertainty quantification calibration metric. Development and internal evaluation of the ML models were performed on the Ghent University Hospital database (752 piperacillin concentrations from 282 patients). Ensuing, ML models were compared with a published PopPK model on a database from the University Medical Centre of Groningen where a different dosing regimen is used (46 piperacillin concentrations from 15 patients.). RESULTS The best performing model was the Catboost model with an RMSE and [Formula: see text] of 31.94-0.64 and 33.53-0.60 for internal evaluation with and without previous concentration. Furthermore, the results prove the added value of the proposed Quantile Ensemble model in providing clinically useful individualized uncertainty predictions and show the limits of homoscedastic methods like Gaussian Processes in clinical applications. CONCLUSIONS Our results show that ML models can consistently estimate piperacillin concentrations with acceptable and high predictive accuracy when identical dosing regimens as in the training data are used while providing highly relevant uncertainty predictions. However, generalization capabilities to other dosing schemes are limited. Notwithstanding, incorporating ML models in therapeutic drug monitoring programs seems definitely promising and the current work provides a basis for validating the model in clinical practice.
Collapse
Affiliation(s)
- Jarne Verhaeghe
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium.
| | - Sofie A M Dhaese
- Department of Internal Medicine and Pediatrics, Ghent University, Ghent, Belgium
| | - Thomas De Corte
- Department of Internal Medicine and Pediatrics, Ghent University, Ghent, Belgium
| | | | - Heleen Aardema
- Department of Critical Care, University Medical Center Groningen, Groningen, The Netherlands
| | - Jan G Zijlstra
- Department of Critical Care, University Medical Center Groningen, Groningen, The Netherlands
| | | | - Veronique Stove
- Department of Diagnostic Sciences, Ghent University, Ghent, Belgium
| | - Pieter Colin
- Department of Anesthesiology, University Medical Center Groningen, Groningen, The Netherlands
| | - Femke Ongenae
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium
| | - Jan J De Waele
- Department of Critical Care Medicine, Ghent University Hospital, Ghent, Belgium
| | - Sofie Van Hoecke
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium.
| |
Collapse
|
9
|
Zhang G, Shi Y, Yin P, Liu F, Fang Y, Li X, Zhang Q, Zhang Z. A machine learning model based on ultrasound image features to assess the risk of sentinel lymph node metastasis in breast cancer patients: Applications of scikit-learn and SHAP. Front Oncol 2022; 12:944569. [PMID: 35957890 PMCID: PMC9359803 DOI: 10.3389/fonc.2022.944569] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2022] [Accepted: 07/01/2022] [Indexed: 11/16/2022] Open
Abstract
Background This study aimed to determine an optimal machine learning (ML) model for evaluating the preoperative diagnostic value of ultrasound signs of breast cancer lesions for sentinel lymph node (SLN) status. Method This study retrospectively analyzed the ultrasound images and postoperative pathological findings of lesions in 952 breast cancer patients. Firstly, the univariate analysis of the relationship between the ultrasonographic features of breast cancer morphological features and SLN metastasis. Then, based on the ultrasound signs of breast cancer lesions, we screened ten ML models: support vector machine (SVM), extreme gradient boosting (XGBoost), random forest (RF), linear discriminant analysis (LDA), logistic regression (LR), naive bayesian model (NB), k-nearest neighbors (KNN), multilayer perceptron (MLP), long short-term memory (LSTM), and convolutional neural network (CNN). The diagnostic performance of the model was evaluated using the area under the receiver operating characteristic (ROC) curve (AUC), Kappa value, accuracy, F1-score, sensitivity, and specificity. Then we constructed a clinical prediction model which was based on the ML algorithm with the best diagnostic performance. Finally, we used SHapley Additive exPlanation (SHAP) to visualize and analyze the diagnostic process of the ML model. Results Of 952 patients with breast cancer, 394 (41.4%) had SLN metastasis, and 558 (58.6%) had no metastasis. Univariate analysis found that the shape, orientation, margin, posterior features, calculations, architectural distortion, duct changes and suspicious lymph node of breast cancer lesions in ultrasound signs were associated with SLN metastasis. Among the 10 ML algorithms, XGBoost had the best comprehensive diagnostic performance for SLN metastasis, with Average-AUC of 0.952, Average-Kappa of 0.763, and Average-Accuracy of 0.891. The AUC of the XGBoost model in the validation cohort was 0.916, the accuracy was 0.846, the sensitivity was 0.870, the specificity was 0.862, and the F1-score was 0.826. The diagnostic performance of the XGBoost model was significantly higher than that of experienced radiologists in some cases (P<0.001). Using SHAP to visualize the interpretation of the ML model screen, it was found that the ultrasonic detection of suspicious lymph nodes, microcalcifications in the primary tumor, burrs on the edge of the primary tumor, and distortion of the tissue structure around the lesion contributed greatly to the diagnostic performance of the XGBoost model. Conclusions The XGBoost model based on the ultrasound signs of the primary breast tumor and its surrounding tissues and lymph nodes has a high diagnostic performance for predicting SLN metastasis. Visual explanation using SHAP made it an effective tool for guiding clinical courses preoperatively.
Collapse
Affiliation(s)
- Gaosen Zhang
- Department of Ultrasound, First Affiliated Hospital of China Medical University, Shenyang, China
| | - Yan Shi
- Department of Ultrasound, Binzhou Medical University Hospital, Binzhou, China
| | - Peipei Yin
- Department of Ultrasound, Binzhou Medical University Hospital, Binzhou, China
| | - Feifei Liu
- Department of Ultrasound Medicine, Peking University People’s Hospital, Beijing, China
| | - Yi Fang
- Department of Ultrasound, First Affiliated Hospital of China Medical University, Shenyang, China
| | - Xiang Li
- Department of Ultrasound, First Affiliated Hospital of China Medical University, Shenyang, China
| | - Qingyu Zhang
- College of Information Science and Engineering, Northeastern University, Shenyang, China
| | - Zhen Zhang
- Department of Ultrasound, First Affiliated Hospital of China Medical University, Shenyang, China
- *Correspondence: Zhen Zhang,
| |
Collapse
|