1
|
G A, K L N, M S AS. Improving sepsis classification performance with artificial intelligence algorithms: A comprehensive overview of healthcare applications. J Crit Care 2024; 83:154815. [PMID: 38723336 DOI: 10.1016/j.jcrc.2024.154815] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 04/10/2024] [Accepted: 04/13/2024] [Indexed: 08/04/2024]
Abstract
PURPOSE This study investigates the potential of machine learning (ML) algorithms in improving sepsis diagnosis and prediction, focusing on their relevance in healthcare decision-making. The primary objective is to contribute to healthcare decision-making by evaluating the performance of various supervised and unsupervised models. MATERIALS AND METHODS Through an extensive literature review, optimal ML models used in sepsis research were identified. Diverse datasets from relevant sources were employed, and rigorous evaluation metrics, including accuracy, specificity, and sensitivity, were applied. Innovative techniques were introduced, such as a Stacked Blended Ensemble Model and Skopt Optimization with Blended Ensemble, incorporating Bayesian optimization for hyperparameter tuning. RESULTS ML algorithms demonstrate efficacy in sepsis diagnosis, presenting an improved balance between specificity and sensitivity, critical for effective clinical decision-making. Classifier ensemble models show enhanced accuracy and efficiency, with novel optimization techniques contributing to improved adaptability. CONCLUSION The study emphasizes the potential benefits of ML algorithms in sepsis management, advocating for ongoing research to optimize performance and ensure ethical utilization in healthcare decision-making. Ethical considerations, interpretability, and transparency are crucial factors in implementing these algorithms in clinical practice.
Collapse
Affiliation(s)
- Anjana G
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India
| | - Nisha K L
- Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kerala, India.
| | - Arun Sankar M S
- School of CS&IT, University College Cork, Cork, Ireland; Electronic and Communications Department, South East Technological University, Carlow, Ireland
| |
Collapse
|
2
|
Liu Y, Wang H, Zhou H, Li M, Hou Y, Zhou S, Wang F, Hoetzlein R, Zhang R. A review of reinforcement learning for natural language processing and applications in healthcare. J Am Med Inform Assoc 2024:ocae215. [PMID: 39208319 DOI: 10.1093/jamia/ocae215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2024] [Revised: 07/01/2024] [Accepted: 08/01/2024] [Indexed: 09/04/2024] Open
Abstract
IMPORTANCE Reinforcement learning (RL) represents a pivotal avenue within natural language processing (NLP), offering a potent mechanism for acquiring optimal strategies in task completion. This literature review studies various NLP applications where RL has demonstrated efficacy, with notable applications in healthcare settings. OBJECTIVES To systematically explore the applications of RL in NLP, focusing on its effectiveness in acquiring optimal strategies, particularly in healthcare settings, and provide a comprehensive understanding of RL's potential in NLP tasks. MATERIALS AND METHODS Adhering to the PRISMA guidelines, an exhaustive literature review was conducted to identify instances where RL has exhibited success in NLP applications, encompassing dialogue systems, machine translation, question-answering, text summarization, and information extraction. Our methodological approach involves closely examining the technical aspects of RL methodologies employed in these applications, analyzing algorithms, states, rewards, actions, datasets, and encoder-decoder architectures. RESULTS The review of 93 papers yields insights into RL algorithms, prevalent techniques, emergent trends, and the fusion of RL methods in NLP healthcare applications. It clarifies the strategic approaches employed, datasets utilized, and the dynamic terrain of RL-NLP systems, thereby offering a roadmap for research and development in RL and machine learning techniques in healthcare. The review also addresses ethical concerns to ensure equity, transparency, and accountability in the evolution and application of RL-based NLP technologies, particularly within sensitive domains such as healthcare. DISCUSSION The findings underscore the promising role of RL in advancing NLP applications, particularly in healthcare, where its potential to optimize decision-making and enhance patient outcomes is significant. However, the ethical challenges and technical complexities associated with RL demand careful consideration and ongoing research to ensure responsible and effective implementation. CONCLUSIONS By systematically exploring RL's applications in NLP and providing insights into technical analysis, ethical implications, and potential advancements, this review contributes to a deeper understanding of RL's role for language processing.
Collapse
Affiliation(s)
- Ying Liu
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
| | - Haozhu Wang
- Amazon Web Service, Seattle, WA 98109, United States
| | - Huixue Zhou
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
- Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States
| | - Mingchen Li
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
| | - Yu Hou
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
| | - Sicheng Zhou
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
- Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States
| | - Fang Wang
- Amazon Web Service, Seattle, WA 98109, United States
| | | | - Rui Zhang
- Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States
| |
Collapse
|
3
|
Liu S, Xu Q, Xu Z, Liu Z, Sun X, Xie G, Feng M, See KC. Reinforcement Learning to Optimize Ventilator Settings for Patients on Invasive Mechanical Ventilation: A Retrospective Study. J Med Internet Res 2024. [PMID: 39219230 DOI: 10.2196/44494] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/04/2024] Open
Abstract
BACKGROUND One of the significant changes in intensive care medicine over the past two decades is the acknowledgement that improper mechanical ventilation settings substantially contribute to pulmonary injury in critically ill patients. Artificial intelligence (AI) solutions can be used to optimize mechanical ventilation settings in intensive care units (ICUs) and to improve patient outcomes. Specifically, machine learning algorithms can be trained on large datasets of patient information and mechanical ventilation settings. These algorithms can then predict patient responses to different ventilation strategies and suggest personalized ventilation settings for individual patients. OBJECTIVE In this study, we aimed to design and evaluate an AI solution that could tailor an optimal ventilator strategy for each critically ill patient who requires mechanical ventilation. METHODS We proposed a reinforcement learning-based AI solution using observational data from multiple ICUs in the US. The primary outcome was hospital mortality. Secondary outcomes were the proportion of optimal oxygen saturation and the proportion of optimal mean arterial blood pressure. We trained our AI agent to recommend low/medium/high levels of three ventilator settings - positive end-expiratory pressure (PEEP), fraction of inspired oxygen (FiO2) and ideal body weight-adjusted tidal volume - according to patients' health conditions. We defined a policy as a set of rules guiding ventilator setting changes given specific clinical scenarios. Off-policy evaluation metrics were applied to evaluate the AI policy. RESULTS We studied 21,595 and 5,105 patients' ICU stays from the eICU Collaborative Research (eICU) and Medical Information Mart for Intensive Care IV (MIMIC-IV) databases respectively. Using the learnt AI policy, we estimated the hospital mortality rate (eICU 12.1±3.1%; MIMIC-IV 29.1±0.9%), proportion of optimal oxygen saturation (eICU 58.7±4.7%; MIMIC-IV 49.0±1.0%), and proportion of optimal mean arterial blood pressure (eICU 31.1±4.5%; MIMIC-IV 41.2±1.0%). Based on multiple quantitative and qualitative evaluation metrics, our proposed AI solution outperformed observed clinical practice.We studied 21,595 5105 and 5,105 21595 patients' ' ICU stays from the Medical Information Mart for Intensive Care -IV (MIMIC-IV)(1) and eICU Collaborative Research (eICU) and Medical Information Mart for Intensive Care IV (MIMIC-IV) databases respectively. Observed hospital mortality rates were 18.2% (eICU) and 31.1% (MIMIC-IV). UsingFor the learnt AI policy, we estimated the hospital mortality rate (eICU 14.7±0.7%; MIMIC-IV 29.1±0.9%), proportion of optimal oxygen saturation (eICU 57.8±1.0%; MIMIC-IV 49.0±1.0%), and proportion of optimal mean arterial blood pressure (eICU 34.7 ± 1.0%; MIMIC-IV 41.2±1.0%). Based on multiple quantitative and qualitative evaluation metrics, our proposed AI solution has potential to outperformed observed clinical practice. CONCLUSIONS Our study found that customizing ventilation settings for individual patients led to lower estimated hospital mortality rates compared to actual rates. This highlights the potential effectiveness of using RL methodology to develop AI models that analyze complex clinical data for optimizing treatment parameters. Additionally, our findings suggest the integration of this model into a clinical decision support system for refining ventilation settings, supporting the need for prospective validation trials. CLINICALTRIAL
Collapse
Affiliation(s)
- Siqi Liu
- NUS Graduate School for Integrative Science and Engineering, National University of Singapore, Singapore, SG
| | - Qianyi Xu
- Saw Swee Hock School of Public Health, National University of Singapore, 12 Science Drive 2, Singapore, SG
| | | | - Zhuo Liu
- Ping An Healthcare Technology, Beijing, CN
| | | | | | - Mengling Feng
- Saw Swee Hock School of Public Health, National University of Singapore, 12 Science Drive 2, Singapore, SG
- Institute of Data Science, National University of Singapore, Singapore, SG
| | - Kay Choong See
- Division of Respiratory and Critical Care Medicine, Department of Medicine, National University Hospital, Singapore, SG
| |
Collapse
|
4
|
Salama V, Godinich B, Geng Y, Humbert-Vidan L, Maule L, Wahid KA, Naser MA, He R, Mohamed ASR, Fuller CD, Moreno AC. Artificial Intelligence and Machine Learning in Cancer Pain: A Systematic Review. J Pain Symptom Manage 2024:S0885-3924(24)00908-4. [PMID: 39097246 DOI: 10.1016/j.jpainsymman.2024.07.025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Revised: 07/22/2024] [Accepted: 07/23/2024] [Indexed: 08/05/2024]
Abstract
BACKGROUND/OBJECTIVES Pain is a challenging multifaceted symptom reported by most cancer patients. This systematic review aims to explore applications of artificial intelligence/machine learning (AI/ML) in predicting pain-related outcomes and pain management in cancer. METHODS A comprehensive search of Ovid MEDLINE, EMBASE and Web of Science databases was conducted using terms: "Cancer," "Pain," "Pain Management," "Analgesics," "Artificial Intelligence," "Machine Learning," and "Neural Networks" published up to September 7, 2023. AI/ML models, their validation and performance were summarized. Quality assessment was conducted using PROBAST risk-of-bias andadherence to TRIPOD guidelines. RESULTS Forty four studies from 2006 to 2023 were included. Nineteen studies used AI/ML for classifying pain after cancer therapy [median AUC 0.80 (range 0.76-0.94)]. Eighteen studies focused on cancer pain research [median AUC 0.86 (range 0.50-0.99)], and 7 focused on applying AI/ML for cancer pain management, [median AUC 0.71 (range 0.47-0.89)]. Median AUC (0.77) of models across all studies. Random forest models demonstrated the highest performance (median AUC 0.81), lasso models had the highest median sensitivity (1), while Support Vector Machine had the highest median specificity (0.74). Overall adherence to TRIPOD guidelines was 70.7%. Overall, high risk-of-bias (77.3%), lack of external validation (14%) and clinical application (23%) was detected. Reporting of model calibration was also missing (5%). CONCLUSION Implementation of AI/ML tools promises significant advances in the classification, risk stratification, and management decisions for cancer pain. Further research focusing on quality improvement, model calibration, rigorous external clinical validation in real healthcare settings is imperative for ensuring its practical and reliable application in clinical practice.
Collapse
Affiliation(s)
- Vivian Salama
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| | - Brandon Godinich
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Medical Education (B.G.), Paul L. Foster School of Medicine, Texas Tech Health Sciences Center, El Paso, TX, USA
| | - Yimin Geng
- Research Medical Library (Y.G.), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Laia Humbert-Vidan
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Laura Maule
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Kareem A Wahid
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Mohamed A Naser
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Renjie He
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Abdallah S R Mohamed
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Clifton D Fuller
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Amy C Moreno
- Department of Radiation Oncology (V.S., B.G., L.H.V., L.M., K.A.W., M.A.N., R.H., A.S.R.M., C.D.F., A.C.M), The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| |
Collapse
|
5
|
Sun K, Roy A, Tobin JM. Artificial intelligence and machine learning: Definition of terms and current concepts in critical care research. J Crit Care 2024; 82:154792. [PMID: 38554543 DOI: 10.1016/j.jcrc.2024.154792] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Revised: 07/05/2023] [Accepted: 07/17/2023] [Indexed: 04/01/2024]
Abstract
With increasing computing power, artificial intelligence (AI) and machine learning (ML) have prospered, which facilitate the analysis of large datasets, especially those found in critical care. It is important to define these terminologies, to inform a standardized approach to critical care research. This manuscript hopes to clarify these terms with examples from medical literature. Three major components that are required for a successful ML implementation: (i) reliable dataset, (ii) ML algorithm, and (iii) unbiased model evaluation, are discussed. A reliable dataset can be structured or unstructured with limited noise, outliers, and missing values. ML, a subset of AI, is typically focused on supervised or unsupervised learning tasks in which the output is based on inputs and derived from iterative pattern recognition algorithms, while AI is the overall ability of a machine to "think" or mimic human behavior; and to analyze data free from human influence. Even with successful implementation, advanced AI and ML algorithms have faced challenges in adoption into practice, mainly due to their lack of interpretability, which hinders trust, buy-in, and engagement from clinicians. Consequently, traditional algorithms, such as linear and logistic regression, that may have reduced predictive power but are highly interpretable, continue to be widely used.
Collapse
Affiliation(s)
- Kai Sun
- Department of Management Science and Statistics, University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249, USA; Department of Anesthesiology, University of Texas Health Sciences Center San Antonio, 7703 Floyd Curl Dr, San Antonio, TX 78229, USA.
| | - Arkajyoti Roy
- Department of Management Science and Statistics, University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249, USA.
| | - Joshua M Tobin
- Department of Anesthesiology, University of Texas Health Sciences Center San Antonio, 7703 Floyd Curl Dr, San Antonio, TX 78229, USA.
| |
Collapse
|
6
|
Zhao Y, Chaw JK, Liu L, Chaw SH, Ang MC, Ting TT. Systematic literature review on reinforcement learning in non-communicable disease interventions. Artif Intell Med 2024; 154:102901. [PMID: 38838400 DOI: 10.1016/j.artmed.2024.102901] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 02/21/2024] [Accepted: 06/02/2024] [Indexed: 06/07/2024]
Abstract
There is evidence that reducing modifiable risk factors and strengthening medical and health interventions can reduce early mortality and economic losses from non-communicable diseases (NCDs). Machine learning (ML) algorithms have been successfully applied to preventing and controlling NCDs. Reinforcement learning (RL) is the most promising of these approaches because of its ability to dynamically adapt interventions to NCD disease progression and its commitment to achieving long-term intervention goals. This paper reviews the preferred algorithms, data sources, design details, and obstacles to clinical application in existing studies to facilitate the early application of RL algorithms in clinical practice research for NCD interventions. We screened 40 relevant papers for quantitative and qualitative analysis using the PRISMA review flow diagram. The results show that researchers tend to use Deep Q-Network (DQN) and Actor-Critic as well as their improved or hybrid algorithms to train and validate RL models on retrospective datasets. Often, the patient's physical condition is the main defining parameter of the state space, while interventions are the main defining parameter of the action space. Mostly, changes in the patient's physical condition are used as a basis for immediate rewards to the agent. Various attempts have been made to address the challenges to clinical application, and several approaches have been proposed from existing research. However, as there is currently no universally accepted solution, the use of RL algorithms in clinical practice for NCD interventions necessitates more comprehensive responses to the issues addressed in this paper, which are safety, interpretability, training efficiency, and the technical aspect of exploitation and exploration in RL algorithms.
Collapse
Affiliation(s)
- Yanfeng Zhao
- Institute of Visual Informatics, National University of Malaysia, Bangi, Selangor, Malaysia
| | - Jun Kit Chaw
- Institute of Visual Informatics, National University of Malaysia, Bangi, Selangor, Malaysia.
| | - Lin Liu
- Henan Vocational University of Science and Technology, Zhoukou, Henan, China
| | - Sook Hui Chaw
- Department of Anaesthesiology, Faculty of Medicine, Universiti Malaya, Kuala Lumpur, Malaysia
| | - Mei Choo Ang
- Institute of Visual Informatics, National University of Malaysia, Bangi, Selangor, Malaysia
| | - Tin Tin Ting
- Faculty of Data Science and Information Technology, INTI International University, Nilai, Negeri Sembilan, Malaysia
| |
Collapse
|
7
|
Figueiredo Prudencio R, Maximo MROA, Colombini EL. A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10237-10257. [PMID: 37030754 DOI: 10.1109/tnnls.2023.3250269] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
With the widespread adoption of deep learning, reinforcement learning (RL) has experienced a dramatic increase in popularity, scaling to previously intractable problems, such as playing complex games from pixel observations, sustaining conversations with humans, and controlling robotic agents. However, there is still a wide range of domains inaccessible to RL due to the high cost and danger of interacting with the environment. Offline RL is a paradigm that learns exclusively from static datasets of previously collected interactions, making it feasible to extract policies from large and diverse training datasets. Effective offline RL algorithms have a much wider range of applications than online RL, being particularly appealing for real-world applications, such as education, healthcare, and robotics. In this work, we contribute with a unifying taxonomy to classify offline RL methods. Furthermore, we provide a comprehensive review of the latest algorithmic breakthroughs in the field using a unified notation as well as a review of existing benchmarks' properties and shortcomings. Additionally, we provide a figure that summarizes the performance of each method and class of methods on different dataset properties, equipping researchers with the tools to decide which type of algorithm is best suited for the problem at hand and identify which classes of algorithms look the most promising. Finally, we provide our perspective on open problems and propose future research directions for this rapidly growing field.
Collapse
|
8
|
Yamamoto K, Sakaguchi M, Onishi A, Yokoyama S, Matsui Y, Yamamoto W, Onizawa H, Fujii T, Murata K, Tanaka M, Hashimoto M, Matsuda S, Morinobu A. Energy landscape analysis and time-series clustering analysis of patient state multistability related to rheumatoid arthritis drug treatment: The KURAMA cohort study. PLoS One 2024; 19:e0302308. [PMID: 38709812 PMCID: PMC11073743 DOI: 10.1371/journal.pone.0302308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 04/02/2024] [Indexed: 05/08/2024] Open
Abstract
Rheumatoid arthritis causes joint inflammation due to immune abnormalities, resulting in joint pain and swelling. In recent years, there have been considerable advancements in the treatment of this disease. However, only approximately 60% of patients achieve remission. Patients with multifactorial diseases shift between states from day to day. Patients may remain in a good or poor state with few or no transitions, or they may switch between states frequently. The visualization of time-dependent state transitions, based on the evaluation axis of stable/unstable states, may provide useful information for achieving rheumatoid arthritis treatment goals. Energy landscape analysis can be used to quantitatively determine the stability/instability of each state in terms of energy. Time-series clustering is another method used to classify transitions into different groups to identify potential patterns within a time-series dataset. The objective of this study was to utilize energy landscape analysis and time-series clustering to evaluate multidimensional time-series data in terms of multistability. We profiled each patient's state transitions during treatment using energy landscape analysis and time-series clustering. Energy landscape analysis divided state transitions into two patterns: "good stability leading to remission" and "poor stability leading to treatment dead-end." The number of patients whose disease status improved increased markedly until approximately 6 months after treatment initiation and then plateaued after 1 year. Time-series clustering grouped patients into three clusters: "toward good stability," "toward poor stability," and "unstable." Patients in the "unstable" cluster are considered to have clinical courses that are difficult to predict; therefore, these patients should be treated with more care. Early disease detection and treatment initiation are important. The evaluation of state multistability enables us to understand a patient's current state in the context of overall state transitions related to rheumatoid arthritis drug treatment and to predict future state transitions.
Collapse
Affiliation(s)
- Keiichi Yamamoto
- Division of Data Science, Center for Industrial Research and Innovation, Translational Research Institute for Medical Innovation, Osaka Dental University, Hirakata City, Osaka, Japan
| | - Masahiko Sakaguchi
- Department of Engineering Informatics, Faculty of Information and Communication Engineering, Osaka Electro-Communication University, Neyagawa City, Osaka, Japan
| | - Akira Onishi
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | | | | | - Wataru Yamamoto
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
- Department of Health Information Management, Kurashiki Sweet Hospital, Nakasho, Kurashiki, Kurashiki City, Okayama Prefecture, Japan
| | - Hideo Onizawa
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Takayuki Fujii
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Koichi Murata
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Masao Tanaka
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Motomu Hashimoto
- Department of Clinical Immunology, Osaka Metropolitan University Graduate School of Medicine, Osaka City, Japan
| | - Shuichi Matsuda
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| | - Akio Morinobu
- Department of Advanced Medicine for Rheumatic Diseases, Kyoto University Graduate School of Medicine, Sakyo, Kyoto, Japan
| |
Collapse
|
9
|
Altara R, Basson CJ, Biondi-Zoccai G, Booz GW. Exploring the Promise and Challenges of Artificial Intelligence in Biomedical Research and Clinical Practice. J Cardiovasc Pharmacol 2024; 83:403-409. [PMID: 38323891 DOI: 10.1097/fjc.0000000000001546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Accepted: 01/26/2024] [Indexed: 02/08/2024]
Abstract
ABSTRACT Artificial intelligence (AI) is poised to revolutionize how science, and biomedical research in particular, are done. With AI, problem-solving and complex tasks using massive data sets can be performed at a much higher rate and dimensionality level compared with humans. With the ability to handle huge data sets and self-learn, AI is already being exploited in drug design, drug repurposing, toxicology, and material identification. AI could also be used in both basic and clinical research in study design, defining outcomes, analyzing data, interpreting findings, and even identifying the most appropriate areas of investigation and funding sources. State-of-the-art AI-based large language models, such as ChatGPT and Perplexity, are positioned to change forever how science is communicated and how scientists interact with one another and their profession, including postpublication appraisal and critique. Like all revolutions, upheaval will follow and not all outcomes can be predicted, necessitating guardrails at the onset, especially to minimize the untoward impact of the many drawbacks of large language models, which include lack of confidentiality, risk of hallucinations, and propagation of mainstream albeit potentially mistaken opinions and perspectives. In this review, we highlight areas of biomedical research that are already being reshaped by AI and how AI is likely to affect it further in the near future. We discuss the potential benefits of AI in biomedical research and address possible risks, some surrounding the creative process, that warrant further reflection.
Collapse
Affiliation(s)
- Raffaele Altara
- Department of Anatomy and Embryology, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
- Department of Pathology, School of Medicine, University of Mississippi Medical Center, Jackson, MS
| | - Cameron J Basson
- School of Medicine, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands
| | - Giuseppe Biondi-Zoccai
- Department of Medical Surgical Sciences and Biotechnologies, Sapienza University of Rome, Latina, Italy
- Mediterranea Cardiocentro, Napoli, Italy; and
| | - George W Booz
- Department of Pharmacology and Toxicology, School of Medicine, University of Mississippi Medical Center, Jackson, MS
| |
Collapse
|
10
|
Singh H, Nim DK, Randhawa AS, Ahluwalia S. Integrating clinical pharmacology and artificial intelligence: potential benefits, challenges, and role of clinical pharmacologists. Expert Rev Clin Pharmacol 2024; 17:381-391. [PMID: 38340012 DOI: 10.1080/17512433.2024.2317963] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 02/08/2024] [Indexed: 02/12/2024]
Abstract
INTRODUCTION The integration of artificial intelligence (AI) into clinical pharmacology could be a potential approach for accelerating drug discovery and development, improving patient care, and streamlining medical research processes. AREAS COVERED We reviewed the current state of AI applications in clinical pharmacology, focusing on drug discovery and development, precision medicine, pharmacovigilance, and other ventures. Key AI applications in clinical pharmacology are examined, including machine learning, natural language processing, deep learning, and reinforcement learning etc. Additionally, the evolving role of clinical pharmacologists, ethical considerations, and challenges in implementing AI in clinical pharmacology are discussed. EXPERT OPINION The AI could be instrumental in accelerating drug discovery, predicting drug safety and efficacy, and optimizing clinical trial designs. It can play a vital role in precision medicine by helping in personalized drug dosing, treatment selection, and predicting drug response based on genetic, clinical, and environmental factors. The role of AI in pharmacovigilance, such as signal detection and adverse event prediction, is also promising. The collaboration between clinical pharmacologists and AI experts also poses certain ethical and practical challenges. Clinical pharmacologists can be instrumental in shaping the future of AI-driven clinical pharmacology and contribute to the improvement of healthcare systems.
Collapse
Affiliation(s)
- Harmanjit Singh
- Department of Pharmacology, Government Medical College & Hospital, Chandigarh, India
| | | | | | | |
Collapse
|
11
|
De Carlo A, Tosca EM, Fantozzi M, Magni P. Reinforcement Learning and PK-PD Models Integration to Personalize the Adaptive Dosing Protocol of Erdafitinib in Patients with Metastatic Urothelial Carcinoma. Clin Pharmacol Ther 2024; 115:825-838. [PMID: 38339803 DOI: 10.1002/cpt.3176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2023] [Accepted: 12/15/2023] [Indexed: 02/12/2024]
Abstract
The integration of pharmacokinetic-pharmacodynamic (PK-PD) modeling and simulations with artificial intelligence/machine learning algorithms is one of the most attractive areas of the pharmacometric research. These hybrid techniques are currently under investigation to perform several tasks, among which precision dosing. In this scenario, this paper presents and evaluates a new framework embedding PK-PD models into a reinforcement learning (RL) algorithm, Q-learning (QL), to personalize pharmacological treatment. Each patient is represented with a set of PK-PD parameters and has a personal QL agent which optimizes the individual treatment. In the training phase, leveraging PK-PD simulations, the QL agent assesses different actions, defined consistently with the clinical knowledge to consider only plausible dose-adjustments, in order to find the optimal rules. The proposed framework was evaluated to optimize the erdafitinib treatment in patients with metastatic urothelial carcinoma. This drug was approved by the US Food and Drug Administration (FDA) with a dose-adaptive protocol based on monitoring the levels of serum phosphate, which represent a biomarker of both treatment efficacy and toxicity. To evaluate the flexibility of the methodology, a heterogeneous virtual population of 141 patients was generated using an erdafitinib population PK (PopPK)-PD literature model. For each patient, treatment response was simulated by using both QL-optimized protocol and the clinical one. QL agents outperform the approved dose-adaptive rules, increasing more than 10% the efficacy and the safety of treatment at each end point. Results confirm the great potentialities of the integration of PopPK-PD models and RL algorithms to optimize precision dosing tasks.
Collapse
Affiliation(s)
- Alessandro De Carlo
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Elena Maria Tosca
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Martina Fantozzi
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - Paolo Magni
- Electrical, Computer, and Biomedical Engineering, University of Pavia, Pavia, Italy
| |
Collapse
|
12
|
Fackler J, Ghobadi K, Gurses AP. Algorithms at the Bedside: Moving Past Development and Validation. Pediatr Crit Care Med 2024; 25:276-278. [PMID: 38451799 DOI: 10.1097/pcc.0000000000003437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 03/09/2024]
Affiliation(s)
- James Fackler
- Department of Anesthesiology and Critical Care Medicine and Pediatrics, Johns Hopkins, Baltimore, MD
| | - Kimia Ghobadi
- Department of Civil and Systems Engineering, The Malone Center for Engineering in Healthcare, The Center for Systems Science and Engineering, Johns Hopkins University, Baltimore, MD
| | - Ayse P Gurses
- Armstrong Institute Center for Health Care Human Factors, Johns Hopkins Medicine, Baltimore, MD
| |
Collapse
|
13
|
Otten M, Jagesar AR, Dam TA, Biesheuvel LA, den Hengst F, Ziesemer KA, Thoral PJ, de Grooth HJ, Girbes ARJ, François-Lavet V, Hoogendoorn M, Elbers PWG. Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment. Crit Care Med 2024; 52:e79-e88. [PMID: 37938042 DOI: 10.1097/ccm.0000000000006100] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2023]
Abstract
OBJECTIVE Reinforcement learning (RL) is a machine learning technique uniquely effective at sequential decision-making, which makes it potentially relevant to ICU treatment challenges. We set out to systematically review, assess level-of-readiness and meta-analyze the effect of RL on outcomes for critically ill patients. DATA SOURCES A systematic search was performed in PubMed, Embase.com, Clarivate Analytics/Web of Science Core Collection, Elsevier/SCOPUS and the Institute of Electrical and Electronics Engineers Xplore Digital Library from inception to March 25, 2022, with subsequent citation tracking. DATA EXTRACTION Journal articles that used an RL technique in an ICU population and reported on patient health-related outcomes were included for full analysis. Conference papers were included for level-of-readiness assessment only. Descriptive statistics, characteristics of the models, outcome compared with clinician's policy and level-of-readiness were collected. RL-health risk of bias and applicability assessment was performed. DATA SYNTHESIS A total of 1,033 articles were screened, of which 18 journal articles and 18 conference papers, were included. Thirty of those were prototyping or modeling articles and six were validation articles. All articles reported RL algorithms to outperform clinical decision-making by ICU professionals, but only in retrospective data. The modeling techniques for the state-space, action-space, reward function, RL model training, and evaluation varied widely. The risk of bias was high in all articles, mainly due to the evaluation procedure. CONCLUSION In this first systematic review on the application of RL in intensive care medicine we found no studies that demonstrated improved patient outcomes from RL-based technologies. All studies reported that RL-agent policies outperformed clinician policies, but such assessments were all based on retrospective off-policy evaluation.
Collapse
Affiliation(s)
- Martijn Otten
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Ameet R Jagesar
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Tariq A Dam
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Laurens A Biesheuvel
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Floris den Hengst
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | | | - Patrick J Thoral
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Harm-Jan de Grooth
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Armand R J Girbes
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| | - Vincent François-Lavet
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Mark Hoogendoorn
- Quantitative Data Analytics Group, Department of Computer Science, Faculty of Science, Vrije Universiteit, Amsterdam, The Netherlands
| | - Paul W G Elbers
- Department of Intensive Care Medicine, Center for Critical Care Computational Intelligence, Amsterdam Medical Data Science (AMDS), Amsterdam Cardiovascular Science (ACS), Amsterdam UMC, Vrije Universiteit, Amsterdam, The Netherlands
| |
Collapse
|
14
|
Nakaizumi D, Miyata S, Uchiyama K, Takahashi I. Development and Validation of a Decision Tree Analysis Model for Predicting Home Discharge in a Convalescent Ward: A Single Institution Study. Phys Ther Res 2024; 27:14-20. [PMID: 38690531 PMCID: PMC11057389 DOI: 10.1298/ptr.e10267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Accepted: 11/17/2023] [Indexed: 05/02/2024]
Abstract
OBJECTIVES Accurately predicting the likelihood of inpatients' home discharge in a convalescent ward is crucial for assisting patients and families in decision-making. While logistic regression analysis has been commonly used, its complexity limits practicality in clinical settings. We focused on decision tree analysis, which is visually straightforward. This study aimed to develop and validate the accuracy of a prediction model for home discharge for inpatients in a convalescent ward using a decision tree analysis. METHODS The cohort consisted of 651 patients admitted to our convalescent ward from 2018 to 2020. We collected data from medical records, including disease classification, sex, age, duration of acute hospitalization, discharge destination (home or nonhome), and Functional Independence Measure (FIM) subitems at admission. We divided the cohort data into training and validation sets and developed a prediction model using decision tree analysis with discharge destination as the target and other variables as predictors. The model's accuracy was validated using the validation data set. RESULTS The decision tree model identified FIM grooming as the first single discriminator of home discharge, diverging at four points and identifying subsequent branching for the duration of acute hospitalization. The model's accuracy was 86.7%, with a sensitivity of 0.96, specificity of 0.52, positive predictive accuracy of 0.88, and negative predictive accuracy of 0.80. The area under the receiver operating characteristic curve was 0.75. CONCLUSION The predictive model demonstrated more than moderate predictive accuracy, suggesting its utility in clinical practice. Grooming emerged as a variable with the highest explanatory power for determining home discharge.
Collapse
Affiliation(s)
- Dai Nakaizumi
- Department of Rehabilitation, Japanese Red Cross Kanazawa Hospital, Japan
- Department of Physical Therapy, Graduate Course of Rehabilitation Science, School of Health Sciences, College of Medical, Pharmaceutical, and Health Sciences, Kanazawa University, Japan
| | - Shingo Miyata
- Department of Rehabilitation, Japanese Red Cross Kanazawa Hospital, Japan
| | - Keita Uchiyama
- Department of Rehabilitation, Japanese Red Cross Kanazawa Hospital, Japan
| | - Ikki Takahashi
- Department of Rehabilitation, Suzuki Clinic Orthopaedics River City, Japan
| |
Collapse
|
15
|
Nguyen TM, Poh KL, Chong SL, Loh SW, Heng YCK, Lee JH. The use of probabilistic graphical models in pediatric sepsis: a feasibility and scoping review. Transl Pediatr 2023; 12:2074-2089. [PMID: 38130578 PMCID: PMC10730969 DOI: 10.21037/tp-23-25] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 10/24/2023] [Indexed: 12/23/2023] Open
Abstract
Background Recent research has demonstrated that machine learning (ML) has the potential to improve several aspects of medical application for critical illness, including sepsis. This scoping review aims to evaluate the feasibility of probabilistic graphical model (PGM) methods in pediatric sepsis application and describe the use of pediatric sepsis definition in these studies. Methods Literature searches were conducted in PubMed, Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL+), and Web of Sciences from 2000-2023. Keywords included "pediatric", "neonates", "infants", "machine learning", "probabilistic graphical model", and "sepsis". Results A total of 3,244 studies were screened, and 72 were included in this scoping review. Sepsis was defined using positive microbiology cultures in 19 studies (26.4%), followed by the 2005's international pediatric sepsis consensus definition in 11 studies (15.3%), and Sepsis-3 definition in seven studies (9.7%). Other sepsis definitions included: bacterial infection, the international classification of diseases, clinicians' assessment, and antibiotic administration time. Among the most common ML approaches used were logistic regression (n=27), random forest (n=24), and Neural Network (n=18). PGMs were used in 13 studies (18.1%), including Bayesian classifiers (n=10), and the Markov Model (n=3). When applied on the same dataset, PGMs show a relatively inferior performance to other ML models in most cases. Other aspects of explainability and transparency were not examined in these studies. Conclusions Current studies suggest that the performance of probabilistic graphic models is relatively inferior to other ML methods. However, its explainability and transparency advantages make it a potentially viable method for several pediatric sepsis studies and applications.
Collapse
Affiliation(s)
- Tuong Minh Nguyen
- Department of Industrial Systems Engineering and Management, College of Design and Engineering, National University of Singapore, SG, Singapore
| | - Kim Leng Poh
- Department of Industrial Systems Engineering and Management, College of Design and Engineering, National University of Singapore, SG, Singapore
| | - Shu-Ling Chong
- Children’s Emergency, KK Women’s and Children’s Hospital, SG, Singapore
- SingHealth-Duke NUS Paediatrics Academic Clinical Programme, Duke-NUS Medical School, SG, Singapore
| | - Sin Wee Loh
- Children’s Intensive Care Unit, KK Women’s and Children’s Hospital, SG, Singapore
| | | | - Jan Hau Lee
- SingHealth-Duke NUS Paediatrics Academic Clinical Programme, Duke-NUS Medical School, SG, Singapore
- Children’s Intensive Care Unit, KK Women’s and Children’s Hospital, SG, Singapore
| |
Collapse
|
16
|
Smit JM, Krijthe JH, Kant WMR, Labrecque JA, Komorowski M, Gommers DAMPJ, van Bommel J, Reinders MJT, van Genderen ME. Causal inference using observational intensive care unit data: a scoping review and recommendations for future practice. NPJ Digit Med 2023; 6:221. [PMID: 38012221 PMCID: PMC10682453 DOI: 10.1038/s41746-023-00961-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 11/05/2023] [Indexed: 11/29/2023] Open
Abstract
This scoping review focuses on the essential role of models for causal inference in shaping actionable artificial intelligence (AI) designed to aid clinicians in decision-making. The objective was to identify and evaluate the reporting quality of studies introducing models for causal inference in intensive care units (ICUs), and to provide recommendations to improve the future landscape of research practices in this domain. To achieve this, we searched various databases including Embase, MEDLINE ALL, Web of Science Core Collection, Google Scholar, medRxiv, bioRxiv, arXiv, and the ACM Digital Library. Studies involving models for causal inference addressing time-varying treatments in the adult ICU were reviewed. Data extraction encompassed the study settings and methodologies applied. Furthermore, we assessed reporting quality of target trial components (i.e., eligibility criteria, treatment strategies, follow-up period, outcome, and analysis plan) and main causal assumptions (i.e., conditional exchangeability, positivity, and consistency). Among the 2184 titles screened, 79 studies met the inclusion criteria. The methodologies used were G methods (61%) and reinforcement learning methods (39%). Studies considered both static (51%) and dynamic treatment regimes (49%). Only 30 (38%) of the studies reported all five target trial components, and only seven (9%) studies mentioned all three causal assumptions. To achieve actionable AI in the ICU, we advocate careful consideration of the causal question of interest, describing this research question as a target trial emulation, usage of appropriate causal inference methods, and acknowledgement (and examination of potential violations of) the causal assumptions.
Collapse
Affiliation(s)
- J M Smit
- Department of Intensive Care, Erasmus University Medical Center, Rotterdam, The Netherlands.
- Pattern Recognition & Bioinformatics group, EEMCS, Delft University of Technology, Delft, The Netherlands.
| | - J H Krijthe
- Pattern Recognition & Bioinformatics group, EEMCS, Delft University of Technology, Delft, The Netherlands
| | - W M R Kant
- Data Science group, Institute for Computing and Information Sciences, Radboud University, Nijmegen, The Netherlands
| | - J A Labrecque
- Department of Epidemiology, Erasmus Medical Center, Rotterdam, The Netherlands
| | - M Komorowski
- Department of Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK
- Intensive Care Unit, Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, UK
| | - D A M P J Gommers
- Department of Intensive Care, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - J van Bommel
- Department of Intensive Care, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - M J T Reinders
- Pattern Recognition & Bioinformatics group, EEMCS, Delft University of Technology, Delft, The Netherlands
| | - M E van Genderen
- Department of Intensive Care, Erasmus University Medical Center, Rotterdam, The Netherlands
| |
Collapse
|
17
|
Caruso PF, Greco M, Ebm C, Angelotti G, Cecconi M. Implementing Artificial Intelligence: Assessing the Cost and Benefits of Algorithmic Decision-Making in Critical Care. Crit Care Clin 2023; 39:783-793. [PMID: 37704340 DOI: 10.1016/j.ccc.2023.03.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/15/2023]
Abstract
This article provides an overview of the most useful artificial intelligence algorithms developed in critical care, followed by a comprehensive outline of the benefits and limitations. We begin by describing how nurses and physicians might be aided by these new technologies. We then move to the possible changes in clinical guidelines with personalized medicine that will allow tailored therapies and probably will increase the quality of the care provided to patients. Finally, we describe how artificial intelligence models can unleash researchers' minds by proposing new strategies, by increasing the quality of clinical practice, and by questioning current knowledge and understanding.
Collapse
Affiliation(s)
- Pier Francesco Caruso
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy; Department of Anesthesiology and Intensive Care, IRCCS Humanitas Research Hospital, Via Manzoni 56, Rozzano, Milan 20089, Italy
| | - Massimiliano Greco
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy; Department of Anesthesiology and Intensive Care, IRCCS Humanitas Research Hospital, Via Manzoni 56, Rozzano, Milan 20089, Italy.
| | - Claudia Ebm
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy
| | - Giovanni Angelotti
- Artificial Intelligence Center, IRCCS Humanitas Research Hospital, Via Manzoni 56, Rozzano, Milan 20089, Italy
| | - Maurizio Cecconi
- Department of Biomedical Sciences, Humanitas University, Via Rita Levi Montalcini 4, Pieve Emanuele, Milan 20072, Italy; Department of Anesthesiology and Intensive Care, IRCCS Humanitas Research Hospital, Via Manzoni 56, Rozzano, Milan 20089, Italy
| |
Collapse
|
18
|
Okada Y, Mertens M, Liu N, Lam SSW, Ong MEH. AI and machine learning in resuscitation: Ongoing research, new concepts, and key challenges. Resusc Plus 2023; 15:100435. [PMID: 37547540 PMCID: PMC10400904 DOI: 10.1016/j.resplu.2023.100435] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/08/2023] Open
Abstract
Aim Artificial intelligence (AI) and machine learning (ML) are important areas of computer science that have recently attracted attention for their application to medicine. However, as techniques continue to advance and become more complex, it is increasingly challenging for clinicians to stay abreast of the latest research. This overview aims to translate research concepts and potential concerns to healthcare professionals interested in applying AI and ML to resuscitation research but who are not experts in the field. Main text We present various research including prediction models using structured and unstructured data, exploring treatment heterogeneity, reinforcement learning, language processing, and large-scale language models. These studies potentially offer valuable insights for optimizing treatment strategies and clinical workflows. However, implementing AI and ML in clinical settings presents its own set of challenges. The availability of high-quality and reliable data is crucial for developing accurate ML models. A rigorous validation process and the integration of ML into clinical practice is essential for practical implementation. We furthermore highlight the potential risks associated with self-fulfilling prophecies and feedback loops, emphasizing the importance of transparency, interpretability, and trustworthiness in AI and ML models. These issues need to be addressed in order to establish reliable and trustworthy AI and ML models. Conclusion In this article, we overview concepts and examples of AI and ML research in the resuscitation field. Moving forward, appropriate understanding of ML and collaboration with relevant experts will be essential for researchers and clinicians to overcome the challenges and harness the full potential of AI and ML in resuscitation.
Collapse
Affiliation(s)
- Yohei Okada
- Duke-NUS Medical School, National University of Singapore, Singapore
- Preventive Services, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Mayli Mertens
- Antwerp Center for Responsible AI, Antwerp University, Belgium
- Centre for Ethics, Department of Philosophy, Antwerp University, Belgium
| | - Nan Liu
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Sean Shao Wei Lam
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Marcus Eng Hock Ong
- Duke-NUS Medical School, National University of Singapore, Singapore
- Department of Emergency Medicine, Singapore General Hospital
| |
Collapse
|
19
|
Alkhodari M, Xiong Z, Khandoker AH, Hadjileontiadis LJ, Leeson P, Lapidaire W. The role of artificial intelligence in hypertensive disorders of pregnancy: towards personalized healthcare. Expert Rev Cardiovasc Ther 2023; 21:531-543. [PMID: 37300317 DOI: 10.1080/14779072.2023.2223978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/04/2023] [Accepted: 06/06/2023] [Indexed: 06/12/2023]
Abstract
INTRODUCTION Guidelines advise ongoing follow-up of patients after hypertensive disorders of pregnancy (HDP) to assess cardiovascular risk and manage future patient-specific pregnancy conditions. However, there are limited tools available to monitor patients, with those available tending to be simple risk assessments that lack personalization. A promising approach could be the emerging artificial intelligence (AI)-based techniques, developed from big patient datasets to provide personalized recommendations for preventive advice. AREAS COVERED In this narrative review, we discuss the impact of integrating AI and big data analysis for personalized cardiovascular care, focusing on the management of HDP. EXPERT OPINION The pathophysiological response of women to pregnancy varies, and deeper insight into each response can be gained through a deeper analysis of the medical history of pregnant women based on clinical records and imaging data. Further research is required to be able to implement AI for clinical cases using multi-modality and multi-organ assessment, and this could expand both knowledge on pregnancy-related disorders and personalized treatment planning.
Collapse
Affiliation(s)
- Mohanad Alkhodari
- Cardiovascular Clinical Research Facility, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
- Healthcare Engineering Innovation Center (HEIC), Department of Biomedical Engineering, Khalifa University of Science and Tehcnology, Abu Dhabi, UAE
| | - Zhaohan Xiong
- Cardiovascular Clinical Research Facility, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| | - Ahsan H Khandoker
- Healthcare Engineering Innovation Center (HEIC), Department of Biomedical Engineering, Khalifa University of Science and Tehcnology, Abu Dhabi, UAE
| | - Leontios J Hadjileontiadis
- Healthcare Engineering Innovation Center (HEIC), Department of Biomedical Engineering, Khalifa University of Science and Tehcnology, Abu Dhabi, UAE
- Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Paul Leeson
- Cardiovascular Clinical Research Facility, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| | - Winok Lapidaire
- Cardiovascular Clinical Research Facility, Radcliffe Department of Medicine, University of Oxford, Oxford, UK
| |
Collapse
|
20
|
Wang M, Sushil M, Miao BY, Butte AJ. Bottom-up and top-down paradigms of artificial intelligence research approaches to healthcare data science using growing real-world big data. J Am Med Inform Assoc 2023; 30:1323-1332. [PMID: 37187158 PMCID: PMC10280344 DOI: 10.1093/jamia/ocad085] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Revised: 04/03/2023] [Accepted: 05/04/2023] [Indexed: 05/17/2023] Open
Abstract
OBJECTIVES As the real-world electronic health record (EHR) data continue to grow exponentially, novel methodologies involving artificial intelligence (AI) are becoming increasingly applied to enable efficient data-driven learning and, ultimately, to advance healthcare. Our objective is to provide readers with an understanding of evolving computational methods and help in deciding on methods to pursue. TARGET AUDIENCE The sheer diversity of existing methods presents a challenge for health scientists who are beginning to apply computational methods to their research. Therefore, this tutorial is aimed at scientists working with EHR data who are early entrants into the field of applying AI methodologies. SCOPE This manuscript describes the diverse and growing AI research approaches in healthcare data science and categorizes them into 2 distinct paradigms, the bottom-up and top-down paradigms to provide health scientists venturing into artificial intelligent research with an understanding of the evolving computational methods and help in deciding on methods to pursue through the lens of real-world healthcare data.
Collapse
Affiliation(s)
- Michelle Wang
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA
| | - Madhumita Sushil
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA
| | - Brenda Y Miao
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA
| | - Atul J Butte
- Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, California, USA
- Department of Pediatrics, University of California, San Francisco, San Francisco, California, USA
| |
Collapse
|
21
|
Development of a Reinforcement Learning Algorithm to Optimize Corticosteroid Therapy in Critically Ill Patients with Sepsis. J Clin Med 2023; 12:jcm12041513. [PMID: 36836046 PMCID: PMC9961939 DOI: 10.3390/jcm12041513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Revised: 01/30/2023] [Accepted: 02/06/2023] [Indexed: 02/17/2023] Open
Abstract
BACKGROUND The optimal indication, dose, and timing of corticosteroids in sepsis is controversial. Here, we used reinforcement learning to derive the optimal steroid policy in septic patients based on data on 3051 ICU admissions from the AmsterdamUMCdb intensive care database. METHODS We identified septic patients according to the 2016 consensus definition. An actor-critic RL algorithm using ICU mortality as a reward signal was developed to determine the optimal treatment policy from time-series data on 277 clinical parameters. We performed off-policy evaluation and testing in independent subsets to assess the algorithm's performance. RESULTS Agreement between the RL agent's policy and the actual documented treatment reached 59%. Our RL agent's treatment policy was more restrictive compared to the actual clinician behavior: our algorithm suggested withholding corticosteroids in 62% of the patient states, versus 52% according to the physicians' policy. The 95% lower bound of the expected reward was higher for the RL agent than clinicians' historical decisions. ICU mortality after concordant action in the testing dataset was lower both when corticosteroids had been withheld and when corticosteroids had been prescribed by the virtual agent. The most relevant variables were vital parameters and laboratory values, such as blood pressure, heart rate, leucocyte count, and glycemia. CONCLUSIONS Individualized use of corticosteroids in sepsis may result in a mortality benefit, but optimal treatment policy may be more restrictive than the routine clinical practice. Whilst external validation is needed, our study motivates a 'precision-medicine' approach to future prospective controlled trials and practice.
Collapse
|
22
|
A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsis. NPJ Digit Med 2023; 6:15. [PMID: 36732666 PMCID: PMC9894526 DOI: 10.1038/s41746-023-00755-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2022] [Accepted: 01/16/2023] [Indexed: 02/04/2023] Open
Abstract
Deep Reinforcement Learning (DRL) has been increasingly attempted in assisting clinicians for real-time treatment of sepsis. While a value function quantifies the performance of policies in such decision-making processes, most value-based DRL algorithms cannot evaluate the target value function precisely and are not as safe as clinical experts. In this study, we propose a Weighted Dueling Double Deep Q-Network with embedded human Expertise (WD3QNE). A target Q value function with adaptive dynamic weight is designed to improve the estimate accuracy and human expertise in decision-making is leveraged. In addition, the random forest algorithm is employed for feature selection to improve model interpretability. We test our algorithm against state-of-the-art value function methods in terms of expected return, survival rate, action distribution and external validation. The results demonstrate that WD3QNE obtains the highest survival rate of 97.81% in MIMIC-III dataset. Our proposed method is capable of providing reliable treatment decisions with embedded clinician expertise.
Collapse
|
23
|
Aguiar-Pérez JM, Pérez-Juárez MÁ. An Insight of Deep Learning Based Demand Forecasting in Smart Grids. SENSORS (BASEL, SWITZERLAND) 2023; 23:1467. [PMID: 36772509 PMCID: PMC9921606 DOI: 10.3390/s23031467] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 01/09/2023] [Accepted: 01/26/2023] [Indexed: 06/18/2023]
Abstract
Smart grids are able to forecast customers' consumption patterns, i.e., their energy demand, and consequently electricity can be transmitted after taking into account the expected demand. To face today's demand forecasting challenges, where the data generated by smart grids is huge, modern data-driven techniques need to be used. In this scenario, Deep Learning models are a good alternative to learn patterns from customer data and then forecast demand for different forecasting horizons. Among the commonly used Artificial Neural Networks, Long Short-Term Memory networks-based on Recurrent Neural Networks-are playing a prominent role. This paper provides an insight into the importance of the demand forecasting issue, and other related factors, in the context of smart grids, and collects some experiences of the use of Deep Learning techniques, for demand forecasting purposes. To have an efficient power system, a balance between supply and demand is necessary. Therefore, industry stakeholders and researchers should make a special effort in load forecasting, especially in the short term, which is critical for demand response.
Collapse
|
24
|
Morris AH, Horvat C, Stagg B, Grainger DW, Lanspa M, Orme J, Clemmer TP, Weaver LK, Thomas FO, Grissom CK, Hirshberg E, East TD, Wallace CJ, Young MP, Sittig DF, Suchyta M, Pearl JE, Pesenti A, Bombino M, Beck E, Sward KA, Weir C, Phansalkar S, Bernard GR, Thompson BT, Brower R, Truwit J, Steingrub J, Hiten RD, Willson DF, Zimmerman JJ, Nadkarni V, Randolph AG, Curley MAQ, Newth CJL, Lacroix J, Agus MSD, Lee KH, deBoisblanc BP, Moore FA, Evans RS, Sorenson DK, Wong A, Boland MV, Dere WH, Crandall A, Facelli J, Huff SM, Haug PJ, Pielmeier U, Rees SE, Karbing DS, Andreassen S, Fan E, Goldring RM, Berger KI, Oppenheimer BW, Ely EW, Pickering BW, Schoenfeld DA, Tocino I, Gonnering RS, Pronovost PJ, Savitz LA, Dreyfuss D, Slutsky AS, Crapo JD, Pinsky MR, James B, Berwick DM. Computer clinical decision support that automates personalized clinical care: a challenging but needed healthcare delivery strategy. J Am Med Inform Assoc 2022; 30:178-194. [PMID: 36125018 PMCID: PMC9748596 DOI: 10.1093/jamia/ocac143] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 07/27/2022] [Accepted: 08/22/2022] [Indexed: 12/15/2022] Open
Abstract
How to deliver best care in various clinical settings remains a vexing problem. All pertinent healthcare-related questions have not, cannot, and will not be addressable with costly time- and resource-consuming controlled clinical trials. At present, evidence-based guidelines can address only a small fraction of the types of care that clinicians deliver. Furthermore, underserved areas rarely can access state-of-the-art evidence-based guidelines in real-time, and often lack the wherewithal to implement advanced guidelines. Care providers in such settings frequently do not have sufficient training to undertake advanced guideline implementation. Nevertheless, in advanced modern healthcare delivery environments, use of eActions (validated clinical decision support systems) could help overcome the cognitive limitations of overburdened clinicians. Widespread use of eActions will require surmounting current healthcare technical and cultural barriers and installing clinical evidence/data curation systems. The authors expect that increased numbers of evidence-based guidelines will result from future comparative effectiveness clinical research carried out during routine healthcare delivery within learning healthcare systems.
Collapse
Affiliation(s)
- Alan H Morris
- Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Christopher Horvat
- Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Brian Stagg
- Department of Ophthalmology and Visual Sciences, Moran Eye Center, University of Utah, Salt Lake City, Utah, USA
| | - David W Grainger
- Department of Biomedical Engineering, University of Utah, Salt Lake City, Utah, USA
| | - Michael Lanspa
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - James Orme
- Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Terry P Clemmer
- Department of Internal Medicine (Critical Care), Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Lindell K Weaver
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Frank O Thomas
- Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA
| | - Colin K Grissom
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Ellie Hirshberg
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Thomas D East
- SYNCRONYS - Chief Executive Officer, Albuquerque, New Mexico, USA
| | - Carrie Jane Wallace
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Michael P Young
- Department of Critical Care, Renown Regional Medical Center, Reno, Nevada, USA
| | - Dean F Sittig
- School of Biomedical Informatics, University of Texas Health Science Center, Houston, Texas, USA
| | - Mary Suchyta
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - James E Pearl
- Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA
- Department of Internal Medicine, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Antinio Pesenti
- Faculty of Medicine and Surgery—Anesthesiology, University of Milan, Milano, Lombardia, Italy
| | - Michela Bombino
- Department of Emergency and Intensive Care, San Gerardo Hospital, Monza (MB), Italy
| | - Eduardo Beck
- Faculty of Medicine and Surgery - Anesthesiology, University of Milan, Ospedale di Desio, Desio, Lombardia, Italy
| | - Katherine A Sward
- Department of Biomedical Informatics, College of Nursing, University of Utah, Salt Lake City, Utah, USA
| | - Charlene Weir
- Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
| | - Shobha Phansalkar
- Wolters Kluwer Health—Clinical Solutions—Medical Informatics, Wolters Kluwer Health, Newton, Massachusetts, USA
| | - Gordon R Bernard
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - B Taylor Thompson
- Pulmonary and Critical Care Division, Department of Internal Medicine, Harvard Medical School, Boston, Massachusetts, USA
| | - Roy Brower
- Department of Medicine, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Jonathon Truwit
- Department of Internal Medicine, Pulmonary and Critical Care, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Jay Steingrub
- Department of Internal Medicine, Pulmonary and Critical Care, University of Massachusetts Medical School, Baystate Campus, Springfield, Massachusetts, USA
| | - R Duncan Hiten
- Department of Internal Medicine, Pulmonary and Critical Care, University of Cincinnati College of Medicine, Cincinnati, Ohio, USA
| | - Douglas F Willson
- Pediatric Critical Care, Department of Pediatrics, Virginia Commonwealth University, Richmond, Virginia, USA
| | - Jerry J Zimmerman
- Division of Pediatric Critical Care Medicine, Department of Pediatrics, University of Washington School of Medicine, Seattle, Washington, USA
| | - Vinay Nadkarni
- Department of Anesthesiology and Critical Care Medicine, The Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Adrienne G Randolph
- Department of Anesthesiology, Critical Care and Pain Medicine, Boston Children's Hospital, Boston, Massachusetts, USA
| | - Martha A Q Curley
- University of Pennsylvania School of Nursing, Philadelphia, Pennsylvania, USA
| | - Christopher J L Newth
- Childrens Hospital Los Angeles, Department of Anesthesiology and Critical Care, University of Southern California Keck School of Medicine, Los Angeles, California, USA
| | - Jacques Lacroix
- Division of Pediatric Critical Care Medicine, Department of Pediatrics, Université de Montréal Faculté de Médecine, Montreal, Quebec, Canada
| | - Michael S D Agus
- Division of Medical Pediatric Critical Care, Department of Pediatrics, Boston Children’s Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Kang Hoe Lee
- Department of Intensive Care Medicine, Ng Teng Fong Hospital and National University Centre of Transplantation, National University Singapore Yong Loo Lin School of Medicine, Singapore
| | - Bennett P deBoisblanc
- Department of Internal Medicine, Pulmonary and Critical Care, Louisiana State University Health Sciences Center, New Orleans, Louisiana, USA
| | - Frederick Alan Moore
- Department of Surgery, University of Florida College of Medicine, Gainesville, Florida, USA
| | - R Scott Evans
- Department of Medical Informatics, Intermountain Healthcare, and Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
| | - Dean K Sorenson
- Department of Medical Informatics, Intermountain Healthcare, Salt Lake City, Utah, USA
| | - Anthony Wong
- Department of Data Science Ann and Robert H Lurie Children's Hospital of Chicago, Chicago, Illinois, USA
| | - Michael V Boland
- Department of Ophthalmology, Massachusetts Ear and Eye Infirmary, Harvard Medical School, Boston, Massachusetts, USA
| | - Willard H Dere
- Endocrinology and Metabolism Division, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, USA
| | - Alan Crandall
- Department of Ophthalmology and Visual Sciences, Moran Eye Center, University of Utah, Salt Lake City, Utah, USA
- Posthumous
| | - Julio Facelli
- Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
| | - Stanley M Huff
- Department of Medical Informatics, Intermountain Healthcare, Department of Biomedical Informatics, University of Utah, and Graphite Health, Salt Lake City, Utah, USA
| | - Peter J Haug
- Department of Medical Informatics, Intermountain Healthcare, and Department of Biomedical Informatics, University of Utah, Salt Lake City, Utah, USA
| | - Ulrike Pielmeier
- Aalborg University Faculty of Engineering and Science - Department of Health Science and Technology, Respiratory and Critical Care Group, Aalborg, Nordjylland, Denmark
| | - Stephen E Rees
- Aalborg University Faculty of Engineering and Science - Department of Health Science and Technology, Respiratory and Critical Care Group, Aalborg, Nordjylland, Denmark
| | - Dan S Karbing
- Aalborg University Faculty of Engineering and Science - Department of Health Science and Technology, Respiratory and Critical Care Group, Aalborg, Nordjylland, Denmark
| | - Steen Andreassen
- Aalborg University Faculty of Engineering and Science - Department of Health Science and Technology, Respiratory and Critical Care Group, Aalborg, Nordjylland, Denmark
| | - Eddy Fan
- Internal Medicine, Pulmonary and Critical Care Division, Institute of Health Policy, Management and Evaluation, University of Toronto Faculty of Medicine, Toronto, Ontario, Canada
| | - Roberta M Goldring
- Department of Internal Medicine, Pulmonary and Critical Care, New York University School of Medicine, New York, New York, USA
| | - Kenneth I Berger
- Department of Internal Medicine, Pulmonary and Critical Care, New York University School of Medicine, New York, New York, USA
| | - Beno W Oppenheimer
- Department of Internal Medicine, Pulmonary and Critical Care, New York University School of Medicine, New York, New York, USA
| | - E Wesley Ely
- Internal Medicine, Pulmonary and Critical Care, Critical Illness, Brain Dysfunction, and Survivorship (CIBS) Center, Vanderbilt University Medical Center, Nashville, Tennessee, USA
- Tennessee Valley Veteran’s Affairs Geriatric Research Education Clinical Center (GRECC), Nashville, Tennessee, USA
| | - Brian W Pickering
- Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota, USA
| | - David A Schoenfeld
- Biostatistics Center, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Irena Tocino
- Department of Radiology, Yale University School of Medicine, New Haven, Connecticut, USA
| | - Russell S Gonnering
- Department of Ophthalmology and Visual Sciences, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Peter J Pronovost
- Department of Anesthesiology and Critical Care Medicine, University Hospitals, Highland Hills, Case Western Reserve University School of Medicine, Cleveland, Ohio, USA
| | - Lucy A Savitz
- Northwest Center for Health Research, Kaiser Permanente, Oakland, California, USA
| | - Didier Dreyfuss
- Assistance Publique—Hôpitaux de Paris, Université de Paris, Sorbonne Université - INSERM unit UMR S_1155 (Common and Rare Kidney Diseases), Paris, France
| | - Arthur S Slutsky
- Interdepartmental Division of Critical Care Medicine, Keenan Research Center, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, University of Toronto, Toronto, Ontario, Canada
| | - James D Crapo
- Department of Internal Medicine, National Jewish Health, Denver, Colorado, USA
| | - Michael R Pinsky
- Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Brent James
- Department of Internal Medicine, Clinical Excellence Research Center (CERC), Stanford University School of Medicine, Stanford, California, USA
| | - Donald M Berwick
- Institute for Healthcare Improvement, Cambridge, Massachusetts, USA
| |
Collapse
|
25
|
Siyam N, Abdallah S. Toward automatic motivator selection for autism behavior intervention therapy. UNIVERSAL ACCESS IN THE INFORMATION SOCIETY 2022; 22:1-23. [PMID: 36160369 PMCID: PMC9483340 DOI: 10.1007/s10209-022-00914-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 08/30/2022] [Indexed: 06/16/2023]
Abstract
Children with autism spectrum disorder (ASD) usually show little interest in academic activities and may display disruptive behavior when presented with assignments. Research indicates that incorporating motivational variables during interventions results in improvements in behavior and academic performance. However, the impact of such motivational variables varies between children. In this paper, we aim to address the problem of selecting the right motivator for children with ASD using reinforcement learning by adapting to the most influential factors impacting the effectiveness of the contingent motivator used. We model the task of selecting a motivator as a Markov decision process problem. The states, actions and rewards design consider the factors that impact the effectiveness of a motivator based on applied behavior analysis as well as learners' individual preferences. We use a Q-learning algorithm to solve the modeled problem. Our proposed solution is then implemented as a mobile application developed for special education plans coordination. To evaluate the motivator selection feature, we conduct a study involving a group of teachers and therapists and assess how the added feature aids the participants in their decision-making process of selecting a motivator. Preliminary results indicated that the motivator selection feature improved the usability of the mobile app. Analysis of the algorithm performance showed promising results and indicated improvement of the recommendations over time.
Collapse
Affiliation(s)
- Nur Siyam
- Faculty of Engineering and IT, British University in Dubai, Dubai, United Arab Emirates
| | - Sherief Abdallah
- Faculty of Engineering and IT, British University in Dubai, Dubai, United Arab Emirates
| |
Collapse
|
26
|
Liu M, Shen X, Pan W. Deep reinforcement learning for personalized treatment recommendation. Stat Med 2022; 41:4034-4056. [PMID: 35716038 PMCID: PMC9427729 DOI: 10.1002/sim.9491] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Revised: 05/22/2022] [Accepted: 05/25/2022] [Indexed: 12/12/2022]
Abstract
In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient-specific molecular and clinical profiles, possibly high-dimensional. To advance cancer treatment, large-scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof-of-concept, we conduct experiments on two large-scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL-based PPORank outperforms the state-of-the-art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied.
Collapse
Affiliation(s)
- Mingyang Liu
- School of StatisticsUniversity of MinnesotaMinneapolisMinnesotaUSA
| | - Xiaotong Shen
- School of StatisticsUniversity of MinnesotaMinneapolisMinnesotaUSA
| | - Wei Pan
- Division of BiostatisticsUniversity of MinnesotaMinneapolisMinnesotaUSA
| |
Collapse
|
27
|
Gottlieb ER, Samuel M, Bonventre JV, Celi LA, Mattie H. Machine Learning for Acute Kidney Injury Prediction in the Intensive Care Unit. Adv Chronic Kidney Dis 2022; 29:431-438. [PMID: 36253026 PMCID: PMC9586459 DOI: 10.1053/j.ackd.2022.06.005] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 06/01/2022] [Accepted: 06/22/2022] [Indexed: 01/25/2023]
Abstract
Machine learning is the field of artificial intelligence in which computers are trained to make predictions or to identify patterns in data through complex mathematical algorithms. It has great potential in critical care to predict outcomes, such as acute kidney injury, and can be used for prognosis and to suggest management strategies. Machine learning can also be used as a research tool to advance our clinical and biochemical understanding of acute kidney injury. In this review, we introduce basic concepts in machine learning and review recent research in each of these domains.
Collapse
Affiliation(s)
- Eric R Gottlieb
- Renal Section, Brigham and Women's Hospital, Boston, MA; Harvard Medical School, Boston, MA; Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA.
| | | | - Joseph V Bonventre
- Renal Section, Brigham and Women's Hospital, Boston, MA; Harvard Medical School, Boston, MA
| | - Leo A Celi
- Harvard Medical School, Boston, MA; Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA; MIT Critical Data, Cambridge, MA; Harvard T.H. Chan School of Public Health, Boston, MA; Beth Israel Deaconess Medical Center, Boston, MA
| | | |
Collapse
|
28
|
Couckuyt A, Seurinck R, Emmaneel A, Quintelier K, Novak D, Van Gassen S, Saeys Y. Challenges in translational machine learning. Hum Genet 2022; 141:1451-1466. [PMID: 35246744 PMCID: PMC8896412 DOI: 10.1007/s00439-022-02439-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2021] [Accepted: 02/08/2022] [Indexed: 11/25/2022]
Abstract
Machine learning (ML) algorithms are increasingly being used to help implement clinical decision support systems. In this new field, we define as "translational machine learning", joint efforts and strong communication between data scientists and clinicians help to span the gap between ML and its adoption in the clinic. These collaborations also improve interpretability and trust in translational ML methods and ultimately aim to result in generalizable and reproducible models. To help clinicians and bioinformaticians refine their translational ML pipelines, we review the steps from model building to the use of ML in the clinic. We discuss experimental setup, computational analysis, interpretability and reproducibility, and emphasize the challenges involved. We highly advise collaboration and data sharing between consortia and institutes to build multi-centric cohorts that facilitate ML methodologies that generalize across centers. In the end, we hope that this review provides a way to streamline translational ML and helps to tackle the challenges that come with it.
Collapse
Affiliation(s)
- Artuur Couckuyt
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
| | - Ruth Seurinck
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
| | - Annelies Emmaneel
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
| | - Katrien Quintelier
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
- Department of Pulmonary Diseases, Erasmus MC, Rotterdam, The Netherlands
| | - David Novak
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
| | - Sofie Van Gassen
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium
| | - Yvan Saeys
- Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Gent, Belgium.
- Data Mining and Modeling for Biomedicine, VIB-UGent Center for Inflammation Research, Gent, Belgium.
| |
Collapse
|
29
|
Girdler B, Caldbeck W, Bae J. Neural Decoders Using Reinforcement Learning in Brain Machine Interfaces: A Technical Review. Front Syst Neurosci 2022; 16:836778. [PMID: 36090185 PMCID: PMC9459159 DOI: 10.3389/fnsys.2022.836778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 06/21/2022] [Indexed: 11/18/2022] Open
Abstract
Creating flexible and robust brain machine interfaces (BMIs) is currently a popular topic of research that has been explored for decades in medicine, engineering, commercial, and machine-learning communities. In particular, the use of techniques using reinforcement learning (RL) has demonstrated impressive results but is under-represented in the BMI community. To shine more light on this promising relationship, this article aims to provide an exhaustive review of RL's applications to BMIs. Our primary focus in this review is to provide a technical summary of various algorithms used in RL-based BMIs to decode neural intention, without emphasizing preprocessing techniques on the neural signals and reward modeling for RL. We first organize the literature based on the type of RL methods used for neural decoding, and then each algorithm's learning strategy is explained along with its application in BMIs. A comparative analysis highlighting the similarities and uniqueness among neural decoders is provided. Finally, we end this review with a discussion about the current stage of RLBMIs including their limitations and promising directions for future research.
Collapse
Affiliation(s)
| | | | - Jihye Bae
- Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, United States
| |
Collapse
|
30
|
Mollura M, Drudi C, Lehman LW, Barbieri R. A Reinforcement Learning Application for Optimal Fluid and Vasopressor Interventions in Septic ICU Patients. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:321-324. [PMID: 36086153 DOI: 10.1109/embc48229.2022.9871055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Sepsis is one of the leading causes of death in ICU and its timely recognition and management are of primary importance. Resuscitation from hypotension in patients with sepsis is one of the first challenges that require fluid and/or vasopressor administrations. Unfortunately, clinical guidelines provide only indications of the strategy that should be adopted in this critical population but personalized strategies are still missing. In this study, we propose a comparative analysis of reinforcement learning applications on ICU data collected in the electronic health records and publicly available within the MIMIC-III database. The ultimate goal of the study is to estimate the optimal fluid and vasopressor administrations. Results show that, after the use of principal component analysis for reducing feature space dimensionality, model performances increased, thus suggesting that additional preprocessing strategies might be used for both reducing the computational cost and refining model performances. Clinical relevance In a context where clinical guidelines are not able to provide the best treatment strategies at a patient level, reinforcement learning applications trained on retrospectively collected data may be used for developing models able to suggest to clinicians the optimal dosage of fluids and/or vasopressors in order to improve 90-day patients' survival.
Collapse
|
31
|
Shiranthika C, Chen KW, Wang CY, Yang CY, Sudantha BH, Li WF. Supervised Optimal Chemotherapy Regimen Based on Offline Reinforcement Learning. IEEE J Biomed Health Inform 2022; 26:4763-4772. [PMID: 35714083 DOI: 10.1109/jbhi.2022.3183854] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
In recent years, reinforcement learning (RL) has achieved a remarkable achievement and it has attracted researchers' attention in modeling real-life scenarios by expanding its research beyond conventional complex games. Prediction of optimal treatment regimens from observational real clinical data is being popularized, and more advanced versions of RL algorithms are being implemented in the literature. However, RL-generated medications still need careful supervision of expertise parties or doctors in healthcare. Hence, in this paper, a Supervised Optimal Chemotherapy Regimen (SOCR) approach to investigate optimal chemotherapy-dosing schedule for cancer patients was presented by using Offline Reinforcement Learning. The optimal policy suggested by the RL approach was supervised by incorporating previous treatment decisions of oncologists, which could add clinical expertise knowledge on algorithmic results. Presented SOCR approach followed a model-based architecture using conservative Q-Learning (CQL) algorithm. The developed model was tested using a manually constructed database of forty Stage-IV colon cancer patients, receiving line-1 chemotherapy treatments, who were clinically classified as 'Bevacizumab based patient' and 'Cetuximab based patient'. Experimental results revealed that the supervision from the oncologists has considered the effect to stabilize chemotherapy regimen and it was suggested that the proposed framework could be successfully used as a supportive model for oncologists in deciding their treatment decisions.
Collapse
|
32
|
Campagnini S, Arienti C, Patrini M, Liuzzi P, Mannini A, Carrozza MC. Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: a systematic review. J Neuroeng Rehabil 2022; 19:54. [PMID: 35659246 PMCID: PMC9166382 DOI: 10.1186/s12984-022-01032-4] [Citation(s) in RCA: 24] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2021] [Accepted: 05/18/2022] [Indexed: 01/21/2023] Open
Abstract
BACKGROUND Rehabilitation medicine is facing a new development phase thanks to a recent wave of rigorous clinical trials aimed at improving the scientific evidence of protocols. This phenomenon, combined with new trends in personalised medical therapies, is expected to change clinical practice dramatically. The emerging field of Rehabilomics is only possible if methodologies are based on biomedical data collection and analysis. In this framework, the objective of this work is to develop a systematic review of machine learning algorithms as solutions to predict motor functional recovery of post-stroke patients after treatment. METHODS We conducted a comprehensive search of five electronic databases using the Patient, Intervention, Comparison and Outcome (PICO) format. We extracted health conditions, population characteristics, outcome assessed, the method for feature extraction and selection, the algorithm used, and the validation approach. The methodological quality of included studies was assessed using the prediction model risk of bias assessment tool (PROBAST). A qualitative description of the characteristics of the included studies as well as a narrative data synthesis was performed. RESULTS A total of 19 primary studies were included. The predictors most frequently used belonged to the areas of demographic characteristics and stroke assessment through clinical examination. Regarding the methods, linear and logistic regressions were the most frequently used and cross-validation was the preferred validation approach. CONCLUSIONS We identified several methodological limitations: small sample sizes, a limited number of external validation approaches, and high heterogeneity among input and output variables. Although these elements prevented a quantitative comparison across models, we defined the most frequently used models given a specific outcome, providing useful indications for the application of more complex machine learning algorithms in rehabilitation medicine.
Collapse
Affiliation(s)
- Silvia Campagnini
- IRCCS Fondazione Don Carlo Gnocchi Onlus, Via di Scandicci 269, 50143, Firenze, Italy.,Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio 34, 56025, Pontedera, Italy
| | - Chiara Arienti
- IRCCS Fondazione Don Carlo Gnocchi Onlus, Via di Scandicci 269, 50143, Firenze, Italy
| | - Michele Patrini
- IRCCS Fondazione Don Carlo Gnocchi Onlus, Via di Scandicci 269, 50143, Firenze, Italy
| | - Piergiuseppe Liuzzi
- IRCCS Fondazione Don Carlo Gnocchi Onlus, Via di Scandicci 269, 50143, Firenze, Italy.,Scuola Superiore Sant'Anna, Viale Rinaldo Piaggio 34, 56025, Pontedera, Italy
| | - Andrea Mannini
- IRCCS Fondazione Don Carlo Gnocchi Onlus, Via di Scandicci 269, 50143, Firenze, Italy.
| | | |
Collapse
|
33
|
Al-Zaiti SS, Alghwiri AA, Hu X, Clermont G, Peace A, Macfarlane P, Bond R. A clinician's guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML). EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022; 3:125-140. [PMID: 36713011 PMCID: PMC9708024 DOI: 10.1093/ehjdh/ztac016] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/20/2021] [Revised: 02/11/2022] [Indexed: 05/06/2023]
Abstract
Developing functional machine learning (ML)-based models to address unmet clinical needs requires unique considerations for optimal clinical utility. Recent debates about the rigours, transparency, explainability, and reproducibility of ML models, terms which are defined in this article, have raised concerns about their clinical utility and suitability for integration in current evidence-based practice paradigms. This featured article focuses on increasing the literacy of ML among clinicians by providing them with the knowledge and tools needed to understand and critically appraise clinical studies focused on ML. A checklist is provided for evaluating the rigour and reproducibility of the four ML building blocks: data curation, feature engineering, model development, and clinical deployment. Checklists like this are important for quality assurance and to ensure that ML studies are rigourously and confidently reviewed by clinicians and are guided by domain knowledge of the setting in which the findings will be applied. Bridging the gap between clinicians, healthcare scientists, and ML engineers can address many shortcomings and pitfalls of ML-based solutions and their potential deployment at the bedside.
Collapse
Affiliation(s)
| | - Alaa A Alghwiri
- Data Science Core, The Provost Office, University of Pittsburgh, Pittsburgh PA, USA
| | - Xiao Hu
- Center for Data Science, Emory University, Atlanta, GA, USA
| | - Gilles Clermont
- Departments of Critical Care Medicine, Mathematics, Clinical and Translational Science, and Industrial Engineering, University of Pittsburgh, Pittsburgh, PA, USA
| | - Aaron Peace
- The Clinical Translational Research and Innovation Centre, Northern Ireland, UK
| | - Peter Macfarlane
- Institute of Health and Wellbeing, Electrocardiology Section, University of Glasgow, Glasgow, UK
| | - Raymond Bond
- School of Computing, Ulster University, Ulster, UK
| |
Collapse
|
34
|
Clinical artificial intelligence quality improvement: towards continual monitoring and updating of AI algorithms in healthcare. NPJ Digit Med 2022; 5:66. [PMID: 35641814 PMCID: PMC9156743 DOI: 10.1038/s41746-022-00611-y] [Citation(s) in RCA: 72] [Impact Index Per Article: 36.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2021] [Accepted: 04/29/2022] [Indexed: 12/13/2022] Open
Abstract
Machine learning (ML) and artificial intelligence (AI) algorithms have the potential to derive insights from clinical data and improve patient outcomes. However, these highly complex systems are sensitive to changes in the environment and liable to performance decay. Even after their successful integration into clinical practice, ML/AI algorithms should be continuously monitored and updated to ensure their long-term safety and effectiveness. To bring AI into maturity in clinical care, we advocate for the creation of hospital units responsible for quality assurance and improvement of these algorithms, which we refer to as “AI-QI” units. We discuss how tools that have long been used in hospital quality assurance and quality improvement can be adapted to monitor static ML algorithms. On the other hand, procedures for continual model updating are still nascent. We highlight key considerations when choosing between existing methods and opportunities for methodological innovation.
Collapse
|
35
|
Awasthi R, Guliani KK, Khan SA, Vashishtha A, Gill MS, Bhatt A, Nagori A, Gupta A, Kumaraguru P, Sethi T. VacSIM: Learning effective strategies for COVID-19 vaccine distribution using reinforcement learning. INTELLIGENCE-BASED MEDICINE 2022; 6:100060. [PMID: 35610985 PMCID: PMC9119863 DOI: 10.1016/j.ibmed.2022.100060] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/15/2021] [Revised: 12/18/2021] [Accepted: 03/29/2022] [Indexed: 12/18/2022]
Abstract
A COVID-19 vaccine is our best bet for mitigating the ongoing onslaught of the pandemic. However, vaccine is also expected to be a limited resource. An optimal allocation strategy, especially in countries with access inequities and temporal separation of hot-spots, might be an effective way of halting the disease spread. We approach this problem by proposing a novel pipeline VacSIM that dovetails Deep Reinforcement Learning models into a Contextual Bandits approach for optimizing the distribution of COVID-19 vaccine. Whereas the Reinforcement Learning models suggest better actions and rewards, Contextual Bandits allow online modifications that may need to be implemented on a day-to-day basis in the real world scenario. We evaluate this framework against a naive allocation approach of distributing vaccine proportional to the incidence of COVID-19 cases in five different States across India (Assam, Delhi, Jharkhand, Maharashtra and Nagaland) and demonstrate up to 9039 potential infections prevented and a significant increase in the efficacy of limiting the spread over a period of 45 days through the VacSIM approach. Our models and the platform are extensible to all states of India and potentially across the globe. We also propose novel evaluation strategies including standard compartmental model-based projections and a causality-preserving evaluation of our model. Since all models carry assumptions that may need to be tested in various contexts, we open source our model VacSIM and contribute a new reinforcement learning environment compatible with OpenAI gym to make it extensible for real-world applications across the globe. 2
Collapse
Affiliation(s)
- Raghav Awasthi
- Indraprastha Institute of Information Technology Delhi, India
| | | | - Saif Ahmad Khan
- Indraprastha Institute of Information Technology Delhi, India
| | | | | | - Arshita Bhatt
- Bhagwan Parshuram Institute of Technology, New Delhi, India
| | - Aditya Nagori
- CSIR-Institute of Genomics and Integrative Biology, New Delhi, India
| | - Aniket Gupta
- Indraprastha Institute of Information Technology Delhi, India
| | | | | |
Collapse
|
36
|
AI and Clinical Decision Making: The Limitations and Risks of Computational Reductionism in Bowel Cancer Screening. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12073341] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/09/2022]
Abstract
Advances in artificial intelligence in healthcare are frequently promoted as ‘solutions’ to improve the accuracy, safety, and quality of clinical decisions, treatments, and care. Despite some diagnostic success, however, AI systems rely on forms of reductive reasoning and computational determinism that embed problematic assumptions about clinical decision-making and clinical practice. Clinician autonomy, experience, and judgement are reduced to inputs and outputs framed as binary or multi-class classification problems benchmarked against a clinician’s capacity to identify or predict disease states. This paper examines this reductive reasoning in AI systems for colorectal cancer (CRC) to highlight their limitations and risks: (1) in AI systems themselves due to inherent biases in (a) retrospective training datasets and (b) embedded assumptions in underlying AI architectures and algorithms; (2) in the problematic and limited evaluations being conducted on AI systems prior to system integration in clinical practice; and (3) in marginalising socio-technical factors in the context-dependent interactions between clinicians, their patients, and the broader health system. The paper argues that to optimise benefits from AI systems and to avoid negative unintended consequences for clinical decision-making and patient care, there is a need for more nuanced and balanced approaches to AI system deployment and evaluation in CRC.
Collapse
|
37
|
Oselio B, Singal AG, Zhang X, Van T, Liu B, Zhu J, Waljee AK. Reinforcement learning evaluation of treatment policies for patients with hepatitis C virus. BMC Med Inform Decis Mak 2022; 22:63. [PMID: 35272662 PMCID: PMC8913329 DOI: 10.1186/s12911-022-01789-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2021] [Accepted: 02/22/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Evaluation of new treatment policies is often costly and challenging in complex conditions, such as hepatitis C virus (HCV) treatment, or in limited-resource settings. We sought to identify hypothetical policies for HCV treatment that could best balance the prevention of cirrhosis while preserving resources (financial or otherwise). METHODS The cohort consisted of 3792 HCV-infected patients without a history of cirrhosis or hepatocellular carcinoma at baseline from the national Veterans Health Administration from 2015 to 2019. To estimate the efficacy of hypothetical treatment policies, we utilized historical data and reinforcement learning to allow for greater flexibility when constructing new HCV treatment strategies. We tested and compared four new treatment policies: a simple stepwise policy based on Aspartate Aminotransferase to Platelet Ratio Index (APRI), a logistic regression based on APRI, a logistic regression on multiple longitudinal and demographic indicators that were prespecified for clinical significance, and a treatment policy based on a risk model developed for HCV infection. RESULTS The risk-based hypothetical treatment policy achieved the lowest overall risk with a score of 0.016 (90% CI 0.016, 0.019) while treating the most high-risk (346.4 ± 1.4) and the fewest low-risk (361.0 ± 20.1) patients. Compared to hypothetical treatment policies that treated approximately the same number of patients (1843.7 vs. 1914.4 patients), the risk-based policy had more untreated time per patient (7968.4 vs. 7742.9 patient visits), signaling cost reduction for the healthcare system. CONCLUSIONS Off-policy evaluation strategies are useful to evaluate hypothetical treatment policies without implementation. If a quality risk model is available, risk-based treatment strategies can reduce overall risk and prioritize patients while reducing healthcare system costs.
Collapse
Affiliation(s)
- Brandon Oselio
- Department of Biostatistics, University of Michigan, Ann Arbor, MI USA
| | - Amit G. Singal
- Department of Internal Medicine, Division of Digestive and Liver Diseases, UT Southwestern Medical Center, Dallas, TX USA
| | - Xuefei Zhang
- Department of Statistics, University of Michigan, Ann Arbor, MI USA
| | - Tony Van
- Health Services Research and Development Center of Clinical Management Research, VA Ann Arbor Healthcare System, 2215 Fuller Road, Gastroenterology 111D, Ann Arbor, MI 48105 USA
| | - Boang Liu
- Department of Statistics, University of Michigan, Ann Arbor, MI USA
- Googleplex, 1600 Amphitheatre Parkway, Mountainview, CA USA
| | - Ji Zhu
- Department of Statistics, University of Michigan, Ann Arbor, MI USA
- Michigan Integrated Center for Health Analytics and Medical Prediction (MiCHAMP), Ann Arbor, MI USA
| | - Akbar K. Waljee
- Health Services Research and Development Center of Clinical Management Research, VA Ann Arbor Healthcare System, 2215 Fuller Road, Gastroenterology 111D, Ann Arbor, MI 48105 USA
- Michigan Integrated Center for Health Analytics and Medical Prediction (MiCHAMP), Ann Arbor, MI USA
- Department of Internal Medicine, Division of Gastroenterology and Hepatology, Michigan Medicine, Ann Arbor, MI USA
| |
Collapse
|
38
|
Nanayakkara T, Clermont G, Langmead CJ, Swigon D. Unifying cardiovascular modelling with deep reinforcement learning for uncertainty aware control of sepsis treatment. PLOS DIGITAL HEALTH 2022; 1:e0000012. [PMID: 36812511 PMCID: PMC9931225 DOI: 10.1371/journal.pdig.0000012] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/14/2021] [Accepted: 11/29/2021] [Indexed: 11/17/2022]
Abstract
Sepsis is a potentially life-threatening inflammatory response to infection or severe tissue damage. It has a highly variable clinical course, requiring constant monitoring of the patient's state to guide the management of intravenous fluids and vasopressors, among other interventions. Despite decades of research, there's still debate among experts on optimal treatment. Here, we combine for the first time, distributional deep reinforcement learning with mechanistic physiological models to find personalized sepsis treatment strategies. Our method handles partial observability by leveraging known cardiovascular physiology, introducing a novel physiology-driven recurrent autoencoder, and quantifies the uncertainty of its own results. Moreover, we introduce a framework for uncertainty-aware decision support with humans in the loop. We show that our method learns physiologically explainable, robust policies, that are consistent with clinical knowledge. Further our method consistently identifies high-risk states that lead to death, which could potentially benefit from more frequent vasopressor administration, providing valuable guidance for future research.
Collapse
Affiliation(s)
- Thesath Nanayakkara
- Department of Mathematics, University of Pittsburgh, Pittsburgh, PA, 15213, United States of America,* E-mail:
| | - Gilles Clermont
- Department of Critical Care Medicine, The Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, University of Pittsburgh School of Medicine Pittsburgh, PA, 15213, United States of America
| | - Christopher James Langmead
- Computational Biology Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 15213, United States of America
| | - David Swigon
- Department of Mathematics, McGowan Institute for Regenerative Medicine, University of Pittsburgh, Pittsburgh, PA, 15213, United States of America
| |
Collapse
|
39
|
Guo H, Li J, Liu H, He J. Learning dynamic treatment strategies for coronary heart diseases by artificial intelligence: real-world data-driven study. BMC Med Inform Decis Mak 2022; 22:39. [PMID: 35168623 PMCID: PMC8845235 DOI: 10.1186/s12911-022-01774-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 02/01/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Coronary heart disease (CHD) has become the leading cause of death and one of the most serious epidemic diseases worldwide. CHD is characterized by urgency, danger and severity, and dynamic treatment strategies for CHD patients are needed. We aimed to build and validate an AI model for dynamic treatment recommendations for CHD patients with the goal of improving patient outcomes and learning best practices from clinicians to help clinical decision support for treating CHD patients. METHODS We formed the treatment strategy as a sequential decision problem, and applied an AI supervised reinforcement learning-long short-term memory (SRL-LSTM) framework that combined supervised learning (SL) and reinforcement learning (RL) with an LSTM network to track patients' states to learn a recommendation model that took a patient's diagnosis and evolving health status as input and provided a treatment recommendation in the form of whether to take specific drugs. The experiments were conducted by leveraging a real-world intensive care unit (ICU) database with 13,762 admitted patients diagnosed with CHD. We compared the performance of the applied SRL-LSTM model and several state-of-the-art SL and RL models in reducing the estimated in-hospital mortality and the Jaccard similarity with clinicians' decisions. We used a random forest algorithm to calculate the feature importance of both the clinician policy and the AI policy to illustrate the interpretability of the AI model. RESULTS Our experimental study demonstrated that the AI model could help reduce the estimated in-hospital mortality through its RL function and learn the best practice from clinicians through its SL function. The similarity between the clinician policy and the AI policy regarding the surviving patients was high, while for the expired patients, it was much lower. The dynamic treatment strategies made by the AI model were clinically interpretable and relied on sensible clinical features extracted according to monitoring indexes and risk factors for CHD patients. CONCLUSIONS We proposed a pipeline for constructing an AI model to learn dynamic treatment strategies for CHD patients that could improve patient outcomes and mimic the best practices of clinicians. And a lot of further studies and efforts are needed to make it practical.
Collapse
Affiliation(s)
- Haihong Guo
- School of Information, Renmin University of China, 59 Zhongguancun Street, Haidian District, Beijing, 100872, China
- Institute of Medical Information/Medical Library, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
- Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, Beijing, China
| | - Jiao Li
- Institute of Medical Information/Medical Library, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Hongyan Liu
- School of Economics and Management, Tsinghua University, Beijing, China
| | - Jun He
- School of Information, Renmin University of China, 59 Zhongguancun Street, Haidian District, Beijing, 100872, China.
- Key Laboratory of Data Engineering and Knowledge Engineering, Ministry of Education, Beijing, China.
| |
Collapse
|
40
|
Liuzzi P, Campagnini S, Fanciullacci C, Arienti C, Patrini M, Carrozza MC, Mannini A. Predicting SARS-CoV-2 infection duration at hospital admission:a deep learning solution. Med Biol Eng Comput 2022; 60:459-470. [PMID: 34993693 PMCID: PMC8739354 DOI: 10.1007/s11517-021-02479-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Accepted: 11/24/2021] [Indexed: 11/25/2022]
Abstract
COVID-19 cases are increasing around the globe with almost 5 million of deaths. We propose here a deep learning model capable of predicting the duration of the infection by means of information available at hospital admission. A total of 222 patients were enrolled in our observational study. Anagraphical and anamnestic data, COVID-19 signs and symptoms, COVID-19 therapy, hematochemical test results, and prior therapies administered to patients are used as predictors. A set of 55 features, all of which can be taken in the first hours of the patient's hospitalization, was considered. Different solutions were compared achieving the best performance with a sequential convolutional neural network-based model merged in an ensemble with two different meta-learners linked in cascade. We obtained a median absolute error of 2.7 days (IQR = 3.0) in predicting the duration of the infection; the error was equally distributed in the infection duration range. This tool could preemptively give an outlook of the COVID-19 patients' expected path and the associated hospitalization effort. The proposed solution could be viable in tackling the huge burden and the logistics complexity of hospitals or rehabilitation centers during the pandemic waves. With data taken ad admission, entering a PCA-based feature selection, a k-fold cross-validated CNN-based model was implemented. After external texting, a median absolute error of 2.7 days [IQR = 3 days].
Collapse
Affiliation(s)
- Piergiuseppe Liuzzi
- Scuola Superiore Sant'Anna, The BioRobotics Institute, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy
- IRCCS Fondazione Don Carlo Gnocchi, via di Scandicci 269, 50143, Firenze, FI, Italy
| | - Silvia Campagnini
- Scuola Superiore Sant'Anna, The BioRobotics Institute, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy.
- IRCCS Fondazione Don Carlo Gnocchi, via di Scandicci 269, 50143, Firenze, FI, Italy.
| | - Chiara Fanciullacci
- IRCCS Fondazione Don Carlo Gnocchi, via di Scandicci 269, 50143, Firenze, FI, Italy
| | - Chiara Arienti
- IRCCS Fondazione Don Carlo Gnocchi, via Alfonso Capecelatro 66, 20148, Milano, FI, Italy
| | - Michele Patrini
- IRCCS Fondazione Don Carlo Gnocchi, via Alfonso Capecelatro 66, 20148, Milano, FI, Italy
| | - Maria Chiara Carrozza
- Scuola Superiore Sant'Anna, The BioRobotics Institute, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy
| | - Andrea Mannini
- Scuola Superiore Sant'Anna, The BioRobotics Institute, Viale Rinaldo Piaggio 34, 56025, Pontedera, PI, Italy
- IRCCS Fondazione Don Carlo Gnocchi, via di Scandicci 269, 50143, Firenze, FI, Italy
| |
Collapse
|
41
|
El-Kareh R, Sittig DF. Enhancing Diagnosis Through Technology: Decision Support, Artificial Intelligence, and Beyond. Crit Care Clin 2022; 38:129-139. [PMID: 34794627 PMCID: PMC8608279 DOI: 10.1016/j.ccc.2021.08.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
Patient care in intensive care environments is complex, time-sensitive, and data-rich, factors that make these settings particularly well-suited to clinical decision support (CDS). A wide range of CDS interventions have been used in intensive care unit environments. The field needs well-designed studies to identify the most effective CDS approaches. Evolving artificial intelligence and machine learning models may reduce information-overload and enable teams to take better advantage of the large volume of patient data available to them. It is vital to effectively integrate new CDS into clinical workflows and to align closely with the cognitive processes of frontline clinicians.
Collapse
Affiliation(s)
- Robert El-Kareh
- University of California, San Diego, 9500 Gilman Drive, #0881 La Jolla, CA 92093-0881, USA.
| | - Dean F Sittig
- School of Biomedical Informatics, The University of Texas Health Science Center at Houston, UT-Memorial Hermann Center for Healthcare Quality & Safety, Houston, TX 77030, USA. https://twitter.com/DeanSittig
| |
Collapse
|
42
|
Zheng H, Zhu J, Xie W, Zhong J. Reinforcement learning assisted oxygen therapy for COVID-19 patients under intensive care. BMC Med Inform Decis Mak 2021; 21:350. [PMID: 34920724 PMCID: PMC8678583 DOI: 10.1186/s12911-021-01712-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2021] [Accepted: 11/12/2021] [Indexed: 12/21/2022] Open
Abstract
BACKGROUND Patients with severe Coronavirus disease 19 (COVID-19) typically require supplemental oxygen as an essential treatment. We developed a machine learning algorithm, based on deep Reinforcement Learning (RL), for continuous management of oxygen flow rate for critically ill patients under intensive care, which can identify the optimal personalized oxygen flow rate with strong potentials to reduce mortality rate relative to the current clinical practice. METHODS We modeled the oxygen flow trajectory of COVID-19 patients and their health outcomes as a Markov decision process. Based on individual patient characteristics and health status, an optimal oxygen control policy is learned by using deep deterministic policy gradient (DDPG) and real-time recommends the oxygen flow rate to reduce the mortality rate. We assessed the performance of proposed methods through cross validation by using a retrospective cohort of 1372 critically ill patients with COVID-19 from New York University Langone Health ambulatory care with electronic health records from April 2020 to January 2021. RESULTS The mean mortality rate under the RL algorithm is lower than the standard of care by 2.57% (95% CI: 2.08-3.06) reduction (P < 0.001) from 7.94% under the standard of care to 5.37% under our proposed algorithm. The averaged recommended oxygen flow rate is 1.28 L/min (95% CI: 1.14-1.42) lower than the rate delivered to patients. Thus, the RL algorithm could potentially lead to better intensive care treatment that can reduce the mortality rate, while saving the oxygen scarce resources. It can reduce the oxygen shortage issue and improve public health during the COVID-19 pandemic. CONCLUSIONS A personalized reinforcement learning oxygen flow control algorithm for COVID-19 patients under intensive care showed a substantial reduction in 7-day mortality rate as compared to the standard of care. In the overall cross validation cohort independent of the training data, mortality was lowest in patients for whom intensivists' actual flow rate matched the RL decisions.
Collapse
Affiliation(s)
- Hua Zheng
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, 334 SN, Boston, MA, 02115, USA
| | - Jiahao Zhu
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, 334 SN, Boston, MA, 02115, USA
| | - Wei Xie
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, 334 SN, Boston, MA, 02115, USA.
| | - Judy Zhong
- Division of Biostatistics, Department of Population Health, New York University School of Medicine, NYU Langone Health, 180 Madison Avenue, 4th Floor, Room 452, New York, NY, 10016, USA.
| |
Collapse
|
43
|
Overcoming Challenges of Applying Reinforcement Learning for Intelligent Vehicle Control. SENSORS 2021; 21:s21237829. [PMID: 34883832 PMCID: PMC8659501 DOI: 10.3390/s21237829] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Revised: 11/18/2021] [Accepted: 11/20/2021] [Indexed: 12/04/2022]
Abstract
Reinforcement learning (RL) is a booming area in artificial intelligence. The applications of RL are endless nowadays, ranging from fields such as medicine or finance to manufacturing or the gaming industry. Although multiple works argue that RL can be key to a great part of intelligent vehicle control related problems, there are many practical problems that need to be addressed, such as safety related problems that can result from non-optimal training in RL. For instance, for an RL agent to be effective it should first cover all the situations during training that it may face later. This is often difficult when applied to the real-world. In this work we investigate the impact of RL applied to the context of intelligent vehicle control. We analyse the implications of RL in path planning tasks and we discuss two possible approaches to overcome the gap between the theorical developments of RL and its practical applications. Specifically, firstly this paper discusses the role of Curriculum Learning (CL) to structure the learning process of intelligent vehicle control in a gradual way. The results show how CL can play an important role in training agents in such context. Secondly, we discuss a method of transferring RL policies from simulation to reality in order to make the agent experience situations in simulation, so it knows how to react to them in reality. For that, we use Arduino Yún controlled robots as our platforms. The results enhance the effectiveness of the presented approach and show how RL policies can be transferred from simulation to reality even when the platforms are resource limited.
Collapse
|
44
|
Falconer N, Abdel-Hafez A, Scott IA, Marxen S, Canaris S, Barras M. Systematic review of machine learning models for personalised dosing of heparin. Br J Clin Pharmacol 2021; 87:4124-4139. [PMID: 33835524 DOI: 10.1111/bcp.14852] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Revised: 03/25/2021] [Accepted: 03/29/2021] [Indexed: 12/18/2022] Open
Abstract
AIM To identify and critically appraise studies of prediction models, developed using machine learning (ML) methods, for determining the optimal dosing of unfractionated heparin (UFH). METHODS Embase, PubMed, CINAHL, Web of Science, International Pharmaceutical Abstracts and IEEE Xplore databases were searched from inception to 31 January 2020 to identify relevant studies using key search terms synonymous with artificial intelligence or ML, 'prediction', 'dose', 'activated partial thromboplastin time (aPTT)' and 'UFH.' Studies had to have used ML methods for developing models that predicted optimal dose of UFH or target therapeutic aPTT levels in the hospital setting. The CHARMS Checklist was used to assess quality and risk of bias of included studies. RESULTS Of 8393 retrieved abstracts, 61 underwent full text review and eight studies met inclusion criteria. Four studies described models for predicting aPTT, three studies described models predicting optimal dose of heparin during dialysis and one study described a model that used surrogate outcomes of clotting and bleeding to predict a therapeutic aPTT. Studies varied widely in reporting of study participants, feature characterisation and selection, handling of missing data, sample size calculations and the intended clinical application of the model. Only one study conducted an external validation and no studies evaluated model impacts in clinical practice. CONCLUSION Studies of ML models for UFH dosing are few and none report a model ready for routine clinical use. Existing studies are limited by low methodological quality, inadequate reporting of study factors and absence of external validation and impact analysis.
Collapse
Affiliation(s)
- Nazanin Falconer
- Department of Pharmacy, Princess Alexandra Hospital, Brisbane, Queensland, 4102, Australia
- School of Pharmacy, The University of Queensland, Brisbane, Queensland, 4102, Australia
- Centre for Health Services Research, The University of Queensland, Level two, Building 33, Princess Alexandra Hospital, Brisbane, 4102, Australia
| | - Ahmad Abdel-Hafez
- Clinical Informatics, Princess Alexandra Hospital, Brisbane, Queensland, 4102, Australia
| | - Ian A Scott
- Department of Internal Medicine and Clinical Epidemiology, Princess Alexandra Hospital, Brisbane, Queensland, Australia
- School of Clinical Medicine, Faculty of Medicine, The University of Queensland, 4102, Australia
| | - Sven Marxen
- Department of Pharmacy, Logan and Beaudesert Hospitals, Meadowbrook, Metro South Health, Brisbane, QLD, 4131, Australia
| | - Stephen Canaris
- Clinical Informatics, Princess Alexandra Hospital, Brisbane, Queensland, 4102, Australia
| | - Michael Barras
- Department of Pharmacy, Princess Alexandra Hospital, Brisbane, Queensland, 4102, Australia
- School of Pharmacy, The University of Queensland, Brisbane, Queensland, 4102, Australia
| |
Collapse
|
45
|
Giordano C, Brennan M, Mohamed B, Rashidi P, Modave F, Tighe P. Accessing Artificial Intelligence for Clinical Decision-Making. Front Digit Health 2021; 3:645232. [PMID: 34713115 PMCID: PMC8521931 DOI: 10.3389/fdgth.2021.645232] [Citation(s) in RCA: 66] [Impact Index Per Article: 22.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2020] [Accepted: 06/01/2021] [Indexed: 11/13/2022] Open
Abstract
Advancements in computing and data from the near universal acceptance and implementation of electronic health records has been formative for the growth of personalized, automated, and immediate patient care models that were not previously possible. Artificial intelligence (AI) and its subfields of machine learning, reinforcement learning, and deep learning are well-suited to deal with such data. The authors in this paper review current applications of AI in clinical medicine and discuss the most likely future contributions that AI will provide to the healthcare industry. For instance, in response to the need to risk stratify patients, appropriately cultivated and curated data can assist decision-makers in stratifying preoperative patients into risk categories, as well as categorizing the severity of ailments and health for non-operative patients admitted to hospitals. Previous overt, traditional vital signs and laboratory values that are used to signal alarms for an acutely decompensating patient may be replaced by continuously monitoring and updating AI tools that can pick up early imperceptible patterns predicting subtle health deterioration. Furthermore, AI may help overcome challenges with multiple outcome optimization limitations or sequential decision-making protocols that limit individualized patient care. Despite these tremendously helpful advancements, the data sets that AI models train on and develop have the potential for misapplication and thereby create concerns for application bias. Subsequently, the mechanisms governing this disruptive innovation must be understood by clinical decision-makers to prevent unnecessary harm. This need will force physicians to change their educational infrastructure to facilitate understanding AI platforms, modeling, and limitations to best acclimate practice in the age of AI. By performing a thorough narrative review, this paper examines these specific AI applications, limitations, and requisites while reviewing a few examples of major data sets that are being cultivated and curated in the US.
Collapse
Affiliation(s)
- Chris Giordano
- Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL, United States
| | - Meghan Brennan
- Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL, United States
| | - Basma Mohamed
- Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL, United States
| | - Parisa Rashidi
- J. Clayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, United States
| | - François Modave
- Department of Health Outcomes & Biomedical Informatics, University of Florida College of Medicine, Gainesville, FL, United States
| | - Patrick Tighe
- Department of Anesthesiology, University of Florida College of Medicine, Gainesville, FL, United States
| |
Collapse
|
46
|
Li D, Gao J, Hong N, Wang H, Su L, Liu C, He J, Jiang H, Wang Q, Long Y, Zhu W. A Clinical Prediction Model to Predict Heparin Treatment Outcomes and Provide Dosage Recommendations: Development and Validation Study. J Med Internet Res 2021; 23:e27118. [PMID: 34014171 PMCID: PMC8176336 DOI: 10.2196/27118] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2021] [Revised: 03/07/2021] [Accepted: 03/24/2021] [Indexed: 01/30/2023] Open
Abstract
BACKGROUND Unfractionated heparin is widely used in the intensive care unit as an anticoagulant. However, weight-based heparin dosing has been shown to be suboptimal and may place patients at unnecessary risk during their intensive care unit stay. OBJECTIVE In this study, we intended to develop and validate a machine learning-based model to predict heparin treatment outcomes and to provide dosage recommendations to clinicians. METHODS A shallow neural network model was adopted in a retrospective cohort of patients from the Multiparameter Intelligent Monitoring in Intensive Care III (MIMIC III) database and patients admitted to the Peking Union Medical College Hospital (PUMCH). We modeled the subtherapeutic, normal, and supratherapeutic activated partial thromboplastin time (aPTT) as the outcomes of heparin treatment and used a group of clinical features for modeling. Our model classifies patients into 3 different therapeutic states. We tested the prediction ability of our model and evaluated its performance by using accuracy, the kappa coefficient, precision, recall, and the F1 score. Furthermore, a dosage recommendation module was designed and evaluated for clinical decision support. RESULTS A total of 3607 patients selected from MIMIC III and 1549 patients admitted to the PUMCH who met our criteria were included in this study. The shallow neural network model showed results of F1 scores 0.887 (MIMIC III) and 0.925 (PUMCH). When compared with the actual dosage prescribed, our model recommended increasing the dosage for 72.2% (MIMIC III, 1240/1718) and 64.7% (PUMCH, 281/434) of the subtherapeutic patients and decreasing the dosage for 80.9% (MIMIC III, 504/623) and 76.7% (PUMCH, 277/361) of the supratherapeutic patients, suggesting that the recommendations can contribute to clinical improvements and that they may effectively reduce the time to optimal dosage in the clinical setting. CONCLUSIONS The evaluation of our model for predicting heparin treatment outcomes demonstrated that the developed model is potentially applicable for reducing the misdosage of heparin and for providing appropriate decision recommendations to clinicians.
Collapse
Affiliation(s)
- Dongkai Li
- Department of Critical Care Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | - Jianwei Gao
- Digital Health China Technologies Co. Ltd., Beijing, China
| | - Na Hong
- Digital Health China Technologies Co. Ltd., Beijing, China
| | - Hao Wang
- Department of Critical Care Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | - Longxiang Su
- Department of Critical Care Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | - Chun Liu
- Digital Health China Technologies Co. Ltd., Beijing, China
| | - Jie He
- Digital Health China Technologies Co. Ltd., Beijing, China
| | - Huizhen Jiang
- Department of Information Center, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Qiang Wang
- Digital Health China Technologies Co. Ltd., Beijing, China
| | - Yun Long
- Department of Critical Care Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China
| | - Weiguo Zhu
- Department of Information Center, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China.,Department of Primary Care and Family Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
47
|
Zheng H, Zhu J, Xie W, Zhong J. Reinforcement Learning Assisted Oxygen Therapy for COVID-19 Patients Under Intensive Care. ARXIV 2021:arXiv:2105.08923v2. [PMID: 34031644 PMCID: PMC8142656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Revised: 11/07/2021] [Indexed: 11/10/2022]
Abstract
Background Patients with severe Coronavirus disease 19 (COVID-19) typically require supplemental oxygen as an essential treatment. We developed a machine learning algorithm, based on deep Reinforcement Learning (RL), for continuous management of oxygen flow rate for critically ill patients under intensive care, which can identify the optimal personalized oxygen flow rate with strong potentials to reduce mortality rate relative to the current clinical practice. Methods We modeled the oxygen flow trajectory of COVID-19 patients and their health outcomes as a Markov decision process. Based on individual patient characteristics and health status, an optimal oxygen control policy is learned by using deep deterministic policy gradient (DDPG) and real-time recommends the oxygen flow rate to reduce the mortality rate. We assessed the performance of proposed methods through cross validation by using a retrospective cohort of 1,372 critically ill patients with COVID-19 from New York University Langone Health ambulatory care with electronic health records from April 2020 to January 2021. Results The mean mortality rate under the RL algorithm is lower than the standard of care by 2.57% (95% CI: 2.08-3.06) reduction (P<0.001) from 7.94% under the standard of care to 5.37 % under our proposed algorithm. The averaged recommended oxygen flow rate is 1.28 L/min (95% CI: 1.14-1.42) lower than the rate delivered to patients. Thus, the RL algorithm could potentially lead to better intensive care treatment that can reduce the mortality rate, while saving the oxygen scarce resources. It can reduce the oxygen shortage issue and improve public health during the COVID-19 pandemic. Conclusions A personalized reinforcement learning oxygen flow control algorithm for COVID-19 patients under intensive care showed a substantial reduction in 7-day mortality rate as compared to the standard of care. In the overall cross validation cohort independent of the training data, mortality was lowest in patients for whom intensivists' actual flow rate matched the RL decisions.
Collapse
Affiliation(s)
- Hua Zheng
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, Boston, MA 02115
| | - Jiahao Zhu
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, Boston, MA 02115
| | - Wei Xie
- Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, Boston, MA 02115
| | - Judy Zhong
- Division of Biostatistics, Department of Population Health, New York University School of Medicine, 180 Madison Avenue, New York, NY 10016
| |
Collapse
|
48
|
Eghbali N, Alhanai T, Ghassemi MM. Patient-Specific Sedation Management via Deep Reinforcement Learning. Front Digit Health 2021; 3:608893. [PMID: 34713090 PMCID: PMC8521809 DOI: 10.3389/fdgth.2021.608893] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Accepted: 02/04/2021] [Indexed: 02/04/2023] Open
Abstract
Introduction: Developing reliable medication dosing guidelines is challenging because individual dose-response relationships are mitigated by both static (e. g., demographic) and dynamic factors (e.g., kidney function). In recent years, several data-driven medication dosing models have been proposed for sedatives, but these approaches have been limited in their ability to assess interindividual differences and compute individualized doses. Objective: The primary objective of this study is to develop an individualized framework for sedative-hypnotics dosing. Method: Using publicly available data (1,757 patients) from the MIMIC IV intensive care unit database, we developed a sedation management agent using deep reinforcement learning. More specifically, we modeled the sedative dosing problem as a Markov Decision Process and developed an RL agent based on a deep deterministic policy gradient approach with a prioritized experience replay buffer to find the optimal policy. We assessed our method's ability to jointly learn an optimal personalized policy for propofol and fentanyl, which are among commonly prescribed sedative-hypnotics for intensive care unit sedation. We compared our model's medication performance against the recorded behavior of clinicians on unseen data. Results: Experimental results demonstrate that our proposed model would assist clinicians in making the right decision based on patients' evolving clinical phenotype. The RL agent was 8% better at managing sedation and 26% better at managing mean arterial compared to the clinicians' policy; a two-sample t-test validated that these performance improvements were statistically significant (p < 0.05). Conclusion: The results validate that our model had better performance in maintaining control variables within their target range, thereby jointly maintaining patients' health conditions and managing their sedation.
Collapse
Affiliation(s)
- Niloufar Eghbali
- Human Augmentation and Artificial Intelligence Laboratory, Department of Computer Science, Michigan State University, East Lansing, MI, United States
| | - Tuka Alhanai
- Laboratory for Computer-Human Intelligence, Division of Engineering, New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - Mohammad M. Ghassemi
- Human Augmentation and Artificial Intelligence Laboratory, Department of Computer Science, Michigan State University, East Lansing, MI, United States
| |
Collapse
|
49
|
Datta S, Li Y, Ruppert MM, Ren Y, Shickel B, Ozrazgat-Baslanti T, Rashidi P, Bihorac A. Reinforcement learning in surgery. Surgery 2021; 170:329-332. [PMID: 33436272 DOI: 10.1016/j.surg.2020.11.040] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2020] [Revised: 11/24/2020] [Accepted: 11/27/2020] [Indexed: 12/30/2022]
Abstract
Patients and physicians make essential decisions regarding diagnostic and therapeutic interventions. These actions should be performed or deferred under time constraints and uncertainty regarding patients' diagnoses and predicted response to treatment. This may lead to cognitive and judgment errors. Reinforcement learning is a subfield of machine learning that identifies a sequence of actions to increase the probability of achieving a predetermined goal. Reinforcement learning has the potential to assist in surgical decision making by recommending actions at predefined intervals and its ability to utilize complex input data, including text, image, and temporal data, in the decision-making process. The algorithm mimics a human trial-and-error learning process to calculate optimum recommendation policies. The article provides insight regarding challenges in the development and application of reinforcement learning in the medical field, with an emphasis on surgical decision making. The review focuses on challenges in formulating reward function describing the ultimate goal and determination of patient states derived from electronic health records, along with the lack of resources to simulate the potential benefits of suggested actions in response to changing physiological states during and after surgery. Although clinical implementation would require secure, interoperable, livestreaming electronic health record data for use by virtual model, development and validation of personalized reinforcement learning models in surgery can contribute to improving care by helping patients and clinicians make better decisions.
Collapse
Affiliation(s)
- Shounak Datta
- Department of Medicine, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Yanjun Li
- NSF Center for Big Learning, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Matthew M Ruppert
- Department of Medicine, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Yuanfang Ren
- Department of Medicine, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Benjamin Shickel
- Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Tezcan Ozrazgat-Baslanti
- Department of Medicine, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Parisa Rashidi
- Department of Biomedical Engineering, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL
| | - Azra Bihorac
- Department of Medicine, College of Medicine, University of Florida, Gainesville, FL; Precision and Intelligent Systems in Medicine (PRISMA(P)), University of Florida, Gainesville, FL.
| |
Collapse
|
50
|
George MM, Tolley NS. AIM in Otolaryngology and Head & Neck Surgery. Artif Intell Med 2021. [DOI: 10.1007/978-3-030-58080-3_198-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|