1
|
Ose B, Sattar Z, Gupta A, Toquica C, Harvey C, Noheria A. Artificial Intelligence Interpretation of the Electrocardiogram: A State-of-the-Art Review. Curr Cardiol Rep 2024; 26:561-580. [PMID: 38753291 DOI: 10.1007/s11886-024-02062-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/17/2024] [Indexed: 06/26/2024]
Abstract
PURPOSE OF REVIEW Artificial intelligence (AI) is transforming electrocardiography (ECG) interpretation. AI diagnostics can reach beyond human capabilities, facilitate automated access to nuanced ECG interpretation, and expand the scope of cardiovascular screening in the population. AI can be applied to the standard 12-lead resting ECG and single-lead ECGs in external monitors, implantable devices, and direct-to-consumer smart devices. We summarize the current state of the literature on AI-ECG. RECENT FINDINGS Rhythm classification was the first application of AI-ECG. Subsequently, AI-ECG models have been developed for screening structural heart disease including hypertrophic cardiomyopathy, cardiac amyloidosis, aortic stenosis, pulmonary hypertension, and left ventricular systolic dysfunction. Further, AI models can predict future events like development of systolic heart failure and atrial fibrillation. AI-ECG exhibits potential in acute cardiac events and non-cardiac applications, including acute pulmonary embolism, electrolyte abnormalities, monitoring drugs therapy, sleep apnea, and predicting all-cause mortality. Many AI models in the domain of cardiac monitors and smart watches have received Food and Drug Administration (FDA) clearance for rhythm classification, while others for identification of cardiac amyloidosis, pulmonary hypertension and left ventricular dysfunction have received breakthrough device designation. As AI-ECG models continue to be developed, in addition to regulatory oversight and monetization challenges, thoughtful clinical implementation to streamline workflows, avoiding information overload and overwhelming of healthcare systems with false positive results is necessary. Research to demonstrate and validate improvement in healthcare efficiency and improved patient outcomes would be required before widespread adoption of any AI-ECG model.
Collapse
Affiliation(s)
- Benjamin Ose
- The University of Kansas School of Medicine, Kansas City, KS, USA
| | - Zeeshan Sattar
- Division of General and Hospital Medicine, The University of Kansas Medical Center, Kansas City, KS, USA
| | - Amulya Gupta
- Department of Cardiovascular Medicine, The University of Kansas Medical Center, Kansas City, KS, USA
- Program for AI & Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center, Kansas City, KS, USA
| | | | - Chris Harvey
- Department of Cardiovascular Medicine, The University of Kansas Medical Center, Kansas City, KS, USA
- Program for AI & Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center, Kansas City, KS, USA
| | - Amit Noheria
- Department of Cardiovascular Medicine, The University of Kansas Medical Center, Kansas City, KS, USA.
- Program for AI & Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center, Kansas City, KS, USA.
| |
Collapse
|
2
|
Dudas D, Dilling TJ, El Naqa I. Improved outcome models with denoising diffusion. Phys Med 2024; 119:103307. [PMID: 38325221 PMCID: PMC10939775 DOI: 10.1016/j.ejmp.2024.103307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 01/12/2024] [Accepted: 01/31/2024] [Indexed: 02/09/2024] Open
Abstract
PURPOSE Radiotherapy outcome modelling often suffers from class imbalance in the modelled endpoints. One of the main options to address this issue is by introducing new synthetically generated datapoints, using generative models, such as Denoising Diffusion Probabilistic Models (DDPM). In this study, we implemented DDPM to improve performance of a tumor local control model, trained on imbalanced dataset, and compare this approach with other common techniques. METHODS A dataset of 535 NSCLC patients treated with SBRT (50 Gy/5 fractions) was used to train a deep learning outcome model for tumor local control prediction. The dataset included complete treatment planning data (planning CT images, 3D planning dose distribution and patient demographics) with sparsely distributed endpoints (6-7 % experiencing local failure). Consequently, we trained a novel conditional 3D DDPM model to generate synthetic treatment planning data. Synthetically generated treatment planning datapoints were used to supplement the real training dataset and the improvement in the model's performance was studied. Obtained results were also compared to other common techniques for class imbalanced training, such as Oversampling, Undersampling, Augmentation, Class Weights, SMOTE and ADASYN. RESULTS Synthetic DDPM-generated data were visually trustworthy, with Fréchet inception distance (FID) below 50. Extending the training dataset with the synthetic data improved the model's performance by more than 10%, while other techniques exhibited only about 4% improvement. CONCLUSIONS DDPM introduces a novel approach to class-imbalanced outcome modelling problems. The model generates realistic synthetic radiotherapy planning data, with a strong potential to increase performance and robustness of outcome models.
Collapse
Affiliation(s)
- D Dudas
- H. Lee Moffitt Cancer Center and Research Institute, Department of Machine Learning, Tampa, FL, USA; Czech Technical University in Prague, Faculty of Nuclear Sciences and Physical Engineering, Prague, Czechia.
| | - T J Dilling
- H. Lee Moffitt Cancer Center and Research Institute, Department of Machine Learning, Tampa, FL, USA
| | - I El Naqa
- H. Lee Moffitt Cancer Center and Research Institute, Department of Machine Learning, Tampa, FL, USA
| |
Collapse
|
3
|
Ciobanu-Caraus O, Aicher A, Kernbach JM, Regli L, Serra C, Staartjes VE. A critical moment in machine learning in medicine: on reproducible and interpretable learning. Acta Neurochir (Wien) 2024; 166:14. [PMID: 38227273 DOI: 10.1007/s00701-024-05892-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 12/14/2023] [Indexed: 01/17/2024]
Abstract
Over the past two decades, advances in computational power and data availability combined with increased accessibility to pre-trained models have led to an exponential rise in machine learning (ML) publications. While ML may have the potential to transform healthcare, this sharp increase in ML research output without focus on methodological rigor and standard reporting guidelines has fueled a reproducibility crisis. In addition, the rapidly growing complexity of these models compromises their interpretability, which currently impedes their successful and widespread clinical adoption. In medicine, where failure of such models may have severe implications for patients' health, the high requirements for accuracy, robustness, and interpretability confront ML researchers with a unique set of challenges. In this review, we discuss the semantics of reproducibility and interpretability, as well as related issues and challenges, and outline possible solutions to counteracting the "black box". To foster reproducibility, standard reporting guidelines need to be further developed and data or code sharing encouraged. Editors and reviewers may equally play a critical role by establishing high methodological standards and thus preventing the dissemination of low-quality ML publications. To foster interpretable learning, the use of simpler models more suitable for medical data can inform the clinician how results are generated based on input data. Model-agnostic explanation tools, sensitivity analysis, and hidden layer representations constitute further promising approaches to increase interpretability. Balancing model performance and interpretability are important to ensure clinical applicability. We have now reached a critical moment for ML in medicine, where addressing these issues and implementing appropriate solutions will be vital for the future evolution of the field.
Collapse
Affiliation(s)
- Olga Ciobanu-Caraus
- Machine Intelligence in Clinical Neuroscience & Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Anatol Aicher
- Machine Intelligence in Clinical Neuroscience & Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Julius M Kernbach
- Department of Neuroradiology, University Hospital Heidelberg, Heidelberg, Germany
| | - Luca Regli
- Machine Intelligence in Clinical Neuroscience & Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Carlo Serra
- Machine Intelligence in Clinical Neuroscience & Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Victor E Staartjes
- Machine Intelligence in Clinical Neuroscience & Microsurgical Neuroanatomy (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland.
| |
Collapse
|
4
|
Johnsson V, Søndergaard MB, Kulasegaram K, Sundberg K, Tiblad E, Herling L, Petersen OB, Tolsgaard MG. Validity evidence supporting clinical skills assessment by artificial intelligence compared with trained clinician raters. MEDICAL EDUCATION 2024; 58:105-117. [PMID: 37615058 DOI: 10.1111/medu.15190] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 07/19/2023] [Accepted: 07/29/2023] [Indexed: 08/25/2023]
Abstract
BACKGROUND Artificial intelligence (AI) is becoming increasingly used in medical education, but our understanding of the validity of AI-based assessments (AIBA) as compared with traditional clinical expert-based assessments (EBA) is limited. In this study, the authors aimed to compare and contrast the validity evidence for the assessment of a complex clinical skill based on scores generated from an AI and trained clinical experts, respectively. METHODS The study was conducted between September 2020 to October 2022. The authors used Kane's validity framework to prioritise and organise their evidence according to the four inferences: scoring, generalisation, extrapolation and implications. The context of the study was chorionic villus sampling performed within the simulated setting. AIBA and EBA were used to evaluate performances of experts, intermediates and novice based on video recordings. The clinical experts used a scoring instrument developed in a previous international consensus study. The AI used convolutional neural networks for capturing features on video recordings, motion tracking and eye movements to arrive at a final composite score. RESULTS A total of 45 individuals participated in the study (22 novices, 12 intermediates and 11 experts). The authors demonstrated validity evidence for scoring, generalisation, extrapolation and implications for both EBA and AIBA. The plausibility of assumptions related to scoring, evidence of reproducibility and relation to different training levels was examined. Issues relating to construct underrepresentation, lack of explainability, and threats to robustness were identified as potential weak links in the AIBA validity argument compared with the EBA validity argument. CONCLUSION There were weak links in the use of AIBA compared with EBA, mainly in their representation of the underlying construct but also regarding their explainability and ability to transfer to other datasets. However, combining AI and clinical expert-based assessments may offer complementary benefits, which is a promising subject for future research.
Collapse
Affiliation(s)
- Vilma Johnsson
- Center for Fetal Medicine, Department of Obstetrics, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
- Faculty of Health and Medical Science, University of Copenhagen, Copenhagen, Denmark
| | - Morten Bo Søndergaard
- Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | - Kulamakan Kulasegaram
- Department of Family and Community Medicine and Scientist, Wilson Centre, Toronto, Ontario, Canada
- Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Karin Sundberg
- Center for Fetal Medicine, Department of Obstetrics, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Eleonor Tiblad
- Center for Fetal Medicine, Karolinska University Hospital, Stockholm, Sweden
- Clinical Epidemiology Division, Department of Medicine Solna, Karolinska Institutet, Stockholm, Sweden
| | - Lotta Herling
- Center for Fetal Medicine, Karolinska University Hospital, Stockholm, Sweden
- Department of Clinical Science, Intervention and Technology, Karolinska Institutet, Stockholm, Sweden
| | - Olav Bjørn Petersen
- Center for Fetal Medicine, Department of Obstetrics, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Martin G Tolsgaard
- Center for Fetal Medicine, Department of Obstetrics, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
- Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
5
|
Wei L, Niraula D, Gates EDH, Fu J, Luo Y, Nyflot MJ, Bowen SR, El Naqa IM, Cui S. Artificial intelligence (AI) and machine learning (ML) in precision oncology: a review on enhancing discoverability through multiomics integration. Br J Radiol 2023; 96:20230211. [PMID: 37660402 PMCID: PMC10546458 DOI: 10.1259/bjr.20230211] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 06/15/2023] [Accepted: 06/27/2023] [Indexed: 09/05/2023] Open
Abstract
Multiomics data including imaging radiomics and various types of molecular biomarkers have been increasingly investigated for better diagnosis and therapy in the era of precision oncology. Artificial intelligence (AI) including machine learning (ML) and deep learning (DL) techniques combined with the exponential growth of multiomics data may have great potential to revolutionize cancer subtyping, risk stratification, prognostication, prediction and clinical decision-making. In this article, we first present different categories of multiomics data and their roles in diagnosis and therapy. Second, AI-based data fusion methods and modeling methods as well as different validation schemes are illustrated. Third, the applications and examples of multiomics research in oncology are demonstrated. Finally, the challenges regarding the heterogeneity data set, availability of omics data, and validation of the research are discussed. The transition of multiomics research to real clinics still requires consistent efforts in standardizing omics data collection and analysis, building computational infrastructure for data sharing and storing, developing advanced methods to improve data fusion and interpretability, and ultimately, conducting large-scale prospective clinical trials to fill the gap between study findings and clinical benefits.
Collapse
Affiliation(s)
- Lise Wei
- Department of Radiation Oncology, University of Michigan, Michigan, United States
| | - Dipesh Niraula
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, United States
| | - Evan D. H. Gates
- Department of Radiation Oncology, University of Washington, Washington, United States
| | - Jie Fu
- Department of Radiation Oncology, Stanford University, Stanford, California, United States
| | - Yi Luo
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, United States
| | - Matthew J. Nyflot
- Department of Radiation Oncology, University of Washington, Washington, United States
| | - Stephen R. Bowen
- Department of Radiation Oncology, University of Washington, Washington, United States
| | - Issam M El Naqa
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, United States
| | - Sunan Cui
- Department of Radiation Oncology, University of Washington, Washington, United States
| |
Collapse
|
6
|
Cui S, Traverso A, Niraula D, Zou J, Luo Y, Owen D, El Naqa I, Wei L. Interpretable artificial intelligence in radiology and radiation oncology. Br J Radiol 2023; 96:20230142. [PMID: 37493248 PMCID: PMC10546466 DOI: 10.1259/bjr.20230142] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2023] [Revised: 05/24/2023] [Accepted: 05/26/2023] [Indexed: 07/27/2023] Open
Abstract
Artificial intelligence has been introduced to clinical practice, especially radiology and radiation oncology, from image segmentation, diagnosis, treatment planning and prognosis. It is not only crucial to have an accurate artificial intelligence model, but also to understand the internal logic and gain the trust of the experts. This review is intended to provide some insights into core concepts of the interpretability, the state-of-the-art methods for understanding the machine learning models, the evaluation of these methods, identifying some challenges and limits of them, and gives some examples of medical applications.
Collapse
Affiliation(s)
- Sunan Cui
- Department of Radiation Oncology, University of Washington, Seattle, WA, United States
| | - Alberto Traverso
- Department of Radiotherapy, Maastro Clinic, Maastricht, Netherlands
| | - Dipesh Niraula
- Department of Machine Learning, Moffitt Cancer Center, FL, United States
| | - Jiaren Zou
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, United States
| | - Yi Luo
- Department of Machine Learning, Moffitt Cancer Center, FL, United States
| | - Dawn Owen
- Department of Radiation Oncology, Mayo Clinic, Rochester, MN, United States
| | - Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, FL, United States
| | - Lise Wei
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, United States
| |
Collapse
|
7
|
El Naqa I, Karolak A, Luo Y, Folio L, Tarhini AA, Rollison D, Parodi K. Translation of AI into oncology clinical practice. Oncogene 2023; 42:3089-3097. [PMID: 37684407 DOI: 10.1038/s41388-023-02826-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Revised: 08/23/2023] [Accepted: 08/25/2023] [Indexed: 09/10/2023]
Abstract
Artificial intelligence (AI) is a transformative technology that is capturing popular imagination and can revolutionize biomedicine. AI and machine learning (ML) algorithms have the potential to break through existing barriers in oncology research and practice such as automating workflow processes, personalizing care, and reducing healthcare disparities. Emerging applications of AI/ML in the literature include screening and early detection of cancer, disease diagnosis, response prediction, prognosis, and accelerated drug discovery. Despite this excitement, only few AI/ML models have been properly validated and fewer have become regulated products for routine clinical use. In this review, we highlight the main challenges impeding AI/ML clinical translation. We present different clinical use cases from the domains of radiology, radiation oncology, immunotherapy, and drug discovery in oncology. We dissect the unique challenges and opportunities associated with each of these cases. Finally, we summarize the general requirements for successful AI/ML implementation in the clinic, highlighting specific examples and points of emphasis including the importance of multidisciplinary collaboration of stakeholders, role of domain experts in AI augmentation, transparency of AI/ML models, and the establishment of a comprehensive quality assurance program to mitigate risks of training bias and data drifts, all culminating toward safer and more beneficial AI/ML applications in oncology labs and clinics.
Collapse
Affiliation(s)
- Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA.
| | - Aleksandra Karolak
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Yi Luo
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Les Folio
- Diagnostic Imaging & Interventional Radiology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Ahmad A Tarhini
- Cutaneous Oncology and Immunology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Dana Rollison
- Department of Cancer Epidemiology, Moffitt Cancer Center, Tampa, FL, 33612, USA
| | - Katia Parodi
- Department of Medical Physics, Ludwig-Maximilians-Universität München, Munich, Germany
| |
Collapse
|
8
|
Drabiak K, Kyzer S, Nemov V, El Naqa I. AI and machine learning ethics, law, diversity, and global impact. Br J Radiol 2023; 96:20220934. [PMID: 37191072 PMCID: PMC10546451 DOI: 10.1259/bjr.20220934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2022] [Revised: 03/20/2023] [Accepted: 03/29/2023] [Indexed: 05/17/2023] Open
Abstract
Artificial intelligence (AI) and its machine learning (ML) algorithms are offering new promise for personalized biomedicine and more cost-effective healthcare with impressive technical capability to mimic human cognitive capabilities. However, widespread application of this promising technology has been limited in the medical domain and expectations have been tampered by ethical challenges and concerns regarding patient privacy, legal responsibility, trustworthiness, and fairness. To balance technical innovation with ethical applications of AI/ML, developers must demonstrate the AI functions as intended and adopt strategies to minimize the risks for failure or bias. This review describes the new ethical challenges created by AI/ML for clinical care and identifies specific considerations for its practice in medicine. We provide an overview of regulatory and legal issues applicable in Europe and the United States, a description of technical aspects to consider, and present recommendations for trustworthy AI/ML that promote transparency, minimize risks of bias or error, and protect the patient well-being.
Collapse
Affiliation(s)
- Katherine Drabiak
- Colleges of Public Health and Medicine, University of South Florida, Tampa, FL, USA
| | - Skylar Kyzer
- Colleges of Public Health and Medicine, University of South Florida, Tampa, FL, USA
| | - Valerie Nemov
- Colleges of Public Health and Medicine, University of South Florida, Tampa, FL, USA
| | - Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, USA
| |
Collapse
|
9
|
Susič D, Syed-Abdul S, Dovgan E, Jonnagaddala J, Gradišek A. Artificial intelligence based personalized predictive survival among colorectal cancer patients. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 231:107435. [PMID: 36842345 DOI: 10.1016/j.cmpb.2023.107435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 12/14/2022] [Accepted: 02/18/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Colorectal cancer is a major health concern. It is now the third most common cancer and the fourth leading cause of cancer mortality worldwide. The aim of this study was to evaluate the performance of machine learning algorithms for predicting survival of colorectal cancer patients 1 to 5 years after diagnosis, and identify the most important variables. METHODS A sample of 1236 patients diagnosed with colorectal cancer and 118 predictor variables has been used. The outcome of interest was a binary variable indicating whether the patient survived the number of years in question or not. 20 predictor variables were selected using mutual information score with the outcome. We implemented 11 machine learning algorithms and evaluated their performance with a 5 by 2-fold cross-validation with stratified folds and with paired Student's t-tests. We compared the results with the Kaplan-Meier estimator and Cox's proportional hazard regression. RESULTS Using the 20 most important predictor variables for each of the survival years, the logistic regression algorithm achieved an area under the receiver operating characteristic curve of 0.850 (0.014 SD, 0.840-0.860 95 % CI) for the 1-year, and 0.872 (0.014 SD, 0.861-0.882 95% CI) for the 5-year survival prediction. Using only the 5 most important predictor variables, the corresponding values are 0.793 (0.020 SD, 0.778-0.807 95% CI) and 0.794 (0.011 SD, 0.785-0.802 95% CI). The most important variables for 1-year prediction were number of R residual, M distant metastasis, overall stage, probable recurrence within 5 years, and tumour length, whereas for 5-year prediction the most important were probable recurrence within 5 years, R residual, M distant metastasis, number of positive lymph nodes, and palliative chemotherapy. Biomarkers do not appear among the top 20 most important ones. For all survival intervals, the probability of the top model agrees with the Kaplan-Meier estimate, both in the interval of one standard deviation and in the 95% confidence interval. CONCLUSIONS The findings suggest that machine learning algorithms can predict the survival probability of colorectal cancer patients and can be used to inform the patients and assist decision-making in clinical care management. In addition, this study unveils the most essential variables for estimating survival short- and long-term among patients with Colorectal cancer.
Collapse
Affiliation(s)
- David Susič
- Jožef Stefan Institute, Jamova cesta 39, SI-1000 Ljubljana, Slovenia; Jožef Stefan International Postgraduate School, Jamova cesta 39, SI-1000 Ljubljana, Slovenia
| | - Shabbir Syed-Abdul
- Graduate Institute of Biomedical Informatics, College of Medical Science and Technology, Taipei Medical University, Taipei 110, Taiwan.
| | - Erik Dovgan
- Jožef Stefan Institute, Jamova cesta 39, SI-1000 Ljubljana, Slovenia
| | | | - Anton Gradišek
- Jožef Stefan Institute, Jamova cesta 39, SI-1000 Ljubljana, Slovenia.
| |
Collapse
|
10
|
Oguma K, Magome T, Someya M, Hasegawa T, Sakata KI. Virtual clinical trial based on outcome modeling with iteratively redistributed extrapolation data. Radiol Phys Technol 2023; 16:262-271. [PMID: 36947353 DOI: 10.1007/s12194-023-00715-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 03/14/2023] [Accepted: 03/15/2023] [Indexed: 03/23/2023]
Abstract
Virtual clinical trials (VCTs) can potentially simulate clinical trials on a computer, but their application with a limited number of past clinical cases is challenging due to the biased estimation of the statistical population. In this study, we developed ExMixup, a novel training technique based on machine learning, using iteratively redistributed extrapolated data. Information obtained from 100 patients with prostate cancer and 385 patients with oropharyngeal cancer was used to predict the recurrence after radiotherapy. Model performance was evaluated by developing outcome prediction models based on three types of training methods: training with original data (baseline), interpolation data (Mixup), and interpolation + extrapolation data (ExMixup). Two types of VCTs were conducted to predict the treatment response of patients with distinct characteristics compared to the training data obtained from patient cohorts categorized under risk classification or cancer stage. The prediction models developed with ExMixup yielded concordance indices (95% confidence intervals) of 0.751 (0.719-0.818) and 0.752 (0.734-0.785) for VCTs on the prostate and oropharyngeal cancer datasets, respectively, which significantly outperformed the baseline and Mixup models (P < 0.01). The proposed approach could enhance the ability of VCTs to predict treatment results in patients excluded from past clinical trials.
Collapse
Affiliation(s)
- Kohei Oguma
- Graduate Division of Health Sciences, Komazawa University, 1-23-1, Komazawa, Setagaya-Ku, Tokyo, 154-8525, Japan
| | - Taiki Magome
- Graduate Division of Health Sciences, Komazawa University, 1-23-1, Komazawa, Setagaya-Ku, Tokyo, 154-8525, Japan.
| | - Masanori Someya
- Department of Radiology, Sapporo Medical University School of Medicine, Sapporo, Japan
| | - Tomokazu Hasegawa
- Department of Radiology, Sapporo Medical University School of Medicine, Sapporo, Japan
| | - Koh-Ichi Sakata
- Department of Radiology, Sapporo Medical University School of Medicine, Sapporo, Japan
| |
Collapse
|
11
|
Ouifak H, Idri A. On the performance and interpretability of Mamdani and Takagi-Sugeno-Kang based neuro-fuzzy systems for medical diagnosis. SCIENTIFIC AFRICAN 2023. [DOI: 10.1016/j.sciaf.2023.e01610] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/04/2023] Open
|
12
|
Mahmood U, Fu Z, Ghosh S, Calhoun V, Plis S. Through the looking glass: Deep interpretable dynamic directed connectivity in resting fMRI. Neuroimage 2022; 264:119737. [PMID: 36356823 PMCID: PMC9844250 DOI: 10.1016/j.neuroimage.2022.119737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 11/01/2022] [Accepted: 11/06/2022] [Indexed: 11/09/2022] Open
Abstract
Brain network interactions are commonly assessed via functional (network) connectivity, captured as an undirected matrix of Pearson correlation coefficients. Functional connectivity can represent static and dynamic relations, but often these are modeled using a fixed choice for the data window Alternatively, deep learning models may flexibly learn various representations from the same data based on the model architecture and the training task. However, the representations produced by deep learning models are often difficult to interpret and require additional posthoc methods, e.g., saliency maps. In this work, we integrate the strengths of deep learning and functional connectivity methods while also mitigating their weaknesses. With interpretability in mind, we present a deep learning architecture that exposes a directed graph layer that represents what the model has learned about relevant brain connectivity. A surprising benefit of this architectural interpretability is significantly improved accuracy in discriminating controls and patients with schizophrenia, autism, and dementia, as well as age and gender prediction from functional MRI data. We also resolve the window size selection problem for dynamic directed connectivity estimation as we estimate windowing functions from the data, capturing what is needed to estimate the graph at each time-point. We demonstrate efficacy of our method in comparison with multiple existing models that focus on classification accuracy, unlike our interpretability-focused architecture. Using the same data but training different models on their own discriminative tasks we are able to estimate task-specific directed connectivity matrices for each subject. Results show that the proposed approach is also more robust to confounding factors compared to standard dynamic functional connectivity models. The dynamic patterns captured by our model are naturally interpretable since they highlight the intervals in the signal that are most important for the prediction. The proposed approach reveals that differences in connectivity among sensorimotor networks relative to default-mode networks are an important indicator of dementia and gender. Dysconnectivity between networks, specially sensorimotor and visual, is linked with schizophrenic patients, however schizophrenic patients show increased intra-network default-mode connectivity compared to healthy controls. Sensorimotor connectivity was important for both dementia and schizophrenia prediction, but schizophrenia is more related to dysconnectivity between networks whereas, dementia bio-markers were mostly intra-network connectivity.
Collapse
Affiliation(s)
- Usman Mahmood
- Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA; Georgia State University, Department of Computer Science, Atlanta, GA, USA.
| | - Zening Fu
- Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA; Georgia State University, Department of Computer Science, Atlanta, GA, USA
| | - Satrajit Ghosh
- McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA USA; Department of Otolaryngology - Head and Neck Surgery, Harvard Medical School, Boston, MA USA
| | - Vince Calhoun
- Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA; Georgia State University, Department of Computer Science, Atlanta, GA, USA; Georgia Institute of Technology, Department of Electrical and Computer Engineering, Atlanta, GA, USA
| | - Sergey Plis
- Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, GA, USA; Georgia State University, Department of Computer Science, Atlanta, GA, USA
| |
Collapse
|
13
|
Leary D, Basran PS. The role of artificial intelligence in veterinary radiation oncology. Vet Radiol Ultrasound 2022; 63 Suppl 1:903-912. [PMID: 36514233 DOI: 10.1111/vru.13162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2021] [Revised: 01/21/2022] [Accepted: 04/12/2022] [Indexed: 12/15/2022] Open
Abstract
Veterinary radiation oncology regularly deploys sophisticated contouring, image registration, and treatment planning optimization software for patient care. Over the past decade, advances in computing power and the rapid development of neural networks, open-source software packages, and data science have been realized and resulted in new research and clinical applications of artificial intelligent (AI) systems in radiation oncology. These technologies differ from conventional software in their level of complexity and ability to learn from representative and local data. We provide clinical and research application examples of AI in human radiation oncology and their potential applications in veterinary medicine throughout the patient's care-path: from treatment simulation, deformable registration, auto-segmentation, automated treatment planning and plan selection, quality assurance, adaptive radiotherapy, and outcomes modeling. These technologies have the potential to offer significant time and cost savings in the veterinary setting; however, since the range of usefulness of these technologies have not been well studied nor understood, care must be taken if adopting AI technologies in clinical practice. Over the next several years, some practical and realizable applications of AI in veterinary radiation oncology include automated segmentation of normal tissues and tumor volumes, deformable registration, multi-criteria plan optimization, and adaptive radiotherapy. Keys in achieving success in adopting AI in veterinary radiation oncology include: establishing "truth-data"; data harmonization; multi-institutional data and collaborations; standardized dose reporting and taxonomy; adopting an open access philosophy, data collection and curation; open-source algorithm development; and transparent and platform-independent code development.
Collapse
Affiliation(s)
- Del Leary
- Department of Environment and Radiological Health Sciences, College of Veterinary Medicine and Biomedical Sciences, Colorado State University, Fort Collins, Colorado, USA
| | - Parminder S Basran
- Department of Clinical Sciences, College of Veterinary Medicine, Cornell University, Ithaca, New York, USA
| |
Collapse
|
14
|
Niraula D, Cui S, Pakela J, Wei L, Luo Y, Ten Haken RK, El Naqa I. Current status and future developments in predicting outcomes in radiation oncology. Br J Radiol 2022; 95:20220239. [PMID: 35867841 PMCID: PMC9793488 DOI: 10.1259/bjr.20220239] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023] Open
Abstract
Advancements in data-driven technologies and the inclusion of information-rich multiomics features have significantly improved the performance of outcomes modeling in radiation oncology. For this current trend to be sustainable, challenges related to robust data modeling such as small sample size, low size to feature ratio, noisy data, as well as issues related to algorithmic modeling such as complexity, uncertainty, and interpretability, need to be mitigated if not resolved. Emerging computational technologies and new paradigms such as federated learning, human-in-the-loop, quantum computing, and novel interpretability methods show great potential in overcoming these challenges and bridging the gap towards precision outcome modeling in radiotherapy. Examples of these promising technologies will be presented and their potential role in improving outcome modeling will be discussed.
Collapse
Affiliation(s)
- Dipesh Niraula
- Department of Machine Learning, H Lee Moffitt Cancer Center and Research Institute, Tampa, USA
| | - Sunan Cui
- Department of Radiation Oncology, Stanford Medicine, Stanford University, Stanford, USA
| | - Julia Pakela
- Department of Radiation Oncology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA
| | - Lise Wei
- Department of Radiation Oncology, University of Michigan, Ann Arbor, USA
| | - Yi Luo
- Department of Machine Learning, H Lee Moffitt Cancer Center and Research Institute, Tampa, USA
| | | | - Issam El Naqa
- Department of Machine Learning, H Lee Moffitt Cancer Center and Research Institute, Tampa, USA
| |
Collapse
|
15
|
Lahmi L, Mamzer MF, Burgun A, Durdux C, Bibault JE. Ethical Aspects of Artificial Intelligence in Radiation Oncology. Semin Radiat Oncol 2022; 32:442-448. [DOI: 10.1016/j.semradonc.2022.06.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
16
|
Artificial Intelligence for Outcome Modeling in Radiotherapy. Semin Radiat Oncol 2022; 32:351-364. [DOI: 10.1016/j.semradonc.2022.06.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
17
|
Cui Y, Zhu J, Duan Z, Liao Z, Wang S, Liu W. Artificial Intelligence in Spinal Imaging: Current Status and Future Directions. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:11708. [PMID: 36141981 PMCID: PMC9517575 DOI: 10.3390/ijerph191811708] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 09/14/2022] [Accepted: 09/15/2022] [Indexed: 06/16/2023]
Abstract
Spinal maladies are among the most common causes of pain and disability worldwide. Imaging represents an important diagnostic procedure in spinal care. Imaging investigations can provide information and insights that are not visible through ordinary visual inspection. Multiscale in vivo interrogation has the potential to improve the assessment and monitoring of pathologies thanks to the convergence of imaging, artificial intelligence (AI), and radiomic techniques. AI is revolutionizing computer vision, autonomous driving, natural language processing, and speech recognition. These revolutionary technologies are already impacting radiology, diagnostics, and other fields, where automated solutions can increase precision and reproducibility. In the first section of this narrative review, we provide a brief explanation of the many approaches currently being developed, with a particular emphasis on those employed in spinal imaging studies. The previously documented uses of AI for challenges involving spinal imaging, including imaging appropriateness and protocoling, image acquisition and reconstruction, image presentation, image interpretation, and quantitative image analysis, are then detailed. Finally, the future applications of AI to imaging of the spine are discussed. AI has the potential to significantly affect every step in spinal imaging. AI can make images of the spine more useful to patients and doctors by improving image quality, imaging efficiency, and diagnostic accuracy.
Collapse
Affiliation(s)
- Yangyang Cui
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| | - Jia Zhu
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| | - Zhili Duan
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| | - Zhenhua Liao
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| | - Song Wang
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| | - Weiqiang Liu
- Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China
- Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
- Biomechanics and Biotechnology Lab, Research Institute of Tsinghua University in Shenzhen, Shenzhen 518057, China
| |
Collapse
|
18
|
Barragán-Montero A, Bibal A, Dastarac MH, Draguet C, Valdés G, Nguyen D, Willems S, Vandewinckele L, Holmström M, Löfman F, Souris K, Sterpin E, Lee JA. Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency. Phys Med Biol 2022; 67:10.1088/1361-6560/ac678a. [PMID: 35421855 PMCID: PMC9870296 DOI: 10.1088/1361-6560/ac678a] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2021] [Accepted: 04/14/2022] [Indexed: 01/26/2023]
Abstract
The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.
Collapse
Affiliation(s)
- Ana Barragán-Montero
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Adrien Bibal
- PReCISE, NaDI Institute, Faculty of Computer Science, UNamur and CENTAL, ILC, UCLouvain, Belgium
| | - Margerie Huet Dastarac
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Camille Draguet
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - Gilmer Valdés
- Department of Radiation Oncology, Department of Epidemiology and Biostatistics, University of California, San Francisco, United States of America
| | - Dan Nguyen
- Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, UT Southwestern Medical Center, United States of America
| | - Siri Willems
- ESAT/PSI, KU Leuven Belgium & MIRC, UZ Leuven, Belgium
| | | | | | | | - Kevin Souris
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| | - Edmond Sterpin
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
- Department of Oncology, Laboratory of Experimental Radiotherapy, KU Leuven, Belgium
| | - John A Lee
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, Institut de Recherche Expérimentale et Clinique (IREC), UCLouvain, Belgium
| |
Collapse
|
19
|
Combining white box models, black box machines and human interventions for interpretable decision strategies. JUDGMENT AND DECISION MAKING 2022. [DOI: 10.1017/s1930297500003594] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/04/2023]
Abstract
AbstractGranting a short-term loan is a critical decision. A great deal of research has concerned the prediction of credit default, notably through Machine Learning (ML) algorithms. However, given that their black-box nature has sometimes led to unwanted outcomes, comprehensibility in ML guided decision-making strategies has become more important. In many domains, transparency and accountability are no longer optional. In this article, instead of opposing white-box against black-box models, we use a multi-step procedure that combines the Fast and Frugal Tree (FFT) methodology of Martignon et al. (2005) and Phillips et al. (2017) with the extraction of post-hoc explainable information from ensemble ML models. New interpretable models are then built thanks to the inclusion of explainable ML outputs chosen by human intervention. Our methodology improves significantly the accuracy of the FFT predictions while preserving their explainable nature. We apply our approach to a dataset of short-term loans granted to borrowers in the UK, and show how complex machine learning can challenge simpler machines and help decision makers.
Collapse
|
20
|
Pakela JM, Knopf A, Dong L, Rucinski A, Zou W. Management of Motion and Anatomical Variations in Charged Particle Therapy: Past, Present, and Into the Future. Front Oncol 2022; 12:806153. [PMID: 35356213 PMCID: PMC8959592 DOI: 10.3389/fonc.2022.806153] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2021] [Accepted: 02/04/2022] [Indexed: 12/14/2022] Open
Abstract
The major aim of radiation therapy is to provide curative or palliative treatment to cancerous malignancies while minimizing damage to healthy tissues. Charged particle radiotherapy utilizing carbon ions or protons is uniquely suited for this task due to its ability to achieve highly conformal dose distributions around the tumor volume. For these treatment modalities, uncertainties in the localization of patient anatomy due to inter- and intra-fractional motion present a heightened risk of undesired dose delivery. A diverse range of mitigation strategies have been developed and clinically implemented in various disease sites to monitor and correct for patient motion, but much work remains. This review provides an overview of current clinical practices for inter and intra-fractional motion management in charged particle therapy, including motion control, current imaging and motion tracking modalities, as well as treatment planning and delivery techniques. We also cover progress to date on emerging technologies including particle-based radiography imaging, novel treatment delivery methods such as tumor tracking and FLASH, and artificial intelligence and discuss their potential impact towards improving or increasing the challenge of motion mitigation in charged particle therapy.
Collapse
Affiliation(s)
- Julia M Pakela
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, United States
| | - Antje Knopf
- Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, Netherlands.,Department I of Internal Medicine, Center for Integrated Oncology Cologne, University Hospital of Cologne, Cologne, Germany
| | - Lei Dong
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, United States
| | - Antoni Rucinski
- Institute of Nuclear Physics, Polish Academy of Sciences, Krakow, Poland
| | - Wei Zou
- Department of Radiation Oncology, University of Pennsylvania, Philadelphia, PA, United States
| |
Collapse
|
21
|
Cui S, Ten Haken RK, El Naqa I. Integrating Multiomics Information in Deep Learning Architectures for Joint Actuarial Outcome Prediction in Non-Small Cell Lung Cancer Patients After Radiation Therapy. Int J Radiat Oncol Biol Phys 2021; 110:893-904. [PMID: 33539966 PMCID: PMC8180510 DOI: 10.1016/j.ijrobp.2021.01.042] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2020] [Revised: 11/10/2020] [Accepted: 01/23/2021] [Indexed: 12/14/2022]
Abstract
PURPOSE Novel actuarial deep learning neural network (ADNN) architectures are proposed for joint prediction of radiation therapy outcomes-radiation pneumonitis (RP) and local control (LC)-in stage III non-small cell lung cancer (NSCLC) patients. Unlike normal tissue complication probability/tumor control probability models that use dosimetric information solely, our proposed models consider complex interactions among multiomics information including positron emission tomography (PET) radiomics, cytokines, and miRNAs. Additional time-to-event information is also used in the actuarial prediction. METHODS AND MATERIALS Three architectures were investigated: ADNN-DVH considered dosimetric information only; ADNN-com integrated multiomics information; and ADNN-com-joint combined RP2 (RP grade ≥2) and LC prediction. In these architectures, differential dose-volume histograms (DVHs) were fed into 1D convolutional neural networks (CNN) for extracting reduced representations. Variational encoders were used to learn representations of imaging and biological data. Reduced representations were fed into Surv-Nets to predict time-to-event probabilities for RP2 and LC independently and jointly by incorporating time information into designated loss functions. RESULTS Models were evaluated on 117 retrospective patients and were independently tested on 25 newly accrued patients prospectively. A multi-institutional RTOG0617 data set of 327 patients was used for external validation. ADNN-DVH yielded cross-validated c-indexes (95% confidence intervals) of 0.660 (0.630-0.690) for RP2 prediction and 0.727 (0.700-0.753) for LC prediction, outperforming a generalized Lyman model for RP2 (0.613 [0.583-0.643]) and a generalized log-logistic model for LC (0.569 [0.545-0.594]). The independent internal test and external validation yielded similar results. ADNN-com achieved an even better performance than ADNN-DVH on both cross-validation and independent internal test. Furthermore, ADNN-com-joint, which yielded performance similar to ADNN-com, realized joint prediction with c-indexes of 0.705 (0.676-0.734) for RP2 and 0.740 (0.714-0.765) for LC and achieved an area under a free-response receiving operator characteristic curve (AU-FROC) of 0.729 (0.697-0.773) for the joint prediction of RP2 and LC. CONCLUSION Novel deep learning architectures that integrate multiomics information outperformed traditional normal tissue complication probability/tumor control probability models in actuarial prediction of RP2 and LC.
Collapse
Affiliation(s)
- Sunan Cui
- Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan; Applied Physics Program, University of Michigan, Ann Arbor, Michigan.
| | - Randall K Ten Haken
- Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan
| | - Issam El Naqa
- Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan
| |
Collapse
|
22
|
Luo W. Predicting Cervical Cancer Outcomes: Statistics, Images, and Machine Learning. Front Artif Intell 2021; 4:627369. [PMID: 34164615 PMCID: PMC8215338 DOI: 10.3389/frai.2021.627369] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2020] [Accepted: 05/25/2021] [Indexed: 01/22/2023] Open
Abstract
Cervical cancer is a very common and severe disease in women worldwide. Accurate prediction of its clinical outcomes will help adjust or optimize the treatment of cervical cancer and benefit the patients. Statistical models, various types of medical images, and machine learning have been used for outcome prediction and obtained promising results. Compared to conventional statistical models, machine learning has demonstrated advantages in dealing with the complexity in large-scale data and discovering prognostic factors. It has great potential in clinical application and improving cervical cancer management. However, the limitations of prediction studies and prediction models including simplification, insufficient data, overfitting and lack of interpretability, indicate that more work is needed to make clinical outcome prediction more accurate, more reliable, and more practical for clinical use.
Collapse
Affiliation(s)
- Wei Luo
- Department of Radiation Medicine, University of Kentucky, Lexington, KY, United States
| |
Collapse
|
23
|
Schelling X, Fernández J, Ward P, Fernández J, Robertson S. Decision Support System Applications for Scheduling in Professional Team Sport. The Team's Perspective. Front Sports Act Living 2021; 3:678489. [PMID: 34151262 PMCID: PMC8213205 DOI: 10.3389/fspor.2021.678489] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2021] [Accepted: 04/27/2021] [Indexed: 01/17/2023] Open
Abstract
Background: Periodization implies the systematic planning of training and competition with the goal of reaching the best possible performance in the most important competition. In team sports, this consists of finding a flight-and-practice schedule that maximizes the opportunities to perform the periodized contents (e.g., trips, practices, games, and days off). This process is conducted whilst considering known constraints (e.g., competitive schedule, roster availability, weather, especial events, holidays, or emotional effect of days away). The way a scheduling decision support system (DSS) leads users to make a decision should allow for flexibility, whilst minimizing users' confusion and facilitating the understanding of the recommendation given by the scheduling decision support system. Traditional approaches to solving scheduling problems use either simulation models, analytical models, heuristic approaches or a combination of these methods. When it comes to evaluate how the scheduling DSS is performing, three overarching aspects need to be reviewed: context satisfaction, process efficiency, and output quality. Appropriate training periodization and scheduling of trips and training sessions are critical for teams to optimize training and recovery processes in order to maximize health and performance. This article presents a methodological framework for designing decision-support systems for scheduling in professional team sports.
Collapse
Affiliation(s)
- Xavier Schelling
- Institute for Health and Sport, Victoria University, Melbourne, VIC, Australia
| | - Jose Fernández
- Faculty of Health Sciences, School of Behavioral and Health Sciences, Australian Catholic University, Fitzroy, VIC, Australia
| | - Patrick Ward
- Human Performance Research Centre, Faculty of Health, University of Technology Sydney (UTS), Sydney, NSW, Australia
| | - Javier Fernández
- Futbol Club Barcelona, Barcelona, Spain.,Department of Computer Science, Universitat Politecnica de Catalunya, Barcelona, Spain
| | - Sam Robertson
- Institute for Health and Sport, Victoria University, Melbourne, VIC, Australia
| |
Collapse
|
24
|
Weller DL, Love TMT, Wiedmann M. Interpretability Versus Accuracy: A Comparison of Machine Learning Models Built Using Different Algorithms, Performance Measures, and Features to Predict E. coli Levels in Agricultural Water. Front Artif Intell 2021; 4:628441. [PMID: 34056577 PMCID: PMC8160515 DOI: 10.3389/frai.2021.628441] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 02/12/2021] [Indexed: 02/02/2023] Open
Abstract
Since E. coli is considered a fecal indicator in surface water, government water quality standards and industry guidance often rely on E. coli monitoring to identify when there is an increased risk of pathogen contamination of water used for produce production (e.g., for irrigation). However, studies have indicated that E. coli testing can present an economic burden to growers and that time lags between sampling and obtaining results may reduce the utility of these data. Models that predict E. coli levels in agricultural water may provide a mechanism for overcoming these obstacles. Thus, this proof-of-concept study uses previously published datasets to train, test, and compare E. coli predictive models using multiple algorithms and performance measures. Since the collection of different feature data carries specific costs for growers, predictive performance was compared for models built using different feature types [geospatial, water quality, stream traits, and/or weather features]. Model performance was assessed against baseline regression models. Model performance varied considerably with root-mean-squared errors and Kendall's Tau ranging between 0.37 and 1.03, and 0.07 and 0.55, respectively. Overall, models that included turbidity, rain, and temperature outperformed all other models regardless of the algorithm used. Turbidity and weather factors were also found to drive model accuracy even when other feature types were included in the model. These findings confirm previous conclusions that machine learning models may be useful for predicting when, where, and at what level E. coli (and associated hazards) are likely to be present in preharvest agricultural water sources. This study also identifies specific algorithm-predictor combinations that should be the foci of future efforts to develop deployable models (i.e., models that can be used to guide on-farm decision-making and risk mitigation). When deploying E. coli predictive models in the field, it is important to note that past research indicates an inconsistent relationship between E. coli levels and foodborne pathogen presence. Thus, models that predict E. coli levels in agricultural water may be useful for assessing fecal contamination status and ensuring compliance with regulations but should not be used to assess the risk that specific pathogens of concern (e.g., Salmonella, Listeria) are present.
Collapse
Affiliation(s)
- Daniel L. Weller
- Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, United States
- Department of Food Science, Cornell University, Ithaca, NY, United States
- Current Affiliation, Department of Environmental and Forest Biology, SUNY College of Environmental Science and Forestry, Syracuse, NY, United States
| | - Tanzy M. T. Love
- Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY, United States
| | - Martin Wiedmann
- Department of Food Science, Cornell University, Ithaca, NY, United States
| |
Collapse
|
25
|
Coates JTT, Pirovano G, El Naqa I. Radiomic and radiogenomic modeling for radiotherapy: strategies, pitfalls, and challenges. J Med Imaging (Bellingham) 2021; 8:031902. [PMID: 33768134 PMCID: PMC7985651 DOI: 10.1117/1.jmi.8.3.031902] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2020] [Accepted: 01/12/2021] [Indexed: 12/14/2022] Open
Abstract
The power of predictive modeling for radiotherapy outcomes has historically been limited by an inability to adequately capture patient-specific variabilities; however, next-generation platforms together with imaging technologies and powerful bioinformatic tools have facilitated strategies and provided optimism. Integrating clinical, biological, imaging, and treatment-specific data for more accurate prediction of tumor control probabilities or risk of radiation-induced side effects are high-dimensional problems whose solutions could have widespread benefits to a diverse patient population-we discuss technical approaches toward this objective. Increasing interest in the above is specifically reflected by the emergence of two nascent fields, which are distinct but complementary: radiogenomics, which broadly seeks to integrate biological risk factors together with treatment and diagnostic information to generate individualized patient risk profiles, and radiomics, which further leverages large-scale imaging correlates and extracted features for the same purpose. We review classical analytical and data-driven approaches for outcomes prediction that serve as antecedents to both radiomic and radiogenomic strategies. Discussion then focuses on uses of conventional and deep machine learning in radiomics. We further consider promising strategies for the harmonization of high-dimensional, heterogeneous multiomics datasets (panomics) and techniques for nonparametric validation of best-fit models. Strategies to overcome common pitfalls that are unique to data-intensive radiomics are also discussed.
Collapse
Affiliation(s)
- James T. T. Coates
- Massachusetts General Hospital & Harvard Medical School, Center for Cancer Research, Boston, Massachusetts, United States
| | - Giacomo Pirovano
- Memorial Sloan Kettering Cancer Center, Department of Radiology, New York, New York, United States
| | - Issam El Naqa
- Moffitt Cancer Center and Research Institute, Department of Machine Learning, Tampa, Florida, United States
| |
Collapse
|
26
|
Ip WY, Yeung FK, Yung SPF, Yu HCJ, So TH, Vardhanabhuti V. Current landscape and potential future applications of artificial intelligence in medical physics and radiotherapy. Artif Intell Med Imaging 2021; 2:37-55. [DOI: 10.35711/aimi.v2.i2.37] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/22/2021] [Revised: 04/01/2021] [Accepted: 04/20/2021] [Indexed: 02/06/2023] Open
Abstract
Artificial intelligence (AI) has seen tremendous growth over the past decade and stands to disrupts the medical industry. In medicine, this has been applied in medical imaging and other digitised medical disciplines, but in more traditional fields like medical physics, the adoption of AI is still at an early stage. Though AI is anticipated to be better than human in certain tasks, with the rapid growth of AI, there is increasing concerns for its usage. The focus of this paper is on the current landscape and potential future applications of artificial intelligence in medical physics and radiotherapy. Topics on AI for image acquisition, image segmentation, treatment delivery, quality assurance and outcome prediction will be explored as well as the interaction between human and AI. This will give insights into how we should approach and use the technology for enhancing the quality of clinical practice.
Collapse
Affiliation(s)
- Wing-Yan Ip
- Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| | - Fu-Ki Yeung
- Medical Physics and Research Department, The Hong Kong Sanitorium & Hospital, Hong Kong SAR, China and Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| | - Shang-Peng Felix Yung
- Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| | | | - Tsz-Him So
- Department of Clinical Oncology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| | - Varut Vardhanabhuti
- Department of Diagnostic Radiology, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
| |
Collapse
|
27
|
Boyle R, Jollans L, Rueda-Delgado LM, Rizzo R, Yener GG, McMorrow JP, Knight SP, Carey D, Robertson IH, Emek-Savaş DD, Stern Y, Kenny RA, Whelan R. Brain-predicted age difference score is related to specific cognitive functions: a multi-site replication analysis. Brain Imaging Behav 2021; 15:327-345. [PMID: 32141032 DOI: 10.1007/s11682-020-00260-3] [Citation(s) in RCA: 41] [Impact Index Per Article: 13.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
Brain-predicted age difference scores are calculated by subtracting chronological age from 'brain' age, which is estimated using neuroimaging data. Positive scores reflect accelerated ageing and are associated with increased mortality risk and poorer physical function. To date, however, the relationship between brain-predicted age difference scores and specific cognitive functions has not been systematically examined using appropriate statistical methods. First, applying machine learning to 1359 T1-weighted MRI scans, we predicted the relationship between chronological age and voxel-wise grey matter data. This model was then applied to MRI data from three independent datasets, significantly predicting chronological age in each dataset: Dokuz Eylül University (n = 175), the Cognitive Reserve/Reference Ability Neural Network study (n = 380), and The Irish Longitudinal Study on Ageing (n = 487). Each independent dataset had rich neuropsychological data. Brain-predicted age difference scores were significantly negatively correlated with performance on measures of general cognitive status (two datasets); processing speed, visual attention, and cognitive flexibility (three datasets); visual attention and cognitive flexibility (two datasets); and semantic verbal fluency (two datasets). As such, there is firm evidence of correlations between increased brain-predicted age differences and reduced cognitive function in some domains that are implicated in cognitive ageing.
Collapse
Affiliation(s)
- Rory Boyle
- Trinity College Institute of Neuroscience, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland
| | - Lee Jollans
- Department of Translational Research in Psychiatry, Max Planck Institute of Psychiatry, München, Germany
| | - Laura M Rueda-Delgado
- Trinity College Institute of Neuroscience, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland
| | - Rossella Rizzo
- Physics Department, University of Calabria, Rende, CS, Italy
| | - Görsev G Yener
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, İzmir, Turkey
- Department of Neurology, Dokuz Eylul University Medical School, İzmir, Turkey
- Brain Dynamics Multidisciplinary Research Center, Dokuz Eylul University, İzmir, Turkey
| | - Jason P McMorrow
- Centre for Advanced Medical Imaging, St. James's Hospital, Dublin 8, Ireland
- School of Medicine, Trinity College Dublin, Dublin 2, Ireland
| | - Silvin P Knight
- School of Medicine, Trinity College Dublin, Dublin 2, Ireland
- The Irish Longitudinal Study on Ageing, Trinity College Dublin, Dublin 2, Ireland
| | - Daniel Carey
- The Irish Longitudinal Study on Ageing, Trinity College Dublin, Dublin 2, Ireland
- Department of Medical Gerontology, Trinity College Dublin, Dublin 2, Ireland
| | - Ian H Robertson
- Trinity College Institute of Neuroscience, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland
- Global Brain Health Institute, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland
| | - Derya D Emek-Savaş
- Department of Neurosciences, Institute of Health Sciences, Dokuz Eylul University, İzmir, Turkey
- Global Brain Health Institute, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland
- Department of Psychology, Faculty of Letters, Dokuz Eylul University, İzmir, Turkey
| | - Yaakov Stern
- Cognitive Neuroscience Division, Department of Neurology, Columbia University, New York, NY, USA
| | - Rose Anne Kenny
- School of Medicine, Trinity College Dublin, Dublin 2, Ireland
- The Irish Longitudinal Study on Ageing, Trinity College Dublin, Dublin 2, Ireland
- Mercer's Institute for Successful Ageing, St. James's Hospital, Dublin 8, Ireland
| | - Robert Whelan
- Trinity College Institute of Neuroscience, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland.
- Global Brain Health Institute, Trinity College Dublin, Lloyd Building, Dublin 2, Ireland.
| |
Collapse
|
28
|
Zaidi H, El Naqa I. Quantitative Molecular Positron Emission Tomography Imaging Using Advanced Deep Learning Techniques. Annu Rev Biomed Eng 2021; 23:249-276. [PMID: 33797938 DOI: 10.1146/annurev-bioeng-082420-020343] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The widespread availability of high-performance computing and the popularity of artificial intelligence (AI) with machine learning and deep learning (ML/DL) algorithms at the helm have stimulated the development of many applications involving the use of AI-based techniques in molecular imaging research. Applications reported in the literature encompass various areas, including innovative design concepts in positron emission tomography (PET) instrumentation, quantitative image reconstruction and analysis techniques, computer-aided detection and diagnosis, as well as modeling and prediction of outcomes. This review reflects the tremendous interest in quantitative molecular imaging using ML/DL techniques during the past decade, ranging from the basic principles of ML/DL techniques to the various steps required for obtaining quantitatively accurate PET data, including algorithms used to denoise or correct for physical degrading factors as well as to quantify tracer uptake and metabolic tumor volume for treatment monitoring or radiation therapy treatment planning and response prediction.This review also addresses future opportunities and current challenges facing the adoption of ML/DL approaches and their role in multimodality imaging.
Collapse
Affiliation(s)
- Habib Zaidi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, 1211 Geneva, Switzerland; .,Geneva Neuroscience Centre, University of Geneva, 1205 Geneva, Switzerland.,Department of Nuclear Medicine and Molecular Imaging, University of Groningen, 9700 RB Groningen, Netherlands.,Department of Nuclear Medicine, University of Southern Denmark, DK-5000 Odense, Denmark
| | - Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, Florida 33612, USA.,Department of Radiation Oncology, University of Michigan, Ann Arbor, Michigan 48109, USA.,Department of Oncology, McGill University, Montreal, Quebec H3A 1G5, Canada
| |
Collapse
|
29
|
Balagurunathan Y, Mitchell R, El Naqa I. Requirements and reliability of AI in the medical context. Phys Med 2021; 83:72-78. [PMID: 33721700 PMCID: PMC8915137 DOI: 10.1016/j.ejmp.2021.02.024] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 02/04/2021] [Accepted: 02/23/2021] [Indexed: 12/12/2022] Open
Abstract
The digital information age has been a catalyst in creating a renewed interest in Artificial Intelligence (AI) approaches, especially the subclass of computer algorithms that are popularly grouped into Machine Learning (ML). These methods have allowed one to go beyond limited human cognitive ability into understanding the complexity in the high dimensional data. Medical sciences have seen a steady use of these methods but have been slow in adoption to improve patient care. There are some significant impediments that have diluted this effort, which include availability of curated diverse data sets for model building, reliable human-level interpretation of these models, and reliable reproducibility of these methods for routine clinical use. Each of these aspects has several limiting conditions that need to be balanced out, considering the data/model building efforts, clinical implementation, integration cost to translational effort with minimal patient level harm, which may directly impact future clinical adoption. In this review paper, we will assess each aspect of the problem in the context of reliable use of the ML methods in oncology, as a representative study case, with the goal to safeguard utility and improve patient care in medicine in general.
Collapse
Affiliation(s)
| | - Ross Mitchell
- Department of Machine Learning, H. Lee. Moffitt Cancer Center, Tampa, FL, USA; Health Data Services, H. Lee. Moffitt Cancer Center, Tampa, FL, USA.
| | - Issam El Naqa
- Department of Machine Learning, H. Lee. Moffitt Cancer Center, Tampa, FL, USA.
| |
Collapse
|
30
|
Barragán-Montero A, Javaid U, Valdés G, Nguyen D, Desbordes P, Macq B, Willems S, Vandewinckele L, Holmström M, Löfman F, Michiels S, Souris K, Sterpin E, Lee JA. Artificial intelligence and machine learning for medical imaging: A technology review. Phys Med 2021; 83:242-256. [PMID: 33979715 PMCID: PMC8184621 DOI: 10.1016/j.ejmp.2021.04.016] [Citation(s) in RCA: 90] [Impact Index Per Article: 30.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/06/2020] [Revised: 04/15/2021] [Accepted: 04/18/2021] [Indexed: 02/08/2023] Open
Abstract
Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.
Collapse
Affiliation(s)
- Ana Barragán-Montero
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium.
| | - Umair Javaid
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium
| | - Gilmer Valdés
- Department of Radiation Oncology, Department of Epidemiology and Biostatistics, University of California, San Francisco, USA
| | - Dan Nguyen
- Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, UT Southwestern Medical Center, USA
| | - Paul Desbordes
- Information and Communication Technologies, Electronics and Applied Mathematics (ICTEAM), UCLouvain, Belgium
| | - Benoit Macq
- Information and Communication Technologies, Electronics and Applied Mathematics (ICTEAM), UCLouvain, Belgium
| | - Siri Willems
- ESAT/PSI, KU Leuven Belgium & MIRC, UZ Leuven, Belgium
| | | | | | | | - Steven Michiels
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium
| | - Kevin Souris
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium
| | - Edmond Sterpin
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium; KU Leuven, Department of Oncology, Laboratory of Experimental Radiotherapy, Belgium
| | - John A Lee
- Molecular Imaging, Radiation and Oncology (MIRO) Laboratory, UCLouvain, Belgium
| |
Collapse
|
31
|
Kazmierska J, Hope A, Spezi E, Beddar S, Nailon WH, Osong B, Ankolekar A, Choudhury A, Dekker A, Redalen KR, Traverso A. From multisource data to clinical decision aids in radiation oncology: The need for a clinical data science community. Radiother Oncol 2020; 153:43-54. [PMID: 33065188 DOI: 10.1016/j.radonc.2020.09.054] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2020] [Revised: 09/25/2020] [Accepted: 09/26/2020] [Indexed: 12/22/2022]
Abstract
Big data are no longer an obstacle; now, by using artificial intelligence (AI), previously undiscovered knowledge can be found in massive data collections. The radiation oncology clinic daily produces a large amount of multisource data and metadata during its routine clinical and research activities. These data involve multiple stakeholders and users. Because of a lack of interoperability, most of these data remain unused, and powerful insights that could improve patient care are lost. Changing the paradigm by introducing powerful AI analytics and a common vision for empowering big data in radiation oncology is imperative. However, this can only be achieved by creating a clinical data science community in radiation oncology. In this work, we present why such a community is needed to translate multisource data into clinical decision aids.
Collapse
Affiliation(s)
- Joanna Kazmierska
- Radiotherapy Department II, Greater Poland Cancer Centre, Poznan, Poland; Electroradiology Department, University of Medical Sciences, Poznan, Poland
| | - Andrew Hope
- Princess Margaret Cancer Centre, Toronto, Canada
| | - Emiliano Spezi
- School of Engineering, Cardiff University, United Kingdom; Department of Medical Physics, Velindre Cancer Centre, Cardiff, United Kingdom
| | - Sam Beddar
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, United States; The UTHealth Graduate School of Biomedical Sciences, Houston, United States
| | - William H Nailon
- Department of Oncology Physics, University of Edinburgh, United Kingdom
| | - Biche Osong
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, The Netherlands
| | - Anshu Ankolekar
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, The Netherlands
| | - Ananya Choudhury
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, The Netherlands
| | - Andre Dekker
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, The Netherlands
| | - Kathrine Røe Redalen
- Department of Physics, Norwegian University of Science and Technology, Trondheim, Norway
| | - Alberto Traverso
- Department of Radiation Oncology (Maastro), GROW School for Oncology, Maastricht University Medical Centre+, The Netherlands.
| |
Collapse
|
32
|
Cui S, Tseng HH, Pakela J, Ten Haken RK, Naqa IE. Introduction to machine and deep learning for medical physicists. Med Phys 2020; 47:e127-e147. [PMID: 32418339 PMCID: PMC7331753 DOI: 10.1002/mp.14140] [Citation(s) in RCA: 52] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2019] [Revised: 01/23/2020] [Accepted: 03/03/2020] [Indexed: 01/01/2023] Open
Abstract
Recent years have witnessed tremendous growth in the application of machine learning (ML) and deep learning (DL) techniques in medical physics. Embracing the current big data era, medical physicists equipped with these state-of-the-art tools should be able to solve pressing problems in modern radiation oncology. Here, a review of the basic aspects involved in ML/DL model building, including data processing, model training, and validation for medical physics applications is presented and discussed. Machine learning can be categorized based on the underlying task into supervised learning, unsupervised learning, or reinforcement learning; each of these categories has its own input/output dataset characteristics and aims to solve different classes of problems in medical physics ranging from automation of processes to predictive analytics. It is recognized that data size requirements may vary depending on the specific medical physics application and the nature of the algorithms applied. Data processing, which is a crucial step for model stability and precision, should be performed before training the model. Deep learning as a subset of ML is able to learn multilevel representations from raw input data, eliminating the necessity for hand crafted features in classical ML. It can be thought of as an extension of the classical linear models but with multilayer (deep) structures and nonlinear activation functions. The logic of going "deeper" is related to learning complex data structures and its realization has been aided by recent advancements in parallel computing architectures and the development of more robust optimization methods for efficient training of these algorithms. Model validation is an essential part of ML/DL model building. Without it, the model being developed cannot be easily trusted to generalize to unseen data. Whenever applying ML/DL, one should keep in mind, according to Amara's law, that humans may tend to overestimate the ability of a technology in the short term and underestimate its capability in the long term. To establish ML/DL role into standard clinical workflow, models considering balance between accuracy and interpretability should be developed. Machine learning/DL algorithms have potential in numerous radiation oncology applications, including automatizing mundane procedures, improving efficiency and safety of auto-contouring, treatment planning, quality assurance, motion management, and outcome predictions. Medical physicists have been at the frontiers of technology translation into medicine and they ought to be prepared to embrace the inevitable role of ML/DL in the practice of radiation oncology and lead its clinical implementation.
Collapse
Affiliation(s)
- Sunan Cui
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, USA; Applied Physics Program, University of Michigan, Ann Arbor, MI 48109, USA
| | - Huan-Hsin Tseng
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, USA
| | - Julia Pakela
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, USA; Applied Physics Program, University of Michigan, Ann Arbor, MI 48109, USA
| | - Randall K. Ten Haken
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, USA
| | - Issam El Naqa
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI 48103, USA
| |
Collapse
|
33
|
El Naqa I, Haider MA, Giger ML, Ten Haken RK. Artificial Intelligence: reshaping the practice of radiological sciences in the 21st century. Br J Radiol 2020; 93:20190855. [PMID: 31965813 PMCID: PMC7055429 DOI: 10.1259/bjr.20190855] [Citation(s) in RCA: 46] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2019] [Revised: 01/12/2020] [Accepted: 01/13/2020] [Indexed: 12/15/2022] Open
Abstract
Advances in computing hardware and software platforms have led to the recent resurgence in artificial intelligence (AI) touching almost every aspect of our daily lives by its capability for automating complex tasks or providing superior predictive analytics. AI applications are currently spanning many diverse fields from economics to entertainment, to manufacturing, as well as medicine. Since modern AI's inception decades ago, practitioners in radiological sciences have been pioneering its development and implementation in medicine, particularly in areas related to diagnostic imaging and therapy. In this anniversary article, we embark on a journey to reflect on the learned lessons from past AI's chequered history. We further summarize the current status of AI in radiological sciences, highlighting, with examples, its impressive achievements and effect on re-shaping the practice of medical imaging and radiotherapy in the areas of computer-aided detection, diagnosis, prognosis, and decision support. Moving beyond the commercial hype of AI into reality, we discuss the current challenges to overcome, for AI to achieve its promised hope of providing better precision healthcare for each patient while reducing cost burden on their families and the society at large.
Collapse
Affiliation(s)
- Issam El Naqa
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, USA
| | - Masoom A Haider
- Department of Medical Imaging and Lunenfeld-Tanenbaum Research Institute, University of Toronto, Toronto, ON, Canada
| | | | - Randall K Ten Haken
- Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, USA
| |
Collapse
|