Machine Learning and Imaging Informatics in Oncology

Construction of a diagnostic model for hepatitis B-related hepatocellular carcinoma using machine learning and artificial neural networks and revealing the correlation by immunoassay

HBV infection profoundly escalates hepatocellular carcinoma (HCC) susceptibility, responsible for a majority of HCC cases. HBV-driven immune-mediated hepatocyte impairment significantly fuels HCC progression. Regrettably, inconspicuous early HCC symptoms often culminate in belated diagnoses. Nevertheless, surgically treated early-stage HCC patients relish augmented five-year survival rates. In contrast, advanced HCC exhibits feeble responses to conventional interventions like radiotherapy, chemotherapy, and surgery, leading to diminished survival rates. This investigation endeavors to unearth diagnostic hallmark genes for HBV-HCC leveraging a bioinformatics framework, thus refining early HBV-HCC detection. Candidate genes were sieved via differential analysis and Weighted Gene Co-Expression Network Analysis (WGCNA). Employing three distinct machine learning algorithms unearthed three feature genes (HHIP, CXCL14, and CDHR2). Melding these genes yielded an innovative Artificial Neural Network (ANN) diagnostic blueprint, portending to alleviate patient encumbrance and elevate life quality. Immunoassay scrutiny unveiled accentuated immune damage in HBV-HCC patients relative to solitary HCC. Through consensus clustering, HBV-HCC was stratified into two subtypes (C1 and C2), the latter potentially indicating milder immune impairment. The diagnostic model grounded in these feature genes showcased robust and transferrable prognostic potentialities, introducing a novel outlook for early HBV-HCC diagnosis. This exhaustive immunological odyssey stands poised to expedite immunotherapeutic curatives' emergence for HBV-HCC.

Leveraging the Academic Artificial Intelligence Silecosystem to Advance the Community Oncology Enterprise

Over the last 75 years, artificial intelligence has evolved from a theoretical concept and novel paradigm describing the role that computers might play in our society to a tool with which we daily engage. In this review, we describe AI in terms of its constituent elements, the synthesis of which we refer to as the AI Silecosystem. Herein, we provide an historical perspective of the evolution of the AI Silecosystem, conceptualized and summarized as a Kuhnian paradigm. This manuscript focuses on the role that the AI Silecosystem plays in oncology and its emerging importance in the care of the community oncology patient. We observe that this important role arises out of a unique alliance between the academic oncology enterprise and community oncology practices. We provide evidence of this alliance by illustrating the practical establishment of the AI Silecosystem at the City of Hope Comprehensive Cancer Center and its team utilization by community oncology providers.

Artificial Intelligence in Evaluation of Permanent Impairment: New Operational Frontiers

Artificial intelligence (AI) and machine learning (ML) span multiple disciplines, including the medico-legal sciences, also with reference to the concept of disease and disability. In this context, the International Classification of Diseases, Injuries, and Causes of Death (ICD) is a standard for the classification of diseases and related problems developed by the World Health Organization (WHO), and it represents a valid tool for statistical and epidemiological studies. Indeed, the International Classification of Functioning, Disability, and Health (ICF) is outlined as a classification that aims to describe the state of health of people in relation to their existential spheres (social, family, work). This paper lays the foundations for proposing an operating model for the use of AI in the assessment of impairments with the aim of making the information system as homogeneous as possible, starting from the main coding systems of the reference pathologies and functional damages. Providing a scientific basis for the understanding and study of health, as well as establishing a common language for the assessment of disability in its various meanings through AI systems, will allow for the improvement and standardization of communication between the various expert users.

Noise-robustness test for ultrasound breast nodule neural network models as medical devices

Background

Deep learning technology has been widely applied to medical image analysis. But due to the limitations of its own imaging principle, ultrasound image has the disadvantages of low resolution and high Speckle Noise density, which not only hinder the diagnosis of patients' conditions but also affect the extraction of ultrasound image features by computer technology.

Objective

In this study, we investigate the robustness of deep convolutional neural network (CNN) for classification, segmentation, and target detection of breast ultrasound image through random Salt & Pepper Noise and Gaussian Noise.

Methods

We trained and validated 9 CNN architectures in 8617 breast ultrasound images, but tested the models with noisy test set. Then, we trained and validated 9 CNN architectures with different levels of noise in these breast ultrasound images, and tested the models with noisy test set. Diseases of each breast ultrasound image in our dataset were annotated and voted by three sonographers based on their malignancy suspiciousness. we use evaluation indexes to evaluate the robustness of the neural network algorithm respectively.

Results

There is a moderate to high impact (The accuracy of the model decreased by about 5%-40%) on model accuracy when Salt and Pepper Noise, Speckle Noise, or Gaussian Noise is introduced to the images respectively. Consequently, DenseNet, UNet++ and Yolov5 were selected as the most robust model based on the selected index. When any two of these three kinds of noise are introduced into the image at the same time, the accuracy of the model will be greatly affected.

Conclusions

Our experimental results reveal new insights: The variation trend of accuracy with the noise level in Each network used for classification tasks and object detection tasks has some unique characteristics. This finding provides us with a method to reveal the black-box architecture of computer-aided diagnosis (CAD) systems. On the other hand, the purpose of this study is to explore the impact of adding noise directly to the image on the performance of neural networks, which is different from the existing articles on robustness in the field of medical image processing. Consequently, it provides a new way to evaluate the robustness of CAD systems in the future.

Prospective Evaluation of Quantitative F-18-FDG-PET/CT for Pre-Operative Thoracic Lymph Node Staging in Patients with Lung Cancer as a Target for Computer-Aided Diagnosis

Purpose: Pre-operative assessment of thoracic lymphonodal (LN) involvement in patients with lung cancer (LC) is crucial when choosing the treatment modality. Visual assessment of F-18-FDG-PET/CT (PET/CT) is well established, however, there is still a need for prospective quantitative data to differentiate benign from malignant lesions which would simplify staging and guide the further implementation of computer-aided diagnosis (CAD). Methods: In this prospective study, 37 patients with confirmed lung cancer (m/f = 24/13; age: 70 [52–83] years) were analyzed. All patients underwent PET/CT and quantitative data (standardized uptake values) were obtained. Histological results were available for 101 thoracic lymph nodes. Quantitative data were matched to determine cut-off values for delineation between benign vs. malignant lymph nodes. Furthermore, a scoring system derived from these cut-off values was established. Statistical analyses were performed through ROC analysis. Results: Quantitative analysis revealed the optimal cut-off values (p < 0.01) for the differentiation between benign and malignant thoracic lymph nodes in patients suffering from lung cancer. The respective areas under the curve (AUC) ranged from 0.86 to 0.94. The highest AUC for a ratio of lymph node to healthy lung tissue was 0.94. The resulting accuracy ranged from 78.2% to 89.1%. A dedicated scoring system led to an AUC of 0.93 with a negative predictive value of 95.4%. Conclusion: Quantitative analysis of F-18-FDG-PET/CT data provides reliable results for delineation between benign and malignant thoracic lymph nodes. Thus, quantitative parameters can improve diagnostic accuracy and reliability and can also facilitate the handling of the steadily increasing number of clinical examinations.

Imaging in Healthcare: A Glance at the Present and a Glimpse Into the Future

The utilization of artificial intelligence (AI) applications in medical imaging relies heavily on imaging informatics. That is a one-of-a-kind professional who works at the crossroads of clinical radiography, data science, and information technology. Imaging informaticians are becoming crucial players in expanding, assessing, and implementing AI in the medical setting. Teleradiology will continue to be a cost-effective healthcare facility that expands. Vendor neutral archive (VNA) isolates image presentation and storing systems, permitting platforms to develop quickly, and is a repository for organization-wide healthcare image data. Efforts are made to incorporate and integrate diagnostic facilities such as radiography and pathology to fulfill the needs and demands of targeted therapy. Developments in computer-aided medical object identification may alter the environment of patient services. Finally, interpreting and processing distinct complex healthcare data will create a data-rich context where evidence-based care and performance development may be driven.

Multisystem Imaging Recommendations/Guidelines: In the Pursuit of Precision Oncology

With an increasing rate of cancers in almost all age groups and advanced screening techniques leading to an early diagnosis and longer longevity of patients with cancers, it is of utmost importance that radiologists assigned with cancer imaging should be prepared to deal with specific expected and unexpected circumstances that may arise during the lifetime of these patients. Tailored integration of preventive and curative interventions with current health plans and global escalation of efforts for timely diagnosis of cancers will pave the path for a cancer-free world. The commonly encountered circumstances in the current era, complicating cancer imaging, include coronavirus disease 2019 infection, pregnancy and lactation, immunocompromised states, bone marrow transplant, and screening of cancers in the relevant population. In this article, we discuss the imaging recommendations pertaining to cancer screening and diagnosis in the aforementioned clinical circumstances.

Prediction of the response to photodynamic therapy in patients with chronic central serous chorioretinopathy based on optical coherence tomography using deep learning

PURPOSE

To assess the prediction of the response to photodynamic therapy (PDT) in chronic central serous chorioretinopathy (CSCR) based on spectral-domain optical coherence tomography (SD-OCT) images using deep learning (DL).

METHODS

Retrospective study including 216 eyes of 175 patients with CSCR and persistent subretinal fluid (SRF) who underwent half-fluence PDT. SD-OCT macular examination was performed before (baseline) and 3 months after treatment. Patients were classified into groups by experts based on the response to PDT: Group 1, complete SRF resorption (n = 100); Group 2, partial SRF resorption (n = 66); and Group 3, absence of any SRF resorption (n = 50). This work proposes different computational approaches: 1st approach compares all groups; 2nd compares groups 1 vs. 2 and 3 together; 3rd compares groups 2 vs. 3.

RESULTS

The mean age was 55.6 ± 10.9 years and 70.3% were males. In the first approach, the algorithm showed a precision of up to 57% to detect the response to treatment in group 1 based on the initial scan, with a mean average accuracy of 0.529 ± 0.035. In the second model, the mean accuracy was higher (0.670 ± 0.046). In the third approach, the algorithm showed a precision of 0.74 ± 0.12 to detect the response to treatment in group 2 (partial SRF resolution) and 0.69 ± 0.15 in group 3 (absence of SRF resolution).

CONCLUSION

Despite the high clinical variability in the response of chronic CSCR to PDT, this DL algorithm offers an objective and promising tool to predict the response to PDT treatment in clinical practice.

Identification of metastatic primary cutaneous squamous cell carcinoma utilizing artificial intelligence analysis of whole slide images

Cutaneous squamous cell carcinoma (cSCC) harbors metastatic potential and causes mortality. However, clinical assessment of metastasis risk is challenging. We approached this challenge by harnessing artificial intelligence (AI) algorithm to identify metastatic primary cSCCs. Residual neural network-architectures were trained with cross-validation to identify metastatic tumors on clinician annotated, hematoxylin and eosin-stained whole slide images representing primary non-metastatic and metastatic cSCCs (n = 104). Metastatic primary tumors were divided into two subgroups, which metastasize rapidly (≤ 180 days) (n = 22) or slowly (> 180 days) (n = 23) after primary tumor detection. Final model was able to predict whether primary tumor was non-metastatic or rapidly metastatic with slide-level area under the receiver operating characteristic curve (AUROC) of 0.747. Furthermore, risk factor (RF) model including prediction by AI, Clark’s level and tumor diameter provided higher AUROC (0.917) than other RF models and predicted high 5-year disease specific survival (DSS) for patients with cSCC with 0 or 1 RFs (100% and 95.7%) and poor DSS for patients with cSCCs with 2 or 3 RFs (41.7% and 40.0%). These results indicate, that AI recognizes unknown morphological features associated with metastasis and may provide added value to clinical assessment of metastasis risk and prognosis of primary cSCC.

Extending the relative seriality formalism for interpretable deep learning of normal tissue complication probability models

We formally demonstrate that the relative seriality (RS) model of normal tissue complication probability (NTCP) can be recast as a simple neural network with one convolutional and one pooling layer. This approach enables us to systematically construct deep relative seriality networks (DRSNs), a new class of mechanistic generalizations of the RS model with radiobiologically interpretable parameters amenable to deep learning. To demonstrate the utility of this formulation, we analyze a simplified example of xerostomia due to irradiation of the parotid gland during alpha radiopharmaceutical therapy. Using a combination of analytical calculations and numerical simulations, we show for both the RS and DRSN cases that the ability of the neural network to generalize without overfitting is tied to ‘stiff’ and ‘sloppy’ directions in the parameter space of the mechanistic model. These results serve as proof-of-concept for radiobiologically interpretable deep learning of NTCP, while simultaneously yielding insight into how such techniques can robustly generalize beyond the training set despite uncertainty in individual parameters.

High-dimensional role of AI and machine learning in cancer research

The role of Artificial Intelligence and Machine Learning in cancer research offers several advantages, primarily scaling up the information processing and increasing the accuracy of the clinical decision-making. The key enabling tools currently in use in Precision, Digital and Translational Medicine, here named as 'Intelligent Systems' (IS), leverage unprecedented data volumes and aim to model their underlying heterogeneous influences and variables correlated with patients' outcomes. As functionality and performance of IS are associated with complex diagnosis and therapy decisions, a rich spectrum of patterns and features detected in high-dimensional data may be critical for inference purposes. Many challenges are also present in such discovery task. First, the generation of interpretable model results from a mix of structured and unstructured input information. Second, the design, and implementation of automated clinical decision processes for drawing disease trajectories and patient profiles. Ultimately, the clinical impacts depend on the data effectively subjected to steps such as harmonisation, integration, validation, etc. The aim of this work is to discuss the transformative value of IS applied to multimodal data acquired through various interrelated cancer domains (high-throughput genomics, experimental biology, medical image processing, radiomics, patient electronic records, etc.).

Bidimensional and Tridimensional Poincaré Maps in Cardiology: A Multiclass Machine Learning Study

Heart rate is a nonstationary signal and its variation may contain indicators of current disease or warnings about impending cardiac diseases. Hence, heart rate variation analysis has become a noninvasive tool to further study the activities of the autonomic nervous system. In this scenario, the Poincaré plot analysis has proven to be a valuable tool to support cardiac diseases diagnosis. The study’s aim is a preliminary exploration of the feasibility of machine learning to classify subjects belonging to five cardiac states (healthy, hypertension, myocardial infarction, congestive heart failure and heart transplanted) using ten unconventional quantitative parameters extracted from bidimensional and three-dimensional Poincaré maps. Knime Analytic Platform was used to implement several machine learning algorithms: Gradient Boosting, Adaptive Boosting, k-Nearest Neighbor and Naïve Bayes. Accuracy, sensitivity and specificity were computed to assess the performances of the predictive models using the leave-one-out cross-validation. The Synthetic Minority Oversampling technique was previously performed for data augmentation considering the small size of the dataset and the number of features. A feature importance, ranked on the basis of the Information Gain values, was computed. Preliminarily, a univariate statistical analysis was performed through one-way Kruskal Wallis plus post-hoc for all the features. Machine learning analysis achieved interesting results in terms of evaluation metrics, such as demonstrated by Adaptive Boosting and k-Nearest Neighbor (accuracies greater than 90%). Gradient Boosting and k-Nearest Neighbor reached even 100% score in sensitivity and specificity, respectively. The most important features according to information gain are in line with the results obtained from the statistical analysis confirming their predictive power. The study shows the proposed combination of unconventional features extracted from Poincaré maps and well-known machine learning algorithms represents a valuable approach to automatically classify patients with different cardiac diseases. Future investigations on enriched datasets will further confirm the potential application of this methodology in diagnostic.

Improving the Efficacy of Deep-Learning Models for Heart Beat Detection on Heterogeneous Datasets

Deep learning (DL) has greatly contributed to bioelectric signal processing, in particular to extract physiological markers. However, the efficacy and applicability of the results proposed in the literature is often constrained to the population represented by the data used to train the models. In this study, we investigate the issues related to applying a DL model on heterogeneous datasets. In particular, by focusing on heart beat detection from electrocardiogram signals (ECG), we show that the performance of a model trained on data from healthy subjects decreases when applied to patients with cardiac conditions and to signals collected with different devices. We then evaluate the use of transfer learning (TL) to adapt the model to the different datasets. In particular, we show that the classification performance is improved, even with datasets with a small sample size. These results suggest that a greater effort should be made towards the generalizability of DL models applied on bioelectric signals, in particular, by retrieving more representative datasets.

Artificial Intelligence in Thyroid Field-A Comprehensive Review

Simple Summary

The incidence of thyroid pathologies has been increasing worldwide. Historically, the detection of thyroid neoplasms relies on medical imaging analysis, depending mainly on the experience of clinicians. The advent of artificial intelligence (AI) techniques led to a remarkable progress in image-recognition tasks. AI represents a powerful tool that may facilitate understanding of thyroid pathologies, but actually, the diagnostic accuracy is uncertain. This article aims to provide an overview of the basic aspects, limitations and open issues of the AI methods applied to thyroid images. Medical experts should be familiar with the workflow of AI techniques in order to avoid misleading outcomes.

Abstract

Artificial intelligence (AI) uses mathematical algorithms to perform tasks that require human cognitive abilities. AI-based methodologies, e.g., machine learning and deep learning, as well as the recently developed research field of radiomics have noticeable potential to transform medical diagnostics. AI-based techniques applied to medical imaging allow to detect biological abnormalities, to diagnostic neoplasms or to predict the response to treatment. Nonetheless, the diagnostic accuracy of these methods is still a matter of debate. In this article, we first illustrate the key concepts and workflow characteristics of machine learning, deep learning and radiomics. We outline considerations regarding data input requirements, differences among these methodologies and their limitations. Subsequently, a concise overview is presented regarding the application of AI methods to the evaluation of thyroid images. We developed a critical discussion concerning limits and open challenges that should be addressed before the translation of AI techniques to the broad clinical use. Clarification of the pitfalls of AI-based techniques results crucial in order to ensure the optimal application for each patient.

Radiomics and machine learning applications in rectal cancer: Current update and future perspectives

The high incidence of rectal cancer in both sexes makes it one of the most common tumors, with significant morbidity and mortality rates. To define the best treatment option and optimize patient outcome, several rectal cancer biological variables must be evaluated. Currently, medical imaging plays a crucial role in the characterization of this disease, and it often requires a multimodal approach. Magnetic resonance imaging is the first-choice imaging modality for local staging and restaging and can be used to detect high-risk prognostic factors. Computed tomography is widely adopted for the detection of distant metastases. However, conventional imaging has recognized limitations, and many rectal cancer characteristics remain assessable only after surgery and histopathology evaluation. There is a growing interest in artificial intelligence applications in medicine, and imaging is by no means an exception. The introduction of radiomics, which allows the extraction of quantitative features that reflect tumor heterogeneity, allows the mining of data in medical images and paved the way for the identification of potential new imaging biomarkers. To manage such a huge amount of data, the use of machine learning algorithms has been proposed. Indeed, without prior explicit programming, they can be employed to build prediction models to support clinical decision making. In this review, current applications and future perspectives of artificial intelligence in medical imaging of rectal cancer are presented, with an imaging modality-based approach and a keen eye on unsolved issues. The results are promising, but the road ahead for translation in clinical practice is rather long.

Machine learning applications in radiation oncology

Machine learning technology has a growing impact on radiation oncology with an increasing presence in research and industry. The prevalence of diverse data including 3D imaging and the 3D radiation dose delivery presents potential for future automation and scope for treatment improvements for cancer patients. Harnessing this potential requires standardization of tools and data, and focused collaboration between fields of expertise. The rapid advancement of radiation oncology treatment technologies presents opportunities for machine learning integration with investments targeted towards data quality, data extraction, software, and engagement with clinical expertise. In this review, we provide an overview of machine learning concepts before reviewing advances in applying machine learning to radiation oncology and integrating these techniques into the radiation oncology workflows. Several key areas are outlined in the radiation oncology workflow where machine learning has been applied and where it can have a significant impact in terms of efficiency, consistency in treatment and overall treatment outcomes. This review highlights that machine learning has key early applications in radiation oncology due to the repetitive nature of many tasks that also currently have human review. Standardized data management of routinely collected imaging and radiation dose data are also highlighted as enabling engagement in research utilizing machine learning and the ability integrate these technologies into clinical workflow to benefit patients. Physicists need to be part of the conversation to facilitate this technical integration.

Machine learning analysis: general features, requirements and cardiovascular applications

Artificial intelligence represents the science which will probably change the future of medicine by solving actually challenging issues. In this special article, the general features of machine learning are discussed. First, a background explanation regarding the division of artificial intelligence, machine learning and deep learning is given and a focus on the structure of machine learning subgroups is shown. The traditional process of a machine learning analysis is described, starting from the collection of data, across features engineering, modelling and till the validation and deployment phase. Due to the several applications of machine learning performed in literature in the last decades and the lack of some guidelines, the need of a standardization for reporting machine learning analysis results emerged. Some possible standards for reporting machine learning results are identified and discussed deeply; these are related to study population (number of subjects), repeatability of the analysis, validation, results, comparison with current practice. The way to the use of machine learning in clinical practice is open and the hope is that, with emerging technology and advanced digital and computational tools, available from hospitalization and subsequently after discharge, it will also be possible, with the help of increasingly powerful hardware, to build assistance strategies useful in clinical practice.

Artificial intelligence for molecular neuroimaging

In recent years, artificial intelligence (AI) or the study of how computers and machines can gain intelligence, has been increasingly applied to problems in medical imaging, and in particular to molecular imaging of the central nervous system. Many AI innovations in medical imaging include improving image quality, segmentation, and automating classification of disease. These advances have led to an increased availability of supportive AI tools to assist physicians in interpreting images and making decisions affecting patient care. This review focuses on the role of AI in molecular neuroimaging, primarily applied to positron emission tomography (PET) and single photon emission computed tomography (SPECT). We emphasize technical innovations such as AI in computed tomography (CT) generation for the purposes of attenuation correction and disease localization, as well as applications in neuro-oncology and neurodegenerative diseases. Limitations and future prospects for AI in molecular brain imaging are also discussed. Just as new equipment such as SPECT and PET revolutionized the field of medical imaging a few decades ago, AI and its related technologies are now poised to bring on further disruptive changes. An understanding of these new technologies and how they work will help physicians adapt their practices and succeed with these new tools.

Deep Neural Networks and Transfer Learning on a Multivariate Physiological Signal Dataset

While Deep Neural Networks (DNNs) and Transfer Learning (TL) have greatly contributed to several medical and clinical disciplines, the application to multivariate physiological datasets is still limited. Current examples mainly focus on one physiological signal and can only utilise applications that are customised for that specific measure, thus it limits the possibility of transferring the trained DNN to other domains. In this study, we composed a dataset (n=813) of six different types of physiological signals (Electrocardiogram, Electrodermal activity, Electromyogram, Photoplethysmogram, Respiration and Acceleration). Signals were collected from 232 subjects using four different acquisition devices. We used a DNN to classify the type of physiological signal and to demonstrate how the TL approach allows the exploitation of the efficiency of DNNs in other domains. After the DNN was trained to optimally classify the type of signal, the features that were automatically extracted by the DNN were used to classify the type of device used for the acquisition using a Support Vector Machine. The dataset, the code and the trained parameters of the DNN are made publicly available to encourage the adoption of DNN and TL in applications with multivariate physiological signals.

Artificial Intelligence and the Medical Physicist: Welcome to the Machine

Artificial intelligence (AI) is a branch of computer science dedicated to giving machines or computers the ability to perform human-like cognitive functions, such as learning, problem-solving, and decision making. Since it is showing superior performance than well-trained human beings in many areas, such as image classification, object detection, speech recognition, and decision-making, AI is expected to change profoundly every area of science, including healthcare and the clinical application of physics to healthcare, referred to as medical physics. As a result, the Italian Association of Medical Physics (AIFM) has created the “AI for Medical Physics” (AI4MP) group with the aims of coordinating the efforts, facilitating the communication, and sharing of the knowledge on AI of the medical physicists (MPs) in Italy. The purpose of this review is to summarize the main applications of AI in medical physics, describe the skills of the MPs in research and clinical applications of AI, and define the major challenges of AI in healthcare.

A deep survival interpretable radiomics model of hepatocellular carcinoma patients

This work aims to identify a new radiomics signature using imaging phenotypes and clinical variables for risk prediction of overall survival (OS) in hepatocellular carcinoma (HCC) patients treated with stereotactic body radiation therapy (SBRT). 167 patients were retrospectively analyzed with repeated nested cross-validation to mitigate overfitting issues. 56 radiomic features were extracted from pre-treatment contrast-enhanced (CE) CT images. 37 clinical factors were obtained from patients' electronic records. Variational autoencoders (VAE) based survival models were designed for radiomics and clinical features and a convolutional neural network (CNN) survival model was used for the CECT. Finally, radiomics, clinical and raw image deep learning network (DNN) models were combined to predict the risk probability for OS. The final models yielded c-indices of 0.579 (95%CI: 0.544-0.621), 0.629 (95%CI: 0.601-0.643), 0.581 (95%CI: 0.553-0.613) and 0.650 (95%CI: 0.635-0.683) for radiomics, clinical, image input and combined models on nested cross validation scheme, respectively. Integrated gradients method was used to interpret the trained models. Our interpretability analysis of the DNN showed that the top ranked features were clinical liver function and liver exclusive of tumor radiomics features, which suggests a prominent role of side effects and toxicities in liver outside the tumor region in determining the survival rate of these patients. In summary, novel deep radiomic analysis provides improved performance for risk assessment of HCC prognosis compared with Cox survival models and may facilitate stratification of HCC patients and personalization of their treatment strategies. Liver function was found to contribute most to the OS for these HCC patients and radiomics can aid in their management.

Robustness of convolutional neural networks in recognition of pigmented skin lesions

BACKGROUND

A basic requirement for artificial intelligence (AI)-based image analysis systems, which are to be integrated into clinical practice, is a high robustness. Minor changes in how those images are acquired, for example, during routine skin cancer screening, should not change the diagnosis of such assistance systems.

OBJECTIVE

To quantify to what extent minor image perturbations affect the convolutional neural network (CNN)-mediated skin lesion classification and to evaluate three possible solutions for this problem (additional data augmentation, test-time augmentation, anti-aliasing).

METHODS

We trained three commonly used CNN architectures to differentiate between dermoscopic melanoma and nevus images. Subsequently, their performance and susceptibility to minor changes ('brittleness') was tested on two distinct test sets with multiple images per lesion. For the first set, image changes, such as rotations or zooms, were generated artificially. The second set contained natural changes that stemmed from multiple photographs taken of the same lesions.

RESULTS

All architectures exhibited brittleness on the artificial and natural test set. The three reviewed methods were able to decrease brittleness to varying degrees while still maintaining performance. The observed improvement was greater for the artificial than for the natural test set, where enhancements were minor.

CONCLUSIONS

Minor image changes, relatively inconspicuous for humans, can have an effect on the robustness of CNNs differentiating skin lesions. By the methods tested here, this effect can be reduced, but not fully eliminated. Thus, further research to sustain the performance of AI classifiers is needed to facilitate the translation of such systems into the clinic.

How to Design AI-Driven Clinical Trials in Nuclear Medicine

Artificial intelligence (AI) is an overarching term for a multitude of technologies which are currently being discussed and introduced in several areas of medicine and in medical imaging specifically. There is, however, limited literature and information about how AI techniques can be integrated into the design of clinical imaging trials. This article will present several aspects of AI being used in trials today and how imaging departments and especially nuclear medicine departments can prepare themselves to be at the forefront of AI-driven clinical trials. Beginning with some basic explanation on AI techniques currently being used and existing challenges of its implementation, it will also cover the logistical prerequisites which have to be in place in nuclear medicine departments to participate successfully in AI-driven clinical trials.

Improving PET Imaging Acquisition and Analysis With Machine Learning: A Narrative Review With Focus on Alzheimer's Disease and Oncology

Machine learning (ML) algorithms have found increasing utility in the medical imaging field and numerous applications in the analysis of digital biomarkers within positron emission tomography (PET) imaging have emerged. Interest in the use of artificial intelligence in PET imaging for the study of neurodegenerative diseases and oncology stems from the potential for such techniques to streamline decision support for physicians providing early and accurate diagnosis and allowing personalized treatment regimens. In this review, the use of ML to improve PET image acquisition and reconstruction is presented, along with an overview of its applications in the analysis of PET images for the study of Alzheimer's disease and oncology.

Introduction to machine and deep learning for medical physicists

Recent years have witnessed tremendous growth in the application of machine learning (ML) and deep learning (DL) techniques in medical physics. Embracing the current big data era, medical physicists equipped with these state-of-the-art tools should be able to solve pressing problems in modern radiation oncology. Here, a review of the basic aspects involved in ML/DL model building, including data processing, model training, and validation for medical physics applications is presented and discussed. Machine learning can be categorized based on the underlying task into supervised learning, unsupervised learning, or reinforcement learning; each of these categories has its own input/output dataset characteristics and aims to solve different classes of problems in medical physics ranging from automation of processes to predictive analytics. It is recognized that data size requirements may vary depending on the specific medical physics application and the nature of the algorithms applied. Data processing, which is a crucial step for model stability and precision, should be performed before training the model. Deep learning as a subset of ML is able to learn multilevel representations from raw input data, eliminating the necessity for hand crafted features in classical ML. It can be thought of as an extension of the classical linear models but with multilayer (deep) structures and nonlinear activation functions. The logic of going "deeper" is related to learning complex data structures and its realization has been aided by recent advancements in parallel computing architectures and the development of more robust optimization methods for efficient training of these algorithms. Model validation is an essential part of ML/DL model building. Without it, the model being developed cannot be easily trusted to generalize to unseen data. Whenever applying ML/DL, one should keep in mind, according to Amara's law, that humans may tend to overestimate the ability of a technology in the short term and underestimate its capability in the long term. To establish ML/DL role into standard clinical workflow, models considering balance between accuracy and interpretability should be developed. Machine learning/DL algorithms have potential in numerous radiation oncology applications, including automatizing mundane procedures, improving efficiency and safety of auto-contouring, treatment planning, quality assurance, motion management, and outcome predictions. Medical physicists have been at the frontiers of technology translation into medicine and they ought to be prepared to embrace the inevitable role of ML/DL in the practice of radiation oncology and lead its clinical implementation.

Balancing accuracy and interpretability of machine learning approaches for radiation treatment outcomes modeling

Radiation outcomes prediction (ROP) plays an important role in personalized prescription and adaptive radiotherapy. A clinical decision may not only depend on an accurate radiation outcomes’ prediction, but also needs to be made based on an informed understanding of the relationship among patients’ characteristics, radiation response and treatment plans. As more patients’ biophysical information become available, machine learning (ML) techniques will have a great potential for improving ROP. Creating explainable ML methods is an ultimate task for clinical practice but remains a challenging one. Towards complete explainability, the interpretability of ML approaches needs to be first explored. Hence, this review focuses on the application of ML techniques for clinical adoption in radiation oncology by balancing accuracy with interpretability of the predictive model of interest. An ML algorithm can be generally classified into an interpretable (IP) or non-interpretable (NIP) (“black box”) technique. While the former may provide a clearer explanation to aid clinical decision-making, its prediction performance is generally outperformed by the latter. Therefore, great efforts and resources have been dedicated towards balancing the accuracy and the interpretability of ML approaches in ROP, but more still needs to be done. In this review, current progress to increase the accuracy for IP ML approaches is introduced, and major trends to improve the interpretability and alleviate the “black box” stigma of ML in radiation outcomes modeling are summarized. Efforts to integrate IP and NIP ML approaches to produce predictive models with higher accuracy and interpretability for ROP are also discussed.