Predicting Response to Neoadjuvant Chemotherapy with PET Imaging Using Convolutional Neural Networks

Impact of artificial intelligence in the management of esophageal, gastric and colorectal malignancies

The incidence of gastrointestinal malignancies has increased over the past decade at an alarming rate. Colorectal and gastric cancers are the third and fifth most commonly diagnosed cancers worldwide but are cited as the second and third leading causes of mortality. Early institution of appropriate therapy from timely diagnosis can optimize patient outcomes. Artificial intelligence (AI)-assisted diagnostic, prognostic, and therapeutic tools can assist in expeditious diagnosis, treatment planning/response prediction, and post-surgical prognostication. AI can intercept neoplastic lesions in their primordial stages, accurately flag suspicious and/or inconspicuous lesions with greater accuracy on radiologic, histopathological, and/or endoscopic analyses, and eliminate over-dependence on clinicians. AI-based models have shown to be on par, and sometimes even outperformed experienced gastroenterologists and radiologists. Convolutional neural networks (state-of-the-art deep learning models) are powerful computational models, invaluable to the field of precision oncology. These models not only reliably classify images, but also accurately predict response to chemotherapy, tumor recurrence, metastasis, and survival rates post-treatment. In this systematic review, we analyze the available evidence about the diagnostic, prognostic, and therapeutic utility of artificial intelligence in gastrointestinal oncology.

Multi-omics approaches for biomarker discovery in predicting the response of esophageal cancer to neoadjuvant therapy: A multidimensional perspective

Neoadjuvant chemoradiotherapy (NCRT) followed by surgery has been established as the standard treatment strategy for operable locally advanced esophageal cancer (EC). However, achieving pathologic complete response (pCR) or near pCR to NCRT is significantly associated with a considerable improvement in survival outcomes, while pCR patients may help organ preservation for patients by active surveillance to avoid planned surgery. Thus, there is an urgent need for improved biomarkers to predict EC chemoradiation response in research and clinical settings. Advances in multiple high-throughput technologies such as next-generation sequencing have facilitated the discovery of novel predictive biomarkers, specifically based on multi-omics data, including genomic/transcriptomic sequencings and proteomic/metabolomic mass spectra. The application of multi-omics data has shown the benefits in improving the understanding of underlying mechanisms of NCRT sensitivity/resistance in EC. Particularly, the prominent development of artificial intelligence (AI) has introduced a new direction in cancer research. The integration of multi-omics data has significantly advanced our knowledge of the disease and enabled the identification of valuable biomarkers for predicting treatment response from diverse dimension levels, especially with rapid advances in biotechnological and AI methodologies. Herein, we summarize the current status of research on the use of multi-omics technologies in predicting NCRT response for EC patients. Current limitations, challenges, and future perspectives of these multi-omics platforms will be addressed to assist in experimental designs and clinical use for further integrated analysis.

A Radiotherapy Dose Map-Guided Deep Learning Method for Predicting Pathological Complete Response in Esophageal Cancer Patients after Neoadjuvant Chemoradiotherapy Followed by Surgery

Esophageal cancer is a deadly disease, and neoadjuvant chemoradiotherapy can improve patient survival, particularly for patients achieving a pathological complete response (ypCR). However, existing imaging methods struggle to accurately predict ypCR. This study explores computer-aided detection methods, considering both imaging data and radiotherapy dose variations to enhance prediction accuracy. It involved patients with node-positive esophageal squamous cell carcinoma undergoing neoadjuvant chemoradiotherapy and surgery, with data collected from 2014 to 2017, randomly split into five subsets for 5-fold cross-validation. The algorithm DCRNet, an advanced version of OCRNet, integrates RT dose distribution into dose contextual representations (DCR), combining dose and pixel representation with ten soft regions. Among the 80 enrolled patients (mean age 55.68 years, primarily male, with stage III disease and middle-part lesions), the ypCR rate was 28.75%, showing no significant demographic or disease differences between the ypCR and non-ypCR groups. Among the three summarization methods, the maximum value across the CTV method produced the best results with an AUC of 0.928. The HRNetV2p model with DCR performed the best among the four backbone models tested, with an AUC of 0.928 (95% CI, 0.884-0.972) based on 5-fold cross-validation, showing significant improvement compared to other models. This underscores DCR-equipped models' superior AUC outcomes. The study highlights the potential of dose-guided deep learning in ypCR prediction, necessitating larger, multicenter studies to validate the results.

Artificial intelligence in esophageal cancer diagnosis and treatment: where are we now?-a narrative review

Background and Objective

Artificial intelligence (AI) use is becoming increasingly prevalent directly or indirectly in daily clinical practice, including esophageal cancer (EC) diagnosis and treatment. Although the limits of its adoption and their clinical benefits are still unknown, any physician related to EC patients' management should be aware of the status and future perspectives of AI use in their field. The purpose of this review is to summarize the existing literature regarding the role of AI in diagnosis and treatment of EC. We have focused on the aids AI entails in the management of this pathology and we have tried to offer an updated perspective to maximize current applications and to identify potential future uses of it.

Methods

Data concerning AI applied to EC diagnosis and treatment is not limited, including direct (those specifically related to them) and indirect (those referring to other specialties as radiology or pathology), applications. However, the clinical relevance of the discussed and presented models is still unknown. We performed a research in PubMed of English and Spanish written studies from January 1970 to June 2022.

Key Content and Findings

Information regarding the role of AI in EC diagnosis and treatment has increased exponentially in recent years. Several models, including different variables and features have been investigated and some of them internally and externally validated. However, the main challenge remains to apply and introduce all these data into clinical practice, and, as some of the discussed studies argue, if the models are able to enhance experienced endoscopists' judgement. Although AI use is increasing steadily in different medical specialties, the truth is, most of the time, the gap between model development and clinical implementation is not closed. Learning to understand the routinely application of AI, as well as future improvements, would lead to a broadened adoption.

Conclusions

Physicians should be aware of the multiple current clinical uses of AI in EC diagnosis and treatment and should take part in their clinical application and future developments to enhance patient care.

Performance of radiomics-based artificial intelligence systems in the diagnosis and prediction of treatment response and survival in esophageal cancer: a systematic review and meta-analysis of diagnostic accuracy

Radiomics can interpret radiological images with more detail and in less time compared to the human eye. Some challenges in managing esophageal cancer can be addressed by incorporating radiomics into image interpretation, treatment planning, and predicting response and survival. This systematic review and meta-analysis provides a summary of the evidence of radiomics in esophageal cancer. The systematic review was carried out using Pubmed, MEDLINE, and Ovid EMBASE databases-articles describing radiomics in esophageal cancer were included. A meta-analysis was also performed; 50 studies were included. For the assessment of treatment response using 18F-FDG PET/computed tomography (CT) scans, seven studies (443 patients) were included in the meta-analysis. The pooled sensitivity and specificity were 86.5% (81.1-90.6) and 87.1% (78.0-92.8). For the assessment of treatment response using CT scans, five studies (625 patients) were included in the meta-analysis, with a pooled sensitivity and specificity of 86.7% (81.4-90.7) and 76.1% (69.9-81.4). The remaining 37 studies formed the qualitative review, discussing radiomics in diagnosis, radiotherapy planning, and survival prediction. This review explores the wide-ranging possibilities of radiomics in esophageal cancer management. The sensitivities of 18F-FDG PET/CT scans and CT scans are comparable, but 18F-FDG PET/CT scans have improved specificity for AI-based prediction of treatment response. Models integrating clinical and radiomic features facilitate diagnosis and survival prediction. More research is required into comparing models and conducting large-scale studies to build a robust evidence base.

Role of PET/CT in Oropharyngeal Cancers

Oropharyngeal squamous cell carcinoma (OPSCC) comprises cancers of the tonsils, tongue base, soft palate, and uvula. The staging of oropharyngeal cancers varies depending upon the presence or absence of human papillomavirus (HPV)-directed pathogenesis. The incidence of HPV-associated oropharyngeal cancer (HPV + OPSCC) is expected to continue to rise over the coming decades. PET/CT is a useful modality for the diagnosis, staging, and follow up of patients with oropharyngeal cancers undergoing treatment and surveillance.

Radiomics combined with transcriptomics to predict response to immunotherapy from patients treated with PD-1/PD-L1 inhibitors for advanced NSCLC

Introduction

In this study, we aim to build radiomics and multiomics models based on transcriptomics and radiomics to predict the response from patients treated with the PD-L1 inhibitor.

Materials and methods

One hundred and ninety-five patients treated with PD-1/PD-L1 inhibitors were included. For all patients, 342 radiomic features were extracted from pretreatment computed tomography scans. The training set was built with 110 patients treated at the Léon Bérard Cancer Center. An independent validation cohort was built with the 85 patients treated in Dijon. The two sets were dichotomized into two classes, patients with disease control and those considered non-responders, in order to predict the disease control at 3 months. Various models were trained with different feature selection methods, and different classifiers were evaluated to build the models. In a second exploratory step, we used transcriptomics to enrich the database and develop a multiomic signature of response to immunotherapy in a 54-patient subgroup. Finally, we considered the HOT/COLD status. We first trained a radiomic model to predict the HOT/COLD status and then prototyped a hybrid model integrating radiomics and the HOT/COLD status to predict the response to immunotherapy.

Results

Radiomic signature for 3 months' progression-free survival (PFS) classification: The most predictive model had an area under the receiver operating characteristic curve (AUROC) of 0.94 on the training set and 0.65 on the external validation set. This model was obtained with the t-test selection method and with a support vector machine (SVM) classifier. Multiomic signature for PFS classification: The most predictive model had an AUROC of 0.95 on the training set and 0.99 on the validation set. Radiomic model to predict the HOT/COLD status: the most predictive model had an AUROC of 0.93 on the training set and 0.86 on the validation set. HOT/COLD radiomic hybrid model for PFS classification: the most predictive model had an AUROC of 0.93 on the training set and 0.90 on the validation set.

Conclusion

In conclusion, radiomics could be used to predict the response to immunotherapy in non-small-cell lung cancer patients. The use of transcriptomics or the HOT/COLD status, together with radiomics, may improve the working of the prediction models.

Efficacy of texture analysis of ultrasonographic images in the differentiation of metastatic and non-metastatic cervical lymph nodes in patients with squamous cell carcinoma of the tongue

OBJECTIVE

We investigated the efficacy of using texture analysis of ultrasonographic images of the cervical lymph nodes of patients with squamous cell carcinoma of the tongue to differentiate between metastatic and non-metastatic lymph nodes.

STUDY DESIGN

We analyzed 32 metastatic and 28 non-metastatic lymph nodes diagnosed by histopathologic examination on presurgical US images. Using the LIFEx texture analysis program, we extracted 36 texture features from the images and calculated the statistical significance of differences in texture features between metastatic and non-metastatic lymph nodes using the t test. To assess the diagnostic ability of the significantly different texture features to discriminate between metastatic and non-metastatic nodes, we performed receiver operating characteristic curve analysis and calculated the area under the curve. We set the cutoff points that maximized the sensitivity and specificity for each curve according to the Youden J statistic.

RESULTS

We found that 20 texture features significantly differed between metastatic and non-metastatic lymph nodes. Among them, only the gray-level run length matrix feature of run length non-uniformity and the gray-level zone length matrix features of gray-level non-uniformity and zone length non-uniformity showed an excellent ability to discriminate between metastatic and non-metastatic lymph nodes as indicated by the area under the curve and the sum of sensitivity and specificity.

CONCLUSIONS

Analysis of the texture features of run length non-uniformity, gray-level non-uniformity, and zone length non-uniformity values allows for differentiation between metastatic and non-metastatic lymph nodes, with the use of gray-level non-uniformity appearing to be the best means of predicting metastatic lymph nodes.

Predicting the efficacy of radiotherapy for esophageal squamous cell carcinoma based on enhanced computed tomography radiomics and combined models

Purpose

This study aimed to investigate the ability of enhanced computed tomography (CT)-based radiomics and dosimetric parameters in predicting response to radiotherapy for esophageal cancer.

Methods

A retrospective analysis of 147 patients diagnosed with esophageal cancer was performed, and the patients were divided into a training group (104 patients) and a validation group (43 patients). In total, 851 radiomics features were extracted from the primary lesions for analysis. Maximum correlation minimum redundancy and minimum least absolute shrinkage and selection operator were utilized for feature screening of radiomics features, and logistic regression was applied to construct a radiotherapy radiomics model for esophageal cancer. Finally, univariate and multivariate parameters were used to identify significant clinical and dosimetric characteristics for constructing combination models. The area evaluated the predictive performance under the receiver operating characteristics (AUC) curve and the accuracy, sensitivity, and specificity of the training and validation cohorts.

Results

Univariate logistic regression analysis revealed statistically significant differences in clinical parameters of sex (p=0.031) and esophageal cancer thickness (p=0.028) on treatment response, whereas dosimetric parameters did not differ significantly in response to treatment. The combined model demonstrated improved discrimination between the training and validation groups, with AUCs of 0.78 (95% confidence interval [CI], 0.69–0.87) and 0.79 (95% CI, 0.65–0.93) in the training and validation groups, respectively.

Conclusion

The combined model has potential application value in predicting the treatment response of patients with esophageal cancer after radiotherapy.

Clinical application of AI-based PET images in oncological patients

Based on the advantages of revealing the functional status and molecular expression of tumor cells, positron emission tomography (PET) imaging has been performed in numerous types of malignant diseases for diagnosis and monitoring. However, insufficient image quality, the lack of a convincing evaluation tool and intra- and interobserver variation in human work are well-known limitations of nuclear medicine imaging and restrict its clinical application. Artificial intelligence (AI) has gained increasing interest in the field of medical imaging due to its powerful information collection and interpretation ability. The combination of AI and PET imaging potentially provides great assistance to physicians managing patients. Radiomics, an important branch of AI applied in medical imaging, can extract hundreds of abstract mathematical features of images for further analysis. In this review, an overview of the applications of AI in PET imaging is provided, focusing on image enhancement, tumor detection, response and prognosis prediction and correlation analyses with pathology or specific gene mutations in several types of tumors. Our aim is to describe recent clinical applications of AI-based PET imaging in malignant diseases and to focus on the description of possible future developments.

Computed tomography-based radiomics machine learning models for prediction of histological invasiveness with sub-centimeter subsolid pulmonary nodules: a retrospective study

To improve the accuracy of preoperative diagnoses and avoid over- or undertreatment, we aimed to develop and compare computed tomography-based radiomics machine learning models for the prediction of histological invasiveness using sub-centimeter subsolid pulmonary nodules. Three predictive models based on radiomics were built using three machine learning classifiers to discriminate the invasiveness of the sub-centimeter subsolid pulmonary nodules. A total of 203 sub-centimeter nodules from 177 patients were collected and assigned randomly to the training set (n = 143) or test set (n = 60). The areas under the curve of the predictive models were 0.743 (95% confidence interval CI [0.661-0.824]) for the logistic regression, 0.828 (95% CI [0.76-0.896]) for the support vector machine, and 0.917 (95% CI [0.869-0.965]) for the XGBoost classifier models in the training set, and 0.803 (95% CI [0.694-0.913]), 0.726 (95% CI [0.598-0.854]), and 0.874 (95% CI [0.776-0.972]) in the test set, respectively. In addition, the decision curve showed that the XGBoost model added more net benefit within the range of 0.06 to 0.93.

Joint EANM/SNMMI guideline on radiomics in nuclear medicine : Jointly supported by the EANM Physics Committee and the SNMMI Physics, Instrumentation and Data Sciences Council

PURPOSE

The purpose of this guideline is to provide comprehensive information on best practices for robust radiomics analyses for both hand-crafted and deep learning-based approaches.

METHODS

In a cooperative effort between the EANM and SNMMI, we agreed upon current best practices and recommendations for relevant aspects of radiomics analyses, including study design, quality assurance, data collection, impact of acquisition and reconstruction, detection and segmentation, feature standardization and implementation, as well as appropriate modelling schemes, model evaluation, and interpretation. We also offer an outlook for future perspectives.

CONCLUSION

Radiomics is a very quickly evolving field of research. The present guideline focused on established findings as well as recommendations based on the state of the art. Though this guideline recognizes both hand-crafted and deep learning-based radiomics approaches, it primarily focuses on the former as this field is more mature. This guideline will be updated once more studies and results have contributed to improved consensus regarding the application of deep learning methods for radiomics. Although methodological recommendations in the present document are valid for most medical image modalities, we focus here on nuclear medicine, and specific recommendations when necessary are made for PET/CT, PET/MR, and quantitative SPECT.

Deep learning based radiomics for gastrointestinal cancer diagnosis and treatment: A minireview

Gastrointestinal (GI) cancers are the major cause of cancer-related mortality globally. Medical imaging is an important auxiliary means for the diagnosis, assessment and prognostic prediction of GI cancers. Radiomics is an emerging and effective technology to decipher the encoded information within medical images, and traditional machine learning is the most commonly used tool. Recent advances in deep learning technology have further promoted the development of radiomics. In the field of GI cancer, although there are several surveys on radiomics, there is no specific review on the application of deep-learning-based radiomics (DLR). In this review, a search was conducted on Web of Science, PubMed, and Google Scholar with an emphasis on the application of DLR for GI cancers, including esophageal, gastric, liver, pancreatic, and colorectal cancers. Besides, the challenges and recommendations based on the findings of the review are comprehensively analyzed to advance DLR.

The efficacy of 18F-FDG-PET-based radiomic and deep-learning features using a machine-learning approach to predict the pathological risk subtypes of thymic epithelial tumors

OBJECTIVE

To examine whether the machine-learning approach using 18-fludeoxyglucose positron emission tomography (18F-FDG-PET)-based radiomic and deep-learning features is useful for predicting the pathological risk subtypes of thymic epithelial tumors (TETs).

METHODS

This retrospective study included 79 TET [27 low-risk thymomas (types A, AB and B1), 31 high-risk thymomas (types B2 and B3) and 21 thymic carcinomas] patients who underwent pre-therapeutic 18F-FDG-PET/CT. High-risk TETs (high-risk thymomas and thymic carcinomas) were 52 patients. The 107 PET-based radiomic features, including SUV-related parameters [maximum SUV (SUVmax), metabolic tumor volume (MTV), and total lesion glycolysis (TLG)] and 1024 deep-learning features extracted from the convolutional neural network were used to predict the pathological risk subtypes of TETs using six different machine-learning algorithms. The area under the curves (AUCs) were calculated to compare the predictive performances.

RESULTS

SUV-related parameters yielded the following AUCs for predicting thymic carcinomas: SUVmax 0.713, MTV 0.442, and TLG 0.479 or high-risk TETs: SUVmax 0.673, MTV 0.533, and TLG 0.539. The best-performing algorithm was the logistic regression model for predicting thymic carcinomas (AUC 0.900, accuracy 81.0%), and the random forest (RF) model for high-risk TETs (AUC 0.744, accuracy 72.2%). The AUC was significantly higher in the logistic regression model than three SUV-related parameters for predicting thymic carcinomas, and in the RF model than MTV and TLG for predicting high-risk TETs (each; p < 0.05).

CONCLUSION

18F-FDG-PET-based radiomic analysis using a machine-learning approach may be useful for predicting the pathological risk subtypes of TETs.

ADVANCES IN KNOWLEDGE

Machine-learning approach using 18F-FDG-PET-based radiomic features has the potential to predict the pathological risk subtypes of TETs.

Radiomics in Oncological PET Imaging: A Systematic Review-Part 2, Infradiaphragmatic Cancers, Blood Malignancies, Melanoma and Musculoskeletal Cancers

The objective of this review was to summarize published radiomics studies dealing with infradiaphragmatic cancers, blood malignancies, melanoma, and musculoskeletal cancers, and assess their quality. PubMed database was searched from January 1990 to February 2022 for articles performing radiomics on PET imaging of at least 1 specified tumor type. Exclusion criteria includd: non-oncological studies; supradiaphragmatic tumors; reviews, comments, cases reports; phantom or animal studies; technical articles without a clinically oriented question; studies including <30 patients in the training cohort. The review database contained PMID, first author, year of publication, cancer type, number of patients, study design, independent validation cohort and objective. This database was completed twice by the same person; discrepant results were resolved by a third reading of the articles. A total of 162 studies met inclusion criteria; 61 (37.7%) studies included >100 patients, 13 (8.0%) were prospective and 61 (37.7%) used an independent validation set. The most represented cancers were esophagus, lymphoma, and cervical cancer (n = 24, n = 24 and n = 19 articles, respectively). Most studies focused on 18F-FDG, and prognostic and response to treatment objectives. Although radiomics and artificial intelligence are technically challenging, new contributions and guidelines help improving research quality over the years and pave the way toward personalized medicine.

Clinical Perspectives for 18F-FDG PET Imaging in Pediatric Oncology: Μetabolic Tumor Volume and Radiomics

Pediatric cancer, although rare, requires the most optimized treatment approach to obtain high survival rates and minimize serious long-term side effects in early adulthood. 18F-FDG PET/CT is most helpful and widely used in staging, recurrence detection, and response assessment in pediatric oncology. The well-known 18F-FDG PET metabolic indices of metabolic tumor volume (MTV) and tumor lesion glycolysis (TLG) have already revealed an independent significant prognostic value for survival in oncologic patients, although the corresponding cut-off values remain study-dependent and not validated for use in clinical practice. Advanced tumor “radiomic” analysis sheds new light into these indices. Numerous patterns of texture 18F-FDG uptake features can be extracted from segmented PET tumor images due to new powerful computational systems supporting complex “deep learning” algorithms. This high number of “quantitative” tumor imaging data, although not decrypted in their majority and once standardized for the different imaging systems and segmentation methods, could be used for the development of new “clinical” models for specific cancer types and, more interestingly, for specific age groups. In addition, data from novel techniques of tumor genome analysis could reveal new genes as biomarkers for prognosis and/or targeted therapies in childhood malignancies. Therefore, this ever-growing information of “radiogenomics”, in which the underlying tumor “genetic profile” could be expressed in the tumor-imaging signature of “radiomics”, possibly represents the next model for precision medicine in pediatric cancer management. This paper reviews 18F-FDG PET image segmentation methods as applied to pediatric sarcomas and lymphomas and summarizes reported findings on the values of metabolic and radiomic features in the assessment of these pediatric tumors.

Machine learning based evaluation of clinical and pretreatment 18F-FDG-PET/CT radiomic features to predict prognosis of cervical cancer patients

PURPOSE

To examine the usefulness of machine learning to predict prognosis in cervical cancer using clinical and radiomic features of 2-deoxy-2-[¹⁸F]fluoro-D-glucose (¹⁸F-FDG) positron emission tomography/computed tomography (CT) (¹⁸F-FDG-PET/CT).

METHODS

This retrospective study included 50 cervical cancer patients who underwent ¹⁸F-FDG-PET/CT before treatment. Four clinical (age, histology, stage, and treatment) and 41 ¹⁸F-FDG-PET-based radiomic features were ranked and a subset of useful features for association with disease progression was selected based on decrease of the Gini impurity. Six machine learning algorithms (random forest, neural network, k-nearest neighbors, naive Bayes, logistic regression, and support vector machine) were compared using the areas under the receiver operating characteristic curve (AUC). Progression-free survival (PFS) was assessed using Cox regression analysis.

RESULTS

The five top predictors of disease progression were: stage, surface area, metabolic tumor volume, gray-level run length non-uniformity (GLRLM_RLNU), and gray-level non-uniformity for run (GLRLM_GLNU). The naive Bayes model was the best-performing classifier for predicting disease progression (AUC = 0.872, accuracy = 0.780, F1 score = 0.781, precision = 0.788, and recall = 0.780). In the naive Bayes model, 5-year PFS was significantly higher in predicted non-progression than predicted progression (80.1% vs. 9.1%, p < 0.001) and was only the independent factor for PFS in multivariate analysis (HR, 6.89; 95% CI, 1.92-24.69; p = 0.003).

CONCLUSION

A machine learning approach based on clinical and pretreatment ¹⁸F-FDG PET-based radiomic features may be useful for predicting tumor progression in cervical cancer patients.

Chemotherapy response prediction with diffuser elapser network

In solid tumors, elevated fluid pressure and inadequate blood perfusion resulting from unbalanced angiogenesis are the prominent reasons for the ineffective drug delivery inside tumors. To normalize the heterogeneous and tortuous tumor vessel structure, antiangiogenic treatment is an effective approach. Additionally, the combined therapy of antiangiogenic agents and chemotherapy drugs has shown promising effects on enhanced drug delivery. However, the need to find the appropriate scheduling and dosages of the combination therapy is one of the main problems in anticancer therapy. Our study aims to generate a realistic response to the treatment schedule, making it possible for future works to use these patient-specific responses to decide on the optimal starting time and dosages of cytotoxic drug treatment. Our dataset is based on our previous in-silico model with a framework for the tumor microenvironment, consisting of a tumor layer, vasculature network, interstitial fluid pressure, and drug diffusion maps. In this regard, the chemotherapy response prediction problem is discussed in the study, putting forth a proof of concept for deep learning models to capture the tumor growth and drug response behaviors simultaneously. The proposed model utilizes multiple convolutional neural network submodels to predict future tumor microenvironment maps considering the effects of ongoing treatment. Since the model has the task of predicting future tumor microenvironment maps, we use two image quality evaluation metrics, which are structural similarity and peak signal-to-noise ratio, to evaluate model performance. We track tumor cell density values of ground truth and predicted tumor microenvironments. The model predicts tumor microenvironment maps seven days ahead with the average structural similarity score of 0.973 and the average peak signal ratio of 35.41 in the test set. It also predicts tumor cell density at the end day of 7 with the mean absolute percentage error of $$2.292\pm 1.820$$ 2.292 ± 1.820 .

Artificial intelligence in cancer diagnostics and therapy: current perspectives

Artificial intelligence (AI) has found its way into every sphere of human life including the field of medicine. Detection of cancer might be AI's most altruistic and convoluted challenge to date in the field of medicine. Embedding AI into various aspects of cancer diagnostics would be of immense use in dealing with the tedious, repetitive, time-consuming job of lesion detection, remove opportunities for human error, and cut costs and time. This would be of great value in cancer screening programs. By using AI algorithms, data from digital images from radiology and pathology that are imperceptible to the human eye can be identified (radiomics and pathomics). Correlating radiomics and pathomics with clinico-demographic-therapy-morbidity-mortality profiles will lead to a greater understanding of cancers. Specific imaging phenotypes have been found to be associated with specific gene-determined molecular pathways involved in cancer pathogenesis (radiogenomics). All these developments would not only help to personalize oncologic practice but also lead to the development of new imaging biomarkers. AI algorithms in oncoimaging and oncopathology will broadly have the following uses: cancer screening (detection of lesions), characterization and grading of tumors, and clinical decision-making and prognostication. However, AI cannot be a foolproof panacea nor can it supplant the role of humans. It can however be a powerful and useful complement to human insights and deeper understanding. Multiple issues like standardization, validity, ethics, privacy, finances, legal liability, training, accreditation, etc., need to be overcome before the vast potential of AI in diagnostic oncology can be fully harnessed.

Methodological quality of machine learning-based quantitative imaging analysis studies in esophageal cancer: a systematic review of clinical outcome prediction after concurrent chemoradiotherapy

PURPOSE

Studies based on machine learning-based quantitative imaging techniques have gained much interest in cancer research. The aim of this review is to critically appraise the existing machine learning-based quantitative imaging analysis studies predicting outcomes of esophageal cancer after concurrent chemoradiotherapy in accordance with PRISMA guidelines.

METHODS

A systematic review was conducted in accordance with PRISMA guidelines. The citation search was performed via PubMed and Embase Ovid databases for literature published before April 2021. From each full-text article, study characteristics and model information were summarized. We proposed an appraisal matrix with 13 items to assess the methodological quality of each study based on recommended best-practices pertaining to quality.

RESULTS

Out of 244 identified records, 37 studies met the inclusion criteria. Study endpoints included prognosis, treatment response, and toxicity after concurrent chemoradiotherapy with reported discrimination metrics in validation datasets between 0.6 and 0.9, with wide variation in quality. A total of 30 studies published within the last 5 years were evaluated for methodological quality and we found 11 studies with at least 6 "good" item ratings.

CONCLUSION

A substantial number of studies lacked prospective registration, external validation, model calibration, and support for use in clinic. To further improve the predictive power of machine learning-based models and translate into real clinical applications in cancer research, appropriate methodologies, prospective registration, and multi-institution validation are recommended.

AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging:: Towards Radiophenomics

Artificial intelligence (AI) techniques have significant potential to enable effective, robust, and automated image phenotyping including the identification of subtle patterns. AI-based detection searches the image space to find the regions of interest based on patterns and features. There is a spectrum of tumor histologies from benign to malignant that can be identified by AI-based classification approaches using image features. The extraction of minable information from images gives way to the field of "radiomics" and can be explored via explicit (handcrafted/engineered) and deep radiomics frameworks. Radiomics analysis has the potential to be used as a noninvasive technique for the accurate characterization of tumors to improve diagnosis and treatment monitoring. This work reviews AI-based techniques, with a special focus on oncological PET and PET/CT imaging, for different detection, classification, and prediction/prognosis tasks. We also discuss needed efforts to enable the translation of AI techniques to routine clinical workflows, and potential improvements and complementary techniques such as the use of natural language processing on electronic health records and neuro-symbolic AI techniques.

Radiomics in breast MRI: current progress toward clinical application in the era of artificial intelligence

Breast magnetic resonance imaging (MRI) is the most sensitive imaging modality for breast cancer diagnosis and is widely used clinically. Dynamic contrast-enhanced MRI is the basis for breast MRI, but ultrafast images, T2-weighted images, and diffusion-weighted images are also taken to improve the characteristics of the lesion. Such multiparametric MRI with numerous morphological and functional data poses new challenges to radiologists, and thus, new tools for reliable, reproducible, and high-volume quantitative assessments are warranted. In this context, radiomics, which is an emerging field of research involving the conversion of digital medical images into mineable data for clinical decision-making and outcome prediction, has been gaining ground in oncology. Recent development in artificial intelligence has promoted radiomics studies in various fields including breast cancer treatment and numerous studies have been conducted. However, radiomics has shown a translational gap in clinical practice, and many issues remain to be solved. In this review, we will outline the steps of radiomics workflow and investigate clinical application of radiomics focusing on breast MRI based on published literature, as well as current discussion about limitations and challenges in radiomics.

Prediction of Neoadjuvant Chemotherapy Response in Osteosarcoma Using Convolutional Neural Network of Tumor Center 18F-FDG PET Images

We compared the accuracy of prediction of the response to neoadjuvant chemotherapy (NAC) in osteosarcoma patients between machine learning approaches of whole tumor utilizing fluorine−¹⁸fluorodeoxyglucose (¹⁸F-FDG) uptake heterogeneity features and a convolutional neural network of the intratumor image region. In 105 patients with osteosarcoma, ¹⁸F-FDG positron emission tomography/computed tomography (PET/CT) images were acquired before (baseline PET0) and after NAC (PET1). Patients were divided into responders and non-responders about neoadjuvant chemotherapy. Quantitative ¹⁸F-FDG heterogeneity features were calculated using LIFEX version 4.0. Receiver operating characteristic (ROC) curve analysis of ¹⁸F-FDG uptake heterogeneity features was used to predict the response to NAC. Machine learning algorithms and 2-dimensional convolutional neural network (2D CNN) deep learning networks were estimated for predicting NAC response with the baseline PET0 images of the 105 patients. ML was performed using the entire tumor image. The accuracy of the 2D CNN prediction model was evaluated using total tumor slices, the center 20 slices, the center 10 slices, and center slice. A total number of 80 patients was used for k-fold validation by five groups with 16 patients. The CNN network test accuracy estimation was performed using 25 patients. The areas under the ROC curves (AUCs) for baseline PET maximum standardized uptake value (SUVmax), total lesion glycolysis (TLG), metabolic tumor volume (MTV), and gray level size zone matrix (GLSZM) were 0.532, 0.507, 0.510, and 0.626, respectively. The texture features test accuracy of machine learning by random forest and support vector machine were 0.55 and 0. 54, respectively. The k-fold validation accuracy and validation accuracy were 0.968 ± 0.01 and 0.610 ± 0.04, respectively. The test accuracy of total tumor slices, the center 20 slices, center 10 slices, and center slices were 0.625, 0.616, 0.628, and 0.760, respectively. The prediction model for NAC response with baseline PET0 texture features machine learning estimated a poor outcome, but the 2D CNN network using ¹⁸F-FDG baseline PET0 images could predict the treatment response before prior chemotherapy in osteosarcoma. Additionally, using the 2D CNN prediction model using a tumor center slice of ¹⁸F-FDG PET images before NAC can help decide whether to perform NAC to treat osteosarcoma patients.

Role of Artificial Intelligence in Theranostics:: Toward Routine Personalized Radiopharmaceutical Therapies

We highlight emerging uses of artificial intelligence (AI) in the field of theranostics, focusing on its significant potential to enable routine and reliable personalization of radiopharmaceutical therapies (RPTs). Personalized RPTs require patient-specific dosimetry calculations accompanying therapy. Additionally we discuss the potential to exploit biological information from diagnostic and therapeutic molecular images to derive biomarkers for absorbed dose and outcome prediction; toward personalization of therapies. We try to motivate the nuclear medicine community to expand and align efforts into making routine and reliable personalization of RPTs a reality.

Radiomics and Digital Image Texture Analysis in Oncology (Review)

One of the most promising areas of diagnosis and prognosis of diseases is radiomics, a science combining radiology, mathematical modeling, and deep machine learning. The main concept of radiomics is image biomarkers (IBMs), the parameters characterizing various pathological changes and calculated based on the analysis of digital image texture. IBMs are used for quantitative assessment of digital imaging results (CT, MRI, ultrasound, PET). The use of IBMs in the form of “virtual biopsy” is of particular relevance in oncology.

The article provides the basic concepts of radiomics identifying the main stages of obtaining IBMs: data collection and preprocessing, tumor segmentation, data detection and extraction, modeling, statistical processing, and data validation. The authors have analyzed the possibilities of using IBMs in oncology, describing the currently known features and advantages of using radiomics and image texture analysis in the diagnosis and prognosis of cancer. The limitations and problems associated with the use of radiomics data are considered.

Although the novel effective tool for performing virtual biopsy of human tissue is at the development stage, quite a few projects have already been implemented, and medical software packages for radiomics analysis of digital images have been created.

Accurate and Feasible Deep Learning Based Semi-Automatic Segmentation in CT for Radiomics Analysis in Pancreatic Neuroendocrine Neoplasms

Current clinical practice or radiomics studies of pancreatic neuroendocrine neoplasms (pNENs) require manual delineation of the lesions in computed tomography (CT) images, which is time-consuming and subjective. We used a semi-automatic deep learning (DL) method for segmentation of pNENs and verified its feasibility in radiomics analysis. This retrospective study included two datasets: Dataset 1, contrast-enhanced CT images (CECT) of 80 and 18 patients respectively collected from two centers; and Dataset 2, CECT of 56 and 16 patients respectively from two centers. A DL-based semi-automatic segmentation model was developed and validated with Dataset 1 and Dataset 2, and the segmentation results were used for radiomics analysis from which the performance was compared against that based on manual segmentation. The mean Dice similarity coefficient of the trained segmentation model was 81.8% and 74.8% for external validation with Dataset 1 and Dataset 2 respectively. Four classifiers frequently used in radiomics studies were trained and tested with leave-one-out cross-validation strategy. For pathological grading prediction with Dataset 1, the area under the receiver operating characteristic curve (AUC) with semi-automatic segmentation was up to 0.76 and 0.87 respectively for internal and external validation. For recurrence study with Dataset 2, the AUC with semi-automatic segmentation was up to 0.78. All these AUCs were not statistically significant from the corresponding results based on manual segmentation. Our study showed that DL-based semi-automatic segmentation is accurate and feasible for the radiomics analysis in pNENs.

Artificial Intelligence in PET: An Industry Perspective

Artificial intelligence (AI) has significant potential to positively impact and advance medical imaging, including positron emission tomography (PET) imaging applications. AI has the ability to enhance and optimize all aspects of the PET imaging chain from patient scheduling, patient setup, protocoling, data acquisition, detector signal processing, reconstruction, image processing, and interpretation. AI poses industry-specific challenges which will need to be addressed and overcome to maximize the future potentials of AI in PET. This article provides an overview of these industry-specific challenges for the development, standardization, commercialization, and clinical adoption of AI and explores the potential enhancements to PET imaging brought on by AI in the near future. In particular, the combination of on-demand image reconstruction, AI, and custom-designed data-processing workflows may open new possibilities for innovation which would positively impact the industry and ultimately patients.

Thinking through the multimodal treatment of localized oesophageal cancer: the point of view of the surgeon

PURPOSE OF REVIEW

This review examines current developments and controversies in the multimodal management of oesophageal cancer, with an emphasis on surgical dilemmas and outcomes from the surgeon's perspective.

RECENT FINDINGS

Despite the advancement of oncological neoadjuvant treatments, there is still no consensus on what regimen is superior. The majority of patients may still fail to respond to neoadjuvant therapy and suffer potential harm without any survival advantage as a result. In patients who do not respond, adjuvant therapy is still often recommended after surgery despite any evidence for its benefit. We examine the implications of different regimens and treatment approaches for both squamous cell cancer and adenocarcinoma of the oesophagus.

SUMMARY

The efficacy of neoadjuvant treatment is highly variable and likely relates to variability of tumour biology. Ongoing work to identify responders, or optimize treatment on an individual patient, should increase the efficacy of multimodal therapy and improve patient outcomes.

Application of a machine learning approach to characterization of liver function using 99mTc-GSA SPECT/CT

PURPOSE

To assess the utility of a machine-learning approach for predicting liver function based on technetium-99 m-galactosyl serum albumin (^99mTc-GSA) single photon emission computed tomography (SPECT)/CT.

METHODS

One hundred twenty-eight patients underwent a ^99mTc-GSA SPECT/CT-based liver function evaluation. All were classified into the low liver-damage or high liver-damage group. Four clinical (age, sex, background liver disease and histological type) and 8 quantitative ^99mTc-GSA SPECT/CT features (receptor index [LHL15], clearance index [HH15], liver-SUVmax, liver-SUVmean, heart-SUVmax, metabolic volume of liver [MVL], total lesion GSA [TL-GSA, liver-SUVmean × MVL] and SUVmax ratio [liver-SUVmax/heart-SUVmax]) were obtained. To predict high liver damage, a machine learning classification with features selection based on Gini impurity and principal component analysis (PCA) were performed using a support vector machine and a random forest (RF) with a five-fold cross-validation scheme. To overcome imbalanced data, stratified sampling was used. The ability to predict high liver damage was evaluated using a receiver operating characteristic (ROC) curve analysis.

RESULTS

Four indices (LHL15, HH15, heart SUVmax and SUVmax ratio) yielded high areas under the ROC curves (AUCs) for predicting high liver damage (range: 0.89-0.93). In a machine learning classification, the RF with selected features (heart SUVmax, SUVmax ratio, LHL15, HH15, and background liver disease) and PCA model yielded the best performance for predicting high liver damage (AUC = 0.956, sensitivity = 96.3%, specificity = 90.0%, accuracy = 91.4%).

CONCLUSION

A machine-learning approach based on clinical and quantitative ^99mTc-GSA SPECT/CT parameters might be useful for predicting liver function.

Machine Learning and Radiomics Applications in Esophageal Cancers Using Non-Invasive Imaging Methods-A Critical Review of Literature

Esophageal cancer (EC) is of public health significance as one of the leading causes of cancer death worldwide. Accurate staging, treatment planning and prognostication in EC patients are of vital importance. Recent advances in machine learning (ML) techniques demonstrate their potential to provide novel quantitative imaging markers in medical imaging. Radiomics approaches that could quantify medical images into high-dimensional data have been shown to improve the imaging-based classification system in characterizing the heterogeneity of primary tumors and lymph nodes in EC patients. In this review, we aim to provide a comprehensive summary of the evidence of the most recent developments in ML application in imaging pertinent to EC patient care. According to the published results, ML models evaluating treatment response and lymph node metastasis achieve reliable predictions, ranging from acceptable to outstanding in their validation groups. Patients stratified by ML models in different risk groups have a significant or borderline significant difference in survival outcomes. Prospective large multi-center studies are suggested to improve the generalizability of ML techniques with standardized imaging protocols and harmonization between different centers.

A review of the application of machine learning in molecular imaging

Molecular imaging (MI) is a science that uses imaging methods to reflect the changes of molecular level in living state and conduct qualitative and quantitative studies on its biological behaviors in imaging. Optical molecular imaging (OMI) and nuclear medical imaging are two key research fields of MI. OMI technology refers to the optical information generated by the imaging target (such as tumors) due to drug intervention and other reasons. By collecting the optical information, researchers can track the motion trajectory of the imaging target at the molecular level. Owing to its high specificity and sensitivity, OMI has been widely used in preclinical research and clinical surgery. Nuclear medical imaging mainly detects ionizing radiation emitted by radioactive substances. It can provide molecular information for early diagnosis, effective treatment and basic research of diseases, which has become one of the frontiers and hot topics in the field of medicine in the world today. Both OMI and nuclear medical imaging technology require a lot of data processing and analysis. In recent years, artificial intelligence technology, especially neural network-based machine learning (ML) technology, has been widely used in MI because of its powerful data processing capability. It provides a feasible strategy to deal with large and complex data for the requirement of MI. In this review, we will focus on the applications of ML methods in OMI and nuclear medical imaging.

Considerable effects of imaging sequences, feature extraction, feature selection, and classifiers on radiomics-based prediction of microvascular invasion in hepatocellular carcinoma using magnetic resonance imaging

Background

Microvascular invasion (MVI) has a significant effect on the prognosis of hepatocellular carcinoma (HCC), but its preoperative identification is challenging. Radiomics features extracted from medical images, such as magnetic resonance (MR) images, can be used to predict MVI. In this study, we explored the effects of different imaging sequences, feature extraction and selection methods, and classifiers on the performance of HCC MVI predictive models.

Methods

After screening against the inclusion criteria, 69 patients with HCC and preoperative gadoxetic acid-enhanced MR images were enrolled. In total, 167 features were extracted from the MR images of each sequence for each patient. Experiments were designed to investigate the effects of imaging sequence, number of gray levels (Ng), quantization algorithm, feature selection method, and classifiers on the performance of radiomics biomarkers in the prediction of HCC MVI. We trained and tested these models using leave-one-out cross-validation (LOOCV).

Results

The radiomics model based on the images of the hepatobiliary phase (HBP) had better predictive performance than those based on the arterial phase (AP), portal venous phase (PVP), and pre-enhanced T1-weighted images [area under the receiver operating characteristic (ROC) curve (AUC) =0.792 vs. 0.641/0.634/0.620, P=0.041/0.021/0.010, respectively]. Compared with the equal-probability and Lloyd-Max algorithms, the radiomics features obtained using the Uniform quantization algorithm had a better performance (AUC =0.643/0.666 vs. 0.792, P=0.002/0.003, respectively). Among the values of 8, 16, 32, 64, and 128, the best predictive performance was achieved when the Ng was 64 (AUC =0.792 vs. 0.584/0.697/0.677/0.734, P<0.001/P=0.039/0.001/0.137, respectively). We used a two-stage feature selection method which combined the least absolute shrinkage and selection operator (LASSO) and recursive feature elimination (RFE) gradient boosting decision tree (GBDT), which achieved better stability than and outperformed LASSO, minimum redundancy maximum relevance (mRMR), and support vector machine (SVM)-RFE (stability =0.967 vs. 0.837/0.623/0.390, respectively; AUC =0.850 vs. 0.792/0.713/0.699, P=0.142/0.007/0.003, respectively). The model based on the radiomics features of HBP images using the GBDT classifier showed a better performance for the preoperative prediction of MVI compared with logistic regression (LR), SVM, and random forest (RF) classifiers (AUC =0.895 vs. 0.850/0.834/0.884, P=0.558/0.229/0.058, respectively). With the optimal combination of these factors, we established the best model, which had an AUC of 0.895, accuracy of 87.0%, specificity of 82.5%, and sensitivity of 93.1%.

Conclusions

Imaging sequences, feature extraction and selection methods, and classifiers can have a considerable effect on the predictive performance of radiomics models for HCC MVI.

PET Molecular Imaging: A Holistic Review of Current Practice and Emerging Perspectives for Diagnosis, Therapeutic Evaluation and Prognosis in Clinical Oncology

PET/CT molecular imaging has been imposed in clinical oncological practice over the past 20 years, driven by its two well-grounded foundations: quantification and radiolabeled molecular probe vectorization. From basic visual interpretation to more sophisticated full kinetic modeling, PET technology provides a unique opportunity to characterize various biological processes with different levels of analysis. In clinical practice, many efforts have been made during the last two decades to standardize image analyses at the international level, but advanced metrics are still under use in practice. In parallel, the integration of PET imaging with radionuclide therapy, also known as radiolabeled theranostics, has paved the way towards highly sensitive radionuclide-based precision medicine, with major breakthroughs emerging in neuroendocrine tumors and prostate cancer. PET imaging of tumor immunity and beyond is also emerging, emphasizing the unique capabilities of PET molecular imaging to constantly adapt to emerging oncological challenges. However, these new horizons face the growing complexity of multidimensional data. In the era of precision medicine, statistical and computer sciences are currently revolutionizing image-based decision making, paving the way for more holistic cancer molecular imaging analyses at the whole-body level.

Application of a Machine Learning Approach for the Analysis of Clinical and Radiomic Features of Pretreatment [18F]-FDG PET/CT to Predict Prognosis of Patients with Endometrial Cancer

PURPOSE

To examine the prognostic significance of pretreatment 2-deoxy-2-[¹⁸F]fluoro-D-glucose ([¹⁸F]-FDG) positron emission tomography (PET)-based radiomic features using a machine learning approach in patients with endometrial cancers.

PROCEDURES

Included in this retrospective study were 53 patients with endometrial cancers who underwent [¹⁸F]-FDG PET/X-ray computed tomography (CT) before treatment. Since two different PET scanners were used, post-reconstruction harmonization was performed for all PET parameters using the ComBat harmonization method. Four clinical (age, histological type, stage, and treatment method) and 40 [¹⁸F]-FDG PET-based radiomic features were ranked, and a subset of useful features was selected based on the decrease in the Gini impurity in terms of associations with disease progression. The machine learning algorithms (random forest, neural network, k-nearest neighbors (kNN), naive Bayes, logistic regression, and support vector machine) were compared using the areas under the receiver operating characteristic curve (AUC) and validated by the random sampling method. Progression-free survival (PFS) and overall survival (OS) were assessed by the Cox regression analysis.

RESULTS

The five best predictors of disease progression were coarseness, gray-level run length nonuniformity, stage, treatment method, and gray-level zone length nonuniformity. The kNN model obtained the best performance classifier for predicting the disease progression (AUC =0.890, accuracy =0.849, F1 score =0.848, precision =0.857, and recall =0.849). Coarseness which was the first ranked radiomic feature was selected for survival analyses, and only coarseness remained as a significant and independent factor for both PFS (hazard ratios (HR), 0.65; 95 % confidence interval [CI], 0.49-0.86; p=0.003) and OS (HR, 0.52; 95 % CI, 0.36-0.76; p<0.001) at multivariate Cox regression analysis.

CONCLUSIONS

[¹⁸F]-FDG PET-based radiomic analysis using a machine learning approach may be useful for predicting tumor progression and prognosis in patients with endometrial cancers.

Artificial Intelligence for Response Evaluation With PET/CT

Positron emission tomography (PET)/computed tomography (CT) are nuclear diagnostic imaging modalities that are routinely deployed for cancer staging and monitoring. They hold the advantage of detecting disease related biochemical and physiologic abnormalities in advance of anatomical changes, thus widely used for staging of disease progression, identification of the treatment gross tumor volume, monitoring of disease, as well as prediction of outcomes and personalization of treatment regimens. Among the arsenal of different functional imaging modalities, nuclear imaging has benefited from early adoption of quantitative image analysis starting from simple standard uptake value normalization to more advanced extraction of complex imaging uptake patterns; thanks to application of sophisticated image processing and machine learning algorithms. In this review, we discuss the application of image processing and machine/deep learning techniques to PET/CT imaging with special focus on the oncological radiotherapy domain as a case study and draw examples from our work and others to highlight current status and future potentials.

Machine and deep learning methods for radiomics

Radiomics is an emerging area in quantitative image analysis that aims to relate large-scale extracted imaging information to clinical and biological endpoints. The development of quantitative imaging methods along with machine learning has enabled the opportunity to move data science research towards translation for more personalized cancer treatments. Accumulating evidence has indeed demonstrated that noninvasive advanced imaging analytics, that is, radiomics, can reveal key components of tumor phenotype for multiple three-dimensional lesions at multiple time points over and beyond the course of treatment. These developments in the use of CT, PET, US, and MR imaging could augment patient stratification and prognostication buttressing emerging targeted therapeutic approaches. In recent years, deep learning architectures have demonstrated their tremendous potential for image segmentation, reconstruction, recognition, and classification. Many powerful open-source and commercial platforms are currently available to embark in new research areas of radiomics. Quantitative imaging research, however, is complex and key statistical principles should be followed to realize its full potential. The field of radiomics, in particular, requires a renewed focus on optimal study design/reporting practices and standardization of image acquisition, feature calculation, and rigorous statistical analysis for the field to move forward. In this article, the role of machine and deep learning as a major computational vehicle for advanced model building of radiomics-based signatures or classifiers, and diverse clinical applications, working principles, research opportunities, and available computational platforms for radiomics will be reviewed with examples drawn primarily from oncology. We also address issues related to common applications in medical physics, such as standardization, feature extraction, model building, and validation.

Application and Construction of Deep Learning Networks in Medical Imaging

Deep learning (DL) approaches are part of the machine learning (ML) subfield concerned with the development of computational models to train artificial intelligence systems. DL models are characterized by automatically extracting high-level features from the input data to learn the relationship between matching datasets. Thus, its implementation offers an advantage over common ML methods that often require the practitioner to have some domain knowledge of the input data to select the best latent representation. As a result of this advantage, DL has been successfully applied within the medical imaging field to address problems, such as disease classification and tumor segmentation for which it is difficult or impossible to determine which image features are relevant. Therefore, taking into consideration the positive impact of DL on the medical imaging field, this article reviews the key concepts associated with its evolution and implementation. The sections of this review summarize the milestones related to the development of the DL field, followed by a description of the elements of deep neural network and an overview of its application within the medical imaging field. Subsequently, the key steps necessary to implement a supervised DL application are defined, and associated limitations are discussed.

A Systematic Review of PET Textural Analysis and Radiomics in Cancer

Background: Although many works have supported the utility of PET radiomics, several authors have raised concerns over the robustness and replicability of the results. This study aimed to perform a systematic review on the topic of PET radiomics and the used methodologies. Methods: PubMed was searched up to 15 October 2020. Original research articles based on human data specifying at least one tumor type and PET image were included, excluding those that apply only first-order statistics and those including fewer than 20 patients. Each publication, cancer type, objective and several methodological parameters (number of patients and features, validation approach, among other things) were extracted. Results: A total of 290 studies were included. Lung (28%) and head and neck (24%) were the most studied cancers. The most common objective was prognosis/treatment response (46%), followed by diagnosis/staging (21%), tumor characterization (18%) and technical evaluations (15%). The average number of patients included was 114 (median = 71; range 20–1419), and the average number of high-order features calculated per study was 31 (median = 26, range 1–286). Conclusions: PET radiomics is a promising field, but the number of patients in most publications is insufficient, and very few papers perform in-depth validations. The role of standardization initiatives will be crucial in the upcoming years.

Deep Neural Network-Based Semantic Segmentation of Microvascular Decompression Images

Image semantic segmentation has been applied more and more widely in the fields of satellite remote sensing, medical treatment, intelligent transportation, and virtual reality. However, in the medical field, the study of cerebral vessel and cranial nerve segmentation based on true-color medical images is in urgent need and has good research and development prospects. We have extended the current state-of-the-art semantic-segmentation network DeepLabv3+ and used it as the basic framework. First, the feature distillation block (FDB) was introduced into the encoder structure to refine the extracted features. In addition, the atrous spatial pyramid pooling (ASPP) module was added to the decoder structure to enhance the retention of feature and boundary information. The proposed model was trained by fine tuning and optimizing the relevant parameters. Experimental results show that the encoder structure has better performance in feature refinement processing, improving target boundary segmentation precision, and retaining more feature information. Our method has a segmentation accuracy of 75.73%, which is 3% better than DeepLabv3+.

Predicting breast cancer response to neoadjuvant chemotherapy using ensemble deep transfer learning based on CT images

OBJECTIVE

To develop an ensemble a deep transfer learning model of CT images for predicting pathologic complete response (pCR) in breast cancer patients undergoing neoadjuvant chemotherapy (NAC).

METHODS

The data were obtained from the public dataset 'QIN-Breast' from The Cancer Imaging Archive (TCIA). CT images were gathered before and after the first cycle of NAC. CT images of 121 breast cancer patients were used to train and test the model. Among these patients, 58 achieved a pCR and 63 showed a non-pCR based pathology examination of surgical results after NAC. The dataset was split into training and testing subsets with a ratio of 7:3. In addition, the number of training samples in the dataset was increased from 656 to 1,968 by performing an image augmentation method. Two deep transfer learning models namely, DenseNet201 and ResNet152V2, and the ensemble model with a concatenation of two models, were trained and tested using CT images.

RESULTS

The ensemble model obtained the highest accuracy of 100% on the testing dataset. Furthermore, we received the best performance of 100% in recall, precision and f1-score value for the ensemble model. This supports the fact that the ensemble model results in better-generalized model and leads to efficient framework. Although a 0.004 and 0.003 difference were seen between the AUC of two base models (DenseNet201 and ResNet152V2) and the proposed ensemble, this increase in the model quality is critical in medical research. T-SNE revealed that in the proposed ensemble, no points were clustered into the wrong class. These results expose the strong performance of the proposed ensemble.

CONCLUSION

The study concluded that the ensemble model can increase the ability to predict breast cancer response to first-cycle NAC than two DenseNet201 and ResNet152V2 models.

Predicting aggressive histopathological features in esophageal cancer with positron emission tomography using a deep convolutional neural network

Background

The presence of lymphovascular invasion (LVI) and perineural invasion (PNI) are of great prognostic importance in esophageal squamous cell carcinoma. Currently, positron emission tomography (PET) scans are the only means of functional assessment prior to treatment. We aimed to predict the presence of LVI and PNI in esophageal squamous cell carcinoma using PET imaging data by training a three-dimensional convolution neural network (3D-CNN).

Methods

Seven hundred and ninety-eight PET scans of patients with esophageal squamous cell carcinoma and 309 PET scans of patients with stage I lung cancer were collected. In the first part of this study, we built a 3D-CNN based on a residual network, ResNet, for a task to classify the scans into esophageal cancer or lung cancer. In the second stage, we collected the PET scans of 278 patients undergoing esophagectomy for a task to classify and predict the presence of LVI/PNI.

Results

In the first part, the model performance attained an area under the receiver operating characteristic curve (AUC) of 0.860. In the second part, we randomly split 80%, 10%, and 10% of our dataset into training set, validation set and testing set, respectively, for a task to classify the scans into the presence of LVI/PNI and evaluated the model performance on the testing set. Our 3D-CNN model attained an AUC of 0.668 in the testing set, which shows a better discriminative ability than random guessing.

Conclusions

A 3D-CNN can be trained, using PET imaging datasets, to predict LNV/PNI in esophageal cancer with acceptable accuracy.

Advanced analytics and artificial intelligence in gastrointestinal cancer: a systematic review of radiomics predicting response to treatment

Purpose

Advanced medical image analytics is increasingly used to predict clinical outcome in patients diagnosed with gastrointestinal tumors. This review provides an overview on the value of radiomics in predicting response to treatment in patients with gastrointestinal tumors.

Methods

A systematic review was conducted, according to PRISMA guidelines. The protocol was prospectively registered (PROSPERO: CRD42019128408). PubMed, Embase, and Cochrane databases were searched. Original studies reporting on the value of radiomics in predicting response to treatment in patients with a gastrointestinal tumor were included. A narrative synthesis of results was conducted. Results were stratified by tumor type. Quality assessment of included studies was performed, according to the radiomics quality score.

Results

The comprehensive literature search identified 1360 unique studies, of which 60 articles were included for analysis. In 37 studies, radiomics models and individual radiomic features showed good predictive performance for response to treatment (area under the curve or accuracy > 0.75). Various strategies to construct predictive models were used. Internal validation of predictive models was often performed, while the majority of studies lacked external validation. None of the studies reported predictive models implemented in clinical practice.

Conclusion

Radiomics is increasingly used to predict response to treatment in patients suffering from gastrointestinal cancer. This review demonstrates its great potential to help predict response to treatment and improve patient selection and early adjustment of treatment strategy in a non-invasive manner.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-020-05142-w.

Artificial intelligence and hybrid imaging: the best match for personalized medicine in oncology

Artificial intelligence (AI) refers to a field of computer science aimed to perform tasks typically requiring human intelligence. Currently, AI is recognized in the broader technology radar within the five key technologies which emerge for their wide-ranging applications and impact in communities, companies, business, and value chain framework alike. However, AI in medical imaging is at an early phase of development, and there are still hurdles to take related to reliability, user confidence, and adoption. The present narrative review aimed to provide an overview on AI-based approaches (distributed learning, statistical learning, computer-aided diagnosis and detection systems, fully automated image analysis tool, natural language processing) in oncological hybrid medical imaging with respect to clinical tasks (detection, contouring and segmentation, prediction of histology and tumor stage, prediction of mutational status and molecular therapies targets, prediction of treatment response, and outcome). Particularly, AI-based approaches have been briefly described according to their purpose and, finally lung cancer-being one of the most extensively malignancy studied by hybrid medical imaging-has been used as illustrative scenario. Finally, we discussed clinical challenges and open issues including ethics, validation strategies, effective data-sharing methods, regulatory hurdles, educational resources, and strategy to facilitate the interaction among different stakeholders. Some of the major changes in medical imaging will come from the application of AI to workflow and protocols, eventually resulting in improved patient management and quality of life. Overall, several time-consuming tasks could be automatized. Machine learning algorithms and neural networks will permit sophisticated analysis resulting not only in major improvements in disease characterization through imaging, but also in the integration of multiple-omics data (i.e., derived from pathology, genomic, proteomics, and demographics) for multi-dimensional disease featuring. Nevertheless, to accelerate the transition of the theory to practice a sustainable development plan considering the multi-dimensional interactions between professionals, technology, industry, markets, policy, culture, and civil society directed by a mindset which will allow talents to thrive is necessary.

Early prediction of neoadjuvant chemotherapy response for advanced breast cancer using PET/MRI image deep learning

This study aimed to investigate the predictive efficacy of positron emission tomography/computed tomography (PET/CT) and magnetic resonance imaging (MRI) for the pathological response of advanced breast cancer to neoadjuvant chemotherapy (NAC). The breast PET/MRI image deep learning model was introduced and compared with the conventional methods. PET/CT and MRI parameters were evaluated before and after the first NAC cycle in patients with advanced breast cancer [n = 56; all women; median age, 49 (range 26–66) years]. The maximum standardized uptake value (SUVmax), metabolic tumor volume (MTV), and total lesion glycolysis (TLG) were obtained with the corresponding baseline values (SUV0, MTV0, and TLG0, respectively) and interim PET images (SUV1, MTV1, and TLG1, respectively). Mean apparent diffusion coefficients were obtained from baseline and interim diffusion MR images (ADC0 and ADC1, respectively). The differences between the baseline and interim parameters were measured (ΔSUV, ΔMTV, ΔTLG, and ΔADC). Subgroup analysis was performed for the HER2-negative and triple-negative groups. Datasets for convolutional neural network (CNN), assigned as training (80%) and test datasets (20%), were cropped from the baseline (PET0, MRI0) and interim (PET1, MRI1) images. Histopathologic responses were assessed using the Miller and Payne system, after three cycles of chemotherapy. Receiver operating characteristic curve analysis was used to assess the performance of the differentiating responders and non-responders. There were six responders (11%) and 50 non-responders (89%). The area under the curve (AUC) was the highest for ΔSUV at 0.805 (95% CI 0.677–0.899). The AUC was the highest for ΔSUV at 0.879 (95% CI 0.722–0.965) for the HER2-negative subtype. AUC improved following CNN application (SUV0:PET0 = 0.652:0.886, SUV1:PET1 = 0.687:0.980, and ADC1:MRI1 = 0.537:0.701), except for ADC0 (ADC0:MRI0 = 0.703:0.602). PET/MRI image deep learning model can predict pathological responses to NAC in patients with advanced breast cancer.

Transcriptomic biomarkers for predicting response to neoadjuvant treatment in oesophageal cancer

Oesophageal cancer is a devastating disease with poor outcomes and is the sixth leading cause of cancer death worldwide. In the setting of resectable disease, there is clear evidence that neoadjuvant chemotherapy and chemoradiotherapy result in improved survival. Disappointingly, only 15%-30% of patients obtain a histopathological response to neoadjuvant therapy, often at the expense of significant toxicity. There are no predictive biomarkers in routine clinical use in this setting and the ability to stratify patients for treatment could dramatically improve outcomes. In this review, we aim to outline current progress in evaluating predictive transcriptomic biomarkers for neoadjuvant therapy in oesophageal cancer and discuss the challenges facing biomarker development in this setting. We place these issues in the wider context of recommendations for biomarker development and reporting. The majority of studies focus on messenger RNA (mRNA) and microRNA (miRNA) biomarkers. These studies report a range of different genes involved in a wide variety of pathways and biological processes, and this is explained to a large extent by the different platforms and analysis methods used. Many studies are also vastly underpowered so are not suitable for identifying a candidate biomarker. Multiple molecular subtypes of oesophageal cancer have been proposed, although little is known about how these relate to clinical outcomes. We anticipate that the accumulating wealth of genomic and transcriptomic data and clinical trial collaborations in the coming years will provide unique opportunities to stratify patients in this poor-prognosis disease and recommend that future biomarker development incorporates well-designed retrospective and prospective analyses.

Radiomics in PET/CT: Current Status and Future AI-Based Evolutions

This short review aims at providing the readers with an update on the current status, as well as future perspectives in the quickly evolving field of radiomics applied to the field of PET/CT imaging. Numerous pitfalls have been identified in study design, data acquisition, segmentation, features calculation and modeling by the radiomics community, and these are often the same issues across all image modalities and clinical applications, however some of these are specific to PET/CT (and SPECT/CT) imaging and therefore the present paper focuses on those. In most cases, recommendations and potential methodological solutions do exist and should therefore be followed to improve the overall quality and reproducibility of published studies. In terms of future evolutions, the techniques from the larger field of artificial intelligence (AI), including those relying on deep neural networks (also known as deep learning) have already shown impressive potential to provide solutions, especially in terms of automation, but also to maybe fully replace the tools the radiomics community has been using until now in order to build the usual radiomics workflow. Some important challenges remain to be addressed before the full impact of AI may be realized but overall the field has made striking advances over the last few years and it is expected advances will continue at a rapid pace.

A Role for FDG PET Radiomics in Personalized Medicine?

Radiomics describes the extraction of multiple features from medical images, including molecular imaging modalities, that with bioinformatic approaches, provide additional clinically relevant information that may be invisible to the human eye. This information may complement standard radiological interpretation with data that may better characterize a disease or that may provide predictive or prognostic information. Progressing from predefined image features, often describing heterogeneity of voxel intensities within a volume of interest, there is increasing use of machine learning to classify disease characteristics and deep learning methods based on artificial neural networks that can learn features without a priori definition and without the need for preprocessing of images. There have been advances in standardization and harmonization of methods to a level that should support multicenter studies. However, in this relatively early phase of research in the field, there are limited aspects that have been adopted into routine practice. Most of the reports in the molecular imaging field describe radiomic approaches in cancer using 18F-fluorodeoxyglucose positron emission tomography (18F-FDG-PET). In this review, we will describe radiomics in molecular imaging and summarize the pertinent literature in lung cancer where reports are most prevalent and mature.