Automatic detection and classification of regions of FDG uptake in whole-body PET-CT lymphoma studies

Role of Novel Quantitative Imaging Techniques in Hematological Malignancies

Hematological malignancies exhibit a widespread distribution, necessitating evaluation of disease activity over the entire body. In clinical practice, visual analysis and semiquantitative parameters are used to assess 18F-FDGPET/CT imaging, which solely represents measurements of disease activity from limited area and may not adequately reflect global disease assessment. An efficient method for assessing the global disease burden of hematological malignancies is to employ PET/computed tomography based novel quantitative parameters. In this article, we explored novel quantitative parameters on PET/CT imaging for assessing global disease burden and the potential role of artificial intelligence (AI) to determine these parameters in evaluation of hematological malignancies.

Review of artificial intelligence clinical applications in Nuclear Medicine

This paper provides an in-depth analysis of the clinical applications of artificial intelligence (AI) in Nuclear Medicine, focusing on three key areas: neurology, cardiology, and oncology. Beginning with neurology, specifically Alzheimer's disease and Parkinson's disease, the paper examines reviews on diagnosis and treatment planning. The same pattern is followed in cardiology studies. In the final section on oncology, the paper explores the various AI applications in multiple cancer types, including lung, head and neck, lymphoma, and pancreatic cancer.

An artificial intelligence method using FDG PET to predict treatment outcome in diffuse large B cell lymphoma patients

Convolutional neural networks (CNNs) may improve response prediction in diffuse large B-cell lymphoma (DLBCL). The aim of this study was to investigate the feasibility of a CNN using maximum intensity projection (MIP) images from 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) baseline scans to predict the probability of time-to-progression (TTP) within 2 years and compare it with the International Prognostic Index (IPI), i.e. a clinically used score. 296 DLBCL 18F-FDG PET/CT baseline scans collected from a prospective clinical trial (HOVON-84) were analysed. Cross-validation was performed using coronal and sagittal MIPs. An external dataset (340 DLBCL patients) was used to validate the model. Association between the probabilities, metabolic tumour volume and Dmaxbulk was assessed. Probabilities for PET scans with synthetically removed tumors were also assessed. The CNN provided a 2-year TTP prediction with an area under the curve (AUC) of 0.74, outperforming the IPI-based model (AUC = 0.68). Furthermore, high probabilities (> 0.6) of the original MIPs were considerably decreased after removing the tumours (< 0.4, generally). These findings suggest that MIP-based CNNs are able to predict treatment outcome in DLBCL.

Role of Artificial Intelligence in PET/CT Imaging for Management of Lymphoma

Our review shows that AI-based analysis of lymphoma whole-body FDG-PET/CT can inform all phases of clinical management including staging, prognostication, treatment planning, and treatment response evaluation. We highlight advancements in the role of neural networks for performing automated image segmentation to calculate PET-based imaging biomarkers such as the total metabolic tumor volume (TMTV). AI-based image segmentation methods are at levels where they can be semi-automatically implemented with minimal human inputs and nearing the level of a second-opinion radiologist. Advances in automated segmentation methods are particularly apparent in the discrimination of lymphomatous vs non-lymphomatous FDG-avid regions, which carries through to automated staging. Automated TMTV calculators, in addition to automated calculation of measures such as Dmax are informing robust models of progression-free survival which can then feed into improved treatment planning.

PSR-Nets: Deep neural networks with prior shift regularization for PET/CT based automatic, accurate, and calibrated whole-body lymphoma segmentation

Lymphoma is a type of lymphatic tissue originated cancer. Automatic and accurate lymphoma segmentation is critical for its diagnosis and prognosis yet challenging due to the severely class-imbalanced problem. Generally, deep neural networks trained with class-observation-frequency based re-weighting loss functions are used to address this problem. However, the majority class can be under-weighted by them, due to the existence of data overlap. Besides, they are more mis-calibrated. To resolve these, we propose a neural network with prior-shift regularization (PSR-Net), which comprises a UNet-like backbone with re-weighting loss functions, and a prior-shift regularization (PSR) module including a prior-shift layer (PSL), a regularizer generation layer (RGL), and an expected prediction confidence updating layer (EPCUL). We first propose a trainable expected prediction confidence (EPC) for each class. Periodically, PSL shifts a prior training dataset to a more informative dataset based on EPCs; RGL presents a generalized informative-voxel-aware (GIVA) loss with EPCs and calculates it on the informative dataset for model finetuning in back-propagation; and EPCUL updates EPCs to refresh PSL and RRL in next forward-propagation. PSR-Net is trained in a two- stage manner. The backbone is first trained with re-weighting loss functions, then we reload the best saved model for the backbone and continue to train it with the weighted sum of the re-weighting loss functions, the GIVA regularizer and the L2 loss function of EPCs for regularization fine-tuning. Extensive experiments are performed based on PET/CT volumes with advanced stage lymphomas. Our PSR-Net achieves 95.12% sensitivity and 87.18% Dice coefficient, demonstrating the effectiveness of PSR-Net, when compared to the baselines and the state-of-the-arts.

Multimodal deep learning model on interim [18F]FDG PET/CT for predicting primary treatment failure in diffuse large B-cell lymphoma

OBJECTIVES

The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim ¹⁸F-fluoro-2-deoxyglucose ([¹⁸F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL.

METHODS

Initially, 205 DLBCL patients undergoing interim [¹⁸F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance.

RESULTS

The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability.

CONCLUSIONS

The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment.

KEY POINTS

• The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.

Artificial Intelligence in Lymphoma PET Imaging:: A Scoping Review (Current Trends and Future Directions)

Malignant lymphomas are a family of heterogenous disorders caused by clonal proliferation of lymphocytes. 18F-FDG-PET has proven to provide essential information for accurate quantification of disease burden, treatment response evaluation, and prognostication. However, manual delineation of hypermetabolic lesions is often a time-consuming and impractical task. Applications of artificial intelligence (AI) may provide solutions to overcome this challenge. Beyond segmentation and detection of lesions, AI could enhance tumor characterization and heterogeneity quantification, as well as treatment response prediction and recurrence risk stratification. In this scoping review, we have systematically mapped and discussed the current applications of AI (such as detection, classification, segmentation as well as the prediction and prognostication) in lymphoma PET.

AI-Based Detection, Classification and Prediction/Prognosis in Medical Imaging:: Towards Radiophenomics

Artificial intelligence (AI) techniques have significant potential to enable effective, robust, and automated image phenotyping including the identification of subtle patterns. AI-based detection searches the image space to find the regions of interest based on patterns and features. There is a spectrum of tumor histologies from benign to malignant that can be identified by AI-based classification approaches using image features. The extraction of minable information from images gives way to the field of "radiomics" and can be explored via explicit (handcrafted/engineered) and deep radiomics frameworks. Radiomics analysis has the potential to be used as a noninvasive technique for the accurate characterization of tumors to improve diagnosis and treatment monitoring. This work reviews AI-based techniques, with a special focus on oncological PET and PET/CT imaging, for different detection, classification, and prediction/prognosis tasks. We also discuss needed efforts to enable the translation of AI techniques to routine clinical workflows, and potential improvements and complementary techniques such as the use of natural language processing on electronic health records and neuro-symbolic AI techniques.

Toward High-Throughput Artificial Intelligence-Based Segmentation in Oncological PET Imaging

Artificial intelligence (AI) techniques for image-based segmentation have garnered much attention in recent years. Convolutional neural networks have shown impressive results and potential toward fully automated segmentation in medical imaging, and particularly PET imaging. To cope with the limited access to annotated data needed in supervised AI methods, given tedious and prone-to-error manual delineations, semi-supervised and unsupervised AI techniques have also been explored for segmentation of tumors or normal organs in single- and bimodality scans. This work reviews existing AI techniques for segmentation tasks and the evaluation criteria for translational AI-based segmentation efforts toward routine adoption in clinical workflows.

Machine learning for evolutive lymphoma and residual masses recognition in whole body diffusion weighted magnetic resonance images

BACKGROUND

After the treatment of the patients with malignant lymphoma, there may persist lesions that must be labeled either as evolutive lymphoma requiring new treatments or as residual masses. We present in this work, a machine learning-based computer-aided diagnosis (CAD) applied to whole-body diffusion-weighted magnetic resonance images.

METHODS

The database consists of a total of 1005 MRI images with evolutive lymphoma and residual masses. More specifically, we propose a novel approach that leverages: (1)-The complementarity of the functional and anatomical criteria of MRI images through a fusion step based on the discrete wavelet transforms (DWT). (2)- The automatic segmentation of the lesions, their localization, and their enumeration using the Chan-Vese algorithm. (3)- The generation of the parametric image which contains the apparent diffusion coefficient value named ADC map. (4)- The features selection through the application of the sequential forward selection (SFS), Entropy, Symmetric uncertainty and Gain Ratio algorithm on 72 extracted features. (5)- The classification of the lesions by applying five well known supervised machine learning classification algorithms: the back-propagation artificial neural network (ANN), the support vector machine (SVM), the K-nearest neighbours (K-NN), Relevance Vectors Machine (RVM), and the random forest (RF) compared to deep learning based on convolutional neural network (CNN). Moreover, this study is achieved with an evaluation of the classification using 335 DW-MR images where 80% of them are used for the training and the remaining 20% for the test.

RESULTS

The obtained accuracy for the five classifiers recorded a slight superiority to the proposed method based on the back-propagation 3-9-1 ANN model which reaches 96,5%. In addition, we compared the proposed method to five other works from the literature. The proposed method gives much better results in terms of SE, SP, accuracy, F₁-measure, and geometric-mean which reaches respectively 96.4%, 90.9%, 95.5%, 0.97, and 91.61%.

CONCLUSIONS

Our initial results suggest that Combining functional, anatomical, and morphological features of ROI's have very good accuracy (97.01%) for evolutive lymphoma and residual masses recognition when we based on the new proposed approach using the back-propagation 3-9-1 ANN model. Proposed method based on machine learning gives less than Deep learning CNN, which is 98.5%.

HD-RDS-UNet: Leveraging Spatial-Temporal Correlation between the Decoder Feature Maps for Lymphoma Segmentation

Lymphoma is a group of malignant tumors originated in the lymphatic system. Automatic and accurate lymphoma segmentation in PET/CT volumes is critical yet challenging in the clinical practice. Recently, UNet-like architectures are widely used for medical image segmentation. The pure UNet-like architectures model the spatial correlation between the feature maps very well, whereas they discard the critical temporal correlation. Some prior work combines UNet with recurrent neural networks (RNNs) to utilize the spatial and temporal correlation simultaneously. However, it is inconvenient to incorporate some advanced techniques for UNet to RNNs, which hampers their further improvements. In this paper, we propose a recurrent dense siamese decoder architecture, which simulates RNNs and can densely utilize the spatial-temporal correlation between the decoder feature maps following a UNet approach. We combine it with a modified hyper dense encoder. Therefore, the proposed model is a UNet with a hyper dense encoder and a recurrent dense siamese decoder (HD-RDS-UNet). To stabilize the training process, we propose a weighted Dice loss with stable gradient and self-adaptive parameters. We perform patient-independent fivefold cross-validation on 3D volumes collected from whole-body PET/CT scans of patients with lymphomas. The experimental results show that the volume-wise average Dice score and sensitivity are 85.58% and 94.63%, respectively. The patient-wise average Dice score and sensitivity are 85.85% and 95.01%, respectively. The different configurations of HD-RDS-UNet consistently show superiority in the performance comparison. Besides, a trained HD-RDS-UNet can be easily pruned, resulting in significantly reduced inference time and memory usage, while keeping very good segmentation performance.

Diffuse large B-cell lymphoma segmentation in PET-CT images via hybrid learning for feature fusion

PURPOSE

Diffuse large B-cell lymphoma (DLBCL) is an aggressive type of lymphoma with high mortality and poor prognosis that especially has a high incidence in Asia. Accurate segmentation of DLBCL lesions is crucial for clinical radiation therapy. However, manual delineation of DLBCL lesions is tedious and time-consuming. Automatic segmentation provides an alternative solution but is difficult for diffuse lesions without the sufficient utilization of multimodality information. Our work is the first study focusing on positron emission tomography and computed tomography (PET-CT) feature fusion for the DLBCL segmentation issue. We aim to improve the fusion performance of complementary information contained in PET-CT imaging with a hybrid learning module in the supervised convolutional neural network.

METHODS

First, two encoder branches extract single-modality features, respectively. Next, the hybrid learning component utilizes them to generate spatial fusion maps which can quantify the contribution of complementary information. Such feature fusion maps are then concatenated with specific-modality (i.e., PET and CT) feature maps to obtain a representation of the final fused feature maps in different scales. Finally, the reconstruction part of our network creates a prediction map of DLBCL lesions by integrating and up-sampling the final fused feature maps from encoder blocks in different scales.

RESULTS

The ability of our method was evaluated to detect foreground and segment lesions in three independent body regions (nasopharynx, chest, and abdomen) of a set of 45 PET-CT scans. Extensive ablation experiments compared our method to four baseline techniques for multimodality fusion (input-level (IL) fusion, multichannel (MC) strategy, multibranch (MB) strategy, and quantitative weighting (QW) fusion). The results showed that our method achieved a high detection accuracy (99.63% in the nasopharynx, 99.51% in the chest, and 99.21% in the abdomen) and had the superiority in segmentation performance with the mean dice similarity coefficient (DSC) of 73.03% and the modified Hausdorff distance (MHD) of 4.39 mm, when compared with the baselines (DSC: IL: 53.08%, MC: 63.59%, MB: 69.98%, and QW: 72.19%; MHD: IL: 12.16 mm, MC: 6.46 mm, MB: 4.83 mm, and QW: 4.89 mm).

CONCLUSIONS

A promising segmentation method has been proposed for the challenging DLBCL lesions in PET-CT images, which improves the understanding of complementary information by feature fusion and may guide clinical radiotherapy. The statistically significant analysis based on P-value calculation has indicated a degree of significant difference between our proposed method and other baselines (almost metrics: P < 0.05). This is a preliminary research using a small sample size, and we will collect data continually to achieve the larger verification study.

Applications of artificial intelligence in oncologic 18F-FDG PET/CT imaging: a systematic review

Artificial intelligence (AI) is a growing field of research that is emerging as a promising adjunct to assist physicians in detection and management of patients with cancer. ¹⁸F-FDG PET imaging helps physicians in detection and management of patients with cancer. In this study we discuss the possible applications of AI in ¹⁸F-FDG PET imaging based on the published studies. A systematic literature review was performed in PubMed on early August 2020 to find the relevant studies. A total of 65 studies were available for review against the inclusion criteria which included studies that developed an AI model based on 18F-FDG PET data in cancer to diagnose, differentiate, delineate, stage, assess response to therapy, determine prognosis, or improve image quality. Thirty-two studies met the inclusion criteria and are discussed in this review. The majority of studies are related to lung cancer. Other studied cancers included breast cancer, cervical cancer, head and neck cancer, lymphoma, pancreatic cancer, and sarcoma. All studies were based on human patients except for one which was performed on rats. According to the included studies, machine learning (ML) models can help in detection, differentiation from benign lesions, segmentation, staging, response assessment, and prognosis determination. Despite the potential benefits of AI in cancer imaging and management, the routine implementation of AI-based models and ¹⁸F-FDG PET-derived radiomics in clinical practice is limited at least partially due to lack of standardized, reproducible, generalizable, and precise techniques.

Automated quantification of baseline imaging PET metrics on FDG PET/CT images of pediatric Hodgkin lymphoma patients

PURPOSE

For pediatric lymphoma, quantitative FDG PET/CT imaging features such as metabolic tumor volume (MTV) are important for prognosis and risk stratification strategies. However, feature extraction is difficult and time-consuming in cases of high disease burden. The purpose of this study was to fully automate the measurement of PET imaging features in PET/CT images of pediatric lymphoma.

METHODS

18F-FDG PET/CT baseline images of 100 pediatric Hodgkin lymphoma patients were retrospectively analyzed. Two nuclear medicine physicians identified and segmented FDG avid disease using PET thresholding methods. Both PET and CT images were used as inputs to a three-dimensional patch-based, multi-resolution pathway convolutional neural network architecture, DeepMedic. The model was trained to replicate physician segmentations using an ensemble of three networks trained with 5-fold cross-validation. The maximum SUV (SUVmax), MTV, total lesion glycolysis (TLG), surface-area-to-volume ratio (SA/MTV), and a measure of disease spread (Dmaxpatient) were extracted from the model output. Pearson's correlation coefficient and relative percent differences were calculated between automated and physician-extracted features.

RESULTS

Median Dice similarity coefficient of patient contours between automated and physician contours was 0.86 (IQR 0.78-0.91). Automated SUVmax values matched exactly the physician determined values in 81/100 cases, with Pearson's correlation coefficient (R) of 0.95. Automated MTV was strongly correlated with physician MTV (R = 0.88), though it was slightly underestimated with a median (IQR) relative difference of - 4.3% (- 10.0-5.7%). Agreement of TLG was excellent (R = 0.94), with median (IQR) relative difference of - 0.4% (- 5.2-7.0%). Median relative percent differences were 6.8% (R = 0.91; IQR 1.6-4.3%) for SA/MTV, and 4.5% (R = 0.51; IQR - 7.5-40.9%) for Dmaxpatient, which was the most difficult feature to quantify automatically.

CONCLUSIONS

An automated method using an ensemble of multi-resolution pathway 3D CNNs was able to quantify PET imaging features of lymphoma on baseline FDG PET/CT images with excellent agreement to reference physician PET segmentation. Automated methods with faster throughput for PET quantitation, such as MTV and TLG, show promise in more accessible clinical and research applications.

Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network

PURPOSE

Lymphoma lesion detection and segmentation on whole-body FDG-PET/CT are a challenging task because of the diversity of involved nodes, organs or physiological uptakes. We sought to investigate the performances of a three-dimensional (3D) convolutional neural network (CNN) to automatically segment total metabolic tumour volume (TMTV) in large datasets of patients with diffuse large B cell lymphoma (DLBCL).

METHODS

The dataset contained pre-therapy FDG-PET/CT from 733 DLBCL patients of 2 prospective LYmphoma Study Association (LYSA) trials. The first cohort (n = 639) was used for training using a 5-fold cross validation scheme. The second cohort (n = 94) was used for external validation of TMTV predictions. Ground truth masks were manually obtained after a 41% SUVmax adaptive thresholding of lymphoma lesions. A 3D U-net architecture with 2 input channels for PET and CT was trained on patches randomly sampled within PET/CTs with a summed cross entropy and Dice similarity coefficient (DSC) loss. Segmentation performance was assessed by the DSC and Jaccard coefficients. Finally, TMTV predictions were validated on the second independent cohort.

RESULTS

Mean DSC and Jaccard coefficients (± standard deviation) in the validations set were 0.73 ± 0.20 and 0.68 ± 0.21, respectively. An underestimation of mean TMTV by - 12 mL (2.8%) ± 263 was found in the validation sets of the first cohort (P = 0.27). In the second cohort, an underestimation of mean TMTV by - 116 mL (20.8%) ± 425 was statistically significant (P = 0.01).

CONCLUSION

Our CNN is a promising tool for automatic detection and segmentation of lymphoma lesions, despite slight underestimation of TMTV. The fully automatic and open-source features of this CNN will allow to increase both dissemination in routine practice and reproducibility of TMTV assessment in lymphoma patients.

Machine Learning in Nuclear Medicine: Part 2-Neural Networks and Clinical Aspects

This article is the second part in our machine learning series. Part 1 provided a general overview of machine learning in nuclear medicine. Part 2 focuses on neural networks. We start with an example illustrating how neural networks work and a discussion of potential applications. Recognizing that there is a spectrum of applications, we focus on recent publications in the areas of image reconstruction, low-dose PET, disease detection, and models used for diagnosis and outcome prediction. Finally, since the way machine learning algorithms are reported in the literature is extremely variable, we conclude with a call to arms regarding the need for standardized reporting of design and outcome metrics and we propose a basic checklist our community might follow going forward.

Convolutional Neural Networks for Automated PET/CT Detection of Diseased Lymph Node Burden in Patients with Lymphoma

PURPOSE

To automatically detect lymph nodes involved in lymphoma on fluorine 18 (¹⁸F) fluorodeoxyglucose (FDG) PET/CT images using convolutional neural networks (CNNs).

MATERIALS AND METHODS

In this retrospective study, baseline disease of 90 patients with lymphoma was segmented on ¹⁸F-FDG PET/CT images (acquired between 2005 and 2011) by a nuclear medicine physician. An ensemble of three-dimensional patch-based, multiresolution pathway CNNs was trained using fivefold cross-validation. Performance was assessed using the true-positive rate (TPR) and number of false-positive (FP) findings. CNN performance was compared with agreement between physicians by comparing the annotations of a second nuclear medicine physician to the first reader in 20 of the patients. Patient TPR was compared using Wilcoxon signed rank tests.

RESULTS

Across all 90 patients, a range of 0-61 nodes per patient was detected. At an average of four FP findings per patient, the method achieved a TPR of 85% (923 of 1087 nodes). Performance varied widely across patients (TPR range, 33%-100%; FP range, 0-21 findings). In the 20 patients labeled by both physicians, a range of 1-49 nodes per patient was detected and labeled. The second reader identified 96% (210 of 219) of nodes with an additional 3.7 per patient compared with the first reader. In the same 20 patients, the CNN achieved a 90% (197 of 219) TPR at 3.7 FP findings per patient.

CONCLUSION

An ensemble of three-dimensional CNNs detected lymph nodes at a performance nearly comparable to differences between two physicians' annotations. This preliminary study is a first step toward automated PET/CT assessment for lymphoma.© RSNA, 2020.

Deep neural network for automatic characterization of lesions on 68Ga-PSMA-11 PET/CT

PURPOSE

This study proposes an automated prostate cancer (PC) lesion characterization method based on the deep neural network to determine tumor burden on ⁶⁸Ga-PSMA-11 PET/CT to potentially facilitate the optimization of PSMA-directed radionuclide therapy.

METHODS

We collected ⁶⁸Ga-PSMA-11 PET/CT images from 193 patients with metastatic PC at three medical centers. For proof-of-concept, we focused on the detection of pelvis bone and lymph node lesions. A deep neural network (triple-combining 2.5D U-Net) was developed for the automated characterization of these lesions. The proposed method simultaneously extracts features from axial, coronal, and sagittal planes, which mimics the workflow of physicians and reduces computational and memory requirements.

RESULTS

Among all the labeled lesions, the network achieved 99% precision, 99% recall, and an F1 score of 99% on bone lesion detection and 94%, precision 89% recall, and an F1 score of 92% on lymph node lesion detection. The segmentation accuracy is lower than the detection. The performance of the network was correlated with the amount of training data.

CONCLUSION

We developed a deep neural network to characterize automatically the PC lesions on ⁶⁸Ga-PSMA-11 PET/CT. The preliminary test within the pelvic area confirms the potential of deep learning methods. Increasing the amount of training data should further enhance the performance of the proposed method and may ultimately allow whole-body assessments.

Artificial intelligence applications for pediatric oncology imaging

Machine learning algorithms can help to improve the accuracy and efficiency of cancer diagnosis, selection of personalized therapies and prediction of long-term outcomes. Artificial intelligence (AI) describes a subset of machine learning that can identify patterns in data and take actions to reach pre-set goals without specific programming. Machine learning tools can help to identify high-risk populations, prescribe personalized screening tests and enrich patient populations that are most likely to benefit from advanced imaging tests. AI algorithms can also help to plan personalized therapies and predict the impact of genomic variations on the sensitivity of normal and tumor tissue to chemotherapy or radiation therapy. The two main bottlenecks for successful AI applications in pediatric oncology imaging to date are the needs for large data sets and appropriate computer and memory power. With appropriate data entry and processing power, deep convolutional neural networks (CNNs) can process large amounts of imaging data, clinical data and medical literature in very short periods of time and thereby accelerate literature reviews, correct diagnoses and personalized treatments. This article provides a focused review of emerging AI applications that are relevant for the pediatric oncology imaging community.

Detection and segmentation of lymphomas in 3D PET images via clustering with entropy-based optimization strategy

PURPOSE

Lymphoma detection and segmentation from PET images are critical tasks for cancer staging and treatment monitoring. However, it is still a challenge owing to the complexities of lymphoma PET data themselves, and the huge computational burdens and memory requirements for 3D volume data. In this work, an entropy-based optimization strategy for clustering is proposed to detect and segment lymphomas in 3D PET images.

METHODS

To reduce computational complexity and add more feature information, billions of voxels in 3D volume data are first aggregated into supervoxels. Then, such supervoxels serve as basic data units for further clustering by using DBSCAN algorithm, in which some new feature attributes based on physical spatial information and prior knowledge are proposed. In addition, more importantly, an entropy-based objective function is constructed to search the most appropriate parameters of DBSCAN to obtain the optimal clustering results by using a genetic algorithm. This step allows to automatically adapt the parameters to each patient. Finally, a series of comparison experiments among various feature attributes are performed.

RESULTS

48 patient data are conducted, showing the combination of three features, supervoxel intensity, geographic coordinates and organ distributions, can achieve good performance and the proposed entropy-based optimization scheme has more advantages than the existing methods.

CONCLUSION

The proposed entropy-based optimization strategy for clustering by integrating physical spatial attributes and prior knowledge can achieve better performance than traditional methods.

Co-Learning Feature Fusion Maps from PET-CT Images of Lung Cancer

The analysis of multi-modality positron emission tomography and computed tomography (PET-CT) images for computer aided diagnosis applications (e.g., detection and segmentation) requires combining the sensitivity of PET to detect abnormal regions with anatomical localization from CT. Current methods for PET-CT image analysis either process the modalities separately or fuse information from each modality based on knowledge about the image analysis task. These methods generally do not consider the spatially varying visual characteristics that encode different information across the different modalities, which have different priorities at different locations. For example, a high abnormal PET uptake in the lungs is more meaningful for tumor detection than physiological PET uptake in the heart. Our aim is to improve fusion of the complementary information in multi-modality PET-CT with a new supervised convolutional neural network (CNN) that learns to fuse complementary information for multi-modality medical image analysis. Our CNN first encodes modality-specific features and then uses them to derive a spatially varying fusion map that quantifies the relative importance of each modality's features across different spatial locations. These fusion maps are then multiplied with the modality-specific feature maps to obtain a representation of the complementary multi-modality information at different locations, which can then be used for image analysis. We evaluated the ability of our CNN to detect and segment multiple regions (lungs, mediastinum, tumors) with different fusion requirements using a dataset of PET-CT images of lung cancer. We compared our method to baseline techniques for multi-modality image fusion (fused inputs (FS), multi-branch (MB) techniques, and multichannel (MC) techniques) and segmentation. Our findings show that our CNN had a significantly higher foreground detection accuracy (99.29%, p < 0:05) than the fusion baselines (FS: 99.00%, MB: 99.08%, TC: 98.92%) and a significantly higher Dice score (63.85%) than recent PET-CT tumor segmentation methods.

Semi-automatic lymphoma detection and segmentation using fully conditional random fields

The detection and delineation of the lymphoma volume are a critical step for its treatment and its outcome prediction. Positron Emission Tomography (PET) is widely used for lymphoma detection. Two common types of approaches can be distinguished for lymphoma detection and segmentation in PET. The first one is ROI dependent which needs a ROI defined by physicians. The second one is based on machine learning methods which need a large learning database. However, such a large standard database is quite rare in medical field. Considering these problems, we propose a new approach that combines PET (metabolic information) with CT (anatomical information). Our approach is semi-automatic, it consists of three steps. First, an anatomical multi-atlas segmentation is applied on CT to locate and remove the organs having physiologic hypermetabolism in PET. Then, CRFs (Conditional Random Fields) detect and segment a set of possible lymphoma volumes in PET. The conditional probabilities used in CRFs are usually estimated by a learning step. In this work, we propose to estimate them in an unsupervised way. The final step is to visualize the detected lymphoma volumes and select the real ones by simply clicking on them. The false detection is low thanks to the first step. Our method is tested on 11 patients. The rate of good detection of lymphoma is 100%. The average of Dice indexes for measuring the lymphoma segmentation performance is 84.4% compared to the manual lymphoma segmentation. Comparing with other methods in terms of Dice index shows the best performance of our method.