1
|
Parghane RV, Basu S. Role of Novel Quantitative Imaging Techniques in Hematological Malignancies. PET Clin 2024:S1556-8598(24)00054-3. [PMID: 38944639 DOI: 10.1016/j.cpet.2024.05.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/01/2024]
Abstract
Hematological malignancies exhibit a widespread distribution, necessitating evaluation of disease activity over the entire body. In clinical practice, visual analysis and semiquantitative parameters are used to assess 18F-FDGPET/CT imaging, which solely represents measurements of disease activity from limited area and may not adequately reflect global disease assessment. An efficient method for assessing the global disease burden of hematological malignancies is to employ PET/computed tomography based novel quantitative parameters. In this article, we explored novel quantitative parameters on PET/CT imaging for assessing global disease burden and the potential role of artificial intelligence (AI) to determine these parameters in evaluation of hematological malignancies.
Collapse
Affiliation(s)
- Rahul V Parghane
- Radiation Medicine Centre (BARC), Tata Memorial Hospital Annexe, Parel, Mumbai, India; Homi Bhabha National Institute, Mumbai, India
| | - Sandip Basu
- Radiation Medicine Centre (BARC), Tata Memorial Hospital Annexe, Parel, Mumbai, India; Homi Bhabha National Institute, Mumbai, India.
| |
Collapse
|
2
|
Choi Y, Al-Masni MA, Jung KJ, Yoo RE, Lee SY, Kim DH. A single stage knowledge distillation network for brain tumor segmentation on limited MR image modalities. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 240:107644. [PMID: 37307766 DOI: 10.1016/j.cmpb.2023.107644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 05/14/2023] [Accepted: 06/03/2023] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Precisely segmenting brain tumors using multimodal Magnetic Resonance Imaging (MRI) is an essential task for early diagnosis, disease monitoring, and surgical planning. Unfortunately, the complete four image modalities utilized in the well-known BraTS benchmark dataset: T1, T2, Fluid-Attenuated Inversion Recovery (FLAIR), and T1 Contrast-Enhanced (T1CE) are not regularly acquired in clinical practice due to the high cost and long acquisition time. Rather, it is common to utilize limited image modalities for brain tumor segmentation. METHODS In this paper, we propose a single stage learning of knowledge distillation algorithm that derives information from the missing modalities for better segmentation of brain tumors. Unlike the previous works that adopted a two-stage framework to distill the knowledge from a pre-trained network into a student network, where the latter network is trained on limited image modality, we train both models simultaneously using a single-stage knowledge distillation algorithm. We transfer the information by reducing the redundancy from a teacher network trained on full image modalities to the student network using Barlow Twins loss on a latent-space level. To distill the knowledge on the pixel level, we further employ a deep supervision idea that trains the backbone networks of both teacher and student paths using Cross-Entropy loss. RESULTS We demonstrate that the proposed single-stage knowledge distillation approach enables improving the performance of the student network in each tumor category with overall dice scores of 91.11% for Tumor Core, 89.70% for Enhancing Tumor, and 92.20% for Whole Tumor in the case of only using the FLAIR and T1CE images, outperforming the state-of-the-art segmentation methods. CONCLUSIONS The outcomes of this work prove the feasibility of exploiting the knowledge distillation in segmenting brain tumors using limited image modalities and hence make it closer to clinical practices.
Collapse
Affiliation(s)
- Yoonseok Choi
- Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Mohammed A Al-Masni
- Department of Artificial Intelligence, College of Software & Convergence Technology, Daeyang AI Center, Sejong University, Seoul 05006, Republic of Korea
| | - Kyu-Jin Jung
- Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul 03722, Republic of Korea
| | - Roh-Eul Yoo
- Department of Radiology, Seoul National University Hospital, 101 Daehak-ro Jongno-gu, Seoul 03080, Republic of Korea; Department of Radiology, Seoul National University College of Medicine, 103 Daehak-ro Jongno-gu, Seoul 03080, Republic of Korea
| | - Seong-Yeong Lee
- Department of Radiology, Seoul National University Hospital, 101 Daehak-ro Jongno-gu, Seoul 03080, Republic of Korea
| | - Dong-Hyun Kim
- Department of Electrical and Electronic Engineering, College of Engineering, Yonsei University, Seoul 03722, Republic of Korea.
| |
Collapse
|
3
|
Rehman MU, Ryu J, Nizami IF, Chong KT. RAAGR2-Net: A brain tumor segmentation network using parallel processing of multiple spatial frames. Comput Biol Med 2023; 152:106426. [PMID: 36565485 DOI: 10.1016/j.compbiomed.2022.106426] [Citation(s) in RCA: 11] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2022] [Revised: 11/16/2022] [Accepted: 12/13/2022] [Indexed: 12/24/2022]
Abstract
Brain tumors are one of the most fatal cancers. Magnetic Resonance Imaging (MRI) is a non-invasive method that provides multi-modal images containing important information regarding the tumor. Many contemporary techniques employ four modalities: T1-weighted (T1), T1-weighted with contrast (T1c), T2-weighted (T2), and fluid-attenuation-inversion-recovery (FLAIR), each of which provides unique and important characteristics for the location of each tumor. Although several modern procedures provide decent segmentation results on the multimodal brain tumor image segmentation benchmark (BraTS) dataset, they lack performance when evaluated simultaneously on all the regions of MRI images. Furthermore, there is still room for improvement due to parameter limitations and computational complexity. Therefore, in this work, a novel encoder-decoder-based architecture is proposed for the effective segmentation of brain tumor regions. Data pre-processing is performed by applying N4 bias field correction, z-score, and 0 to 1 resampling to facilitate model training. To minimize the loss of location information in different modules, a residual spatial pyramid pooling (RASPP) module is proposed. RASPP is a set of parallel layers using dilated convolution. In addition, an attention gate (AG) module is used to efficiently emphasize and restore the segmented output from extracted feature maps. The proposed modules attempt to acquire rich feature representations by combining knowledge from diverse feature maps and retaining their local information. The performance of the proposed deep network based on RASPP, AG, and recursive residual (R2) block termed RAAGR2-Net is evaluated on the BraTS benchmarks. The experimental results show that the suggested network outperforms existing networks that exhibit the usefulness of the proposed modules for "fine" segmentation. The code for this work is made available online at: https://github.com/Rehman1995/RAAGR2-Net.
Collapse
Affiliation(s)
- Mobeen Ur Rehman
- Department of Electronics and Information Engineering, Jeonbuk National University, Jeonju 54896, South Korea.
| | - Jihyoung Ryu
- Electronics and Telecommunications Research Institute, 176-11 Cheomdan Gwagi-ro, Buk-gu, Gwangju 61012, Republic of Korea.
| | - Imran Fareed Nizami
- Department of Electrical Engineering, Bahria University, Islamabad, Pakistan.
| | - Kil To Chong
- Department of Electronics and Information Engineering, Jeonbuk National University, Jeonju 54896, South Korea; Advances Electronics and Information Research Center, Jeonbuk National University, Jeonju 54896, South Korea.
| |
Collapse
|
4
|
Kotsyfakis S, Iliaki-Giannakoudaki E, Anagnostopoulos A, Papadokostaki E, Giannakoudakis K, Goumenakis M, Kotsyfakis M. The application of machine learning to imaging in hematological oncology: A scoping review. Front Oncol 2022; 12:1080988. [PMID: 36605438 PMCID: PMC9808781 DOI: 10.3389/fonc.2022.1080988] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Accepted: 12/05/2022] [Indexed: 12/24/2022] Open
Abstract
Background Here, we conducted a scoping review to (i) establish which machine learning (ML) methods have been applied to hematological malignancy imaging; (ii) establish how ML is being applied to hematological cancer radiology; and (iii) identify addressable research gaps. Methods The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews guidelines. The inclusion criteria were (i) pediatric and adult patients with suspected or confirmed hematological malignancy undergoing imaging (population); (ii) any study using ML techniques to derive models using radiological images to apply to the clinical management of these patients (concept); and (iii) original research articles conducted in any setting globally (context). Quality Assessment of Diagnostic Accuracy Studies 2 criteria were used to assess diagnostic and segmentation studies, while the Newcastle-Ottawa scale was used to assess the quality of observational studies. Results Of 53 eligible studies, 33 applied diverse ML techniques to diagnose hematological malignancies or to differentiate them from other diseases, especially discriminating gliomas from primary central nervous system lymphomas (n=18); 11 applied ML to segmentation tasks, while 9 applied ML to prognostication or predicting therapeutic responses, especially for diffuse large B-cell lymphoma. All studies reported discrimination statistics, but no study calculated calibration statistics. Every diagnostic/segmentation study had a high risk of bias due to their case-control design; many studies failed to provide adequate details of the reference standard; and only a few studies used independent validation. Conclusion To deliver validated ML-based models to radiologists managing hematological malignancies, future studies should (i) adhere to standardized, high-quality reporting guidelines such as the Checklist for Artificial Intelligence in Medical Imaging; (ii) validate models in independent cohorts; (ii) standardize volume segmentation methods for segmentation tasks; (iv) establish comprehensive prospective studies that include different tumor grades, comparisons with radiologists, optimal imaging modalities, sequences, and planes; (v) include side-by-side comparisons of different methods; and (vi) include low- and middle-income countries in multicentric studies to enhance generalizability and reduce inequity.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Michail Kotsyfakis
- Biology Center of the Czech Academy of Sciences, Budweis (Ceske Budejovice), Czechia,*Correspondence: Michail Kotsyfakis,
| |
Collapse
|
5
|
Wang M, Jiang H, Shi T, Wang Z, Guo J, Lu G, Wang Y, Yao YD. PSR-Nets: Deep neural networks with prior shift regularization for PET/CT based automatic, accurate, and calibrated whole-body lymphoma segmentation. Comput Biol Med 2022; 151:106215. [PMID: 36306584 DOI: 10.1016/j.compbiomed.2022.106215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Revised: 10/04/2022] [Accepted: 10/15/2022] [Indexed: 12/27/2022]
Abstract
Lymphoma is a type of lymphatic tissue originated cancer. Automatic and accurate lymphoma segmentation is critical for its diagnosis and prognosis yet challenging due to the severely class-imbalanced problem. Generally, deep neural networks trained with class-observation-frequency based re-weighting loss functions are used to address this problem. However, the majority class can be under-weighted by them, due to the existence of data overlap. Besides, they are more mis-calibrated. To resolve these, we propose a neural network with prior-shift regularization (PSR-Net), which comprises a UNet-like backbone with re-weighting loss functions, and a prior-shift regularization (PSR) module including a prior-shift layer (PSL), a regularizer generation layer (RGL), and an expected prediction confidence updating layer (EPCUL). We first propose a trainable expected prediction confidence (EPC) for each class. Periodically, PSL shifts a prior training dataset to a more informative dataset based on EPCs; RGL presents a generalized informative-voxel-aware (GIVA) loss with EPCs and calculates it on the informative dataset for model finetuning in back-propagation; and EPCUL updates EPCs to refresh PSL and RRL in next forward-propagation. PSR-Net is trained in a two- stage manner. The backbone is first trained with re-weighting loss functions, then we reload the best saved model for the backbone and continue to train it with the weighted sum of the re-weighting loss functions, the GIVA regularizer and the L2 loss function of EPCs for regularization fine-tuning. Extensive experiments are performed based on PET/CT volumes with advanced stage lymphomas. Our PSR-Net achieves 95.12% sensitivity and 87.18% Dice coefficient, demonstrating the effectiveness of PSR-Net, when compared to the baselines and the state-of-the-arts.
Collapse
Affiliation(s)
- Meng Wang
- Department of Software College, Northeastern University, Shenyang 110819, China
| | - Huiyan Jiang
- Department of Software College, Northeastern University, Shenyang 110819, China; Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang 110819, China.
| | - Tianyu Shi
- Department of Software College, Northeastern University, Shenyang 110819, China
| | - Zhiguo Wang
- Department of Nuclear Medicine, General Hospital of Northern Military Area, Shenyang 110016, China
| | - Jia Guo
- Department of Nuclear Medicine, General Hospital of Northern Military Area, Shenyang 110016, China
| | - Guoxiu Lu
- Department of Nuclear Medicine, General Hospital of Northern Military Area, Shenyang 110016, China
| | - Youchao Wang
- Department of Nuclear Medicine, General Hospital of Northern Military Area, Shenyang 110016, China
| | - Yu-Dong Yao
- Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ 07030, USA
| |
Collapse
|
6
|
Hasani N, Paravastu SS, Farhadi F, Yousefirizi F, Morris MA, Rahmim A, Roschewski M, Summers RM, Saboury B. Artificial Intelligence in Lymphoma PET Imaging:: A Scoping Review (Current Trends and Future Directions). PET Clin 2022; 17:145-174. [PMID: 34809864 PMCID: PMC8735853 DOI: 10.1016/j.cpet.2021.09.006] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Malignant lymphomas are a family of heterogenous disorders caused by clonal proliferation of lymphocytes. 18F-FDG-PET has proven to provide essential information for accurate quantification of disease burden, treatment response evaluation, and prognostication. However, manual delineation of hypermetabolic lesions is often a time-consuming and impractical task. Applications of artificial intelligence (AI) may provide solutions to overcome this challenge. Beyond segmentation and detection of lesions, AI could enhance tumor characterization and heterogeneity quantification, as well as treatment response prediction and recurrence risk stratification. In this scoping review, we have systematically mapped and discussed the current applications of AI (such as detection, classification, segmentation as well as the prediction and prognostication) in lymphoma PET.
Collapse
Affiliation(s)
- Navid Hasani
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA; University of Queensland Faculty of Medicine, Ochsner Clinical School, New Orleans, LA 70121, USA
| | - Sriram S Paravastu
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA
| | - Faraz Farhadi
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA
| | - Fereshteh Yousefirizi
- Department of Integrative Oncology, BC Cancer Research Institute, Vancouver, BC, Canada
| | - Michael A Morris
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA; Department of Computer Science and Electrical Engineering, University of Maryland-Baltimore Country, Baltimore, MD, USA
| | - Arman Rahmim
- Department of Integrative Oncology, BC Cancer Research Institute, Vancouver, BC, Canada; Department of Radiology, BC Cancer Research Institute, University of British Columbia, 675 West 10th Avenue, Vancouver, British Columbia, V5Z 1L3, Canada
| | - Mark Roschewski
- Lymphoid Malignancies Branch, Center for Cancer Research, National Institutes of Health, Bethesda, MD, USA
| | - Ronald M Summers
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA.
| | - Babak Saboury
- Department of Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, 9000 Rockville Pike, Building 10, Room 1C455, Bethesda, MD 20892, USA; Department of Computer Science and Electrical Engineering, University of Maryland-Baltimore Country, Baltimore, MD, USA; Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.
| |
Collapse
|
7
|
Lapuyade-Lahorgue J, Ruan S. Segmentation of multicorrelated images with copula models and conditionally random fields. J Med Imaging (Bellingham) 2022; 9:014001. [PMID: 35024379 PMCID: PMC8741411 DOI: 10.1117/1.jmi.9.1.014001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2021] [Accepted: 12/16/2021] [Indexed: 01/11/2023] Open
Abstract
Purpose: Multisource images are interesting in medical imaging. Indeed, multisource images enable the use of complementary information of different sources such as for T1 and T2 modalities in MRI imaging. However, such multisource data can also be subject to redundancy and correlation. The question is how to efficiently fuse the multisource information without reinforcing the redundancy. We propose a method for segmenting multisource images that are statistically correlated. Approach: The method that we propose is the continuation of a prior work in which we introduce the copula model in hidden Markov fields (HMF). To achieve the multisource segmentations, we use a functional measure of dependency called "copula." This copula is incorporated in the conditionally random fields (CRF). Contrary to HMF, where we consider a prior knowledge on the hidden states modeled by an HMF, in CRF, there is no prior information and only the distribution of the hidden states conditionally to the observations can be known. This conditional distribution depends on the data and can be modeled by an energy function composed of two terms. The first one groups the voxels having similar intensities in the same class. As for the second term, it encourages a pair of voxels to be in the same class if the difference between their intensities is not too big. Results: A comparison between HMF and CRF is performed via theory and experimentations using both simulated and real data from BRATS 2013. Moreover, our method is compared with different state-of-the-art methods, which include supervised (convolutional neural networks) and unsupervised (hierarchical MRF). Our unsupervised method gives similar results as decision trees for synthetic images and as convolutional neural networks for real images; both methods are supervised. Conclusions: We compare two statistical methods using the copula: HMF and CRF to deal with multicorrelated images. We demonstrate the interest of using copula. In both models, the copula considerably improves the results compared with individual segmentations.
Collapse
Affiliation(s)
- Jérôme Lapuyade-Lahorgue
- University of Rouen, LITIS, Eq. Quantif, Rouen, France,Address all correspondence to Jérôme Lapuyade-Lahorgue,
| | - Su Ruan
- University of Rouen, LITIS, Eq. Quantif, Rouen, France
| |
Collapse
|
8
|
Zhou T, Canu S, Vera P, Ruan S. Feature-enhanced generation and multi-modality fusion based deep neural network for brain tumor segmentation with missing MR modalities. Neurocomputing 2021. [DOI: 10.1016/j.neucom.2021.09.032] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
|
9
|
Wang M, Jiang H, Shi T, Yao YD. HD-RDS-UNet: Leveraging Spatial-Temporal Correlation between the Decoder Feature Maps for Lymphoma Segmentation. IEEE J Biomed Health Inform 2021; 26:1116-1127. [PMID: 34351864 DOI: 10.1109/jbhi.2021.3102612] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Lymphoma is a group of malignant tumors originated in the lymphatic system. Automatic and accurate lymphoma segmentation in PET/CT volumes is critical yet challenging in the clinical practice. Recently, UNet-like architectures are widely used for medical image segmentation. The pure UNet-like architectures model the spatial correlation between the feature maps very well, whereas they discard the critical temporal correlation. Some prior work combines UNet with recurrent neural networks (RNNs) to utilize the spatial and temporal correlation simultaneously. However, it is inconvenient to incorporate some advanced techniques for UNet to RNNs, which hampers their further improvements. In this paper, we propose a recurrent dense siamese decoder architecture, which simulates RNNs and can densely utilize the spatial-temporal correlation between the decoder feature maps following a UNet approach. We combine it with a modified hyper dense encoder. Therefore, the proposed model is a UNet with a hyper dense encoder and a recurrent dense siamese decoder (HD-RDS-UNet). To stabilize the training process, we propose a weighted Dice loss with stable gradient and self-adaptive parameters. We perform patient-independent fivefold cross-validation on 3D volumes collected from whole-body PET/CT scans of patients with lymphomas. The experimental results show that the volume-wise average Dice score and sensitivity are 85.58% and 94.63%, respectively. The patient-wise average Dice score and sensitivity are 85.85% and 95.01%, respectively. The different configurations of HD-RDS-UNet consistently show superiority in the performance comparison. Besides, a trained HD-RDS-UNet can be easily pruned, resulting in significantly reduced inference time and memory usage, while keeping very good segmentation performance.
Collapse
|
10
|
Zhou T, Canu S, Vera P, Ruan S. Latent Correlation Representation Learning for Brain Tumor Segmentation With Missing MRI Modalities. IEEE TRANSACTIONS ON IMAGE PROCESSING : A PUBLICATION OF THE IEEE SIGNAL PROCESSING SOCIETY 2021; 30:4263-4274. [PMID: 33830924 DOI: 10.1109/tip.2021.3070752] [Citation(s) in RCA: 40] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Magnetic Resonance Imaging (MRI) is a widely used imaging technique to assess brain tumor. Accurately segmenting brain tumor from MR images is the key to clinical diagnostics and treatment planning. In addition, multi-modal MR images can provide complementary information for accurate brain tumor segmentation. However, it's common to miss some imaging modalities in clinical practice. In this paper, we present a novel brain tumor segmentation algorithm with missing modalities. Since it exists a strong correlation between multi-modalities, a correlation model is proposed to specially represent the latent multi-source correlation. Thanks to the obtained correlation representation, the segmentation becomes more robust in the case of missing modality. First, the individual representation produced by each encoder is used to estimate the modality independent parameter. Then, the correlation model transforms all the individual representations to the latent multi-source correlation representations. Finally, the correlation representations across modalities are fused via attention mechanism into a shared representation to emphasize the most important features for segmentation. We evaluate our model on BraTS 2018 and BraTS 2019 dataset, it outperforms the current state-of-the-art methods and produces robust results when one or more modalities are missing.
Collapse
|
11
|
Li S, Jiang H, Li H, Yao YD. AW-SDRLSE: Adaptive Weighting and Scalable Distance Regularized Level Set Evolution for Lymphoma Segmentation on PET Images. IEEE J Biomed Health Inform 2021; 25:1173-1184. [PMID: 32841130 DOI: 10.1109/jbhi.2020.3017546] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Accurate lymphoma segmentation on Positron Emission Tomography (PET) images is of great importance for medical diagnoses, such as for distinguishing benign and malignant. To this end, this paper proposes an adaptive weighting and scalable distance regularized level set evolution (AW-SDRLSE) method for delineating lymphoma boundaries on 2D PET slices. There are three important characteristics with respect to AW-SDRLSE: 1) A scalable distance regularization term is proposed and a parameter q can control the contour's convergence rate and precision in theory. 2) A novel dynamic annular mask is proposed to calculate mean intensities of local interior and exterior regions and further define the region energy term. 3) As the level set method is sensitive to parameters, we thus propose an adaptive weighting strategy for the length and area energy terms using local region intensity and boundary direction information. AW-SDRLSE is evaluated on 90 cases of real PET data with a mean Dice coefficient of 0.8796. Comparative results demonstrate the accuracy and robustness of AW-SDRLSE as well as its performance advantages as compared with related level set methods. In addition, experimental results indicate that AW-SDRLSE can be a fine segmentation method for improving the lymphoma segmentation results obtained by deep learning (DL) methods significantly.
Collapse
|
12
|
Pinochet P, Eude F, Becker S, Shah V, Sibille L, Toledano MN, Modzelewski R, Vera P, Decazes P. Evaluation of an Automatic Classification Algorithm Using Convolutional Neural Networks in Oncological Positron Emission Tomography. Front Med (Lausanne) 2021; 8:628179. [PMID: 33718406 PMCID: PMC7953145 DOI: 10.3389/fmed.2021.628179] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 01/25/2021] [Indexed: 12/11/2022] Open
Abstract
Introduction: Our aim was to evaluate the performance in clinical research and in clinical routine of a research prototype, called positron emission tomography (PET) Assisted Reporting System (PARS) (Siemens Healthineers) and based on a convolutional neural network (CNN), which is designed to detect suspected cancer sites in fluorine-18 fluorodeoxyglucose (18F-FDG) PET/computed tomography (CT). Method: We retrospectively studied two cohorts of patients. The first cohort consisted of research-based patients who underwent PET scans as part of the initial workup for diffuse large B-cell lymphoma (DLBCL). The second cohort consisted of patients who underwent PET scans as part of the evaluation of miscellaneous cancers in clinical routine. In both cohorts, we assessed the correlation between manually and automatically segmented total metabolic tumor volumes (TMTVs), and the overlap between both segmentations (Dice score). For the research cohort, we also compared the prognostic value for progression-free survival (PFS) and overall survival (OS) of manually and automatically obtained TMTVs. Results: For the first cohort (research cohort), data from 119 patients were retrospectively analyzed. The median Dice score between automatic and manual segmentations was 0.65. The intraclass correlation coefficient between automatically and manually obtained TMTVs was 0.68. Both TMTV results were predictive of PFS (hazard ratio: 2.1 and 3.3 for automatically based and manually based TMTVs, respectively) and OS (hazard ratio: 2.4 and 3.1 for automatically based and manually based TMTVs, respectively). For the second cohort (routine cohort), data from 430 patients were retrospectively analyzed. The median Dice score between automatic and manual segmentations was 0.48. The intraclass correlation coefficient between automatically and manually obtained TMTVs was 0.61. Conclusion: The TMTVs determined for the research cohort remain predictive of total and PFS for DLBCL. However, the segmentations and TMTVs determined automatically by the algorithm need to be verified and, sometimes, corrected to be similar to the manual segmentation.
Collapse
Affiliation(s)
- Pierre Pinochet
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France
| | - Florian Eude
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France
| | - Stéphanie Becker
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France.,LITIS Quantif-EA 4108, University of Rouen, Rouen, France
| | - Vijay Shah
- Siemens Medical Solutions USA, Inc., Knoxville, TN, United States
| | - Ludovic Sibille
- Siemens Medical Solutions USA, Inc., Knoxville, TN, United States
| | | | - Romain Modzelewski
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France.,LITIS Quantif-EA 4108, University of Rouen, Rouen, France
| | - Pierre Vera
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France.,LITIS Quantif-EA 4108, University of Rouen, Rouen, France
| | - Pierre Decazes
- Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France.,LITIS Quantif-EA 4108, University of Rouen, Rouen, France
| |
Collapse
|
13
|
Weisman AJ, Kieler MW, Perlman S, Hutchings M, Jeraj R, Kostakoglu L, Bradshaw TJ. Comparison of 11 automated PET segmentation methods in lymphoma. Phys Med Biol 2020; 65:235019. [PMID: 32906088 DOI: 10.1088/1361-6560/abb6bd] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023]
Abstract
Segmentation of lymphoma lesions in FDG PET/CT images is critical in both assessing individual lesions and quantifying patient disease burden. Simple thresholding methods remain common despite the large heterogeneity in lymphoma lesion location, size, and contrast. Here, we assess 11 automated PET segmentation methods for their use in two scenarios: individual lesion segmentation and patient-level disease quantification in lymphoma. Lesions on 18F-FDG PET/CT scans of 90 lymphoma patients were contoured by a nuclear medicine physician. Thresholding, active contours, clustering, adaptive region-growing, and convolutional neural network (CNN) methods were implemented on all physician-identified lesions. Lesion-level segmentation was evaluated using multiple segmentation performance metrics (Dice, Hausdorff Distance). Patient-level quantification of total disease burden (SUVtotal) and metabolic tumor volume (MTV) was assessed using Spearman's correlation coefficients between the segmentation output and physician contours. Lesion segmentation and patient quantification performance was compared to inter-physician agreement in a subset of 20 patients segmented by a second nuclear medicine physician. In total, 1223 lesions with median tumor-to-background ratio of 4.0 and volume of 1.8 cm3, were evaluated. When assessed for lesion segmentation, a 3D CNN, DeepMedic, achieved the highest performance across all evaluation metrics. DeepMedic, clustering methods, and an iterative threshold method had lesion-level segmentation performance comparable to the degree of inter-physician agreement. For patient-level SUVtotal and MTV quantification, all methods except 40% and 50% SUVmax and adaptive region-growing achieved a performance that was similar the agreement of the two physicians. Multiple methods, including a 3D CNN, clustering, and an iterative threshold method, achieved both good lesion-level segmentation and patient-level quantification performance in a population of 90 lymphoma patients. These methods are thus recommended over thresholding methods such as 40% and 50% SUVmax, which were consistently found to be significantly outside the limits defined by inter-physician agreement.
Collapse
Affiliation(s)
- Amy J Weisman
- Department of Medical Physics, University of Wisconsin-Madison, Madison, WI, United States of America
| | | | | | | | | | | | | |
Collapse
|
14
|
Zhou T, Canu S, Ruan S. Fusion based on attention mechanism and context constraint for multi-modal brain tumor segmentation. Comput Med Imaging Graph 2020; 86:101811. [PMID: 33232843 DOI: 10.1016/j.compmedimag.2020.101811] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2020] [Revised: 10/06/2020] [Accepted: 10/23/2020] [Indexed: 11/18/2022]
Abstract
This paper presents a 3D brain tumor segmentation network from multi-sequence MRI datasets based on deep learning. We propose a three-stage network: generating constraints, fusion under constraints and final segmentation. In the first stage, an initial 3D U-Net segmentation network is introduced to produce an additional context constraint for each tumor region. Under the obtained constraint, multi-sequence MRI are then fused using an attention mechanism to achieve three single tumor region segmentations. Considering the location relationship of the tumor regions, a new loss function is introduced to deal with the multiple class segmentation problem. Finally, a second 3D U-Net network is applied to combine and refine the three single prediction results. In each stage, only 8 initial filters are used, allowing to decrease significantly the number of parameters to be estimated. We evaluated our method on BraTS 2017 dataset. The results are promising in terms of dice score, hausdorff distance, and the amount of memory required for training.
Collapse
Affiliation(s)
- Tongxue Zhou
- Université de Rouen Normandie, LITIS - QuantIF, Rouen 76183, France; INSA de Rouen, LITIS - Apprentissage, Rouen 76800, France; Normandie Univ, INSA Rouen, UNIROUEN, UNIHAVRE, LITIS, France
| | - Stéphane Canu
- INSA de Rouen, LITIS - Apprentissage, Rouen 76800, France; Normandie Univ, INSA Rouen, UNIROUEN, UNIHAVRE, LITIS, France
| | - Su Ruan
- Université de Rouen Normandie, LITIS - QuantIF, Rouen 76183, France; Normandie Univ, INSA Rouen, UNIROUEN, UNIHAVRE, LITIS, France.
| |
Collapse
|
15
|
Blanc-Durand P, Jégou S, Kanoun S, Berriolo-Riedinger A, Bodet-Milin C, Kraeber-Bodéré F, Carlier T, Le Gouill S, Casasnovas RO, Meignan M, Itti E. Fully automatic segmentation of diffuse large B cell lymphoma lesions on 3D FDG-PET/CT for total metabolic tumour volume prediction using a convolutional neural network. Eur J Nucl Med Mol Imaging 2020; 48:1362-1370. [PMID: 33097974 DOI: 10.1007/s00259-020-05080-7] [Citation(s) in RCA: 58] [Impact Index Per Article: 14.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2020] [Accepted: 10/15/2020] [Indexed: 01/29/2023]
Abstract
PURPOSE Lymphoma lesion detection and segmentation on whole-body FDG-PET/CT are a challenging task because of the diversity of involved nodes, organs or physiological uptakes. We sought to investigate the performances of a three-dimensional (3D) convolutional neural network (CNN) to automatically segment total metabolic tumour volume (TMTV) in large datasets of patients with diffuse large B cell lymphoma (DLBCL). METHODS The dataset contained pre-therapy FDG-PET/CT from 733 DLBCL patients of 2 prospective LYmphoma Study Association (LYSA) trials. The first cohort (n = 639) was used for training using a 5-fold cross validation scheme. The second cohort (n = 94) was used for external validation of TMTV predictions. Ground truth masks were manually obtained after a 41% SUVmax adaptive thresholding of lymphoma lesions. A 3D U-net architecture with 2 input channels for PET and CT was trained on patches randomly sampled within PET/CTs with a summed cross entropy and Dice similarity coefficient (DSC) loss. Segmentation performance was assessed by the DSC and Jaccard coefficients. Finally, TMTV predictions were validated on the second independent cohort. RESULTS Mean DSC and Jaccard coefficients (± standard deviation) in the validations set were 0.73 ± 0.20 and 0.68 ± 0.21, respectively. An underestimation of mean TMTV by - 12 mL (2.8%) ± 263 was found in the validation sets of the first cohort (P = 0.27). In the second cohort, an underestimation of mean TMTV by - 116 mL (20.8%) ± 425 was statistically significant (P = 0.01). CONCLUSION Our CNN is a promising tool for automatic detection and segmentation of lymphoma lesions, despite slight underestimation of TMTV. The fully automatic and open-source features of this CNN will allow to increase both dissemination in routine practice and reproducibility of TMTV assessment in lymphoma patients.
Collapse
Affiliation(s)
- Paul Blanc-Durand
- Department of Nuclear Medicine, CHU H. Mondor, AP-HP, F-94010, Créteil, France. .,LYmphoma Study Association (LYSA), Pierre-Bénite, France. .,INSERM IMRB Team 8, U-PEC, F-94000, Créteil, France. .,INRIA Epione Team, Sophia Antipolis, France. .,Service de Médecine Nucléaire, CHU Henri Mondor, 51 ave. Du Mal de Lattre de Tassigny, 94010, Créteil, France.
| | | | - Salim Kanoun
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Nuclear Medicine, Institut C. Regaud, F-31000, Toulouse, France
| | - Alina Berriolo-Riedinger
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Nuclear Medicine, Centre G.-F. Leclerc, F-21000, Dijon, France
| | - Caroline Bodet-Milin
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Nuclear Medicine, CHU de Nantes, F-44000, Nantes, France.,CRCINA, INSERM, CNRS, Université d'Angers, Université de Nantes, Nantes, France
| | - Françoise Kraeber-Bodéré
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Nuclear Medicine, CHU de Nantes, F-44000, Nantes, France.,CRCINA, INSERM, CNRS, Université d'Angers, Université de Nantes, Nantes, France
| | - Thomas Carlier
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Nuclear Medicine, CHU de Nantes, F-44000, Nantes, France.,CRCINA, INSERM, CNRS, Université d'Angers, Université de Nantes, Nantes, France
| | - Steven Le Gouill
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Hematology, CHU de Nantes, F-44000, Nantes, France
| | - René-Olivier Casasnovas
- LYmphoma Study Association (LYSA), Pierre-Bénite, France.,Department of Hematology, CHU Le Bocage, F-21000, Dijon, France
| | - Michel Meignan
- LYmphoma Study Association (LYSA), Pierre-Bénite, France
| | - Emmanuel Itti
- Department of Nuclear Medicine, CHU H. Mondor, AP-HP, F-94010, Créteil, France.,LYmphoma Study Association (LYSA), Pierre-Bénite, France.,INSERM IMRB Team 8, U-PEC, F-94000, Créteil, France
| |
Collapse
|
16
|
Weisman AJ, Kieler MW, Perlman SB, Hutchings M, Jeraj R, Kostakoglu L, Bradshaw TJ. Convolutional Neural Networks for Automated PET/CT Detection of Diseased Lymph Node Burden in Patients with Lymphoma. Radiol Artif Intell 2020; 2:e200016. [PMID: 33937842 PMCID: PMC8082306 DOI: 10.1148/ryai.2020200016] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2020] [Revised: 04/20/2020] [Accepted: 05/01/2020] [Indexed: 05/01/2023]
Abstract
PURPOSE To automatically detect lymph nodes involved in lymphoma on fluorine 18 (18F) fluorodeoxyglucose (FDG) PET/CT images using convolutional neural networks (CNNs). MATERIALS AND METHODS In this retrospective study, baseline disease of 90 patients with lymphoma was segmented on 18F-FDG PET/CT images (acquired between 2005 and 2011) by a nuclear medicine physician. An ensemble of three-dimensional patch-based, multiresolution pathway CNNs was trained using fivefold cross-validation. Performance was assessed using the true-positive rate (TPR) and number of false-positive (FP) findings. CNN performance was compared with agreement between physicians by comparing the annotations of a second nuclear medicine physician to the first reader in 20 of the patients. Patient TPR was compared using Wilcoxon signed rank tests. RESULTS Across all 90 patients, a range of 0-61 nodes per patient was detected. At an average of four FP findings per patient, the method achieved a TPR of 85% (923 of 1087 nodes). Performance varied widely across patients (TPR range, 33%-100%; FP range, 0-21 findings). In the 20 patients labeled by both physicians, a range of 1-49 nodes per patient was detected and labeled. The second reader identified 96% (210 of 219) of nodes with an additional 3.7 per patient compared with the first reader. In the same 20 patients, the CNN achieved a 90% (197 of 219) TPR at 3.7 FP findings per patient. CONCLUSION An ensemble of three-dimensional CNNs detected lymph nodes at a performance nearly comparable to differences between two physicians' annotations. This preliminary study is a first step toward automated PET/CT assessment for lymphoma.© RSNA, 2020.
Collapse
|
17
|
Accurate segmentation of overlapping cells in cervical cytology with deep convolutional neural networks. Neurocomputing 2019. [DOI: 10.1016/j.neucom.2019.06.086] [Citation(s) in RCA: 26] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
18
|
Detection and segmentation of lymphomas in 3D PET images via clustering with entropy-based optimization strategy. Int J Comput Assist Radiol Surg 2019; 14:1715-1724. [PMID: 31401714 DOI: 10.1007/s11548-019-02049-2] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2019] [Accepted: 08/05/2019] [Indexed: 01/10/2023]
Abstract
PURPOSE Lymphoma detection and segmentation from PET images are critical tasks for cancer staging and treatment monitoring. However, it is still a challenge owing to the complexities of lymphoma PET data themselves, and the huge computational burdens and memory requirements for 3D volume data. In this work, an entropy-based optimization strategy for clustering is proposed to detect and segment lymphomas in 3D PET images. METHODS To reduce computational complexity and add more feature information, billions of voxels in 3D volume data are first aggregated into supervoxels. Then, such supervoxels serve as basic data units for further clustering by using DBSCAN algorithm, in which some new feature attributes based on physical spatial information and prior knowledge are proposed. In addition, more importantly, an entropy-based objective function is constructed to search the most appropriate parameters of DBSCAN to obtain the optimal clustering results by using a genetic algorithm. This step allows to automatically adapt the parameters to each patient. Finally, a series of comparison experiments among various feature attributes are performed. RESULTS 48 patient data are conducted, showing the combination of three features, supervoxel intensity, geographic coordinates and organ distributions, can achieve good performance and the proposed entropy-based optimization scheme has more advantages than the existing methods. CONCLUSION The proposed entropy-based optimization strategy for clustering by integrating physical spatial attributes and prior knowledge can achieve better performance than traditional methods.
Collapse
|
19
|
Guo B, Tan X, Ke Q, Cen H. Prognostic value of baseline metabolic tumor volume and total lesion glycolysis in patients with lymphoma: A meta-analysis. PLoS One 2019; 14:e0210224. [PMID: 30625203 PMCID: PMC6326501 DOI: 10.1371/journal.pone.0210224] [Citation(s) in RCA: 66] [Impact Index Per Article: 13.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2018] [Accepted: 12/17/2018] [Indexed: 01/11/2023] Open
Abstract
Whether baseline metabolic tumor volume (TMTV) and total lesion glycolysis (TLG) measured by FDG-PET/CT affected prognosis of patients with lymphoma was controversial. We searched PubMed, EMBASE and Cochrane to identify studies assessing the effect of baseline TMTV and TLG on the survival of lymphoma patients. Pooled hazard ratios (HR) for overall survival (OS) and progression-free survival (PFS) were calculated, along with 95% confidence intervals (CI). Twenty-seven eligible studies including 2,729 patients were analysed. Patients with high baseline TMTV showed a worse prognosis with an HR of 3.05 (95% CI 2.55–3.64, p<0.00001) for PFS and an HR of 3.07 (95% CI 2.47–3.82, p<0.00001) for OS. Patients with high baseline TLG also showed a worse prognosis with an HR of 3.44 (95% CI 2.37–5.01, p<0.00001) for PFS and an HR of 3.08 (95% CI 1.84–5.16, p<0.00001) for OS. A high baseline TMTV was significantly associated with worse survival in DLBCL patients treated with R-CHOP (OS, pooled HR = 3.52; PFS, pooled HR = 2.93). A high baseline TLG was significantly associated with worse survival in DLBCL patients treated with R-CHOP (OS, pooled HR = 3.06; PFS, pooled HR = 2.93). The negative effect of high baseline TMTV on PFS was demonstrated in HL (pooled HR = 3.89). A high baseline TMTV was significantly associated with worse survival in ENKL patients (OS, pooled HR = 2.24; PFS, pooled HR = 3.25). A high baseline TLG was significantly associated with worse survival in ENKL patients (OS, pooled HR = 2.58; PFS, pooled HR = 2.99). High baseline TMTV or TLG predict significantly worse PFS and OS in patients with lymphoma. Future studies are warranted to explore whether TMTV or TLG could be integrated into various prognostic models for clinical decision making.
Collapse
Affiliation(s)
- Baoping Guo
- Department of Chemotherapy, Guangxi Cancer Hospital and of Guangxi Medical University Affiliated Cancer Hospital, Nanning, Guangxi, People’s Republic of China
| | - Xiaohong Tan
- Department of Chemotherapy, Guangxi Cancer Hospital and of Guangxi Medical University Affiliated Cancer Hospital, Nanning, Guangxi, People’s Republic of China
| | - Qing Ke
- Department of Chemotherapy, Guangxi Cancer Hospital and of Guangxi Medical University Affiliated Cancer Hospital, Nanning, Guangxi, People’s Republic of China
| | - Hong Cen
- Department of Chemotherapy, Guangxi Cancer Hospital and of Guangxi Medical University Affiliated Cancer Hospital, Nanning, Guangxi, People’s Republic of China
- * E-mail:
| |
Collapse
|