1
|
Zhang X, Broersen A, Sokooti H, Ramasamy A, Kitslaar P, Parasa R, Karaduman M, Mohammed ASAJ, Bourantas CV, Dijkstra J. Cross-sectional angle prediction of lipid-rich and calcified tissue on computed tomography angiography images. Int J Comput Assist Radiol Surg 2024; 19:971-981. [PMID: 38478204 PMCID: PMC11098906 DOI: 10.1007/s11548-024-03086-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2023] [Accepted: 02/26/2024] [Indexed: 03/21/2024]
Abstract
PURPOSE The assessment of vulnerable plaque characteristics and distribution is important to stratify cardiovascular risk in a patient. Computed tomography angiography (CTA) offers a promising alternative to invasive imaging but is limited by the fact that the range of Hounsfield units (HU) in lipid-rich areas overlaps with the HU range in fibrotic tissue and that the HU range of calcified plaques overlaps with the contrast within the contrast-filled lumen. This paper is to investigate whether lipid-rich and calcified plaques can be detected more accurately on cross-sectional CTA images using deep learning methodology. METHODS Two deep learning (DL) approaches are proposed, a 2.5D Dense U-Net and 2.5D Mask-RCNN, which separately perform the cross-sectional plaque detection in the Cartesian and polar domain. The spread-out view is used to evaluate and show the prediction result of the plaque regions. The accuracy and F1-score are calculated on a lesion level for the DL and conventional plaque detection methods. RESULTS For the lipid-rich plaques, the median and mean values of the F1-score calculated by the two proposed DL methods on 91 lesions were approximately 6 and 3 times higher than those of the conventional method. For the calcified plaques, the F1-score of the proposed methods was comparable to those of the conventional method. The median F1-score of the Dense U-Net-based method was 3% higher than that of the conventional method. CONCLUSION The two methods proposed in this paper contribute to finer cross-sectional predictions of lipid-rich and calcified plaques compared to studies focusing only on longitudinal prediction. The angular prediction performance of the proposed methods outperforms the convincing conventional method for lipid-rich plaque and is comparable for calcified plaque.
Collapse
Affiliation(s)
- Xiaotong Zhang
- Division of Image Processing, Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Alexander Broersen
- Division of Image Processing, Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | | | - Anantharaman Ramasamy
- Cardiology, Barts Heart Centre, Barts Health NHS Trust, London, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, London, UK
| | | | - Ramya Parasa
- Cardiology, Barts Heart Centre, Barts Health NHS Trust, London, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, London, UK
- The Essex Cardiothoracic Centre, Basildon, UK
| | | | | | - Christos V Bourantas
- Cardiology, Barts Heart Centre, Barts Health NHS Trust, London, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, London, UK
| | - Jouke Dijkstra
- Division of Image Processing, Radiology, Leiden University Medical Center, Leiden, The Netherlands.
| |
Collapse
|
2
|
Shou Q, Zhao C, Shao X, Jann K, Kim H, Helmer KG, Lu H, Wang DJJ. Transformer-based deep learning denoising of single and multi-delay 3D arterial spin labeling. Magn Reson Med 2024; 91:803-818. [PMID: 37849048 PMCID: PMC10841192 DOI: 10.1002/mrm.29887] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Revised: 09/13/2023] [Accepted: 09/21/2023] [Indexed: 10/19/2023]
Abstract
PURPOSE To present a Swin Transformer-based deep learning (DL) model (SwinIR) for denoising single-delay and multi-delay 3D arterial spin labeling (ASL) and compare its performance with convolutional neural network (CNN) and other Transformer-based methods. METHODS SwinIR and CNN-based spatial denoising models were developed for single-delay ASL. The models were trained on 66 subjects (119 scans) and tested on 39 subjects (44 scans) from three different vendors. Spatiotemporal denoising models were developed using another dataset (6 subjects, 10 scans) of multi-delay ASL. A range of input conditions was tested for denoising single and multi-delay ASL, respectively. The performance was evaluated using similarity metrics, spatial SNR and quantification accuracy of cerebral blood flow (CBF), and arterial transit time (ATT). RESULTS SwinIR outperformed CNN and other Transformer-based networks, whereas pseudo-3D models performed better than 2D models for denoising single-delay ASL. The similarity metrics and image quality (SNR) improved with more slices in pseudo-3D models and further improved when using M0 as input, but introduced greater biases for CBF quantification. Pseudo-3D models with three slices achieved optimal balance between SNR and accuracy, which can be generalized to different vendors. For multi-delay ASL, spatiotemporal denoising models had better performance than spatial-only models with reduced biases in fitted CBF and ATT maps. CONCLUSIONS SwinIR provided better performance than CNN and other Transformer-based methods for denoising both single and multi-delay 3D ASL data. The proposed model offers flexibility to improve image quality and/or reduce scan time for 3D ASL to facilitate its clinical use.
Collapse
Affiliation(s)
- Qinyang Shou
- Laboratory of Functional MRI Technology (LOFT), Stevens Neuro Imaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| | - Chenyang Zhao
- Laboratory of Functional MRI Technology (LOFT), Stevens Neuro Imaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| | - Xingfeng Shao
- Laboratory of Functional MRI Technology (LOFT), Stevens Neuro Imaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| | - Kay Jann
- Laboratory of Functional MRI Technology (LOFT), Stevens Neuro Imaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| | - Hosung Kim
- Laboratory of Neuro Imaging (LONI), Stevens Neuroimaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| | - Karl G. Helmer
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown, MA, USA
| | - Hanzhang Lu
- Department of Radiology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Danny JJ Wang
- Laboratory of Functional MRI Technology (LOFT), Stevens Neuro Imaging and Informatics Institute, University of Southern California, Los Angeles, CA, United States
| |
Collapse
|
3
|
Liu P, Sun Y, Zhao X, Yan Y. Deep learning algorithm performance in contouring head and neck organs at risk: a systematic review and single-arm meta-analysis. Biomed Eng Online 2023; 22:104. [PMID: 37915046 PMCID: PMC10621161 DOI: 10.1186/s12938-023-01159-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2023] [Accepted: 09/21/2023] [Indexed: 11/03/2023] Open
Abstract
PURPOSE The contouring of organs at risk (OARs) in head and neck cancer radiation treatment planning is a crucial, yet repetitive and time-consuming process. Recent studies have applied deep learning (DL) algorithms to automatically contour head and neck OARs. This study aims to conduct a systematic review and meta-analysis to summarize and analyze the performance of DL algorithms in contouring head and neck OARs. The objective is to assess the advantages and limitations of DL algorithms in contour planning of head and neck OARs. METHODS This study conducted a literature search of Pubmed, Embase and Cochrane Library databases, to include studies related to DL contouring head and neck OARs, and the dice similarity coefficient (DSC) of four categories of OARs from the results of each study are selected as effect sizes for meta-analysis. Furthermore, this study conducted a subgroup analysis of OARs characterized by image modality and image type. RESULTS 149 articles were retrieved, and 22 studies were included in the meta-analysis after excluding duplicate literature, primary screening, and re-screening. The combined effect sizes of DSC for brainstem, spinal cord, mandible, left eye, right eye, left optic nerve, right optic nerve, optic chiasm, left parotid, right parotid, left submandibular, and right submandibular are 0.87, 0.83, 0.92, 0.90, 0.90, 0.71, 0.74, 0.62, 0.85, 0.85, 0.82, and 0.82, respectively. For subgroup analysis, the combined effect sizes for segmentation of the brainstem, mandible, left optic nerve, and left parotid gland using CT and MRI images are 0.86/0.92, 0.92/0.90, 0.71/0.73, and 0.84/0.87, respectively. Pooled effect sizes using 2D and 3D images of the brainstem, mandible, left optic nerve, and left parotid gland for contouring are 0.88/0.87, 0.92/0.92, 0.75/0.71 and 0.87/0.85. CONCLUSIONS The use of automated contouring technology based on DL algorithms is an essential tool for contouring head and neck OARs, achieving high accuracy, reducing the workload of clinical radiation oncologists, and providing individualized, standardized, and refined treatment plans for implementing "precision radiotherapy". Improving DL performance requires the construction of high-quality data sets and enhancing algorithm optimization and innovation.
Collapse
Affiliation(s)
- Peiru Liu
- General Hospital of Northern Theater Command, Department of Radiation Oncology, Shenyang, China
- Beifang Hospital of China Medical University, Shenyang, China
| | - Ying Sun
- General Hospital of Northern Theater Command, Department of Radiation Oncology, Shenyang, China
| | - Xinzhuo Zhao
- Shenyang University of Technology, School of Electrical Engineering,, Shenyang, China
| | - Ying Yan
- General Hospital of Northern Theater Command, Department of Radiation Oncology, Shenyang, China.
| |
Collapse
|
4
|
Ramasamy A, Sokooti H, Zhang X, Tzorovili E, Bajaj R, Kitslaar P, Broersen A, Amersey R, Jain A, Ozkor M, Reiber JHC, Dijkstra J, Serruys PW, Moon JC, Mathur A, Baumbach A, Torii R, Pugliese F, Bourantas CV. Novel near-infrared spectroscopy-intravascular ultrasound-based deep-learning methodology for accurate coronary computed tomography plaque quantification and characterization. EUROPEAN HEART JOURNAL OPEN 2023; 3:oead090. [PMID: 37908441 PMCID: PMC10615127 DOI: 10.1093/ehjopen/oead090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/12/2023] [Revised: 07/16/2023] [Accepted: 08/17/2023] [Indexed: 11/02/2023]
Abstract
Aims Coronary computed tomography angiography (CCTA) is inferior to intravascular imaging in detecting plaque morphology and quantifying plaque burden. We aim to, for the first time, train a deep-learning (DL) methodology for accurate plaque quantification and characterization in CCTA using near-infrared spectroscopy-intravascular ultrasound (NIRS-IVUS). Methods and results Seventy patients were prospectively recruited who underwent CCTA and NIRS-IVUS imaging. Corresponding cross sections were matched using an in-house developed software, and the estimations of NIRS-IVUS for the lumen, vessel wall borders, and plaque composition were used to train a convolutional neural network in 138 vessels. The performance was evaluated in 48 vessels and compared against the estimations of NIRS-IVUS and the conventional CCTA expert analysis. Sixty-four patients (186 vessels, 22 012 matched cross sections) were included. Deep-learning methodology provided estimations that were closer to NIRS-IVUS compared with the conventional approach for the total atheroma volume (ΔDL-NIRS-IVUS: -37.8 ± 89.0 vs. ΔConv-NIRS-IVUS: 243.3 ± 183.7 mm3, variance ratio: 4.262, P < 0.001) and percentage atheroma volume (-3.34 ± 5.77 vs. 17.20 ± 7.20%, variance ratio: 1.578, P < 0.001). The DL methodology detected lesions more accurately than the conventional approach (Area under the curve (AUC): 0.77 vs. 0.67, P < 0.001) and quantified minimum lumen area (ΔDL-NIRS-IVUS: -0.35 ± 1.81 vs. ΔConv-NIRS-IVUS: 1.37 ± 2.32 mm2, variance ratio: 1.634, P < 0.001), maximum plaque burden (4.33 ± 11.83% vs. 5.77 ± 16.58%, variance ratio: 2.071, P = 0.004), and calcific burden (-51.2 ± 115.1 vs. -54.3 ± 144.4, variance ratio: 2.308, P < 0.001) more accurately than conventional approach. The DL methodology was able to segment a vessel on CCTA in 0.3 s. Conclusions The DL methodology developed for CCTA analysis from co-registered NIRS-IVUS and CCTA data enables rapid and accurate assessment of lesion morphology and is superior to expert analysts (Clinicaltrials.gov: NCT03556644).
Collapse
Affiliation(s)
- Anantharaman Ramasamy
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
| | | | - Xiaotong Zhang
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Evangelia Tzorovili
- Pragmatic Clinical Trials Unit, Centre for Evaluation and Methods, Wolfson Institute of Population Health, Queen Mary University of London, London, UK
| | - Retesh Bajaj
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
| | - Pieter Kitslaar
- Medis Medical Imaging Systems, Leiden, The Netherlands
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Alexander Broersen
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Rajiv Amersey
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
| | - Ajay Jain
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
| | - Mick Ozkor
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
| | - Johan H C Reiber
- Medis Medical Imaging Systems, Leiden, The Netherlands
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Jouke Dijkstra
- Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, The Netherlands
| | - Patrick W Serruys
- Faculty of Medicine, National Heart and Lung Institute, Imperial College London, Cale Street, London SW3 6LY, UK
- Department of Cardiology, National University of Ireland, Galway, Ireland
| | - James C Moon
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Institute of Cardiovascular Sciences, University College London, Gower Street, London WC1E 6BT, UK
| | - Anthony Mathur
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
| | - Andreas Baumbach
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
| | - Ryo Torii
- Department of Mechanical Engineering, University College London, Torrington Place, London WC1E 7JE, UK
| | - Francesca Pugliese
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
| | - Christos V Bourantas
- Department of Cardiology, Barts Heart Centre, Barts Health NHS Trust, West Smithfield, London EC1A 7BE, UK
- Centre for Cardiovascular Medicine and Devices, William Harvey Research Institute, Queen Mary University of London, Mile End Road, London E1 4NS, UK
- Institute of Cardiovascular Sciences, University College London, Gower Street, London WC1E 6BT, UK
| |
Collapse
|
5
|
Shou Q, Zhao C, Shao X, Jann K, Helmer KG, Lu H, Wang DJ. Transformer based deep learning denoising of single and multi-delay 3D Arterial Spin Labeling. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.04.24.23288718. [PMID: 37162975 PMCID: PMC10168491 DOI: 10.1101/2023.04.24.23288718] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Purpose To present a Swin Transformer-based deep learning (DL) model for denoising of single-delay and multi-delay 3D arterial spin labeling (ASL) and compare its performance with convolutional neural network (CNN) methods. Methods Swin Transformer and CNN-based spatial denoising models were developed for single-delay ASL. The models were trained on 59 subjects (104 scans) and tested on 44 subjects (57 scans) from 3 different vendors. Spatiotemporal denoising models were developed using another dataset (6 subjects, 10 scans) of multi-delay ASL. A range of input conditions was tested for denoising single and multi-delay ASL respectively. The performance was evaluated using similarity metrics, spatial signal-to-noise ratio (SNR) and quantification accuracy of cerebral blood flow (CBF) and arterial transit time (ATT). Results Swin Transformer outperformed CNN-based networks, whereas pseudo-3D models showed better performance than 2D models for denoising single-delay ASL. The similarity metrics and image quality (SNR) improved with more slices in pseudo-3D models, and further improved when using M0 as input but introduced greater biases for CBF quantification. Pseudo-3D models with 3 slices as input achieved optimal balance between SNR and accuracy, which can be generalized to different vendors. For multi-delay, spatiotemporal denoising models had better performance than spatial-only models with reduced biases in fitted CBF and ATT maps. Conclusions Swin Transformer DL models provided better performance than CNN methods for denoising both single and multi-delay 3D ASL data. The proposed model offers flexibility to improve image quality and/or reduce scan time for 3D ASL to facilitate its clinical use.
Collapse
|
6
|
Cheon W, Jeong S, Jeong JH, Lim YK, Shin D, Lee SB, Lee DY, Lee SU, Suh YG, Moon SH, Kim TH, Kim H. Interobserver Variability Prediction of Primary Gross Tumor in a Patient with Non-Small Cell Lung Cancer. Cancers (Basel) 2022; 14:cancers14235893. [PMID: 36497374 PMCID: PMC9741368 DOI: 10.3390/cancers14235893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Revised: 11/25/2022] [Accepted: 11/27/2022] [Indexed: 12/03/2022] Open
Abstract
This research addresses the problem of interobserver variability (IOV), in which different oncologists manually delineate varying primary gross tumor volume (pGTV) contours, adding risk to targeted radiation treatments. Thus, a method of IOV reduction is urgently needed. Hypothesizing that the radiation oncologist’s IOV may shrink with the aid of IOV maps, we propose IOV prediction network (IOV-Net), a deep-learning model that uses the fuzzy membership function to produce high-quality maps based on computed tomography (CT) images. To test the prediction accuracy, a ground-truth pGTV IOV map was created using the manual contour delineations of radiation therapy structures provided by five expert oncologists. Then, we tasked IOV-Net with producing a map of its own. The mean squared error (prediction vs. ground truth) and its standard deviation were 0.0038 and 0.0005, respectively. To test the clinical feasibility of our method, CT images were divided into two groups, and oncologists from our institution created manual contours with and without IOV map guidance. The Dice similarity coefficient and Jaccard index increased by ~6 and 7%, respectively, and the Hausdorff distance decreased by 2.5 mm, indicating a statistically significant IOV reduction (p < 0.05). Hence, IOV-net and its resultant IOV maps have the potential to improve radiation therapy efficacy worldwide.
Collapse
|
7
|
SIP-UNet: Sequential Inputs Parallel UNet Architecture for Segmentation of Brain Tissues from Magnetic Resonance Images. MATHEMATICS 2022. [DOI: 10.3390/math10152755] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Proper analysis of changes in brain structure can lead to a more accurate diagnosis of specific brain disorders. The accuracy of segmentation is crucial for quantifying changes in brain structure. In recent studies, UNet-based architectures have outperformed other deep learning architectures in biomedical image segmentation. However, improving segmentation accuracy is challenging due to the low resolution of medical images and insufficient data. In this study, we present a novel architecture that combines three parallel UNets using a residual network. This architecture improves upon the baseline methods in three ways. First, instead of using a single image as input, we use three consecutive images. This gives our model the freedom to learn from neighboring images as well. Additionally, the images are individually compressed and decompressed using three different UNets, which prevents the model from merging the features of the images. Finally, following the residual network architecture, the outputs of the UNets are combined in such a way that the features of the image corresponding to the output are enhanced by a skip connection. The proposed architecture performed better than using a single conventional UNet and other UNet variants.
Collapse
|
8
|
Vu MH, Norman G, Nyholm T, Lofstedt T. A Data-Adaptive Loss Function for Incomplete Data and Incremental Learning in Semantic Image Segmentation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1320-1330. [PMID: 34965206 DOI: 10.1109/tmi.2021.3139161] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
In the last years, deep learning has dramatically improved the performances in a variety of medical image analysis applications. Among different types of deep learning models, convolutional neural networks have been among the most successful and they have been used in many applications in medical imaging. Training deep convolutional neural networks often requires large amounts of image data to generalize well to new unseen images. It is often time-consuming and expensive to collect large amounts of data in the medical image domain due to expensive imaging systems, and the need for experts to manually make ground truth annotations. A potential problem arises if new structures are added when a decision support system is already deployed and in use. Since the field of radiation therapy is constantly developing, the new structures would also have to be covered by the decision support system. In the present work, we propose a novel loss function to solve multiple problems: imbalanced datasets, partially-labeled data, and incremental learning. The proposed loss function adapts to the available data in order to utilize all available data, even when some have missing annotations. We demonstrate that the proposed loss function also works well in an incremental learning setting, where an existing model is easily adapted to semi-automatically incorporate delineations of new organs when they appear. Experiments on a large in-house dataset show that the proposed method performs on par with baseline models, while greatly reducing the training time and eliminating the hassle of maintaining multiple models in practice.
Collapse
|
9
|
Zhang F, Wells WM, O'Donnell LJ. Deep Diffusion MRI Registration (DDMReg): A Deep Learning Method for Diffusion MRI Registration. IEEE TRANSACTIONS ON MEDICAL IMAGING 2022; 41:1454-1467. [PMID: 34968177 PMCID: PMC9273049 DOI: 10.1109/tmi.2021.3139507] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
In this paper, we present a deep learning method, DDMReg, for accurate registration between diffusion MRI (dMRI) datasets. In dMRI registration, the goal is to spatially align brain anatomical structures while ensuring that local fiber orientations remain consistent with the underlying white matter fiber tract anatomy. DDMReg is a novel method that uses joint whole-brain and tract-specific information for dMRI registration. Based on the successful VoxelMorph framework for image registration, we propose a novel registration architecture that leverages not only whole brain information but also tract-specific fiber orientation information. DDMReg is an unsupervised method for deformable registration between pairs of dMRI datasets: it does not require nonlinearly pre-registered training data or the corresponding deformation fields as ground truth. We perform comparisons with four state-of-the-art registration methods on multiple independently acquired datasets from different populations (including teenagers, young and elderly adults) and different imaging protocols and scanners. We evaluate the registration performance by assessing the ability to align anatomically corresponding brain structures and ensure fiber spatial agreement between different subjects after registration. Experimental results show that DDMReg obtains significantly improved registration performance compared to the state-of-the-art methods. Importantly, we demonstrate successful generalization of DDMReg to dMRI data from different populations with varying ages and acquired using different acquisition protocols and different scanners.
Collapse
|
10
|
Muijzer MB, Heslinga FG, Couwenberg F, Noordmans HJ, Oahalou A, Pluim JPW, Veta M, Wisse RPL. Automatic evaluation of graft orientation during Descemet membrane endothelial keratoplasty using intraoperative OCT. BIOMEDICAL OPTICS EXPRESS 2022; 13:2683-2694. [PMID: 35774322 PMCID: PMC9203112 DOI: 10.1364/boe.446519] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/21/2021] [Revised: 12/31/2021] [Accepted: 02/24/2022] [Indexed: 05/27/2023]
Abstract
Correct Descemet Membrane Endothelial Keratoplasty (DMEK) graft orientation is imperative for success of DMEK surgery, but intraoperative evaluation can be challenging. We present a method for automatic evaluation of the graft orientation in intraoperative optical coherence tomography (iOCT), exploiting the natural rolling behavior of the graft. The method encompasses a deep learning model for graft segmentation, post-processing to obtain a smooth line representation, and curvature calculations to determine graft orientation. For an independent test set of 100 iOCT-frames, the automatic method correctly identified graft orientation in 78 frames and obtained an area under the receiver operating characteristic curve (AUC) of 0.84. When we replaced the automatic segmentation with the manual masks, the AUC increased to 0.92, corresponding to an accuracy of 86%. In comparison, two corneal specialists correctly identified graft orientation in 90% and 91% of the iOCT-frames.
Collapse
Affiliation(s)
- Marc B. Muijzer
- Utrecht Cornea Research Group, Ophthalmology Department, University Medical Center Utrecht, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
- Contributed equally
| | - Friso G. Heslinga
- Department of Biomedical Engineering, Eindhoven University of Technology, Postbus 513, 5600 MB, Eindhoven, The Netherlands
- Contributed equally
| | - Floor Couwenberg
- University of Twente, Drienerlolaan 5, 7522 NB Enschede, The Netherlands
| | - Herke-Jan Noordmans
- Medical technical and Clinical physics department, University Medical Center Utrecht, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | | | - Josien P. W. Pluim
- Department of Biomedical Engineering, Eindhoven University of Technology, Postbus 513, 5600 MB, Eindhoven, The Netherlands
- Image Sciences Institute, University Medical Center Utrecht, Utrecht University, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
| | - Mitko Veta
- Department of Biomedical Engineering, Eindhoven University of Technology, Postbus 513, 5600 MB, Eindhoven, The Netherlands
- Contributed equally
| | - Robert P. L. Wisse
- Utrecht Cornea Research Group, Ophthalmology Department, University Medical Center Utrecht, Heidelberglaan 100, 3584CX, Utrecht, The Netherlands
- Contributed equally
| |
Collapse
|
11
|
Szmul A, Chandy E, Veiga C, Jacob J, Stavropoulou A, Landau D, Hiley CT, McClelland JR. A Novel and Automated Approach to Classify Radiation Induced Lung Tissue Damage on CT Scans. Cancers (Basel) 2022; 14:1341. [PMID: 35267649 PMCID: PMC8909378 DOI: 10.3390/cancers14051341] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2021] [Revised: 02/18/2022] [Accepted: 02/25/2022] [Indexed: 02/01/2023] Open
Abstract
Radiation-induced lung damage (RILD) is a common side effect of radiotherapy (RT). The ability to automatically segment, classify, and quantify different types of lung parenchymal change is essential to uncover underlying patterns of RILD and their evolution over time. A RILD dedicated tissue classification system was developed to describe lung parenchymal tissue changes on a voxel-wise level. The classification system was automated for segmentation of five lung tissue classes on computed tomography (CT) scans that described incrementally increasing tissue density, ranging from normal lung (Class 1) to consolidation (Class 5). For ground truth data generation, we employed a two-stage data annotation approach, akin to active learning. Manual segmentation was used to train a stage one auto-segmentation method. These results were manually refined and used to train the stage two auto-segmentation algorithm. The stage two auto-segmentation algorithm was an ensemble of six 2D Unets using different loss functions and numbers of input channels. The development dataset used in this study consisted of 40 cases, each with a pre-radiotherapy, 3-, 6-, 12-, and 24-month follow-up CT scans (n = 200 CT scans). The method was assessed on a hold-out test dataset of 6 cases (n = 30 CT scans). The global Dice score coefficients (DSC) achieved for each tissue class were: Class (1) 99% and 98%, Class (2) 71% and 44%, Class (3) 56% and 26%, Class (4) 79% and 47%, and Class (5) 96% and 92%, for development and test subsets, respectively. The lowest values for the test subsets were caused by imaging artefacts or reflected subgroups that occurred infrequently and with smaller overall parenchymal volumes. We performed qualitative evaluation on the test dataset presenting manual and auto-segmentation to a blinded independent radiologist to rate them as 'acceptable', 'minor disagreement' or 'major disagreement'. The auto-segmentation ratings were similar to the manual segmentation, both having approximately 90% of cases rated as acceptable. The proposed framework for auto-segmentation of different lung tissue classes produces acceptable results in the majority of cases and has the potential to facilitate future large studies of RILD.
Collapse
Affiliation(s)
- Adam Szmul
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
| | - Edward Chandy
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
- Sussex Cancer Centre, Royal Sussex County Hospital, Brighton BN2 5BE, UK
- UCL Cancer Institute, University College London, London WC1E 6BT, UK; (D.L.); (C.T.H.)
| | - Catarina Veiga
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
| | - Joseph Jacob
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
- UCL Respiratory Department, University College London Hospital, London NW1 2PG, UK
| | - Alkisti Stavropoulou
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
| | - David Landau
- UCL Cancer Institute, University College London, London WC1E 6BT, UK; (D.L.); (C.T.H.)
| | - Crispin T. Hiley
- UCL Cancer Institute, University College London, London WC1E 6BT, UK; (D.L.); (C.T.H.)
- University College Hospital, University College London, London NW1 2BU, UK
| | - Jamie R. McClelland
- Centre for Medical Image Computing, Department of Medical Physics and Biomedical Engineering, University College London, London WC1E 6BT, UK; (E.C.); (C.V.); (J.J.); (A.S.); (J.R.M.)
| |
Collapse
|
12
|
Lei T, Wang R, Zhang Y, Wan Y, Liu C, Nandi AK. DefED-Net: Deformable Encoder-Decoder Network for Liver and Liver Tumor Segmentation. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2022. [DOI: 10.1109/trpms.2021.3059780] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
|
13
|
Araújo JDL, da Cruz LB, Diniz JOB, Ferreira JL, Silva AC, de Paiva AC, Gattass M. Liver segmentation from computed tomography images using cascade deep learning. Comput Biol Med 2022; 140:105095. [PMID: 34902610 DOI: 10.1016/j.compbiomed.2021.105095] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 11/17/2021] [Accepted: 11/27/2021] [Indexed: 12/18/2022]
Abstract
BACKGROUND Liver segmentation is a fundamental step in the treatment planning and diagnosis of liver cancer. However, manual segmentation of liver is time-consuming because of the large slice quantity and subjectiveness associated with the specialist's experience, which can lead to segmentation errors. Thus, the segmentation process can be automated using computational methods for better time efficiency and accuracy. However, automatic liver segmentation is a challenging task, as the liver can vary in shape, ill-defined borders, and lesions, which affect its appearance. We aim to propose an automatic method for liver segmentation using computed tomography (CT) images. METHODS The proposed method, based on deep convolutional neural network models and image processing techniques, comprise of four main steps: (1) image preprocessing, (2) initial segmentation, (3) reconstruction, and (4) final segmentation. RESULTS We evaluated the proposed method using 131 CT images from the LiTS image base. An average sensitivity of 95.45%, an average specificity of 99.86%, an average Dice coefficient of 95.64%, an average volumetric overlap error (VOE) of 8.28%, an average relative volume difference (RVD) of -0.41%, and an average Hausdorff distance (HD) of 26.60 mm were achieved. CONCLUSIONS This study demonstrates that liver segmentation, even when lesions are present in CT images, can be efficiently performed using a cascade approach and including a reconstruction step based on deep convolutional neural networks.
Collapse
Affiliation(s)
- José Denes Lima Araújo
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil.
| | - Luana Batista da Cruz
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil.
| | - João Otávio Bandeira Diniz
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil; Federal Institute of Maranhão, BR-226, SN, Campus Grajaú, Vila Nova, 65 940-000, Grajaú, MA, Brazil.
| | - Jonnison Lima Ferreira
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil; Federal Institute of Amazonas, Rua Santos Dumont, SN, Campus Tabatinga, Vila Verde, 69 640-000, Tabatinga, AM, Brazil.
| | - Aristófanes Corrêa Silva
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil.
| | - Anselmo Cardoso de Paiva
- Applied Computing Group (NCA - UFMA), Federal University of Maranhão, Av. dos Portugueses, SN, Campus do Bacanga, Bacanga, 65 085-580, São Luís, MA, Brazil.
| | - Marcelo Gattass
- Pontifical Catholic University of Rio de Janeiro, R. São Vicente, 225, Gávea, 22 453-900, Rio de Janeiro, RJ, Brazil.
| |
Collapse
|
14
|
Fang Y, Wang J, Ou X, Ying H, Hu C, Zhang Z, Hu W. The impact of training sample size on deep learning-based organ auto-segmentation for head-and-neck patients. Phys Med Biol 2021; 66. [PMID: 34450599 DOI: 10.1088/1361-6560/ac2206] [Citation(s) in RCA: 25] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Accepted: 08/27/2021] [Indexed: 12/23/2022]
Abstract
To investigate the impact of training sample size on the performance of deep learning-based organ auto-segmentation for head-and-neck cancer patients, a total of 1160 patients with head-and-neck cancer who received radiotherapy were enrolled in this study. Patient planning CT images and regions of interest (ROIs) delineation, including the brainstem, spinal cord, eyes, lenses, optic nerves, temporal lobes, parotids, larynx and body, were collected. An evaluation dataset with 200 patients were randomly selected and combined with Dice similarity index to evaluate the model performances. Eleven training datasets with different sample sizes were randomly selected from the remaining 960 patients to form auto-segmentation models. All models used the same data augmentation methods, network structures and training hyperparameters. A performance estimation model of the training sample size based on the inverse power law function was established. Different performance change patterns were found for different organs. Six organs had the best performance with 800 training samples and others achieved their best performance with 600 training samples or 400 samples. The benefit of increasing the size of the training dataset gradually decreased. Compared to the best performance, optic nerves and lenses reached 95% of their best effect at 200, and the other organs reached 95% of their best effect at 40. For the fitting effect of the inverse power law function, the fitted root mean square errors of all ROIs were less than 0.03 (left eye: 0.024, others: <0.01), and theRsquare of all ROIs except for the body was greater than 0.5. The sample size has a significant impact on the performance of deep learning-based auto-segmentation. The relationship between sample size and performance depends on the inherent characteristics of the organ. In some cases, relatively small samples can achieve satisfactory performance.
Collapse
Affiliation(s)
- Yingtao Fang
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Jiazhou Wang
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Xiaomin Ou
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Hongmei Ying
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Chaosu Hu
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Zhen Zhang
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| | - Weigang Hu
- Department of Radiation Oncology, Fudan University Shanghai Cancer Center, Shanghai, People's Republic of China.,Department of Oncology, Shanghai Medical College, Fudan University, Shanghai, People's Republic of China.,Shanghai Key Laboratory of Radiation Oncology, Shanghai, People's Republic of China
| |
Collapse
|