1
|
Zhang R, Chung ACS. EfficientQ: An efficient and accurate post-training neural network quantization method for medical image segmentation. Med Image Anal 2024; 97:103277. [PMID: 39094461 DOI: 10.1016/j.media.2024.103277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2023] [Revised: 05/09/2024] [Accepted: 07/15/2024] [Indexed: 08/04/2024]
Abstract
Model quantization is a promising technique that can simultaneously compress and accelerate a deep neural network by limiting its computation bit-width, which plays a crucial role in the fast-growing AI industry. Despite model quantization's success in producing well-performing low-bit models, the quantization process itself can still be expensive, which may involve a long fine-tuning stage on a large, well-annotated training set. To make the quantization process more efficient in terms of both time and data requirements, this paper proposes a fast and accurate post-training quantization method, namely EfficientQ. We develop this new method with a layer-wise optimization strategy and leverage the powerful alternating direction method of multipliers (ADMM) algorithm to ensure fast convergence. Furthermore, a weight regularization scheme is incorporated to provide more guidance for the optimization of the discrete weights, and a self-adaptive attention mechanism is proposed to combat the class imbalance problem. Extensive comparison and ablation experiments are conducted on two publicly available medical image segmentation datasets, i.e., LiTS and BraTS2020, and the results demonstrate the superiority of the proposed method over various existing post-training quantization methods in terms of both accuracy and optimization speed. Remarkably, with EfficientQ, the quantization of a practical 3D UNet only requires less than 5 min on a single GPU and one data sample. The source code is available at https://github.com/rongzhao-zhang/EfficientQ.
Collapse
Affiliation(s)
- Rongzhao Zhang
- Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.
| | - Albert C S Chung
- Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.
| |
Collapse
|
2
|
Ji Z, Liu J, Mu J, Zhang H, Dai C, Yuan N, Ganchev I. ResDAC-Net: a novel pancreas segmentation model utilizing residual double asymmetric spatial kernels. Med Biol Eng Comput 2024; 62:2087-2100. [PMID: 38457066 PMCID: PMC11190007 DOI: 10.1007/s11517-024-03052-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Accepted: 02/13/2024] [Indexed: 03/09/2024]
Abstract
The pancreas not only is situated in a complex abdominal background but is also surrounded by other abdominal organs and adipose tissue, resulting in blurred organ boundaries. Accurate segmentation of pancreatic tissue is crucial for computer-aided diagnosis systems, as it can be used for surgical planning, navigation, and assessment of organs. In the light of this, the current paper proposes a novel Residual Double Asymmetric Convolution Network (ResDAC-Net) model. Firstly, newly designed ResDAC blocks are used to highlight pancreatic features. Secondly, the feature fusion between adjacent encoding layers fully utilizes the low-level and deep-level features extracted by the ResDAC blocks. Finally, parallel dilated convolutions are employed to increase the receptive field to capture multiscale spatial information. ResDAC-Net is highly compatible to the existing state-of-the-art models, according to three (out of four) evaluation metrics, including the two main ones used for segmentation performance evaluation (i.e., DSC and Jaccard index).
Collapse
Affiliation(s)
- Zhanlin Ji
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, China
| | - Jianuo Liu
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, China
| | - Juncheng Mu
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, China
| | - Haiyang Zhang
- Department of Computing, Xi'an Jiaotong-Liverpool University, Suzhou, China
| | - Chenxu Dai
- Department of Artificial Intelligence, North China University of Science and Technology, Tangshan, 063009, China
| | - Na Yuan
- Intelligence and Information Engineering College, Tangshan University, Tangshan, 063000, China.
| | - Ivan Ganchev
- Telecommunications Research Centre (TRC), University of Limerick, Limerick, V94 T9PX, Ireland.
- Department of Computer Systems, University of Plovdiv "Paisii Hilendarski", Plovdiv, 4000, Bulgaria.
- Institute of Mathematics and Informatics-Bulgarian Academy of Sciences, Sofia, 1040, Bulgaria.
| |
Collapse
|
3
|
Song Z, Asiedu M, Wang S, Li Q, Ozturk A, Mittal V, Schoen S, Ramaswamy S, Pierce TT, Samir AE, Eldar YC, Chandrakasan A, Kumar V. Memory-efficient low-compute segmentation algorithms for bladder-monitoring smart ultrasound devices. Sci Rep 2023; 13:16450. [PMID: 37777523 PMCID: PMC10542811 DOI: 10.1038/s41598-023-42000-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 09/04/2023] [Indexed: 10/02/2023] Open
Abstract
Post-operative urinary retention is a medical condition where patients cannot urinate despite having a full bladder. Ultrasound imaging of the bladder is used to estimate urine volume for early diagnosis and management of urine retention. Moreover, the use of bladder ultrasound can reduce the need for an indwelling urinary catheter and the risk of catheter-associated urinary tract infection. Wearable ultrasound devices combined with machine-learning based bladder volume estimation algorithms reduce the burdens of nurses in hospital settings and improve outpatient care. However, existing algorithms are memory and computation intensive, thereby demanding the use of expensive GPUs. In this paper, we develop and validate a low-compute memory-efficient deep learning model for accurate bladder region segmentation and urine volume calculation. B-mode ultrasound bladder images of 360 patients were divided into training and validation sets; another 74 patients were used as the test dataset. Our 1-bit quantized models with 4-bits and 6-bits skip connections achieved an accuracy within [Formula: see text] and [Formula: see text], respectively, of a full precision state-of-the-art neural network (NN) without any floating-point operations and with an [Formula: see text] and [Formula: see text] reduction in memory requirements to fit under 150 kB. The means and standard deviations of the volume estimation errors, relative to estimates from ground-truth clinician annotations, were [Formula: see text] ml and [Formula: see text] ml, respectively. This lightweight NN can be easily integrated on the wearable ultrasound device for automated and continuous monitoring of urine volume. Our approach can potentially be extended to other clinical applications, such as monitoring blood pressure and fetal heart rate.
Collapse
Affiliation(s)
- Zhiye Song
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
| | - Mercy Asiedu
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | - Shuhang Wang
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | - Qian Li
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
- Department of Ultrasound, Shenzhen University General Hospital, Shenzhen, Guangdong, China
| | - Arinc Ozturk
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | - Vipasha Mittal
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Scott Schoen
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | | | - Theodore T Pierce
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | - Anthony E Samir
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| | - Yonina C Eldar
- Department of Computer Science and Applied Mathematics, Weizmann institute of Science, Rehovot, Israel
| | - Anantha Chandrakasan
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Viksit Kumar
- Center for Ultrasound Research and Translation, Massachusetts General Hospital, Boston, MA, USA
| |
Collapse
|
4
|
Multi-target segmentation of pancreas and pancreatic tumor based on fusion of attention mechanism. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
5
|
Huang ML, Wu YZ. Semantic segmentation of pancreatic medical images by using convolutional neural network. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103458] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
6
|
Abstract
Artificial intelligence (AI) has illuminated a clear path towards an evolving health-care system replete with enhanced precision and computing capabilities. Medical imaging analysis can be strengthened by machine learning as the multidimensional data generated by imaging naturally lends itself to hierarchical classification. In this Review, we describe the role of machine intelligence in image-based endocrine cancer diagnostics. We first provide a brief overview of AI and consider its intuitive incorporation into the clinical workflow. We then discuss how AI can be applied for the characterization of adrenal, pancreatic, pituitary and thyroid masses in order to support clinicians in their diagnostic interpretations. This Review also puts forth a number of key evaluation criteria for machine learning in medicine that physicians can use in their appraisals of these algorithms. We identify mitigation strategies to address ongoing challenges around data availability and model interpretability in the context of endocrine cancer diagnosis. Finally, we delve into frontiers in systems integration for AI, discussing automated pipelines and evolving computing platforms that leverage distributed, decentralized and quantum techniques.
Collapse
Affiliation(s)
| | - Ihab R Kamel
- Department of Imaging & Imaging Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Harrison X Bai
- Department of Imaging & Imaging Sciences, Johns Hopkins University School of Medicine, Baltimore, MD, USA.
| |
Collapse
|
7
|
Zhang R, Chung ACS. MedQ: Lossless ultra-low-bit neural network quantization for medical image segmentation. Med Image Anal 2021; 73:102200. [PMID: 34416578 DOI: 10.1016/j.media.2021.102200] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2020] [Revised: 06/30/2021] [Accepted: 07/26/2021] [Indexed: 10/20/2022]
Abstract
Implementing deep convolutional neural networks (CNNs) with boolean arithmetic is ideal for eliminating the notoriously high computational expense of deep learning models. However, although lossless model compression via weight-only quantization has been achieved in previous works, it is still an open problem about how to reduce the computation precision of CNNs without losing performance, especially for medical image segmentation tasks where data dimension is high and annotation is scarce. This paper presents a novel CNN quantization framework that can squeeze a deep model (both parameters and activation) to extremely low bitwidth, e.g., 1∼2 bits, while maintaining its high performance. In the new method, we first design a strong baseline quantizer with an optimizable quantization range. Then, to relieve the back-propagation difficulty caused by the discontinuous quantization function, we design a radical residual connection scheme that allows gradients to flow through every quantized layer freely. Moreover, a tanh-based derivative function is used to further boost gradient flow and a distributional loss is employed to regularize the model output. Extensive experiments and ablation studies are conducted on two well-established public 3D segmentation datasets, i.e., BRATS2020 and LiTS. Experimental results evidence that our framework not only outperforms state-of-the-art quantization approaches significantly, but also achieves lossless performance on both datasets with ternary (2-bit) quantization.
Collapse
Affiliation(s)
- Rongzhao Zhang
- Lo Kwee-Seong Medical Image Analysis Laboratory, Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong.
| | - Albert C S Chung
- Lo Kwee-Seong Medical Image Analysis Laboratory, Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong.
| |
Collapse
|
8
|
Sengupta K, Srivastava PR. Quantum algorithm for quicker clinical prognostic analysis: an application and experimental study using CT scan images of COVID-19 patients. BMC Med Inform Decis Mak 2021; 21:227. [PMID: 34330278 PMCID: PMC8323083 DOI: 10.1186/s12911-021-01588-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 07/18/2021] [Indexed: 12/31/2022] Open
Abstract
BACKGROUND In medical diagnosis and clinical practice, diagnosing a disease early is crucial for accurate treatment, lessening the stress on the healthcare system. In medical imaging research, image processing techniques tend to be vital in analyzing and resolving diseases with a high degree of accuracy. This paper establishes a new image classification and segmentation method through simulation techniques, conducted over images of COVID-19 patients in India, introducing the use of Quantum Machine Learning (QML) in medical practice. METHODS This study establishes a prototype model for classifying COVID-19, comparing it with non-COVID pneumonia signals in Computed tomography (CT) images. The simulation work evaluates the usage of quantum machine learning algorithms, while assessing the efficacy for deep learning models for image classification problems, and thereby establishes performance quality that is required for improved prediction rate when dealing with complex clinical image data exhibiting high biases. RESULTS The study considers a novel algorithmic implementation leveraging quantum neural network (QNN). The proposed model outperformed the conventional deep learning models for specific classification task. The performance was evident because of the efficiency of quantum simulation and faster convergence property solving for an optimization problem for network training particularly for large-scale biased image classification task. The model run-time observed on quantum optimized hardware was 52 min, while on K80 GPU hardware it was 1 h 30 min for similar sample size. The simulation shows that QNN outperforms DNN, CNN, 2D CNN by more than 2.92% in gain in accuracy measure with an average recall of around 97.7%. CONCLUSION The results suggest that quantum neural networks outperform in COVID-19 traits' classification task, comparing to deep learning w.r.t model efficacy and training time. However, a further study needs to be conducted to evaluate implementation scenarios by integrating the model within medical devices.
Collapse
Affiliation(s)
- Kinshuk Sengupta
- Microsoft Corporation, New Delhi
, India
- Department of Information System, Indian Institute of Management, Rohtak, India
- City Southern Bypass, Sunaria, Rohtak, Haryana 124010 India
| | - Praveen Ranjan Srivastava
- Department of Information System, Indian Institute of Management, Rohtak, India
- City Southern Bypass, Sunaria, Rohtak, Haryana 124010 India
| |
Collapse
|
9
|
Huang M, Huang C, Yuan J, Kong D. A Semiautomated Deep Learning Approach for Pancreas Segmentation. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:3284493. [PMID: 34306587 PMCID: PMC8272661 DOI: 10.1155/2021/3284493] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/25/2021] [Revised: 05/28/2021] [Accepted: 06/21/2021] [Indexed: 12/03/2022]
Abstract
Accurate pancreas segmentation from 3D CT volumes is important for pancreas diseases therapy. It is challenging to accurately delineate the pancreas due to the poor intensity contrast and intrinsic large variations in volume, shape, and location. In this paper, we propose a semiautomated deformable U-Net, i.e., DUNet for the pancreas segmentation. The key innovation of our proposed method is a deformable convolution module, which adaptively adds learned offsets to each sampling position of 2D convolutional kernel to enhance feature representation. Combining deformable convolution module with U-Net enables our DUNet to flexibly capture pancreatic features and improve the geometric modeling capability of U-Net. Moreover, a nonlinear Dice-based loss function is designed to tackle the class-imbalanced problem in the pancreas segmentation. Experimental results show that our proposed method outperforms all comparison methods on the same NIH dataset.
Collapse
Affiliation(s)
- Meixiang Huang
- The School of Mathematical Sciences, Zhejiang University, Hangzhou 310027, China
| | - Chongfei Huang
- The School of Mathematical Sciences, Zhejiang University, Hangzhou 310027, China
| | - Jing Yuan
- The School of Mathematical Sciences, Zhejiang University, Hangzhou 310027, China
- The School of Mathematics and Statistics, Xidian University, Xi'an 710069, China
| | - Dexing Kong
- The School of Mathematical Sciences, Zhejiang University, Hangzhou 310027, China
| |
Collapse
|
10
|
Shirokikh B, Shevtsov A, Dalechina A, Krivov E, Kostjuchenko V, Golanov A, Gombolevskiy V, Morozov S, Belyaev M. Accelerating 3D Medical Image Segmentation by Adaptive Small-Scale Target Localization. J Imaging 2021; 7:35. [PMID: 34460634 PMCID: PMC8321270 DOI: 10.3390/jimaging7020035] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 01/28/2021] [Accepted: 02/05/2021] [Indexed: 11/25/2022] Open
Abstract
The prevailing approach for three-dimensional (3D) medical image segmentation is to use convolutional networks. Recently, deep learning methods have achieved human-level performance in several important applied problems, such as volumetry for lung-cancer diagnosis or delineation for radiation therapy planning. However, state-of-the-art architectures, such as U-Net and DeepMedic, are computationally heavy and require workstations accelerated with graphics processing units for fast inference. However, scarce research has been conducted concerning enabling fast central processing unit computations for such networks. Our paper fills this gap. We propose a new segmentation method with a human-like technique to segment a 3D study. First, we analyze the image at a small scale to identify areas of interest and then process only relevant feature-map patches. Our method not only reduces the inference time from 10 min to 15 s but also preserves state-of-the-art segmentation quality, as we illustrate in the set of experiments with two large datasets.
Collapse
Affiliation(s)
- Boris Shirokikh
- Center for Neurobiology and Brain Restoration, Skolkovo Institute of Science and Technology, 121205 Moscow, Russia; (A.S.); (M.B.)
| | - Alexey Shevtsov
- Center for Neurobiology and Brain Restoration, Skolkovo Institute of Science and Technology, 121205 Moscow, Russia; (A.S.); (M.B.)
- Sector of Data Analysis for Neuroscience, Kharkevich Institute for Information Transmission Problems, 127051 Moscow, Russia;
- Department of Radio Engineering and Cybernetics, Moscow Institute of Physics and Technology, 141701 Moscow, Russia
| | | | - Egor Krivov
- Sector of Data Analysis for Neuroscience, Kharkevich Institute for Information Transmission Problems, 127051 Moscow, Russia;
- Department of Radio Engineering and Cybernetics, Moscow Institute of Physics and Technology, 141701 Moscow, Russia
| | | | - Andrey Golanov
- Department of Radiosurgery and Radiation, Burdenko Neurosurgery Institute, 125047 Moscow, Russia;
| | - Victor Gombolevskiy
- Medical Research Department, Research and Practical Clinical Center of Diagnostics and Telemedicine Technologies of the Department of Health Care of Moscow, 127051 Moscow, Russia; (V.G.); (S.M.)
| | - Sergey Morozov
- Medical Research Department, Research and Practical Clinical Center of Diagnostics and Telemedicine Technologies of the Department of Health Care of Moscow, 127051 Moscow, Russia; (V.G.); (S.M.)
| | - Mikhail Belyaev
- Center for Neurobiology and Brain Restoration, Skolkovo Institute of Science and Technology, 121205 Moscow, Russia; (A.S.); (M.B.)
| |
Collapse
|
11
|
Bagheri MH, Roth H, Kovacs W, Yao J, Farhadi F, Li X, Summers RM. Technical and Clinical Factors Affecting Success Rate of a Deep Learning Method for Pancreas Segmentation on CT. Acad Radiol 2020; 27:689-695. [PMID: 31537506 DOI: 10.1016/j.acra.2019.08.014] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2019] [Revised: 08/21/2019] [Accepted: 08/27/2019] [Indexed: 02/07/2023]
Abstract
PURPOSE Accurate pancreas segmentation has application in surgical planning, assessment of diabetes, and detection and analysis of pancreatic tumors. Factors that affect pancreas segmentation accuracy have not been previously reported. The purpose of this study is to identify technical and clinical factors that adversely affect the accuracy of pancreas segmentation on CT. METHOD AND MATERIALS In this IRB and HIPAA compliant study, a deep convolutional neural network was used for pancreas segmentation in a publicly available archive of 82 portal-venous phase abdominal CT scans of 53 men and 29 women. The accuracies of the segmentations were evaluated by the Dice similarity coefficient (DSC). The DSC was then correlated with demographic and clinical data (age, gender, height, weight, body mass index), CT technical factors (image pixel size, slice thickness, presence or absence of oral contrast), and CT imaging findings (volume and attenuation of pancreas, visceral abdominal fat, and CT attenuation of the structures within a 5 mm neighborhood of the pancreas). RESULTS The average DSC was 78% ± 8%. Factors that were statistically significantly correlated with DSC included body mass index (r = 0.34, p < 0.01), visceral abdominal fat (r = 0.51, p < 0.0001), volume of the pancreas (r = 0.41, p = 0.001), standard deviation of CT attenuation within the pancreas (r = 0.30, p = 0.01), and median and average CT attenuation in the immediate neighborhood of the pancreas (r = -0.53, p < 0.0001 and r = -0.52, p < 0.0001). There were no significant correlations between the DSC and the height, gender, or mean CT attenuation of the pancreas. CONCLUSION Increased visceral abdominal fat and accumulation of fat within or around the pancreas are major factors associated with more accurate segmentation of the pancreas. Potential applications of our findings include assessment of pancreas segmentation difficulty of a particular scan or dataset and identification of methods that work better for more challenging pancreas segmentations.
Collapse
|
12
|
|
13
|
Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D. Attention gated networks: Learning to leverage salient regions in medical images. Med Image Anal 2019; 53:197-207. [PMID: 30802813 PMCID: PMC7610718 DOI: 10.1016/j.media.2019.01.012] [Citation(s) in RCA: 533] [Impact Index Per Article: 106.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2018] [Revised: 01/15/2019] [Accepted: 01/18/2019] [Indexed: 02/07/2023]
Abstract
We propose a novel attention gate (AG) model for medical image analysis that automatically learns to focus on target structures of varying shapes and sizes. Models trained with AGs implicitly learn to suppress irrelevant regions in an input image while highlighting salient features useful for a specific task. This enables us to eliminate the necessity of using explicit external tissue/organ localisation modules when using convolutional neural networks (CNNs). AGs can be easily integrated into standard CNN models such as VGG or U-Net architectures with minimal computational overhead while increasing the model sensitivity and prediction accuracy. The proposed AG models are evaluated on a variety of tasks, including medical image classification and segmentation. For classification, we demonstrate the use case of AGs in scan plane detection for fetal ultrasound screening. We show that the proposed attention mechanism can provide efficient object localisation while improving the overall prediction performance by reducing false positives. For segmentation, the proposed architecture is evaluated on two large 3D CT abdominal datasets with manual annotations for multiple organs. Experimental results show that AG models consistently improve the prediction performance of the base architectures across different datasets and training sizes while preserving computational efficiency. Moreover, AGs guide the model activations to be focused around salient regions, which provides better insights into how model predictions are made. The source code for the proposed AG models is publicly available.
Collapse
Affiliation(s)
- Jo Schlemper
- BioMedIA, Imperial College London, London, SW7 2AZ, UK.
| | - Ozan Oktay
- BioMedIA, Imperial College London, London, SW7 2AZ, UK; HeartFlow, Redwood City, CA 94063, USA.
| | | | | | | | - Ben Glocker
- BioMedIA, Imperial College London, London, SW7 2AZ, UK
| | | |
Collapse
|
14
|
Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients. Int J Comput Assist Radiol Surg 2018; 14:43-52. [PMID: 30430361 DOI: 10.1007/s11548-018-1888-2] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2018] [Accepted: 11/06/2018] [Indexed: 01/19/2023]
Abstract
PURPOSE Deep convolutional neural networks in their various forms are currently achieving or outperforming state-of-the-art results on several medical imaging tasks. We aim to make these developments available to the so far unsolved task of accurate correspondence finding-especially with regard to image registration. METHODS We propose a two-step hybrid approach to make deep learned features accessible to a discrete optimization-based registration method. In a first step, in order to extract expressive binary local descriptors, we train a deep network architecture on a patch-based landmark retrieval problem as auxiliary task. As second step at runtime within a MRF-regularised dense displacement sampling, their binary nature enables highly efficient similarity computations, thus making them an ideal candidate to replace the so far used handcrafted local feature descriptors during the registration process. RESULTS We evaluate our approach on finding correspondences between highly non-rigidly deformed lung CT scans from different breathing states. Although the CNN-based descriptors excell at an auxiliary learning task for finding keypoint correspondences, self-similarity-based descriptors yield more accurate registration results. However, a combination of both approaches turns out to generate the most robust features for registration. CONCLUSION We present a three-dimensional framework for large lung motion estimation based on the combination of CNN-based and handcrafted descriptors efficiently employed in a discrete registration method. Achieving best results by combining learned and handcrafted features encourages further research in this direction.
Collapse
|