1
|
Automatic Classification of Simulated Breast Tomosynthesis Whole Images for the Presence of Microcalcification Clusters Using Deep CNNs. J Imaging 2022; 8:jimaging8090231. [PMID: 36135397 PMCID: PMC9503015 DOI: 10.3390/jimaging8090231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 07/26/2022] [Accepted: 08/04/2022] [Indexed: 11/30/2022] Open
Abstract
Microcalcification clusters (MCs) are among the most important biomarkers for breast cancer, especially in cases of nonpalpable lesions. The vast majority of deep learning studies on digital breast tomosynthesis (DBT) are focused on detecting and classifying lesions, especially soft-tissue lesions, in small regions of interest previously selected. Only about 25% of the studies are specific to MCs, and all of them are based on the classification of small preselected regions. Classifying the whole image according to the presence or absence of MCs is a difficult task due to the size of MCs and all the information present in an entire image. A completely automatic and direct classification, which receives the entire image, without prior identification of any regions, is crucial for the usefulness of these techniques in a real clinical and screening environment. The main purpose of this work is to implement and evaluate the performance of convolutional neural networks (CNNs) regarding an automatic classification of a complete DBT image for the presence or absence of MCs (without any prior identification of regions). In this work, four popular deep CNNs are trained and compared with a new architecture proposed by us. The main task of these trainings was the classification of DBT cases by absence or presence of MCs. A public database of realistic simulated data was used, and the whole DBT image was taken into account as input. DBT data were considered without and with preprocessing (to study the impact of noise reduction and contrast enhancement methods on the evaluation of MCs with CNNs). The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance. Very promising results were achieved with a maximum AUC of 94.19% for the GoogLeNet. The second-best AUC value was obtained with a new implemented network, CNN-a, with 91.17%. This CNN had the particularity of also being the fastest, thus becoming a very interesting model to be considered in other studies. With this work, encouraging outcomes were achieved in this regard, obtaining similar results to other studies for the detection of larger lesions such as masses. Moreover, given the difficulty of visualizing the MCs, which are often spread over several slices, this work may have an important impact on the clinical analysis of DBT images.
Collapse
|
2
|
A Decision Support System Based on BI-RADS and Radiomic Classifiers to Reduce False Positive Breast Calcifications at Digital Breast Tomosynthesis: A Preliminary Study. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11062503] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
Digital breast tomosynthesis (DBT) studies were introduced as a successful help for the detection of calcification, which can be a primary sign of cancer. Expert radiologists are able to detect suspicious calcifications in DBT, but a high number of calcifications with non-malignant diagnosis at biopsy have been reported (false positives, FP). In this study, a radiomic approach was developed and applied on DBT images with the aim to reduce the number of benign calcifications addressed to biopsy and to give the radiologists a helpful decision support system during their diagnostic activity. This allows personalizing patient management on the basis of personalized risk. For this purpose, 49 patients showing microcalcifications on DBT images were retrospectively included, classified by BI-RADS (Breast Imaging-Reporting and Data System) and analyzed. After segmentation of microcalcifications from DBT images, radiomic features were extracted. Features were then selected with respect to their stability within different segmentations and their repeatability in test–retest studies. Stable radiomic features were used to train, validate and test (nested 10-fold cross-validation) a preliminary machine learning radiomic classifier that, combined with BI-RADS classification, allowed a reduction in FP of a factor of 2 and an improvement in positive predictive value of 50%.
Collapse
|
3
|
Martín-Noguerol T, Paulano-Godino F, López-Ortega R, Górriz JM, Riascos RF, Luna A. Artificial intelligence in radiology: relevance of collaborative work between radiologists and engineers for building a multidisciplinary team. Clin Radiol 2020; 76:317-324. [PMID: 33358195 DOI: 10.1016/j.crad.2020.11.113] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2020] [Accepted: 11/20/2020] [Indexed: 12/13/2022]
Abstract
The use of artificial intelligence (AI) algorithms in the field of radiology is becoming more common. Several studies have demonstrated the potential utility of machine learning (ML) and deep learning (DL) techniques as aids for radiologists to solve specific radiological challenges. The decision-making process, the establishment of specific clinical or radiological targets, the profile of the different professionals involved in the development of AI solutions, and the relation with partnerships and stakeholders are only some of the main issues that have to be faced and solved prior to starting the development of radiological AI solutions. Among all the players in this multidisciplinary team, the communication between radiologists and data scientists is essential for a successful collaborative work. There are specific skills that are inherent to radiological and medical training that are critical for identifying anatomical or clinical targets as well as for segmenting or labelling lesions. These skills would then have to be transferred, explained, and taught to the data science experts to facilitate their comprehension and integration into ML or DL algorithms. On the other hand, there is a wide range of complex software packages, deep neural-network architectures, and data transfer processes for which radiologists need the expertise of software engineers and data scientists in order to select the optimal manner to analyse and post-process this amount of data. This paper offers a summary of the top five challenges faced by radiologists and data scientists including tips and tricks to build a successful AI team.
Collapse
Affiliation(s)
| | | | | | - J M Górriz
- Department of Signal Theory, Telematics and Communications, University of Granada, Granada, Spain
| | - R F Riascos
- Department of Neuroradiology, The University of Texas Health Science Center at Houston, McGovern Medical School, Houston, TX, USA
| | - A Luna
- MRI Unit, Radiology Department. HT Medica, Jaén, Spain
| |
Collapse
|
4
|
Mota AM, Clarkson MJ, Almeida P, Peralta L, Matela N. Impact of total variation minimization in volume rendering visualization of breast tomosynthesis data. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 195:105534. [PMID: 32480190 DOI: 10.1016/j.cmpb.2020.105534] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/11/2020] [Revised: 04/23/2020] [Accepted: 05/07/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Total Variation (TV) minimization algorithms have achieved great attention due to the virtue of decreasing noise while preserving edges. The purpose of this work is to implement and evaluate two TV minimization methods in 3D. Their performance is analyzed through 3D visualization of digital breast tomosynthesis (DBT) data with volume rendering. METHODS Both filters were studied with real phantom and one clinical DBT data. One algorithm was applied sequentially to all slices and the other was applied to the entire volume at once. The suitable Lagrange multiplier used in each filter equation was studied to reach the minimum 3D TV and the maximum contrast-to-noise ratio (CNR). Imaging blur was measured at 0° and 90° using two disks with different diameters (0.5 mm and 5.0 mm) and equal thickness. The quality of unfiltered and filtered data was analyzed with volume rendering at 0° and 90°. RESULTS For phantom data, with the sequential filter, a decrease of 25% in 3D TV value and an increase of 19% and 30% in CNR at 0° and 90°, respectively, were observed. When the filter is applied directly in 3D, TV value was reduced by 35% and an increase of 36% was achieved both for CNR at 0° and 90°. For the smaller disk, variations of 0% in width at half maximum (FWHM) at 0° and a decrease of about 2.5% for FWHM at 90° were observed for both filters. For the larger disk, there was a 2.5% increase in FWHM at 0° for both filters and a decrease of 6.28% and 1.69% in FWHM at 90° with the sequential filter and the 3D filter, respectively. When applied to clinical data, the performance of each filter was consistent with that obtained with the phantom. CONCLUSIONS Data analysis confirmed the relevance of these methods in improving quality of DBT images. Additionally, this type of 3D visualization showed that it may play an important complementary role in DBT imaging. It allows to visualize all DBT data at once and to analyze properly filters applied to all the three dimensions. Concise Abstract Total Variation (TV) minimization algorithms are one compressed sensing technique that has achieved great attention due to the virtue of decrease noise while preserve edges transitions. The purpose of this work is to solve the same TV minimization problem in DBT data, by studying two 3D filters. The obtained results were analyzed at 0° and 90° with a 3D visualization through volume rendering. The filters differ in their application. One considers a slice-by-slice optimization, sequentially traversing all slices of the data. The other considers the intensity values of adjacent slices to make this optimization on each voxel. The performance of each filter was also tested with a clinical case. The results obtained were very encouraging with a significantly increased contrast to noise ratio at 0° and 90° and a small reduction in blur at 90° (slight reduction of the out-of-plane artifact).
Collapse
Affiliation(s)
- A M Mota
- Faculdade de Ciências, Instituto de Biofísica e Engenharia Biomédica, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal.
| | - M J Clarkson
- Department of Medical Physics and Biomedical Engineering and the Centre for Medical Image Computing, University College London, London, UK
| | - P Almeida
- Faculdade de Ciências, Instituto de Biofísica e Engenharia Biomédica, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal
| | - L Peralta
- Departamento de Física da Faculdade de Ciências da Universidade de Lisboa, Lisboa, Portugal; Laboratório de Instrumentação e Física Experimental de Partículas, Lisboa, Portugal
| | - N Matela
- Faculdade de Ciências, Instituto de Biofísica e Engenharia Biomédica, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal
| |
Collapse
|
5
|
Martín Noguerol T, Paulano-Godino F, Martín-Valdivia MT, Menias CO, Luna A. Strengths, Weaknesses, Opportunities, and Threats Analysis of Artificial Intelligence and Machine Learning Applications in Radiology. J Am Coll Radiol 2020; 16:1239-1247. [PMID: 31492401 DOI: 10.1016/j.jacr.2019.05.047] [Citation(s) in RCA: 90] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2019] [Revised: 05/26/2019] [Accepted: 05/29/2019] [Indexed: 12/13/2022]
Abstract
Currently, the use of artificial intelligence (AI) in radiology, particularly machine learning (ML), has become a reality in clinical practice. Since the end of the last century, several ML algorithms have been introduced for a wide range of common imaging tasks, not only for diagnostic purposes but also for image acquisition and postprocessing. AI is now recognized to be a driving initiative in every aspect of radiology. There is growing evidence of the advantages of AI in radiology creating seamless imaging workflows for radiologists or even replacing radiologists. Most of the current AI methods have some internal and external disadvantages that are impeding their ultimate implementation in the clinical arena. As such, AI can be considered a portion of a business trying to be introduced in the health care market. For this reason, this review analyzes the current status of AI, and specifically ML, applied to radiology from the scope of strengths, weaknesses, opportunities, and threats (SWOT) analysis.
Collapse
Affiliation(s)
| | | | - María Teresa Martín-Valdivia
- SINAI Research Group, Computer Science Department, Advanced Studies Center in ICT (CEATIC), Universidad de Jaén, Jaén, Spain
| | | | - Antonio Luna
- MRI Unit, Radiology Department, Health Time, Jaén, Spain.
| |
Collapse
|
6
|
Chan HP, Samala RK, Hadjiiski LM. CAD and AI for breast cancer-recent development and challenges. Br J Radiol 2020; 93:20190580. [PMID: 31742424 PMCID: PMC7362917 DOI: 10.1259/bjr.20190580] [Citation(s) in RCA: 80] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2019] [Revised: 11/13/2019] [Accepted: 11/17/2019] [Indexed: 12/15/2022] Open
Abstract
Computer-aided diagnosis (CAD) has been a popular area of research and development in the past few decades. In CAD, machine learning methods and multidisciplinary knowledge and techniques are used to analyze the patient information and the results can be used to assist clinicians in their decision making process. CAD may analyze imaging information alone or in combination with other clinical data. It may provide the analyzed information directly to the clinician or correlate the analyzed results with the likelihood of certain diseases based on statistical modeling of the past cases in the population. CAD systems can be developed to provide decision support for many applications in the patient care processes, such as lesion detection, characterization, cancer staging, treatment planning and response assessment, recurrence and prognosis prediction. The new state-of-the-art machine learning technique, known as deep learning (DL), has revolutionized speech and text recognition as well as computer vision. The potential of major breakthrough by DL in medical image analysis and other CAD applications for patient care has brought about unprecedented excitement of applying CAD, or artificial intelligence (AI), to medicine in general and to radiology in particular. In this paper, we will provide an overview of the recent developments of CAD using DL in breast imaging and discuss some challenges and practical issues that may impact the advancement of artificial intelligence and its integration into clinical workflow.
Collapse
Affiliation(s)
- Heang-Ping Chan
- Department of Radiology, University of Michigan, Ann Arbor, MI, United States
| | - Ravi K. Samala
- Department of Radiology, University of Michigan, Ann Arbor, MI, United States
| | | |
Collapse
|
7
|
Sutphin C, Olson E, Motai Y, Lee SJ, Kim JG, Takabe K. Elastographic Tomosynthesis From X-Ray Strain Imaging of Breast Cancer. IEEE JOURNAL OF TRANSLATIONAL ENGINEERING IN HEALTH AND MEDICINE-JTEHM 2019; 7:4300312. [PMID: 31497411 PMCID: PMC6726464 DOI: 10.1109/jtehm.2019.2935721] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/25/2019] [Revised: 06/12/2019] [Accepted: 08/07/2019] [Indexed: 11/18/2022]
Abstract
Noncancerous breast tissue and cancerous breast tissue have different elastic properties. In particular, cancerous breast tumors are stiff when compared to the noncancerous surrounding tissue. This difference in elasticity can be used as a means for detection through the method of elastographic tomosynthesis by means of physical modulation. This paper deals with a method to visualize elasticity of soft tissues, particularly breast tissues, via x-ray tomosynthesis. X-ray tomosynthesis is now used to visualize breast tissues with better resolution than the conventional single-shot mammography. The advantage of X-ray tomosynthesis over X-ray CT is that fewer projections are needed than CT to perform the reconstruction, thus radiation exposure and cost are both reduced. Two phantoms were used for the testing of this method, a physical phantom and an in silico phantom. The standard root mean square error in the tomosynthesis for the physical phantom was 2.093 and the error in the in silico phantom was negligible. The elastographs were created through the use of displacement and strain graphing. A Gaussian Mixture Model with an expectation–maximization clustering algorithm was applied in three dimensions with an error of 16.667%. The results of this paper have been substantial when using phantom data. There are no equivalent comparisons yet in 3D x-ray elastographic tomosynthesis. Tomosynthesis with and without physical modulation in the 3D elastograph can identify feature groupings used for biopsy. The studies have potential to be applied to human test data used as a guide for biopsy to improve accuracy of diagnosis results. Further research on this topic could prove to yield new techniques for human patient diagnosis purposes.
Collapse
Affiliation(s)
- Corey Sutphin
- 1Department of Electrical and Computer EngineeringVirginia Commonwealth UniversityRichmondVA23284USA
| | - Eric Olson
- 1Department of Electrical and Computer EngineeringVirginia Commonwealth UniversityRichmondVA23284USA
| | - Yuichi Motai
- 1Department of Electrical and Computer EngineeringVirginia Commonwealth UniversityRichmondVA23284USA
| | - Suk Jin Lee
- 2TSYS School of Computer ScienceColumbus State UniversityColumbusGA31907USA
| | - Jae G Kim
- 3Imaging Software LabNano-ray Co., Ltd.Daegu601-604South Korea
| | | |
Collapse
|
8
|
Samala RK, Hadjiiski L, Helvie MA, Richter CD, Cha KH. Breast Cancer Diagnosis in Digital Breast Tomosynthesis: Effects of Training Sample Size on Multi-Stage Transfer Learning Using Deep Neural Nets. IEEE TRANSACTIONS ON MEDICAL IMAGING 2019; 38:686-696. [PMID: 31622238 PMCID: PMC6812655 DOI: 10.1109/tmi.2018.2870343] [Citation(s) in RCA: 89] [Impact Index Per Article: 14.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
In this paper, we developed a deep convolutional neural network (CNN) for the classification of malignant and benign masses in digital breast tomosynthesis (DBT) using a multi-stage transfer learning approach that utilized data from similar auxiliary domains for intermediate-stage fine-tuning. Breast imaging data from DBT, digitized screen-film mammography, and digital mammography totaling 4039 unique regions of interest (1797 malignant and 2242 benign) were collected. Using cross validation, we selected the best transfer network from six transfer networks by varying the level up to which the convolutional layers were frozen. In a single-stage transfer learning approach, knowledge from CNN trained on the ImageNet data was fine-tuned directly with the DBT data. In a multi-stage transfer learning approach, knowledge learned from ImageNet was first fine-tuned with the mammography data and then fine-tuned with the DBT data. Two transfer networks were compared for the second-stage transfer learning by freezing most of the CNN structures versus freezing only the first convolutional layer. We studied the dependence of the classification performance on training sample size for various transfer learning and fine-tuning schemes by varying the training data from 1% to 100% of the available sets. The area under the receiver operating characteristic curve (AUC) was used as a performance measure. The view-based AUC on the test set for single-stage transfer learning was 0.85 ± 0.05 and improved significantly (p <; 0.05$ ) to 0.91 ± 0.03 for multi-stage learning. This paper demonstrated that, when the training sample size from the target domain is limited, an additional stage of transfer learning using data from a similar auxiliary domain is advantageous.
Collapse
|
9
|
Zhang F, Wu S, Zhang C, Chen Q, Yang X, Jiang K, Zheng J. Multi-domain features for reducing false positives in automated detection of clustered microcalcifications in digital breast tomosynthesis. Med Phys 2019; 46:1300-1308. [PMID: 30661242 DOI: 10.1002/mp.13394] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2018] [Revised: 12/26/2018] [Accepted: 01/03/2019] [Indexed: 01/04/2023] Open
Abstract
PURPOSE In digital breast tomosynthesis (DBT) imaging, a microcalcification (MC) cluster may span across different slices and blurring exists in the out-of-focus slices. We developed a radiomics approach to extract features from focus slice and combine multiple spatial domains to reduce false positives (FPs) in an automated pipeline of detecting MC clusters. METHODS We performed a retrospective study on a cohort of 290 Chinese women patients with a total of 580 DBT volumes. We developed an automated MC detection pipeline that consists of two stages: an initial detection to identify a set of MC candidates that may include many FPs, followed by a radiomics-based classification model to identify and reduce the FPs. We extract both two-dimensional (2D) and three-dimensional (3D) radiomics features from multiple spatial domains, including a focus slice, projection image, and tomographic volume. A linear discriminant classifier was used coupled with a sequential forward feature selection procedure. The free-response operating characteristics (FROC) curve and partial area under the FROC curve (pAUC) in the FP rate range of 0 to 2 per DBT volume were used to evaluate the model's performance. RESULTS At a sensitivity of 90%, the FP rate was reduced from 1.3 to 0.2 per DBT volume after applying the multi-domain-based classification on the initial detections. The multi-domain yielded a significantly higher pAUC compared to the initial detection (increase of pAUC = 0.2278, P < 0.0001), focus slice (increase of pAUC = 0.0345, P = 0.0152), project image (increase of pAUC = 0.1043, P < 0.0001), and tomographic volume (increase of pAUC = 0.0791, P = 0.0032). CONCLUSION The radiomic features extracted from the three domains may provide complementary information and their integration can significantly reduce FPs in automated detection of MCs in DBT volumes on a large Chinese women population.
Collapse
Affiliation(s)
- Fan Zhang
- University of Science and Technology of China, Hefei, 230026, China.,Department of Medical Imaging, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Shandong Wu
- Departments of Radiology, Biomedical Informatics, Bioengineering, Intelligent Systems, and Computer Science, University of Pittsburgh, 4200 Fifth Ave, Pittsburgh, PA, 15260, USA
| | - Cheng Zhang
- Department of Medical Imaging, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Qian Chen
- Department of Radiology, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou, 215000, China
| | - Xiaodong Yang
- Department of Medical Imaging, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| | - Ke Jiang
- Department of Radiology, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou, 215000, China
| | - Jian Zheng
- Department of Medical Imaging, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, China
| |
Collapse
|
10
|
Gordon MN, Hadjiiski LM, Cha KH, Samala RK, Chan HP, Cohan RH, Caoili EM. Deep-learning convolutional neural network: Inner and outer bladder wall segmentation in CT urography. Med Phys 2019; 46:634-648. [PMID: 30520055 DOI: 10.1002/mp.13326] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2018] [Revised: 09/30/2018] [Accepted: 11/15/2018] [Indexed: 11/08/2022] Open
Abstract
PURPOSE We are developing a computerized segmentation tool for the inner and outer bladder wall as a part of an image analysis pipeline for CT urography (CTU). MATERIALS AND METHODS A data set of 172 CTU cases was collected retrospectively with Institutional Review Board (IRB) approval. The data set was randomly split into two independent sets of training (81 cases) and testing (92 cases) which were manually outlined for both the inner and outer wall. We trained a deep-learning convolutional neural network (DL-CNN) to distinguish the bladder wall from the inside and outside of the bladder using neighborhood information. Approximately, 240 000 regions of interest (ROIs) of 16 × 16 pixels in size were extracted from regions in the training cases identified by the manually outlined inner and outer bladder walls to form a training set for the DL-CNN; half of the ROIs were selected to include the bladder wall and the other half were selected to exclude the bladder wall with some of these ROIs being inside the bladder and the rest outside the bladder entirely. The DL-CNN trained on these ROIs was applied to the cases in the test set slice-by-slice to generate a bladder wall likelihood map where the gray level of a given pixel represents the likelihood that a given pixel would belong to the bladder wall. We then used the DL-CNN likelihood map as an energy term in the energy equation of a cascaded level sets method to segment the inner and outer bladder wall. The DL-CNN segmentation with level sets was compared to the three-dimensional (3D) hand-segmented contours as a reference standard. RESULTS For the inner wall contour, the training set achieved the average volume intersection, average volume error, average absolute volume error, and average distance of 90.0 ± 8.7%, -4.2 ± 18.4%, 12.9 ± 13.9%, and 3.0 ± 1.6 mm, respectively. The corresponding values for the test set were 86.9 ± 9.6%, -8.3 ± 37.7%, 18.4 ± 33.8%, and 3.4 ± 1.8 mm, respectively. For the outer wall contour, the training set achieved the values of 93.7 ± 3.9%, -7.8 ± 11.4%, 10.3 ± 9.3%, and 3.0 ± 1.2 mm, respectively. The corresponding values for the test set were 87.5 ± 9.9%, -1.2 ± 20.8%, 11.9 ± 17.0%, and 3.5 ± 2.3 mm, respectively. CONCLUSIONS Our study demonstrates that DL-CNN-assisted level sets can effectively segment bladder walls from the inner bladder and outer structures despite a lack of consistent distinctions along the inner wall. However, even with the addition of level sets, the inner and outer walls may still be over-segmented and the DL-CNN-assisted level sets may incorrectly segment parts of the prostate that overlap with the outer bladder wall. The outer wall segmentation was improved compared to our previous method and the DL-CNN-assisted level sets were also able to segment the inner bladder wall with similar performance. This study shows the DL-CNN-assisted level set segmentation tool can effectively segment the inner and outer wall of the bladder.
Collapse
Affiliation(s)
- Marshall N Gordon
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Lubomir M Hadjiiski
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Kenny H Cha
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Ravi K Samala
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Heang-Ping Chan
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Richard H Cohan
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| | - Elaine M Caoili
- Department of Radiology, The University of Michigan, Ann Arbor, MI, 48109-0904, USA
| |
Collapse
|
11
|
Bladder Cancer Treatment Response Assessment in CT using Radiomics with Deep-Learning. Sci Rep 2017; 7:8738. [PMID: 28821822 PMCID: PMC5562694 DOI: 10.1038/s41598-017-09315-w] [Citation(s) in RCA: 109] [Impact Index Per Article: 13.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2016] [Accepted: 07/18/2017] [Indexed: 02/06/2023] Open
Abstract
Cross-sectional X-ray imaging has become the standard for staging most solid organ malignancies. However, for some malignancies such as urinary bladder cancer, the ability to accurately assess local extent of the disease and understand response to systemic chemotherapy is limited with current imaging approaches. In this study, we explored the feasibility that radiomics-based predictive models using pre- and post-treatment computed tomography (CT) images might be able to distinguish between bladder cancers with and without complete chemotherapy responses. We assessed three unique radiomics-based predictive models, each of which employed different fundamental design principles ranging from a pattern recognition method via deep-learning convolution neural network (DL-CNN), to a more deterministic radiomics feature-based approach and then a bridging method between the two, utilizing a system which extracts radiomics features from the image patterns. Our study indicates that the computerized assessment using radiomics information from the pre- and post-treatment CT of bladder cancer patients has the potential to assist in assessment of treatment response.
Collapse
|
12
|
Samala RK, Chan HP, Hadjiiski L, Helvie MA, Wei J, Cha K. Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. Med Phys 2017; 43:6654. [PMID: 27908154 DOI: 10.1118/1.4967345] [Citation(s) in RCA: 138] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
PURPOSE Develop a computer-aided detection (CAD) system for masses in digital breast tomosynthesis (DBT) volume using a deep convolutional neural network (DCNN) with transfer learning from mammograms. METHODS A data set containing 2282 digitized film and digital mammograms and 324 DBT volumes were collected with IRB approval. The mass of interest on the images was marked by an experienced breast radiologist as reference standard. The data set was partitioned into a training set (2282 mammograms with 2461 masses and 230 DBT views with 228 masses) and an independent test set (94 DBT views with 89 masses). For DCNN training, the region of interest (ROI) containing the mass (true positive) was extracted from each image. False positive (FP) ROIs were identified at prescreening by their previously developed CAD systems. After data augmentation, a total of 45 072 mammographic ROIs and 37 450 DBT ROIs were obtained. Data normalization and reduction of non-uniformity in the ROIs across heterogeneous data was achieved using a background correction method applied to each ROI. A DCNN with four convolutional layers and three fully connected (FC) layers was first trained on the mammography data. Jittering and dropout techniques were used to reduce overfitting. After training with the mammographic ROIs, all weights in the first three convolutional layers were frozen, and only the last convolution layer and the FC layers were randomly initialized again and trained using the DBT training ROIs. The authors compared the performances of two CAD systems for mass detection in DBT: one used the DCNN-based approach and the other used their previously developed feature-based approach for FP reduction. The prescreening stage was identical in both systems, passing the same set of mass candidates to the FP reduction stage. For the feature-based CAD system, 3D clustering and active contour method was used for segmentation; morphological, gray level, and texture features were extracted and merged with a linear discriminant classifier to score the detected masses. For the DCNN-based CAD system, ROIs from five consecutive slices centered at each candidate were passed through the trained DCNN and a mass likelihood score was generated. The performances of the CAD systems were evaluated using free-response ROC curves and the performance difference was analyzed using a non-parametric method. RESULTS Before transfer learning, the DCNN trained only on mammograms with an AUC of 0.99 classified DBT masses with an AUC of 0.81 in the DBT training set. After transfer learning with DBT, the AUC improved to 0.90. For breast-based CAD detection in the test set, the sensitivity for the feature-based and the DCNN-based CAD systems was 83% and 91%, respectively, at 1 FP/DBT volume. The difference between the performances for the two systems was statistically significant (p-value < 0.05). CONCLUSIONS The image patterns learned from the mammograms were transferred to the mass detection on DBT slices through the DCNN. This study demonstrated that large data sets collected from mammography are useful for developing new CAD systems for DBT, alleviating the problem and effort of collecting entirely new large data sets for the new modality.
Collapse
Affiliation(s)
- Ravi K Samala
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| | - Heang-Ping Chan
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| | - Lubomir Hadjiiski
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| | - Mark A Helvie
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| | - Jun Wei
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| | - Kenny Cha
- Department of Radiology, University of Michigan, Ann Arbor, Michigan 48109
| |
Collapse
|
13
|
Cha KH, Hadjiiski L, Samala RK, Chan HP, Caoili EM, Cohan RH. Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. Med Phys 2016; 43:1882. [PMID: 27036584 DOI: 10.1118/1.4944498] [Citation(s) in RCA: 171] [Impact Index Per Article: 19.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
Abstract
PURPOSE The authors are developing a computerized system for bladder segmentation in CT urography (CTU) as a critical component for computer-aided detection of bladder cancer. METHODS A deep-learning convolutional neural network (DL-CNN) was trained to distinguish between the inside and the outside of the bladder using 160 000 regions of interest (ROI) from CTU images. The trained DL-CNN was used to estimate the likelihood of an ROI being inside the bladder for ROIs centered at each voxel in a CTU case, resulting in a likelihood map. Thresholding and hole-filling were applied to the map to generate the initial contour for the bladder, which was then refined by 3D and 2D level sets. The segmentation performance was evaluated using 173 cases: 81 cases in the training set (42 lesions, 21 wall thickenings, and 18 normal bladders) and 92 cases in the test set (43 lesions, 36 wall thickenings, and 13 normal bladders). The computerized segmentation accuracy using the DL likelihood map was compared to that using a likelihood map generated by Haar features and a random forest classifier, and that using our previous conjoint level set analysis and segmentation system (CLASS) without using a likelihood map. All methods were evaluated relative to the 3D hand-segmented reference contours. RESULTS With DL-CNN-based likelihood map and level sets, the average volume intersection ratio, average percent volume error, average absolute volume error, average minimum distance, and the Jaccard index for the test set were 81.9% ± 12.1%, 10.2% ± 16.2%, 14.0% ± 13.0%, 3.6 ± 2.0 mm, and 76.2% ± 11.8%, respectively. With the Haar-feature-based likelihood map and level sets, the corresponding values were 74.3% ± 12.7%, 13.0% ± 22.3%, 20.5% ± 15.7%, 5.7 ± 2.6 mm, and 66.7% ± 12.6%, respectively. With our previous CLASS with local contour refinement (LCR) method, the corresponding values were 78.0% ± 14.7%, 16.5% ± 16.8%, 18.2% ± 15.0%, 3.8 ± 2.3 mm, and 73.9% ± 13.5%, respectively. CONCLUSIONS The authors demonstrated that the DL-CNN can overcome the strong boundary between two regions that have large difference in gray levels and provides a seamless mask to guide level set segmentation, which has been a problem for many gradient-based segmentation methods. Compared to our previous CLASS with LCR method, which required two user inputs to initialize the segmentation, DL-CNN with level sets achieved better segmentation performance while using a single user input. Compared to the Haar-feature-based likelihood map, the DL-CNN-based likelihood map could guide the level sets to achieve better segmentation. The results demonstrate the feasibility of our new approach of using DL-CNN in combination with level sets for segmentation of the bladder.
Collapse
Affiliation(s)
- Kenny H Cha
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| | - Lubomir Hadjiiski
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| | - Ravi K Samala
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| | - Heang-Ping Chan
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| | - Elaine M Caoili
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| | - Richard H Cohan
- Department of Radiology, The University of Michigan, Ann Arbor, Michigan 48109-0904
| |
Collapse
|
14
|
Cha KH, Hadjiiski LM, Samala RK, Chan HP, Cohan RH, Caoili EM, Paramagul C, Alva A, Weizer AZ. Bladder Cancer Segmentation in CT for Treatment Response Assessment: Application of Deep-Learning Convolution Neural Network-A Pilot Study. ACTA ACUST UNITED AC 2016; 2:421-429. [PMID: 28105470 PMCID: PMC5241049 DOI: 10.18383/j.tom.2016.00184] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Assessing the response of bladder cancer to neoadjuvant chemotherapy is crucial for reducing morbidity and increasing quality of life of patients. Changes in tumor volume during treatment is generally used to predict treatment outcome. We are developing a method for bladder cancer segmentation in CT using a pilot data set of 62 cases. 65 000 regions of interests were extracted from pre-treatment CT images to train a deep-learning convolution neural network (DL-CNN) for tumor boundary detection using leave-one-case-out cross-validation. The results were compared to our previous AI-CALS method. For all lesions in the data set, the longest diameter and its perpendicular were measured by two radiologists, and 3D manual segmentation was obtained from one radiologist. The World Health Organization (WHO) criteria and the Response Evaluation Criteria In Solid Tumors (RECIST) were calculated, and the prediction accuracy of complete response to chemotherapy was estimated by the area under the receiver operating characteristic curve (AUC). The AUCs were 0.73 ± 0.06, 0.70 ± 0.07, and 0.70 ± 0.06, respectively, for the volume change calculated using DL-CNN segmentation, the AI-CALS and the manual contours. The differences did not achieve statistical significance. The AUCs using the WHO criteria were 0.63 ± 0.07 and 0.61 ± 0.06, while the AUCs using RECIST were 0.65 ± 007 and 0.63 ± 0.06 for the two radiologists, respectively. Our results indicate that DL-CNN can produce accurate bladder cancer segmentation for calculation of tumor size change in response to treatment. The volume change performed better than the estimations from the WHO criteria and RECIST for the prediction of complete response.
Collapse
Affiliation(s)
- Kenny H Cha
- Department of Radiology, University of Michigan, Ann Arbor, Michigan
| | | | - Ravi K Samala
- Department of Radiology, University of Michigan, Ann Arbor, Michigan
| | - Heang-Ping Chan
- Department of Radiology, University of Michigan, Ann Arbor, Michigan
| | - Richard H Cohan
- Department of Radiology, University of Michigan, Ann Arbor, Michigan
| | - Elaine M Caoili
- Department of Radiology, University of Michigan, Ann Arbor, Michigan
| | | | - Ajjai Alva
- Department of Internal Medicine, Hematology-Oncology, University of Michigan, Ann Arbor, Michigan
| | - Alon Z Weizer
- Department of Urology, Comprehensive Cancer Center, University of Michigan, Ann Arbor, Michigan
| |
Collapse
|
15
|
Samala RK, Chan HP, Hadjiiski LM, Helvie MA. Analysis of computer-aided detection techniques and signal characteristics for clustered microcalcifications on digital mammography and digital breast tomosynthesis. Phys Med Biol 2016; 61:7092-7112. [PMID: 27648708 DOI: 10.1088/0031-9155/61/19/7092] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
With IRB approval, digital breast tomosynthesis (DBT) images of human subjects were collected using a GE GEN2 DBT prototype system. Corresponding digital mammograms (DMs) of the same subjects were collected retrospectively from patient files. The data set contained a total of 237 views of DBT and equal number of DM views from 120 human subjects, each included 163 views with microcalcification clusters (MCs) and 74 views without MCs. The data set was separated into training and independent test sets. The pre-processing, object prescreening and segmentation, false positive reduction and clustering strategies for MC detection by three computer-aided detection (CADe) systems designed for DM, DBT, and a planar projection image generated from DBT were analyzed. Receiver operating characteristic (ROC) curves based on features extracted from microcalcifications and free-response ROC (FROC) curves based on scores from MCs were used to quantify the performance of the systems. Jackknife FROC (JAFROC) and non-parametric analysis methods were used to determine the statistical difference between the FROC curves. The difference between the CADDM and CADDBT systems when the false positive rate was estimated from cases without MCs did not reach statistical significance. The study indicates that the large search space in DBT may not be a limiting factor for CADe to achieve similar performance as that observed in DM.
Collapse
Affiliation(s)
- Ravi K Samala
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109-5842, USA
| | | | | | | |
Collapse
|
16
|
Three-Dimensional Computer-Aided Detection of Microcalcification Clusters in Digital Breast Tomosynthesis. BIOMED RESEARCH INTERNATIONAL 2016; 2016:8651573. [PMID: 27274993 PMCID: PMC4870350 DOI: 10.1155/2016/8651573] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/10/2015] [Revised: 02/12/2016] [Accepted: 02/17/2016] [Indexed: 11/17/2022]
Abstract
We propose computer-aided detection (CADe) algorithm for microcalcification (MC) clusters in reconstructed digital breast tomosynthesis (DBT) images. The algorithm consists of prescreening, MC detection, clustering, and false-positive (FP) reduction steps. The DBT images containing the MC-like objects were enhanced by a multiscale Hessian-based three-dimensional (3D) objectness response function and a connected-component segmentation method was applied to extract the cluster seed objects as potential clustering centers of MCs. Secondly, a signal-to-noise ratio (SNR) enhanced image was also generated to detect the individual MC candidates and prescreen the MC-like objects. Each cluster seed candidate was prescreened by counting neighboring individual MC candidates nearby the cluster seed object according to several microcalcification clustering criteria. As a second step, we introduced bounding boxes for the accepted seed candidate, clustered all the overlapping cubes, and examined. After the FP reduction step, the average number of FPs per case was estimated to be 2.47 per DBT volume with a sensitivity of 83.3%.
Collapse
|
17
|
Samala RK, Chan HP, Lu Y, Hadjiiski LM, Wei J, Helvie MA. Computer-aided detection system for clustered microcalcifications in digital breast tomosynthesis using joint information from volumetric and planar projection images. Phys Med Biol 2015; 60:8457-79. [PMID: 26464355 DOI: 10.1088/0031-9155/60/21/8457] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
We propose a novel approach for the detection of microcalcification clusters (MCs) using joint information from digital breast tomosynthesis (DBT) volume and planar projection (PPJ) image. A data set of 307 DBT views was collected with IRB approval using a prototype DBT system. The system acquires 21 projection views (PVs) from a wide tomographic angle of 60° (60°-21PV) at about twice the dose of a digital mammography (DM) system, which allows us the flexibility of simulating other DBT acquisition geometries using a subset of the PVs. In this study, we simulated a 30° DBT geometry using the central 11 PVs (30°-11PV). The narrower tomographic angle is closer to DBT geometries commercially available or under development and the dose is matched approximately to that of a DM. We developed a new joint-CAD system for detection of clustered microcalcifications. The DBT volume was reconstructed with a multiscale bilateral filtering regularized method and a PPJ image was generated from the reconstructed volume. Task-specific detection strategies were designed to combine information from the DBT volume and the PPJ image. The data set was divided into a training set (127 views with MCs) and an independent test set (104 views with MCs and 76 views without MCs). The joint-CAD system outperformed the individual CAD systems for DBT volume or PPJ image alone; the differences in the test performances were statistically significant (p < 0.05) using JAFROC analysis.
Collapse
Affiliation(s)
- Ravi K Samala
- Department of Radiology, University of Michigan, Ann Arbor, MI 48109842, USA
| | | | | | | | | | | |
Collapse
|