1
|
Zhou X, Wang X, Ma H, Zhang J, Wang X, Bai X, Zhang L, Long J, Chen J, Le H, He W, Zhao S, Xia J, Yang G. Customized T-time inner sampling network with uncertainty-aware data augmentation strategy for multi-annotated lesion segmentation. Comput Biol Med 2024; 180:108990. [PMID: 39126788 DOI: 10.1016/j.compbiomed.2024.108990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2024] [Revised: 07/09/2024] [Accepted: 08/02/2024] [Indexed: 08/12/2024]
Abstract
Segmentation in medical images is inherently ambiguous. It is crucial to capture the uncertainty in lesion segmentations to assist cancer diagnosis and further interventions. Recent works have made great progress in generating multiple plausible segmentation results as diversified references to account for the uncertainty in lesion segmentations. However, the efficiency of existing models is limited, and the uncertainty information lying in multi-annotated datasets remains to be fully utilized. In this study, we propose a series of methods to corporately deal with the above limitation and leverage the abundant information in multi-annotated datasets: (1) Customized T-time Inner Sampling Network to promote the modeling flexibility and efficiently generate samples matching the ground-truth distribution of a number of annotators; (2) Uncertainty Degree defined for quantitatively measuring the uncertainty of each sample and the imbalance of the whole multi-annotated dataset from a brand new perspective; (3) Uncertainty-aware Data Augmentation Strategy to help probabilistic models adaptively fit samples with different ranges of uncertainty. We have evaluated each of them on both the publicly available lung nodule dataset and our in-house Liver Tumor dataset. Results show that our proposed methods achieves the overall best performance on both accuracy and efficiency, demonstrating its great potential in lesion segmentations and more downstream tasks in real clinical scenarios.
Collapse
Affiliation(s)
- Xi Zhou
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Xinxin Wang
- Department of Artificial Intelligence, Sun Yat-sen University, Guangzhou, 510006, China.
| | - Haiqin Ma
- Shantou University Medical College, Shantou, 515041, China.
| | - Jianjian Zhang
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Xiaomei Wang
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Xiuxiu Bai
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Li Zhang
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Jia Long
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Jiakuan Chen
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Hongbo Le
- Department of Radiology, South China Hospital, Medical School, Shenzhen University, Shenzhen, 5181116, China.
| | - Wenjie He
- Department of Radiology, The First Affiliated Hospital of Shenzhen University, Shenzhen University, Shenzhen second people's hospital, Shenzhen, 518035, China.
| | - Shen Zhao
- Department of Artificial Intelligence, Sun Yat-sen University, Guangzhou, 510006, China.
| | - Jun Xia
- Department of Radiology, The First Affiliated Hospital of Shenzhen University, Shenzhen University, Shenzhen second people's hospital, Shenzhen, 518035, China.
| | - Guang Yang
- Bioengineering Department and Imperial-X, Imperial College London, London W12 7SL, UK; National Heart and Lung Institute, Imperial College London, London SW7 2AZ, UK; Cardiovascular Research Centre, Royal Brompton Hospital, London SW3 6NP, UK; School of Biomedical Engineering and Imaging Sciences, King's College London, London WC2R 2LS, UK.
| |
Collapse
|
2
|
Demuth S, Paris J, Faddeenkov I, De Sèze J, Gourraud PA. Clinical applications of deep learning in neuroinflammatory diseases: A scoping review. Rev Neurol (Paris) 2024:S0035-3787(24)00522-8. [PMID: 38772806 DOI: 10.1016/j.neurol.2024.04.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2024] [Revised: 03/26/2024] [Accepted: 04/09/2024] [Indexed: 05/23/2024]
Abstract
BACKGROUND Deep learning (DL) is an artificial intelligence technology that has aroused much excitement for predictive medicine due to its ability to process raw data modalities such as images, text, and time series of signals. OBJECTIVES Here, we intend to give the clinical reader elements to understand this technology, taking neuroinflammatory diseases as an illustrative use case of clinical translation efforts. We reviewed the scope of this rapidly evolving field to get quantitative insights about which clinical applications concentrate the efforts and which data modalities are most commonly used. METHODS We queried the PubMed database for articles reporting DL algorithms for clinical applications in neuroinflammatory diseases and the radiology.healthairegister.com website for commercial algorithms. RESULTS The review included 148 articles published between 2018 and 2024 and five commercial algorithms. The clinical applications could be grouped as computer-aided diagnosis, individual prognosis, functional assessment, the segmentation of radiological structures, and the optimization of data acquisition. Our review highlighted important discrepancies in efforts. The segmentation of radiological structures and computer-aided diagnosis currently concentrate most efforts with an overrepresentation of imaging. Various model architectures have addressed different applications, relatively low volume of data, and diverse data modalities. We report the high-level technical characteristics of the algorithms and synthesize narratively the clinical applications. Predictive performances and some common a priori on this topic are finally discussed. CONCLUSION The currently reported efforts position DL as an information processing technology, enhancing existing modalities of paraclinical investigations and bringing perspectives to make innovative ones actionable for healthcare.
Collapse
Affiliation(s)
- S Demuth
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France.
| | - J Paris
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - I Faddeenkov
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France
| | - J De Sèze
- Inserm U1119 : biopathologie de la myéline, neuroprotection et stratégies thérapeutiques, University of Strasbourg, 1, rue Eugène-Boeckel - CS 60026, 67084 Strasbourg, France; Department of Neurology, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France; Inserm CIC 1434 Clinical Investigation Center, University Hospital of Strasbourg, 1, avenue Molière, 67200 Strasbourg, France
| | - P-A Gourraud
- Inserm U1064, CR2TI - Center for Research in Transplantation and Translational Immunology, Nantes University, 44000 Nantes, France; "Data clinic", Department of Public Health, University Hospital of Nantes, Nantes, France
| |
Collapse
|
3
|
Jan M, Spangaro A, Lenartowicz M, Mattiazzi Usaj M. From pixels to insights: Machine learning and deep learning for bioimage analysis. Bioessays 2024; 46:e2300114. [PMID: 38058114 DOI: 10.1002/bies.202300114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2023] [Revised: 10/25/2023] [Accepted: 11/13/2023] [Indexed: 12/08/2023]
Abstract
Bioimage analysis plays a critical role in extracting information from biological images, enabling deeper insights into cellular structures and processes. The integration of machine learning and deep learning techniques has revolutionized the field, enabling the automated, reproducible, and accurate analysis of biological images. Here, we provide an overview of the history and principles of machine learning and deep learning in the context of bioimage analysis. We discuss the essential steps of the bioimage analysis workflow, emphasizing how machine learning and deep learning have improved preprocessing, segmentation, feature extraction, object tracking, and classification. We provide examples that showcase the application of machine learning and deep learning in bioimage analysis. We examine user-friendly software and tools that enable biologists to leverage these techniques without extensive computational expertise. This review is a resource for researchers seeking to incorporate machine learning and deep learning in their bioimage analysis workflows and enhance their research in this rapidly evolving field.
Collapse
Affiliation(s)
- Mahta Jan
- Department of Chemistry and Biology, Toronto Metropolitan University, Toronto, Canada
| | - Allie Spangaro
- Department of Chemistry and Biology, Toronto Metropolitan University, Toronto, Canada
| | - Michelle Lenartowicz
- Department of Chemistry and Biology, Toronto Metropolitan University, Toronto, Canada
| | - Mojca Mattiazzi Usaj
- Department of Chemistry and Biology, Toronto Metropolitan University, Toronto, Canada
| |
Collapse
|
4
|
Chen Z, Hua S, Gao J, Chen Y, Gong Y, Shen Y, Tang X, Emu Y, Jin W, Hu C. A dual-stage partially interpretable neural network for joint suppression of bSSFP banding and flow artifacts in non-phase-cycled cine imaging. J Cardiovasc Magn Reson 2023; 25:68. [PMID: 37993824 PMCID: PMC10666342 DOI: 10.1186/s12968-023-00988-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2023] [Accepted: 11/12/2023] [Indexed: 11/24/2023] Open
Abstract
PURPOSE To develop a partially interpretable neural network for joint suppression of banding and flow artifacts in non-phase-cycled bSSFP cine imaging. METHODS A dual-stage neural network consisting of a voxel-identification (VI) sub-network and artifact-suppression (AS) sub-network is proposed. The VI sub-network provides identification of artifacts, which guides artifact suppression and improves interpretability. The AS sub-network reduces banding and flow artifacts. Short-axis cine images of 12 frequency offsets from 28 healthy subjects were used to train and test the dual-stage network. An additional 77 patients were retrospectively enrolled to evaluate its clinical generalizability. For healthy subjects, artifact suppression performance was analyzed by comparison with traditional phase cycling. The partial interpretability provided by the VI sub-network was analyzed via correlation analysis. Generalizability was evaluated for cine obtained with different sequence parameters and scanners. For patients, artifact suppression performance and partial interpretability of the network were qualitatively evaluated by 3 clinicians. Cardiac function before and after artifact suppression was assessed via left ventricular ejection fraction (LVEF). RESULTS For the healthy subjects, visual inspection and quantitative analysis found a considerable reduction of banding and flow artifacts by the proposed network. Compared with traditional phase cycling, the proposed network improved flow artifact scores (4.57 ± 0.23 vs 3.40 ± 0.38, P = 0.002) and overall image quality (4.33 ± 0.22 vs 3.60 ± 0.38, P = 0.002). The VI sub-network well identified the location of banding and flow artifacts in the original movie and significantly correlated with the change of signal intensities in these regions. Changes of imaging parameters or the scanner did not cause a significant change of overall image quality relative to the baseline dataset, suggesting a good generalizability. For the patients, qualitative analysis showed a significant improvement of banding artifacts (4.01 ± 0.50 vs 2.77 ± 0.40, P < 0.001), flow artifacts (4.22 ± 0.38 vs 2.97 ± 0.57, P < 0.001), and image quality (3.91 ± 0.45 vs 2.60 ± 0.43, P < 0.001) relative to the original cine. The artifact suppression slightly reduced the LVEF (mean bias = -1.25%, P = 0.01). CONCLUSIONS The dual-stage network simultaneously reduces banding and flow artifacts in bSSFP cine imaging with a partial interpretability, sparing the need for sequence modification. The method can be easily deployed in a clinical setting to identify artifacts and improve cine image quality.
Collapse
Affiliation(s)
- Zhuo Chen
- National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, School of Biomedical Engineering, Shanghai Jiao Tong University, 415 S Med-X Center, 1954 Huashan Road, Shanghai, 200030, China
| | - Sha Hua
- Department of Cardiovascular Medicine, Heart Failure Center, Ruijin Hospital Lu Wan Branch, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Juan Gao
- National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, School of Biomedical Engineering, Shanghai Jiao Tong University, 415 S Med-X Center, 1954 Huashan Road, Shanghai, 200030, China
| | - Yanjia Chen
- Department of Cardiovascular Medicine, Heart Failure Center, Ruijin Hospital Lu Wan Branch, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yiwen Gong
- Department of Cardiovascular Medicine, Heart Failure Center, Ruijin Hospital Lu Wan Branch, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yiwen Shen
- Department of Cardiovascular Medicine, Heart Failure Center, Ruijin Hospital Lu Wan Branch, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Xin Tang
- National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, School of Biomedical Engineering, Shanghai Jiao Tong University, 415 S Med-X Center, 1954 Huashan Road, Shanghai, 200030, China
| | - Yixin Emu
- National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, School of Biomedical Engineering, Shanghai Jiao Tong University, 415 S Med-X Center, 1954 Huashan Road, Shanghai, 200030, China
| | - Wei Jin
- Department of Cardiovascular Medicine, Heart Failure Center, Ruijin Hospital Lu Wan Branch, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Chenxi Hu
- National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy, School of Biomedical Engineering, Shanghai Jiao Tong University, 415 S Med-X Center, 1954 Huashan Road, Shanghai, 200030, China.
| |
Collapse
|
5
|
Nguyen N, Bohak C, Engel D, Mindek P, Strnad O, Wonka P, Li S, Ropinski T, Viola I. Finding Nano-Ötzi: Cryo-Electron Tomography Visualization Guided by Learned Segmentation. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; 29:4198-4214. [PMID: 35749328 DOI: 10.1109/tvcg.2022.3186146] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Cryo-electron tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural details. Existing volume visualization methods, however, are not able to reveal details of interest due to low signal-to-noise ratio. In order to design more powerful transfer functions, we propose leveraging soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning, where we combine the advantages of two segmentation algorithms. First, the weak segmentation algorithm provides good results for propagating sparse user-provided labels to other voxels in the same volume and is used to generate dense pseudo-labels. Second, the powerful deep-learning-based segmentation algorithm learns from these pseudo-labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses deep-learning-based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through frequency distribution analysis. Furthermore, our visualization uses gradient-free ambient occlusion shading to further suppress the visual presence of noise, and to give structural detail the desired prominence. The cryo-ET data studied in our technical experiments are based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.
Collapse
|
6
|
Hemelings R, Elen B, Schuster AK, Blaschko MB, Barbosa-Breda J, Hujanen P, Junglas A, Nickels S, White A, Pfeiffer N, Mitchell P, De Boever P, Tuulonen A, Stalmans I. A generalizable deep learning regression model for automated glaucoma screening from fundus images. NPJ Digit Med 2023; 6:112. [PMID: 37311940 DOI: 10.1038/s41746-023-00857-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2022] [Accepted: 06/01/2023] [Indexed: 06/15/2023] Open
Abstract
A plethora of classification models for the detection of glaucoma from fundus images have been proposed in recent years. Often trained with data from a single glaucoma clinic, they report impressive performance on internal test sets, but tend to struggle in generalizing to external sets. This performance drop can be attributed to data shifts in glaucoma prevalence, fundus camera, and the definition of glaucoma ground truth. In this study, we confirm that a previously described regression network for glaucoma referral (G-RISK) obtains excellent results in a variety of challenging settings. Thirteen different data sources of labeled fundus images were utilized. The data sources include two large population cohorts (Australian Blue Mountains Eye Study, BMES and German Gutenberg Health Study, GHS) and 11 publicly available datasets (AIROGS, ORIGA, REFUGE1, LAG, ODIR, REFUGE2, GAMMA, RIM-ONEr3, RIM-ONE DL, ACRIMA, PAPILA). To minimize data shifts in input data, a standardized image processing strategy was developed to obtain 30° disc-centered images from the original data. A total of 149,455 images were included for model testing. Area under the receiver operating characteristic curve (AUC) for BMES and GHS population cohorts were at 0.976 [95% CI: 0.967-0.986] and 0.984 [95% CI: 0.980-0.991] on participant level, respectively. At a fixed specificity of 95%, sensitivities were at 87.3% and 90.3%, respectively, surpassing the minimum criteria of 85% sensitivity recommended by Prevent Blindness America. AUC values on the eleven publicly available data sets ranged from 0.854 to 0.988. These results confirm the excellent generalizability of a glaucoma risk regression model trained with homogeneous data from a single tertiary referral center. Further validation using prospective cohort studies is warranted.
Collapse
Affiliation(s)
- Ruben Hemelings
- Research Group Ophthalmology, Department of Neurosciences, KU Leuven, Herestraat 49, 3000, Leuven, Belgium.
- Flemish Institute for Technological Research (VITO), Boeretang 200, 2400, Mol, Belgium.
| | - Bart Elen
- Flemish Institute for Technological Research (VITO), Boeretang 200, 2400, Mol, Belgium
| | - Alexander K Schuster
- Department of Ophthalmology, University Medical Center Mainz, Langenbeckstr. 1, 55131, Mainz, Germany
| | | | - João Barbosa-Breda
- Research Group Ophthalmology, Department of Neurosciences, KU Leuven, Herestraat 49, 3000, Leuven, Belgium
- Cardiovascular R&D Center, Faculty of Medicine of the University of Porto, Alameda Prof. Hernâni Monteiro, 4200-319, Porto, Portugal
- Department of Ophthalmology, Centro Hospitalar e Universitário São João, Alameda Prof. Hernâni Monteiro, 4200-319, Porto, Portugal
| | - Pekko Hujanen
- Tays Eye Centre, Tampere University Hospital, Tampere, Finland
| | - Annika Junglas
- Department of Ophthalmology, University Medical Center Mainz, Langenbeckstr. 1, 55131, Mainz, Germany
| | - Stefan Nickels
- Department of Ophthalmology, University Medical Center Mainz, Langenbeckstr. 1, 55131, Mainz, Germany
| | - Andrew White
- Department of Ophthalmology, The University of Sydney, Sydney, NSW, Australia
| | - Norbert Pfeiffer
- Department of Ophthalmology, University Medical Center Mainz, Langenbeckstr. 1, 55131, Mainz, Germany
| | - Paul Mitchell
- Department of Ophthalmology, The University of Sydney, Sydney, NSW, Australia
| | - Patrick De Boever
- Centre for Environmental Sciences, Hasselt University, Agoralaan building D, 3590, Diepenbeek, Belgium
- University of Antwerp, Department of Biology, 2610, Wilrijk, Belgium
| | - Anja Tuulonen
- Tays Eye Centre, Tampere University Hospital, Tampere, Finland
| | - Ingeborg Stalmans
- Research Group Ophthalmology, Department of Neurosciences, KU Leuven, Herestraat 49, 3000, Leuven, Belgium
- Ophthalmology Department, UZ Leuven, Herestraat 49, 3000, Leuven, Belgium
| |
Collapse
|
7
|
Lapierre-Landry M, Liu Y, Bayat M, Wilson DL, Jenkins MW. Digital labeling for 3D histology: segmenting blood vessels without a vascular contrast agent using deep learning. BIOMEDICAL OPTICS EXPRESS 2023; 14:2416-2431. [PMID: 37342724 PMCID: PMC10278624 DOI: 10.1364/boe.480230] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/12/2023] [Accepted: 02/20/2023] [Indexed: 06/23/2023]
Abstract
Recent advances in optical tissue clearing and three-dimensional (3D) fluorescence microscopy have enabled high resolution in situ imaging of intact tissues. Using simply prepared samples, we demonstrate here "digital labeling," a method to segment blood vessels in 3D volumes solely based on the autofluorescence signal and a nuclei stain (DAPI). We trained a deep-learning neural network based on the U-net architecture using a regression loss instead of a commonly used segmentation loss to achieve better detection of small vessels. We achieved high vessel detection accuracy and obtained accurate vascular morphometrics such as vessel length density and orientation. In the future, such digital labeling approach could easily be transferred to other biological structures.
Collapse
Affiliation(s)
| | - Yehe Liu
- Department of Biomedical Engineering, Case Western Reserve University, USA
| | - Mahdi Bayat
- Department of Electrical, Computer and Systems Engineering, Case Western Reserve University, USA
| | - David L. Wilson
- Department of Biomedical Engineering, Case Western Reserve University, USA
- Department of Radiology, Case Western Reserve University, USA
| | - Michael W. Jenkins
- Department of Biomedical Engineering, Case Western Reserve University, USA
- Department of Pediatrics, School of
Medicine, Case Western Reserve University, USA
| |
Collapse
|
8
|
A Soft Label Method for Medical Image Segmentation with Multirater Annotations. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2023; 2023:1883597. [PMID: 36851939 PMCID: PMC9966563 DOI: 10.1155/2023/1883597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 10/04/2022] [Accepted: 10/06/2022] [Indexed: 02/20/2023]
Abstract
In medical image analysis, collecting multiple annotations from different clinical raters is a typical practice to mitigate possible diagnostic errors. For such multirater labels' learning problems, in addition to majority voting, it is a common practice to use soft labels in the form of full-probability distributions obtained by averaging raters as ground truth to train the model, which benefits from uncertainty contained in soft labels. However, the potential information contained in soft labels is rarely studied, which may be the key to improving the performance of medical image segmentation with multirater annotations. In this work, we aim to improve soft label methods by leveraging interpretable information from multiraters. Considering that mis-segmentation occurs in areas with weak supervision of annotations and high difficulty of images, we propose to reduce the reliance on local uncertain soft labels and increase the focus on image features. Therefore, we introduce local self-ensembling learning with consistency regularization, forcing the model to concentrate more on features rather than annotations, especially in regions with high uncertainty measured by the pixelwise interclass variance. Furthermore, we utilize a label smoothing technique to flatten each rater's annotation, alleviating overconfidence of structural edges in annotations. Without introducing additional parameters, our method improves the accuracy of the soft label baseline by 4.2% and 2.7% on a synthetic dataset and a fundus dataset, respectively. In addition, quantitative comparisons show that our method consistently outperforms existing multirater strategies as well as state-of-the-art methods. This work provides a simple yet effective solution for the widespread multirater label segmentation problems in clinical diagnosis.
Collapse
|
9
|
Wang L, Ye X, Ju L, He W, Zhang D, Wang X, Huang Y, Feng W, Song K, Ge Z. Medical matting: Medical image segmentation with uncertainty from the matting perspective. Comput Biol Med 2023; 158:106714. [PMID: 37003068 DOI: 10.1016/j.compbiomed.2023.106714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 02/10/2023] [Accepted: 02/26/2023] [Indexed: 03/05/2023]
Abstract
High-quality manual labeling of ambiguous and complex-shaped targets with binary masks can be challenging. The weakness of insufficient expression of binary masks is prominent in segmentation, especially in medical scenarios where blurring is prevalent. Thus, reaching a consensus among clinicians through binary masks is more difficult in multi-person labeling cases. These inconsistent or uncertain areas are related to the lesions' structure and may contain anatomical information conducive to providing an accurate diagnosis. However, recent research focuses on uncertainties of model training and data labeling. None of them has investigated the influence of the ambiguous nature of the lesion itself. Inspired by image matting, this paper introduces a soft mask called alpha matte to medical scenes. It can describe the lesions with more details better than a binary mask. Moreover, it can also be used as a new uncertainty quantification method to represent uncertain areas, filling the gap in research on the uncertainty of lesion structure. In this work, we introduce a multi-task framework to generate binary masks and alpha mattes, which outperforms all state-of-the-art matting algorithms compared. The uncertainty map is proposed to imitate the trimap in matting methods, which can highlight fuzzy areas and improve matting performance. We have created three medical datasets with alpha mattes to address the lack of available matting datasets in medical fields and evaluated the effectiveness of our proposed method on them comprehensively. Furthermore, experiments demonstrate that the alpha matte is a more effective labeling method than the binary mask from both qualitative and quantitative aspects.
Collapse
|
10
|
Garcea F, Serra A, Lamberti F, Morra L. Data augmentation for medical imaging: A systematic literature review. Comput Biol Med 2023; 152:106391. [PMID: 36549032 DOI: 10.1016/j.compbiomed.2022.106391] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Revised: 11/22/2022] [Accepted: 11/29/2022] [Indexed: 12/13/2022]
Abstract
Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.
Collapse
Affiliation(s)
- Fabio Garcea
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Alessio Serra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Fabrizio Lamberti
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Lia Morra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy.
| |
Collapse
|
11
|
A Soft Label Deep Learning to Assist Breast Cancer Target Therapy and Thyroid Cancer Diagnosis. Cancers (Basel) 2022; 14:cancers14215312. [PMID: 36358732 PMCID: PMC9657740 DOI: 10.3390/cancers14215312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 10/20/2022] [Accepted: 10/25/2022] [Indexed: 11/17/2022] Open
Abstract
According to the World Health Organization Report 2022, cancer is the most common cause of death contributing to nearly one out of six deaths worldwide. Early cancer diagnosis and prognosis have become essential in reducing the mortality rate. On the other hand, cancer detection is a challenging task in cancer pathology. Trained pathologists can detect cancer, but their decisions are subjective to high intra- and inter-observer variability, which can lead to poor patient care owing to false-positive and false-negative results. In this study, we present a soft label fully convolutional network (SL-FCN) to assist in breast cancer target therapy and thyroid cancer diagnosis, using four datasets. To aid in breast cancer target therapy, the proposed method automatically segments human epidermal growth factor receptor 2 (HER2) amplification in fluorescence in situ hybridization (FISH) and dual in situ hybridization (DISH) images. To help in thyroid cancer diagnosis, the proposed method automatically segments papillary thyroid carcinoma (PTC) on Papanicolaou-stained fine needle aspiration and thin prep whole slide images (WSIs). In the evaluation of segmentation of HER2 amplification in FISH and DISH images, we compare the proposed method with thirteen deep learning approaches, including U-Net, U-Net with InceptionV5, Ensemble of U-Net with Inception-v4, Inception-Resnet-v2 encoder, and ResNet-34 encoder, SegNet, FCN, modified FCN, YOLOv5, CPN, SOLOv2, BCNet, and DeepLabv3+ with three different backbones, including MobileNet, ResNet, and Xception, on three clinical datasets, including two DISH datasets on two different magnification levels and a FISH dataset. The result on DISH breast dataset 1 shows that the proposed method achieves high accuracy of 87.77 ± 14.97%, recall of 91.20 ± 7.72%, and F1-score of 81.67 ± 17.76%, while, on DISH breast dataset 2, the proposed method achieves high accuracy of 94.64 ± 2.23%, recall of 83.78 ± 6.42%, and F1-score of 85.14 ± 6.61% and, on the FISH breast dataset, the proposed method achieves high accuracy of 93.54 ± 5.24%, recall of 83.52 ± 13.15%, and F1-score of 86.98 ± 9.85%, respectively. Furthermore, the proposed method outperforms most of the benchmark approaches by a significant margin (p <0.001). In evaluation of segmentation of PTC on Papanicolaou-stained WSIs, the proposed method is compared with three deep learning methods, including Modified FCN, U-Net, and SegNet. The experimental result demonstrates that the proposed method achieves high accuracy of 99.99 ± 0.01%, precision of 92.02 ± 16.6%, recall of 90.90 ± 14.25%, and F1-score of 89.82 ± 14.92% and significantly outperforms the baseline methods, including U-Net and FCN (p <0.001). With the high degree of accuracy, precision, and recall, the results show that the proposed method could be used in assisting breast cancer target therapy and thyroid cancer diagnosis with faster evaluation and minimizing human judgment errors.
Collapse
|
12
|
Ryselis K, Blažauskas T, Damaševičius R, Maskeliūnas R. Agrast-6: Abridged VGG-Based Reflected Lightweight Architecture for Binary Segmentation of Depth Images Captured by Kinect. SENSORS (BASEL, SWITZERLAND) 2022; 22:6354. [PMID: 36080813 PMCID: PMC9460068 DOI: 10.3390/s22176354] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/03/2022] [Revised: 08/15/2022] [Accepted: 08/19/2022] [Indexed: 06/15/2023]
Abstract
Binary object segmentation is a sub-area of semantic segmentation that could be used for a variety of applications. Semantic segmentation models could be applied to solve binary segmentation problems by introducing only two classes, but the models to solve this problem are more complex than actually required. This leads to very long training times, since there are usually tens of millions of parameters to learn in this category of convolutional neural networks (CNNs). This article introduces a novel abridged VGG-16 and SegNet-inspired reflected architecture adapted for binary segmentation tasks. The architecture has 27 times fewer parameters than SegNet but yields 86% segmentation cross-intersection accuracy and 93% binary accuracy. The proposed architecture is evaluated on a large dataset of depth images collected using the Kinect device, achieving an accuracy of 99.25% in human body shape segmentation and 87% in gender recognition tasks.
Collapse
|
13
|
Wahid KA, Olson B, Jain R, Grossberg AJ, El-Habashy D, Dede C, Salama V, Abobakr M, Mohamed ASR, He R, Jaskari J, Sahlsten J, Kaski K, Fuller CD, Naser MA. Muscle and adipose tissue segmentations at the third cervical vertebral level in patients with head and neck cancer. Sci Data 2022; 9:470. [PMID: 35918336 PMCID: PMC9346108 DOI: 10.1038/s41597-022-01587-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 07/22/2022] [Indexed: 11/09/2022] Open
Abstract
The accurate determination of sarcopenia is critical for disease management in patients with head and neck cancer (HNC). Quantitative determination of sarcopenia is currently dependent on manually-generated segmentations of skeletal muscle derived from computed tomography (CT) cross-sectional imaging. This has prompted the increasing utilization of machine learning models for automated sarcopenia determination. However, extant datasets currently do not provide the necessary manually-generated skeletal muscle segmentations at the C3 vertebral level needed for building these models. In this data descriptor, a set of 394 HNC patients were selected from The Cancer Imaging Archive, and their skeletal muscle and adipose tissue was manually segmented at the C3 vertebral level using sliceOmatic. Subsequently, using publicly disseminated Python scripts, we generated corresponding segmentations files in Neuroimaging Informatics Technology Initiative format. In addition to segmentation data, additional clinical demographic data germane to body composition analysis have been retrospectively collected for these patients. These data are a valuable resource for studying sarcopenia and body composition analysis in patients with HNC.
Collapse
Affiliation(s)
- Kareem A Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Brennan Olson
- Department of Radiation Medicine, Oregon Health & Science University, Portland, Oregon, USA.,Medical Scientist Training Program, Oregon Health & Science University, Portland, Oregon, USA
| | - Rishab Jain
- Department of Radiation Medicine, Oregon Health & Science University, Portland, Oregon, USA
| | - Aaron J Grossberg
- Department of Radiation Medicine, Oregon Health & Science University, Portland, Oregon, USA
| | - Dina El-Habashy
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA.,Department of Clinical Oncology, Menoufia University, Shibin Al Kawm, Egypt
| | - Cem Dede
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Vivian Salama
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Moamen Abobakr
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Abdallah S R Mohamed
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Renjie He
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Joel Jaskari
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Jaakko Sahlsten
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Kimmo Kaski
- Department of Computer Science, Aalto University School of Science, Espoo, Finland
| | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA.
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA.
| |
Collapse
|
14
|
Rajaraman S, Zamzmi G, Yang F, Xue Z, Jaeger S, Antani SK. Uncertainty Quantification in Segmenting Tuberculosis-Consistent Findings in Frontal Chest X-rays. Biomedicines 2022; 10:biomedicines10061323. [PMID: 35740345 PMCID: PMC9220007 DOI: 10.3390/biomedicines10061323] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 05/30/2022] [Accepted: 06/03/2022] [Indexed: 12/10/2022] Open
Abstract
Deep learning (DL) methods have demonstrated superior performance in medical image segmentation tasks. However, selecting a loss function that conforms to the data characteristics is critical for optimal performance. Further, the direct use of traditional DL models does not provide a measure of uncertainty in predictions. Even high-quality automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust. In this study, we aim to investigate the benefits of (i) selecting an appropriate loss function and (ii) quantifying uncertainty in predictions using a VGG16-based-U-Net model with the Monto–Carlo (MCD) Dropout method for segmenting Tuberculosis (TB)-consistent findings in frontal chest X-rays (CXRs). We determine an optimal uncertainty threshold based on several uncertainty-related metrics. This threshold is used to select and refer highly uncertain cases to an expert. Experimental results demonstrate that (i) the model trained with a modified Focal Tversky loss function delivered superior segmentation performance (mean average precision (mAP): 0.5710, 95% confidence interval (CI): (0.4021,0.7399)), (ii) the model with 30 MC forward passes during inference further improved and stabilized performance (mAP: 0.5721, 95% CI: (0.4032,0.7410), and (iii) an uncertainty threshold of 0.7 is observed to be optimal to refer highly uncertain cases.
Collapse
|
15
|
Daeschler SC, Bourget MH, Derakhshan D, Sharma V, Asenov SI, Gordon T, Cohen-Adad J, Borschel GH. Rapid, automated nerve histomorphometry through open-source artificial intelligence. Sci Rep 2022; 12:5975. [PMID: 35396530 PMCID: PMC8993871 DOI: 10.1038/s41598-022-10066-6] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2021] [Accepted: 03/21/2022] [Indexed: 12/23/2022] Open
Abstract
We aimed to develop and validate a deep learning model for automated segmentation and histomorphometry of myelinated peripheral nerve fibers from light microscopic images. A convolutional neural network integrated in the AxonDeepSeg framework was trained for automated axon/myelin segmentation using a dataset of light-microscopic cross-sectional images of osmium tetroxide-stained rat nerves including various axonal regeneration stages. In a second dataset, accuracy of automated segmentation was determined against manual axon/myelin labels. Automated morphometry results, including axon diameter, myelin sheath thickness and g-ratio were compared against manual straight-line measurements and morphometrics extracted from manual labels with AxonDeepSeg as a reference standard. The neural network achieved high pixel-wise accuracy for nerve fiber segmentations with a mean (± standard deviation) ground truth overlap of 0.93 (± 0.03) for axons and 0.99 (± 0.01) for myelin sheaths, respectively. Nerve fibers were identified with a sensitivity of 0.99 and a precision of 0.97. For each nerve fiber, the myelin thickness, axon diameter, g-ratio, solidity, eccentricity, orientation, and individual x -and y-coordinates were determined automatically. Compared to manual morphometry, automated histomorphometry showed superior agreement with the reference standard while reducing the analysis time to below 2.5% of the time needed for manual morphometry. This open-source convolutional neural network provides rapid and accurate morphometry of entire peripheral nerve cross-sections. Given its easy applicability, it could contribute to significant time savings in biomedical research while extracting unprecedented amounts of objective morphologic information from large image datasets.
Collapse
Affiliation(s)
- Simeon Christian Daeschler
- SickKids Research Institute, Neuroscience and Mental Health Program, Hospital for Sick Children (SickKids), Toronto, ON, Canada.
| | - Marie-Hélène Bourget
- NeuroPoly Laboratory, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada
| | | | - Vasudev Sharma
- NeuroPoly Laboratory, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada.,University of Toronto, Toronto, ON, Canada
| | - Stoyan Ivaylov Asenov
- NeuroPoly Laboratory, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada
| | - Tessa Gordon
- SickKids Research Institute, Neuroscience and Mental Health Program, Hospital for Sick Children (SickKids), Toronto, ON, Canada.,Division of Plastic and Reconstructive Surgery, the Hospital for Sick Children, Toronto, ON, Canada
| | - Julien Cohen-Adad
- NeuroPoly Laboratory, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada.,Functional Neuroimaging Unit, CRIUGM, University of Montreal, Montreal, QC, Canada.,Mila - Quebec AI Institute, Montreal, QC, Canada
| | - Gregory Howard Borschel
- SickKids Research Institute, Neuroscience and Mental Health Program, Hospital for Sick Children (SickKids), Toronto, ON, Canada.,University of Toronto, Toronto, ON, Canada.,Division of Plastic and Reconstructive Surgery, the Hospital for Sick Children, Toronto, ON, Canada.,Indiana University School of Medicine, Indianapolis, IN, USA
| |
Collapse
|
16
|
Automated pancreas segmentation and volumetry using deep neural network on computed tomography. Sci Rep 2022; 12:4075. [PMID: 35260710 PMCID: PMC8904764 DOI: 10.1038/s41598-022-07848-3] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Accepted: 02/23/2022] [Indexed: 12/14/2022] Open
Abstract
Pancreas segmentation is necessary for observing lesions, analyzing anatomical structures, and predicting patient prognosis. Therefore, various studies have designed segmentation models based on convolutional neural networks for pancreas segmentation. However, the deep learning approach is limited by a lack of data, and studies conducted on a large computed tomography dataset are scarce. Therefore, this study aims to perform deep-learning-based semantic segmentation on 1006 participants and evaluate the automatic segmentation performance of the pancreas via four individual three-dimensional segmentation networks. In this study, we performed internal validation with 1,006 patients and external validation using the cancer imaging archive pancreas dataset. We obtained mean precision, recall, and dice similarity coefficients of 0.869, 0.842, and 0.842, respectively, for internal validation via a relevant approach among the four deep learning networks. Using the external dataset, the deep learning network achieved mean precision, recall, and dice similarity coefficients of 0.779, 0.749, and 0.735, respectively. We expect that generalized deep-learning-based systems can assist clinical decisions by providing accurate pancreatic segmentation and quantitative information of the pancreas for abdominal computed tomography.
Collapse
|
17
|
Marin T, Zhuo Y, Lahoud RM, Tian F, Ma X, Xing F, Moteabbed M, Liu X, Grogg K, Shusharina N, Woo J, Ma C, Chen YLE, El Fakhri G. Deep learning-based GTV contouring modeling inter- and intra- observer variability in sarcomas. Radiother Oncol 2022; 167:269-276. [PMID: 34808228 PMCID: PMC8934266 DOI: 10.1016/j.radonc.2021.09.034] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Revised: 09/21/2021] [Accepted: 09/29/2021] [Indexed: 02/03/2023]
Abstract
BACKGROUND AND PURPOSE The delineation of the gross tumor volume (GTV) is a critical step for radiation therapy treatment planning. The delineation procedure is typically performed manually which exposes two major issues: cost and reproducibility. Delineation is a time-consuming process that is subject to inter- and intra-observer variability. While methods have been proposed to predict GTV contours, typical approaches ignore variability and therefore fail to utilize the valuable confidence information offered by multiple contours. MATERIALS AND METHODS In this work we propose an automatic GTV contouring method for soft-tissue sarcomas from X-ray computed tomography (CT) images, using deep learning by integrating inter- and intra-observer variability in the learned model. Sixty-eight patients with soft tissue and bone sarcomas were considered in this evaluation, all underwent pre-operative CT imaging used to perform GTV delineation. Four radiation oncologists and radiologists performed three contouring trials each for all patients. We quantify variability by defining confidence levels based on the frequency of inclusion of a given voxel into the GTV and use a deep convolutional neural network to learn GTV confidence maps. RESULTS Results were compared to confidence maps from the four readers as well as ground-truth consensus contours established jointly by all readers. The resulting continuous Dice score between predicted and true confidence maps was 87% and the Hausdorff distance was 14 mm. CONCLUSION Results demonstrate the ability of the proposed method to predict accurate contours while utilizing variability and as such it can be used to improve clinical workflow.
Collapse
Affiliation(s)
- Thibault Marin
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Yue Zhuo
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Rita Maria Lahoud
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Fei Tian
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Xiaoyue Ma
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Fangxu Xing
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Maryam Moteabbed
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Department of Radiation Oncology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Xiaofeng Liu
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Kira Grogg
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Nadya Shusharina
- Department of Radiation Oncology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Jonghye Woo
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Chao Ma
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Yen-Lin E. Chen
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Department of Radiation Oncology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America
| | - Georges El Fakhri
- Gordon Center for Medical Imaging, Department of Radiology, Massachusetts General Hospital, Boston MA, 02114, United States of America,Harvard Medical School, Boston MA, 02115, United States of America,Corresponding author,
| |
Collapse
|
18
|
Deep Learning-Based Segmentation of Various Brain Lesions for Radiosurgery. APPLIED SCIENCES-BASEL 2021. [DOI: 10.3390/app11199180] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Abstract
Semantic segmentation of medical images with deep learning models is rapidly being developed. In this study, we benchmarked state-of-the-art deep learning segmentation algorithms on our clinical stereotactic radiosurgery dataset. The dataset consists of 1688 patients with various brain lesions (pituitary tumors, meningioma, schwannoma, brain metastases, arteriovenous malformation, and trigeminal neuralgia), and we divided the dataset into a training set (1557 patients) and test set (131 patients). This study demonstrates the strengths and weaknesses of deep-learning algorithms in a fairly practical scenario. We compared the model performances concerning their sampling method, model architecture, and the choice of loss functions, identifying suitable settings for their applications and shedding light on the possible improvements. Evidence from this study led us to conclude that deep learning could be promising in assisting the segmentation of brain lesions even if the training dataset was of high heterogeneity in lesion types and sizes.
Collapse
|
19
|
Lemay A, Gros C, Zhuo Z, Zhang J, Duan Y, Cohen-Adad J, Liu Y. Automatic multiclass intramedullary spinal cord tumor segmentation on MRI with deep learning. NEUROIMAGE-CLINICAL 2021; 31:102766. [PMID: 34352654 PMCID: PMC8350366 DOI: 10.1016/j.nicl.2021.102766] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/30/2021] [Revised: 06/22/2021] [Accepted: 07/17/2021] [Indexed: 11/25/2022]
Abstract
Automatic spinal cord tumor segmentation with deep learning. Multi-class model for tumor, edema, and cavity. Model trained to recognize astrocytoma, ependymoma, and hemangioblastoma. Multi-contrast input for more robustness: Gd-T1w and T2w. Method and model are available in open-source Spinal Cord Toolbox (SCT).
Spinal cord tumors lead to neurological morbidity and mortality. Being able to obtain morphometric quantification (size, location, growth rate) of the tumor, edema, and cavity can result in improved monitoring and treatment planning. Such quantification requires the segmentation of these structures into three separate classes. However, manual segmentation of three-dimensional structures is time consuming, tedious and prone to intra- and inter-rater variability, motivating the development of automated methods. Here, we tailor a model adapted to the spinal cord tumor segmentation task. Data were obtained from 343 patients using gadolinium-enhanced T1-weighted and T2-weighted MRI scans with cervical, thoracic, and/or lumbar coverage. The dataset includes the three most common intramedullary spinal cord tumor types: astrocytomas, ependymomas, and hemangioblastomas. The proposed approach is a cascaded architecture with U-Net-based models that segments tumors in a two-stage process: locate and label. The model first finds the spinal cord and generates bounding box coordinates. The images are cropped according to this output, leading to a reduced field of view, which mitigates class imbalance. The tumor is then segmented. The segmentation of the tumor, cavity, and edema (as a single class) reached 76.7 ± 1.5% of Dice score and the segmentation of tumors alone reached 61.8 ± 4.0% Dice score. The true positive detection rate was above 87% for tumor, edema, and cavity. To the best of our knowledge, this is the first fully automatic deep learning model for spinal cord tumor segmentation. The multiclass segmentation pipeline is available in the Spinal Cord Toolbox (https://spinalcordtoolbox.com/). It can be run with custom data on a regular computer within seconds.
Collapse
Affiliation(s)
- Andreanne Lemay
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada; Mila, Quebec AI Institute, Canada
| | - Charley Gros
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada; Mila, Quebec AI Institute, Canada
| | - Zhizheng Zhuo
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Jie Zhang
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Yunyun Duan
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Julien Cohen-Adad
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada; Mila, Quebec AI Institute, Canada; Functional Neuroimaging Unit, CRIUGM, Université de Montréal, Montreal, QC, Canada.
| | - Yaou Liu
- Department of Radiology, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
| |
Collapse
|
20
|
Saha A, Hosseinzadeh M, Huisman H. End-to-end prostate cancer detection in bpMRI via 3D CNNs: Effects of attention mechanisms, clinical priori and decoupled false positive reduction. Med Image Anal 2021; 73:102155. [PMID: 34245943 DOI: 10.1016/j.media.2021.102155] [Citation(s) in RCA: 51] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2021] [Revised: 05/30/2021] [Accepted: 06/08/2021] [Indexed: 01/22/2023]
Abstract
We present a multi-stage 3D computer-aided detection and diagnosis (CAD) model2 for automated localization of clinically significant prostate cancer (csPCa) in bi-parametric MR imaging (bpMRI). Deep attention mechanisms drive its detection network, targeting salient structures and highly discriminative feature dimensions across multiple resolutions. Its goal is to accurately identify csPCa lesions from indolent cancer and the wide range of benign pathology that can afflict the prostate gland. Simultaneously, a decoupled residual classifier is used to achieve consistent false positive reduction, without sacrificing high sensitivity or computational efficiency. In order to guide model generalization with domain-specific clinical knowledge, a probabilistic anatomical prior is used to encode the spatial prevalence and zonal distinction of csPCa. Using a large dataset of 1950 prostate bpMRI paired with radiologically-estimated annotations, we hypothesize that such CNN-based models can be trained to detect biopsy-confirmed malignancies in an independent cohort. For 486 institutional testing scans, the 3D CAD system achieves 83.69±5.22% and 93.19±2.96% detection sensitivity at 0.50 and 1.46 false positive(s) per patient, respectively, with 0.882±0.030 AUROC in patient-based diagnosis -significantly outperforming four state-of-the-art baseline architectures (U-SEResNet, UNet++, nnU-Net, Attention U-Net) from recent literature. For 296 external biopsy-confirmed testing scans, the ensembled CAD system shares moderate agreement with a consensus of expert radiologists (76.69%; kappa = 0.51±0.04) and independent pathologists (81.08%; kappa = 0.56±0.06); demonstrating strong generalization to histologically-confirmed csPCa diagnosis.
Collapse
Affiliation(s)
- Anindo Saha
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen 6525 GA, the Netherlands.
| | - Matin Hosseinzadeh
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen 6525 GA, the Netherlands
| | - Henkjan Huisman
- Diagnostic Image Analysis Group, Radboud University Medical Center, Nijmegen 6525 GA, the Netherlands
| |
Collapse
|
21
|
Bautin P, Cohen-Adad J. Minimum detectable spinal cord atrophy with automatic segmentation: Investigations using an open-access dataset of healthy participants. NEUROIMAGE: CLINICAL 2021; 32:102849. [PMID: 34624638 PMCID: PMC8503570 DOI: 10.1016/j.nicl.2021.102849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 09/07/2021] [Accepted: 09/28/2021] [Indexed: 11/20/2022] Open
Abstract
Evaluate the robustness of an automated analysis pipeline for detecting SC atrophy. Simulate spinal cord atrophy and scan-rescan variability. Fully automated analysis method available on an open access database. Evaluation of sample size and inter/intra-subject variability for T1w and T2w images.
Spinal cord atrophy is a well-known biomarker in multiple sclerosis (MS) and other diseases. It is measured by segmenting the spinal cord on an MRI image and computing the average cross-sectional area (CSA) over a few slices. Introduced about 25 years ago, this procedure is highly sensitive to the quality of the segmentation and is prone to rater-bias. Recently, fully-automated spinal cord segmentation methods, which remove the rater-bias and enable the automated analysis of large populations, have been introduced. A lingering question related to these automated methods is: How reliable are they at detecting atrophy? In this study, we evaluated the precision and accuracy of automated atrophy measurements by simulating scan-rescan experiments. Spinal cord MRI data from the open-access spine-generic project were used. The dataset aggregates 42 sites worldwide and consists of 260 healthy subjects and includes T1w and T2w contrasts. To simulate atrophy, each volume was globally rescaled at various scaling factors. Moreover, to simulate patient repositioning, random rigid transformations were applied. Using the DeepSeg algorithm from the Spinal Cord Toolbox, the spinal cord was segmented and vertebral levels were identified. Then, the average CSA between C3-C5 vertebral levels was computed for each Monte Carlo sample, allowing us to derive measures of atrophy, intra/inter-subject variability, and sample-size calculations. The minimum sample size required to detect an atrophy of 2% between unpaired study arms, commonly seen in MS studies, was 467 +/− 13.9 using T1w and 467 +/− 3.2 using T2w images. The minimum sample size to detect a longitudinal atrophy (between paired study arms) of 0.8% was 60 +/− 25.1 using T1w and 10 +/− 1.2 using T2w images. At the intra-subject level, the estimated CSA, observed in this study, showed good precision compared to other studies with COVs (across Monte Carlo transformations) of 0.8% for T1w and 0.6% for T2w images. While these sample sizes seem small, we would like to stress that these results correspond to a “best case” scenario, in that the dataset used here was of particularly good quality and the model for simulating atrophy does not encompass all the variability met in real-life datasets. The simulated atrophy and scan-rescan variability may over-simplify the biological reality. The proposed framework is open-source and available at https://csa-atrophy.readthedocs.io/.
Collapse
Affiliation(s)
- Paul Bautin
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada
| | - Julien Cohen-Adad
- NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada; Functional Neuroimaging Unit, CRIUGM, Université de Montréal, Montreal, QC, Canada; Mila - Quebec AI Institute, Montreal, QC, Canada.
| |
Collapse
|