1
|
Liu Z, Yuan Y, Zhang C, Zhu Q, Xu X, Yuan M, Tan W. Hierarchical classification of early microscopic lung nodule based on cascade network. Health Inf Sci Syst 2024; 12:13. [PMID: 38404714 PMCID: PMC10891040 DOI: 10.1007/s13755-024-00273-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2023] [Accepted: 01/08/2024] [Indexed: 02/27/2024] Open
Abstract
Purpose Early-stage lung cancer is typically characterized clinically by the presence of isolated lung nodules. Thousands of cases are examined each year, and one case usually contains numerous lung CT slices. Detecting and classifying early microscopic lung nodules is demanding due to their diminutive dimensions and restricted characterization capabilities. Therefore, a lung nodule classification model that performs well and is sensitive to microscopic lung nodules is needed to accurately classify lung nodules. Methods This paper uses the Resnet34 network as a basic classification model. A new cascade lung nodule classification method is proposed to classify lung nodules into 6 classes instead of the traditional 2 or 4 classes. It can effectively classify six different nodule types including ground-glass and solid nodules, benign and malignant nodules, and nodules with predominantly ground-glass or solid components. Results In this paper, the traditional multi-classification method and the cascade classification method proposed in this paper were tested using real lung nodule data collected in the clinic. The test results demonstrate that the cascade classification method in this study achieves an accuracy of 80.04% , outperforming the conventional multi-classification approach. Conclusions Different from the existing methods for categorizing the benign and malignant nature of lung nodules, the approach presented in this paper can classify lung nodules into 6 categories more accurately. At the same time, This paper proposes a rapid, precise, and dependable approach for classifying six distinct categories of lung nodules, which increases the accuracy categorization compared with the traditional multivariate categorization method.
Collapse
Affiliation(s)
- Ziang Liu
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, 110189 China
- College of Computer Science and Engineering, Northeastern University, Shenyang, 110189 China
| | - Ye Yuan
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, 110189 China
- College of Computer Science and Engineering, Northeastern University, Shenyang, 110189 China
| | - Cui Zhang
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, 110189 China
- College of Computer Science and Engineering, Northeastern University, Shenyang, 110189 China
| | - Quan Zhu
- Department of Thoracic Surgery, The First Affiliated Hospital of Nanjing Medical University, Nanjing, 210029 China
| | - Xinfeng Xu
- Department of Thoracic Surgery, The First Affiliated Hospital of Nanjing Medical University, Nanjing, 210029 China
| | - Mei Yuan
- Department of Thoracic Surgery, The First Affiliated Hospital of Nanjing Medical University, Nanjing, 210029 China
| | - Wenjun Tan
- Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, 110189 China
- College of Computer Science and Engineering, Northeastern University, Shenyang, 110189 China
| |
Collapse
|
2
|
Liu Y, Chen L, Fan M, Zhang T, Chen J, Li X, Lv Y, Zheng P, Chen F, Sun G. Application of AI-assisted MRI for the identification of surgical target areas in pediatric hip and periarticular infections. BMC Musculoskelet Disord 2024; 25:428. [PMID: 38824518 PMCID: PMC11143611 DOI: 10.1186/s12891-024-07548-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/21/2023] [Accepted: 05/27/2024] [Indexed: 06/03/2024] Open
Abstract
OBJECTIVE To develop an AI-assisted MRI model to identify surgical target areas in pediatric hip and periarticular infections. METHODS A retrospective study was conducted on the pediatric patients with hip and periarticular infections who underwent Magnetic Resonance Imaging(MRI)examinations from January 2010 to January 2023 in three hospitals in China. A total of 7970 axial Short Tau Inversion Recovery (STIR) images were selected, and the corresponding regions of osteomyelitis (label 1) and abscess (label 2) were labeled using the Labelme software. The images were randomly divided into training group, validation group, and test group at a ratio of 7:2:1. A Mask R-CNN model was constructed and optimized, and the performance of identifying label 1 and label 2 was evaluated using receiver operating characteristic (ROC) curves. Calculation of the average time it took for the model and specialists to process an image in the test group. Comparison of the accuracy of the model in the interpretation of MRI images with four orthopaedic surgeons, with statistical significance set at P < 0.05. RESULTS A total of 275 patients were enrolled, comprising 197 males and 78 females, with an average age of 7.10 ± 3.59 years, ranging from 0.00 to 14.00 years. The area under curve (AUC), accuracy, sensitivity, specificity, precision, and F1 score for the model to identify label 1 were 0.810, 0.976, 0.995, 0.969, 0.922, and 0.957, respectively. The AUC, accuracy, sensitivity, specificity, precision, and F1 score for the model to identify label 2 were 0.890, 0.957, 0.969, 0.915, 0.976, and 0.972, respectively. The model demonstrated a significant speed advantage, taking only 0.2 s to process an image compared to average 10 s required by the specialists. The model identified osteomyelitis with an accuracy of 0.976 and abscess with an accuracy of 0.957, both statistically better than the four orthopaedic surgeons, P < 0.05. CONCLUSION The Mask R-CNN model is reliable for identifying surgical target areas in pediatric hip and periarticular infections, offering a more convenient and rapid option. It can assist unexperienced physicians in pre-treatment assessments, reducing the risk of missed and misdiagnosis.
Collapse
Affiliation(s)
- Yuwen Liu
- Department of Orthopaedic Surgery, Children's Hospital of Nanjing Medical University, Nanjing, China
| | - Lingyu Chen
- Department of Computer Science and Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Mingjie Fan
- Department of Orthopaedic Surgery, Children's Hospital of Nanjing Medical University, Nanjing, China
| | - Tao Zhang
- Department of Orthopaedic Surgery, Qinghai Women's and Children's Hospital, Xining, China
| | - Jie Chen
- Department of Orthopaedic Surgery, Wuxi Children's Hospital, Wuxi, China
| | - Xiaohui Li
- Department of Radiology, Children's Hospital of Nanjing Medical University, Nanjing, China
| | - Yunhao Lv
- Department of Computer Science and Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China
| | - Pengfei Zheng
- Department of Orthopaedic Surgery, Children's Hospital of Nanjing Medical University, Nanjing, China.
| | - Fang Chen
- Department of Computer Science and Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, China.
| | - Guixin Sun
- Department of Traumatic Surgery, Shanghai East Hospital, Nanjing Medical University, Shanghai, China.
| |
Collapse
|
3
|
Zhao Q, Chang CW, Yang X, Zhao L. Robust explanation supervision for false positive reduction in pulmonary nodule detection. Med Phys 2024; 51:1687-1701. [PMID: 38224306 PMCID: PMC10939846 DOI: 10.1002/mp.16937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 11/08/2023] [Accepted: 12/12/2023] [Indexed: 01/16/2024] Open
Abstract
BACKGROUND Lung cancer is the deadliest and second most common cancer in the United States due to the lack of symptoms for early diagnosis. Pulmonary nodules are small abnormal regions that can be potentially correlated to the occurrence of lung cancer. Early detection of these nodules is critical because it can significantly improve the patient's survival rates. Thoracic thin-sliced computed tomography (CT) scanning has emerged as a widely used method for diagnosing and prognosis lung abnormalities. PURPOSE The standard clinical workflow of detecting pulmonary nodules relies on radiologists to analyze CT images to assess the risk factors of cancerous nodules. However, this approach can be error-prone due to the various nodule formation causes, such as pollutants and infections. Deep learning (DL) algorithms have recently demonstrated remarkable success in medical image classification and segmentation. As an ever more important assistant to radiologists in nodule detection, it is imperative ensure the DL algorithm and radiologist to better understand the decisions from each other. This study aims to develop a framework integrating explainable AI methods to achieve accurate pulmonary nodule detection. METHODS A robust and explainable detection (RXD) framework is proposed, focusing on reducing false positives in pulmonary nodule detection. Its implementation is based on an explanation supervision method, which uses nodule contours of radiologists as supervision signals to force the model to learn nodule morphologies, enabling improved learning ability on small dataset, and enable small dataset learning ability. In addition, two imputation methods are applied to the nodule region annotations to reduce the noise within human annotations and allow the model to have robust attributions that meet human expectations. The 480, 265, and 265 CT image sets from the public Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset are used for training, validation, and testing. RESULTS Using only 10, 30, 50, and 100 training samples sequentially, our method constantly improves the classification performance and explanation quality of baseline in terms of Area Under the Curve (AUC) and Intersection over Union (IoU). In particular, our framework with a learnable imputation kernel improves IoU from baseline by 24.0% to 80.0%. A pre-defined Gaussian imputation kernel achieves an even greater improvement, from 38.4% to 118.8% from baseline. Compared to the baseline trained on 100 samples, our method shows less drop in AUC when trained on fewer samples. A comprehensive comparison of interpretability shows that our method aligns better with expert opinions. CONCLUSIONS A pulmonary nodule detection framework was demonstrated using public thoracic CT image datasets. The framework integrates the robust explanation supervision (RES) technique to ensure the performance of nodule classification and morphology. The method can reduce the workload of radiologists and enable them to focus on the diagnosis and prognosis of the potential cancerous pulmonary nodules at the early stage to improve the outcomes for lung cancer patients.
Collapse
Affiliation(s)
- Qilong Zhao
- Department of Computer Science, Emory University, Atlanta, GA 30308
| | - Chih-Wei Chang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA 30308
| | - Xiaofeng Yang
- Department of Radiation Oncology and Winship Cancer Institute, Emory University, Atlanta, GA 30308
| | - Liang Zhao
- Department of Computer Science, Emory University, Atlanta, GA 30308
| |
Collapse
|
4
|
Tomassini S, Falcionelli N, Bruschi G, Sbrollini A, Marini N, Sernani P, Morettini M, Müller H, Dragoni AF, Burattini L. On-cloud decision-support system for non-small cell lung cancer histology characterization from thorax computed tomography scans. Comput Med Imaging Graph 2023; 110:102310. [PMID: 37979340 DOI: 10.1016/j.compmedimag.2023.102310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 10/25/2023] [Accepted: 11/03/2023] [Indexed: 11/20/2023]
Abstract
Non-Small Cell Lung Cancer (NSCLC) accounts for about 85% of all lung cancers. Developing non-invasive techniques for NSCLC histology characterization may not only help clinicians to make targeted therapeutic treatments but also prevent subjects from undergoing lung biopsy, which is challenging and could lead to clinical implications. The motivation behind the study presented here is to develop an advanced on-cloud decision-support system, named LUCY, for non-small cell LUng Cancer histologY characterization directly from thorax Computed Tomography (CT) scans. This aim was pursued by selecting thorax CT scans of 182 LUng ADenocarcinoma (LUAD) and 186 LUng Squamous Cell carcinoma (LUSC) subjects from four openly accessible data collections (NSCLC-Radiomics, NSCLC-Radiogenomics, NSCLC-Radiomics-Genomics and TCGA-LUAD), in addition to the implementation and comparison of two end-to-end neural networks (the core layer of whom is a convolutional long short-term memory layer), the performance evaluation on test dataset (NSCLC-Radiomics-Genomics) from a subject-level perspective in relation to NSCLC histological subtype location and grade, and the dynamic visual interpretation of the achieved results by producing and analyzing one heatmap video for each scan. LUCY reached test Area Under the receiver operating characteristic Curve (AUC) values above 77% in all NSCLC histological subtype location and grade groups, and a best AUC value of 97% on the entire dataset reserved for testing, proving high generalizability to heterogeneous data and robustness. Thus, LUCY is a clinically-useful decision-support system able to timely, non-invasively and reliably provide visually-understandable predictions on LUAD and LUSC subjects in relation to clinically-relevant information.
Collapse
Affiliation(s)
- Selene Tomassini
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Nicola Falcionelli
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Giulia Bruschi
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Agnese Sbrollini
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Niccolò Marini
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
| | - Paolo Sernani
- Department of Law, University of Macerata (UNIMC), Macerata, Italy
| | - Micaela Morettini
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
| | - Aldo Franco Dragoni
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy
| | - Laura Burattini
- Department of Information Engineering, Università Politecnica delle Marche (UNIVPM), Ancona, Italy.
| |
Collapse
|
5
|
Pang S, Du A, Orgun MA, Wang Y, Sheng QZ, Wang S, Huang X, Yu Z. Beyond CNNs: Exploiting Further Inherent Symmetries in Medical Image Segmentation. IEEE TRANSACTIONS ON CYBERNETICS 2023; 53:6776-6787. [PMID: 36044511 DOI: 10.1109/tcyb.2022.3195447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Automatic tumor or lesion segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because, although the human visual system can detect symmetries in 2-D images effectively, regular CNNs can only exploit translation invariance, overlooking further inherent symmetries existing in medical images, such as rotations and reflections. To solve this problem, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning more precise representations. First, kernel-based equivariant operations are devised on each orientation, which allows it to effectively address the gaps of learning symmetries in existing approaches. Then, to keep segmentation networks globally equivariant, we design distinctive group layers with layer-wise symmetry constraints. Finally, based on our novel framework, extensive experiments conducted on real-world clinical data demonstrate that a group equivariant Res-UNet (called GER-UNet) outperforms its regular CNN-based counterpart and the state-of-the-art segmentation methods in the tasks of hepatic tumor segmentation, COVID-19 lung infection segmentation, and retinal vessel detection. More importantly, the newly built GER-UNet also shows potential in reducing the sample complexity and the redundancy of filters, upgrading current segmentation CNNs, and delineating organs on other medical imaging modalities.
Collapse
|
6
|
Modak S, Abdel-Raheem E, Rueda L. Applications of Deep Learning in Disease Diagnosis of Chest Radiographs: A Survey on Materials and Methods. BIOMEDICAL ENGINEERING ADVANCES 2023. [DOI: 10.1016/j.bea.2023.100076] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023] Open
|
7
|
Liu F, Gao L, Wan J, Lyu ZL, Huang YY, Liu C, Han M. Recognition of Digital Dental X-ray Images Using a Convolutional Neural Network. J Digit Imaging 2023; 36:73-79. [PMID: 36109403 PMCID: PMC9984574 DOI: 10.1007/s10278-022-00694-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Revised: 07/26/2022] [Accepted: 08/16/2022] [Indexed: 11/30/2022] Open
Abstract
Digital dental X-ray images are an important basis for diagnosing dental diseases, especially endodontic and periodontal diseases. Conventional diagnostic methods depend on the experience of doctors, so they are highly subjective and consume more energy than other approaches. The current computer-aided interpretation technology has low accuracy and poor lesion classification. This study proposes an efficient and accurate method for identifying common lesions in digital dental X-ray images by a convolutional neural network (CNN). In total, 188 digital dental X-ray images that were previously diagnosed as periapical periodontitis, dental caries, periapical cysts, and other common dental diseases by dentists in Qilu Hospital of Shandong University were collected and augmented. The images and labels were inputted into four CNN models for training, including visual geometry group (VGG)-16, InceptionV3, residual network (ResNet)-50, and densely connected convolutional networks (DenseNet)-121. The average classification accuracy of the four trained network models on the test set was 95.9%, while the classification accuracy of the trained DenseNet-121 network model reached 99.5%. It is demonstrated that the use of CNNs to interpret digital dental X-ray images is an efficient and accurate way to conduct auxiliary diagnoses of dental diseases.
Collapse
Affiliation(s)
- Feng Liu
- School of Information Science and Engineering, Shandong University, Qingdao, 266237, People's Republic of China
| | - Lei Gao
- Department of First Operating Room, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China
| | - Jun Wan
- School of Information Science and Engineering, Shandong University, Qingdao, 266237, People's Republic of China
| | - Zhi-Lei Lyu
- Department of Oral Radiology, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China
| | - Ying-Ying Huang
- Department of Oral Radiology, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China
| | - Chao Liu
- Department of Oral and Maxillofacial Surgery, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China.
- Department of Oral Surgery, Shanghai Key Laboratory of Stomatology, National Clinical Research Center of Stomatology, Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200011, People's Republic of China.
| | - Min Han
- School of Information Science and Engineering, Shandong University, Qingdao, 266237, People's Republic of China.
| |
Collapse
|
8
|
Garcea F, Serra A, Lamberti F, Morra L. Data augmentation for medical imaging: A systematic literature review. Comput Biol Med 2023; 152:106391. [PMID: 36549032 DOI: 10.1016/j.compbiomed.2022.106391] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2022] [Revised: 11/22/2022] [Accepted: 11/29/2022] [Indexed: 12/13/2022]
Abstract
Recent advances in Deep Learning have largely benefited from larger and more diverse training sets. However, collecting large datasets for medical imaging is still a challenge due to privacy concerns and labeling costs. Data augmentation makes it possible to greatly expand the amount and variety of data available for training without actually collecting new samples. Data augmentation techniques range from simple yet surprisingly effective transformations such as cropping, padding, and flipping, to complex generative models. Depending on the nature of the input and the visual task, different data augmentation strategies are likely to perform differently. For this reason, it is conceivable that medical imaging requires specific augmentation strategies that generate plausible data samples and enable effective regularization of deep neural networks. Data augmentation can also be used to augment specific classes that are underrepresented in the training set, e.g., to generate artificial lesions. The goal of this systematic literature review is to investigate which data augmentation strategies are used in the medical domain and how they affect the performance of clinical tasks such as classification, segmentation, and lesion detection. To this end, a comprehensive analysis of more than 300 articles published in recent years (2018-2022) was conducted. The results highlight the effectiveness of data augmentation across organs, modalities, tasks, and dataset sizes, and suggest potential avenues for future research.
Collapse
Affiliation(s)
- Fabio Garcea
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Alessio Serra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Fabrizio Lamberti
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy
| | - Lia Morra
- Dipartimento di Automatica e Informatica, Politecnico di Torino, C.so Duca degli Abruzzi, 24, Torino, 10129, Italy.
| |
Collapse
|
9
|
Liu J, Cao L, Akin O, Tian Y. Robust and accurate pulmonary nodule detection with self-supervised feature learning on domain adaptation. FRONTIERS IN RADIOLOGY 2022; 2:1041518. [PMID: 37492669 PMCID: PMC10365286 DOI: 10.3389/fradi.2022.1041518] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/11/2022] [Accepted: 11/28/2022] [Indexed: 07/27/2023]
Abstract
Medical imaging data annotation is expensive and time-consuming. Supervised deep learning approaches may encounter overfitting if trained with limited medical data, and further affect the robustness of computer-aided diagnosis (CAD) on CT scans collected by various scanner vendors. Additionally, the high false-positive rate in automatic lung nodule detection methods prevents their applications in daily clinical routine diagnosis. To tackle these issues, we first introduce a novel self-learning schema to train a pre-trained model by learning rich feature representatives from large-scale unlabeled data without extra annotation, which guarantees a consistent detection performance over novel datasets. Then, a 3D feature pyramid network (3DFPN) is proposed for high-sensitivity nodule detection by extracting multi-scale features, where the weights of the backbone network are initialized by the pre-trained model and then fine-tuned in a supervised manner. Further, a High Sensitivity and Specificity (HS2 ) network is proposed to reduce false positives by tracking the appearance changes among continuous CT slices on Location History Images (LHI) for the detected nodule candidates. The proposed method's performance and robustness are evaluated on several publicly available datasets, including LUNA16, SPIE-AAPM, LungTIME, and HMS. Our proposed detector achieves the state-of-the-art result of 90.6 % sensitivity at 1 / 8 false positive per scan on the LUNA16 dataset. The proposed framework's generalizability has been evaluated on three additional datasets (i.e., SPIE-AAPM, LungTIME, and HMS) captured by different types of CT scanners.
Collapse
Affiliation(s)
- Jingya Liu
- The City College of New York, New York, NY, USA
| | | | - Oguz Akin
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Yingli Tian
- The City College of New York, New York, NY, USA
| |
Collapse
|
10
|
Tomassini S, Falcionelli N, Sernani P, Burattini L, Dragoni AF. Lung nodule diagnosis and cancer histology classification from computed tomography data by convolutional neural networks: A survey. Comput Biol Med 2022; 146:105691. [PMID: 35691714 DOI: 10.1016/j.compbiomed.2022.105691] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2022] [Revised: 05/26/2022] [Accepted: 05/31/2022] [Indexed: 11/30/2022]
Abstract
Lung cancer is among the deadliest cancers. Besides lung nodule classification and diagnosis, developing non-invasive systems to classify lung cancer histological types/subtypes may help clinicians to make targeted treatment decisions timely, having a positive impact on patients' comfort and survival rate. As convolutional neural networks have proven to be responsible for the significant improvement of the accuracy in lung cancer diagnosis, with this survey we intend to: show the contribution of convolutional neural networks not only in identifying malignant lung nodules but also in classifying lung cancer histological types/subtypes directly from computed tomography data; point out the strengths and weaknesses of slice-based and scan-based approaches employing convolutional neural networks; and highlight the challenges and prospective solutions to successfully apply convolutional neural networks for such classification tasks. To this aim, we conducted a comprehensive analysis of relevant Scopus-indexed studies involved in lung nodule diagnosis and cancer histology classification up to January 2022, dividing the investigation in convolutional neural network-based approaches fed with planar or volumetric computed tomography data. Despite the application of convolutional neural networks in lung nodule diagnosis and cancer histology classification is a valid strategy, some challenges raised, mainly including the lack of publicly-accessible annotated data, together with the lack of reproducibility and clinical interpretability. We believe that this survey will be helpful for future studies involved in lung nodule diagnosis and cancer histology classification prior to lung biopsy by means of convolutional neural networks.
Collapse
Affiliation(s)
- Selene Tomassini
- Department of Information Engineering, Engineering Faculty, Università Politecnica delle Marche, Ancona, Italy.
| | - Nicola Falcionelli
- Department of Information Engineering, Engineering Faculty, Università Politecnica delle Marche, Ancona, Italy.
| | - Paolo Sernani
- Department of Information Engineering, Engineering Faculty, Università Politecnica delle Marche, Ancona, Italy.
| | - Laura Burattini
- Department of Information Engineering, Engineering Faculty, Università Politecnica delle Marche, Ancona, Italy.
| | - Aldo Franco Dragoni
- Department of Information Engineering, Engineering Faculty, Università Politecnica delle Marche, Ancona, Italy.
| |
Collapse
|
11
|
An improved CNN-based architecture for automatic lung nodule classification. Med Biol Eng Comput 2022; 60:1977-1986. [PMID: 35524089 DOI: 10.1007/s11517-022-02578-0] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2021] [Accepted: 04/22/2022] [Indexed: 10/18/2022]
Abstract
Lung cancer is one of the most critical diseases due to its significant death rate compared to all other types of cancer. The early diagnosis of lung cancer that improves the patient's chance of surviving is mostly done in two phases: screening through CT scan imaging modality and, more importantly the medical expert's reading of the scan, which is a time-consuming task and is vulnerable to errors. It is difficult to differentiate between malignant and benign nodules and biopsies are highly invasive, and patients with benign nodules may undergo unnecessary procedures. In this study, we propose a CNN-based computer-aided diagnosis system to automatically classify pulmonary nodules into benign or malignant. The proposed network architecture is based on AlexNet architecture that experiments with several types of layer ordering, hyperparameters, and functions for the various sides of the network. To build a well-trained model, several pre-processing steps are applied to the entire dataset, for instance segmentation, normalization, and zero centering. Finally, the proposed system obtained results with 98.7% accuracy, 98.6% sensitivity, and 98.9% specificity. The proposed model achieved superior performance compared to the AlexNet. The modifications in the original AlexNet is done to get a reasonable structure that has high nodule analysis sensitivity.
Collapse
|
12
|
Wong KK, Cummock JS, Li G, Ghosh R, Xu P, Volpi JJ, Wong STC. Automatic Segmentation in Acute Ischemic Stroke: Prognostic Significance of Topological Stroke Volumes on Stroke Outcome. Stroke 2022; 53:2896-2905. [PMID: 35545938 DOI: 10.1161/strokeaha.121.037982] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Stroke infarct volume predicts patient disability and has utility for clinical trial outcomes. Accurate infarct volume measurement requires manual segmentation of stroke boundaries in diffusion-weighted magnetic resonance imaging scans which is time-consuming and subject to variability. Automatic infarct segmentation should be robust to rotation and reflection; however, prior work has not encoded this property into deep learning architecture. Here, we use rotation-reflection equivariance and train a deep learning model to segment stroke volumes in a large cohort of well-characterized patients with acute ischemic stroke in different vascular territories. METHODS In this retrospective study, patients were selected from a stroke registry at Houston Methodist Hospital. Eight hundred seventy-five patients with acute ischemic stroke in any brain area who had magnetic resonance imaging with diffusion-weighted imaging were included for analysis and split 80/20 for training/testing. Infarct volumes were manually segmented by consensus of 3 independent clinical experts and cross-referenced against radiology reports. A rotation-reflection equivariant model was developed based on U-Net and grouped convolutions. Segmentation performance was evaluated using Dice score, precision, and recall. Ninety-day modified Rankin Scale outcome prediction was also evaluated using clinical variables and segmented stroke volumes in different brain regions. RESULTS Segmentation model Dice scores are 0.88 (95% CI, 0.87-0.89; training) and 0.85 (0.82-0.88; testing). The modified Rankin Scale outcome prediction AUC using stroke volume in 30 refined brain regions based upon modified Rankin Scale-relevance areas adjusted for clinical variables was 0.80 (0.76-0.83) with an accuracy of 0.75 (0.72-0.78). CONCLUSIONS We trained a deep learning model with encoded rotation-reflection equivariance to segment acute ischemic stroke lesions in diffusion- weighted imaging using a large data set from the Houston Methodist stroke center. The model achieved competitive performance in 175 well-balanced hold-out testing cases that include strokes from different vascular territories. Furthermore, the location specific stroke volume segmentations from the deep learning model combined with clinical factors demonstrated high AUC and accuracy for 90-day modified Rankin Scale in an outcome prediction model.
Collapse
Affiliation(s)
- Kelvin K Wong
- Department of Radiology, Systems Medicine and Bioengineering, Houston Methodist Cancer Center, Houston Methodist Hospital, Weill Cornell Medicine, TX (K.K.W., J.S.C., R.G., S.T.C.W.).,The Ting Tsung and Wei Fong Chao Center for BRAIN, Houston Methodist Hospital, TX (K.K.W., S.T.C.W.).,Department of Radiology, Houston Methodist Institute for Academic Medicine, TX. (K.K.W., S.T.C.W.)
| | - Jonathon S Cummock
- Department of Radiology, Systems Medicine and Bioengineering, Houston Methodist Cancer Center, Houston Methodist Hospital, Weill Cornell Medicine, TX (K.K.W., J.S.C., R.G., S.T.C.W.)
| | - Guihua Li
- Department of Neurology, Guangdong Second People's Hospital, China (G.L.)
| | - Rahul Ghosh
- Department of Radiology, Systems Medicine and Bioengineering, Houston Methodist Cancer Center, Houston Methodist Hospital, Weill Cornell Medicine, TX (K.K.W., J.S.C., R.G., S.T.C.W.).,MD/PhD Program, Texas A&M University College of Medicine, Bryan. (J.S.C., R.G.)
| | - Pingyi Xu
- Department of Neurology, The First Affiliated Hospital of Guangzhou Medical University, Guangdong, China (P.X.)
| | - John J Volpi
- Department of Neurology, Houston Methodist Institute for Academic Medicine, TX. (J.J.V.).,MD/PhD Program, Texas A&M University College of Medicine, Bryan. (J.S.C., R.G.)
| | - Stephen T C Wong
- Department of Radiology, Systems Medicine and Bioengineering, Houston Methodist Cancer Center, Houston Methodist Hospital, Weill Cornell Medicine, TX (K.K.W., J.S.C., R.G., S.T.C.W.).,The Ting Tsung and Wei Fong Chao Center for BRAIN, Houston Methodist Hospital, TX (K.K.W., S.T.C.W.).,Department of Radiology, Houston Methodist Institute for Academic Medicine, TX. (K.K.W., S.T.C.W.).,Department of Neuroscience and Experimental Therapeutics, Texas A&M University College of Medicine, Bryan. (S.T.C.W.)
| |
Collapse
|
13
|
Research Progress of Machine Learning and Deep Learning in Intelligent Diagnosis of the Coronary Atherosclerotic Heart Disease. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2022; 2022:3016532. [PMID: 35516452 PMCID: PMC9064517 DOI: 10.1155/2022/3016532] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2021] [Revised: 02/27/2022] [Accepted: 03/04/2022] [Indexed: 11/17/2022]
Abstract
The coronary atherosclerotic heart disease is a common cardiovascular disease with high morbidity, disability, and societal burden. Early, precise, and comprehensive diagnosis of the coronary atherosclerotic heart disease is of great significance. The rise of artificial intelligence technologies, represented by machine learning and deep learning, provides new methods to address the above issues. In recent years, artificial intelligence has achieved an extraordinary progress in multiple aspects of coronary atherosclerotic heart disease diagnosis, including the construction of intelligent diagnostic models based on artificial intelligence algorithms, applications of artificial intelligence algorithms in coronary angiography, coronary CT angiography, intravascular imaging, cardiac magnetic resonance, and functional parameters. This paper presents a comprehensive review of the technical background and current state of research on the application of artificial intelligence in the diagnosis of the coronary atherosclerotic heart disease and analyzes recent challenges and perspectives in this field.
Collapse
|
14
|
Min Y, Hu L, Wei L, Nie S. Computer-aided detection of pulmonary nodules based on convolutional neural networks: a review. Phys Med Biol 2022; 67. [DOI: 10.1088/1361-6560/ac568e] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2021] [Accepted: 02/18/2022] [Indexed: 02/08/2023]
Abstract
Abstract
Computer-aided detection (CADe) technology has been proven to increase the detection rate of pulmonary nodules that has important clinical significance for the early diagnosis of lung cancer. In this study, we systematically review the latest techniques in pulmonary nodule CADe based on deep learning models with convolutional neural networks in computed tomography images. First, the brief descriptions and popular architecture of convolutional neural networks are introduced. Second, several common public databases and evaluation metrics are briefly described. Third, state-of-the-art approaches with excellent performances are selected. Subsequently, we combine the clinical diagnostic process and the traditional four steps of pulmonary nodule CADe into two stages, namely, data preprocessing and image analysis. Further, the major optimizations of deep learning models and algorithms are highlighted according to the progressive evaluation effect of each method, and some clinical evidence is added. Finally, various methods are summarized and compared. The innovative or valuable contributions of each method are expected to guide future research directions. The analyzed results show that deep learning-based methods significantly transformed the detection of pulmonary nodules, and the design of these methods can be inspired by clinical imaging diagnostic procedures. Moreover, focusing on the image analysis stage will result in improved returns. In particular, optimal results can be achieved by optimizing the steps of candidate nodule generation and false positive reduction. End-to-end methods, with greater operating speeds and lower computational consumptions, are superior to other methods in CADe of pulmonary nodules.
Collapse
|
15
|
Parida PK, Dora L, Swain M, Agrawal S, Panda R. Data science methodologies in smart healthcare: a review. HEALTH AND TECHNOLOGY 2022. [DOI: 10.1007/s12553-022-00648-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
|
16
|
Li J, Wu J, Zhao Z, Zhang Q, Shao J, Wang C, Qiu Z, Li W. Artificial intelligence-assisted decision making for prognosis and drug efficacy prediction in lung cancer patients: a narrative review. J Thorac Dis 2022; 13:7021-7033. [PMID: 35070384 PMCID: PMC8743400 DOI: 10.21037/jtd-21-864] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2021] [Accepted: 08/30/2021] [Indexed: 02/05/2023]
Abstract
Objective In this review, we aim to present frontier studies in patients with lung cancer as it related to artificial intelligence (AI)-assisted decision-making and summarize the latest advances, challenges and future trend in this field. Background Despite increasing survival rate in cancer patients over the last decades, lung cancer remains one of the leading causes of death worldwide. The early diagnosis, accurate evaluation and individualized treatment are vital approaches to improve the survival rate of patients with lung cancer. Thus, decision making based on these approaches requires accuracy and efficiency beyond manpower. Recent advances in AI and precision medicine have provided a fertile environment for the development of AI-based models. These models have the potential to assist radiologists and oncologists in detecting lung cancer, predicting prognosis and developing personalized treatment plans for better outcomes of the patients. Methods We searched literature from 2000 through July 31th, 2021 in Medline/PubMed, the Web of Science, the Cochrane Library, ACM Digital Library, INSPEC and EMBASE. Key words such as “artificial intelligence”, “AI”, “deep learning”, “lung cancer”, “NSCLC”, “SCLC” were combined to identify related literatures. These literatures were then selected by two independent authors. Articles chosen by only one author will be examined by another author to determine whether this article was relative and valuable. The selected literatures were read by all authors and discussed to draw reliable conclusions. Conclusions AI, especially for those based on deep learning and radiomics, is capable of assisting clinical decision making from many aspects, for its quantitatively interpretation of patients’ information and its potential to deal with the dynamics, individual differences and heterogeneity of lung cancer. Hopefully, remaining problems such as insufficient data and poor interpretability may be solved to put AI-based models into clinical practice.
Collapse
Affiliation(s)
- Jingwei Li
- Department of Respiratory and Critical Care Medicine, West China Medical School/West China Hospital, Sichuan University, Chengdu, China.,West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Jiayang Wu
- West China School of Public Health/West China Fourth Hospital, Sichuan University, Chengdu, China
| | - Zhehao Zhao
- West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Qiran Zhang
- West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Jun Shao
- Department of Respiratory and Critical Care Medicine, West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Chengdi Wang
- Department of Respiratory and Critical Care Medicine, West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Zhixin Qiu
- Department of Respiratory and Critical Care Medicine, West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| | - Weimin Li
- Department of Respiratory and Critical Care Medicine, West China Medical School/West China Hospital, Sichuan University, Chengdu, China
| |
Collapse
|
17
|
Lin FY, Chang YC, Huang HY, Li CC, Chen YC, Chen CM. A radiomics approach for lung nodule detection in thoracic CT images based on the dynamic patterns of morphological variation. Eur Radiol 2022; 32:3767-3777. [PMID: 35020016 DOI: 10.1007/s00330-021-08456-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2021] [Revised: 09/20/2021] [Accepted: 11/02/2021] [Indexed: 11/28/2022]
Abstract
OBJECTIVES To propose and evaluate a set of radiomic features, called morphological dynamics features, for pulmonary nodule detection, which were rooted in the dynamic patterns of morphological variation and needless precise lesion segmentation. MATERIALS AND METHODS Two datasets were involved, namely, university hospital (UH) and LIDC datasets, comprising 72 CT scans (360 nodules) and 888 CT scans (2230 nodules), respectively. Each nodule was annotated by multiple radiologists. Denoted the category of nodules identified by at least k radiologists as ALk. A nodule detection algorithm, called CAD-MD algorithm, was proposed based on the morphological dynamics radiomic features, characterizing a lesion by ten sets of the same features with different values extracted from ten different thresholding results. Each nodule candidate was classified by a two-level classifier, including ten decision trees and a random forest, respectively. The CAD-MD algorithm was compared with a deep learning approach, the N-Net, using the UH dataset. RESULTS On the AL1 and AL2 of the UH dataset, the AUC of the AFROC curves were 0.777 and 0.851 for the CAD-MD algorithm and 0.478 and 0.472 for the N-Net, respectively. The CAD-MD algorithm achieved the sensitivities of 84.4% and 91.4% with 2.98 and 3.69 FPs/scan and the N-Net 74.4% and 80.7% with 3.90 and 4.49 FPs/scan, respectively. On the LIDC dataset, the CAD-MD algorithm attained the sensitivities of 87.6%, 89.2%, 92.2%, and 95.0% with 4 FPs/scan for AL1-AL4, respectively. CONCLUSION The morphological dynamics radiomic features might serve as an effective set of radiomic features for lung nodule detection. KEY POINTS • Texture features varied with such CT system settings as reconstruction kernels of CT images, CT scanner models, and parameter settings, and so on. • Shape and first-order statistics were shown to be the most robust features against variation in CT imaging parameters. • The morphological dynamics radiomic features, which mainly characterized the dynamic patterns of morphological variation, were shown to be effective for lung nodule detection.
Collapse
Affiliation(s)
- Fan-Ya Lin
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, No. 1, Sec. 1, Jen-Ai Road, Taipei, 100, Taiwan
| | - Yeun-Chung Chang
- Department of Medical Imaging, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei, Taiwan
| | | | - Chia-Chen Li
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, No. 1, Sec. 1, Jen-Ai Road, Taipei, 100, Taiwan
| | - Yi-Chang Chen
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, No. 1, Sec. 1, Jen-Ai Road, Taipei, 100, Taiwan.,Department of Medical Imaging, Cardinal Tien Hospital, New Taipei City, Taiwan
| | - Chung-Ming Chen
- Department of Biomedical Engineering, College of Medicine and College of Engineering, National Taiwan University, No. 1, Sec. 1, Jen-Ai Road, Taipei, 100, Taiwan.
| |
Collapse
|
18
|
Chen L, Liu K, Shen H, Ye H, Liu H, Yu L, Li J, Zhao K, Zhu W. Multi-Modality Attention-Guided Three-Dimensional Detection of Non-Small Cell Lung Cancer in 18F-FDG PET/CT Images. IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES 2022. [DOI: 10.1109/trpms.2021.3072064] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
19
|
Spherical coordinates transformation pre-processing in Deep Convolution Neural Networks for brain tumor segmentation in MRI. Med Biol Eng Comput 2021; 60:121-134. [PMID: 34729681 DOI: 10.1007/s11517-021-02464-1] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2021] [Accepted: 10/21/2021] [Indexed: 12/29/2022]
Abstract
Magnetic Resonance Imaging (MRI) is used in everyday clinical practice to assess brain tumors. Deep Convolutional Neural Networks (DCNN) have recently shown very promising results in brain tumor segmentation tasks; however, DCNN models fail the task when applied to volumes that are different from the training dataset. One of the reasons is due to the lack of data standardization to adjust for different models and MR machines. In this work, a 3D spherical coordinates transform during the pre-processing phase has been hypothesized to improve DCNN models' accuracy and to allow more generalizable results even when the model is trained on small and heterogeneous datasets and translated into different domains. Indeed, the spherical coordinate system avoids several standardization issues since it works independently of resolution and imaging settings. The model trained on spherical transform pre-processed inputs resulted in superior performance over the Cartesian-input trained model on predicting gliomas' segmentation on Tumor Core and Enhancing Tumor classes, achieving a further improvement in accuracy by merging the two models together. The proposed model is not resolution-dependent, thus improving segmentation accuracy and theoretically solving some transfer learning problems related to the domain shifting, at least in terms of image resolution in the datasets.
Collapse
|
20
|
Zheng S, Shen Z, Pei C, Ding W, Lin H, Zheng J, Pan L, Zheng B, Huang L. Interpretative computer-aided lung cancer diagnosis: From radiology analysis to malignancy evaluation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 210:106363. [PMID: 34478913 DOI: 10.1016/j.cmpb.2021.106363] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/26/2021] [Accepted: 08/13/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVE Computer-aided diagnosis (CAD) systems promote accurate diagnosis and reduce the burden of radiologists. A CAD system for lung cancer diagnosis includes nodule candidate detection and nodule malignancy evaluation. Recently, deep learning-based pulmonary nodule detection has reached satisfactory performance ready for clinical application. However, deep learning-based nodule malignancy evaluation depends on heuristic inference from low-dose computed tomography (LDCT) volume to malignant probability, and lacks clinical cognition. METHODS In this paper, we propose a joint radiology analysis and malignancy evaluation network called R2MNet to evaluate pulmonary nodule malignancy via the analysis of radiological characteristics. Radiological features are extracted as channel descriptor to highlight specific regions of the input volume that are critical for nodule malignancy evaluation. In addition, for model explanations, we propose channel-dependent activation mapping (CDAM) to visualize features and shed light on the decision process of deep neural networks (DNNs). RESULTS Experimental results on the lung image database consortium image collection (LIDC-IDRI) dataset demonstrate that the proposed method achieved an area under curve (AUC) of 96.27% and 97.52% on nodule radiology analysis and nodule malignancy evaluation, respectively. In addition, explanations of CDAM features proved that the shape and density of nodule regions are two critical factors that influence a nodule to be inferred as malignant. This process conforms to the diagnosis cognition of experienced radiologists. CONCLUSION The network inference process conforms to the diagnostic procedure of radiologists and increases the confidence of evaluation results by incorporating radiology analysis with nodule malignancy evaluation. Besides, model interpretation with CDAM features shed light on the focus regions of DNNs during the estimation of nodule malignancy probabilities.
Collapse
Affiliation(s)
- Shaohua Zheng
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| | - Zhiqiang Shen
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| | - Chenhao Pei
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| | - Wangbin Ding
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| | - Haojin Lin
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| | - Jiepeng Zheng
- Thoracic Department, Fujian Medical University Union Hospital, Fuzhou 350001, China
| | - Lin Pan
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China.
| | - Bin Zheng
- Thoracic Department, Fujian Medical University Union Hospital, Fuzhou 350001, China.
| | - Liqin Huang
- College of Physics and Information Engineering, Fuzhou University, Fuzhou 350108, China
| |
Collapse
|
21
|
Martins Jarnalo CO, Linsen PVM, Blazís SP, van der Valk PHM, Dickerscheid DBM. Clinical evaluation of a deep-learning-based computer-aided detection system for the detection of pulmonary nodules in a large teaching hospital. Clin Radiol 2021; 76:838-845. [PMID: 34404517 DOI: 10.1016/j.crad.2021.07.012] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2021] [Accepted: 07/15/2021] [Indexed: 12/17/2022]
Abstract
AIM To evaluate a deep-learning-based computer-aided detection (DL-CAD) software system for pulmonary nodule detection on computed tomography (CT) images and assess its added value in the clinical practice of a large teaching hospital. MATERIALS AND METHODS A retrospective analysis was performed of 145 chest CT examinations by comparing the output of the DL-CAD software with a reference standard based on the consensus reading of three radiologists. For every nodule in each scan, the location, composition, and maximum diameter in the axial plane were recorded. The subgroup of chest CT examinations (n = 97) without any nodules was used to determine the negative predictive value at the given clinical sensitivity threshold setting. RESULTS The radiologists found 91 nodules and the CAD system 130 nodules of which 80 were true positive. The measured sensitivity was 88% and the mean false-positive rate was 1.04 false positives/scan. The negative predictive value was 95%. For 23 nodules, there was a size discrepancy of which 19 (83%) were measured smaller by the radiologist. The agreement of nodule composition between the CAD results and the reference standard was 95%. CONCLUSIONS The present study found a sensitivity of 88% and a false-positive rate of 1.04 false positives/scan, which match the vendor specification. Together with the measured negative predictive value of 95% the system performs very well; however, these rates are still not good enough to replace the radiologist, even for the specific task of nodule detection. Furthermore, a surprisingly high rate of overestimation of nodule size was observed, which can lead to too many follow-up examinations.
Collapse
Affiliation(s)
- C O Martins Jarnalo
- Department of Radiology, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands.
| | - P V M Linsen
- Department of Radiology, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands
| | - S P Blazís
- Department of Clinical Physics, FP, the Netherlands
| | - P H M van der Valk
- Department of Radiology, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands
| | | |
Collapse
|
22
|
Chlap P, Min H, Vandenberg N, Dowling J, Holloway L, Haworth A. A review of medical image data augmentation techniques for deep learning applications. J Med Imaging Radiat Oncol 2021; 65:545-563. [PMID: 34145766 DOI: 10.1111/1754-9485.13261] [Citation(s) in RCA: 162] [Impact Index Per Article: 54.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2021] [Accepted: 05/23/2021] [Indexed: 12/21/2022]
Abstract
Research in artificial intelligence for radiology and radiotherapy has recently become increasingly reliant on the use of deep learning-based algorithms. While the performance of the models which these algorithms produce can significantly outperform more traditional machine learning methods, they do rely on larger datasets being available for training. To address this issue, data augmentation has become a popular method for increasing the size of a training dataset, particularly in fields where large datasets aren't typically available, which is often the case when working with medical images. Data augmentation aims to generate additional data which is used to train the model and has been shown to improve performance when validated on a separate unseen dataset. This approach has become commonplace so to help understand the types of data augmentation techniques used in state-of-the-art deep learning models, we conducted a systematic review of the literature where data augmentation was utilised on medical images (limited to CT and MRI) to train a deep learning model. Articles were categorised into basic, deformable, deep learning or other data augmentation techniques. As artificial intelligence models trained using augmented data make their way into the clinic, this review aims to give an insight to these techniques and confidence in the validity of the models produced.
Collapse
Affiliation(s)
- Phillip Chlap
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,Liverpool and Macarthur Cancer Therapy Centre, Liverpool Hospital, Sydney, New South Wales, Australia
| | - Hang Min
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,The Australian e-Health and Research Centre, CSIRO Health and Biosecurity, Brisbane, Queensland, Australia
| | - Nym Vandenberg
- Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia
| | - Jason Dowling
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,The Australian e-Health and Research Centre, CSIRO Health and Biosecurity, Brisbane, Queensland, Australia
| | - Lois Holloway
- South Western Sydney Clinical School, University of New South Wales, Sydney, New South Wales, Australia.,Ingham Institute for Applied Medical Research, Sydney, New South Wales, Australia.,Liverpool and Macarthur Cancer Therapy Centre, Liverpool Hospital, Sydney, New South Wales, Australia.,Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia.,Centre for Medical Radiation Physics, University of Wollongong, Wollongong, New South Wales, Australia
| | - Annette Haworth
- Institute of Medical Physics, University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
23
|
3D CNN with Visual Insights for Early Detection of Lung Cancer Using Gradient-Weighted Class Activation. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:6695518. [PMID: 33777347 PMCID: PMC7979307 DOI: 10.1155/2021/6695518] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 02/09/2021] [Accepted: 02/23/2021] [Indexed: 11/17/2022]
Abstract
The 3D convolutional neural network is able to make use of the full nonlinear 3D context information of lung nodule detection from the DICOM (Digital Imaging and Communications in Medicine) images, and the Gradient Class Activation has shown to be useful for tailoring classification tasks and localization interpretation for fine-grained features and visual explanation for the internal working. Gradient-weighted class activation plays a crucial role for clinicians and radiologists in terms of trusting and adopting the model. Practitioners not only rely on a model that can provide high precision but also really want to gain the respect of radiologists. So, in this paper, we explored the lung nodule classification using the improvised 3D AlexNet with lightweight architecture. Our network employed the full nature of the multiview network strategy. We have conducted the binary classification (benign and malignant) on computed tomography (CT) images from the LUNA 16 database conglomerate and database image resource initiative. The results obtained are through the 10-fold cross-validation. Experimental results have shown that the proposed lightweight architecture achieved a superior classification accuracy of 97.17% on LUNA 16 dataset when compared with existing classification algorithms and low-dose CT scan images as well.
Collapse
|
24
|
Lafarge MW, Bekkers EJ, Pluim JP, Duits R, Veta M. Roto-translation equivariant convolutional networks: Application to histopathology image analysis. Med Image Anal 2021; 68:101849. [DOI: 10.1016/j.media.2020.101849] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2019] [Revised: 06/03/2020] [Accepted: 08/14/2020] [Indexed: 01/23/2023]
|
25
|
Blazis SP, Dickerscheid DBM, Linsen PVM, Martins Jarnalo CO. Effect of CT reconstruction settings on the performance of a deep learning based lung nodule CAD system. Eur J Radiol 2021; 136:109526. [PMID: 33453573 DOI: 10.1016/j.ejrad.2021.109526] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2020] [Revised: 12/23/2020] [Accepted: 01/04/2021] [Indexed: 01/08/2023]
Abstract
PURPOSE To study the effect of different reconstruction parameter settings on the performance of a commercially available deep learning based pulmonary nodule CAD system. MATERIALS AND METHODS We performed a retrospective analysis of 24 chest CT scans, reconstructed at 16 different reconstruction settings for two different iterative reconstruction algorithms (SAFIRE and ADMIRE) varying in slice thickness, kernel size and iterative reconstruction level strength using a commercially available deep learning pulmonary nodule CAD system. The DL-CAD software was evaluated at 25 different sensitivity threshold settings and nodules detected by the DL-CAD software were matched against a reference standard based on the consensus reading of three radiologists. RESULTS A total of 384 CT reconstructions was analysed from 24 patients, resulting in a total of 5786 found nodules. We matched the detected nodules against the reference standard, defined by a team of thoracic radiologists, and showed a gradual drop in recall, and an improvement in precision when the iterative strength levels were increased for a constant kernel size. The optimal DL-CAD threshold setting for use in our clinical workflow was found to be 0.88 with an F2 of 0.73 ± 0.053. CONCLUSIONS The DL-CAD system behaves differently on IR data than on FBP data, there is a gradual drop in recall, and growth in precision when the iterative strength levels are increased. As a result, caution should be taken when implementing deep learning software in a hospital with multiple CT scanners and different reconstruction protocols. To the best of our knowledge, this is the first study that demonstrates this result from a DL-CAD system on clinical data.
Collapse
Affiliation(s)
- Stephan P Blazis
- Department of Clinical Physics, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands.
| | | | - Philip V M Linsen
- Department of Radiology, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands
| | | |
Collapse
|
26
|
Spiculation Sign Recognition in a Pulmonary Nodule Based on Spiking Neural P Systems. BIOMED RESEARCH INTERNATIONAL 2020; 2020:6619076. [PMID: 33426059 PMCID: PMC7775132 DOI: 10.1155/2020/6619076] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/10/2020] [Revised: 12/04/2020] [Accepted: 12/11/2020] [Indexed: 11/18/2022]
Abstract
The spiculation sign is one of the main signs to distinguish benign and malignant pulmonary nodules. In order to effectively extract the image feature of a pulmonary nodule for the spiculation sign distinguishment, a new spiculation sign recognition model is proposed based on the doctors' diagnosis process of pulmonary nodules. A maximum density projection model is established to fuse the local three-dimensional information into the two-dimensional image. The complete boundary of a pulmonary nodule is extracted by the improved Snake model, which can take full advantage of the parallel calculation of the Spike Neural P Systems to build a new neural network structure. In this paper, our experiments show that the proposed algorithm can accurately extract the boundary of a pulmonary nodule and effectively improve the recognition rate of the spiculation sign.
Collapse
|
27
|
Singh SP, Wang L, Gupta S, Goli H, Padmanabhan P, Gulyás B. 3D Deep Learning on Medical Images: A Review. SENSORS (BASEL, SWITZERLAND) 2020; 20:E5097. [PMID: 32906819 PMCID: PMC7570704 DOI: 10.3390/s20185097] [Citation(s) in RCA: 155] [Impact Index Per Article: 38.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Revised: 08/31/2020] [Accepted: 09/03/2020] [Indexed: 12/20/2022]
Abstract
The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) CNNs have been employed for the analysis of medical images. In this paper, we trace the history of how the 3D CNN was developed from its machine learning roots, we provide a brief mathematical description of 3D CNN and provide the preprocessing steps required for medical images before feeding them to 3D CNNs. We review the significant research in the field of 3D medical imaging analysis using 3D CNNs (and its variants) in different medical areas such as classification, segmentation, detection and localization. We conclude by discussing the challenges associated with the use of 3D CNNs in the medical imaging domain (and the use of deep learning models in general) and possible future trends in the field.
Collapse
Affiliation(s)
- Satya P. Singh
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 608232, Singapore; (S.P.S.); (B.G.)
- Cognitive Neuroimaging Centre, Nanyang Technological University, Singapore 636921, Singapore
| | - Lipo Wang
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, Singapore;
| | - Sukrit Gupta
- School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798, Singapore; (S.G.); (H.G.)
| | - Haveesh Goli
- School of Computer Science and Engineering, Nanyang Technological University, Singapore 639798, Singapore; (S.G.); (H.G.)
| | - Parasuraman Padmanabhan
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 608232, Singapore; (S.P.S.); (B.G.)
- Cognitive Neuroimaging Centre, Nanyang Technological University, Singapore 636921, Singapore
| | - Balázs Gulyás
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 608232, Singapore; (S.P.S.); (B.G.)
- Cognitive Neuroimaging Centre, Nanyang Technological University, Singapore 636921, Singapore
- Department of Clinical Neuroscience, Karolinska Institute, 17176 Stockholm, Sweden
| |
Collapse
|
28
|
Yu J, Yang B, Wang J, Leader J, Wilson D, Pu J. 2D CNN versus 3D CNN for false-positive reduction in lung cancer screening. J Med Imaging (Bellingham) 2020; 7:051202. [PMID: 33062802 PMCID: PMC7550796 DOI: 10.1117/1.jmi.7.5.051202] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Accepted: 09/28/2020] [Indexed: 11/14/2022] Open
Abstract
Purpose: To clarify whether and to what extent three-dimensional (3D) convolutional neural network (CNN) is superior to 2D CNN when applied to reduce false-positive nodule detections in the scenario of low-dose computed tomography (CT) lung cancer screening. Approach: We established a dataset consisting of 1600 chest CT examinations acquired on different subjects from various sources. There were in total 18,280 candidate nodules in these CT examinations, among which 9185 were nodules and 9095 were not nodules. For each candidate nodule, we extracted a number of cubic subvolumes with a dimension of 72 × 72 × 72 mm 3 by rotating the CT examinations randomly for 25 times prior to the extraction of the axis-aligned subvolumes. These subvolumes were split into three groups in a ratio of 8 ∶ 1 ∶ 1 for training, validation, and independent testing purposes. We developed a multiscale CNN architecture and implemented its 2D and 3D versions to classify pulmonary nodules into two categories, namely true positive and false positive. The performance of the 2D/3D-CNN classification schemes was evaluated using the area under the receiver operating characteristic curves (AUC). The p -values and the 95% confidence intervals (CI) were calculated. Results: The AUC for the optimal 2D-CNN model is 0.9307 (95% CI: 0.9285 to 0.9330) with a sensitivity of 92.70% and a specificity of 76.21%. The 3D-CNN model with the best performance had an AUC of 0.9541 (95% CI: 0.9495 to 0.9583) with a sensitivity of 89.98% and a specificity of 87.30%. The developed multiscale CNN architecture had a better performance than the vanilla architecture did. Conclusions: The 3D-CNN model has a better performance in false-positive reduction compared with its 2D counterpart; however, the improvement is relatively limited and demands more computational resources for training purposes.
Collapse
Affiliation(s)
- Juezhao Yu
- University of Pittsburgh, Departments of Radiology and Bioengineering, Pittsburgh, Pennsylvania, United States
| | - Bohan Yang
- University of Pittsburgh, Departments of Radiology and Bioengineering, Pittsburgh, Pennsylvania, United States
| | - Jing Wang
- University of Pittsburgh, Departments of Radiology and Bioengineering, Pittsburgh, Pennsylvania, United States
| | - Joseph Leader
- University of Pittsburgh, Departments of Radiology and Bioengineering, Pittsburgh, Pennsylvania, United States
| | - David Wilson
- University of Pittsburgh, Department of Medicine, Pittsburgh, Pennsylvania, United States
| | - Jiantao Pu
- University of Pittsburgh, Departments of Radiology and Bioengineering, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
29
|
Marquette CH, Boutros J, Benzaquen J, Ferreira M, Pastre J, Pison C, Padovani B, Bettayeb F, Fallet V, Guibert N, Basille D, Ilie M, Hofman V, Hofman P. Circulating tumour cells as a potential biomarker for lung cancer screening: a prospective cohort study. THE LANCET RESPIRATORY MEDICINE 2020; 8:709-716. [PMID: 32649919 DOI: 10.1016/s2213-2600(20)30081-3] [Citation(s) in RCA: 71] [Impact Index Per Article: 17.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/05/2019] [Revised: 02/11/2020] [Accepted: 02/12/2020] [Indexed: 12/17/2022]
Abstract
BACKGROUND Lung cancer screening with low-dose chest CT (LDCT) reduces the mortality of eligible individuals. Blood signatures might act as a standalone screening tool, refine the selection of patients at risk, or help to classify undetermined nodules detected on LDCT. We previously showed that circulating tumour cells (CTCs) could be detected, using the isolation by size of epithelial tumour cell technique (ISET), long before the cancer was diagnosed radiologically. We aimed to test whether CTCs could be used as a biomarker for lung cancer screening. METHODS We did a prospective, multicentre, cohort study in 21 French university centres. Participants had to be eligible for lung cancer screening as per National Lung Screening Trial criteria and have chronic obstructive pulmonary disease with a fixed airflow limitation defined as post-bronchodilator FEV1/FVC ratio of less than 0·7. Any cancer, other than basocellular skin carcinomas, detected within the previous 5 years was the main exclusion criterion. Participants had three screening rounds at 1-year intervals (T0 [baseline], T1, and T2), which involved LDCT, clinical examination, and a blood test for CTCs detection. Participants and investigators were masked to the results of CTC detection, and cytopathologists were masked to clinical and radiological findings. Our primary objective was to test the diagnostic performance of CTC detection using the ISET technique in lung cancer screening, compared with cancers diagnosed by final pathology, or follow up if pathology was unavailable as the gold standard. This study is registered with ClinicalTrials.gov identifier, number NCT02500693. FINDINGS Between Oct 30, 2015, and Feb 2, 2017, we enrolled 614 participants, predominantly men (437 [71%]), aged 65·1 years (SD 6·5), and heavy smokers (52·7 pack-years [SD 21·5]). 81 (13%) participants dropped out between baseline and T1, and 56 (11%) did between T1 and T2. Nodules were detected on 178 (29%) of 614 baseline LDCTs. 19 participants (3%) were diagnosed with a prevalent lung cancer at T0 and 19 were diagnosed with incident lung cancer (15 (3%) of 533 at T1 and four (1%) of 477 at T2). Extrapulmonary cancers were diagnosed in 27 (4%) of participants. Overall 28 (2%) of 1187 blood samples were not analysable. At baseline, the sensitivity of CTC detection for lung cancer detection was 26·3% (95% CI 11·8-48·8). ISET was unable to predict lung cancer or extrapulmonary cancer development. INTERPRETATION CTC detection using ISET is not suitable for lung cancer screening. FUNDING French Government, Conseil Départemental 06, Fondation UNICE, Fondation Aveni, Fondation de France, Ligue Contre le Cancer-Comité des Alpes-Maritimes, ARC (Canc'Air Genexposomics), Claire de Divonne-Pollner, Enca Faidhi, Basil Faidhi, Fabienne Mourou, Michel Mourou, Leonid Fridlyand, cogs4cancer, and the Fondation Masikini.
Collapse
Affiliation(s)
- Charles-Hugo Marquette
- Department of Pulmonary Medicine and Oncology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Institute of Research on Cancer and Aging, Centre National de la Recherche Scientifique, Institut National de la Santé et de la Recherche Médicale, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France.
| | - Jacques Boutros
- Department of Pulmonary Medicine and Oncology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France
| | - Jonathan Benzaquen
- Department of Pulmonary Medicine and Oncology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Institute of Research on Cancer and Aging, Centre National de la Recherche Scientifique, Institut National de la Santé et de la Recherche Médicale, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Marion Ferreira
- Department of Pulmonary Medicine, Centre Hospitalier Régional Universitaire Tours, Tours, France
| | - Jean Pastre
- Department of Pulmonary Medicine, Hôpital Européen Georges Pompidou, Paris, France
| | - Christophe Pison
- Centre Hospitalier Universitaire Grenoble Alpes, Service Hospitalier Universitaire Pneumologie Physiologie, Université Grenoble Alpes, Grenoble, France
| | - Bernard Padovani
- Department of Radiology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Faiza Bettayeb
- Clinique des bronches, allergies, et sommeil, Centre Hospitalier Universitaire de Marseille, Institut National de la Santé et de la Recherche Médicale, Centre Recherche en Cardiovasculaire et Nutrition, Aix Marseille Université, Marseille, France
| | - Vincent Fallet
- Sorbonne Université, Groupe de Recherche Clinique 4, Theranoscan, Assistance Publique - Hôpitaux de Paris, Service de Pneumologie, Hôpital Tenon, Paris, France
| | - Nicolas Guibert
- Department of Pulmonary Medicine, Centre Hospitalier Universitaire Toulouse, Toulouse, France
| | - Damien Basille
- Department of Pulmonary Medicine, Centre Hospitalier Universitaire d'Amiens, Amiens, France
| | - Marius Ilie
- Laboratory of Clinical and Experimental Pathology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Hospital-Related Biobank, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Institute of Research on Cancer and Aging, Centre National de la Recherche Scientifique, Institut National de la Santé et de la Recherche Médicale, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Véronique Hofman
- Laboratory of Clinical and Experimental Pathology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Hospital-Related Biobank, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Institute of Research on Cancer and Aging, Centre National de la Recherche Scientifique, Institut National de la Santé et de la Recherche Médicale, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France
| | - Paul Hofman
- Laboratory of Clinical and Experimental Pathology, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Hospital-Related Biobank, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, University Hospital Federation OncoAge, Nice, France; Institute of Research on Cancer and Aging, Centre National de la Recherche Scientifique, Institut National de la Santé et de la Recherche Médicale, Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, Nice, France
| | | |
Collapse
|
30
|
Farhat H, Sakr GE, Kilany R. Deep learning applications in pulmonary medical imaging: recent updates and insights on COVID-19. MACHINE VISION AND APPLICATIONS 2020; 31:53. [PMID: 32834523 PMCID: PMC7386599 DOI: 10.1007/s00138-020-01101-5] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/18/2020] [Revised: 06/21/2020] [Accepted: 07/07/2020] [Indexed: 05/07/2023]
Abstract
Shortly after deep learning algorithms were applied to Image Analysis, and more importantly to medical imaging, their applications increased significantly to become a trend. Likewise, deep learning applications (DL) on pulmonary medical images emerged to achieve remarkable advances leading to promising clinical trials. Yet, coronavirus can be the real trigger to open the route for fast integration of DL in hospitals and medical centers. This paper reviews the development of deep learning applications in medical image analysis targeting pulmonary imaging and giving insights of contributions to COVID-19. It covers more than 160 contributions and surveys in this field, all issued between February 2017 and May 2020 inclusively, highlighting various deep learning tasks such as classification, segmentation, and detection, as well as different pulmonary pathologies like airway diseases, lung cancer, COVID-19 and other infections. It summarizes and discusses the current state-of-the-art approaches in this research domain, highlighting the challenges, especially with COVID-19 pandemic current situation.
Collapse
Affiliation(s)
- Hanan Farhat
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| | - George E. Sakr
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| | - Rima Kilany
- Saint Joseph University of Beirut, Mar Roukos, Beirut, Lebanon
| |
Collapse
|
31
|
Application of artificial intelligence in surgery. Front Med 2020; 14:417-430. [DOI: 10.1007/s11684-020-0770-0] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2019] [Accepted: 03/05/2020] [Indexed: 12/14/2022]
|
32
|
Local rotation invariance in 3D CNNs. Med Image Anal 2020; 65:101756. [PMID: 32623274 DOI: 10.1016/j.media.2020.101756] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2019] [Revised: 03/10/2020] [Accepted: 06/15/2020] [Indexed: 11/23/2022]
Abstract
Locally Rotation Invariant (LRI) image analysis was shown to be fundamental in many applications and in particular in medical imaging where local structures of tissues occur at arbitrary rotations. LRI constituted the cornerstone of several breakthroughs in texture analysis, including Local Binary Patterns (LBP), Maximum Response 8 (MR8) and steerable filterbanks. Whereas globally rotation invariant Convolutional Neural Networks (CNN) were recently proposed, LRI was very little investigated in the context of deep learning. LRI designs allow learning filters accounting for all orientations, which enables a drastic reduction of trainable parameters and training data when compared to standard 3D CNNs. In this paper, we propose and compare several methods to obtain LRI CNNs with directional sensitivity. Two methods use orientation channels (responses to rotated kernels), either by explicitly rotating the kernels or using steerable filters. These orientation channels constitute a locally rotation equivariant representation of the data. Local pooling across orientations yields LRI image analysis. Steerable filters are used to achieve a fine and efficient sampling of 3D rotations as well as a reduction of trainable parameters and operations, thanks to a parametric representations involving solid Spherical Harmonics (SH),which are products of SH with associated learned radial profiles. Finally, we investigate a third strategy to obtain LRI based on rotational invariants calculated from responses to a learned set of solid SHs. The proposed methods are evaluated and compared to standard CNNs on 3D datasets including synthetic textured volumes composed of rotated patterns, and pulmonary nodule classification in CT. The results show the importance of LRI image analysis while resulting in a drastic reduction of trainable parameters, outperforming standard 3D CNNs trained with rotational data augmentation.
Collapse
|
33
|
Lei Y, Tian Y, Shan H, Zhang J, Wang G, Kalra MK. Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping. Med Image Anal 2020; 60:101628. [DOI: 10.1016/j.media.2019.101628] [Citation(s) in RCA: 16] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2019] [Revised: 12/04/2019] [Accepted: 12/06/2019] [Indexed: 10/25/2022]
|
34
|
Mastouri R, Khlifa N, Neji H, Hantous-Zannad S. Deep learning-based CAD schemes for the detection and classification of lung nodules from CT images: A survey. JOURNAL OF X-RAY SCIENCE AND TECHNOLOGY 2020; 28:591-617. [PMID: 32568165 DOI: 10.3233/xst-200660] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
BACKGROUND Lung cancer is the most common cancer in the world. Computed tomography (CT) is the standard medical imaging modality for early lung nodule detection and diagnosis that improves patient's survival rate. Recently, deep learning algorithms, especially convolutional neural networks (CNNs), have become a preferred methodology for developing computer-aided detection and diagnosis (CAD) schemes of lung CT images. OBJECTIVE Several CNN-based research projects have been initiated to design robust and efficient CAD schemes for the detection and classification of lung nodules. This paper reviews the recent works in this area and gives an insight into technical progress. METHODS First, a brief overview of CNN models and their basic structures is presented in this investigation. Then, we provide an analytic comparison of the existing approaches to discover recent trend and upcoming challenges. We also introduce an objective description of both handcrafted and deep learning features, as well as the types of nodules, the medical imaging modalities, the widely used databases, and related works in the last three years. The articles presented in this work were selected from various databases. About 57% of reviewed articles published in the last year. RESULTS Our analysis reveals that several methods achieved promising performance with high sensitivity rates ranging from 66% to 100% under the false-positive rates ranging from 1 to 15 per CT scan. It can be noted that CNN models have contributed to the accurate detection and early diagnosis of lung nodules. CONCLUSIONS From the critical discussion and an outline for prospective directions, this survey provide researchers valuable information to master the deep learning concepts and to deepen their knowledge of the trend and latest techniques in developing CAD schemes of lung CT images.
Collapse
Affiliation(s)
- Rekka Mastouri
- University of Tunis el Manar, Higher Institute of Medical Technologies of Tunis, Research Laboratory of Biophysics and Medical Technologies, 1006 Tunis, Tunisia
| | - Nawres Khlifa
- University of Tunis el Manar, Higher Institute of Medical Technologies of Tunis, Research Laboratory of Biophysics and Medical Technologies, 1006 Tunis, Tunisia
| | - Henda Neji
- University of Tunis el Manar, Faculty of Medicine of Tunis, 1007 Tunis, Tunisia
- Department of Medical Imaging, Abderrahmen Mami Hospital, 2035 Ariana, Tunisia
| | - Saoussen Hantous-Zannad
- University of Tunis el Manar, Faculty of Medicine of Tunis, 1007 Tunis, Tunisia
- Department of Medical Imaging, Abderrahmen Mami Hospital, 2035 Ariana, Tunisia
| |
Collapse
|
35
|
Artificial Intelligence-Based Thyroid Nodule Classification Using Information from Spatial and Frequency Domains. J Clin Med 2019; 8:jcm8111976. [PMID: 31739517 PMCID: PMC6912332 DOI: 10.3390/jcm8111976] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Revised: 11/08/2019] [Accepted: 11/10/2019] [Indexed: 12/25/2022] Open
Abstract
Image-based computer-aided diagnosis (CAD) systems have been developed to assist doctors in the diagnosis of thyroid cancer using ultrasound thyroid images. However, the performance of these systems is strongly dependent on the selection of detection and classification methods. Although there are previous researches on this topic, there is still room for enhancement of the classification accuracy of the existing methods. To address this issue, we propose an artificial intelligence-based method for enhancing the performance of the thyroid nodule classification system. Thus, we extract image features from ultrasound thyroid images in two domains: spatial domain based on deep learning, and frequency domain based on Fast Fourier transform (FFT). Using the extracted features, we perform a cascade classifier scheme for classifying the input thyroid images into either benign (negative) or malign (positive) cases. Through expensive experiments using a public dataset, the thyroid digital image database (TDID) dataset, we show that our proposed method outperforms the state-of-the-art methods and produces up-to-date classification results for the thyroid nodule classification problem.
Collapse
|