1
|
Kalavakonda V, Mohamed S, Abhay L, Muthu S. Automated Screening of Hip X-rays for Osteoporosis by Singh's Index Using Machine Learning Algorithms. Indian J Orthop 2024; 58:1449-1457. [PMID: 39324087 PMCID: PMC11420408 DOI: 10.1007/s43465-024-01246-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/01/2024] [Accepted: 07/22/2024] [Indexed: 09/11/2024]
Abstract
Introduction Osteoporosis is a significant and growing global public health problem, projected to increase in the next decade. The Singh Index (SI) is a simple, semi-quantitative evaluation tool for diagnosing osteoporosis with plain hip radiographs based on the visibility of the trabecular pattern in the proximal femur. This work aims to develop an automated tool to diagnose osteoporosis using SI of hip radiograph images with the help of machine learning algorithms. Methods We used 830 hip X-ray images collected from Indian men and women aged between 20 and 70 which were annotated and labeled for appropriate SI. We employed three state-of-the-art machine learning algorithms-Vision Transformer (ViT), MobileNet-V3, and a Stacked Convolutional Neural Network (CNN)-for image pre-processing, feature extraction, and automation. Each algorithm was evaluated and compared for accuracy, precision, recall, and generalization capabilities to diagnose osteoporosis. Results The ViT model achieved an overall accuracy of 62.6% with macro-averages of 0.672, 0.597, and 0.622 for precision, recall, and F1 score, respectively. MobileNet-V3 presented a more encouraging accuracy of 69.6% with macro-averages for precision, recall, and F1 score of 0.845, 0.636, and 0.652, respectively. The stacked CNN model demonstrated the strongest performance, achieving an accuracy of 93.6% with well-balanced precision, recall, and F1-score metrics. Conclusion The superior accuracy, precision-recall balance, and high F1-scores of the stacked CNN model make it the most reliable tool for screening radiographs and diagnosing osteoporosis using the SI.
Collapse
Affiliation(s)
- Vijaya Kalavakonda
- Department of Computing Technologies, School of Computing, Faculty of Engineering and Technology, SRM Institute of Science and Technology, Chennai, India
| | - Sameer Mohamed
- Sri Ramachandra Institute of Higher Education and Research, Chennai, India
| | - Lal Abhay
- Department of Computing Technologies, School of Computing, Faculty of Engineering and Technology, SRM Institute of Science and Technology, Chennai, India
| | - Sathish Muthu
- Orthopaedic Research Group, Coimbatore, Tamil Nadu India
- Department of Biotechnology, Faculty of Engineering, Karpagam Academy of Higher Education, Coimbatore, Tamil Nadu India
- Department of Orthopaedics, Government Medical College, Karur, Tamil Nadu India
| |
Collapse
|
2
|
Takahashi S, Sakaguchi Y, Kouno N, Takasawa K, Ishizu K, Akagi Y, Aoyama R, Teraya N, Bolatkan A, Shinkai N, Machino H, Kobayashi K, Asada K, Komatsu M, Kaneko S, Sugiyama M, Hamamoto R. Comparison of Vision Transformers and Convolutional Neural Networks in Medical Image Analysis: A Systematic Review. J Med Syst 2024; 48:84. [PMID: 39264388 PMCID: PMC11393140 DOI: 10.1007/s10916-024-02105-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Accepted: 08/31/2024] [Indexed: 09/13/2024]
Abstract
In the rapidly evolving field of medical image analysis utilizing artificial intelligence (AI), the selection of appropriate computational models is critical for accurate diagnosis and patient care. This literature review provides a comprehensive comparison of vision transformers (ViTs) and convolutional neural networks (CNNs), the two leading techniques in the field of deep learning in medical imaging. We conducted a survey systematically. Particular attention was given to the robustness, computational efficiency, scalability, and accuracy of these models in handling complex medical datasets. The review incorporates findings from 36 studies and indicates a collective trend that transformer-based models, particularly ViTs, exhibit significant potential in diverse medical imaging tasks, showcasing superior performance when contrasted with conventional CNN models. Additionally, it is evident that pre-training is important for transformer applications. We expect this work to help researchers and practitioners select the most appropriate model for specific medical image analysis tasks, accounting for the current state of the art and future trends in the field.
Collapse
Affiliation(s)
- Satoshi Takahashi
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Yusuke Sakaguchi
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Department of Neurosurgery, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Nobuji Kouno
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
- Department of Surgery, Graduate School of Medicine, Kyoto University, Yoshida-konoe-cho, Sakyo-ku, Kyoto, 606-8303, Japan
| | - Ken Takasawa
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Kenichi Ishizu
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Yu Akagi
- Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo Bunkyo-ku, Tokyo, 113-8655, Japan
| | - Rina Aoyama
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Department of Obstetrics and Gynecology, Showa University School of Medicine, 1-5-8 Hatanodai, Shinagawa-ku, Tokyo, 142-8666, Japan
| | - Naoki Teraya
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Department of Obstetrics and Gynecology, Showa University School of Medicine, 1-5-8 Hatanodai, Shinagawa-ku, Tokyo, 142-8666, Japan
| | - Amina Bolatkan
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Norio Shinkai
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Hidenori Machino
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Kazuma Kobayashi
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Ken Asada
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Masaaki Komatsu
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan
| | - Syuzo Kaneko
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan
| | - Masashi Sugiyama
- RIKEN Center for Advanced Intelligence Project, Tokyo, 103-0027, Japan
| | - Ryuji Hamamoto
- Division of Medical AI Research and Development, National Cancer Center Research Institute, 5-1-1 Tsukiji, Chuo-ku, Tokyo, 104-0045, Japan.
- Cancer Translational Research Team, RIKEN Center for Advanced Intelligence Project, 1-4-1 Nihonbashi, Chuo-ku, Tokyo, 103-0027, Japan.
| |
Collapse
|
3
|
Tchetchenian A, Zekelman L, Chen Y, Rushmore J, Zhang F, Yeterian EH, Makris N, Rathi Y, Meijering E, Song Y, O'Donnell LJ. Deep multimodal saliency parcellation of cerebellar pathways: Linking microstructure and individual function through explainable multitask learning. Hum Brain Mapp 2024; 45:e70008. [PMID: 39185598 PMCID: PMC11345609 DOI: 10.1002/hbm.70008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2024] [Revised: 07/18/2024] [Accepted: 08/10/2024] [Indexed: 08/27/2024] Open
Abstract
Parcellation of human cerebellar pathways is essential for advancing our understanding of the human brain. Existing diffusion magnetic resonance imaging tractography parcellation methods have been successful in defining major cerebellar fibre tracts, while relying solely on fibre tract structure. However, each fibre tract may relay information related to multiple cognitive and motor functions of the cerebellum. Hence, it may be beneficial for parcellation to consider the potential importance of the fibre tracts for individual motor and cognitive functional performance measures. In this work, we propose a multimodal data-driven method for cerebellar pathway parcellation, which incorporates both measures of microstructure and connectivity, and measures of individual functional performance. Our method involves first training a multitask deep network to predict various cognitive and motor measures from a set of fibre tract structural features. The importance of each structural feature for predicting each functional measure is then computed, resulting in a set of structure-function saliency values that are clustered to parcellate cerebellar pathways. We refer to our method as Deep Multimodal Saliency Parcellation (DeepMSP), as it computes the saliency of structural measures for predicting cognitive and motor functional performance, with these saliencies being applied to the task of parcellation. Applying DeepMSP to a large-scale dataset from the Human Connectome Project Young Adult study (n = 1065), we found that it was feasible to identify multiple cerebellar pathway parcels with unique structure-function saliency patterns that were stable across training folds. We thoroughly experimented with all stages of the DeepMSP pipeline, including network selection, structure-function saliency representation, clustering algorithm, and cluster count. We found that a 1D convolutional neural network architecture and a transformer network architecture both performed comparably for the multitask prediction of endurance, strength, reading decoding, and vocabulary comprehension, with both architectures outperforming a fully connected network architecture. Quantitative experiments demonstrated that a proposed low-dimensional saliency representation with an explicit measure of motor versus cognitive category bias achieved the best parcellation results, while a parcel count of four was most successful according to standard cluster quality metrics. Our results suggested that motor and cognitive saliencies are distributed across the cerebellar white matter pathways. Inspection of the final k = 4 parcellation revealed that the highest-saliency parcel was most salient for the prediction of both motor and cognitive performance scores and included parts of the middle and superior cerebellar peduncles. Our proposed saliency-based parcellation framework, DeepMSP, enables multimodal, data-driven tractography parcellation. Through utilising both structural features and functional performance measures, this parcellation strategy may have the potential to enhance the study of structure-function relationships of the cerebellar pathways.
Collapse
Affiliation(s)
- Ari Tchetchenian
- Biomedical Image Computing Group, School of Computer Science and EngineeringUniversity of New South Wales (UNSW)SydneyNew South WalesAustralia
| | - Leo Zekelman
- Department of Radiology, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Harvard UniversityCambridgeMassachusettsUSA
| | - Yuqian Chen
- Department of Radiology, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Jarrett Rushmore
- Department of PsychiatryMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of NeurologyMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of Anatomy and NeurobiologyBoston University School of MedicineBostonMassachusettsUSA
| | - Fan Zhang
- School of Information and Communication EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina
| | | | - Nikos Makris
- Department of PsychiatryMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of NeurologyMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyMassachusetts General Hospital, Harvard Medical SchoolBostonMassachusettsUSA
- Department of Psychiatry, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Yogesh Rathi
- Department of Radiology, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of Psychiatry, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Erik Meijering
- Biomedical Image Computing Group, School of Computer Science and EngineeringUniversity of New South Wales (UNSW)SydneyNew South WalesAustralia
| | - Yang Song
- Biomedical Image Computing Group, School of Computer Science and EngineeringUniversity of New South Wales (UNSW)SydneyNew South WalesAustralia
| | - Lauren J. O'Donnell
- Department of Radiology, Brigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| |
Collapse
|
4
|
Shimada Y, Ojima T, Takaoka Y, Sugano A, Someya Y, Hirabayashi K, Homma T, Kitamura N, Akemoto Y, Tanabe K, Sato F, Yoshimura N, Tsuchiya T. Prediction of visceral pleural invasion of clinical stage I lung adenocarcinoma using thoracoscopic images and deep learning. Surg Today 2024; 54:540-550. [PMID: 37864054 DOI: 10.1007/s00595-023-02756-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Accepted: 09/13/2023] [Indexed: 10/22/2023]
Abstract
PURPOSE To develop deep learning models using thoracoscopic images to identify visceral pleural invasion (VPI) in patients with clinical stage I lung adenocarcinoma, and to verify if these models can be applied clinically. METHODS Two deep learning models, one based on a convolutional neural network (CNN) and the other based on a vision transformer (ViT), were applied and trained via 463 images (VPI negative: 269 images, VPI positive: 194 images) captured from surgical videos of 81 patients. Model performances were validated via an independent test dataset containing 46 images (VPI negative: 28 images, VPI positive: 18 images) from 46 test patients. RESULTS The areas under the receiver operating characteristic curves of the CNN-based and ViT-based models were 0.77 and 0.84 (p = 0.304), respectively. The accuracy, sensitivity, specificity, and positive and negative predictive values were 73.91, 83.33, 67.86, 62.50, and 86.36% for the CNN-based model and 78.26, 77.78, 78.57, 70.00, and 84.62% for the ViT-based model, respectively. These models' diagnostic abilities were comparable to those of board-certified thoracic surgeons and tended to be superior to those of non-board-certified thoracic surgeons. CONCLUSION The deep learning model systems can be utilized in clinical applications via data expansion.
Collapse
Affiliation(s)
- Yoshifumi Shimada
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Toshihiro Ojima
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Yutaka Takaoka
- Data Science Center for Medicine and Hospital Management, Toyama University Hospital, 2630 Sugitani, Toyama, Japan
- Center for Data Science and Artificial Intelligence Research Promotion, Toyama University Hospital, 2630 Sugitani, Toyama, Japan
| | - Aki Sugano
- Data Science Center for Medicine and Hospital Management, Toyama University Hospital, 2630 Sugitani, Toyama, Japan
- Center for Clinical Research, Toyama University Hospital, 2630 Sugitani, Toyama, Japan
| | - Yoshiaki Someya
- Center for Data Science and Artificial Intelligence Research Promotion, Toyama University Hospital, 2630 Sugitani, Toyama, Japan
| | - Kenichi Hirabayashi
- Department of Diagnostic Pathology, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Takahiro Homma
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Naoya Kitamura
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Yushi Akemoto
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Keitaro Tanabe
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Fumitaka Sato
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Naoki Yoshimura
- Department of Cardiovascular Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan
| | - Tomoshi Tsuchiya
- Department of Thoracic Surgery, University of Toyama, 2630 Sugitani, Toyama, Japan.
| |
Collapse
|
5
|
Quishpe-Usca A, Cuenca-Dominguez S, Arias-Viñansaca A, Bosmediano-Angos K, Villalba-Meneses F, Ramírez-Cando L, Tirado-Espín A, Cadena-Morejón C, Almeida-Galárraga D, Guevara C. The effect of hair removal and filtering on melanoma detection: a comparative deep learning study with AlexNet CNN. PeerJ Comput Sci 2024; 10:e1953. [PMID: 38660169 PMCID: PMC11041978 DOI: 10.7717/peerj-cs.1953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2023] [Accepted: 03/03/2024] [Indexed: 04/26/2024]
Abstract
Melanoma is the most aggressive and prevalent form of skin cancer globally, with a higher incidence in men and individuals with fair skin. Early detection of melanoma is essential for the successful treatment and prevention of metastasis. In this context, deep learning methods, distinguished by their ability to perform automated and detailed analysis, extracting melanoma-specific features, have emerged. These approaches excel in performing large-scale analysis, optimizing time, and providing accurate diagnoses, contributing to timely treatments compared to conventional diagnostic methods. The present study offers a methodology to assess the effectiveness of an AlexNet-based convolutional neural network (CNN) in identifying early-stage melanomas. The model is trained on a balanced dataset of 10,605 dermoscopic images, and on modified datasets where hair, a potential obstructive factor, was detected and removed allowing for an assessment of how hair removal affects the model's overall performance. To perform hair removal, we propose a morphological algorithm combined with different filtering techniques for comparison: Fourier, Wavelet, average blur, and low-pass filters. The model is evaluated through 10-fold cross-validation and the metrics of accuracy, recall, precision, and the F1 score. The results demonstrate that the proposed model performs the best for the dataset where we implemented both a Wavelet filter and hair removal algorithm. It has an accuracy of 91.30%, a recall of 87%, a precision of 95.19%, and an F1 score of 90.91%.
Collapse
Affiliation(s)
- Angélica Quishpe-Usca
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Stefany Cuenca-Dominguez
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Araceli Arias-Viñansaca
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Karen Bosmediano-Angos
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Fernando Villalba-Meneses
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Lenin Ramírez-Cando
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Andrés Tirado-Espín
- School of Mathematical and Computational Sciences, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Carolina Cadena-Morejón
- School of Mathematical and Computational Sciences, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Diego Almeida-Galárraga
- School of Biological Sciences and Engineering, Yachay Tech University, San Miguel de Urcuquí, Imbabura, Ecuador
| | - Cesar Guevara
- Quantitative Methods Department, CUNEF Universidad, Madrid, Madrid, Spain
| |
Collapse
|
6
|
Huma C, Hawon L, Sarisha J, Erdal T, Kevin C, Valentina KA. Advances in the field of developing biomarkers for re-irradiation: a how-to guide to small, powerful data sets and artificial intelligence. EXPERT REVIEW OF PRECISION MEDICINE AND DRUG DEVELOPMENT 2024; 9:3-16. [PMID: 38550554 PMCID: PMC10972602 DOI: 10.1080/23808993.2024.2325936] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Accepted: 02/28/2024] [Indexed: 04/01/2024]
Abstract
Introduction Patient selection remains challenging as the clinical use of re-irradiation (re-RT) increases. Re-RT data is limited to retrospective studies and small prospective single-institution reports, resulting in small, heterogenous data sets. Validated prognostic and predictive biomarkers are derived from large-volume studies with long-term follow-up. This review aims to examine existing re-RT publications and available data sets and discuss strategies using artificial intelligence (AI) to approach small data sets to optimize the use of re-RT data. Methods Re-RT publications were identified where associated public data was present. The existing literature on small data sets to identify biomarkers was also explored. Results Publications with associated public data were identified, with glioma and nasopharyngeal cancers emerging as the most common tumor sites where the use of re-RT was the primary management approach. Existing and emerging AI strategies have been used to approach small data sets including data generation, augmentation, discovery, and transfer learning. Conclusions Further data is needed to generate adaptive frameworks, improve the collection of specimens for molecular analysis, and improve the interpretability of results in re-RT data.
Collapse
Affiliation(s)
- Chaudhry Huma
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| | - Lee Hawon
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| | - Jagasia Sarisha
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| | - Tasci Erdal
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| | - Camphausen Kevin
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| | - Krauze Andra Valentina
- Radiation Oncology Branch, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Building 10, Bethesda, MD, 20892, United States
| |
Collapse
|
7
|
Singh S, Kumar M, Kumar A, Verma BK, Abhishek K, Selvarajan S. Efficient pneumonia detection using Vision Transformers on chest X-rays. Sci Rep 2024; 14:2487. [PMID: 38291130 PMCID: PMC10827725 DOI: 10.1038/s41598-024-52703-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 01/22/2024] [Indexed: 02/01/2024] Open
Abstract
Pneumonia is a widespread and acute respiratory infection that impacts people of all ages. Early detection and treatment of pneumonia are essential for avoiding complications and enhancing clinical results. We can reduce mortality, improve healthcare efficiency, and contribute to the global battle against a disease that has plagued humanity for centuries by devising and deploying effective detection methods. Detecting pneumonia is not only a medical necessity but also a humanitarian imperative and a technological frontier. Chest X-rays are a frequently used imaging modality for diagnosing pneumonia. This paper examines in detail a cutting-edge method for detecting pneumonia implemented on the Vision Transformer (ViT) architecture on a public dataset of chest X-rays available on Kaggle. To acquire global context and spatial relationships from chest X-ray images, the proposed framework deploys the ViT model, which integrates self-attention mechanisms and transformer architecture. According to our experimentation with the proposed Vision Transformer-based framework, it achieves a higher accuracy of 97.61%, sensitivity of 95%, and specificity of 98% in detecting pneumonia from chest X-rays. The ViT model is preferable for capturing global context, comprehending spatial relationships, and processing images that have different resolutions. The framework establishes its efficacy as a robust pneumonia detection solution by surpassing convolutional neural network (CNN) based architectures.
Collapse
Affiliation(s)
| | - Manoj Kumar
- JSS Academy of Technical Education, Noida, India
| | - Abhay Kumar
- National Institute of Technology Patna, Patna, India
| | | | | | - Shitharth Selvarajan
- School of Built Environment, Engineering and Computing, Leeds Beckett University, LS1 3HE, Leeds, UK.
| |
Collapse
|
8
|
Fanizzi A, Fadda F, Comes MC, Bove S, Catino A, Di Benedetto E, Milella A, Montrone M, Nardone A, Soranno C, Rizzo A, Guven DC, Galetta D, Massafra R. Comparison between vision transformers and convolutional neural networks to predict non-small lung cancer recurrence. Sci Rep 2023; 13:20605. [PMID: 37996651 PMCID: PMC10667245 DOI: 10.1038/s41598-023-48004-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2023] [Accepted: 11/21/2023] [Indexed: 11/25/2023] Open
Abstract
Non-Small cell lung cancer (NSCLC) is one of the most dangerous cancers, with 85% of all new lung cancer diagnoses and a 30-55% of recurrence rate after surgery. Thus, an accurate prediction of recurrence risk in NSCLC patients during diagnosis could be essential to drive targeted therapies preventing either overtreatment or undertreatment of cancer patients. The radiomic analysis of CT images has already shown great potential in solving this task; specifically, Convolutional Neural Networks (CNNs) have already been proposed providing good performances. Recently, Vision Transformers (ViTs) have been introduced, reaching comparable and even better performances than traditional CNNs in image classification. The aim of the proposed paper was to compare the performances of different state-of-the-art deep learning algorithms to predict cancer recurrence in NSCLC patients. In this work, using a public database of 144 patients, we implemented a transfer learning approach, involving different Transformers architectures like pre-trained ViTs, pre-trained Pyramid Vision Transformers, and pre-trained Swin Transformers to predict the recurrence of NSCLC patients from CT images, comparing their performances with state-of-the-art CNNs. Although, the best performances in this study are reached via CNNs with AUC, Accuracy, Sensitivity, Specificity, and Precision equal to 0.91, 0.89, 0.85, 0.90, and 0.78, respectively, Transformer architectures reach comparable ones with AUC, Accuracy, Sensitivity, Specificity, and Precision equal to 0.90, 0.86, 0.81, 0.89, and 0.75, respectively. Based on our preliminary experimental results, it appears that Transformers architectures do not add improvements in terms of predictive performance to the addressed problem.
Collapse
Affiliation(s)
- Annarita Fanizzi
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Federico Fadda
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Maria Colomba Comes
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Samantha Bove
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy.
| | - Annamaria Catino
- Unità Operativa Complessa di Oncologia Toracica, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Erika Di Benedetto
- Unità Operativa Complessa di Oncologia Medica, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Angelo Milella
- Dipartimento di ElettronicaInformazione e Bioingegneria, Politecnico di Milano, Via Giuseppe Ponzio, 34, 20133, Milan, Italy
| | - Michele Montrone
- Unità Operativa Complessa di Oncologia Toracica, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Annalisa Nardone
- Unità Operativa Complessa di Radioterapia, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Clara Soranno
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Alessandro Rizzo
- Unità Operativa Complessa di Oncologia Medica 'Don Tonino Bello', I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Deniz Can Guven
- Department of Medical Oncology, Hacettepe University Cancer Institute, 06100, Sihhiye, Ankara, Turkey
| | - Domenico Galetta
- Unità Operativa Complessa di Oncologia Toracica, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| | - Raffaella Massafra
- Struttura Semplice Dipartimentale Fisica Sanitaria, I.R.C.C.S. Istituto Tumori 'Giovanni Paolo II', Viale Orazio Flacco 65, 70124, Bari, Italy
| |
Collapse
|
9
|
Pachetti E, Colantonio S. 3D-Vision-Transformer Stacking Ensemble for Assessing Prostate Cancer Aggressiveness from T2w Images. Bioengineering (Basel) 2023; 10:1015. [PMID: 37760117 PMCID: PMC10525095 DOI: 10.3390/bioengineering10091015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 07/27/2023] [Accepted: 08/20/2023] [Indexed: 09/29/2023] Open
Abstract
Vision transformers represent the cutting-edge topic in computer vision and are usually employed on two-dimensional data following a transfer learning approach. In this work, we propose a trained-from-scratch stacking ensemble of 3D-vision transformers to assess prostate cancer aggressiveness from T2-weighted images to help radiologists diagnose this disease without performing a biopsy. We trained 18 3D-vision transformers on T2-weighted axial acquisitions and combined them into two- and three-model stacking ensembles. We defined two metrics for measuring model prediction confidence, and we trained all the ensemble combinations according to a five-fold cross-validation, evaluating their accuracy, confidence in predictions, and calibration. In addition, we optimized the 18 base ViTs and compared the best-performing base and ensemble models by re-training them on a 100-sample bootstrapped training set and evaluating each model on the hold-out test set. We compared the two distributions by calculating the median and the 95% confidence interval and performing a Wilcoxon signed-rank test. The best-performing 3D-vision-transformer stacking ensemble provided state-of-the-art results in terms of area under the receiving operating curve (0.89 [0.61-1]) and exceeded the area under the precision-recall curve of the base model of 22% (p < 0.001). However, it resulted to be less confident in classifying the positive class.
Collapse
Affiliation(s)
- Eva Pachetti
- “Alessandro Faedo” Institute of Information Science and Technologies (ISTI), National Research Council of Italy (CNR), 56127 Pisa, Italy;
- Department of Information Engineering (DII), University of Pisa, 56122 Pisa, Italy
| | - Sara Colantonio
- “Alessandro Faedo” Institute of Information Science and Technologies (ISTI), National Research Council of Italy (CNR), 56127 Pisa, Italy;
| |
Collapse
|
10
|
Ichikawa S, Itadani H, Sugimori H. Prediction of body weight from chest radiographs using deep learning with a convolutional neural network. Radiol Phys Technol 2023; 16:127-134. [PMID: 36637719 DOI: 10.1007/s12194-023-00697-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 01/07/2023] [Accepted: 01/09/2023] [Indexed: 01/14/2023]
Abstract
Accurate body weights are not necessarily available in routine clinical practice. This study aimed to investigate whether body weight can be predicted from chest radiographs using deep learning. Deep-learning models with a convolutional neural network (CNN) were trained and tested using chest radiographs from 85,849 patients. The CNN models were evaluated by calculating the mean absolute error (MAE) and Spearman's rank correlation coefficient (ρ). The MAEs of the CNN models were 2.63 kg and 3.35 kg for female and male patients, respectively. The predicted body weight was significantly correlated with the actual body weight (ρ = 0.917, p < 0.001 for females; ρ = 0.915, p < 0.001 for males). The body weight was predicted using chest radiographs by applying deep learning. Our method is potentially useful for radiation dose management, determination of the contrast medium dose, and estimation of the specific absorption rate in patients with unknown body weights.
Collapse
Affiliation(s)
- Shota Ichikawa
- Graduate School of Health Sciences, Hokkaido University, Kita-12, Nishi-5, Kita-ku, Sapporo, 060-0812, Japan.,Department of Radiological Technology, Kurashiki Central Hospital, 1-1-1 Miwa, Kurashiki, Okayama, 710-8602, Japan
| | - Hideki Itadani
- Department of Radiological Technology, Kurashiki Central Hospital, 1-1-1 Miwa, Kurashiki, Okayama, 710-8602, Japan
| | - Hiroyuki Sugimori
- Faculty of Health Sciences, Hokkaido University, Kita-12, Nishi-5, Kita-ku, Sapporo, 060-0812, Japan.
| |
Collapse
|