1
|
Hashimoto F, Onishi Y, Ote K, Tashima H, Yamaya T. Two-step optimization for accelerating deep image prior-based PET image reconstruction. Radiol Phys Technol 2024:10.1007/s12194-024-00831-9. [PMID: 39096446 DOI: 10.1007/s12194-024-00831-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2024] [Revised: 07/25/2024] [Accepted: 07/27/2024] [Indexed: 08/05/2024]
Abstract
Deep learning, particularly convolutional neural networks (CNNs), has advanced positron emission tomography (PET) image reconstruction. However, it requires extensive, high-quality training datasets. Unsupervised learning methods, such as deep image prior (DIP), have shown promise for PET image reconstruction. Although DIP-based PET image reconstruction methods demonstrate superior performance, they involve highly time-consuming calculations. This study proposed a two-step optimization method to accelerate end-to-end DIP-based PET image reconstruction and improve PET image quality. The proposed two-step method comprised a pre-training step using conditional DIP denoising, followed by an end-to-end reconstruction step with fine-tuning. Evaluations using Monte Carlo simulation data demonstrated that the proposed two-step method significantly reduced the computation time and improved the image quality, thereby rendering it a practical and efficient approach for end-to-end DIP-based PET image reconstruction.
Collapse
Affiliation(s)
- Fumio Hashimoto
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan.
- Graduate School of Science and Engineering, Chiba University, 1-33, Yayoicho,Inage-Ku, Chiba, 263-8522, Japan.
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa,Inage-Ku, Chiba, 263-8555, Japan.
| | - Yuya Onishi
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan
| | - Kibo Ote
- Central Research Laboratory, Hamamatsu Photonics K. K, 5000 Hirakuchi, Hamana-Ku, Hamamatsu, 434-8601, Japan
| | - Hideaki Tashima
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa,Inage-Ku, Chiba, 263-8555, Japan
| | - Taiga Yamaya
- Graduate School of Science and Engineering, Chiba University, 1-33, Yayoicho,Inage-Ku, Chiba, 263-8522, Japan
- National Institutes for Quantum Science and Technology, 4-9-1, Anagawa,Inage-Ku, Chiba, 263-8555, Japan
| |
Collapse
|
2
|
Moshe YH, Buchsweiler Y, Teicher M, Artzi M. Handling Missing MRI Data in Brain Tumors Classification Tasks: Usage of Synthetic Images vs. Duplicate Images and Empty Images. J Magn Reson Imaging 2024; 60:561-573. [PMID: 37864370 DOI: 10.1002/jmri.29072] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Revised: 09/30/2023] [Accepted: 10/02/2023] [Indexed: 10/22/2023] Open
Abstract
BACKGROUND Deep-learning is widely used for lesion classification. However, in the clinic patient data often has missing images. PURPOSE To evaluate the use of generated, duplicate and empty(black) images for replacing missing MRI data in AI brain tumor classification tasks. STUDY TYPE Retrospective. POPULATION 224 patients (local-dataset; low-grade-glioma (LGG) = 37, high-grade-glioma (HGG) = 187) and 335 patients (public-dataset (BraTS); LGG = 76, HGG = 259). The local-dataset was divided into training (64), validation (16), and internal-test-data (20), while the public-dataset was an independent test-set. FIELD STRENGTH/SEQUENCE T1WI, T1WI+C, T2WI, and FLAIR images (1.5T/3.0T-MR), obtained from different suppliers. ASSESSMENT Three image-to-image translation generative-adversarial-network (Pix2Pix-GAN) models were trained on the local-dataset, to generate T1WI, T2WI, and FLAIR images. The rating-and-preference-judgment assessment was performed by three human-readers (radiologist (MD) and two MRI-technicians). Resnet152 was used for classification, and inference was performed on both datasets, with baseline input, and with missing data replaced by 1) generated images; 2) duplication of existing images; and 3) black images. STATISTICAL TESTS The similarity between the generated and the original images was evaluated using the peak-signal-to-noise-ratio (PSNR) and the structural-similarity-index-measure (SSIM). Classification results were evaluated using accuracy, F1-score and the Kolmogorov-Smirnov test and distance. RESULTS For baseline-state, the classification model reached to accuracy = 0.93,0.82 on the local and public-datasets. For the missing-data methods, high similarity was obtained between the generated and the original images with mean PSNR = 35.65,32.94 and SSIM = 0.87,0.91 on the local and public-datasets; 39% of the generated-images were labeled as real images by the human-readers. The classification model using generated-images to replace missing images produced the highest results with mean accuracy = 0.91,0.82 compared to 0.85,0.79 for duplicated and 0.77,0.68 for use of black images; DATA CONCLUSION: The feasibility for inference classification model on an MRI dataset with missing images using the Pix2pix-GAN generated images, was shown. The stability and generalization ability of the model was demonstrated by producing consistent results on two independent datasets. LEVEL OF EVIDENCE 3 TECHNICAL EFFICACY: Stage 5.
Collapse
Affiliation(s)
- Yael H Moshe
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel
- Department of Mathematics, Bar Ilan University, Ramat Gan, Israel
| | - Yuval Buchsweiler
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel
- The Iby and Aladar Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv, Israel
| | - Mina Teicher
- Department of Mathematics, Bar Ilan University, Ramat Gan, Israel
- Gonda Brain Research Center, Bar-Ilan University, Ramat Gan, Israel
| | - Moran Artzi
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel
- Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
- Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel
| |
Collapse
|
3
|
Konar D, Bhattacharyya S, Gandhi TK, Panigrahi BK, Jiang R. 3-D Quantum-Inspired Self-Supervised Tensor Network for Volumetric Segmentation of Medical Images. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10312-10325. [PMID: 37022399 DOI: 10.1109/tnnls.2023.3240238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
This article introduces a novel shallow 3-D self-supervised tensor neural network in quantum formalism for volumetric segmentation of medical images with merits of obviating training and supervision. The proposed network is referred to as the 3-D quantum-inspired self-supervised tensor neural network (3-D-QNet). The underlying architecture of 3-D-QNet is composed of a trinity of volumetric layers, viz., input, intermediate, and output layers interconnected using an S -connected third-order neighborhood-based topology for voxelwise processing of 3-D medical image data, suitable for semantic segmentation. Each of the volumetric layers contains quantum neurons designated by qubits or quantum bits. The incorporation of tensor decomposition in quantum formalism leads to faster convergence of network operations to preclude the inherent slow convergence problems faced by the classical supervised and self-supervised networks. The segmented volumes are obtained once the network converges. The suggested 3-D-QNet is tailored and tested on the BRATS 2019 Brain MR image dataset and the Liver Tumor Segmentation Challenge (LiTS17) dataset extensively in our experiments. The 3-D-QNet has achieved promising dice similarity (DS) as compared with the time-intensive supervised convolutional neural network (CNN)-based models, such as 3-D-UNet, voxelwise residual network (VoxResNet), Dense-Res-Inception Net (DRINet), and 3-D-ESPNet, thereby showing a potential advantage of our self-supervised shallow network on facilitating semantic segmentation.
Collapse
|
4
|
Liu W, Wang D, Liu L, Zhou Z. Assessing the Influence of B-US, CDFI, SE, and Patient Age on Predicting Molecular Subtypes in Breast Lesions Using Deep Learning Algorithms. JOURNAL OF ULTRASOUND IN MEDICINE : OFFICIAL JOURNAL OF THE AMERICAN INSTITUTE OF ULTRASOUND IN MEDICINE 2024; 43:1375-1388. [PMID: 38581195 DOI: 10.1002/jum.16460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2023] [Revised: 03/01/2024] [Accepted: 03/25/2024] [Indexed: 04/08/2024]
Abstract
OBJECTIVES Our study aims to investigate the impact of B-mode ultrasound (B-US) imaging, color Doppler flow imaging (CDFI), strain elastography (SE), and patient age on the prediction of molecular subtypes in breast lesions. METHODS Totally 2272 multimodal ultrasound imaging was collected from 198 patients. The ResNet-18 network was employed to predict four molecular subtypes from B-US imaging, CDFI, and SE of patients with different ages. All the images were split into training and testing datasets by the ratio of 80%:20%. The predictive performance on testing dataset was evaluated through 5 metrics including mean accuracy, precision, recall, F1-scores, and confusion matrix. RESULTS Based on B-US imaging, the test mean accuracy is 74.50%, the precision is 74.84%, the recall is 72.48%, and the F1-scores is 0.73. By combining B-US imaging with CDFI, the results were increased to 85.41%, 85.03%, 85.05%, and 0.84, respectively. With the integration of B-US imaging and SE, the results were changed to 75.64%, 74.69%, 73.86%, and 0.74, respectively. Using images from patients under 40 years old, the results were 90.48%, 90.88%, 88.47%, and 0.89. When images from patients who are above 40 years old, they were changed to 81.96%, 83.12%, 80.5%, and 0.81, respectively. CONCLUSION Multimodal ultrasound imaging can be used to accurately predict the molecular subtypes of breast lesions. In addition to B-US imaging, CDFI rather than SE contribute further to improve predictive performance. The predictive performance is notably better for patients under 40 years old compared with those who are 40 years old and above.
Collapse
Affiliation(s)
- Weiyong Liu
- Department of Ultrasound, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Dongyue Wang
- School of Management, Hefei University of Technology, Hefei, China
- Key Laboratory of Process Optimization and Intelligent Decision-Making, Ministry of Education, Hefei, China
- Ministry of Education Engineering Research Center for Intelligent Decision-Making & Information System Technologies, Hefei, China
| | - Le Liu
- Department of Ultrasound, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, China
| | - Zhiguo Zhou
- Reliable Intelligence and Medical Innovation Laboratory (RIMI Lab), Department of Biostatistics & Data Science, University of Kansas Medical Center, Kansas City, Kansas, USA
| |
Collapse
|
5
|
Border SP, Tomaszewski JE, Yoshida T, Kopp JB, Hodgin JB, Clapp WL, Rosenberg AZ, Buyon JP, Sarder P. Investigating quantitative histological characteristics in renal pathology using HistoLens. Sci Rep 2024; 14:17528. [PMID: 39080444 PMCID: PMC11289473 DOI: 10.1038/s41598-024-68406-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/12/2024] [Accepted: 07/23/2024] [Indexed: 08/02/2024] Open
Abstract
HistoLens is an open-source graphical user interface developed using MATLAB AppDesigner for visual and quantitative analysis of histological datasets. HistoLens enables users to interrogate sets of digitally annotated whole slide images to efficiently characterize histological differences between disease and experimental groups. Users can dynamically visualize the distribution of 448 hand-engineered features quantifying color, texture, morphology, and distribution across microanatomic sub-compartments. Additionally, users can map differentially detected image features within the images by highlighting affected regions. We demonstrate the utility of HistoLens to identify hand-engineered features that correlate with pathognomonic renal glomerular characteristics distinguishing diabetic nephropathy and amyloid nephropathy from the histologically unremarkable glomeruli in minimal change disease. Additionally, we examine the use of HistoLens for glomerular feature discovery in the Tg26 mouse model of HIV-associated nephropathy. We identify numerous quantitative glomerular features distinguishing Tg26 transgenic mice from wild-type mice, corresponding to a progressive renal disease phenotype. Thus, we demonstrate an off-the-shelf and ready-to-use toolkit for quantitative renal pathology applications.
Collapse
Affiliation(s)
- Samuel P Border
- Section of Quantitative Health, Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, 1600 SW Archer Rd., Gainesville, FL, 32608, USA
| | - John E Tomaszewski
- Department of Pathology & Anatomical Sciences, University at Buffalo, Buffalo, NY, USA
| | - Teruhiko Yoshida
- Kidney Disease Section, Kidney Diseases Branch, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, MD, USA
| | - Jeffrey B Kopp
- Kidney Disease Section, Kidney Diseases Branch, National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, MD, USA
| | - Jeffrey B Hodgin
- Department of Pathology, University of Michigan, Ann Arbor, MI, USA
| | - William L Clapp
- Department of Pathology, Immunology and Laboratory Medicine, University of Florida, Gainesville, FL, USA
| | - Avi Z Rosenberg
- Department of Pathology, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Jill P Buyon
- New York University Grossman School of Medicine, New York, NY, USA
| | - Pinaki Sarder
- Section of Quantitative Health, Division of Nephrology, Hypertension, and Renal Transplantation, Department of Medicine, University of Florida, 1600 SW Archer Rd., Gainesville, FL, 32608, USA.
| |
Collapse
|
6
|
Han K, Ryu CH, Lee CL, Han TH. Deep learning-based material decomposition of iodine and calcium in mobile photon counting detector CT. PLoS One 2024; 19:e0306627. [PMID: 39058758 PMCID: PMC11280148 DOI: 10.1371/journal.pone.0306627] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Accepted: 06/20/2024] [Indexed: 07/28/2024] Open
Abstract
Photon-counting detector (PCD)-based computed tomography (CT) offers several advantages over conventional energy-integrating detector-based CT. Among them, the ability to discriminate energy exhibits significant potential for clinical applications because it provides material-specific information. That is, material decomposition (MD) can be achieved through energy discrimination. In this study, deep learning-based material decomposition was performed using live animal data. We propose MD-Unet, which is a deep learning strategy for material decomposition based on an Unet architecture trained with data from three energy bins. To mitigate the data insufficiency, we developed a pretrained model incorporating various simulation data forms and augmentation strategies. Incorporating these approaches into model training results in enhanced precision in material decomposition, thereby enabling the identification of distinct materials at individual pixel locations. The trained network was applied to the acquired animal data to evaluate material decomposition results. Compared with conventional methods, the newly generated MD-Unet demonstrated more accurate material decomposition imaging. Moreover, the network demonstrated an improved material decomposition ability and significantly reduced noise. In addition, they can potentially offer an enhancement level similar to that of a typical contrast agent. This implies that it can acquire images of the same quality with fewer contrast agents administered to patients, thereby demonstrating its significant clinical value.
Collapse
Affiliation(s)
- Kwanhee Han
- Health & Medical Equipment Business Unit, Samsung Electronics, Suwon-si, Gyeonggi-do, Korea
- Department of Digital Media and Communications Engineering, Sungkyunkwan University, Suwon-si, Gyeonggi-do, Korea
| | - Chang Ho Ryu
- Department of Artificial Intelligence, Sungkyunkwan University, Suwon-si, Gyeonggi-do, Korea
| | - Chang-Lae Lee
- Health & Medical Equipment Business Unit, Samsung Electronics, Suwon-si, Gyeonggi-do, Korea
| | - Tae Hee Han
- Department of Semiconductor Systems Engineering, Sungkyunkwan University, Suwon-si, Gyeonggi-do, Korea
| |
Collapse
|
7
|
Getzmann JM, Zantonelli G, Messina C, Albano D, Serpi F, Gitto S, Sconfienza LM. The use of artificial intelligence in musculoskeletal ultrasound: a systematic review of the literature. LA RADIOLOGIA MEDICA 2024:10.1007/s11547-024-01856-1. [PMID: 39001961 DOI: 10.1007/s11547-024-01856-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/27/2024] [Accepted: 07/04/2024] [Indexed: 07/15/2024]
Abstract
PURPOSE To systematically review the use of artificial intelligence (AI) in musculoskeletal (MSK) ultrasound (US) with an emphasis on AI algorithm categories and validation strategies. MATERIAL AND METHODS An electronic literature search was conducted for articles published up to January 2024. Inclusion criteria were the use of AI in MSK US, involvement of humans, English language, and ethics committee approval. RESULTS Out of 269 identified papers, 16 studies published between 2020 and 2023 were included. The research was aimed at predicting diagnosis and/or segmentation in a total of 11 (69%) out of 16 studies. A total of 11 (69%) studies used deep learning (DL)-based algorithms, three (19%) studies employed conventional machine learning (ML)-based algorithms, and two (12%) studies employed both conventional ML- and DL-based algorithms. Six (38%) studies used cross-validation techniques with K-fold cross-validation being the most frequently employed (n = 4, 25%). Clinical validation with separate internal test datasets was reported in nine (56%) papers. No external clinical validation was reported. CONCLUSION AI is a topic of increasing interest in MSK US research. In future studies, attention should be paid to the use of validation strategies, particularly regarding independent clinical validation performed on external datasets.
Collapse
Affiliation(s)
| | - Giulia Zantonelli
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| | - Carmelo Messina
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
- UOC Radiodiagnostica, ASST Centro Specialistico Ortopedico Traumatologico Gaetano Pini-CTO, Milan, Italy
| | - Domenico Albano
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
- Dipartimento Di Scienze Biomediche, Chirurgiche Ed Odontoiatriche, Università Degli Studi Di Milano, Milan, Italy
| | | | - Salvatore Gitto
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy.
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy.
| | - Luca Maria Sconfienza
- IRCCS Istituto Ortopedico Galeazzi, Milan, Italy
- Dipartimento Di Scienze Biomediche Per La Salute, Università Degli Studi Di Milano, Milan, Italy
| |
Collapse
|
8
|
Hao J, Chen S. Language-aware multiple datasets detection pretraining for DETRs. Neural Netw 2024; 179:106506. [PMID: 38996689 DOI: 10.1016/j.neunet.2024.106506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2024] [Revised: 05/17/2024] [Accepted: 07/02/2024] [Indexed: 07/14/2024]
Abstract
Pretraining on large-scale datasets can boost the performance of object detectors while the annotated datasets for object detection are hard to scale up due to the high labor cost. What we possess are numerous isolated filed-specific datasets, thus, it is appealing to jointly pretrain models across aggregation of datasets to enhance data volume and diversity. In this paper, we propose a strong framework for utilizing Multiple datasets to pretrain DETR-like detectors, termed METR, without the need for manual label spaces integration. It converts the typical multi-classification in object detection into binary classification by introducing a pre-trained language model. Specifically, we design a category extraction module for extracting potential categories involved in an image and assign these categories into different queries by language embeddings. Each query is only responsible for predicting a class-specific object. Besides, to adapt our novel detection paradigm, we propose a Class-wise Bipartite Matching strategy that limits the ground truths to match queries assigned to the same category. Extensive experiments demonstrate that METR achieves extraordinary results on either multi-task joint training or the pretrain & finetune paradigm. Notably, our pre-trained models have high flexible transferability and increase the performance upon various DETR-like detectors on COCO val2017 benchmark. Our code is publicly available at: https://github.com/isbrycee/METR.
Collapse
Affiliation(s)
- Jing Hao
- VIS, Baidu Inc., Beijing, 100000, China.
| | - Song Chen
- VIS, Baidu Inc., Beijing, 100000, China.
| |
Collapse
|
9
|
Feng R, Li S, Zhang Y. AI-powered microscopy image analysis for parasitology: integrating human expertise. Trends Parasitol 2024; 40:633-646. [PMID: 38824067 DOI: 10.1016/j.pt.2024.05.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2024] [Revised: 05/06/2024] [Accepted: 05/07/2024] [Indexed: 06/03/2024]
Abstract
Microscopy image analysis plays a pivotal role in parasitology research. Deep learning (DL), a subset of artificial intelligence (AI), has garnered significant attention. However, traditional DL-based methods for general purposes are data-driven, often lacking explainability due to their black-box nature and sparse instructional resources. To address these challenges, this article presents a comprehensive review of recent advancements in knowledge-integrated DL models tailored for microscopy image analysis in parasitology. The massive amounts of human expert knowledge from parasitologists can enhance the accuracy and explainability of AI-driven decisions. It is expected that the adoption of knowledge-integrated DL models will open up a wide range of applications in the field of parasitology.
Collapse
Affiliation(s)
- Ruijun Feng
- College of Science, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, China; School of Computer Science and Engineering, University of New South Wales, Sydney, Australia
| | - Sen Li
- College of Science, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, China
| | - Yang Zhang
- College of Science, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055, China.
| |
Collapse
|
10
|
Wang G, Jiang N, Ma Y, Chen D, Wu J, Li G, Liang D, Yan T. Connectional-style-guided contextual representation learning for brain disease diagnosis. Neural Netw 2024; 175:106296. [PMID: 38653077 DOI: 10.1016/j.neunet.2024.106296] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2023] [Revised: 01/26/2024] [Accepted: 04/06/2024] [Indexed: 04/25/2024]
Abstract
Structural magnetic resonance imaging (sMRI) has shown great clinical value and has been widely used in deep learning (DL) based computer-aided brain disease diagnosis. Previous DL-based approaches focused on local shapes and textures in brain sMRI that may be significant only within a particular domain. The learned representations are likely to contain spurious information and have poor generalization ability in other diseases and datasets. To facilitate capturing meaningful and robust features, it is necessary to first comprehensively understand the intrinsic pattern of the brain that is not restricted within a single data/task domain. Considering that the brain is a complex connectome of interlinked neurons, the connectional properties in the brain have strong biological significance, which is shared across multiple domains and covers most pathological information. In this work, we propose a connectional style contextual representation learning model (CS-CRL) to capture the intrinsic pattern of the brain, used for multiple brain disease diagnosis. Specifically, it has a vision transformer (ViT) encoder and leverages mask reconstruction as the proxy task and Gram matrices to guide the representation of connectional information. It facilitates the capture of global context and the aggregation of features with biological plausibility. The results indicate that CS-CRL achieves superior accuracy in multiple brain disease diagnosis tasks across six datasets and three diseases and outperforms state-of-the-art models. Furthermore, we demonstrate that CS-CRL captures more brain-network-like properties, and better aggregates features, is easier to optimize, and is more robust to noise, which explains its superiority in theory.
Collapse
Affiliation(s)
- Gongshu Wang
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| | - Ning Jiang
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| | - Yunxiao Ma
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| | - Duanduan Chen
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| | - Jinglong Wu
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| | - Guoqi Li
- Institute of Automation, Chinese Academy of Sciences, Beijing, China.
| | - Dong Liang
- Research Center for Medical AI, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
| | - Tianyi Yan
- School of Medical Technology, Beijing Institute of Technology, Beijing, China.
| |
Collapse
|
11
|
Yuan G, Cai L, Qu W, Zhou Z, Liang P, Chen J, Xu C, Zhang J, Wang S, Chu Q, Li Z. Identification of Calculous Pyonephrosis by CT-Based Radiomics and Deep Learning. Bioengineering (Basel) 2024; 11:662. [PMID: 39061744 PMCID: PMC11274102 DOI: 10.3390/bioengineering11070662] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2024] [Revised: 06/18/2024] [Accepted: 06/22/2024] [Indexed: 07/28/2024] Open
Abstract
Urgent detection of calculous pyonephrosis is crucial for surgical planning and preventing severe outcomes. This study aims to evaluate the performance of computed tomography (CT)-based radiomics and a three-dimensional convolutional neural network (3D-CNN) model, integrated with independent clinical factors, to identify patients with calculous pyonephrosis. We recruited 182 patients receiving either percutaneous nephrostomy tube placement or percutaneous nephrolithotomy for calculous hydronephrosis or pyonephrosis. The regions of interest were manually delineated on plain CT images and the CT attenuation value (HU) was measured. Radiomics analysis was performed using least absolute shrinkage and selection operator (LASSO). A 3D-CNN model was also developed. The better-performing machine-learning model was combined with independent clinical factors to build a comprehensive clinical machine-learning model. The performance of these models was assessed using receiver operating characteristic analysis and decision curve analysis. Fever, blood neutrophils, and urine leukocytes were independent risk factors for pyonephrosis. The radiomics model showed higher area under the curve (AUC) than the 3D-CNN model and HU (0.876 vs. 0.599, 0.578; p = 0.003, 0.002) in the testing cohort. The clinical machine-learning model surpassed the clinical model in both the training (0.975 vs. 0.904, p = 0.019) and testing (0.967 vs. 0.889, p = 0.045) cohorts.
Collapse
Affiliation(s)
- Guanjie Yuan
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Lingli Cai
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Weinuo Qu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Ziling Zhou
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Ping Liang
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Jun Chen
- Bayer Healthcare, Wuhan 430000, China;
| | - Chuou Xu
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| | - Jiaqiao Zhang
- Department of Urology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China;
| | - Shaogang Wang
- Department of Urology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China;
| | - Qian Chu
- Department of Oncology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China;
| | - Zhen Li
- Department of Radiology, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan 430030, China; (G.Y.); (L.C.); (W.Q.); (Z.Z.); (P.L.); (Z.L.)
| |
Collapse
|
12
|
Chang J, Hatfield B. Advancements in computer vision and pathology: Unraveling the potential of artificial intelligence for precision diagnosis and beyond. Adv Cancer Res 2024; 161:431-478. [PMID: 39032956 DOI: 10.1016/bs.acr.2024.05.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/23/2024]
Abstract
The integration of computer vision into pathology through slide digitalization represents a transformative leap in the field's evolution. Traditional pathology methods, while reliable, are often time-consuming and susceptible to intra- and interobserver variability. In contrast, computer vision, empowered by artificial intelligence (AI) and machine learning (ML), promises revolutionary changes, offering consistent, reproducible, and objective results with ever-increasing speed and scalability. The applications of advanced algorithms and deep learning architectures like CNNs and U-Nets augment pathologists' diagnostic capabilities, opening new frontiers in automated image analysis. As these technologies mature and integrate into digital pathology workflows, they are poised to provide deeper insights into disease processes, quantify and standardize biomarkers, enhance patient outcomes, and automate routine tasks, reducing pathologists' workload. However, this transformative force calls for cross-disciplinary collaboration between pathologists, computer scientists, and industry innovators to drive research and development. While acknowledging its potential, this chapter addresses the limitations of AI in pathology, encompassing technical, practical, and ethical considerations during development and implementation.
Collapse
Affiliation(s)
- Justin Chang
- Virginia Commonwealth University Health System, Richmond, VA, United States
| | - Bryce Hatfield
- Virginia Commonwealth University Health System, Richmond, VA, United States.
| |
Collapse
|
13
|
Ren G, Kumar A, Mahmoud SS, Fang Q. A deep neural network and transfer learning combined method for cross-task classification of error-related potentials. Front Hum Neurosci 2024; 18:1394107. [PMID: 38933146 PMCID: PMC11199896 DOI: 10.3389/fnhum.2024.1394107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2024] [Accepted: 05/22/2024] [Indexed: 06/28/2024] Open
Abstract
Background Error-related potentials (ErrPs) are electrophysiological responses that naturally occur when humans perceive wrongdoing or encounter unexpected events. It offers a distinctive means of comprehending the error-processing mechanisms within the brain. A method for detecting ErrPs with high accuracy holds significant importance for various ErrPs-based applications, such as human-in-the-loop Brain-Computer Interface (BCI) systems. Nevertheless, current methods fail to fulfill the generalization requirements for detecting such ErrPs due to the high non-stationarity of EEG signals across different tasks and the limited availability of ErrPs datasets. Methods This study introduces a deep learning-based model that integrates convolutional layers and transformer encoders for the classification of ErrPs. Subsequently, a model training strategy, grounded in transfer learning, is proposed for the effective training of the model. The datasets utilized in this study are available for download from the publicly accessible databases. Results In cross-task classification, an average accuracy of about 78% was achieved, exceeding the baseline. Furthermore, in the leave-one-subject-out, within-session, and cross-session classification scenarios, the proposed model outperformed the existing techniques with an average accuracy of 71.81, 78.74, and 77.01%, respectively. Conclusions Our approach contributes to mitigating the challenge posed by limited datasets in the ErrPs field, achieving this by reducing the requirement for extensive training data for specific target tasks. This may serve as inspiration for future studies that concentrate on ErrPs and their applications.
Collapse
Affiliation(s)
| | | | | | - Qiang Fang
- Department of Biomedical Engineering, Shantou University, Shantou, China
| |
Collapse
|
14
|
Pérez Cáceres M, Gauvin A, Dumais F, Iorio-Morin C. A Practical Roadmap to Implementing Deep Learning Segmentation in the Clinical Neuroimaging Research Workflow. World Neurosurg 2024; 189:193-200. [PMID: 38866234 DOI: 10.1016/j.wneu.2024.06.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/14/2024] [Revised: 06/04/2024] [Accepted: 06/05/2024] [Indexed: 06/14/2024]
Abstract
BACKGROUND Thanks to the proliferation of open-source tools, we are seeing an exponential growth of machine-learning applications, and its integration has become more accessible, particularly for segmentation tools in neuroimaging. METHODS This article explores a generalized methodology that harnesses these tools and aims/enables to expedite and enhance the reproducibility of clinical research. Herein, critical considerations include hardware, software, neural network training strategies, and data labeling guidelines. More specifically, we advocate an iterative approach to model training and transfer learning, focusing on internal validation and outlier handling early in the labeling process and fine-tuning later. RESULTS The iterative refinement process allows experts to intervene and improve model reliability while cutting down on their time spent in manual work. A seamless integration of the final model's predictions into clinical research is proposed to ensure standardized and reproducible results. CONCLUSIONS In short, this article provides a comprehensive framework for accelerating research using machine-learning techniques for image segmentation.
Collapse
Affiliation(s)
| | - Alexandre Gauvin
- Department of Neurosurgery, CHUS (Centre Hôspitalier de L'Université de Sherbrooke) Fleurimont Hospital, Montréal, Québec, Canada
| | - Félix Dumais
- Computer Science Department, University of Sherbrooke, Montréal, Québec, Canada
| | - Christian Iorio-Morin
- Department of Neurosurgery, CHUS (Centre Hôspitalier de L'Université de Sherbrooke) Fleurimont Hospital, Montréal, Québec, Canada
| |
Collapse
|
15
|
Qi L, Jiang Z, Shi W, Qu F, Feng G. GMIM: Self-supervised pre-training for 3D medical image segmentation with adaptive and hierarchical masked image modeling. Comput Biol Med 2024; 176:108547. [PMID: 38728994 DOI: 10.1016/j.compbiomed.2024.108547] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2023] [Revised: 04/07/2024] [Accepted: 04/28/2024] [Indexed: 05/12/2024]
Abstract
Self-supervised pre-training and fully supervised fine-tuning paradigms have received much attention to solve the data annotation problem in deep learning fields. Compared with traditional pre-training on large natural image datasets, medical self-supervised learning methods learn rich representations derived from unlabeled data itself thus avoiding the distribution shift between different image domains. However, nowadays state-of-the-art medical pre-training methods were specifically designed for downstream tasks making them less flexible and difficult to apply to new tasks. In this paper, we propose grid mask image modeling, a flexible and general self-supervised method to pre-train medical vision transformers for 3D medical image segmentation. Our goal is to guide networks to learn the correlations between organs and tissues by reconstructing original images based on partial observations. The relationships are consistent within the human body and invariant to disease type or imaging modality. To achieve this, we design a Siamese framework consisting of an online branch and a target branch. An adaptive and hierarchical masking strategy is employed in the online branch to (1) learn the boundaries or small contextual mutation regions within images; (2) to learn high-level semantic representations from deeper layers of the multiscale encoder. In addition, the target branch provides representations for contrastive learning to further reduce representation redundancy. We evaluate our method through segmentation performance on two public datasets. The experimental results demonstrate our method outperforms other self-supervised methods. Codes are available at https://github.com/mobiletomb/Gmim.
Collapse
Affiliation(s)
- Liangce Qi
- Department of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China.
| | - Zhengang Jiang
- Department of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528400, Guangzhou, China.
| | - Weili Shi
- Department of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China; Zhongshan Institute of Changchun University of Science and Technology, Zhongshan, 528400, Guangzhou, China
| | - Feng Qu
- Department of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China
| | - Guanyuan Feng
- Department of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China
| |
Collapse
|
16
|
Yue Y, Li N, Zhang G, Xing W, Zhu Z, Liu X, Song S, Ta D. A transformer-guided cross-modality adaptive feature fusion framework for esophageal gross tumor volume segmentation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 251:108216. [PMID: 38761412 DOI: 10.1016/j.cmpb.2024.108216] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 04/17/2024] [Accepted: 05/06/2024] [Indexed: 05/20/2024]
Abstract
BACKGROUND AND OBJECTIVE Accurate segmentation of esophageal gross tumor volume (GTV) indirectly enhances the efficacy of radiotherapy for patients with esophagus cancer. In this domain, learning-based methods have been employed to fuse cross-modality positron emission tomography (PET) and computed tomography (CT) images, aiming to improve segmentation accuracy. This fusion is essential as it combines functional metabolic information from PET with anatomical information from CT, providing complementary information. While the existing three-dimensional (3D) segmentation method has achieved state-of-the-art (SOTA) performance, it typically relies on pure-convolution architectures, limiting its ability to capture long-range spatial dependencies due to convolution's confinement to a local receptive field. To address this limitation and further enhance esophageal GTV segmentation performance, this work proposes a transformer-guided cross-modality adaptive feature fusion network, referred to as TransAttPSNN, which is based on cross-modality PET/CT scans. METHODS Specifically, we establish an attention progressive semantically-nested network (AttPSNN) by incorporating the convolutional attention mechanism into the progressive semantically-nested network (PSNN). Subsequently, we devise a plug-and-play transformer-guided cross-modality adaptive feature fusion model, which is inserted between the multi-scale feature counterparts of a two-stream AttPSNN backbone (one for the PET modality flow and another for the CT modality flow), resulting in the proposed TransAttPSNN architecture. RESULTS Through extensive four-fold cross-validation experiments on the clinical PET/CT cohort. The proposed approach acquires a Dice similarity coefficient (DSC) of 0.76 ± 0.13, a Hausdorff distance (HD) of 9.38 ± 8.76 mm, and a Mean surface distance (MSD) of 1.13 ± 0.94 mm, outperforming the SOTA competing methods. The qualitative results show a satisfying consistency with the lesion areas. CONCLUSIONS The devised transformer-guided cross-modality adaptive feature fusion module integrates the strengths of PET and CT, effectively enhancing the segmentation performance of esophageal GTV. The proposed TransAttPSNN has further advanced the research of esophageal GTV segmentation.
Collapse
Affiliation(s)
- Yaoting Yue
- Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai 200438, PR China
| | - Nan Li
- Department of Nuclear Medicine, Fudan University Shanghai Cancer Center, Shanghai 201321, PR China
| | - Gaobo Zhang
- Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai 200438, PR China
| | - Wenyu Xing
- Academy for Engineering and Technology, Fudan University, Shanghai 200433, PR China
| | - Zhibin Zhu
- Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai 200438, PR China; School of Physics and Electromechanical Engineering, Hexi University, Zhangye 734000, Gansu, PR China
| | - Xin Liu
- Academy for Engineering and Technology, Fudan University, Shanghai 200433, PR China.
| | - Shaoli Song
- Department of Nuclear Medicine, Fudan University Shanghai Cancer Center, Shanghai 201321, PR China.
| | - Dean Ta
- Department of Biomedical Engineering, School of Information Science and Technology, Fudan University, Shanghai 200438, PR China; Academy for Engineering and Technology, Fudan University, Shanghai 200433, PR China.
| |
Collapse
|
17
|
Rousta F, Esteki A, Shalbaf A, Sadeghi A, Moghadam PK, Voshagh A. Application of artificial intelligence in pancreas endoscopic ultrasound imaging- A systematic review. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 250:108205. [PMID: 38703435 DOI: 10.1016/j.cmpb.2024.108205] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/04/2024] [Revised: 04/13/2024] [Accepted: 04/24/2024] [Indexed: 05/06/2024]
Abstract
The pancreas is a vital organ in digestive system which has significant health implications. It is imperative to evaluate and identify malignant pancreatic lesions promptly in light of the high mortality rate linked to such malignancies. Endoscopic Ultrasound (EUS) is a non-invasive precise technique to detect pancreas disorders, but it is highly operator dependent. Artificial intelligence (AI), including traditional machine learning (ML) and deep learning (DL) techniques can play a pivotal role to enhancing the performance of EUS regardless of operator. AI performs a critical function in the detection, classification, and segmentation of medical images. The utilization of AI-assisted systems has improved the accuracy and productivity of pancreatic analysis, including the detection of diverse pancreatic disorders (e.g., pancreatitis, masses, and cysts) as well as landmarks and parenchyma. This systematic review examines the rapidly developing domain of AI-assisted system in EUS of the pancreas. Its objective is to present a thorough study of the present research status and developments in this area. This paper explores the significant challenges of AI-assisted system in pancreas EUS imaging, highlights the potential of AI techniques in addressing these challenges, and suggests the scope for future research in domain of AI-assisted EUS systems.
Collapse
Affiliation(s)
- Fatemeh Rousta
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ali Esteki
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ahmad Shalbaf
- Department of Biomedical Engineering and Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
| | - Amir Sadeghi
- Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Pardis Ketabi Moghadam
- Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Ardalan Voshagh
- Faculty of Electrical Engineering, Shahid Beheshti University, Tehran, Iran
| |
Collapse
|
18
|
Mavridis C, Economopoulos TL, Benetos G, Matsopoulos GK. Aorta Segmentation in 3D CT Images by Combining Image Processing and Machine Learning Techniques. Cardiovasc Eng Technol 2024; 15:359-373. [PMID: 38388764 DOI: 10.1007/s13239-024-00720-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2023] [Accepted: 01/30/2024] [Indexed: 02/24/2024]
Abstract
PURPOSE Aorta segmentation is extremely useful in clinical practice, allowing the diagnosis of numerous pathologies, such as dissections, aneurysms and occlusive disease. In such cases, image segmentation is prerequisite for applying diagnostic algorithms, which in turn allow the prediction of possible complications and enable risk assessment, which is crucial in saving lives. The aim of this paper is to present a novel fully automatic 3D segmentation method, which combines basic image processing techniques and more advanced machine learning algorithms, for detecting and modelling the aorta in 3D CT imaging data. METHODS An initial intensity threshold-based segmentation procedure is followed by a classification-based segmentation approach, based on a Markov Random Field network. The result of the proposed two-stage segmentation process is modelled and visualized. RESULTS The proposed methodology was applied to 16 3D CT data sets and the extracted aortic segments were reconstructed as 3D models. The performance of segmentation was evaluated both qualitatively and quantitatively against other commonly used segmentation techniques, in terms of the accuracy achieved, compared to the actual aorta, which was defined manually by experts. CONCLUSION The proposed methodology achieved superior segmentation performance, compared to all compared segmentation techniques, in terms of the accuracy of the extracted 3D aortic model. Therefore, the proposed segmentation scheme could be used in clinical practice, such as in treatment planning and assessment, as it can speed up the evaluation of the medical imaging data, which is commonly a lengthy and tedious process.
Collapse
Affiliation(s)
- Christos Mavridis
- Department of Electrical and Computer Engineering, National Technical University of Athens, 15780, Athens, Greece.
| | - Theodore L Economopoulos
- Department of Electrical and Computer Engineering, National Technical University of Athens, 15780, Athens, Greece
| | - Georgios Benetos
- Department of CT and MRI, Lefkos Stavros Clinic, 11528, Athens, Greece
| | - George K Matsopoulos
- Department of Electrical and Computer Engineering, National Technical University of Athens, 15780, Athens, Greece
| |
Collapse
|
19
|
Zhou Z, Islam MT, Xing L. Multibranch CNN With MLP-Mixer-Based Feature Exploration for High-Performance Disease Diagnosis. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:7351-7362. [PMID: 37028335 DOI: 10.1109/tnnls.2023.3250490] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Deep learning-based diagnosis is becoming an indispensable part of modern healthcare. For high-performance diagnosis, the optimal design of deep neural networks (DNNs) is a prerequisite. Despite its success in image analysis, existing supervised DNNs based on convolutional layers often suffer from their rudimentary feature exploration ability caused by the limited receptive field and biased feature extraction of conventional convolutional neural networks (CNNs), which compromises the network performance. Here, we propose a novel feature exploration network named manifold embedded multilayer perceptron (MLP) mixer (ME-Mixer), which utilizes both supervised and unsupervised features for disease diagnosis. In the proposed approach, a manifold embedding network is employed to extract class-discriminative features; then, two MLP-Mixer-based feature projectors are adopted to encode the extracted features with the global reception field. Our ME-Mixer network is quite general and can be added as a plugin to any existing CNN. Comprehensive evaluations on two medical datasets are performed. The results demonstrate that their approach greatly enhances the classification accuracy in comparison with different configurations of DNNs with acceptable computational complexity.
Collapse
|
20
|
Xu H, Li C, Zhang L, Ding Z, Lu T, Hu H. Immunotherapy efficacy prediction through a feature re-calibrated 2.5D neural network. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 249:108135. [PMID: 38569256 DOI: 10.1016/j.cmpb.2024.108135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 03/11/2024] [Accepted: 03/13/2024] [Indexed: 04/05/2024]
Abstract
BACKGROUND AND OBJECTIVE Lung cancer continues to be a leading cause of cancer-related mortality worldwide, with immunotherapy emerging as a promising therapeutic strategy for advanced non-small cell lung cancer (NSCLC). Despite its potential, not all patients experience benefits from immunotherapy, and the current biomarkers used for treatment selection possess inherent limitations. As a result, the implementation of imaging-based biomarkers to predict the efficacy of lung cancer treatments offers a promising avenue for improving therapeutic outcomes. METHODS This study presents an automatic system for immunotherapy efficacy prediction on the subjects with lung cancer, facilitating significant clinical implications. Our model employs an advanced 2.5D neural network that incorporates 2D intra-slice feature extraction and 3D inter-slice feature aggregation. We further present a lesion-focused prior to guide the re-calibration for intra-slice features, and a attention-based re-calibration for the inter-slice features. Finally, we design an accumulated back-propagation strategy to optimize network parameters in a memory-efficient fashion. RESULTS We demonstrate that the proposed method achieves impressive performance on an in-house clinical dataset, surpassing existing state-of-the-art models. Furthermore, the proposed model exhibits increased efficiency in inference for each subject on average. To further validate the effectiveness of our model and its components, we conducted comprehensive and in-depth ablation experiments and discussions. CONCLUSION The proposed model showcases the potential to enhance physicians' diagnostic performance due to its impressive performance in predicting immunotherapy efficacy, thereby offering significant clinical application value. Moreover, we conduct adequate comparison experiments of the proposed methods and existing advanced models. These findings contribute to our understanding of the proposed model's effectiveness and serve as motivation for future work in immunotherapy efficacy prediction.
Collapse
Affiliation(s)
- Haipeng Xu
- Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fujian 350014, China.
| | - Chenxin Li
- Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong 999077, SAR, China.
| | - Longfeng Zhang
- Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fujian 350014, China.
| | - Zhiyuan Ding
- School of Informatics, Xiamen University, Fujian 350014, China.
| | - Tao Lu
- Department of Radiology, Fujian Medical University Cancer Hospital and Fujian Cancer Hospital, Fujian 350014, China.
| | - Huihua Hu
- Clinical Oncology School of Fujian Medical University, Fujian Cancer Hospital, Fujian 350014, China.
| |
Collapse
|
21
|
Joo J, Kim SY, Kim D, Lee JE, Lee SM, Suh SY, Kim SJ, Kim SC. Enhancing automated strabismus classification with limited data: Data augmentation using StyleGAN2-ADA. PLoS One 2024; 19:e0303355. [PMID: 38787813 PMCID: PMC11125505 DOI: 10.1371/journal.pone.0303355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 04/24/2024] [Indexed: 05/26/2024] Open
Abstract
In this study, we propose a generative data augmentation technique to overcome the challenges of severely limited data when designing a deep learning-based automated strabismus diagnosis system. We implement a generative model based on the StyleGAN2-ADA model for system design and assess strabismus classification performance using two classifiers. We evaluate the capability of our proposed method against traditional data augmentation techniques and confirm a substantial enhancement in performance. Furthermore, we conduct experiments to explore the relationship between the diagnosis agreement among ophthalmologists and the generation performance of the generative model. Beyond FID, we validate the generative samples on the classifier to establish their practicality. Through these experiments, we demonstrate that the generative model-based data augmentation improves overall quantitative performance in scenarios of extreme data scarcity and effectively mitigates overfitting issues during deep learning model training.
Collapse
Affiliation(s)
- Jaehan Joo
- Department of Electronics Engineering, Pusan National University, Busan, Republic of Korea
| | - Sang Yoon Kim
- Department of Ophthalmology, Pusan National University School of Medicine and Research Institute for Convergence of Biomedical Science and Technology Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Donghwan Kim
- Department of Electronics Engineering, Pusan National University, Busan, Republic of Korea
| | - Ji-Eun Lee
- Department of Ophthalmology, Pusan National University School of Medicine and Research Institute for Convergence of Biomedical Science and Technology Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Seung Min Lee
- Department of Ophthalmology, Pusan National University School of Medicine and Research Institute for Convergence of Biomedical Science and Technology Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Su Youn Suh
- Department of Ophthalmology, Pusan National University School of Medicine and Research Institute for Convergence of Biomedical Science and Technology Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Su-Jin Kim
- Department of Ophthalmology, Pusan National University School of Medicine and Research Institute for Convergence of Biomedical Science and Technology Pusan National University Yangsan Hospital, Yangsan, Republic of Korea
| | - Suk Chan Kim
- Department of Electronics Engineering, Pusan National University, Busan, Republic of Korea
| |
Collapse
|
22
|
Liu H, Hu J, Zhou J, Yu R. Application of deep learning to pressure injury staging. J Wound Care 2024; 33:368-378. [PMID: 38683775 DOI: 10.12968/jowc.2024.33.5.368] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
OBJECTIVE Accurate assessment of pressure injuries (PIs) is necessary for a good outcome. Junior and non-specialist nurses have less experience with PIs and lack clinical practice, and so have difficulty staging them accurately. In this work, a deep learning-based system for PI staging and tissue classification is proposed to help improve its accuracy and efficiency in clinical practice, and save healthcare costs. METHOD A total of 1610 cases of PI and their corresponding photographs were collected from clinical practice, and each sample was accurately staged and the tissues labelled by experts for training a Mask Region-based Convolutional Neural Network (Mask R-CNN, Facebook Artificial Intelligence Research, Meta, US) object detection and instance segmentation network. A recognition system was set up to automatically stage and classify the tissues of the remotely uploaded PI photographs. RESULTS On a test set of 100 samples, the average precision of this model for stage recognition reached 0.603, which exceeded that of the medical personnel involved in the comparative evaluation, including an enterostomal therapist. CONCLUSION In this study, the deep learning-based PI staging system achieved the evaluation performance of a nurse with professional training in wound care. This low-cost system could help overcome the difficulty of identifying PIs by junior and non-specialist nurses, and provide valuable auxiliary clinical information.
Collapse
Affiliation(s)
- Han Liu
- Jiulongpo District People's Hospital, Chongqing, China
| | - Juan Hu
- The First Affiliated Hospital of Chongqing Medical and Pharmaceutical College, Chongqing, China
| | | | - Rong Yu
- Shulan Hospital, Hangzhou, China
| |
Collapse
|
23
|
Daneshpajooh V, Ahmad D, Toth J, Bascom R, Higgins WE. Automatic lesion detection for narrow-band imaging bronchoscopy. J Med Imaging (Bellingham) 2024; 11:036002. [PMID: 38827776 PMCID: PMC11138083 DOI: 10.1117/1.jmi.11.3.036002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Revised: 04/04/2024] [Accepted: 05/14/2024] [Indexed: 06/05/2024] Open
Abstract
Purpose Early detection of cancer is crucial for lung cancer patients, as it determines disease prognosis. Lung cancer typically starts as bronchial lesions along the airway walls. Recent research has indicated that narrow-band imaging (NBI) bronchoscopy enables more effective bronchial lesion detection than other bronchoscopic modalities. Unfortunately, NBI video can be hard to interpret because physicians currently are forced to perform a time-consuming subjective visual search to detect bronchial lesions in a long airway-exam video. As a result, NBI bronchoscopy is not regularly used in practice. To alleviate this problem, we propose an automatic two-stage real-time method for bronchial lesion detection in NBI video and perform a first-of-its-kind pilot study of the method using NBI airway exam video collected at our institution. Approach Given a patient's NBI video, the first method stage entails a deep-learning-based object detection network coupled with a multiframe abnormality measure to locate candidate lesions on each video frame. The second method stage then draws upon a Siamese network and a Kalman filter to track candidate lesions over multiple frames to arrive at final lesion decisions. Results Tests drawing on 23 patient NBI airway exam videos indicate that the method can process an incoming video stream at a real-time frame rate, thereby making the method viable for real-time inspection during a live bronchoscopic airway exam. Furthermore, our studies showed a 93% sensitivity and 86% specificity for lesion detection; this compares favorably to a sensitivity and specificity of 80% and 84% achieved over a series of recent pooled clinical studies using the current time-consuming subjective clinical approach. Conclusion The method shows potential for robust lesion detection in NBI video at a real-time frame rate. Therefore, it could help enable more common use of NBI bronchoscopy for bronchial lesion detection.
Collapse
Affiliation(s)
- Vahid Daneshpajooh
- The Pennsylvania State University, School of Electrical Engineering and Computer Science, University Park, Pennsylvania, United States
| | - Danish Ahmad
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - Jennifer Toth
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - Rebecca Bascom
- The Pennsylvania State University, College of Medicine, Hershey, Pennsylvania, United States
| | - William E. Higgins
- The Pennsylvania State University, School of Electrical Engineering and Computer Science, University Park, Pennsylvania, United States
| |
Collapse
|
24
|
Rong Y, Chen Q, Fu Y, Yang X, Al-Hallaq HA, Wu QJ, Yuan L, Xiao Y, Cai B, Latifi K, Benedict SH, Buchsbaum JC, Qi XS. NRG Oncology Assessment of Artificial Intelligence Deep Learning-Based Auto-segmentation for Radiation Therapy: Current Developments, Clinical Considerations, and Future Directions. Int J Radiat Oncol Biol Phys 2024; 119:261-280. [PMID: 37972715 PMCID: PMC11023777 DOI: 10.1016/j.ijrobp.2023.10.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 09/16/2023] [Accepted: 10/14/2023] [Indexed: 11/19/2023]
Abstract
Deep learning neural networks (DLNN) in Artificial intelligence (AI) have been extensively explored for automatic segmentation in radiotherapy (RT). In contrast to traditional model-based methods, data-driven AI-based models for auto-segmentation have shown high accuracy in early studies in research settings and controlled environment (single institution). Vendor-provided commercial AI models are made available as part of the integrated treatment planning system (TPS) or as a stand-alone tool that provides streamlined workflow interacting with the main TPS. These commercial tools have drawn clinics' attention thanks to their significant benefit in reducing the workload from manual contouring and shortening the duration of treatment planning. However, challenges occur when applying these commercial AI-based segmentation models to diverse clinical scenarios, particularly in uncontrolled environments. Contouring nomenclature and guideline standardization has been the main task undertaken by the NRG Oncology. AI auto-segmentation holds the potential clinical trial participants to reduce interobserver variations, nomenclature non-compliance, and contouring guideline deviations. Meanwhile, trial reviewers could use AI tools to verify contour accuracy and compliance of those submitted datasets. In recognizing the growing clinical utilization and potential of these commercial AI auto-segmentation tools, NRG Oncology has formed a working group to evaluate the clinical utilization and potential of commercial AI auto-segmentation tools. The group will assess in-house and commercially available AI models, evaluation metrics, clinical challenges, and limitations, as well as future developments in addressing these challenges. General recommendations are made in terms of the implementation of these commercial AI models, as well as precautions in recognizing the challenges and limitations.
Collapse
Affiliation(s)
- Yi Rong
- Mayo Clinic Arizona, Phoenix, AZ
| | - Quan Chen
- City of Hope Comprehensive Cancer Center Duarte, CA
| | - Yabo Fu
- Memorial Sloan Kettering Cancer Center, Commack, NY
| | | | | | | | - Lulin Yuan
- Virginia Commonwealth University, Richmond, VA
| | - Ying Xiao
- University of Pennsylvania/Abramson Cancer Center, Philadelphia, PA
| | - Bin Cai
- The University of Texas Southwestern Medical Center, Dallas, TX
| | | | - Stanley H Benedict
- University of California Davis Comprehensive Cancer Center, Sacramento, CA
| | | | - X Sharon Qi
- University of California Los Angeles, Los Angeles, CA
| |
Collapse
|
25
|
Dritsas S, Chua KWD, Goh ZH, Simpson RE. Classification, registration and segmentation of ear canal impressions using convolutional neural networks. Med Image Anal 2024; 94:103152. [PMID: 38531210 DOI: 10.1016/j.media.2024.103152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 12/12/2023] [Accepted: 03/20/2024] [Indexed: 03/28/2024]
Abstract
Today, fitting bespoke hearing aids involves injecting silicone into patients' ears to produce ear canal molds. These are subsequently 3D scanned to create digital ear canal impressions. However, before digital impressions can be used they require a substantial amount of effort in manual 3D editing. In this article, we present computational methods to pre-process ear canal impressions. The aim is to create automation tools to assist the hearing aid design, manufacturing and fitting processes as well as normalizing anatomical data to assist the study of the outer ear canal's morphology. The methods include classifying the handedness of the impression into left and right ear types, orienting the geometries onto the same coordinate system sense, and removing extraneous artifacts introduced by the silicone mold. We investigate the use of convolutional neural networks for performing these semantic tasks and evaluate their accuracy using a dataset of 3000 ear canal impressions. The neural networks proved highly effective at performing these tasks with 95.8% adjusted accuracy in classification, 92.3% within 20° angular error in registration and 93.4% intersection over union in segmentation.
Collapse
Affiliation(s)
- Stylianos Dritsas
- Singapore University of Technology and Design, 8 Somapah Road, 487372, Singapore.
| | | | - Zhi Hwee Goh
- Singapore University of Technology and Design, 8 Somapah Road, 487372, Singapore
| | | |
Collapse
|
26
|
Morrison S, Gatsonis C, Eloyan A, Steingrimsson JA. Survival analysis using deep learning with medical imaging. Int J Biostat 2024; 20:1-12. [PMID: 37312249 PMCID: PMC11074924 DOI: 10.1515/ijb-2022-0113] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 02/24/2023] [Indexed: 06/15/2023]
Abstract
There is widespread interest in using deep learning to build prediction models for medical imaging data. These deep learning methods capture the local structure of the image and require no manual feature extraction. Despite the importance of modeling survival in the context of medical data analysis, research on deep learning methods for modeling the relationship of imaging and time-to-event data is still under-developed. We provide an overview of deep learning methods for time-to-event outcomes and compare several deep learning methods to Cox model based methods through the analysis of a histology dataset of gliomas.
Collapse
Affiliation(s)
- Samantha Morrison
- Department of Biostatistics, School of Public Health, Brown University, Providence, RI, USA
| | - Constantine Gatsonis
- Department of Biostatistics, School of Public Health, Brown University, Providence, RI, USA
| | - Ani Eloyan
- Department of Biostatistics, School of Public Health, Brown University, Providence, RI, USA
| | - Jon Arni Steingrimsson
- Department of Biostatistics, School of Public Health, Brown University, Providence, RI, USA
| |
Collapse
|
27
|
Migliorelli G, Fiorentino MC, Di Cosmo M, Villani FP, Mancini A, Moccia S. On the use of contrastive learning for standard-plane classification in fetal ultrasound imaging. Comput Biol Med 2024; 174:108430. [PMID: 38613892 DOI: 10.1016/j.compbiomed.2024.108430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Revised: 03/06/2024] [Accepted: 04/07/2024] [Indexed: 04/15/2024]
Abstract
BACKGROUND To investigate the effectiveness of contrastive learning, in particular SimClr, in reducing the need for large annotated ultrasound (US) image datasets for fetal standard plane identification. METHODS We explore SimClr advantage in the cases of both low and high inter-class variability, considering at the same time how classification performance varies according to different amounts of labels used. This evaluation is performed by exploiting contrastive learning through different training strategies. We apply both quantitative and qualitative analyses, using standard metrics (F1-score, sensitivity, and precision), Class Activation Mapping (CAM), and t-Distributed Stochastic Neighbor Embedding (t-SNE). RESULTS When dealing with high inter-class variability classification tasks, contrastive learning does not bring a significant advantage; whereas it results to be relevant for low inter-class variability classification, specifically when initialized with ImageNet weights. CONCLUSIONS Contrastive learning approaches are typically used when a large number of unlabeled data is available, which is not representative of US datasets. We proved that SimClr either as pre-training with backbone initialized via ImageNet weights or used in an end-to-end dual-task may impact positively the performance over standard transfer learning approaches, under a scenario in which the dataset is small and characterized by low inter-class variability.
Collapse
Affiliation(s)
| | | | - Mariachiara Di Cosmo
- Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy
| | | | - Adriano Mancini
- Department of Information Engineering, Università Politecnica delle Marche, Ancona, Italy
| | - Sara Moccia
- The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy
| |
Collapse
|
28
|
Guo X, Xu L, Li S, Xu M, Chu Y, Jiang Q. Cascade-EC Network: Recognition of Gastrointestinal Multiple Lesions Based on EfficientNet and CA_stm_Retinanet. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024:10.1007/s10278-024-01096-9. [PMID: 38587768 DOI: 10.1007/s10278-024-01096-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Revised: 03/17/2024] [Accepted: 03/20/2024] [Indexed: 04/09/2024]
Abstract
Capsule endoscopy (CE) is non-invasive and painless during gastrointestinal examination. However, capsule endoscopy can increase the workload of image reviewing for clinicians, making it prone to missed and misdiagnosed diagnoses. Current researches primarily concentrated on binary classifiers, multiple classifiers targeting fewer than four abnormality types and detectors within a specific segment of the digestive tract, and segmenters for a single type of anomaly. Due to intra-class variations, the task of creating a unified scheme for detecting multiple gastrointestinal diseases is particularly challenging. A cascade neural network designed in this study, Cascade-EC, can automatically identify and localize four types of gastrointestinal lesions in CE images: angiectasis, bleeding, erosion, and polyp. Cascade-EC consists of EfficientNet for image classification and CA_stm_Retinanet for lesion detection and location. As the first layer of Cascade-EC, the EfficientNet network classifies CE images. CA_stm_Retinanet, as the second layer, performs the target detection and location task on the classified image. CA_stm_Retinanet adopts the general architecture of Retinanet. Its feature extraction module is the CA_stm_Backbone from the stack of CA_stm Block. CA_stm Block adopts the split-transform-merge strategy and introduces the coordinate attention. The dataset in this study is from Shanghai East Hospital, collected by PillCam SB3 and AnKon capsule endoscopes, which contains a total of 7936 images of 317 patients from the years 2017 to 2021. In the testing set, the average precision of Cascade-EC in the multi-lesions classification task was 94.55%, the average recall was 90.60%, and the average F1 score was 92.26%. The mean mAP@ 0.5 of Cascade-EC for detecting the four types of diseases is 85.88%. The experimental results show that compared with a single target detection network, Cascade-EC has better performance and can effectively assist clinicians to classify and detect multiple lesions in CE images.
Collapse
Affiliation(s)
- Xudong Guo
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China.
| | - Lei Xu
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Shengnan Li
- School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
| | - Meidong Xu
- Endoscopy Center, Department of Gastroenterology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China.
| | - Yuan Chu
- Endoscopy Center, Department of Gastroenterology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China
| | - Qinfen Jiang
- Department of Information Management, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, 200120, China
| |
Collapse
|
29
|
Loizillon S, Bottani S, Maire A, Ströer S, Dormont D, Colliot O, Burgos N. Automatic motion artefact detection in brain T1-weighted magnetic resonance images from a clinical data warehouse using synthetic data. Med Image Anal 2024; 93:103073. [PMID: 38176355 DOI: 10.1016/j.media.2023.103073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 12/12/2023] [Accepted: 12/21/2023] [Indexed: 01/06/2024]
Abstract
Containing the medical data of millions of patients, clinical data warehouses (CDWs) represent a great opportunity to develop computational tools. Magnetic resonance images (MRIs) are particularly sensitive to patient movements during image acquisition, which will result in artefacts (blurring, ghosting and ringing) in the reconstructed image. As a result, a significant number of MRIs in CDWs are corrupted by these artefacts and may be unusable. Since their manual detection is impossible due to the large number of scans, it is necessary to develop tools to automatically exclude (or at least identify) images with motion in order to fully exploit CDWs. In this paper, we propose a novel transfer learning method from research to clinical data for the automatic detection of motion in 3D T1-weighted brain MRI. The method consists of two steps: a pre-training on research data using synthetic motion, followed by a fine-tuning step to generalise our pre-trained model to clinical data, relying on the labelling of 4045 images. The objectives were both (1) to be able to exclude images with severe motion, (2) to detect mild motion artefacts. Our approach achieved excellent accuracy for the first objective with a balanced accuracy nearly similar to that of the annotators (balanced accuracy>80 %). However, for the second objective, the performance was weaker and substantially lower than that of human raters. Overall, our framework will be useful to take advantage of CDWs in medical imaging and highlight the importance of a clinical validation of models trained on research data.
Collapse
Affiliation(s)
- Sophie Loizillon
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, Paris 75013, France
| | - Simona Bottani
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, Paris 75013, France
| | - Aurélien Maire
- AP-HP, Innovation & Données - Département des Services Numériques, Paris 75012, France
| | - Sebastian Ströer
- AP-HP, Hôpital de la Pitié Salpêtrière, Department of Neuroradiology, Paris 75013, France
| | - Didier Dormont
- AP-HP, Hôpital de la Pitié Salpêtrière, Department of Neuroradiology, Paris 75013, France; Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, DMU DIAMENT, Paris 75013, France
| | - Olivier Colliot
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, Paris 75013, France
| | - Ninon Burgos
- Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM, CNRS, Inria, Inserm, AP-HP, Hôpital de la Pitié-Salpêtrière, Paris 75013, France.
| |
Collapse
|
30
|
Aktar M, Xiao Y, Tehrani AKZ, Tampieri D, Rivaz H, Kersten-Oertel M. SCANED: Siamese collateral assessment network for evaluation of collaterals from ischemic damage. Comput Med Imaging Graph 2024; 113:102346. [PMID: 38364600 DOI: 10.1016/j.compmedimag.2024.102346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 01/31/2024] [Accepted: 01/31/2024] [Indexed: 02/18/2024]
Abstract
This study conducts collateral evaluation from ischemic damage using a deep learning-based Siamese network, addressing the challenges associated with a small and imbalanced dataset. The collateral network provides an alternative oxygen and nutrient supply pathway in ischemic stroke cases, influencing treatment decisions. Research in this area focuses on automated collateral assessment using deep learning (DL) methods to expedite decision-making processes and enhance accuracy. Our study employed a 3D ResNet-based Siamese network, referred to as SCANED, to classify collaterals as good/intermediate or poor. Utilizing non-contrast computed tomography (NCCT) images, the network automates collateral identification and assessment by analyzing tissue degeneration around the ischemic site. Relevant features from the left/right hemispheres were extracted, and Euclidean Distance (ED) was employed for similarity measurement. Finally, dichotomized classification of good/intermediate or poor collateral is performed by SCANED using an optimal threshold derived from ROC analysis. SCANED provides a sensitivity of 0.88, a specificity of 0.63, and a weighted F1 score of 0.86 in the dichotomized classification.
Collapse
Affiliation(s)
- Mumu Aktar
- Concordia University, Gina Cody School of Engineering and Computer Science, 1455 De Maisonneuve Blvd. W., Montreal, H3g 1M8, Quebec, Canada.
| | - Yiming Xiao
- Concordia University, Gina Cody School of Engineering and Computer Science, 1455 De Maisonneuve Blvd. W., Montreal, H3g 1M8, Quebec, Canada
| | - Ali K Z Tehrani
- Concordia University, Gina Cody School of Engineering and Computer Science, 1455 De Maisonneuve Blvd. W., Montreal, H3g 1M8, Quebec, Canada
| | - Donatella Tampieri
- Queens University, Department of Diagnostic Radiology, Kingston Health Sciences Centre, Kingston General Hospital 76 Stuart Street Kingston, K7L 2V7, Ontario, Canada
| | - Hassan Rivaz
- Concordia University, Gina Cody School of Engineering and Computer Science, 1455 De Maisonneuve Blvd. W., Montreal, H3g 1M8, Quebec, Canada
| | - Marta Kersten-Oertel
- Concordia University, Gina Cody School of Engineering and Computer Science, 1455 De Maisonneuve Blvd. W., Montreal, H3g 1M8, Quebec, Canada
| |
Collapse
|
31
|
Lundin J, Suutala A, Holmström O, Henriksson S, Valkamo S, Kaingu H, Kinyua F, Muinde M, Lundin M, Diwan V, Mårtensson A, Linder N. Diagnosis of soil-transmitted helminth infections with digital mobile microscopy and artificial intelligence in a resource-limited setting. PLoS Negl Trop Dis 2024; 18:e0012041. [PMID: 38602896 PMCID: PMC11008773 DOI: 10.1371/journal.pntd.0012041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 02/28/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND Infections caused by soil-transmitted helminths (STHs) are the most prevalent neglected tropical diseases and result in a major disease burden in low- and middle-income countries, especially in school-aged children. Improved diagnostic methods, especially for light intensity infections, are needed for efficient, control and elimination of STHs as a public health problem, as well as STH management. Image-based artificial intelligence (AI) has shown promise for STH detection in digitized stool samples. However, the diagnostic accuracy of AI-based analysis of entire microscope slides, so called whole-slide images (WSI), has previously not been evaluated on a sample-level in primary healthcare settings in STH endemic countries. METHODOLOGY/PRINCIPAL FINDINGS Stool samples (n = 1,335) were collected during 2020 from children attending primary schools in Kwale County, Kenya, prepared according to the Kato-Katz method at a local primary healthcare laboratory and digitized with a portable whole-slide microscopy scanner and uploaded via mobile networks to a cloud environment. The digital samples of adequate quality (n = 1,180) were split into a training (n = 388) and test set (n = 792) and a deep-learning system (DLS) developed for detection of STHs. The DLS findings were compared with expert manual microscopy and additional visual assessment of the digital samples in slides with discordant results between the methods. Manual microscopy detected 15 (1.9%) Ascaris lumbricoides, 172 (21.7%) Tricuris trichiura and 140 (17.7%) hookworm (Ancylostoma duodenale or Necator americanus) infections in the test set. Importantly, more than 90% of all STH positive cases represented light intensity infections. With manual microscopy as the reference standard, the sensitivity of the DLS as the index test for detection of A. lumbricoides, T. trichiura and hookworm was 80%, 92% and 76%, respectively. The corresponding specificity was 98%, 90% and 95%. Notably, in 79 samples (10%) classified as negative by manual microscopy for a specific species, STH eggs were detected by the DLS and confirmed correct by visual inspection of the digital samples. CONCLUSIONS/SIGNIFICANCE Analysis of digitally scanned stool samples with the DLS provided high diagnostic accuracy for detection of STHs. Importantly, a substantial number of light intensity infections were missed by manual microscopy but detected by the DLS. Thus, analysis of WSIs with image-based AI may provide a future tool for improved detection of STHs in a primary healthcare setting, which in turn could facilitate monitoring and evaluation of control programs.
Collapse
Affiliation(s)
- Johan Lundin
- Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | - Antti Suutala
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | - Oscar Holmström
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | - Samuel Henriksson
- Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden
| | - Severi Valkamo
- Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden
| | | | - Felix Kinyua
- Kinondo Kwetu Hospital, Kinondo, Kwale County, Kenya
| | - Martin Muinde
- Kinondo Kwetu Hospital, Kinondo, Kwale County, Kenya
| | - Mikael Lundin
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | - Vinod Diwan
- Department of Global Public Health, Karolinska Institutet, Stockholm, Sweden
| | - Andreas Mårtensson
- Global Health & Migration Unit, Department of Women’s and Children’s Health, Uppsala University, Uppsala, Sweden
| | - Nina Linder
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
- Global Health & Migration Unit, Department of Women’s and Children’s Health, Uppsala University, Uppsala, Sweden
| |
Collapse
|
32
|
Lindner C, Riquelme R, San Martín R, Quezada F, Valenzuela J, Maureira JP, Einersen M. Improving the radiological diagnosis of hepatic artery thrombosis after liver transplantation: Current approaches and future challenges. World J Transplant 2024; 14:88938. [PMID: 38576750 PMCID: PMC10989478 DOI: 10.5500/wjt.v14.i1.88938] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Revised: 12/03/2023] [Accepted: 12/29/2023] [Indexed: 03/15/2024] Open
Abstract
Hepatic artery thrombosis (HAT) is a devastating vascular complication following liver transplantation, requiring prompt diagnosis and rapid revascularization treatment to prevent graft loss. At present, imaging modalities such as ultrasound, computed tomography, and magnetic resonance play crucial roles in diagnosing HAT. Although imaging techniques have improved sensitivity and specificity for HAT diagnosis, they have limitations that hinder the timely diagnosis of this complication. In this sense, the emergence of artificial intelligence (AI) presents a transformative opportunity to address these diagnostic limitations. The develo pment of machine learning algorithms and deep neural networks has demon strated the potential to enhance the precision diagnosis of liver transplant com plications, enabling quicker and more accurate detection of HAT. This article examines the current landscape of imaging diagnostic techniques for HAT and explores the emerging role of AI in addressing future challenges in the diagnosis of HAT after liver transplant.
Collapse
Affiliation(s)
- Cristian Lindner
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| | - Raúl Riquelme
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| | - Rodrigo San Martín
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| | - Frank Quezada
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| | - Jorge Valenzuela
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| | - Juan P Maureira
- Department of Statistics, Catholic University of Maule, Talca 3460000, Chile
| | - Martín Einersen
- Department of Radiology, Faculty of Medicine, University of Concepción, Concepción 4030000, Chile
- Neurovascular Unit, Department of Radiology, Hospital Clínico Regional Guillermo Grant Benavente, Concepción 4030000, Chile
| |
Collapse
|
33
|
Abubakar H, Al-Turjman F, Ameen ZS, Mubarak AS, Altrjman C. A hybridized feature extraction for COVID-19 multi-class classification on computed tomography images. Heliyon 2024; 10:e26939. [PMID: 38463848 PMCID: PMC10920381 DOI: 10.1016/j.heliyon.2024.e26939] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2023] [Revised: 02/20/2024] [Accepted: 02/21/2024] [Indexed: 03/12/2024] Open
Abstract
COVID-19 has killed more than 5 million individuals worldwide within a short time. It is caused by SARS-CoV-2 which continuously mutates and produces more transmissible new different strains. It is therefore of great significance to diagnose COVID-19 early to curb its spread and reduce the death rate. Owing to the COVID-19 pandemic, traditional diagnostic methods such as reverse-transcription polymerase chain reaction (RT-PCR) are ineffective for diagnosis. Medical imaging is among the most effective techniques of respiratory disorders detection through machine learning and deep learning. However, conventional machine learning methods depend on extracted and engineered features, whereby the optimum features influence the classifier's performance. In this study, Histogram of Oriented Gradient (HOG) and eight deep learning models were utilized for feature extraction while K-Nearest Neighbour (KNN) and Support Vector Machines (SVM) were used for classification. A combined feature of HOG and deep learning feature was proposed to improve the performance of the classifiers. VGG-16 + HOG achieved 99.4 overall accuracy with SVM. This indicates that our proposed concatenated feature can enhance the SVM classifier's performance in COVID-19 detection.
Collapse
Affiliation(s)
- Hassana Abubakar
- Biomedical Engineering Department, Faculty of Engineering, Near East University, Mersin 10, Turkey
| | - Fadi Al-Turjman
- Artificial Intelligence Engineering Department, AI and Robotics Institute, Near East University, Mersin 10, Turkey
- Research Center for AI and IoT, Faculty of Engineering, University of Kyrenia, Mersin 10, Turkey
| | - Zubaida S. Ameen
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Auwalu S. Mubarak
- Operational Research Center in Healthcare, Near East University, Mersin 10, Turkey
| | - Chadi Altrjman
- Waterloo University, 200 University Avenue West. Waterloo, ON, Canada
| |
Collapse
|
34
|
Lambert T, Brunner C, Kil D, Wuyts R, D'Hondt E, Montaldo G, Urban A. A deep learning classification task for brain navigation in rodents using micro-Doppler ultrasound imaging. Heliyon 2024; 10:e27432. [PMID: 38495198 PMCID: PMC10943389 DOI: 10.1016/j.heliyon.2024.e27432] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2023] [Revised: 02/28/2024] [Accepted: 02/28/2024] [Indexed: 03/19/2024] Open
Abstract
Positioning and navigation are essential components of neuroimaging as they improve the quality and reliability of data acquisition, leading to advances in diagnosis, treatment outcomes, and fundamental understanding of the brain. Functional ultrasound imaging is an emerging technology providing high-resolution images of the brain vasculature, allowing for the monitoring of brain activity. However, as the technology is relatively new, there is no standardized tool for inferring the position in the brain from the vascular images. In this study, we present a deep learning-based framework designed to address this challenge. Our approach uses an image classification task coupled with a regression on the resulting probabilities to determine the position of a single image. To evaluate its performance, we conducted experiments using a dataset of 51 rat brain scans. The training positions were extracted at intervals of 375 μm, resulting in a positioning error of 176 μm. Further GradCAM analysis revealed that the predictions were primarily driven by subcortical vascular structures. Finally, we assessed the robustness of our method in a cortical stroke where the brain vasculature is severely impaired. Remarkably, no specific increase in the number of misclassifications was observed, confirming the method's reliability in challenging conditions. Overall, our framework provides accurate and flexible positioning, not relying on a pre-registered reference but rather on conserved vascular patterns.
Collapse
Affiliation(s)
- Théo Lambert
- Neuro-Electronics Research Flanders, Leuven, Belgium
- VIB, Leuven, Belgium
- Imec, Leuven, Belgium
- Department of Neuroscience, Faculty of Medicine, KU Leuven, Leuven, Belgium
| | - Clément Brunner
- Neuro-Electronics Research Flanders, Leuven, Belgium
- VIB, Leuven, Belgium
- Imec, Leuven, Belgium
- Department of Neuroscience, Faculty of Medicine, KU Leuven, Leuven, Belgium
| | - Dries Kil
- Neuro-Electronics Research Flanders, Leuven, Belgium
- VIB, Leuven, Belgium
- Imec, Leuven, Belgium
- Department of Neuroscience, Faculty of Medicine, KU Leuven, Leuven, Belgium
| | | | | | - Gabriel Montaldo
- Neuro-Electronics Research Flanders, Leuven, Belgium
- VIB, Leuven, Belgium
- Imec, Leuven, Belgium
- Department of Neuroscience, Faculty of Medicine, KU Leuven, Leuven, Belgium
| | - Alan Urban
- Neuro-Electronics Research Flanders, Leuven, Belgium
- VIB, Leuven, Belgium
- Imec, Leuven, Belgium
- Department of Neuroscience, Faculty of Medicine, KU Leuven, Leuven, Belgium
| |
Collapse
|
35
|
Saluja S, Trivedi MC, Saha A. Deep CNNs for glioma grading on conventional MRIs: Performance analysis, challenges, and future directions. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:5250-5282. [PMID: 38872535 DOI: 10.3934/mbe.2024232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2024]
Abstract
The increasing global incidence of glioma tumors has raised significant healthcare concerns due to their high mortality rates. Traditionally, tumor diagnosis relies on visual analysis of medical imaging and invasive biopsies for precise grading. As an alternative, computer-assisted methods, particularly deep convolutional neural networks (DCNNs), have gained traction. This research paper explores the recent advancements in DCNNs for glioma grading using brain magnetic resonance images (MRIs) from 2015 to 2023. The study evaluated various DCNN architectures and their performance, revealing remarkable results with models such as hybrid and ensemble based DCNNs achieving accuracy levels of up to 98.91%. However, challenges persisted in the form of limited datasets, lack of external validation, and variations in grading formulations across diverse literature sources. Addressing these challenges through expanding datasets, conducting external validation, and standardizing grading formulations can enhance the performance and reliability of DCNNs in glioma grading, thereby advancing brain tumor classification and extending its applications to other neurological disorders.
Collapse
Affiliation(s)
- Sonam Saluja
- Department of Computer Science and Engineering, National Institute of Technology Agartala, Tripura 799046, India
| | - Munesh Chandra Trivedi
- Department of Computer Science and Engineering, National Institute of Technology Agartala, Tripura 799046, India
| | - Ashim Saha
- Department of Computer Science and Engineering, National Institute of Technology Agartala, Tripura 799046, India
| |
Collapse
|
36
|
Avola D, Cannistraci I, Cascio M, Cinque L, Fagioli A, Foresti GL, Rodolà E, Solito L. MV-MS-FETE: Multi-view multi-scale feature extractor and transformer encoder for stenosis recognition in echocardiograms. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 245:108037. [PMID: 38271793 DOI: 10.1016/j.cmpb.2024.108037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Revised: 12/27/2023] [Accepted: 01/15/2024] [Indexed: 01/27/2024]
Abstract
BACKGROUND aortic stenosis is a common heart valve disease that mainly affects older people in developed countries. Its early detection is crucial to prevent the irreversible disease progression and, eventually, death. A typical screening technique to detect stenosis uses echocardiograms; however, variations introduced by other tissues, camera movements, and uneven lighting can hamper the visual inspection, leading to misdiagnosis. To address these issues, effective solutions involve employing deep learning algorithms to assist clinicians in detecting and classifying stenosis by developing models that can predict this pathology from single heart views. Although promising, the visual information conveyed by a single image may not be sufficient for an accurate diagnosis, especially when using an automatic system; thus, this indicates that different solutions should be explored. METHODOLOGY following this rationale, this paper proposes a novel deep learning architecture, composed of a multi-view, multi-scale feature extractor, and a transformer encoder (MV-MS-FETE) to predict stenosis from parasternal long and short-axis views. In particular, starting from the latter, the designed model extracts relevant features at multiple scales along its feature extractor component and takes advantage of a transformer encoder to perform the final classification. RESULTS experiments were performed on the recently released Tufts medical echocardiogram public dataset, which comprises 27,788 images split into training, validation, and test sets. Due to the recent release of this collection, tests were also conducted on several state-of-the-art models to create multi-view and single-view benchmarks. For all models, standard classification metrics were computed (e.g., precision, F1-score). The obtained results show that the proposed approach outperforms other multi-view methods in terms of accuracy and F1-score and has more stable performance throughout the training procedure. Furthermore, the experiments also highlight that multi-view methods generally perform better than their single-view counterparts. CONCLUSION this paper introduces a novel multi-view and multi-scale model for aortic stenosis recognition, as well as three benchmarks to evaluate it, effectively providing multi-view and single-view comparisons that fully highlight the model's effectiveness in aiding clinicians in performing diagnoses while also producing several baselines for the aortic stenosis recognition task.
Collapse
Affiliation(s)
- Danilo Avola
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| | - Irene Cannistraci
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| | - Marco Cascio
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| | - Luigi Cinque
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| | - Alessio Fagioli
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy.
| | - Gian Luca Foresti
- Department of Mathematics, Computer Science and Physics, University of Udine, 33100 Udine, Italy
| | - Emanuele Rodolà
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| | - Luciana Solito
- Department of Computer Science, Sapienza University, Via Salaria 113, 00185, Rome, Italy
| |
Collapse
|
37
|
Ilyas T, Ahmad K, Arsa DMS, Jeong YC, Kim H. Enhancing medical image analysis with unsupervised domain adaptation approach across microscopes and magnifications. Comput Biol Med 2024; 170:108055. [PMID: 38295480 DOI: 10.1016/j.compbiomed.2024.108055] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 01/05/2024] [Accepted: 01/26/2024] [Indexed: 02/02/2024]
Abstract
In the domain of medical image analysis, deep learning models are heralding a revolution, especially in detecting complex and nuanced features characteristic of diseases like tumors and cancers. However, the robustness and adaptability of these models across varied imaging conditions and magnifications remain a formidable challenge. This paper introduces the Fourier Adaptive Recognition System (FARS), a pioneering model primarily engineered to address adaptability in malarial parasite recognition. Yet, the foundational principles guiding FARS lend themselves seamlessly to broader applications, including tumor and cancer diagnostics. FARS capitalizes on the untapped potential of transitioning from bounding box labels to richer semantic segmentation labels, enabling a more refined examination of microscopy slides. With the integration of adversarial training and the Color Domain Aware Fourier Domain Adaptation (F2DA), the model ensures consistent feature extraction across diverse microscopy configurations. The further inclusion of category-dependent context attention amplifies FARS's cross-domain versatility. Evidenced by a substantial elevation in cross-magnification performance from 31.3% mAP to 55.19% mAP and a 15.68% boost in cross-domain adaptability, FARS positions itself as a significant advancement in malarial parasite recognition. Furthermore, the core methodologies of FARS can serve as a blueprint for enhancing precision in other realms of medical image analysis, especially in the complex terrains of tumor and cancer imaging. The code is available at; https://github.com/Mr-TalhaIlyas/FARS.
Collapse
Affiliation(s)
- Talha Ilyas
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea.
| | - Khubaib Ahmad
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea
| | - Dewa Made Sri Arsa
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Department of Information Technology, Universitas Udayana, Bali, 80361, Indonesia
| | - Yong Chae Jeong
- Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Division of Electronics Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea
| | - Hyongsuk Kim
- Division of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Republic of Korea; Core Research Institute of Intelligent Robots, Jeonbuk National University, Jeonju, 54896, Republic of Korea.
| |
Collapse
|
38
|
Wang J, Yang X, Jia X, Xue W, Chen R, Chen Y, Zhu X, Liu L, Cao Y, Zhou J, Ni D, Gu N. Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training. Comput Biol Med 2024; 171:108087. [PMID: 38364658 DOI: 10.1016/j.compbiomed.2024.108087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Revised: 01/04/2024] [Accepted: 01/27/2024] [Indexed: 02/18/2024]
Abstract
Thyroid nodule classification and segmentation in ultrasound images are crucial for computer-aided diagnosis; however, they face limitations owing to insufficient labeled data. In this study, we proposed a multi-view contrastive self-supervised method to improve thyroid nodule classification and segmentation performance with limited manual labels. Our method aligns the transverse and longitudinal views of the same nodule, thereby enabling the model to focus more on the nodule area. We designed an adaptive loss function that eliminates the limitations of the paired data. Additionally, we adopted a two-stage pre-training to exploit the pre-training on ImageNet and thyroid ultrasound images. Extensive experiments were conducted on a large-scale dataset collected from multiple centers. The results showed that the proposed method significantly improves nodule classification and segmentation performance with limited manual labels and outperforms state-of-the-art self-supervised methods. The two-stage pre-training also significantly exceeded ImageNet pre-training.
Collapse
Affiliation(s)
- Jian Wang
- Key Laboratory for Bio-Electromagnetic Environment and Advanced Medical Theranostics, School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, 211166, China
| | - Xin Yang
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Xiaohong Jia
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025, China
| | - Wufeng Xue
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Rusi Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Yanlin Chen
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Xiliang Zhu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Lian Liu
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China
| | - Yan Cao
- Shenzhen RayShape Medical Technology Co., Ltd, Shenzhen, 518051, China
| | - Jianqiao Zhou
- Department of Ultrasound, Ruijin Hospital, Shanghai Jiaotong University School of Medicine, Shanghai, 200025, China.
| | - Dong Ni
- National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, School of Biomedical Engineering, Health Science Center, Shenzhen University, Shenzhen, 518073, China; Marshall Laboratory of Biomedical Engineering, Shenzhen University, Shenzhen, 518073, China; Medical UltraSound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, 518073, China.
| | - Ning Gu
- Key Laboratory for Bio-Electromagnetic Environment and Advanced Medical Theranostics, School of Biomedical Engineering and Informatics, Nanjing Medical University, Nanjing, 211166, China; Cardiovascular Disease Research Center, Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Medical School, Nanjing University, Nanjing, 210093, China.
| |
Collapse
|
39
|
Shao D, Yang H, Liu C, Ma L. AFANet: Adaptive feature aggregation for polyp segmentation. Med Eng Phys 2024; 125:104118. [PMID: 38508807 DOI: 10.1016/j.medengphy.2024.104118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2023] [Revised: 01/15/2024] [Accepted: 02/13/2024] [Indexed: 03/22/2024]
Abstract
In terms of speed and accuracy, the deep learning-based polyp segmentation method is superior. It is essential for the early detection and treatment of colorectal cancer and has the potential to greatly reduce the disease's overall prevalence. Due to the various forms and sizes of polyps, as well as the blurring of the boundaries between the polyp region and the surrounding mucus, most existing algorithms are unable to provide highly accurate colorectal polyp segmentation. Therefore, to overcome these obstacles, we propose an adaptive feature aggregation network (AFANet). It contains two main modules: the Multi-modal Balancing Attention Module (MMBA) and the Global Context Module (GCM). The MMBA extracts improved local characteristics for inference by integrating local contextual information while paying attention to them in three regions: foreground, background, and border. The GCM takes global information from the top of the encoder and sends it to the decoder layer in order to further investigate global contextual feature information in the pathologic picture. Dice of 92.11 % and 94.76 % and MIoU of 91.07 % and 94.54 %, respectively, are achieved by comprehensive experimental validation of our proposed technique on two benchmark datasets, Kvasir-SEG and CVCClinicDB. The experimental results demonstrate that the strategy outperforms other cutting-edge approaches.
Collapse
Affiliation(s)
- Dangguo Shao
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
| | - Haiqiong Yang
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
| | - Cuiyin Liu
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China.
| | - Lei Ma
- Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, China
| |
Collapse
|
40
|
Bharadwaj UU, Chin CT, Majumdar S. Practical Applications of Artificial Intelligence in Spine Imaging: A Review. Radiol Clin North Am 2024; 62:355-370. [PMID: 38272627 DOI: 10.1016/j.rcl.2023.10.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2024]
Abstract
Artificial intelligence (AI), a transformative technology with unprecedented potential in medical imaging, can be applied to various spinal pathologies. AI-based approaches may improve imaging efficiency, diagnostic accuracy, and interpretation, which is essential for positive patient outcomes. This review explores AI algorithms, techniques, and applications in spine imaging, highlighting diagnostic impact and challenges with future directions for integrating AI into spine imaging workflow.
Collapse
Affiliation(s)
- Upasana Upadhyay Bharadwaj
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4th Street, Byers Hall, Suite 203, Room 203D, San Francisco, CA 94158, USA
| | - Cynthia T Chin
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 505 Parnassus Avenue, Box 0628, San Francisco, CA 94143, USA.
| | - Sharmila Majumdar
- Department of Radiology and Biomedical Imaging, University of California San Francisco, 1700 4th Street, Byers Hall, Suite 203, Room 203D, San Francisco, CA 94158, USA
| |
Collapse
|
41
|
Al Muhaisen S, Safi O, Ulayan A, Aljawamis S, Fakhoury M, Baydoun H, Abuquteish D. Artificial Intelligence-Powered Mammography: Navigating the Landscape of Deep Learning for Breast Cancer Detection. Cureus 2024; 16:e56945. [PMID: 38665752 PMCID: PMC11044525 DOI: 10.7759/cureus.56945] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/26/2024] [Indexed: 04/28/2024] Open
Abstract
Worldwide, breast cancer (BC) is one of the most commonly diagnosed malignancies in women. Early detection is key to improving survival rates and health outcomes. This literature review focuses on how artificial intelligence (AI), especially deep learning (DL), can enhance the ability of mammography, a key tool in BC detection, to yield more accurate results. Artificial intelligence has shown promise in reducing diagnostic errors and increasing early cancer detection chances. Nevertheless, significant challenges exist, including the requirement for large amounts of high-quality data and concerns over data privacy. Despite these hurdles, AI and DL are advancing the field of radiology, offering better ways to diagnose, detect, and treat diseases. The U.S. Food and Drug Administration (FDA) has approved several AI diagnostic tools. Yet, the full potential of these technologies, especially for more advanced screening methods like digital breast tomosynthesis (DBT), depends on further clinical studies and the development of larger databases. In summary, this review highlights the exciting potential of AI in BC screening. It calls for more research and validation to fully employ the power of AI in clinical practice, ensuring that these technologies can help save lives by improving diagnosis accuracy and efficiency.
Collapse
Affiliation(s)
| | - Omar Safi
- Medicine, Faculty of Medicine, The Hashemite University, Zarqa, JOR
| | - Ahmad Ulayan
- Medicine, Faculty of Medicine, The Hashemite University, Zarqa, JOR
| | - Sara Aljawamis
- Medicine, Faculty of Medicine, The Hashemite University, Zarqa, JOR
| | - Maryam Fakhoury
- Medicine, Faculty of Medicine, The Hashemite University, Zarqa, JOR
| | - Haneen Baydoun
- Diagnostic Radiology, King Hussein Cancer Center, Amman, JOR
| | - Dua Abuquteish
- Microbiology, Pathology and Forensic Medicine, Faculty of Medicine, The Hashemite University, Zarqa, JOR
- Pathology and Laboratory Medicine, King Hussein Cancer Center, Amman, JOR
| |
Collapse
|
42
|
De A, Mishra N, Chang HT. An approach to the dermatological classification of histopathological skin images using a hybridized CNN-DenseNet model. PeerJ Comput Sci 2024; 10:e1884. [PMID: 38435616 PMCID: PMC10909212 DOI: 10.7717/peerj-cs.1884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2023] [Accepted: 01/29/2024] [Indexed: 03/05/2024]
Abstract
This research addresses the challenge of automating skin disease diagnosis using dermatoscopic images. The primary issue lies in accurately classifying pigmented skin lesions, which traditionally rely on manual assessment by dermatologists and are prone to subjectivity and time consumption. By integrating a hybrid CNN-DenseNet model, this study aimed to overcome the complexities of differentiating various skin diseases and automating the diagnostic process effectively. Our methodology involved rigorous data preprocessing, exploratory data analysis, normalization, and label encoding. Techniques such as model hybridization, batch normalization and data fitting were employed to optimize the model architecture and data fitting. Initial iterations of our convolutional neural network (CNN) model achieved an accuracy of 76.22% on the test data and 75.69% on the validation data. Recognizing the need for improvement, the model was hybridized with DenseNet architecture and ResNet architecture was implemented for feature extraction and then further trained on the HAM10000 and PAD-UFES-20 datasets. Overall, our efforts resulted in a hybrid model that demonstrated an impressive accuracy of 95.7% on the HAM10000 dataset and 91.07% on the PAD-UFES-20 dataset. In comparison to recently published works, our model stands out because of its potential to effectively diagnose skin diseases such as melanocytic nevi, melanoma, benign keratosis-like lesions, basal cell carcinoma, actinic keratoses, vascular lesions, and dermatofibroma, all of which rival the diagnostic accuracy of real-world clinical specialists but also offer customization potential for more nuanced clinical uses.
Collapse
Affiliation(s)
- Anubhav De
- School of Computing Science & Engineering, VIT Bhopal University, Madhya Pradesh, India
| | - Nilamadhab Mishra
- School of Computing Science & Engineering, VIT Bhopal University, Madhya Pradesh, India
| | - Hsien-Tsung Chang
- Department of Computer Science and Information Engineering, Chang Gung University, Taoyuan, Taiwan
- Department of Physical Medicine and Rehabilitation, Chang Gung Memorial Hospital, Taoyuan, Taiwan
- Artificial Intelligence Research Center, Chang Gung University, Taoyuan, Taiwan
- Bachelor Program in Artificial Intelligence, Chang Gung University, Taoyuan, Taiwan
| |
Collapse
|
43
|
Perkgoz C. Identifying optical microscope images of CVD-grown two-dimensional MoS 2 by convolutional neural networks and transfer learning. PeerJ Comput Sci 2024; 10:e1885. [PMID: 38435565 PMCID: PMC10909165 DOI: 10.7717/peerj-cs.1885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/29/2024] [Indexed: 03/05/2024]
Abstract
Background In Complementary Metal-Oxide Semiconductor (CMOS) technology, scaling down has been a key strategy to improve chip performance and reduce power losses. However, challenges such as sub-threshold leakage and gate leakage, resulting from short-channel effects, contribute to an increase in distributed static power. Two-dimensional transition metal dichalcogenides (2D TMDs) emerge as potential solutions, serving as channel materials with steep sub-threshold swings and lower power consumption. However, the production and development of these 2-dimensional materials require some time-consuming tasks. In order to employ them in different fields, including chip technology, it is crucial to ensure that their production meets the required standards of quality and uniformity; in this context, deep learning techniques show significant potential. Methods This research introduces a transfer learning-based deep convolutional neural network (CNN) to classify chemical vapor deposition (CVD) grown molybdenum disulfide (MoS2) flakes based on their uniformity or the occurrence of defects affecting electronic properties. Acquiring and labeling a sufficient number of microscope images for CNN training may not be realistic. To address this challenge, artificial images were generated using Fresnel equations to pre-train the CNN. Subsequently, accuracy was improved through fine-tuning with a limited set of real images. Results The proposed transfer learning-based CNN method significantly improved all measurement metrics with respect to the ordinary CNNs. The initial CNN, trained with limited data and without transfer learning, achieved 68% average accuracy for binary classification. Through transfer learning and artificial images, the same CNN achieved 85% average accuracy, demonstrating an average increase of approximately 17%. While this study specifically focuses on MoS2 structures, the same methodology can be extended to other 2-dimensional materials by simply incorporating their specific parameters when generating artificial images.
Collapse
Affiliation(s)
- Cahit Perkgoz
- Department of Computer Engineering, Eskisehir Technical University, Eskişehir, Turkey
| |
Collapse
|
44
|
Jodeiri A, Seyedarabi H, Danishvar S, Shafiei SH, Sales JG, Khoori M, Rahimi S, Mortazavi SMJ. Concurrent Learning Approach for Estimation of Pelvic Tilt from Anterior-Posterior Radiograph. Bioengineering (Basel) 2024; 11:194. [PMID: 38391680 PMCID: PMC10886461 DOI: 10.3390/bioengineering11020194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2024] [Revised: 02/02/2024] [Accepted: 02/08/2024] [Indexed: 02/24/2024] Open
Abstract
Accurate and reliable estimation of the pelvic tilt is one of the essential pre-planning factors for total hip arthroplasty to prevent common post-operative complications such as implant impingement and dislocation. Inspired by the latest advances in deep learning-based systems, our focus in this paper has been to present an innovative and accurate method for estimating the functional pelvic tilt (PT) from a standing anterior-posterior (AP) radiography image. We introduce an encoder-decoder-style network based on a concurrent learning approach called VGG-UNET (VGG embedded in U-NET), where a deep fully convolutional network known as VGG is embedded at the encoder part of an image segmentation network, i.e., U-NET. In the bottleneck of the VGG-UNET, in addition to the decoder path, we use another path utilizing light-weight convolutional and fully connected layers to combine all extracted feature maps from the final convolution layer of VGG and thus regress PT. In the test phase, we exclude the decoder path and consider only a single target task i.e., PT estimation. The absolute errors obtained using VGG-UNET, VGG, and Mask R-CNN are 3.04 ± 2.49, 3.92 ± 2.92, and 4.97 ± 3.87, respectively. It is observed that the VGG-UNET leads to a more accurate prediction with a lower standard deviation (STD). Our experimental results demonstrate that the proposed multi-task network leads to a significantly improved performance compared to the best-reported results based on cascaded networks.
Collapse
Affiliation(s)
- Ata Jodeiri
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666, Iran
- Faculty of Advanced Medical Sciences, Tabriz University of Medical Sciences, Tabriz 51656, Iran
| | - Hadi Seyedarabi
- Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz 51666, Iran
| | - Sebelan Danishvar
- College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK
| | - Seyyed Hossein Shafiei
- Orthopedic Surgery Research Centre, Sina University Hospital, School of Medicine, Tehran University of Medical Sciences, Tehran 51656, Iran
| | - Jafar Ganjpour Sales
- Department of Orthopedic Surgery, Shohada Hospital, Tabriz University of Medical Sciences, Tabriz 51656, Iran
| | - Moein Khoori
- Joint Reconstruction Research Center (JRRC), Tehran University of Medical Sciences, Tehran 51656, Iran
| | - Shakiba Rahimi
- Orthopedic Surgery Research Centre, Sina University Hospital, School of Medicine, Tehran University of Medical Sciences, Tehran 51656, Iran
| | | |
Collapse
|
45
|
Gifani P, Shalbaf A. Transfer Learning with Pretrained Convolutional Neural Network for Automated Gleason Grading of Prostate Cancer Tissue Microarrays. JOURNAL OF MEDICAL SIGNALS & SENSORS 2024; 14:4. [PMID: 38510670 PMCID: PMC10950311 DOI: 10.4103/jmss.jmss_42_22] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Revised: 12/20/2022] [Accepted: 03/22/2023] [Indexed: 03/22/2024]
Abstract
Background The Gleason grading system has been the most effective prediction for prostate cancer patients. This grading system provides this possibility to assess prostate cancer's aggressiveness and then constitutes an important factor for stratification and therapeutic decisions. However, determining Gleason grade requires highly-trained pathologists and is time-consuming and tedious, and suffers from inter-pathologist variability. To remedy these limitations, this paper introduces an automatic methodology based on transfer learning with pretrained convolutional neural networks (CNNs) for automatic Gleason grading of prostate cancer tissue microarray (TMA). Methods Fifteen pretrained (CNNs): Efficient Nets (B0-B5), NasNetLarge, NasNetMobile, InceptionV3, ResNet-50, SeResnet 50, Xception, DenseNet121, ResNext50, and inception_resnet_v2 were fine-tuned on a dataset of prostate carcinoma TMA images. Six pathologists separately identified benign and cancerous areas for each prostate TMA image by allocating benign, 3, 4, or 5 Gleason grade for 244 patients. The dataset was labeled by these pathologists and majority vote was applied on pixel-wise annotations to obtain a unified label. Results Results showed the NasnetLarge architecture is the best model among them in the classification of prostate TMA images of 244 patients with accuracy of 0.93 and area under the curve of 0.98. Conclusion Our study can act as a highly trained pathologist to categorize the prostate cancer stages with more objective and reproducible results.
Collapse
Affiliation(s)
- Parisa Gifani
- Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
| | - Ahmad Shalbaf
- Cancer Research Center, Shahid Beheshti University of Medical Sciences, Tehran, Iran
- Department of Biomedical Engineering and Medical Physics, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| |
Collapse
|
46
|
Çetin-Kaya Y, Kaya M. A Novel Ensemble Framework for Multi-Classification of Brain Tumors Using Magnetic Resonance Imaging. Diagnostics (Basel) 2024; 14:383. [PMID: 38396422 PMCID: PMC10888105 DOI: 10.3390/diagnostics14040383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 02/01/2024] [Accepted: 02/06/2024] [Indexed: 02/25/2024] Open
Abstract
Brain tumors can have fatal consequences, affecting many body functions. For this reason, it is essential to detect brain tumor types accurately and at an early stage to start the appropriate treatment process. Although convolutional neural networks (CNNs) are widely used in disease detection from medical images, they face the problem of overfitting in the training phase on limited labeled and insufficiently diverse datasets. The existing studies use transfer learning and ensemble models to overcome these problems. When the existing studies are examined, it is evident that there is a lack of models and weight ratios that will be used with the ensemble technique. With the framework proposed in this study, several CNN models with different architectures are trained with transfer learning and fine-tuning on three brain tumor datasets. A particle swarm optimization-based algorithm determined the optimum weights for combining the five most successful CNN models with the ensemble technique. The results across three datasets are as follows: Dataset 1, 99.35% accuracy and 99.20 F1-score; Dataset 2, 98.77% accuracy and 98.92 F1-score; and Dataset 3, 99.92% accuracy and 99.92 F1-score. We achieved successful performances on three brain tumor datasets, showing that the proposed framework is reliable in classification. As a result, the proposed framework outperforms existing studies, offering clinicians enhanced decision-making support through its high-accuracy classification performance.
Collapse
Affiliation(s)
- Yasemin Çetin-Kaya
- Department of Computer Engineering, Faculty of Engineering and Architecture, Tokat Gaziosmanpaşa University, Tokat 60250, Turkey
| | - Mahir Kaya
- Department of Computer Engineering, Faculty of Engineering and Architecture, Tokat Gaziosmanpaşa University, Tokat 60250, Turkey
| |
Collapse
|
47
|
Gouzou D, Taimori A, Haloubi T, Finlayson N, Wang Q, Hopgood JR, Vallejo M. Applications of machine learning in time-domain fluorescence lifetime imaging: a review. Methods Appl Fluoresc 2024; 12:022001. [PMID: 38055998 PMCID: PMC10851337 DOI: 10.1088/2050-6120/ad12f7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Revised: 09/25/2023] [Accepted: 12/06/2023] [Indexed: 12/08/2023]
Abstract
Many medical imaging modalities have benefited from recent advances in Machine Learning (ML), specifically in deep learning, such as neural networks. Computers can be trained to investigate and enhance medical imaging methods without using valuable human resources. In recent years, Fluorescence Lifetime Imaging (FLIm) has received increasing attention from the ML community. FLIm goes beyond conventional spectral imaging, providing additional lifetime information, and could lead to optical histopathology supporting real-time diagnostics. However, most current studies do not use the full potential of machine/deep learning models. As a developing image modality, FLIm data are not easily obtainable, which, coupled with an absence of standardisation, is pushing back the research to develop models which could advance automated diagnosis and help promote FLIm. In this paper, we describe recent developments that improve FLIm image quality, specifically time-domain systems, and we summarise sensing, signal-to-noise analysis and the advances in registration and low-level tracking. We review the two main applications of ML for FLIm: lifetime estimation and image analysis through classification and segmentation. We suggest a course of action to improve the quality of ML studies applied to FLIm. Our final goal is to promote FLIm and attract more ML practitioners to explore the potential of lifetime imaging.
Collapse
Affiliation(s)
- Dorian Gouzou
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| | - Ali Taimori
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Tarek Haloubi
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Neil Finlayson
- Neil Finlayson is with Institute for Integrated Micro and Nano Systems, School of Engineering, University ofEdinburgh, Edinburgh EH9 3FF, United Kingdom
| | - Qiang Wang
- Qiang Wang is with Centre for Inflammation Research, University of Edinburgh, Edinburgh, EH16 4TJ, United Kingdom
| | - James R Hopgood
- Tarek Haloubi, Ali Taimori, and James R. Hopgood are with Institute for Imaging, Data and Communication, School of Engineering, University of Edinburgh, Edinburgh, EH9 3FG, United Kingdom
| | - Marta Vallejo
- Dorian Gouzou and Marta Vallejo are with Institute of Signals, Sensors and Systems, School of Engineering and Physical Sciences, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom
| |
Collapse
|
48
|
Park HC, Joo Y, Lee OJ, Lee K, Song TK, Choi C, Choi MH, Yoon C. Automated classification of liver fibrosis stages using ultrasound imaging. BMC Med Imaging 2024; 24:36. [PMID: 38321373 PMCID: PMC10848434 DOI: 10.1186/s12880-024-01209-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 01/21/2024] [Indexed: 02/08/2024] Open
Abstract
BACKGROUND Ultrasound imaging is the most frequently performed for the patients with chronic hepatitis or liver cirrhosis. However, ultrasound imaging is highly operator dependent and interpretation of ultrasound images is subjective, thus well-trained radiologist is required for evaluation. Automated classification of liver fibrosis could alleviate the shortage of skilled radiologist especially in low-to-middle income countries. The purposed of this study is to evaluate deep convolutional neural networks (DCNNs) for classifying the degree of liver fibrosis according to the METAVIR score using US images. METHODS We used ultrasound (US) images from two tertiary university hospitals. A total of 7920 US images from 933 patients were used for training/validation of DCNNs. All patient were underwent liver biopsy or hepatectomy, and liver fibrosis was categorized based on pathology results using the METAVIR score. Five well-established DCNNs (VGGNet, ResNet, DenseNet, EfficientNet and ViT) was implemented to predict the METAVIR score. The performance of DCNNs for five-level (F0/F1/F2/F3/F4) classification was evaluated through area under the receiver operating characteristic curve (AUC) with 95% confidential interval, accuracy, sensitivity, specificity, positive and negative likelihood ratio. RESULTS Similar mean AUC values were achieved for five models; VGGNet (0.96), ResNet (0.96), DenseNet (0.95), EfficientNet (0.96), and ViT (0.95). The same mean accuracy (0.94) and specificity values (0.96) were yielded for all models. In terms of sensitivity, EffcientNet achieved highest mean value (0.85) while the other models produced slightly lower values range from 0.82 to 0.84. CONCLUSION In this study, we demonstrated that DCNNs can classify the staging of liver fibrosis according to METAVIR score with high performance using conventional B-mode images. Among them, EfficientNET that have fewer parameters and computation cost produced highest performance. From the results, we believe that DCNNs based classification of liver fibrosis may allow fast and accurate diagnosis of liver fibrosis without needs of additional equipment for add-on test and may be powerful tool for supporting radiologists in clinical practice.
Collapse
Grants
- NTIS Number: 9991007146 the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety
- HI21C0940110021 the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea
- No. 2022-0-00101 the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT)
- the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety
- the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea
- the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT)
Collapse
Affiliation(s)
- Hyun-Cheol Park
- Division of Industrial Mathematics, National Institute for Mathematical Sciences, 70, Yuseong-daero, Yuseong-gu, 34047, Daejeon, Republic of Korea
| | - YunSang Joo
- Department of Computer Engineering, Gachon University, 1342, Seongnam-daero, Sujeong-gu, 13120, Seongnam-si, Gyeonggi-do, Republic of Korea
| | - O-Joun Lee
- Department of Artificial Intelligence, The Catholic University of Korea, 43, Jibong-ro, 14662, Bucheon-si, Gyeonggi-do, Republic of Korea
| | - Kunkyu Lee
- Department of Electronic Engineering, Sogang University, 35 Baekbeom-ro, 04107, Seoul, Republic of Korea
| | - Tai-Kyong Song
- Department of Electronic Engineering, Sogang University, 35 Baekbeom-ro, 04107, Seoul, Republic of Korea
| | - Chang Choi
- Department of Computer Engineering, Gachon University, 1342, Seongnam-daero, Sujeong-gu, 13120, Seongnam-si, Gyeonggi-do, Republic of Korea
| | - Moon Hyung Choi
- Department of Radiology, College of Medicine, The Catholic University of Korea, 222 Banpo-daero, Seoul, Republic of Korea.
| | - Changhan Yoon
- Department of Biomedical Engineering, Department of Nanoscience and Engineering, Inje University, Inje-ro 197, 50834, Gimhae, Gyeongnam, Republic of Korea.
| |
Collapse
|
49
|
Ritsche P, Franchi MV, Faude O, Finni T, Seynnes O, Cronin NJ. Fully Automated Analysis of Muscle Architecture from B-Mode Ultrasound Images with DL_Track_US. ULTRASOUND IN MEDICINE & BIOLOGY 2024; 50:258-267. [PMID: 38007322 DOI: 10.1016/j.ultrasmedbio.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/13/2023] [Revised: 10/25/2023] [Accepted: 10/26/2023] [Indexed: 11/27/2023]
Abstract
OBJECTIVE B-mode ultrasound can be used to image musculoskeletal tissues, but one major bottleneck is analyses of muscle architectural parameters (i.e., muscle thickness, pennation angle and fascicle length), which are most often performed manually. METHODS In this study we trained two different neural networks (classic U-Net and U-Net with VGG16 pre-trained encoder) to detect muscle fascicles and aponeuroses using a set of labeled musculoskeletal ultrasound images. We determined the best-performing model based on intersection over union and loss metrics. We then compared neural network predictions on an unseen test set with those obtained via manual analysis and two existing semi/automated analysis approaches (simple muscle architecture analysis [SMA] and UltraTrack). DL_Track_US detects the locations of the superficial and deep aponeuroses, as well as multiple fascicle fragments per image. RESULTS For single images, DL_Track_US yielded results similar to those produced by a non-trainable automated method (SMA; mean difference in fascicle length: 5.1 mm) and human manual analysis (mean difference: -2.4 mm). Between-method differences in pennation angle were within 1.5°, and mean differences in muscle thickness were less than 1 mm. Similarly, for videos, there was overlap between the results produced with UltraTrack and DL_Track_US, with intraclass correlations ranging between 0.19 and 0.88. CONCLUSION DL_Track_US is fully automated and open source and can estimate fascicle length, pennation angle and muscle thickness from single images or videos, as well as from multiple superficial muscles. We also provide a user interface and all necessary code and training data for custom model development.
Collapse
Affiliation(s)
- Paul Ritsche
- Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland.
| | - Martino V Franchi
- Department of Biomedical Sciences, University of Padova, Padova, Italy
| | - Oliver Faude
- Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
| | - Taija Finni
- Faculty of Sport and Health Sciences, University of Jyvaskyla, Jyvaskyla, Finland
| | - Olivier Seynnes
- Department for Physical Performance, Norwegian School of Sport Sciences, Oslo, Norway
| | - Neil J Cronin
- Faculty of Sport and Health Sciences, University of Jyvaskyla, Jyvaskyla, Finland; School of Sport & Exercise, University of Gloucestershire, Gloucester, UK
| |
Collapse
|
50
|
Sirimewan D, Bazli M, Raman S, Mohandes SR, Kineber AF, Arashpour M. Deep learning-based models for environmental management: Recognizing construction, renovation, and demolition waste in-the-wild. JOURNAL OF ENVIRONMENTAL MANAGEMENT 2024; 351:119908. [PMID: 38169254 DOI: 10.1016/j.jenvman.2023.119908] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/02/2023] [Revised: 12/04/2023] [Accepted: 12/11/2023] [Indexed: 01/05/2024]
Abstract
The construction industry generates a substantial volume of solid waste, often destinated for landfills, causing significant environmental pollution. Waste recycling is decisive in managing waste yet challenging due to labor-intensive sorting processes and the diverse forms of waste. Deep learning (DL) models have made remarkable strides in automating domestic waste recognition and sorting. However, the application of DL models to recognize the waste derived from construction, renovation, and demolition (CRD) activities remains limited due to the context-specific studies conducted in previous research. This paper aims to realistically capture the complexity of waste streams in the CRD context. The study encompasses collecting and annotating CRD waste images in real-world, uncontrolled environments. It then evaluates the performance of state-of-the-art DL models for automatically recognizing CRD waste in-the-wild. Several pre-trained networks are utilized to perform effectual feature extraction and transfer learning during DL model training. The results demonstrated that DL models, whether integrated with larger or lightweight backbone networks can recognize the composition of CRD waste streams in-the-wild which is useful for automated waste sorting. The outcome of the study emphasized the applicability of DL models in recognizing and sorting solid waste across various industrial domains, thereby contributing to resource recovery and encouraging environmental management efforts.
Collapse
Affiliation(s)
- Diani Sirimewan
- Department of Civil Engineering, Monash University, Melbourne, Australia.
| | - Milad Bazli
- Faculty of Science and Technology, Charles Darwin University, Australia.
| | - Sudharshan Raman
- Civil Engineering Discipline, School of Engineering, Monash University, Malaysia.
| | | | - Ahmed Farouk Kineber
- Department of Civil Engineering, Prince Sattam Bin Abdulaziz University, Saudi Arabia.
| | - Mehrdad Arashpour
- Department of Civil Engineering, Monash University, Melbourne, Australia.
| |
Collapse
|