1
|
Lavric A, Beguni C, Zadobrischi E, Căilean AM, Avătămăniței SA. A Comprehensive Survey on Emerging Assistive Technologies for Visually Impaired Persons: Lighting the Path with Visible Light Communications and Artificial Intelligence Innovations. SENSORS (BASEL, SWITZERLAND) 2024; 24:4834. [PMID: 39123881 PMCID: PMC11314945 DOI: 10.3390/s24154834] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/04/2024] [Revised: 07/01/2024] [Accepted: 07/23/2024] [Indexed: 08/12/2024]
Abstract
In the context in which severe visual impairment significantly affects human life, this article emphasizes the potential of Artificial Intelligence (AI) and Visible Light Communications (VLC) in developing future assistive technologies. Toward this path, the article summarizes the features of some commercial assistance solutions, and debates the characteristics of VLC and AI, emphasizing their compatibility with blind individuals' needs. Additionally, this work highlights the AI potential in the efficient early detection of eye diseases. This article also reviews the existing work oriented toward VLC integration in blind persons' assistive applications, showing the existing progress and emphasizing the high potential associated with VLC use. In the end, this work provides a roadmap toward the development of an integrated AI-based VLC assistance solution for visually impaired people, pointing out the high potential and some of the steps to follow. As far as we know, this is the first comprehensive work which focuses on the integration of AI and VLC technologies in visually impaired persons' assistance domain.
Collapse
Affiliation(s)
- Alexandru Lavric
- Department of Computers, Electronics and Automation, Stefan cel Mare University of Suceava, 720229 Suceava, Romania; (A.L.); (C.B.); (E.Z.)
| | - Cătălin Beguni
- Department of Computers, Electronics and Automation, Stefan cel Mare University of Suceava, 720229 Suceava, Romania; (A.L.); (C.B.); (E.Z.)
- Integrated Center for Research, Development and Innovation in Advanced Materials, Nanotechnologies and Distributed Systems for Fabrication and Control, Stefan cel Mare University of Suceava, 720229 Suceava, Romania;
| | - Eduard Zadobrischi
- Department of Computers, Electronics and Automation, Stefan cel Mare University of Suceava, 720229 Suceava, Romania; (A.L.); (C.B.); (E.Z.)
- Integrated Center for Research, Development and Innovation in Advanced Materials, Nanotechnologies and Distributed Systems for Fabrication and Control, Stefan cel Mare University of Suceava, 720229 Suceava, Romania;
| | - Alin-Mihai Căilean
- Department of Computers, Electronics and Automation, Stefan cel Mare University of Suceava, 720229 Suceava, Romania; (A.L.); (C.B.); (E.Z.)
- Integrated Center for Research, Development and Innovation in Advanced Materials, Nanotechnologies and Distributed Systems for Fabrication and Control, Stefan cel Mare University of Suceava, 720229 Suceava, Romania;
| | - Sebastian-Andrei Avătămăniței
- Integrated Center for Research, Development and Innovation in Advanced Materials, Nanotechnologies and Distributed Systems for Fabrication and Control, Stefan cel Mare University of Suceava, 720229 Suceava, Romania;
- East European Border Scientific and Technological Park, 725500 Siret, Romania
| |
Collapse
|
2
|
Cao Y, le Yu X, Yao H, Jin Y, Lin K, Shi C, Cheng H, Lin Z, Jiang J, Gao H, Shen M. ScLNet: A cornea with scleral lens OCT layers segmentation dataset and new multi-task model. Heliyon 2024; 10:e33911. [PMID: 39071564 PMCID: PMC11283045 DOI: 10.1016/j.heliyon.2024.e33911] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Revised: 06/28/2024] [Accepted: 06/28/2024] [Indexed: 07/30/2024] Open
Abstract
Objective To develop deep learning methods with high accuracy for segmenting irregular corneas and detecting the tear fluid reservoir (TFR) boundary under the scleral lens. Additionally, this study aims to provide a publicly available cornea with scleral lens OCT dataset, including manually labeled layer masks for training and validation of segmentation algorithms. This study introduces ScLNet, a dataset comprising cornea with Scleral Lens (ScL) optical coherence tomography (OCT) images with layer annotations, and a multi-task network designed to achieve rapid, accurate, automated segmentation of scleral lens with regular and irregular corneas. Methods We created a dataset comprising 31,360 OCT images with scleral lens annotations. The network architecture includes an encoder with multi-scale input and a context coding layer, along with two decoders for specific tasks. The primary task focuses on predicting ScL, TFR, and cornea regions, while the auxiliary task, aimed at predicting the boundaries of ScL, TFR, and cornea, enhances feature extraction for the main task. Segmentation results were compared with state-of-the-art methods and evaluated using Dice similarity coefficient (DSC), intersection over union (IoU), Matthews correlation coefficient (MCC), Precision, and Hausdorff distance (HD). Results ScLNet achieves 98.22 % DSC, 96.50 % IoU, 98.13 % MCC, 98.35 % Precision, and 3.6840 HD (in pixels) in segmenting ScL; 97.78 % DSC, 95.66 % IoU, 97.71 % MCC, 97.70 % Precision, and 3.7838 HD (in pixels) in segmenting TFR; and 99.22 % DSC, 98.45 % IoU, 99.15 % MCC, 99.14 % Precision, and 3.5355 HD (in pixels) in segmenting cornea. The layer interfaces recognized by ScLNet closely align with expert annotations, as evidenced by high IoU scores. Boundary metrics further confirm its effectiveness. Conclusion We constructed a dataset of corneal OCT images with ScL wearing, which includes regular and irregular cornea patients. The proposed ScLNet achieves high accuracy in extracting ScL, TFR, and corneal layer masks and boundaries from OCT images of the dataset.
Collapse
Affiliation(s)
- Yang Cao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Xiang le Yu
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Han Yao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Yue Jin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Kuangqing Lin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Ce Shi
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Hongling Cheng
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Zhiyang Lin
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Jun Jiang
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| | - Hebei Gao
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
- School of Artificial Intelligence, Wenzhou Polytechnic, Wenzhou, 325035, China
| | - Meixiao Shen
- Oujiang Laboratory (Zhejiang Lab for Regenerative Medicine, Vision and Brain Health), Eye Hospital and School of Ophthalmology and Optometry, Wenzhou Medical University, Wenzhou, 325000, China
| |
Collapse
|
3
|
Yang X, Zheng Y, Mei C, Jiang G, Tian B, Wang L. UGLS: an uncertainty guided deep learning strategy for accurate image segmentation. Front Physiol 2024; 15:1362386. [PMID: 38651048 PMCID: PMC11033460 DOI: 10.3389/fphys.2024.1362386] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2024] [Accepted: 03/26/2024] [Indexed: 04/25/2024] Open
Abstract
Accurate image segmentation plays a crucial role in computer vision and medical image analysis. In this study, we developed a novel uncertainty guided deep learning strategy (UGLS) to enhance the performance of an existing neural network (i.e., U-Net) in segmenting multiple objects of interest from images with varying modalities. In the developed UGLS, a boundary uncertainty map was introduced for each object based on its coarse segmentation (obtained by the U-Net) and then combined with input images for the fine segmentation of the objects. We validated the developed method by segmenting optic cup (OC) regions from color fundus images and left and right lung regions from Xray images. Experiments on public fundus and Xray image datasets showed that the developed method achieved a average Dice Score (DS) of 0.8791 and a sensitivity (SEN) of 0.8858 for the OC segmentation, and 0.9605, 0.9607, 0.9621, and 0.9668 for the left and right lung segmentation, respectively. Our method significantly improved the segmentation performance of the U-Net, making it comparable or superior to five sophisticated networks (i.e., AU-Net, BiO-Net, AS-Net, Swin-Unet, and TransUNet).
Collapse
Affiliation(s)
- Xiaoguo Yang
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Yanyan Zheng
- Wenzhou People’s Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou, China
| | - Chenyang Mei
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Gaoqiang Jiang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Bihan Tian
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| | - Lei Wang
- School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
4
|
Mei C, Yang X, Zhou M, Zhang S, Chen H, Yang X, Wang L. Semi-supervised image segmentation using a residual-driven mean teacher and an exponential Dice loss. Artif Intell Med 2024; 148:102757. [PMID: 38325920 DOI: 10.1016/j.artmed.2023.102757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Revised: 11/13/2023] [Accepted: 12/29/2023] [Indexed: 02/09/2024]
Abstract
Semi-supervised segmentation plays an important role in computer vision and medical image analysis and can alleviate the burden of acquiring abundant expert-annotated images. In this paper, we developed a residual-driven semi-supervised segmentation method (termed RDMT) based on the classical mean teacher (MT) framework by introducing a novel model-level residual perturbation and an exponential Dice (eDice) loss. The introduced perturbation was integrated into the exponential moving average (EMA) scheme to enhance the performance of the MT, while the eDice loss was used to improve the detection sensitivity of a given network to object boundaries. We validated the developed method by applying it to segment 3D Left Atrium (LA) and 2D optic cup (OC) from the public LASC and REFUGE datasets based on the V-Net and U-Net, respectively. Extensive experiments demonstrated that the developed method achieved the average Dice score of 0.8776 and 0.7751, when trained on 10% and 20% labeled images, respectively for the LA and OC regions depicted on the LASC and REFUGE datasets. It significantly outperformed the MT and can compete with several existing semi-supervised segmentation methods (i.e., HCMT, UAMT, DTC and SASS).
Collapse
Affiliation(s)
- Chenyang Mei
- School of Ophthalmology & Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou 325027, China
| | - Xiaoguo Yang
- Department of Neurology, Wenzhou People's Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou 325041, China
| | - Mi Zhou
- Department of Neurology, Wenzhou People's Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou 325041, China; School of Medicine, Shanghai University, Shanghai 200444, China
| | - Shaodan Zhang
- School of Ophthalmology & Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou 325027, China
| | - Hao Chen
- School of Ophthalmology & Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou 325027, China
| | - Xiaokai Yang
- Department of Neurology, Wenzhou People's Hospital, The Third Affiliated Hospital of Shanghai University, Wenzhou 325041, China; School of Medicine, Shanghai University, Shanghai 200444, China.
| | - Lei Wang
- School of Ophthalmology & Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou 325027, China.
| |
Collapse
|
5
|
Pu J, Gezer NS, Ren S, Alpaydin AO, Avci ER, Risbano MG, Rivera-Lebron B, Chan SYW, Leader JK. Automated detection and segmentation of pulmonary embolisms on computed tomography pulmonary angiography (CTPA) using deep learning but without manual outlining. Med Image Anal 2023; 89:102882. [PMID: 37482032 PMCID: PMC10528048 DOI: 10.1016/j.media.2023.102882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Revised: 05/26/2023] [Accepted: 06/26/2023] [Indexed: 07/25/2023]
Abstract
We present a novel computer algorithm to automatically detect and segment pulmonary embolisms (PEs) on computed tomography pulmonary angiography (CTPA). This algorithm is based on deep learning but does not require manual outlines of the PE regions. Given a CTPA scan, both intra- and extra-pulmonary arteries were firstly segmented. The arteries were then partitioned into several parts based on size (radius). Adaptive thresholding and constrained morphological operations were used to identify suspicious PE regions within each part. The confidence of a suspicious region to be PE was scored based on its contrast in the arteries. This approach was applied to the publicly available RSNA Pulmonary Embolism CT Dataset (RSNA-PE) to identify three-dimensional (3-D) PE negative and positive image patches, which were used to train a 3-D Recurrent Residual U-Net (R2-Unet) to automatically segment PE. The feasibility of this computer algorithm was validated on an independent test set consisting of 91 CTPA scans acquired from a different medical institute, where the PE regions were manually located and outlined by a thoracic radiologist (>18 years' experience). An R2-Unet model was also trained and validated on the manual outlines using a 5-fold cross-validation method. The CNN model trained on the high-confident PE regions showed a Dice coefficient of 0.676±0.168 and a false positive rate of 1.86 per CT scan, while the CNN model trained on the manual outlines demonstrated a Dice coefficient of 0.647±0.192 and a false positive rate of 4.20 per CT scan. The former model performed significantly better than the latter model (p<0.01). The promising performance of the developed PE detection and segmentation algorithm suggests the feasibility of training a deep learning network without dedicating significant efforts to manual annotations of the PE regions on CTPA scans.
Collapse
Affiliation(s)
- Jiantao Pu
- Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA; Department of Bioengineering, University of Pittsburgh, Pittsburgh, PA 15213, USA.
| | | | - Shangsi Ren
- Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| | | | - Emre Ruhat Avci
- Department of Radiology, Dokuz Eylul University, Izmir, Turkey
| | - Michael G Risbano
- Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | | | | | - Joseph K Leader
- Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| |
Collapse
|
6
|
Pu L, Leader JK, Ali A, Geng Z, Wilson D. Predicting left/right lung volumes, thoracic cavity volume, and heart volume from subject demographics to improve lung transplant. J Med Imaging (Bellingham) 2023; 10:051806. [PMID: 37077858 PMCID: PMC10108239 DOI: 10.1117/1.jmi.10.5.051806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2022] [Accepted: 03/21/2023] [Indexed: 04/21/2023] Open
Abstract
Purpose Lung transplantation is the standard treatment for end-stage lung diseases. A crucial factor affecting its success is size matching between the donor's lungs and the recipient's thorax. Computed tomography (CT) scans can accurately determine recipient's lung size, but donor's lung size is often unknown due to the absence of medical images. We aim to predict donor's right/left/total lung volume, thoracic cavity, and heart volume from only subject demographics to improve the accuracy of size matching. Approach A cohort of 4610 subjects with chest CT scans and basic demographics (i.e., age, gender, race, smoking status, smoking history, weight, and height) was used in this study. The right and left lungs, thoracic cavity, and heart depicted on chest CT scans were automatically segmented using U-Net, and their volumes were computed. Eight machine learning models [i.e., random forest, multivariate linear regression, support vector machine, extreme gradient boosting (XGBoost), multilayer perceptron (MLP), decision tree, k -nearest neighbors, and Bayesian regression) were developed and used to predict the volume measures from subject demographics. The 10-fold cross-validation method was used to evaluate the performances of the prediction models. R -squared (R 2 ), mean absolute error (MAE), and mean absolute percentage error (MAPE) were used as performance metrics. Results The MLP model demonstrated the best performance for predicting the thoracic cavity volume (R 2 : 0.628, MAE: 0.736 L, MAPE: 10.9%), right lung volume (R 2 : 0.501, MAE: 0.383 L, MAPE: 13.9%), and left lung volume (R 2 : 0.507, MAE: 0.365 L, MAPE: 15.2%), and the XGBoost model demonstrated the best performance for predicting the total lung volume (R 2 : 0.514, MAE: 0.728 L, MAPE: 14.0%) and heart volume (R 2 : 0.430, MAE: 0.075 L, MAPE: 13.9%). Conclusions Our results demonstrate the feasibility of predicting lung, heart, and thoracic cavity volumes from subject demographics with superior performance compared with available studies in predicting lung volumes.
Collapse
Affiliation(s)
- Lucas Pu
- North Allegheny High School, Wexford, Pennsylvania, United States
| | - Joseph K Leader
- University of Pittsburgh School of Medicine, Department of Radiology, Pittsburgh, Pennsylvania, United States
| | - Alaa Ali
- University of Pittsburgh School of Medicine, Department of Radiology, Pittsburgh, Pennsylvania, United States
| | - Zihan Geng
- Carnegie Mellon University, Department of Statistics and Data Science, Pittsburgh, Pennsylvania, United States
| | - David Wilson
- University of Pittsburgh School of Medicine, Department of Medicine, Pittsburgh, Pennsylvania, United States
| |
Collapse
|
7
|
Hu K, Zhang X, Lee D, Xiong D, Zhang Y, Gao X. Boundary-Guided and Region-Aware Network With Global Scale-Adaptive for Accurate Segmentation of Breast Tumors in Ultrasound Images. IEEE J Biomed Health Inform 2023; 27:4421-4432. [PMID: 37310830 DOI: 10.1109/jbhi.2023.3285789] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Breast ultrasound (BUS) image segmentation is a critical procedure in the diagnosis and quantitative analysis of breast cancer. Most existing methods for BUS image segmentation do not effectively utilize the prior information extracted from the images. In addition, breast tumors have very blurred boundaries, various sizes and irregular shapes, and the images have a lot of noise. Thus, tumor segmentation remains a challenge. In this article, we propose a BUS image segmentation method using a boundary-guided and region-aware network with global scale-adaptive (BGRA-GSA). Specifically, we first design a global scale-adaptive module (GSAM) to extract features of tumors of different sizes from multiple perspectives. GSAM encodes the features at the top of the network in both channel and spatial dimensions, which can effectively extract multi-scale context and provide global prior information. Moreover, we develop a boundary-guided module (BGM) for fully mining boundary information. BGM guides the decoder to learn the boundary context by explicitly enhancing the extracted boundary features. Simultaneously, we design a region-aware module (RAM) for realizing the cross-fusion of diverse layers of breast tumor diversity features, which can facilitate the network to improve the learning ability of contextual features of tumor regions. These modules enable our BGRA-GSA to capture and integrate rich global multi-scale context, multi-level fine-grained details, and semantic information to facilitate accurate breast tumor segmentation. Finally, the experimental results on three publicly available datasets show that our model achieves highly effective segmentation of breast tumors even with blurred boundaries, various sizes and shapes, and low contrast.
Collapse
|
8
|
Pu L, Gezer NS, Ashraf SF, Ocak I, Dresser DE, Dhupar R. Automated segmentation of five different body tissues on computed tomography using deep learning. Med Phys 2023; 50:178-191. [PMID: 36008356 PMCID: PMC11186697 DOI: 10.1002/mp.15932] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 07/27/2020] [Accepted: 08/04/2022] [Indexed: 01/25/2023] Open
Abstract
PURPOSE To develop and validate a computer tool for automatic and simultaneous segmentation of five body tissues depicted on computed tomography (CT) scans: visceral adipose tissue (VAT), subcutaneous adipose tissue (SAT), intermuscular adipose tissue (IMAT), skeletal muscle (SM), and bone. METHODS A cohort of 100 CT scans acquired on different subjects were collected from The Cancer Imaging Archive-50 whole-body positron emission tomography-CTs, 25 chest, and 25 abdominal. Five different body tissues (i.e., VAT, SAT, IMAT, SM, and bone) were manually annotated. A training-while-annotating strategy was used to improve the annotation efficiency. The 10-fold cross-validation method was used to develop and validate the performance of several convolutional neural networks (CNNs), including UNet, Recurrent Residual UNet (R2Unet), and UNet++. A grid-based three-dimensional patch sampling operation was used to train the CNN models. The CNN models were also trained and tested separately for each body tissue to see if they could achieve a better performance than segmenting them jointly. The paired sample t-test was used to statistically assess the performance differences among the involved CNN models RESULTS: When segmenting the five body tissues simultaneously, the Dice coefficients ranged from 0.826 to 0.840 for VAT, from 0.901 to 0.908 for SAT, from 0.574 to 0.611 for IMAT, from 0.874 to 0.889 for SM, and from 0.870 to 0.884 for bone, which were significantly higher than the Dice coefficients when segmenting the body tissues separately (p < 0.05), namely, from 0.744 to 0.819 for VAT, from 0.856 to 0.896 for SAT, from 0.433 to 0.590 for IMAT, from 0.838 to 0.871 for SM, and from 0.803 to 0.870 for bone. CONCLUSION There were no significant differences among the CNN models in segmenting body tissues, but jointly segmenting body tissues achieved a better performance than segmenting them separately.
Collapse
Affiliation(s)
- Lucy Pu
- Department, of Cardiothoracic Surgery, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
- North Allegheny Senior High School, Wexford, USA
| | - Naciye S Gezer
- Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | | | - Iclal Ocak
- Department of Radiology, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Daniel E. Dresser
- Department of Pathology, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
| | - Rajeev Dhupar
- Department, of Cardiothoracic Surgery, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USA
- Surgical Services Division, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania, USA
| |
Collapse
|
9
|
Karri M, Annavarapu CSR, Acharya UR. Explainable multi-module semantic guided attention based network for medical image segmentation. Comput Biol Med 2022; 151:106231. [PMID: 36335811 DOI: 10.1016/j.compbiomed.2022.106231] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 10/05/2022] [Accepted: 10/11/2022] [Indexed: 12/27/2022]
Abstract
Automated segmentation of medical images is crucial for disease diagnosis and treatment planning. Medical image segmentation has been improved based on the convolutional neural networks (CNNs) models. Unfortunately, they are still limited by scenarios in which the segmentation objective has large variations in size, boundary, position, and shape. Moreover, current CNNs have low explainability, restricting their use in clinical decisions. In this paper, we involve substantial use of various attentions in a CNN model and present an explainable multi-module semantic guided attention based network (MSGA-Net) for explainable and highly accurate medical image segmentation, which involves considering the most significant spatial regions, boundaries, scales, and channels. Specifically, we present a multi-scale attention module (MSA) to extract the most salient features at various scales from medical images. Then, we propose a semantic region-guided attention mechanism (SRGA) including location attention (LAM), channel-wise attention (CWA), and edge attention (EA) modules to extract the most important spatial, channel-wise, boundary-related features for interested regions. Moreover, we present a sequence of fine-tuning steps with the SRGA module to gradually weight the significance of interesting regions while simultaneously reducing the noise. In this work, we experimented with three different types of medical images such as dermoscopic images (HAM10000 dataset), multi-organ CT images (CHAOS 2019 dataset), and Brain tumor MRI images (BraTS 2020 dataset). Extensive experiments on all types of medical images revealed that our proposed MSGA-Net substantially increased the overall performance of all metrics over the existing models. Moreover, displaying the attention feature maps has more explainability than state-of-the-art models.
Collapse
Affiliation(s)
- Meghana Karri
- Computer Science and Engineering Department, Indian Institute of Technology (ISM), Dhanbad, 826004, Jharkhand, India.
| | | | - U Rajendra Acharya
- Ngee Ann Polytechnic, Department of Electronics and Computer Engineering, 599489, Singapore; Department of Biomedical Engineering, School of science and Technology, SUSS university, Singapore; Department of Biomedical Informatics and Medical Engineering, Asia university, Taichung, Taiwan.
| |
Collapse
|
10
|
Zheng Q, Zhang X, Zhang J, Bai F, Huang S, Pu J, Chen W, Wang L. A texture-aware U-Net for identifying incomplete blinking from eye videography. Biomed Signal Process Control 2022; 75. [DOI: 10.1016/j.bspc.2022.103630] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
|
11
|
Shi C, Zhang J, Zhang X, Shen M, Chen H, Wang L. A recurrent skip deep learning network for accurate image segmentation. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103533] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
|
12
|
Wang L, Shen M, Shi C, Zhou Y, Chen Y, Pu J, Chen H. EE-Net: An edge-enhanced deep learning network for jointly identifying corneal micro-layers from optical coherence tomography. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2021.103213] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/06/2023]
|