1
|
Gou F, Liu J, Xiao C, Wu J. Research on Artificial-Intelligence-Assisted Medicine: A Survey on Medical Artificial Intelligence. Diagnostics (Basel) 2024; 14:1472. [PMID: 39061610 PMCID: PMC11275417 DOI: 10.3390/diagnostics14141472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2024] [Revised: 07/04/2024] [Accepted: 07/05/2024] [Indexed: 07/28/2024] Open
Abstract
With the improvement of economic conditions and the increase in living standards, people's attention in regard to health is also continuously increasing. They are beginning to place their hopes on machines, expecting artificial intelligence (AI) to provide a more humanized medical environment and personalized services, thus greatly expanding the supply and bridging the gap between resource supply and demand. With the development of IoT technology, the arrival of the 5G and 6G communication era, and the enhancement of computing capabilities in particular, the development and application of AI-assisted healthcare have been further promoted. Currently, research on and the application of artificial intelligence in the field of medical assistance are continuously deepening and expanding. AI holds immense economic value and has many potential applications in regard to medical institutions, patients, and healthcare professionals. It has the ability to enhance medical efficiency, reduce healthcare costs, improve the quality of healthcare services, and provide a more intelligent and humanized service experience for healthcare professionals and patients. This study elaborates on AI development history and development timelines in the medical field, types of AI technologies in healthcare informatics, the application of AI in the medical field, and opportunities and challenges of AI in the field of medicine. The combination of healthcare and artificial intelligence has a profound impact on human life, improving human health levels and quality of life and changing human lifestyles.
Collapse
Affiliation(s)
- Fangfang Gou
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
| | - Jun Liu
- The Second People's Hospital of Huaihua, Huaihua 418000, China
| | - Chunwen Xiao
- The Second People's Hospital of Huaihua, Huaihua 418000, China
| | - Jia Wu
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton, VIC 3800, Australia
| |
Collapse
|
2
|
Zhai C, Xu J, Yang Y, Xie F, Cao L, Wang K, Zhou Y, Ding X, Yin J, Ding X, Hu H, Yu H. Heterogeneous Analysis of Extracellular Vesicles for Osteosarcoma Diagnosis. Anal Chem 2024; 96:9486-9492. [PMID: 38814722 DOI: 10.1021/acs.analchem.4c00941] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/01/2024]
Abstract
Osteosarcoma (OS) is the most prevalent primary tumor of bones, often diagnosed late with a poor prognosis. Currently, few effective biomarkers or diagnostic methods have been developed for early OS detection with high confidence, especially for metastatic OS. Tumor-derived extracellular vesicles (EVs) are emerging as promising biomarkers for early cancer diagnosis through liquid biopsy. Here, we report a plasmonic imaging-based biosensing technique, termed subpopulation protein analysis by single EV counting (SPASEC), for size-dependent EV subpopulation analysis. In our SPASEC platform, EVs are accurately sized and counted on plasmonic sensor chips coated with OS-specific antibodies. Subsequently, EVs are categorized into distinct subpopulations based on their sizes, and the membrane proteins of each size-dependent subpopulation are profiled. We measured the heterogeneous expression levels of the EV markers (CD63, BMP2, GD2, and N-cadherin) in each of the EV subsets from both OS cell lines and clinical plasma samples. Using the linear discriminant analysis (LDA) model, the combination of four markers is applied to classify the healthy donors (n = 37), nonmetastatic OS patients (n = 13), and metastatic patients (n = 12) with an area under the curve of 0.95, 0.92, and 0.99, respectively. SPASEC provides accurate EV sensing technology for early OS diagnosis.
Collapse
Affiliation(s)
- Chunhui Zhai
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Jiaying Xu
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Yuting Yang
- School of Sensing Science and Engineering, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Feng Xie
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Li Cao
- Shanghai Clinical Research Ward (SCRW), Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200233, China
| | - Kai Wang
- Shanghai Clinical Research Ward (SCRW), Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200233, China
| | - Yan Zhou
- Department of Oncology, Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200233, China
| | - Xiaomin Ding
- Shanghai Clinical Research Ward (SCRW), Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200233, China
| | - Junyi Yin
- Oncology Department of Tongji Hospital of Tongji University, No. 389 Xincun Road, Shanghai, 200065, China
| | - Xianting Ding
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Haiyan Hu
- Shanghai Clinical Research Ward (SCRW), Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200233, China
| | - Hui Yu
- School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200030, China
| |
Collapse
|
3
|
Liu X, Du P, Dai Z, Yi R, Liu W, Wu H, Geng D, Liu J. SRTRP-Net: A multi-task learning network for segmentation and prediction of stereotactic radiosurgery treatment response in brain metastases. Comput Biol Med 2024; 175:108503. [PMID: 38688125 DOI: 10.1016/j.compbiomed.2024.108503] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Revised: 03/30/2024] [Accepted: 04/21/2024] [Indexed: 05/02/2024]
Abstract
Before the Stereotactic Radiosurgery (SRS) treatment, it is of great clinical significance to avoid secondary genetic damage and guide the personalized treatment plans for patients with brain metastases (BM) by predicting the response to SRS treatment of brain metastatic lesions. Thus, we developed a multi-task learning model termed SRTRP-Net to provide prior knowledge of BM ROI and predict the SRS treatment response of the lesion. In dual-encoder tumor segmentation Network (DTS-Net), two parallel encoders encode the original and mirrored multi-modal MRI images. The differences in the dual-encoder features between foreground and background are enhanced by the symmetrical visual difference block (SVDB). In the bottom layer of the encoder, a transformer is used to extract local contextual features in the spatial and depth dimensions of low-resolution images. Then, the decoder of DTS-Net provides the prior knowledge for predicting the response to SRS treatment by performing BM segmentation. SRS response prediction network (SRP-Net) directly utilizes shared multi-modal MRI features weighted by the signed distance map (SDM) of the masks. The bidirectional multi-dimensional feature fusion module (BMDF) fuses the shared features and the clinical text information features to obtain comprehensive tumor information for characterizing tumors and predicting SRS treatment response. Experiments based on internal and external clinical datasets have shown that SRTRP-Net achieves comparable or better results. We believe that SRTRP-Net can help clinicians accurately develop personalized first-time treatment regimens for BM patients and improve their survival.
Collapse
Affiliation(s)
- Xiao Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100089, China.
| | - Peng Du
- Department of Radiology, the Second Affiliated Hospital of Xuzhou Medical University, Jiangsu, 221000, China.
| | - Zhiguang Dai
- CSSC Systems Engineering Research Institute, Beijing, 100094, China.
| | - Rumeng Yi
- CSSC Systems Engineering Research Institute, Beijing, 100094, China.
| | - Weifan Liu
- College of Science, Beijing Forestry University, Beijing, 100089, China.
| | - Hao Wu
- Huashan Hospital, Fudan University, Shanghai, 200020, China.
| | - Daoying Geng
- Huashan Hospital, Fudan University, Shanghai, 200020, China.
| | - Jie Liu
- School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100089, China.
| |
Collapse
|
4
|
Wu Y, Li J, Wang X, Zhang Z, Zhao S. DECIDE: A decoupled semantic and boundary learning network for precise osteosarcoma segmentation by integrating multi-modality MRI. Comput Biol Med 2024; 174:108308. [PMID: 38581998 DOI: 10.1016/j.compbiomed.2024.108308] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/17/2024] [Accepted: 03/12/2024] [Indexed: 04/08/2024]
Abstract
Automated Osteosarcoma Segmentation in Multi-modality MRI (AOSMM) holds clinical significance for effective tumor evaluation and treatment planning. However, the precision of AOSMM is challenged by the diverse characteristics of multi-modality MRI and the inherent heterogeneity and boundary ambiguity of osteosarcoma. While numerous methods have made significant strides in automated osteosarcoma segmentation, they primarily focused on the use of a single MRI modality and overlooked the potential benefits of integrating complementary information from other MRI modalities. Furthermore, they did not adequately model the long-range dependencies of complex tumor features, which may lead to insufficiently discriminative feature representations. To this end, we propose a decoupled semantic and boundary learning network (DECIDE) to achieve precise AOSMM with three functional modules. The Multi-modality Feature Fusion and Recalibration (MFR) module adaptively fuses and recalibrates multi-modality features by exploiting their channel-wise dependencies to compute low-rank attention weights for effectively aggregating useful information from different MRI modalities, which promotes complementary learning between multi-modality MRI and enables a more comprehensive tumor characterization. The Lesion Attention Enhancement (LAE) module employs spatial and channel attention mechanisms to capture global contextual dependencies over local features, significantly enhancing the discriminability and representational capacity of intricate tumor features. The Boundary Context Aggregation (BCA) module further enhances semantic representations by utilizing boundary information for effective context aggregation while also ensuring intra-class consistency in cases of boundary ambiguity. Substantial experiments demonstrate that DECIDE achieves exceptional performance in osteosarcoma segmentation, surpassing state-of-the-art methods in terms of accuracy and stability.
Collapse
Affiliation(s)
- Yinhao Wu
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China
| | - Jianqi Li
- The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, 510080, China
| | - Xinxin Wang
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China
| | - Zhaohui Zhang
- The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, 510080, China.
| | - Shen Zhao
- Department of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, 518107, China.
| |
Collapse
|
5
|
Zhou Z, Xie P, Dai Z, Wu J. Self-supervised tumor segmentation and prognosis prediction in osteosarcoma using multiparametric MRI and clinical characteristics. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2024; 244:107974. [PMID: 38154327 DOI: 10.1016/j.cmpb.2023.107974] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2022] [Revised: 11/26/2023] [Accepted: 12/07/2023] [Indexed: 12/30/2023]
Abstract
BACKGROUND AND OBJECTIVE Osteosarcoma has a high mortality among malignant bone tumors. MRI-based tumor segmentation and prognosis prediction are helpful to assist doctors in detecting osteosarcoma, evaluating the patient's status, and improving patient survival. Current intelligent diagnostic approaches focus on segmentation with single-parameter MRI, which ignores the nature of MRI resulting in poor performance, and lacks the connection with prognosis prediction. Besides, osteosarcoma is a rare disease, and their few labeled data may lead to model overfitting. METHODS We propose a three-stage pipeline for segmentation and prognosis prediction of osteosarcoma to assist doctors in diagnosis. First, we propose the Multiparameter Fusion Contrast Learning (MPFCLR) algorithm to share pre-training weights for the segmentation model using unlabeled data. Then, we construct a multiparametric fusion network (MPFNet), which fuses the complementary features from multiparametric MRI (CE-T1WI, T2WI). It can automatically segment tumor and necrotic regions. Finally, a fusion nomogram is constructed by segmentation masks and clinical characteristics (volume, tumor spread) to predict the patient's prognostic status. RESULTS Our experiments used data from 136 patients at the Second Xiangya Hospital in China. According to experiments, the MPFNet achieves 84.19 % mean DSC and 84.56 % mean F1-score in segmenting tumor and necrotic regions, surpassing existing models and single-parameter MRI input for osteosarcoma segmentation. Besides, MPFCLR improves the segmentation performance and convergence speed. In prognosis prediction, our fusion nomogram (C-index: 0.806, 95 %CI: 0.758-0.854) is better than radiomics (C-index: 0.753, 95 %CI: 0.685-0.841) and clinical (C-index: 0.794, 95 %CI: 0.735-0.854) nomograms in predictive performance. Compared to the comparison models, our model is closest to the prediction model based on physician annotations. Moreover, it can accurately distinguish the patients' prognostic status with good or poor. CONCLUSION Our proposed solution can provide references for clinicians to detect osteosarcoma, evaluate patient status, and make personalized decisions. It can reduce delayed treatment or overtreatment and improve patient survival.
Collapse
Affiliation(s)
- Zhixun Zhou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Peng Xie
- Department of Spine Surgery, The Second Xiangya Hospital, Central South University, Changsha 410011, China
| | - Zhehao Dai
- Department of Spine Surgery, The Second Xiangya Hospital, Central South University, Changsha 410011, China
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China; Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton VIC 3800, Australia.
| |
Collapse
|
6
|
He Z, Liu J, Gou F, Wu J. An Innovative Solution Based on TSCA-ViT for Osteosarcoma Diagnosis in Resource-Limited Settings. Biomedicines 2023; 11:2740. [PMID: 37893113 PMCID: PMC10604772 DOI: 10.3390/biomedicines11102740] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 09/24/2023] [Accepted: 10/08/2023] [Indexed: 10/29/2023] Open
Abstract
Identifying and managing osteosarcoma pose significant challenges, especially in resource-constrained developing nations. Advanced diagnostic methods involve isolating the nucleus from cancer cells for comprehensive analysis. However, two main challenges persist: mitigating image noise during the capture and transmission of cellular sections, and providing an efficient, accurate, and cost-effective solution for cell nucleus segmentation. To tackle these issues, we introduce the Twin-Self and Cross-Attention Vision Transformer (TSCA-ViT). This pioneering AI-based system employs a directed filtering algorithm for noise reduction and features an innovative transformer architecture with a twin attention mechanism for effective segmentation. The model also incorporates cross-attention-enabled skip connections to augment spatial information. We evaluated our method on a dataset of 1000 osteosarcoma pathology slide images from the Second People's Hospital of Huaihua, achieving a remarkable average precision of 97.7%. This performance surpasses traditional methodologies. Furthermore, TSCA-ViT offers enhanced computational efficiency owing to its fewer parameters, which results in reduced time and equipment costs. These findings underscore the superior efficacy and efficiency of TSCA-ViT, offering a promising approach for addressing the ongoing challenges in osteosarcoma diagnosis and treatment, particularly in settings with limited resources.
Collapse
Affiliation(s)
- Zengxiao He
- School of Computer Science and Engineering, Central South University, Changsha 410083, China;
| | - Jun Liu
- The Second People’s Hospital of Huaihua, Huaihua 418000, China
| | - Fangfang Gou
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China;
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China;
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China;
- Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton, VIC 3800, Australia
| |
Collapse
|
7
|
Rich JM, Bhardwaj LN, Shah A, Gangal K, Rapaka MS, Oberai AA, Fields BKK, Matcuk GR, Duddalwar VA. Deep learning image segmentation approaches for malignant bone lesions: a systematic review and meta-analysis. FRONTIERS IN RADIOLOGY 2023; 3:1241651. [PMID: 37614529 PMCID: PMC10442705 DOI: 10.3389/fradi.2023.1241651] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 07/28/2023] [Indexed: 08/25/2023]
Abstract
Introduction Image segmentation is an important process for quantifying characteristics of malignant bone lesions, but this task is challenging and laborious for radiologists. Deep learning has shown promise in automating image segmentation in radiology, including for malignant bone lesions. The purpose of this review is to investigate deep learning-based image segmentation methods for malignant bone lesions on Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and Positron-Emission Tomography/CT (PET/CT). Method The literature search of deep learning-based image segmentation of malignant bony lesions on CT and MRI was conducted in PubMed, Embase, Web of Science, and Scopus electronic databases following the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). A total of 41 original articles published between February 2017 and March 2023 were included in the review. Results The majority of papers studied MRI, followed by CT, PET/CT, and PET/MRI. There was relatively even distribution of papers studying primary vs. secondary malignancies, as well as utilizing 3-dimensional vs. 2-dimensional data. Many papers utilize custom built models as a modification or variation of U-Net. The most common metric for evaluation was the dice similarity coefficient (DSC). Most models achieved a DSC above 0.6, with medians for all imaging modalities between 0.85-0.9. Discussion Deep learning methods show promising ability to segment malignant osseous lesions on CT, MRI, and PET/CT. Some strategies which are commonly applied to help improve performance include data augmentation, utilization of large public datasets, preprocessing including denoising and cropping, and U-Net architecture modification. Future directions include overcoming dataset and annotation homogeneity and generalizing for clinical applicability.
Collapse
Affiliation(s)
- Joseph M. Rich
- Keck School of Medicine, University of Southern California, Los Angeles, CA, United States
| | - Lokesh N. Bhardwaj
- Keck School of Medicine, University of Southern California, Los Angeles, CA, United States
| | - Aman Shah
- Department of Applied Biostatistics and Epidemiology, University of Southern California, Los Angeles, CA, United States
| | - Krish Gangal
- Bridge UnderGrad Science Summer Research Program, Irvington High School, Fremont, CA, United States
| | - Mohitha S. Rapaka
- Department of Biology, University of Texas at Austin, Austin, TX, United States
| | - Assad A. Oberai
- Department of Aerospace and Mechanical Engineering Department, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, United States
| | - Brandon K. K. Fields
- Department of Radiology & Biomedical Imaging, University of California, San Francisco, San Francisco, CA, United States
| | - George R. Matcuk
- Department of Radiology, Cedars-Sinai Medical Center, Los Angeles, CA, United States
| | - Vinay A. Duddalwar
- Department of Radiology, Keck School of Medicine of the University of Southern California, Los Angeles, CA, United States
- Department of Radiology, USC Radiomics Laboratory, Keck School of Medicine, University of Southern California, Los Angeles, CA, United States
| |
Collapse
|
8
|
Wu J, Yuan T, Zeng J, Gou F. A Medically Assisted Model for Precise Segmentation of Osteosarcoma Nuclei on Pathological Images. IEEE J Biomed Health Inform 2023; 27:3982-3993. [PMID: 37216252 DOI: 10.1109/jbhi.2023.3278303] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Osteosarcoma is the most common malignant bone tumor with a high degree of malignancy and misdiagnosis rates. Pathological images are crucial for its diagnosis. However, underdeveloped regions currently lack sufficient high-level pathologists, leading to uncertain diagnostic accuracy and efficiency. Existing research on pathological image segmentation often neglects the differences in staining styles and lack of data, without considering medical backgrounds. To alleviate the difficulty in diagnosing osteosarcoma in underdeveloped areas, an intelligent assisted diagnosis and treatment scheme for osteosarcoma pathological images, ENMViT, is proposed. ENMViT utilizes KIN to achieve normalization of mismatched images with limited GPU resources and uses traditional data enhancement methods, such as cleaning, cropping, mosaic, Laplacian sharpening, and other techniques to alleviate the issue of insufficient data. A multi-path semantic segmentation network combining Transformer and CNN is used to segment images, and the degree of edge offset in the spatial domain is introduced into the loss function. Finally, noise is filtered according to the size of the connecting domain. This article experimented on more than 2000 osteosarcoma pathological images from Central South University. The experimental results demonstrate that this scheme performs well in each stage of the osteosarcoma pathological image processing, and the segmentation results' IoU index is 9.4% higher than the comparative models, demonstrating its significant value in the medical industry.
Collapse
|
9
|
Yildiz Potter I, Yeritsyan D, Mahar S, Wu J, Nazarian A, Vaziri A, Vaziri A. Automated Bone Tumor Segmentation and Classification as Benign or Malignant Using Computed Tomographic Imaging. J Digit Imaging 2023; 36:869-878. [PMID: 36627518 PMCID: PMC10287871 DOI: 10.1007/s10278-022-00771-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2022] [Revised: 12/23/2022] [Accepted: 12/27/2022] [Indexed: 01/12/2023] Open
Abstract
The purpose of this study was to pair computed tomography (CT) imaging and machine learning for automated bone tumor segmentation and classification to aid clinicians in determining the need for biopsy. In this retrospective study (March 2005-October 2020), a dataset of 84 femur CT scans (50 females and 34 males, 20 years and older) with definitive histologic confirmation of bone lesion (71% malignant) were leveraged to perform automated tumor segmentation and classification. Our method involves a deep learning architecture that receives a DICOM slice and predicts (i) a segmentation mask over the estimated tumor region, and (ii) a corresponding class as benign or malignant. Class prediction for each case is then determined via majority voting. Statistical analysis was conducted via fivefold cross validation, with results reported as averages along with 95% confidence intervals. Despite the imbalance between benign and malignant cases in our dataset, our approach attains similar classification performances in specificity (75%) and sensitivity (79%). Average segmentation performance attains 56% Dice score and reaches up to 80% for an image slice in each scan. The proposed approach establishes the first steps in developing an automated deep learning method on bone tumor segmentation and classification from CT imaging. Our approach attains comparable quantitative performance to existing deep learning models using other imaging modalities, including X-ray. Moreover, visual analysis of bone tumor segmentation indicates that our model is capable of learning typical tumor characteristics and provides a promising direction in aiding the clinical decision process for biopsy.
Collapse
Affiliation(s)
| | - Diana Yeritsyan
- Beth Israel Deaconess Medical Center (BIDMC), Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, USA
| | - Sarah Mahar
- Beth Israel Deaconess Medical Center (BIDMC), Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, USA
| | - Jim Wu
- Beth Israel Deaconess Medical Center (BIDMC), Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, USA
| | - Ara Nazarian
- Beth Israel Deaconess Medical Center (BIDMC), Harvard Medical School, 330 Brookline Avenue, Boston, MA, 02215, USA
| | - Aidin Vaziri
- BioSensics LLC, 57 Chapel Street, Newton, MA, 02458, USA
| | - Ashkan Vaziri
- BioSensics LLC, 57 Chapel Street, Newton, MA, 02458, USA
| |
Collapse
|
10
|
Zhan X, Liu J, Long H, Zhu J, Tang H, Gou F, Wu J. An Intelligent Auxiliary Framework for Bone Malignant Tumor Lesion Segmentation in Medical Image Analysis. Diagnostics (Basel) 2023; 13:diagnostics13020223. [PMID: 36673032 PMCID: PMC9858155 DOI: 10.3390/diagnostics13020223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2022] [Revised: 12/17/2022] [Accepted: 01/04/2023] [Indexed: 01/11/2023] Open
Abstract
Bone malignant tumors are metastatic and aggressive, with poor treatment outcomes and prognosis. Rapid and accurate diagnosis is crucial for limb salvage and increasing the survival rate. There is a lack of research on deep learning to segment bone malignant tumor lesions in medical images with complex backgrounds and blurred boundaries. Therefore, we propose a new intelligent auxiliary framework for the medical image segmentation of bone malignant tumor lesions, which consists of a supervised edge-attention guidance segmentation network (SEAGNET). We design a boundary key points selection module to supervise the learning of edge attention in the model to retain fine-grained edge feature information. We precisely locate malignant tumors by instance segmentation networks while extracting feature maps of tumor lesions in medical images. The rich contextual-dependent information in the feature map is captured by mixed attention to better understand the uncertainty and ambiguity of the boundary, and edge attention learning is used to guide the segmentation network to focus on the fuzzy boundary of the tumor region. We implement extensive experiments on real-world medical data to validate our model. It validates the superiority of our method over the latest segmentation methods, achieving the best performance in terms of the Dice similarity coefficient (0.967), precision (0.968), and accuracy (0.996). The results prove the important contribution of the framework in assisting doctors to improve the accuracy of diagnosis and clinical efficiency.
Collapse
Affiliation(s)
- Xiangbing Zhan
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
| | - Jun Liu
- The Second People’s Hospital of Huaihua, Huaihua 418000, China
- Correspondence: (J.L.); (H.L.); (J.W.)
| | - Huiyun Long
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
- Correspondence: (J.L.); (H.L.); (J.W.)
| | - Jun Zhu
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
| | - Haoyu Tang
- The First People’s Hospital of Huaihua, Huaihua 418000, China
| | - Fangfang Gou
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
| | - Jia Wu
- State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University, Guiyang 550025, China
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, VIC 3800, Australia
- Correspondence: (J.L.); (H.L.); (J.W.)
| |
Collapse
|
11
|
Tang H, Huang H, Liu J, Zhu J, Gou F, Wu J. AI-Assisted Diagnosis and Decision-Making Method in Developing Countries for Osteosarcoma. Healthcare (Basel) 2022; 10:2313. [PMID: 36421636 PMCID: PMC9690527 DOI: 10.3390/healthcare10112313] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2022] [Revised: 10/28/2022] [Accepted: 11/15/2022] [Indexed: 10/29/2023] Open
Abstract
Osteosarcoma is a malignant tumor derived from primitive osteogenic mesenchymal cells, which is extremely harmful to the human body and has a high mortality rate. Early diagnosis and treatment of this disease is necessary to improve the survival rate of patients, and MRI is an effective tool for detecting osteosarcoma. However, due to the complex structure and variable location of osteosarcoma, cancer cells are highly heterogeneous and prone to aggregation and overlap, making it easy for doctors to inaccurately predict the area of the lesion. In addition, in developing countries lacking professional medical systems, doctors need to examine mass of osteosarcoma MRI images of patients, which is time-consuming and inefficient, and may result in misjudgment and omission. For the sake of reducing labor cost and improve detection efficiency, this paper proposes an Attention Condenser-based MRI image segmentation system for osteosarcoma (OMSAS), which can help physicians quickly locate the lesion area and achieve accurate segmentation of the osteosarcoma tumor region. Using the idea of AttendSeg, we constructed an Attention Condenser-based residual structure network (ACRNet), which greatly reduces the complexity of the structure and enables smaller hardware requirements while ensuring the accuracy of image segmentation. The model was tested on more than 4000 samples from two hospitals in China. The experimental results demonstrate that our model has higher efficiency, higher accuracy and lighter structure for osteosarcoma MRI image segmentation compared to other existing models.
Collapse
Affiliation(s)
- Haojun Tang
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Hui Huang
- The First People’s Hospital of Huaihua, Huaihua 418000, China
| | - Jun Liu
- The Second People’s Hospital of Huaihua, Huaihua 418000, China
| | - Jun Zhu
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
| | - Fangfang Gou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton, VIC 3800, Australia
| |
Collapse
|
12
|
Liu F, Zhu J, Lv B, Yang L, Sun W, Dai Z, Gou F, Wu J. Auxiliary Segmentation Method of Osteosarcoma MRI Image Based on Transformer and U-Net. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2022; 2022:9990092. [PMID: 36419505 PMCID: PMC9678467 DOI: 10.1155/2022/9990092] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/17/2022] [Revised: 10/24/2022] [Accepted: 10/28/2022] [Indexed: 07/28/2023]
Abstract
One of the most prevalent malignant bone tumors is osteosarcoma. The diagnosis and treatment cycle are long and the prognosis is poor. It takes a lot of time to manually identify osteosarcoma from osteosarcoma magnetic resonance imaging (MRI). Medical image processing technology has greatly alleviated the problems faced by medical diagnoses. However, MRI images of osteosarcoma are characterized by high noise and blurred edges. The complex features increase the difficulty of lesion area identification. Therefore, this study proposes an osteosarcoma MRI image segmentation method (OSTransnet) based on Transformer and U-net. This technique primarily addresses the issues of fuzzy tumor edge segmentation and overfitting brought on by data noise. First, we optimize the dataset by changing the precise spatial distribution of noise and the data-increment image rotation process. The tumor is then segmented based on the model of U-Net and Transformer with edge improvement. It compensates for the limitations of U-semantic Net by using channel-based transformers. Finally, we also add an edge enhancement module (BAB) and a combined loss function to improve the performance of edge segmentation. The method's accuracy and stability are demonstrated by the detection and training results based on more than 4,000 MRI images of osteosarcoma, which also demonstrate how well the method works as an adjunct to clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Feng Liu
- School of Information Engineering, Shandong Youth University of Political Science, Jinan, Shandong, China
- New Technology Research and Development Center of Intelligent Information Controlling in Universities of Shandong, Jinan 250103, China
| | - Jun Zhu
- The First People's Hospital of Huaihua, Huaihua 418000, Hunan, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, Hunan, China
| | - Baolong Lv
- School of Modern Service Management, Shandong Youth University of Political Science, Jinan, China
| | - Lei Yang
- School of Computer Science and Technology, Shandong Janzhu University, Jinan, China
| | - Wenyan Sun
- School of Information Engineering, Shandong Youth University of Political Science, Jinan, Shandong, China
| | - Zhehao Dai
- Department of Spine Surgery, The Second Xiangya Hospital, Central South University, Changsha 410011, China
| | - Fangfang Gou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, Clayton, Victoria 3800, Australia
| |
Collapse
|
13
|
Gou F, Liu J, Zhu J, Wu J. A Multimodal Auxiliary Classification System for Osteosarcoma Histopathological Images Based on Deep Active Learning. Healthcare (Basel) 2022; 10:2189. [PMID: 36360530 PMCID: PMC9690420 DOI: 10.3390/healthcare10112189] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 10/27/2022] [Accepted: 10/28/2022] [Indexed: 10/29/2023] Open
Abstract
Histopathological examination is an important criterion in the clinical diagnosis of osteosarcoma. With the improvement of hardware technology and computing power, pathological image analysis systems based on artificial intelligence have been widely used. However, classifying numerous intricate pathology images by hand is a tiresome task for pathologists. The lack of labeling data makes the system costly and difficult to build. This study constructs a classification assistance system (OHIcsA) based on active learning (AL) and a generative adversarial network (GAN). The system initially uses a small, labeled training set to train the classifier. Then, the most informative samples from the unlabeled images are selected for expert annotation. To retrain the network, the final chosen images are added to the initial labeled dataset. Experiments on real datasets show that our proposed method achieves high classification performance with an AUC value of 0.995 and an accuracy value of 0.989 using a small amount of labeled data. It reduces the cost of building a medical system. Clinical diagnosis can be aided by the system's findings, which can also increase the effectiveness and verifiable accuracy of doctors.
Collapse
Affiliation(s)
- Fangfang Gou
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
| | - Jun Liu
- The Second People’s Hospital of Huaihua, Huaihua 418000, China
| | - Jun Zhu
- The First People’s Hospital of Huaihua, Huaihua 418000, China
- Collaborative Innovation Center for Medical Artificial Intelligence and Big Data Decision Making Assistance, Hunan University of Medicine, Huaihua 418000, China
| | - Jia Wu
- School of Computer Science and Engineering, Central South University, Changsha 410083, China
- Research Center for Artificial Intelligence, Monash University, Melbourne, VIC 3800, Australia
| |
Collapse
|
14
|
Auxiliary Segmentation Method of Osteosarcoma in MRI Images Based on Denoising and Local Enhancement. Healthcare (Basel) 2022; 10:healthcare10081468. [PMID: 36011123 PMCID: PMC9408522 DOI: 10.3390/healthcare10081468] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Revised: 08/01/2022] [Accepted: 08/03/2022] [Indexed: 12/23/2022] Open
Abstract
Osteosarcoma is a bone tumor which is malignant. There are many difficulties when doctors manually identify patients’ MRI images to complete the diagnosis. The osteosarcoma in MRI images is very complex, making its recognition and segmentation resource-consuming. Automatic osteosarcoma area segmentation can solve these problems to a certain extent. However, existing studies usually fail to balance segmentation accuracy and efficiency. They are either sensitive to noise with low accuracy or time-consuming. So we propose an auxiliary segmentation method based on denoising and local enhancement. The method first optimizes the osteosarcoma images, including removing noise using the Edge Enhancement based Transformer for Medical Image Denoising (Eformer) and using a non-parameter method to localize and enhance the tumor region in MRI images. Osteosarcoma was then segmented by Deep Feature Aggregation for Real-Time Semantic Segmentation (DFANet). Our method achieves impressive segmentation accuracy. Moreover, it is efficient in both time and space. It can provide information about the location and extent of the osteosarcoma as a basis for further diagnosis.
Collapse
|