1
|
Dong X, Chen G, Zhu Y, Ma B, Ban X, Wu N, Ming Y. Artificial intelligence in skeletal metastasis imaging. Comput Struct Biotechnol J 2024; 23:157-164. [PMID: 38144945 PMCID: PMC10749216 DOI: 10.1016/j.csbj.2023.11.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 11/02/2023] [Accepted: 11/02/2023] [Indexed: 12/26/2023] Open
Abstract
In the field of metastatic skeletal oncology imaging, the role of artificial intelligence (AI) is becoming more prominent. Bone metastasis typically indicates the terminal stage of various malignant neoplasms. Once identified, it necessitates a comprehensive revision of the initial treatment regime, and palliative care is often the only resort. Given the gravity of the condition, the diagnosis of bone metastasis should be approached with utmost caution. AI techniques are being evaluated for their efficacy in a range of tasks within medical imaging, including object detection, disease classification, region segmentation, and prognosis prediction in medical imaging. These methods offer a standardized solution to the frequently subjective challenge of image interpretation.This subjectivity is most desirable in bone metastasis imaging. This review describes the basic imaging modalities of bone metastasis imaging, along with the recent developments and current applications of AI in the respective imaging studies. These concrete examples emphasize the importance of using computer-aided systems in the clinical setting. The review culminates with an examination of the current limitations and prospects of AI in the realm of bone metastasis imaging. To establish the credibility of AI in this domain, further research efforts are required to enhance the reproducibility and attain robust level of empirical support.
Collapse
Affiliation(s)
- Xiying Dong
- Department of Orthopedic Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing 100730, China
- Key Laboratory of Big Data for Spinal Deformities, Chinese Academy of Medical Sciences, Beijing 100730, China
- Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, China
- Department of Urology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, 100021 Beijing, China
| | - Guilin Chen
- Department of Orthopedic Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing 100730, China
- Key Laboratory of Big Data for Spinal Deformities, Chinese Academy of Medical Sciences, Beijing 100730, China
- Graduate School of Peking Union Medical College, Beijing 100730, China
| | - Yuanpeng Zhu
- Department of Orthopedic Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing 100730, China
- Key Laboratory of Big Data for Spinal Deformities, Chinese Academy of Medical Sciences, Beijing 100730, China
- Graduate School of Peking Union Medical College, Beijing 100730, China
| | - Boyuan Ma
- School of Intelligence Science and Technology, University of Science and Technology Beijing, Beijing, China
| | - Xiaojuan Ban
- School of Intelligence Science and Technology, University of Science and Technology Beijing, Beijing, China
| | - Nan Wu
- Department of Orthopedic Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing 100730, China
- Key Laboratory of Big Data for Spinal Deformities, Chinese Academy of Medical Sciences, Beijing 100730, China
- Beijing Key Laboratory for Genetic Research of Skeletal Deformity, Beijing 100730, China
| | - Yue Ming
- Department of Nuclear Medicine (PET-CT Center), National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
| |
Collapse
|
2
|
Ceranka J, Wuts J, Chiabai O, Lecouvet F, Vandemeulebroucke J. Computer-aided diagnosis of skeletal metastases in multi-parametric whole-body MRI. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107811. [PMID: 37742486 DOI: 10.1016/j.cmpb.2023.107811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 09/07/2023] [Accepted: 09/11/2023] [Indexed: 09/26/2023]
Abstract
The confident detection of metastatic bone disease is essential to improve patients' comfort and increase life expectancy. Multi-parametric magnetic resonance imaging (MRI) has been successfully used for monitoring of metastatic bone disease, allowing for comprehensive and holistic evaluation of the total tumour volume and treatment response assessment. The major challenges of radiological reading of whole-body MRI come from the amount of data to be reviewed and the scattered distribution of metastases, often of complex shapes. This makes bone lesion detection and quantification demanding for a radiologist and prone to error. Additionally, whole-body MRI are often corrupted with multiple spatial and intensity distortions, which further degrade the performance of image reading and image processing algorithms. In this work we propose a fully automated computer-aided diagnosis system for the detection and segmentation of metastatic bone disease using whole-body multi-parametric MRI. The system consists of an extensive image preprocessing pipeline aiming at enhancing the image quality, followed by a deep learning framework for detection and segmentation of metastatic bone disease. The system outperformed state-of-the-art methodologies, achieving a detection sensitivity of 63% with a mean of 6.44 false positives per image, and an average lesion Dice coefficient of 0.53. A provided ablation study performed to investigate the relative importance of image preprocessing shows that introduction of region of interest mask and spatial registration have a significant impact on detection and segmentation performance in whole-body MRI. The proposed computer-aided diagnosis system allows for automatic quantification of disease infiltration and could provide a valuable tool during radiological examination of whole-body MRI.
Collapse
Affiliation(s)
- Jakub Ceranka
- Vrije Universiteit Brussel, Department of Electronics and Informatics, Pleinlaan 2, Brussels, 1050, Belgium; imec, Kapeldreef 75, Leuven, B-3001, Belgium.
| | - Joris Wuts
- Vrije Universiteit Brussel, Department of Electronics and Informatics, Pleinlaan 2, Brussels, 1050, Belgium; imec, Kapeldreef 75, Leuven, B-3001, Belgium; Cliniques universitaires Saint Luc, Institut de Recherche Expérimentale et Clinique (IREC), Université catholique de Louvain (UCLouvain), Avenue Hippocrate 10, Brussels, 1200, Belgium.
| | - Ophélye Chiabai
- Cliniques universitaires Saint Luc, Institut de Recherche Expérimentale et Clinique (IREC), Université catholique de Louvain (UCLouvain), Avenue Hippocrate 10, Brussels, 1200, Belgium
| | - Frédéric Lecouvet
- Cliniques universitaires Saint Luc, Institut de Recherche Expérimentale et Clinique (IREC), Université catholique de Louvain (UCLouvain), Avenue Hippocrate 10, Brussels, 1200, Belgium
| | - Jef Vandemeulebroucke
- Vrije Universiteit Brussel, Department of Electronics and Informatics, Pleinlaan 2, Brussels, 1050, Belgium; imec, Kapeldreef 75, Leuven, B-3001, Belgium; Universitair Ziekenhuis Brussel, Department of Radiology, Laarbeeklaan 101, Brussels, 1090, Belgium
| |
Collapse
|
3
|
Chen YY, Yu PN, Lai YC, Hsieh TC, Cheng DC. Bone Metastases Lesion Segmentation on Breast Cancer Bone Scan Images with Negative Sample Training. Diagnostics (Basel) 2023; 13:3042. [PMID: 37835785 PMCID: PMC10572884 DOI: 10.3390/diagnostics13193042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Revised: 09/18/2023] [Accepted: 09/22/2023] [Indexed: 10/15/2023] Open
Abstract
The use of deep learning methods for the automatic detection and quantification of bone metastases in bone scan images holds significant clinical value. A fast and accurate automated system for segmenting bone metastatic lesions can assist clinical physicians in diagnosis. In this study, a small internal dataset comprising 100 breast cancer patients (90 cases of bone metastasis and 10 cases of non-metastasis) and 100 prostate cancer patients (50 cases of bone metastasis and 50 cases of non-metastasis) was used for model training. Initially, all image labels were binary. We used the Otsu thresholding method or negative mining to generate a non-metastasis mask, thereby transforming the image labels into three classes. We adopted the Double U-Net as the baseline model and made modifications to its output activation function. We changed the activation function to SoftMax to accommodate multi-class segmentation. Several methods were used to enhance model performance, including background pre-processing to remove background information, adding negative samples to improve model precision, and using transfer learning to leverage shared features between two datasets, which enhances the model's performance. The performance was investigated via 10-fold cross-validation and computed on a pixel-level scale. The best model we achieved had a precision of 69.96%, a sensitivity of 63.55%, and an F1-score of 66.60%. Compared to the baseline model, this represents an 8.40% improvement in precision, a 0.56% improvement in sensitivity, and a 4.33% improvement in the F1-score. The developed system has the potential to provide pre-diagnostic reports for physicians in final decisions and the calculation of the bone scan index (BSI) with the combination with bone skeleton segmentation.
Collapse
Affiliation(s)
- Yi-You Chen
- Department of Biomedical Imaging and Radiological Science, China Medical University, Taichung 404, Taiwan; (Y.-Y.C.); (P.-N.Y.)
| | - Po-Nien Yu
- Department of Biomedical Imaging and Radiological Science, China Medical University, Taichung 404, Taiwan; (Y.-Y.C.); (P.-N.Y.)
| | - Yung-Chi Lai
- Department of Nuclear Medicine, Feng Yuan Hospital, Ministry of Health and Welfare, Taichung 420, Taiwan;
| | - Te-Chun Hsieh
- Department of Biomedical Imaging and Radiological Science, China Medical University, Taichung 404, Taiwan; (Y.-Y.C.); (P.-N.Y.)
- Department of Nuclear Medicine and PET Center, China Medical University Hospital, Taichung 404, Taiwan
| | - Da-Chuan Cheng
- Department of Biomedical Imaging and Radiological Science, China Medical University, Taichung 404, Taiwan; (Y.-Y.C.); (P.-N.Y.)
| |
Collapse
|
4
|
Deng S, Li C, Cao J, Cui Z, Du J, Fu Z, Yang H, Chen P. Organ-on-a-chip meets artificial intelligence in drug evaluation. Theranostics 2023; 13:4526-4558. [PMID: 37649608 PMCID: PMC10465229 DOI: 10.7150/thno.87266] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 08/02/2023] [Indexed: 09/01/2023] Open
Abstract
Drug evaluation has always been an important area of research in the pharmaceutical industry. However, animal welfare protection and other shortcomings of traditional drug development models pose obstacles and challenges to drug evaluation. Organ-on-a-chip (OoC) technology, which simulates human organs on a chip of the physiological environment and functionality, and with high fidelity reproduction organ-level of physiology or pathophysiology, exhibits great promise for innovating the drug development pipeline. Meanwhile, the advancement in artificial intelligence (AI) provides more improvements for the design and data processing of OoCs. Here, we review the current progress that has been made to generate OoC platforms, and how human single and multi-OoCs have been used in applications, including drug testing, disease modeling, and personalized medicine. Moreover, we discuss issues facing the field, such as large data processing and reproducibility, and point to the integration of OoCs and AI in data analysis and automation, which is of great benefit in future drug evaluation. Finally, we look forward to the opportunities and challenges faced by the coupling of OoCs and AI. In summary, advancements in OoCs development, and future combinations with AI, will eventually break the current state of drug evaluation.
Collapse
Affiliation(s)
- Shiwen Deng
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
| | - Caifeng Li
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
- Robot Intelligent Laboratory of Traditional Chinese Medicine, Experimental Research Center, China Academy of Chinese Medical Sciences & MEGAROBO, Beijing 100700, China
| | - Junxian Cao
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
| | - Zhao Cui
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
- Institute of Chinese Materia Medica, China Academy of Chinese Medical Sciences, Beijing 100700, China
| | - Jiang Du
- Yunnan Biovalley Pharmaceutical Co., Ltd, Kunming 650503, China
| | - Zheng Fu
- Robot Intelligent Laboratory of Traditional Chinese Medicine, Experimental Research Center, China Academy of Chinese Medical Sciences & MEGAROBO, Beijing 100700, China
| | - Hongjun Yang
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
- Robot Intelligent Laboratory of Traditional Chinese Medicine, Experimental Research Center, China Academy of Chinese Medical Sciences & MEGAROBO, Beijing 100700, China
| | - Peng Chen
- Beijing Key Laboratory of Traditional Chinese Medicine Basic Research on Prevention and Treatment for Major Diseases, Experimental Research Center, China Academy of Chinese Medical Sciences, Beijing 100700, China
- Yunnan Biovalley Pharmaceutical Co., Ltd, Kunming 650503, China
- Robot Intelligent Laboratory of Traditional Chinese Medicine, Experimental Research Center, China Academy of Chinese Medical Sciences & MEGAROBO, Beijing 100700, China
| |
Collapse
|
5
|
Segmentation of lung cancer-caused metastatic lesions in bone scan images using self-defined model with deep supervision. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
|
6
|
Lacroix M, Aouad T, Feydy J, Biau D, Larousserie F, Fournier L, Feydy A. Artificial intelligence in musculoskeletal oncology imaging: A critical review of current applications. Diagn Interv Imaging 2023; 104:18-23. [PMID: 36270953 DOI: 10.1016/j.diii.2022.10.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2022] [Accepted: 10/05/2022] [Indexed: 01/10/2023]
Abstract
Artificial intelligence (AI) is increasingly being studied in musculoskeletal oncology imaging. AI has been applied to both primary and secondary bone tumors and assessed for various predictive tasks that include detection, segmentation, classification, and prognosis. Still, in the field of clinical research, further efforts are needed to improve AI reproducibility and reach an acceptable level of evidence in musculoskeletal oncology. This review describes the basic principles of the most common AI techniques, including machine learning, deep learning and radiomics. Then, recent developments and current results of AI in the field of musculoskeletal oncology are presented. Finally, limitations and future perspectives of AI in this field are discussed.
Collapse
Affiliation(s)
- Maxime Lacroix
- Department of Radiology, Hôpital Européen Georges Pompidou, Assistance Publique-Hôpitaux de Paris, Paris, 75015, France; Université Paris Cité, Faculté de Médecine, Paris, 75006, France; PARCC UMRS 970, INSERM, Paris 75015, France
| | - Theodore Aouad
- Université Paris-Saclay, CentraleSupélec, Inria, Centre for Visual Computing, 91190, Gif-sur-Yvette, France
| | - Jean Feydy
- Université Paris Cité, HeKA team, Inria Paris, Inserm, 75006, Paris, France
| | - David Biau
- Université Paris Cité, Faculté de Médecine, Paris, 75006, France; Department of Orthopedic Surgery, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, 75014, France
| | - Frédérique Larousserie
- Université Paris Cité, Faculté de Médecine, Paris, 75006, France; Department of Pathology, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, 75014, France
| | - Laure Fournier
- Department of Radiology, Hôpital Européen Georges Pompidou, Assistance Publique-Hôpitaux de Paris, Paris, 75015, France; Université Paris Cité, Faculté de Médecine, Paris, 75006, France; PARCC UMRS 970, INSERM, Paris 75015, France
| | - Antoine Feydy
- Université Paris Cité, Faculté de Médecine, Paris, 75006, France; Department of Radiology, Hôpital Cochin, Assistance Publique-Hôpitaux de Paris, Paris, 75014, France
| |
Collapse
|
7
|
Integrating Transfer Learning and Feature Aggregation into Self-defined Convolutional Neural Network for Automated Detection of Lung Cancer Bone Metastasis. J Med Biol Eng 2022. [DOI: 10.1007/s40846-022-00770-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
|
8
|
Lin Q, Gao R, Luo M, Wang H, Cao Y, Man Z, Wang R. Semi-supervised segmentation of metastasis lesions in bone scan images. Front Mol Biosci 2022; 9:956720. [DOI: 10.3389/fmolb.2022.956720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 10/13/2022] [Indexed: 11/13/2022] Open
Abstract
To develop a deep image segmentation model that automatically identifies and delineates lesions of skeletal metastasis in bone scan images, facilitating clinical diagnosis of lung cancer–caused bone metastasis by nuclear medicine physicians. A semi-supervised segmentation model is proposed, comprising the feature extraction subtask and pixel classification subtask. During the feature extraction stage, cascaded layers which include the dilated residual convolution, inception connection, and feature aggregation learn the hierarchal representations of low-resolution bone scan images. During the pixel classification stage, each pixel is first classified into categories in a semi-supervised manner, and the boundary of pixels belonging to an individual lesion is then delineated using a closed curve. Experimental evaluation conducted on 2,280 augmented samples (112 original images) demonstrates that the proposed model performs well for automated segmentation of metastasis lesions, with a score of 0.692 for DSC if the model is trained using 37% of the labeled samples. The self-defined semi-supervised segmentation model can be utilized as an automated clinical tool to detect and delineate metastasis lesions in bone scan images, using only a few manually labeled image samples. Nuclear medicine physicians need only attend to those segmented lesions while ignoring the background when they diagnose bone metastasis using low-resolution images. More images of patients from multiple centers are typically needed to further improve the scalability and performance of the model via mitigating the impacts of variability in size, shape, and intensity of bone metastasis lesions.
Collapse
|
9
|
Meena T, Roy S. Bone Fracture Detection Using Deep Supervised Learning from Radiological Images: A Paradigm Shift. Diagnostics (Basel) 2022; 12:diagnostics12102420. [PMID: 36292109 PMCID: PMC9600559 DOI: 10.3390/diagnostics12102420] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 10/04/2022] [Accepted: 10/05/2022] [Indexed: 01/16/2023] Open
Abstract
Bone diseases are common and can result in various musculoskeletal conditions (MC). An estimated 1.71 billion patients suffer from musculoskeletal problems worldwide. Apart from musculoskeletal fractures, femoral neck injuries, knee osteoarthritis, and fractures are very common bone diseases, and the rate is expected to double in the next 30 years. Therefore, proper and timely diagnosis and treatment of a fractured patient are crucial. Contrastingly, missed fractures are a common prognosis failure in accidents and emergencies. This causes complications and delays in patients’ treatment and care. These days, artificial intelligence (AI) and, more specifically, deep learning (DL) are receiving significant attention to assist radiologists in bone fracture detection. DL can be widely used in medical image analysis. Some studies in traumatology and orthopaedics have shown the use and potential of DL in diagnosing fractures and diseases from radiographs. In this systematic review, we provide an overview of the use of DL in bone imaging to help radiologists to detect various abnormalities, particularly fractures. We have also discussed the challenges and problems faced in the DL-based method, and the future of DL in bone imaging.
Collapse
|
10
|
Manganelli Conforti P, D’Acunto M, Russo P. Deep Learning for Chondrogenic Tumor Classification through Wavelet Transform of Raman Spectra. SENSORS (BASEL, SWITZERLAND) 2022; 22:s22197492. [PMID: 36236597 PMCID: PMC9571786 DOI: 10.3390/s22197492] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 09/16/2022] [Accepted: 09/23/2022] [Indexed: 05/22/2023]
Abstract
The grading of cancer tissues is still one of the main challenges for pathologists. The development of enhanced analysis strategies hence becomes crucial to accurately identify and further deal with each individual case. Raman spectroscopy (RS) is a promising tool for the classification of tumor tissues as it allows us to obtain the biochemical maps of the tissues under analysis and to observe their evolution in terms of biomolecules, proteins, lipid structures, DNA, vitamins, and so on. However, its potential could be further improved by providing a classification system which would be able to recognize the sample tumor category by taking as input the raw Raman spectroscopy signal; this could provide more reliable responses in shorter time scales and could reduce or eliminate false-positive or -negative diagnoses. Deep Learning techniques have become ubiquitous in recent years, with models able to perform classification with high accuracy in most diverse fields of research, e.g., natural language processing, computer vision, medical imaging. However, deep models often rely on huge labeled datasets to produce reasonable accuracy, otherwise occurring in overfitting issues when the training data is insufficient. In this paper, we propose a chondrogenic tumor CLAssification through wavelet transform of RAman spectra (CLARA), which is able to classify with high accuracy Raman spectra obtained from bone tissues. CLARA recognizes and grades the tumors in the evaluated dataset with 97% accuracy by exploiting a classification pipeline consisting of the division of the original task in two binary classification steps, where the first is performed on the original RS signals while the latter is accomplished through the use of a hybrid temporal-frequency 2D transform.
Collapse
Affiliation(s)
| | - Mario D’Acunto
- CNR-IBF, Istituto di Biofisica, Via Moruzzi 1, 56124 Pisa, Italy
| | - Paolo Russo
- DIAG Department, Sapienza University of Rome, Via Ariosto 25, 00185 Roma, Italy
- Correspondence:
| |
Collapse
|
11
|
Guo Y, Lin Q, Zhao S, Li T, Cao Y, Man Z, Zeng X. Automated detection of lung cancer-caused metastasis by classifying scintigraphic images using convolutional neural network with residual connection and hybrid attention mechanism. Insights Imaging 2022; 13:24. [PMID: 35138479 PMCID: PMC8828823 DOI: 10.1186/s13244-022-01162-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2021] [Accepted: 01/20/2022] [Indexed: 12/03/2022] Open
Abstract
Background Whole-body bone scan is the widely used tool for surveying bone metastases caused by various primary solid tumors including lung cancer. Scintigraphic images are characterized by low specificity, bringing a significant challenge to manual analysis of images by nuclear medicine physicians. Convolutional neural network can be used to develop automated classification of images by automatically extracting hierarchal features and classifying high-level features into classes. Results Using convolutional neural network, a multi-class classification model has been developed to detect skeletal metastasis caused by lung cancer using clinical whole-body scintigraphic images. The proposed method consisted of image aggregation, hierarchal feature extraction, and high-level feature classification. Experimental evaluations on a set of clinical scintigraphic images have shown that the proposed multi-class classification network is workable for automated detection of lung cancer-caused metastasis, with achieving average scores of 0.7782, 0.7799, 0.7823, 0.7764, and 0.8364 for accuracy, precision, recall, F-1 score, and AUC value, respectively. Conclusions The proposed multi-class classification model can not only predict whether an image contains lung cancer-caused metastasis, but also differentiate between subclasses of lung cancer (i.e., adenocarcinoma and non-adenocarcinoma). On the context of two-class (i.e., the metastatic and non-metastatic) classification, the proposed model obtained a higher score of 0.8310 for accuracy metric.
Collapse
Affiliation(s)
- Yanru Guo
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China
| | - Qiang Lin
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China. .,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China. .,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Northwest Minzu University, Lanzhou, Gansu, China.
| | - Shaofang Zhao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China
| | - Tongtong Li
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China
| | - Yongchun Cao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Northwest Minzu University, Lanzhou, Gansu, China
| | - Zhengxing Man
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, Gansu, China.,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Northwest Minzu University, Lanzhou, Gansu, China
| | - Xianwu Zeng
- Department of Nuclear Medicine, Gansu Provincial Tumor Hospital, Lanzhou, Gansu, China.
| |
Collapse
|
12
|
Li MD, Ahmed SR, Choy E, Lozano-Calderon SA, Kalpathy-Cramer J, Chang CY. Artificial intelligence applied to musculoskeletal oncology: a systematic review. Skeletal Radiol 2022; 51:245-256. [PMID: 34013447 DOI: 10.1007/s00256-021-03820-w] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Revised: 05/13/2021] [Accepted: 05/13/2021] [Indexed: 02/02/2023]
Abstract
Developments in artificial intelligence have the potential to improve the care of patients with musculoskeletal tumors. We performed a systematic review of the published scientific literature to identify the current state of the art of artificial intelligence applied to musculoskeletal oncology, including both primary and metastatic tumors, and across the radiology, nuclear medicine, pathology, clinical research, and molecular biology literature. Through this search, we identified 252 primary research articles, of which 58 used deep learning and 194 used other machine learning techniques. Articles involving deep learning have mostly involved bone scintigraphy, histopathology, and radiologic imaging. Articles involving other machine learning techniques have mostly involved transcriptomic analyses, radiomics, and clinical outcome prediction models using medical records. These articles predominantly present proof-of-concept work, other than the automated bone scan index for bone metastasis quantification, which has translated to clinical workflows in some regions. We systematically review and discuss this literature, highlight opportunities for multidisciplinary collaboration, and identify potentially clinically useful topics with a relative paucity of research attention. Musculoskeletal oncology is an inherently multidisciplinary field, and future research will need to integrate and synthesize noisy siloed data from across clinical, imaging, and molecular datasets. Building the data infrastructure for collaboration will help to accelerate progress towards making artificial intelligence truly useful in musculoskeletal oncology.
Collapse
Affiliation(s)
- Matthew D Li
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA. .,Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
| | - Syed Rakin Ahmed
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.,Harvard Medical School, Harvard Graduate Program in Biophysics, Harvard University, Cambridge, MA, USA.,Geisel School of Medicine At Dartmouth, Dartmouth College, Hanover, NH, USA
| | - Edwin Choy
- Division of Hematology Oncology, Department of Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Santiago A Lozano-Calderon
- Department of Orthopedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jayashree Kalpathy-Cramer
- Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Connie Y Chang
- Division of Musculoskeletal Imaging and Intervention, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
13
|
Li T, Lin Q, Guo Y, Zhao S, Zeng X, Man Z, Cao Y, Hu Y. Automated detection of skeletal metastasis of lung cancer with bone scans using convolutional nuclear network. Phys Med Biol 2021; 67. [PMID: 34933282 DOI: 10.1088/1361-6560/ac4565] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/17/2021] [Accepted: 12/21/2021] [Indexed: 11/12/2022]
Abstract
Bone scan is widely used for surveying bone metastases caused by various solid tumors. Scintigraphic images are characterized by inferior spatial resolution, bringing a significant challenge to manual analysis of images by nuclear medicine physicians. We present in this work a new framework for automatically classifying scintigraphic images collected from patients clinically diagnosed with lung cancer. The framework consists of data preparation and image classification. In the data preparation stage, data augmentation is used to enlarge the dataset, followed by image fusion and thoracic region extraction. In the image classification stage, we use a self-defined convolutional neural network consisting of feature extraction, feature aggregation, and feature classification sub-networks. The developed multi-class classification network can not only predict whether a bone scan image contains bone metastasis but also tell which subcategory of lung cancer that a bone metastasis metastasized from is present in the image. Experimental evaluations on a set of clinical bone scan images have shown that the proposed multi-class classification network is workable for automated classification of metastatic images, with achieving average scores of 0.7392, 0.7592, 0.7242, and 0.7292 for accuracy, precision, recall, and F-1 score, respectively.
Collapse
Affiliation(s)
- Tongtong Li
- Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, Gansu, 730030, CHINA
| | - Qiang Lin
- School of Mathematics and Computer Science, Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, 730030, CHINA
| | - Yanru Guo
- Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, Gansu, 730030, CHINA
| | - Shaofang Zhao
- Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, Gansu, 730030, CHINA
| | - Xianwu Zeng
- Gansu Provincial Cancer Hospital, No. 2, Dongjie Rd., Lanzhou, Gansu, 730050, CHINA
| | - Zhengxing Man
- Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, Gansu, 730030, CHINA
| | - Yongchun Cao
- Northwest Minzu University, No. 1, Xibei Xincun Rd., Lanzhou, Gansu, 730030, CHINA
| | - Yonghua Hu
- Gansu University of Chinese Medicine, No. 35, Dingxi Donglu Rd., Lanzhou, 730000, CHINA
| |
Collapse
|
14
|
Liu X, Han C, Cui Y, Xie T, Zhang X, Wang X. Detection and Segmentation of Pelvic Bones Metastases in MRI Images for Patients With Prostate Cancer Based on Deep Learning. Front Oncol 2021; 11:773299. [PMID: 34912716 PMCID: PMC8666439 DOI: 10.3389/fonc.2021.773299] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Accepted: 11/08/2021] [Indexed: 12/23/2022] Open
Abstract
Objective To establish and evaluate the 3D U-Net model for automated segmentation and detection of pelvic bone metastases in patients with prostate cancer (PCa) using diffusion-weighted imaging (DWI) and T1 weighted imaging (T1WI) images. Methods The model consisted of two 3D U-Net algorithms. A total of 859 patients with clinically suspected or confirmed PCa between January 2017 and December 2020 were enrolled for the first 3D U-Net development of pelvic bony structure segmentation. Then, 334 PCa patients were selected for the model development of bone metastases segmentation. Additionally, 63 patients from January to May 2021 were recruited for the external evaluation of the network. The network was developed using DWI and T1WI images as input. Dice similarity coefficient (DSC), volumetric similarity (VS), and Hausdorff distance (HD) were used to evaluate the segmentation performance. Sensitivity, specificity, and area under the curve (AUC) were used to evaluate the detection performance at the patient level; recall, precision, and F1-score were assessed at the lesion level. Results The pelvic bony structures segmentation on DWI and T1WI images had mean DSC and VS values above 0.85, and the HD values were <15 mm. In the testing set, the AUC of the metastases detection at the patient level were 0.85 and 0.80 on DWI and T1WI images. At the lesion level, the F1-score achieved 87.6% and 87.8% concerning metastases detection on DWI and T1WI images, respectively. In the external dataset, the AUC of the model for M-staging was 0.94 and 0.89 on DWI and T1WI images. Conclusion The deep learning-based 3D U-Net network yields accurate detection and segmentation of pelvic bone metastases for PCa patients on DWI and T1WI images, which lays a foundation for the whole-body skeletal metastases assessment.
Collapse
Affiliation(s)
- Xiang Liu
- Department of Radiology, Peking University First Hospital, Beijing, China
| | - Chao Han
- Department of Radiology, Peking University First Hospital, Beijing, China
| | - Yingpu Cui
- Department of Radiology, Peking University First Hospital, Beijing, China
| | - Tingting Xie
- Department of Radiology, Peking University Shenzhen Hospital, Shenzhen, China
| | - Xiaodong Zhang
- Department of Radiology, Peking University First Hospital, Beijing, China
| | - Xiaoying Wang
- Department of Radiology, Peking University First Hospital, Beijing, China
| |
Collapse
|
15
|
Lin Q, Cao C, Li T, Cao Y, Man Z, Wang H. Multiclass classification of whole-body scintigraphic images using a self-defined convolutional neural network with attention modules. Med Phys 2021; 48:5782-5793. [PMID: 34455613 PMCID: PMC9135133 DOI: 10.1002/mp.15196] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2021] [Revised: 08/20/2021] [Accepted: 08/22/2021] [Indexed: 01/26/2023] Open
Abstract
Purpose A self‐defined convolutional neural network is developed to automatically classify whole‐body scintigraphic images of concern (i.e., the normal, metastasis, arthritis, and thyroid carcinoma), automatically detecting diseases with whole‐body bone scintigraphy. Methods A set of parameter transformation operations are first used to augment the original dataset of whole‐body bone scintigraphic images. A hybrid attention mechanism including the spatial and channel attention module is then introduced to develop a deep classification network, Dscint, which consists of eight weight layers, one hybrid attention module, two normalization modules, two fully connected layers, and one softmax layer. Results Experimental evaluations conducted on a set of whole‐body scintigraphic images show that the proposed deep classification network, Dscint, performs well for automated detection of diseases by classifying the images of concerns, achieving the accuracy, precision, recall, specificity, and F‐1 score of 0.9801, 0.9795, 0.9791, 0.9933, and 0.9792, respectively, on the test data in the augmented dataset. A comparative analysis of Dscint and several classical deep classification networks (i.e., AlexNet, ResNet, VGGNet, DenseNet, and Inception‐v4) reveals that our self‐defined network, Dscint, performs best on classifying whole‐body scintigraphic images on the same dataset. Conclusions The self‐defined deep classification network, Dscint, can be utilized to automatically determine whether a whole‐body scintigraphic image is either normal or contains diseases of concern. Specifically, better performance of Dscint is obtained on images with lesions that are present in relatively fixed locations like thyroid carcinoma than those with lesions occurring in nonfixed locations of bone tissue.
Collapse
Affiliation(s)
- Qiang Lin
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, China.,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Lanzhou, China
| | - Chuangui Cao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, China
| | - Tongtong Li
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, China
| | - Yongchun Cao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, China.,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Lanzhou, China
| | - Zhengxing Man
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, China.,Key Laboratory of Streaming Data Computing Technologies and Application, Northwest Minzu University, Lanzhou, China.,Key Laboratory of China's Ethnic Languages and Information Technology of Ministry of Education, Lanzhou, China
| | - Haijun Wang
- Department of Nuclear Medicine, Gansu Provincial Hospital, Lanzhou, China
| |
Collapse
|
16
|
Lin Q, Cao C, Li T, Man Z, Cao Y, Wang H. dSPIC: a deep SPECT image classification network for automated multi-disease, multi-lesion diagnosis. BMC Med Imaging 2021; 21:122. [PMID: 34380441 PMCID: PMC8359584 DOI: 10.1186/s12880-021-00653-w] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2020] [Accepted: 07/27/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Functional imaging especially the SPECT bone scintigraphy has been accepted as the effective clinical tool for diagnosis, treatment, evaluation, and prevention of various diseases including metastasis. However, SPECT imaging is brightly characterized by poor resolution, low signal-to-noise ratio, as well as the high sensitivity and low specificity because of the visually similar characteristics of lesions between diseases on imaging findings. METHODS Focusing on the automated diagnosis of diseases with whole-body SPECT scintigraphic images, in this work, a self-defined convolutional neural network is developed to survey the presence or absence of diseases of concern. The data preprocessing mainly including data augmentation is first conducted to cope with the problem of limited samples of SPECT images by applying the geometric transformation operations and generative adversarial network techniques on the original SPECT imaging data. An end-to-end deep SPECT image classification network named dSPIC is developed to extract the optimal features from images and then to classify these images into classes, including metastasis, arthritis, and normal, where there may be multiple diseases existing in a single image. RESULTS A group of real-world data of whole-body SPECT images is used to evaluate the self-defined network, obtaining a best (worst) value of 0.7747 (0.6910), 0.7883 (0.7407), 0.7863 (0.6956), 0.8820 (0.8273) and 0.7860 (0.7230) for accuracy, precision, sensitivity, specificity, and F-1 score, respectively, on the testing samples from the original and augmented datasets. CONCLUSIONS The prominent classification performance in contrast to other related deep classifiers including the classical AlexNet network demonstrates that the built deep network dSPIC is workable and promising for the multi-disease, multi-lesion classification task of whole-body SPECT bone scintigraphy images.
Collapse
Affiliation(s)
- Qiang Lin
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, 730030, China. .,Key Lab of Streaming Data Computing and Applications, Northwest Minzu University, Lanzhou, 730030, China.
| | - Chuangui Cao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, 730030, China.,Key Lab of Streaming Data Computing and Applications, Northwest Minzu University, Lanzhou, 730030, China
| | - Tongtong Li
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, 730030, China.,Key Lab of Streaming Data Computing and Applications, Northwest Minzu University, Lanzhou, 730030, China
| | - Zhengxing Man
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, 730030, China.,Key Lab of Streaming Data Computing and Applications, Northwest Minzu University, Lanzhou, 730030, China
| | - Yongchun Cao
- School of Mathematics and Computer Science, Northwest Minzu University, Lanzhou, 730030, China.,Key Lab of Streaming Data Computing and Applications, Northwest Minzu University, Lanzhou, 730030, China
| | - Haijun Wang
- Department of Nuclear Medicine, Gansu Provincial Hospital, Lanzhou, 730000, China
| |
Collapse
|